Medical apps for smartphones: lack of evidence
undermines quality and safety
Arthur Willem Gerard Buijink,1Benjamin Jelle Visser,2
Increasing numbers of healthcare professionals are using
smartphones and their associated applications (apps) in
daily clinical care. While these medical apps hold great
potential for improving clinical practice, little is known
about the possible dangers associated with their use.
Breaches of patient confidentiality, conflicts of interests
and malfunctioning clinical decision-making apps could
all negatively impact on patient care. We propose
several strategies to enhance the development of evi-
dence-based medical apps while retaining their open
nature. The increasing use of medical apps calls for
broader discussion across medicine’s organising and
accrediting bodies. The field of medical apps is currently
one of the most dynamic in medicine, with real potential
to change the way evidence-based healthcare is deliv-
ered in the future. Establishing appropriate regulatory
procedures will enable this potential to be fulfilled,
while at all times ensuring the safety of the patient.
Smartphone applications—so-called apps—are becoming
increasingly popular among medical professionals.1–3By
2015, 500 million smartphone users worldwide will be
using a medical application.4Studies report that over
85% of health professionals use a smartphone, and
30–50% use medical apps in clinical care.5–7Apps have
huge potential to improve patient practice, system effi-
ciency and communication by providing a quick refer-
ence tool accessible at the point of care.8To date, there
are 10 000 apps available in the ‘medical section’ of
Apple’s ‘App store’ and over 3000 on Google’s ‘Play
store’.9 10Since these platforms facilitate development
and distribution of mobile applications by clinicians and
other developers, rapid proliferation of the market will
However, there has been minimal description of the
dangers posed by medical apps within medical literature.
Recent studies have addressed the lackof evidence and pro-
fessional medical involvement in their design and develop-
ment, raising concerns regarding the reliability and
accuracy of their medical content, and the consequences
for patient safety.2 3It has been proposed that medical apps
should be peer-reviewed by clinical experts and that regu-
latory measures should be increased in order to safeguard
quality of care.1Regulation and guidance are urgently
needed. Medical professionals must be made aware that
some apps contain unreliable, non-peer-reviewed content
so that they can choose carefully which apps to use in clin-
ical care.11In this paper, we propose possible strategies that
could enable the medical app market to be controlled and
evidence based, while simultaneously minimising unneces-
sary bureaucracy so as not to hinder app development.
Lack of clinical involvement in medical apps
Most medical apps lack authenticity details; authors,
manufacturers and distributors are not listed and refer-
ences are unavailable or out-of-date. To illustrate,
searching for an app to interpret arterial blood gas
values produces a list of dozens claiming to have such a
function, but few would be useful to physicians. Instead,
most are irrelevant, non-evidence-based or even down-
right dangerous due to inaccurate content. Furthermore,
it is unclear as to whether these apps will be updated if
new evidence arises. Two recent studies in the fields of
dermatology and microbiology revealed that less than
35% of medical apps had medical expert involvement
111-reviewed pain-management apps were found to
have no medical professional involvement.12Only 12%
reported a physician as the app’s author.
As with any new healthcare technology, it is important
to establish where responsibility and control lies. Indeed,
this was the topic of a research project undertaken
10 years ago13that recommended ways of accrediting
healthcare-related software, and proposed a scheme like
Conformité Européene (CE) marking.
Dangers of medical apps in clinical practice
The very nature of smart phones poses a potential risk.
As medical apps are increasingly used to support diag-
nosis and management of diseases (eg, apps that allow
the user to input patient-specific information along with
reference material to automatically diagnose a disease or
condition), facilitating the appropriate use of informa-
disease severity, was recalled from app stores because it
was giving erroneous scores in comparison with those
calculated using the official formula.14Until now, there
has been no reported harm to a patient caused by a
recalled app. However, without app safety standards, it
is only a matter of time before medical errors will be
made and unintended harm to a patient will occur.
Several websites have recently been launched by
medical professionals to index, provide commentary and
review medical applications.15 16Although this is a
good starting point for peer-reviewing apps, the current
assessment criteria do not address the scientific evidence
for their content, but rather matters of usability, design
and content control.
industry are progressively developing and merchandising
more medical apps for healthcare professionals.18Using
such apps could raise substantial ethical issues. For
instance, conflicts of interest may lead to conscious and
unconscious bias in prescribing habits. Pharmaceutical
companies may use these apps for marketing purposes,
1Department of Neurology,
Academic Medical Center—
University of Amsterdam,
Amsterdam, The Netherlands
2Department of Internal
Medicine, Academic Medical
3Sobell Department of Motor
Neuroscience and Movement
Disorders, Institute of
Neurology, University College
London, London, UK
Arthur Willem Gerard Buijink,
Department of Neurology—Room
D2-136, Academic Medical
Center, PO Box 22660,
Amsterdam 1100 DD,
Evidence-Based Medicine Month 2012 | volume 0 | number 0 |
Evidence-Based Medicine Online First, published on August 25, 2012 as 10.1136/eb-2012-100885
Copyright Article author (or their employer) 2012. Produced by BMJ Publishing Group Ltd under licence.
influencing treatment options and presenting informa-
tion in favour of their own drugs, with knock-on effects
for patients’ care.
Regulation by government health authorities
Recently, the American Food and Drug Administration
(FDA) published a draft guideline on how to regulate
medical apps.19The FDA states that an app can be con-
sidered a medical device when ‘it is used as an accessory
to a regulated medical device or transforms a mobile
platform into a regulated medical device’.19As an
example, they describe that a light-emitting diode (LED),
included on most mobile phones, can be used to illu-
minate objects. In this case, neither an app controlling
the LED nor the mobile phone itself are considered
medical devices. When, however, an app is promoted for
providing a light source to examine patients, it would
meet the definition of a device. The FDA plans to
actively regulate certain types of apps (box 1). In our
opinion, this is a positive development. Nevertheless, at
the same time, government health authorities should not
over-regulate medical apps so as to retain their open
nature. The regulation process should be managed pri-
marily by the healthcare community itself. However, it
would be beneficial for government health authorities to
provide official certification marks guaranteeing the
quality of apps so that physicians can make an informed
choice as to whether an app has evidence-based
Adoption of medical apps by medical publishers
We believe there to be various ways in which medical
apps could be developed to ensure quality and safety.
First of all, the content of all medical apps should be
evidence based, externally peer reviewed by medical
professionals and provide up-to-date clinical informa-
tion. A peer-review system could be implemented by
allowing physicians’ associations or patient organisa-
tions related to a specific app’s topic to ‘adopt’ or
develop peer-reviewed apps. In addition, guidelines for
medical apps, such as the future FDA guideline, should
be used by app-developers and app-reviewers to preserve
and monitor their quality and reliability. Evaluating the
impact these technologies might have on improving
healthcare would be highly advantageous, reducing the
likelihood of medical mistakes and protecting patients.
Medical app-developers should be encouraged to register
their app in an international registry and to submit a
premarket notification to accrediting bodies and medical
experts in order to assess the effectiveness and safety of
the proposed app. It should be noted that one disadvan-
tage of such a system is that it would significantly
decelerate innovations in this industry.
Medical apps selected by individual hospitals
Following the completion of medical app development,
reliable, high-quality apps would be placed in app
stores. Owing to the vast quantity of medical apps avail-
able, finding an appropriate and usable app can be
problematic for physicians. At present, there are several
methods by which medical apps are advertised. One
example is the subset of medical apps offered by specia-
lised commercial companies as part of a substore of
‘useful and high-quality’ apps.20Pharmaceutical com-
panies are also widely involved in the development and
distribution of medical apps.8The inevitable aim of such
companies, however, is to make profit, and a conflict of
interest cannot be ruled out. One way to bypass this
problem would be to allow hospitals, or physicians’
associations, to preselect apps and make them accessible
to their employees, comparable with a medical library.
Future of medical apps
The advances in mobile health technology and the adop-
tion of smart phones means that medical apps will be of
vital importance and an integral part of daily medical
practice in the near future. But while the rapid develop-
ment of medical apps is engaging the attention of health-
care professionals and improving accessibility to medical
knowledge, there is increasing concern regarding the
potential dangers related to the use of medical apps. We
are convinced that, to some degree, medical apps should
be regulated, and that they need to be thoroughly peer-
reviewed in order to ensure validity. Medical applications
should have an assured quality, be scientifically sound
and cost-effective in their use. All stakeholders in the
mobile medical market should be involved in the regula-
tion process. A shared decision-making approach in the
creation of a regulatory guideline would both facilitate its
acceptance among all stakeholders and enhance compli-
ance to the guideline. Figure 1 provides a clear overview
of the different stakeholders in the medical app field.
Governmental healthcare authorities should provide
guidelines which app-developers and reviewers should
follow. Hospitals, healthcare institutions, medical pub-
lishing companies and physicians’ accrediting bodies
play a pivotal role in selecting and providing apps for
healthcare professionals. Looking ahead, we believe that,
since mobile technology has acquired a dominant role in
society, further research on the use and implementation
of medical apps in clinical practice will be necessary. The
integration of medical apps will significantly contribute
to accessible and evidenced-based healthcare. Medical
apps constitute one of medicine’s most dynamic contem-
porary fields, with considerable potential to change the
way healthcare is delivered in the future. Establishing
and Drug Administration intents to
Groups of medical apps the Food
Apps that control a medical devise or display,
store, analyse or transmit patient-specific
medical device data (such as an
Apps that, with help of formulae or
algorithms, output patient-specific results,
such as a diagnosis, treatment
recommendation or differential diagnosis
Apps that transform the mobile platform into
a regulated medical device by using
attachments or sensors or similar medical
Evidence-Based Medicine Month 2012 | volume 0 | number 0 |
appropriate regulatory procedures will enable this poten- Download full-text
tial to be fulfilled, while at all times ensuring patient
Contributors AWGB and BJV contributed equally to
Competing interests None.
1. Visser BJ, Buijink AW. Need to peer-review medical
applications for smart phones. J Telemed Telecare 2012;18:124.
2. Visvanathan A, Hamilton A, Brady RR. Smartphone apps in
microbiology-is better regulation required? Clin Microbiol
3. Hamilton AD, Brady RR. Medical professional involvement in
smartphone ‘apps’ in dermatology. Br J Dermatol 2012;167:220–1.
4. Research2Guidance. Global smart phone application market
report 2010, update 1st half year 2010. 2010.
5. Cox A. Smartphone Apps, Monitoring & mHealth Strategies
2011–2016. 1 Dec 2011. Juniper Research.
6. Franko OI, Tirrell TF. Smartphone app use among medical
providers in ACGME training programs. J Med Syst Published
Online First 3 November 2011. doi: 10.1007/s10916-011-9798-7.
7. Franko OI. Smartphone apps for orthopaedic surgeons. Clin
Orthop Relat Res 2011;469:2042–8.
8. Baumgart DC. Smartphones in clinical practice, medical
education, and research. Arch Intern Med 2011;171:1294–6.
9. Apple Press Info. Apple’s App Store Downloads Top 15 Billion.
20 Aug 2012).
10. AppBrain. Android statistics. Top categories. http://www.
appbrain.com/stats/android-market-app-categories (accessed 13
11. Sherwin-Smith J, Pritchard-Jones R. Medical applications: the
future of regulation. Bull R Coll Surg Engl 2012;94:12–13.
12. Rosser BA, Eccleston C. Smartphone applications for pain
management. J Telemed Telecare 2011;17:308–12.
13. Rigby M, Forsstrom J, Roberts R, et al. Verifying quality
and safety in health informatics services. BMJ 2001;323:552–6.
14. Phillips B. ‘Pfizer Rheumatology Calculator’ iPhone /Android
Application—important information. http://www.mhra.gov.uk/
(accessed 20 Aug 2012).
15. Medical App Journal. http://medicalappjournal.com (accessed 20
16. Husain I, Misra S, Wodajo F. iMedicalApps. http://www.
imedicalapps.com (accessed 13 May 2012).
17. Nolan T. A smarter way to practise. BMJ 2011;342:d1124.
18. Adams B. Digital Pharma Directory—mobile apps. http://www.
16 Feb 2012).
19. U.S. Food and Drug Administration. Draft Guidance for
Industry and Food and Drug Administration Staff—Mobile
Medical Applications (accessed 20 Aug 2012).
20. Happtique. Happtique Mobile Health Source. http://www.
happtique.com (accessed 13 May 2012).
Figure 1 Different stakeholders in developing a guideline for the regulation of medical apps.
Evidence-Based Medicine Month 2012 | volume 0 | number 0 |