Content uploaded by Jana Zvárová
Author content
All content in this area was uploaded by Jana Zvárová
Content may be subject to copyright.
Empowering Clinicians by eHealth
Technologies in Decision-Making Tasks
Jana ZVÁROVÁ a,b,1, Helena HEROUTOVÁ b, Hana GRÜNFELDOVÁ a,c,
Karel ZVÁRA a,d, David BUCHTELA a,b
aCenter of Biomedical Informatics, Prague, Czech Republic
bDepartment of Medical Informatics,
Institute of Computer Science of the Academy of Sciences, Prague, Czech Republic
cMunicipal Hospital Caslav, Czech Republic
dEuroMISE s.r.o., Prague, Czech Republic
Abstract. We present three types of eHealth applications that can enhance quality
of clinical decision-making. Formalized electronic medical guidelines are bringing
medical knowledge close to clinicians. eHealth tools for evaluation knowledge and
competency in a given clinical decision-making problem are demonstrated by
systems ExaMe and TECOM. The TECOM system supports training of clinical
competence in a given decision-making problems using real clinical cases. The
TECOM system estimates the decision-maker abilities using a coefficient of
prediction or a classical error rate. Transfer of data and knowledge important for
clinical decision-making without language barriers is demonstrated on features of
the European Journal for Biomedical Informatics.
Keywords. clinical decision-making, eHealth, medical guidelines, education,
evaluation
1. Introduction
Medical decision-making is the “core” of clinicians’ working activities. Clinicians not
only use their medical knowledge to make diagnostic, therapeutic and prognostic
decisions, but they also coordinate patient care over time and among multiple providers
and settings. Attempts to evaluate medical decision-making processes and the effort to
get insight in their nature have recently become very up-to-date due to the penetration
of ehealth technologies in medical practice. Formalization of existing medical
knowledge and new results in medical research make it more and more easier to
assimilate all the information, which might be useful in making medical decisions. The
role of biomedical ontologies in medical decision support and diagnostic processes
were described in many different ways, see [1], as well as new approaches to medical
education and formalization of medical knowledge were discussed in [2, 3]. Moreover,
new methods for evaluating clinical decision abilities that contribute to measuring of
clinical competence were published [4, 5].
Further we will mention three types of eHealth applications and their realizations
in the Czech environment that can enhance quality of clinical decision-making.
1 Corresponding Author: Jana Zvárová, Pod Vodarenskou vezi 2, 182 07 Prague, Czech Republic; E-mail:
zvarova@euromise.cz.
Medical Informatics in a United and Healthy Europe
K.-P. Adlassnig et al. (Eds.)
IOS Press, 2009
© 2009 European Federation for Medical Informatics. All rights reserved.
doi:10.3233/978-1-60750-044-5-683
683
2. Formalized Electronic Medical Guidelines
Medical guidelines are used for clinician decision support. They are intended to
improve the quality of patient care and reduce costs. Unfortunately, finding information
contained in conventional (free text form) guidelines may be difficult. A prerequisite
for developing decision support systems that use guidelines is creating formalized
electronic medical guidelines. A number of groups are actively developing computer
interpretable guideline representation languages for this purpose. The Arden Syntax [6]
is perhaps the best known representing language, but there are many other related
languages successfully introduced for representing medical knowledge, such as Asbru
[7], EON [8], GUIDE [9], PRODIGY [10], PROforma [11], GLIF [12], etc. We
designed knowledge representation model [13] based on the GLIF3.5 specification. It is
the universal tool for the formalization of knowledge stored in a free text form (e.g.,
medical guidelines). The XML representation of the graphic model and the creation of
the data interface in a paramodel form make it possible to use it in different types of
applications and to connect to real data from different sources (e.g., XML or CSV files).
There is a big potential of formalized electronic clinical guidelines to enhance quality
of clinical decision-making, decease the number of medical errors and save human and
financial resources. However the use of formalized electronic clinical guidelines is also
highly dependent on factors of legal, social and ethical environment.
3. Tools for Evaluation Knowledge and Competency in a Given Clinical Decision-
Making Problem
Although clinicians learn to cope with uncertainty, they seldom master it. In this case
eHealth applications that support learning processes for a given decision-making task
can rapidly increase quality of clinical decision-making. We have developed two
applications that can be used in this area: ExaMe system and system TECOM.
3.1. ExaMe System
Since 1998 the ExaMe system for evaluation of a targeted knowledge has been
developing [14]. The idea of the system is based on generalized multiple-choice
questions, with no prior restrictions on the number of given answers. The only
restriction is that at least one answer is correct and at least one wrong. This new idea
has led to new concepts of standardization of test results and also to new research
problems in statistics.
Evaluation by the ExaMe system is performed using fixed or automated test. A
fixed test is appropriate for evaluation of the group of students in computer classroom
connected to Internet. An automated test is appropriate for self-evaluation on remote
places. Students can pass evaluations by automated tests by themselves and the final
results of the tests are displayed immediately. The displayed results also give
explanation to students why some answers were not correct. The ubiquity of the
Internet and its World Wide Web applications made it possible to realize the new
educational goals in an innovative and creative way.
New features of the ExaMe evaluation system and statistical issues of evaluation
were described in [15]. During the last decades the reliability of didactical tests has
often been examined. It is easy to show (see for example [16]) that equivalently the
Jana Zvárováetal. /Empowering Clinicians by eHealth Technologies in Decision-Making Tasks684
reliability can be expressed as the squared value of the correlation between the
observed score X and the true score T, corr2(X,T). It means that the reliability of a
didactical test can be understood as the strength of the relationship between the score
reached by a student and his true knowledge. The ExaMe system covers also
knowledge on decision-support systems, expert systems and medical guidelines and
can be used for evaluation of students’ knowledge in the field.
Higher education programme on Biomedical Informatics for Ph.D. studies was
presented in [17]. Nowadays, topics of biomedical informatics are covered by five
different types of courses in the Czech language. The courses are focusing on health
information systems and electronic health records, telemedicine, bioinformatics and
biostatistics, knowledge discovery and decision support systems, standards,
interoperability, safety and security, evidence-based medicine and other topics. The
total number of participants in these courses in the years 2006–2007 was 132, the
number of successful graduates was 109. Target groups in these courses were
postgraduate doctoral students of medical faculties, clinicians and general practitioners,
health managers and other healthcare workers. Participants in these courses were
coming from more than 60 workplaces. These courses highly used e-learning tools, e.g.,
electronic books, ExaME program, multimedia presentations of lectures and different
software tools.
3.2. TECOM System
An important part of teaching medical decision-making is the method of measuring the
increase in the decision-maker’s ability to make correct decisions. The new TECOM
system (TEsting COMpetency) can evaluate decision-making ability of decision-
makers for any decision-making task. Simultaneously, clinical competence expressed
by the prediction coefficient is calculated. The use of the TECOM program is
demonstrated by an example from cardiology using real cases from the Municipal
hospital in Caslav.
Table 1. Part of the data matrix from the field of cardiology
Diagnosis Sex Age Height Weight BMI Hypertension
Cardiac infarction male 53 181 86 26.3 yes
Cardiac infarction female 59 * 76 * no
Cardiac infarction female 71 155 69 28.7 no
Cardiac infarction male 72 175 94 30.7 yes
Cardiac infarction female 80 * * * yes
Cardiac infarction male 78 * 72 * yes
Cardiac infarction female 47 159 100 39.6 yes
Pulmonary embolism male 42 * 89 * no
Pulmonary embolism female 63 165 68 25 no
Pulmonary embolism male 71 * * * yes
Pulmonary embolism female 81 * * * yes
Pulmonary embolism male 83 * 72 * Yes
The TECOM system was created in C++ Builder. In essence, it is capable to test
competency of decision makers in any area. The system acquires information for a
decision-making task from a data matrix that can contain any type of data. The data
matrix contains information on cases (e.g., patients) in the rows and correct decisions
as headings of rows. Correct decisions in the data matrix (e.g., correct diagnosis,
optimal therapy, validated prediction of a patient health state) should be carefully
validated by experts (e.g., physicians). Table 1 shows an example as a part of the data
Jana Zvárováetal. /Empowering Clinicians by eHealth Technologies in Decision-Making Tasks 685
matrix for a decision-making task from cardiology translated to English. We use the
TECOM system for evaluating competencies of Czech clinicians and medical students,
therefore we developed the Czech version of the TECOM system. However, the
TECOM system can be translated to other languages if necessary.
The data matrix, partly displayed in Table 1, is based on real cases described in
medical reports generated in the Municipal hospital in Caslav. 76 patients (cases) were
included in the data matrix and the data model contains 75 features describing each
patient. The headers in the first row are used by the TECOM system for the questions
to be asked. A generator of random numbers selects one row (i.e., data about an
examination of one patient and the final correct diagnosis, which of course the tested
student cannot see). The student can ask about values of symptoms, signs and
laboratory tests for the selected patient. The student can stop the process of asking
about these values any time. After that a new window appears with a list of possible
decisions (e.g., diagnoses). Then a student can distribute 100 points among possible
decisions. If the student is quite sure about the correctness of his/her final decision,
he/she gives 100 points to this decision. If he/she is not quite sure about the final
decision he/she divides points among more decisions, where one decision has a
maximal number of points. The decision with the highest number of points (highest
subjective probability) is the final decision made by the student. In case that the final
decision of the student is the same as the correct decision in the data matrix we classify
the final decision of the student as the correct one.
Then decision-making results of a student are evaluated by the traditional error rate
technique (percentage of false and correct decisions)and by the prediction coefficient Q,
¦
n
i
i
npQ
1
2
1),1(log (1)
where n is the number of cases and pi is the probability of the correct outcome at the
i-th case (i.e., pi states how much the student’s final decision coincides with the correct
decision described in the data matrix, e.g., diagnosis given by an experienced
cardiologist). The prediction coefficient was proposed in [4] and it seems to reflect well
the competence of a student (decision-maker) in the given decision-making problem.
The TECOM system can help clinicians reveal more explicitly their decision-
making competencies and enhance their medical knowledge from cases and correct
decisions stored in the data matrix.
4. European Journal for Biomedical Informatics
In the year 2005 we came with a new initiative to publish the European Journal for
Biomedical Informatics (EJBI) on Internet (http://www.ejbi.eu/). The journal gives the
possibility to publish papers in original English versions with translations to other
European languages simultaneously. The multilingual versions of papers help to solve
problems with terminology and support semantic interoperability. Methods of
Information in Medicine (Methods) is a journal stressing the basic methodology and
scientific fundamentals of data, information and knowledge in medicine and health care.
As an official journal of IMIA, it has an international focus and readership. The editors
of Methods and EJBI agreed on future more close collaboration with the aim to better
meet future demands of timely and high-quality, peer-reviewed publications for a broad
readership, including possibilities for multilingual publications. Therefore EJBI is
Jana Zvárováetal. /Empowering Clinicians by eHealth Technologies in Decision-Making Tasks686
bringing new knowledge to clinicians in their own language that avoids
misunderstandings of notions or procedures.
5. Conclusion
We demonstrated several eHealth applications that can help clinicians to cope better
with uncertainty in decision-making tasks and increase their knowledge and skills. We
stressed the role of education and training in the field as well as the knowledge transfer
in the most understandable native language of clinicians.
Acknowledgement. The work was supported by the project 1M06014 of MSMT CR.
References
[1] Bodenreider, O. (2008) Biomedical ontologies in action: Role in knowledge management, data
integration and decision support. IMIA Yearbook of Medical Informatics 2008, Methods of Information
in Medicine 47(Suppl 1):67–79.
[2] Hovenga, E., Bricknel, L. (2006) Changing academic roles – New approaches to teaching and distance
learning in Australia. Methods of Information in Medicine 45, 288–293.
[3] Van Bemmel, J.H. (1985) Formalization of medical knowledge. In Van Bemmel, J.H., Grémy, F.,
Zvárová, J. (Eds.) Medical Decision Making: Diagnostic Strategies and Expert Systems, North-Holland,
Amsterdam, 1–11.
[4] Shapiro, A.R. (1977) The evaluation of clinical predictions. New England Journal of Medicine
296:1509–1514.
[5] Zvarova, J., Jedlickova, A., Zvara, K. (1991) Computer-based evaluation of antibiotic therapy
predictions. International Journal of Biomedical Computing 29:207–213.
[6] Buchtela, D., Peleška, J., Veselý, A., Zvárová, J., Zvolský, M. (2008) Guideline knowledge
representation model. European Journal for Biomedical Informatics,
http://www.ejbi.eu/articles/200812/34/1.html.
[7] Peleg, M., Ogunyemi, O., Tu, S. et al. (2001) Using features of Arden Syntax with object-oriented
medical data models for guideline modeling. AMIA Symposium Proceedings 2001, 523–527.
[8] Shahar, Y., Miksch, S., Johnson, P. (1998) The Asgaard project: A task-specific framework for the
application and critiquing of time-oriented clinical guidelines. Artificial Intelligence in Medicine 14:29–
51.
[9] Tu, S.W., Musen, M.A. (1999) A flexible approach to guideline modeling. AMIA Symposium
Proceedings 1999, 420–424.
[10] Quaglini, S., Stefanelli, M., Lanzola, G., Caporusso, V., Panzarasa, S. (2001) Flexible guideline-based
patient careflow systems. Artificial Intelligence in Medicine 22:65–80.
[11] Johnson, P.D., Tu, S.W., Booth, N., Sugden, B., Purves, I.N. (2000) Using scenarios in chronic disease
management guidelines for primary care. AMIA Symposium Proceedings 2000, 389–393.
[12] Fox, J., Patkar, V., Thomson, R. (2006) Decision support for healthcare: The PROforma evidence base.
Informatics in Primary Care 14:49–56.
[13] Ohno-Machado, L., Gennari, J.H., Murphy, S.N. et al. (1998) The GuideLine Interchange Format: A
model for representing guidelines. Journal of the American Medical Informatics Association 5:357–372.
[14] Zvarova, J., Zvara, K. (2000) Evaluation of knowledge using ExaMe program on the internet. In
Iakovidis, I., Maglaveras, S., Trakatellis, A. (Eds.) User Acceptance of Health Telematics Applications,
IOS Press, Amsterdam, 145–151.
[15] Martinkova, P., Zvara Jr, K., Zvarova, J., Zvara, K. (2006) The new features of the ExaMe evaluation
system and reliability of its fixed tests. Methods of Information in Medicine 45(3):310–315.
[16] Sireci, S.G., Wainer, H., Braun, H. (1998) Psychometrics, Overview. In Armitage, P., Colton, T. (Eds.)
Encyclopaedia of Biostatistics, John Wiley&Sons, New York, 3577–3602.
[17] Zvarova, J. (2008) Education in biomedical informatics and eHealth. In Blobel, B., Pharow, P., Nerlich,
M. (Eds.) Combining Health Telematics, Telemedicine, Biomedical Engineering and Bioinformatics to
the Edge, IOS Press, Amsterdam, 323–337.
Jana Zvárováetal. /Empowering Clinicians by eHealth Technologies in Decision-Making Tasks 687
