ArticlePDF Available

Abstract and Figures

Background: In many telemedicine applications, the correct use of medical device at the point of need is essential to provide an appropriate service. Some applications may require untrained people to interact with medical devices and patients: care delivery in transportation, military actions, home care and telemedicine training.Appropriate operation of medical device and correct connection with patient's body are crucial. In these scenarios, tailored applications of Augmented Reality can offer a valid support by guiding untrained people at the point of need. This study aims to explore the feasibility of using Augmented Reality in telemedicine applications, by facilitating an acceptable use of biomedical equipment by any unskilled person. In particular, a prototype system was built in order to estimate how untrained users, with limited or no knowledge, can effectively interact with an ECG device and properly placing ECG electrodes on patient's chest. Methods: An Augmented Reality application was built to support untrained users in performing an ECG test. Simple markers attached to the ECG device and onto patient's thorax allow camera calibration. Once objects and their pose in the space are recognized, the video of the current scene is enriched, in real-time, with additional pointers, text boxes and audio that help the untrained operator to perform the appropriate sequence of operations. All the buttons, switches, ports of the ECG device together with the location of precordial leads were coded and indicated. Some user's voice commands were also included to improve usability. Results: Ten untrained volunteers, supported by the augmented reality, were able to carry out a complete ECG test first on a mannequin and then on a real patient in a reasonable time (about 8 minutes on average). Average positioning errors of precordial electrodes resulted less than 3 mm for the mannequin and less than 7 mm for the real patient. These preliminary findings suggest the effectiveness of the developed application and the validity of clinical ECG recordings. Conclusion: This application can be adapted to support the use of other medical equipment as well as other telemedicine tasks and it could be performed with a Tablet or a Smartphone.
Content may be subject to copyright.
R E S E A R C H Open Access
Telemedicine supported by Augmented Reality:
an interactive guide for untrained people in
performing an ECG test
Paolo Bifulco
, Fabio Narducci
, Raffaele Vertucci
, Pasquale Ambruosi
, Mario Cesarelli
and Maria Romano
* Correspondence:
Department of Electrical
Engineering and Information
Technology, University of Naples
Federico II, Naples, Italy
Full list of author information is
available at the end of the article
Background: In many telemedicine applications, the correct use of medical device
at the point of need is essential to provide an appropriate service. Some applications
may require untrained people to interact with medical devices and patients: care
delivery in transportation, military actions, home care and telemedicine training.
Appropriate operation of medical device and correct connection with patients body
are crucial. In these scenarios, tailored applications of Augmented Reality can offer a
valid support by guiding untrained people at the point of need. This study aims to
explore the feasibility of using Augmented Reality in telemedicine applications, by
facilitating an acceptable use of biomedical equipment by any unskilled person. In
particular, a prototype system was built in order to estimate how untrained users,
with limited or no knowledge, can effectively interact with an ECG device and
properly placing ECG electrodes on patients chest.
Methods: An Augmented Reality application was built to support untrained users in
performing an ECG test. Simple markers attached to the ECG device and onto
patients thorax allow camera calibration. Once objects and their pose in the space
are recognized, the video of the current scene is enriched, in real-time, with additional
pointers, text boxes and audio that help the untrained operator to perform the
appropriate sequence of operations. All the buttons, switches, ports of the ECG device
together with the location of precordial leads were coded and indicated. Some users
voice commands were also included to improve usability.
Results: Ten untrained volunteers, supported by the augmented reality, were able to
carry out a complete ECG test first on a mannequin and then on a real patient in a
reasonable time (about 8 minutes on average). Average positioning errors of precordial
electrodes resulted less than 3 mm for the mannequin and less than 7 mm for the real
patient. These preliminary findings suggest the effectiveness of the developed
application and the validity of clinical ECG recordings.
Conclusion: This application can be adapted to support the use of other medical
equipment as well as other telemedicine tasks and it could be performed with a Tablet
or a Smartphone.
Keywords: Augmented Reality, Untrained user, ECG device operation, Electrode
© 2014 Bifulco et al.; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative
Commons Attribution License (, which permits unrestricted use, distribution, and
reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication
waiver ( applies to the data made available in this article, unless otherwise
Bifulco et al. BioMedical Engineering OnLine 2014, 13:153
Telemedicine refers to the use of telecommunications and information technologies for
the delivery of medical services where is needed [1-3]. For many applications, the cor-
rect usage of medical device at the point of need is essential to provide an appropriate
service, but there are some practical situations that may require untrained or inexpert
people to interact with medical devices and patients. Some examples are telemedicine
services on transportation (e.g. aircrafts [4-6], boats [7,8], trains, etc.), application dur-
ing military actions [9], on islands or remote areas [10,11], some emergency applica-
tions [12,13], but also home care telemedicine supported by family members [14-16],
elderly care [17,18], operators training and so on.
In these cases, untrained or improvised (but necessary) actors can, involuntarily, use
medical instruments in an inappropriate manner and/or make improper connection
between the patient and the medical device seriously invalidating the telemedicine
In these scenarios, tailored applications of Augmented Reality (AR) can offer a valid
support by guiding non-trained people to a correct usage of medical devices at the
point of need. Augmented reality basically consists of a live view of the real-world in
which some elements of the scene are augmented(enriched, enhanced) by computer-
generated information such as graphics, texts and sounds. The application domains for
AR are numerous and extend in different fields such as training and support, design,
medicine, entertainment and cultural heritage [19-21].
Recently, there is a growing interest about AR in medicine. The main applications of
AR are in the field of surgery, rehabilitation and teaching/training. Interventional medi-
cine, surgery [22,23], laparoscopy and other procedures (e.g. needle biopsy [24]) can be
assisted by integrating preoperative and intraoperative anatomic and functional data
improving the visual perception of the surgeon [20,25-27]. Obviously, surgery AR appli-
cations require very accurate registration and camera calibration [28]. AR and virtual
reality have long since found use in rehabilitation and particularly in neurorehabilita-
tion, by guiding and aiding the patient to perform therapy [29,30]. Teaching and train-
ing of students or physicians can take great advantage by AR, which can be further
enriched with direct haptic and auditory feedback [31-33]. Superimposition in real time
of images from US, CT or MR scans can also help in learning [20].
There are only very few examples of AR applications in telemedicine, among these
there are systems of virtual reality supporting distance teaching of minimally invasive
surgery and systems for interactive telemedicine in the operating theatre [34,35]; some
low-cost peripherals to support telehealth, visualization, education and collaborative
systems [36]; some applications of distance training for the restoration of motor func-
tion, supported by virtual reality [37].
The capability of AR to provide live support to users in operating on instrumenta-
tions is also of interest for this study. AR can support and guide workers in operating,
actuate, disassembling and maintaining complex devices or systems. As example, it was
proposed a mixed reality environment aimed to improve the effectiveness of servicing
and repair procedures in mission critical systems, while reducing the time required for
the intervention. Also technicians with no previous experience on specific, complex de-
vices were able to perform an assigned maintenance task when supported by the AR
application. In maintenance activities there are well-defined sequences of procedures to
Bifulco et al. BioMedical Engineering OnLine 2014, 13:153 Page 2 of 16
be done in a relatively static environment. These features allow a defined design space,
supporting a wide variety of systems and technologies. It was proved effective in sup-
porting workers for maintenance activities [38].
As feasibility study for new possible telemedicine services, an AR application was
built to support untrained users in performing an ECG-test. In particular, the objective
of this study was to analytical assess the benefits of exploiting augmented reality princi-
ples in order to estimate how untrained users with limited or no knowledge can prop-
erly interact with an ECG device and properly placing ECG electrodes on patients
chest. The proposed AR application was assessed in terms of efficacy and clinical
In many clinical activities, 12-lead electrocardiogram is an essential medical investiga-
tion. On the other hand, it should carefully carried out. Misplaced ECG electrodes can
cause changes in ECG recordings, which could have an impact on clinical decisions
[39]. Incorrect electrode cable connections, reversal of electrodes, inadequate place-
ment of the electrodes are common source of error (changes the true ECG morph-
ology) and can conceal or simulate different pathology such as, myocardial ischemia or
infarction, arrhythmias, ventricular hypertrophy [40-43]. It is also worth mentioning
that untrained service providers are one of the key barriers to implementation of tele-
medicine services.
Overall system description
The developed AR application guides the user to perform a predefined sequence of
simple tasks on the ECG device (e.g. connect cables, press buttons, check indicator
lights, etc.) and on the patient (e.g. connect electrodes in a specific position, etc.), driv-
ing users attention from time to time to the relevant item.
A flow-chart of activities was created to describe all possible sequences of simple
tasks necessary to carry out an electrocardiographic test using that particular ECG de-
vice. The developed AR application drives the user to perform each predefined task by
presenting text, graphics and audio messages to him/her (see Figure 1).
Specifically designed sets of markers were attached to the ECG device and to the pa-
tient: this allows the AR engine to evaluate on real-time the 3D pose of these objects
with respect to the user.
This permits to indicate or highlights a specific point or a part of the objects (e.g. a
button to be pressed) by superimposing opportune signs to the current scene. The op-
erators worn a Head Mounted Display (HMD) coupled to a webcam in order to see the
virtual contents that augment the current scene (future developments will involve the
use of a single Tablet or Smartphone). Figure 1 presents a general scheme of the pro-
posed AR application.
The augmented reality engine
The Augmented Reality engine is in charge of users head tracking and scene augmen-
tation/rendering. The tracking system of the developed application is based on the
ARToolKit, an open source AR library [44]. The ARToolKit video tracking libraries
compute the current camera position and orientation relative to physical markers in
Bifulco et al. BioMedical Engineering OnLine 2014, 13:153 Page 3 of 16
real time. Very simple markers (i.e. black squares containing specific identification sym-
bols) are used. Markers are rigidly attached to the objects of interest to estimate users
perspective, i.e. the relative 3D pose of these objects with respect to the camera (cam-
era calibration). For each captured frame, the AR engine performs a defined sequence
of steps. First of all, it looks for known markers in the framed scene. A thresholding op-
eration turns the coloured image into a binary one in order to make the detection of
the markers easier and faster. Once detected all visible markers, the engine computes
the users perspective view (i.e. users relative 3D positioning with respect to the marked
objects). At this stage, the AR engine loads pre-recorded information about each
marked object (e.g. its geometry, the internal spatial coordinates of its parts, etc.) and
then is possible to superimpose virtual objects (e.g. labels, arrows, spots etc) to the real
scene. The augmentedscene is finally displayed to the user by means of proper dis-
plays, such as the HMD.
In normal environmental conditions (i.e. acceptable lighting of the scene, absence of
strong reflections or excessive shadows, etc.), a single marker could be enough to
achieve a reliable augmentation of the scene. However, the marker should be always in
the central field of view of the camera and should be entirely and clearly visible to re-
duce the risk of detection miss. In several contexts, arranging a marker in the middle
of operational field could simply be unfeasible or it could even interfere with the opera-
tions. Furthermore, object characterized by uneven surfaces or significant rotations of
displacements can hinder a continuous and stable detection of the users perspective.
Previous studies [45] have shown that the use of multiple markers (or a constellation
of markers) for each object of interest can address most of the problems arising in
practical situations, thus delivering an inherently more robust and more accurate track-
ing of object, even by using small markers. A useful advantage of the multi marker ap-
proach is that it is easily scalable. In fact, by adding other markers it is possible to
easily widen the tracking volume as desired. The AR application includes a calibration
function meant to measure and compensate the geometrical distortion generated by
Figure 1 System architecture. Scheme of the operations of the developed AR application to guide an
untrained user to correctly perform an ECG test.
Bifulco et al. BioMedical Engineering OnLine 2014, 13:153 Page 4 of 16
the lens of the camera. In addition, a specific procedure allows a manual fine tuning and
coregistration between the camera and its virtual counterpart in charge of rendering the re-
quired graphics. Each of the six degrees of freedom, including the focal length of the camera
and also the threshold used for marker identification, can be precisely adjusted. This task is
performed only once, generally, on the very first system start-up. An ad-hoc graphic user
interface supports this fine tuning calibration procedure. This procedure can be recalled
when the equipment configuration is changed. The AR application memorizes and ex-
changes information using XML (eXtensible Markup Language) files. The language is
widely used and it allows extensibility, unambiguously and large compatibility. All informa-
tion of the accurate location of any relevant element of the objects and of the working envir-
onment is memorized in a database of XML files. More technically, an XML object file
defines the objects points of interest. These include Cartesian coordinates and properties of
augmented aids, such as the location of textual information, audio messages, and the inter-
action region on the object surface to be highlighted. The AR engine builds up the virtual
scene by means of a DOM (Document Object Model) XML parser. To find the required
data in the application database, a XML-based XPath query language is used. Combining
the information from tracker and from XML database, the AR engine is therefore able to lo-
cate users perspective view in the real world and to extract all the required augmented con-
tents from the repository to properly guide him/her during the intervention.
AR application development and set-up
The AR application has to support an untrained user to perform an ECG-test on a pa-
tient. Starting from the established medical guidelines and technical instructions of the
device exploited, each ECG procedure was carefully analyzed and subdivided in very
simple tasks (or steps), achieving the whole formalization of the all possible sequences
of steps. Once we obtained the whole flow chart, it was coded as a deterministic finite
automaton (DFA) that has a finite number of possible states and precise rules for step-
ping form one state to another, producing unique runs of the automaton. Each proced-
ure is an automaton run that starts at the root and ends in a leaf. A particular state
represents a precise ECG-test step and the links with other steps define the order of
the execution of the entire procedure. DFA results particularly suited to model either
simple or complex procedures in an easy and comprehensive way. The DFA representa-
tion was also translated into several XML files where each one represents a run of the
automaton or, on the other hand, a possible procedure. The syntax of this file is very
simple, a tag < step > defines an elementary task of any procedure. Once the parser
reaches the end of the XML document, the procedure is completed.
User is invited to execute each task by an audio message, while a correspondent
text-box is presented on the users screen. Simultaneously, real scene is augmented by
adding graphics (i.e. pointers, spots, etc.) to drive users attention on a particular point
of the objects (e.g. the button to be pressed, etc.). Once accomplished the task, user
sends a vocal command to the AR application to step to the next task. Only very few
vocal commands are allowed such as: Go Nextto pass to the next step; Go Backto
return to the previous step; Redo Instructionto listen to once again the voice
prompt of the current step.
Basically, the ECG-test procedure includes: connection of the electrodes to the pa-
tient (including the six precordial leads V1-V6), connection of the patient cable to the
Bifulco et al. BioMedical Engineering OnLine 2014, 13:153 Page 5 of 16
ECG device, a sequence of operation on the ECG device leading to record 12-leads
ECG on a strip chart.
Each marker used to augment an object is a black square (of 4 cm per side) contain-
ing a unique identification symbol. Markers can be easily printed on paper or card-
board and have to be attached on the object (markers should be rigid and fixed with
respect to the object to be augmented). To allow more reliable and stable functioning,
a costellation of three markers was associated to the ECG device and a different set of
three to the patients thorax. The multi-markers approach significantly improves the ac-
curacy of the augmentation [46] compared to single marker approaches [47,48], thus
reducing the matching error between the rendering of the augmenting point and its
physical location in the real scene. The portable ECG device considered in this applica-
tion was a Cardiette Microruler 12/1. The three markers were arranged in line and
horizontally aligned, spaced from each other by 2 cm and attached to the top of the
front part of the ECG device (see Figure 2a). The geometrical localization of the follow-
ing parts of the ECG device was accurately measured with respect to central point of
the markers costellation (located in the geometrical center of the middle marker) and
then recorded in XML file: the main switch (placed on the left lateral side of the de-
vice), the connector of the patient cable (placed on the right lateral side), all the but-
tons (eight in total, placed on the main panel), all the LEDs (twelve in total, placed on
the main panel), the accessible parts of the chart printer (placed on the main panel).
For the augmentation of the patients thorax, the three markers were arranged on the
tips of a T-shaped stucture showed in Figure 2b. The T-structure was made flat and
rigid, its horizontal and vertical segment of the T-structure measured 10 cm (empiric-
ally chosen according the mean size of adults thorax [49]). The T-structure was applied
with stickers onto the patients thorax, taking care to align the vertical segment along
the sternum and positioning the upper side of the lower marker (recognizable in figure
by its C-like symbol) in correspondence of the xiphoid process (the lower end of the
sternum), which can be easily recognizable by touching (see Figure 2b). The anatomical
landmarks for the standard precordial electrode positions were fixed in accordance
with the current ECG international standards [50]. The precordial electrode positions
are: V1 and V2 at the fourth intercostals space to the right and left sternal border, re-
spectively; V4 at the fifth left intercostal space in the mid-clavicular line; V3 midway
between V2 and V4; and V5 and V6 at the horizontal level of V4 in the anterior and
midaxillary lines, respectively [39,51]. Coordinates x, y and z were obtained for each of
the precordial electrodes with respect to the reference system fixed to the T-shape
marker set (see Figure 2c) and opportunely coded in the XML file. The x-, y- and z-
axis correspond to the latero-lateral, cranial-caudal and dorsal-ventral patients anatom-
ical axes, respectively.
The user (see Figure 3a and 3b) worn an HMD (Silicon Micro Display ST1080) offer-
ing a 1920x1080 pixels video resolution at 60Hz, 45 degrees diagonal field of view, 24
bit stereo audio and a weight of 180 grams. To provide a vision in augmented reality
through the HMD, it was coupled with a Logitech HD Pro Webcam C910 that offers a
very wide diagonal field of view of 83 degrees, 4.3 mm focal length and a maximum
video resolution of 1920x1080 pixels. However, in order to get the best results in term
of augmentation of the scene, we preferred to use a 640x480 pixels video resolution
that allows to work at 30 FPS.
Bifulco et al. BioMedical Engineering OnLine 2014, 13:153 Page 6 of 16
Figure 2 (See legend on next page.)
Bifulco et al. BioMedical Engineering OnLine 2014, 13:153 Page 7 of 16
Experiments setting
To assess the feasibility of the AR application and its benefit, two separated experimental
tests were designed: the first on a life-size mannequin and the second on a real patient. The
experimental tests were carried out involving two groups of 10 people each (14 men and 6
women in total, mean age of 31.3 and median age of 30.5) with no medical expertise, no ex-
perience of ECG test and who never used an ECG device. At each tester was asked to wear
the HMD and, after familiarizing with the system for few minutes, the experimental trial
(See figure on previous page.)
Figure 2 Portable ECG with markers. The portable ECG device adopted for this study with attached on
the correspondent three markers. The side of each marker measures 4 cm (a). The rigid T-shaped frame en-
closing the three markers placed on the thorax of a mannequin. The upper side of the lowest marker is
placed in correspond to the xiphoid process (b). Schematic representation of the reference frame adopted
in the trials and the relative positions of the precordial electrodes V1, V2, V3, V4, V5 and V6 (c).
Figure 3 Untrained user on a mannequin and on the portable ECG. Pictures representing an untrained
user while placing precordial electrodes on the mannequin (a) and operating the ECG-device (b).
Bifulco et al. BioMedical Engineering OnLine 2014, 13:153 Page 8 of 16
was started. The first group of 10 people carried out complete ECG recording sessions
interacting with the ECG device and placing electrodes on the mannequin (see Figure 4a-d).
The second test was performed on a volunteer, male adult acting as patient who lay
supine on a table and breathed normally to resemble a practical case (see Figure 4a-b,
4e-f). Similarly, the second group of 10 untrained people was asked to perform
complete ECG recordings only supported by the AR application.
The ability to perform the required operations, the time required to complete the ECG
recording, the positioning errors of the electrodes and the users judgment were collected
for all the tester. In particular, the three spatial components of the distance between an ex-
pected (true) location of an electrode and its actual placement were recorded as the error
committed by the untrained user. This information was used to quantitatively assess the
efficacy and the clinical acceptability of the developed AR application.
Results and discussion
Each of the untrained tester was able to carry out in an appropriate manner the
ECG-test with the only support of the AR application. All the testers reported that
Figure 4 Examples of augmented scenes. Examples of augmented real scenes: the power button of the
ECG device is surrounded by a light green rectangle, while audio and text invite the user to press it (a);a
little red spot indicates the LED to be checked by the users (b); the location of the V3 precordial electrode
on the mannequin is highlighted with a red circle (c); the location of the V6 precordial electrode is
highlighted with a red circle (d) please note the inclination with respect to the plane of the markers; the
location of the V3 precordial electrode on a real patient is highlighted with a red circle (e); placement of
the V5 precordial electrode on the patients thorax, the location of V5 is highlighted with a red circle (f).
Bifulco et al. BioMedical Engineering OnLine 2014, 13:153 Page 9 of 16
they had not encountered any particular difficulty in interacting with the AR appli-
cation and in carrying out all the requested actions. Some of them reported only
minor problems in perception of distances in the direction perpendicular to their
plane of view. This is due to the use of a single camera, which obviously fails to
accurately render the depth of the scene. The AR application was perceived as in-
tuitive and easy to use, the opportunity to interact with vocal commands was par-
ticularly appreciated.
The execution time over both tests was on average of 8 minutes (including 3 minutes
of ECG recording in the automatic mode) that can be considered acceptable for prac-
tical purposes.
Bar plots in Figure 5 show the averages and standard deviations of the x, y and z
components of errors (computed on all testers) in placement of the precordial elec-
trodes on the mannequin (Figure 5a) and patients chest (Figure 5b). Averages can be
considered as an indication of the accuracy, while standard deviations as a measure of
the precision (repeatability) achieved during the trials.
In addition and in a more concise way, Table 1 reports errors measured as distance be-
tween the expected and the occurred position for each of the precordial electrodes V1-V6.
The average, standard deviation and maximum of these displacement errors are available
for both tests (mannequin and real patient).
For the first test, the average errors in electrode positioning on the mannequin re-
sulted less than or equal to 3 mm, while the standard deviation less than 5 mm.
Involvement of the real patient instead of the mannequin led to a slight increase of
the errors committed. This can be reasonably due to thorax motion due to breathing
or other little movements of the patient. Nevertheless, even in the test of the real pa-
tient the mispositioning of the electrodes resulted less than 7 mm on average and
reached a maximum of 16 mm on V6, these data support the effectiveness of the AR
procedure and the clinical acceptability of the recorded ECG. As a matter of fact, elec-
trode malposition exceeding 25 mm is associated with potentially significant ECG
changes [39]. Taking into account this threshold value and observing the results
achieved during both tests (Figure 5a-b and Table 1), the average errors in electrode
positioning resulted reasonably acceptable and comparable with placement errors usu-
ally made by technicians and nurses in an emergency care department [52]. This sup-
ports the clinical validity of the acquired ECG waveforms by means of the developed
AR system.
It is worth noting that in both tests only for V5 and V6 significant variations of posi-
tioning were registered on the z-axis. This is inherent to the specific positions of these
two electrodes and their relative positioning with respect to the markers on the thorax.
When the user directs his gaze towards V5 or V6, the plane on which lies the marker
(x- and y-axes represented in Figure 2c) results significantly angled and then the 3D
pose errors increase [48]. This mainly occurs because of the natural curvature of the
human thorax. Indeed, many studies have shown that the accuracy of positioning vir-
tual object on the real scene mainly depends on camera distance and viewing angle
with respect to the markers and also on other factors (e.g. size of the marker, focal
length, field of view, pixel resolution, etc.) [48,53,54].
However, in the specific case, the distance and the relative angle between camera and
markers are actually limited. Indeed, when placing the electrodes onto a patient or
Bifulco et al. BioMedical Engineering OnLine 2014, 13:153 Page 10 of 16
Figure 5 Average x, y and z components of positioning errors for precordial electrodes. Averages
(bars) and Standard Deviations (segments) of the errors (distance from the expected location reported in
centimeters for each of the x, y and z axes of the space) in the placement of the precordial electrodes on
the mannequin (a) and on the patients chest (b) by the ten untrained users. V1, V2, V3, V4, V5 and V6 are
the six precordial locations;
Bifulco et al. BioMedical Engineering OnLine 2014, 13:153 Page 11 of 16
when manually operating on the electrocardiograph, the distance between the camera
and markers cannot be greater than the length of operators arm, nor the angle of gaze
be particularly tilted. Hence, the need for manual interaction with objects marked se-
verely restricts the space in which augmented reality operates.
Due to the perspective of the scene taken by the camera, the greatest pose error of
an AR marker lies in the direction that connects the marker with the camera. However,
in this specific application (e.g. in placing the electrodes V1 to V4) the error made on
the z-axis of Figure 2c (coincident with the most probable line of sight of user) results
in practice negligible for the precordial electrodes V1 to V4: the users hand must stops
on patients skin. This, obviously, does not hold for the electrodes V5 and V6 because
of their intrinsic positions. Consequently, the placement of these electrodes suffers
from greater errors (an higher component on z axis, see Figure 5a-b, and consequently
an increase in the positioning error, see Table 1).
Lastly, it is also interesting to note that the errors are consistent with data reported by
the developers of the system ARToolKit [48]. Indeed, if we consider that usually the oper-
ator is located at approximately 30-40 cm from the patient (and therefore from the
markers) with an angle between 0° and 45° the errors predicted by previous studies [48,53]
are contained within +/- 5 mm; while for a greater angle (e.g. 45°-80°) they increase to ap-
proximately +/-12 mm. In the extreme case in which the inclination of the users gaze line
reaches or exceeds 90 degrees with respect to the marker plane, the markers are no longer
in sight making impossible to coherently augment the scene. Even if this event did not
occur during the trials, it is possible and should be taken into account.
This study has highlighted the possibility of using Augmented Reality to support un-
trained user while performing an ECG-test. The developed AR application is at the
proof-of-concept stage and can certainly be improved. Furthermore, it can be easily
generalized to the use of other medical equipment.
AR can have a relevant and positive impact on various telemedicine applications. For
example, telemedicine applications involving acquisition of diagnostic signals onboard
of a plane or a ship by an untrained crew member or even passengers, as well as in the
battlefield by untrained soldiers where it can obtain very beneficial effect. Also other
home-care solutions, where a relative may be request to interact with medical device or
emergency application (e.g. use of a defibrillator by an inexpert user), can take advantage
Table 1 Positioning errors for precordial electrodes
Precordial lead
Test V1 V2 V3 V4 V5 V6
Mannequin Average error [cm] 0.30 0.18 0.23 0.24 0.22 0.27
± SD of the error [cm] ± 0.10 ± 0.10 ± 0.13 ± 0.19 ± 0.11 ± 0.41
Max error [cm] 0.45 0.40 0.45 0.54 0.41 1.42
Patient Average error [cm] 0.40 0.27 0.44 0.46 0.49 0.69
± SD of the error [cm] ± 0.07 ± 0.20 ± 0.20 ± 0.22 ± 0.18 ± 0.28
Max error [cm] 0.50 0.73 0.78 0.81 0.79 1.56
Average, standard deviation and maximum of the displacement errors done by the ten users of the developed AR
application in placing precordial electrodes (V1-V6) during the tests on mannequin and real patient. All data are
expressed in centimeters. The displacement errors were computed as the square root of the sum of the square of the
three Cartesian components (x, y, and z) of the displacement vector.
Bifulco et al. BioMedical Engineering OnLine 2014, 13:153 Page 12 of 16
of such AR applications. More appropriate and reliable usage of medical device can be ob-
tained, but also opens new horizons for novel and/or more advanced telemedicine
However, some issues and question should be fully addressed before moving to clin-
ical applications. While no particular difficulties arose in using AR with medical device
(mainly because they are rigid and their 3D geometry known and fixed), problems
occur when augmentinga generic, alive, human body: large anatomical variations and
patients motion can be accurately taken into account. It is worth to remember that
even if some parts of the human body can be roughly considered rigid, change in pro-
portions should be accounted, e.g., between adults of infants, but also between different
body types. Some parts of the body are in continuous motion such as the chest during
breathing. However, in the conducted trials, with the patient breathing normally, this
phenomenon did not significantly affect the results. These considerations lead to the
use of non-rigid constellations of markers (e.g. proportionally scalable in geometric
proportions) to take into account different body types, or even to a marker-less recog-
nition of the body (however, this requires the use of sophisticated software and can ad-
versely affect the real-time operation). The final AR application should also consider
and properly instruct the operator on how to face with special situation (e.g. in the case
of very hairy patients). The flexibility of the XML language offers the possibility to eas-
ily enrich or to customize the set of operations and warnings. For example, a simple re-
minder for the operator to shave the skin or clip small patches of hair for proper
sticking of electrodes, whether it is the case, can be included in the final AR
This study, being essentially a proof of concept, does not provide solutions to these
problems. Only a small variation in the markers T-structure has been designed to over-
come the problem of excessive users slant. An improved version was obtained by add-
ing two extra markers placed adjacently to the left marker (that indicated by the
trapezoidal symbol) and the lower marker (that indicated by the C-like symbol), but ar-
ranged at 90 degree with respect to the T-structure plane (i.e. laying in anatomical
Figure 6 An improved marker set. Picture of the improved T-structure including the two extra markers
oriented perpendicularly to the patients coronal plane.
Bifulco et al. BioMedical Engineering OnLine 2014, 13:153 Page 13 of 16
sagittal planes). Figure 6 shows the new marker constellation for the patients body.
Very preliminary results seem to confirm the expectations.
Further developments of this study are oriented towards the realization of this AR appli-
cation on mobile devices, i.e. tablets or smartphones (see Figure 7). These devices intrin-
sically incorporate a camera, a relatively large screen, a processor capable of running the
software, and other multimedia interfaces such as speakers and microphone. Possibly, the
software can be also downloaded, at the very moment, from the Internet. The use of tab-
lets or smartphones, today widely disseminated, could bring further benefits.
Written informed consent was obtained from the patient for the publication of this re-
port and any accompanying images.
Competing interest
RV is employed at the Selex ES, an international leader in electronic and information technologies for defense systems
and aerospace, the other authors declare no competing interests.
All authors conceived the study and planned research activity. FN and PA developed the necessary software
application. All the authors planned the practical tests carried out by FN and PA. PA, PB and MR drafted the
manuscript. All authors contributed to revise it and approved its final version.
Authors are very thankful to Selex ES to provide strong support to this study. Selex ES, a Finmeccanica company, is an
international leader in electronic and information technologies for defense systems, aerospace, data, infrastructures,
land security and protection and sustainable smartsolutions.
Author details
Department of Electrical Engineering and Information Technology, University of Naples Federico II, Naples, Italy.
VRLab, University of Salerno, Salerno, Italy.
Selex ES, Giugliano, Italy.
Received: 27 March 2014 Accepted: 16 October 2014
Published: 21 November 2014
Figure 7 Possible AR application on smartphones. Possible, future scenario involving the use of the
proposed AR application on smartphones.
Bifulco et al. BioMedical Engineering OnLine 2014, 13:153 Page 14 of 16
1. Wootton R, Craig J, Patterson V: Introduction to telemedicine. 2nd edition. London: The Royal Society of Medicine
Press Ltd; 2006.
2. Norris AC: Essentials of Telemedicine and Telecare. John Wiley & Sons, Ltd. Published Online: 23 April 2002; Print
ISBN: 97804715315; Online ISBN: 9780470846346; doi:10.1002/0470846348.
3. Bracale M, Monteagudo JL, Rossing N, Hakansson S, Reponen J, Bifulco P: Telemedicine in Europe - the
TELEPLANS project. J Telemed Telecare 2002, 8(Suppl 5):308310.
4. Weinlich M, Nieuwkamp N, Stueben U, Marzi I, Walcher F: Telemedical assistance for in-flight emergencies on
intercontinental commercial aircraft. J Telemed Telecare 2009, 15(Suppl 8):409413.
5. Beltrame F, Boddy K, Maryni P: Adopting telemedicine services in the airline framework. IEEE Trans Inf Technol
Biomed June 2001, 5(Suppl 2):171177.
6. Ferrer-Roca O, Díaz De León RD, de Latorre FJ, Suárez-Delgado M, Di Persia L, Cordo M: Aviation medicine:
challenges for telemedicine. J Telemed Telecare 2002, 8(Suppl 1):14.
7. Amenta F, Dauri A, Rizzo N: Telemedicine and medical care to ships without a doctor on board. J Telemed
Telecare 1998, 4(Suppl 1):4445.
8. Gargiulo P, Helgason T, Bifulco P, Bracale M: Telemedicine applications for seamen in the North Atlantic.InProceed.
of The 12th Nordic Baltic Conference on Biomedical Engineering & Medical Physics. Reykjavik, Iceland; 2002.
9. Wilson KL, Doswell JT, Fashola OS, Debeatham W, Darko N, Walker TM, Danner OK, Matthews LR, Weaver WL:
Using augmented reality as a clinical support tool to assist combat medics in the treatment of tension
pneumothoraces. Mil Med 2013, 178(Suppl 9):981985.
10. Bracale M, Cesarelli M, Bifulco P: Telemedicine services for two islands in the Bay of Naples. J Telemed Telecare
2002, 8(Suppl 1):510.
11. Martiniuk A, Negin J, Hersch F, Dalipanda T, Jagilli R, Houasia P, Gorringe L, Christie A: Telemedicine in the
Solomon Islands: 2006 to 2009. J Telemed Telecare 2011, 17(Suppl 5):251256.
12. Drew BJ, Sommargren CE, Schindler DM, Zegre J, Benedict K, Krucoff MW: Novel electrocardiogram
configurations and transmission procedures in the prehospital setting: effect on ischemia and arrhythmia
determination. J Electrocardiol October 2006, 39(Suppl 4):157160.
13. Oldenburg M, Baur X, Schlaich C: Assessment of three conventional automated external defibrillators in
seafaring telemedicine. Occup Med (Lond) March 2012, 62(Suppl 2):117122.
14. Silver HJ, Wellman NS, Galindo-Ciocon D, Johnson P: Family caregivers of older adults on home enteral
nutrition have multiple unmet task-related training needs and low overall preparedness for caregiving. JAm
Diet Assoc January 2004, 104(Suppl 1):4350.
15. Silver HJ, Wellman NS: Family caregiver training is needed to improve outcomes for older adults using home
care technologies. J Am Diet Assoc June 2002, 102(Suppl 6):831836.
16. Romano M, Cesarelli M, DAddio G, Mazzoleni MC, Bifulco P, Ferrara N, Rengo F: Telemedicine fetal
phonocardiography surveillance: An Italian satisfactory experience. Stud Health Technol Inform 2010, 155:176181.
17. Shah MN, McDermott R, Gillespie SM, Philbrick EB, Nelson D: Potential of telemedicine to provide acute medical
care for adults in senior living communities. Acad Emerg Med February 2013, 20(Suppl 2):162168.
18. Shah MN, Morris D, Jones CM, Gillespie SM, Nelson DL, McConnochie KM, Dozier A: A qualitative evaluation of a
telemedicine-enhanced emergency care program for older adults. J Am Geriatr Soc 2013, 61(Suppl 4):571576.
19. Thomas B, Close B, Donoghue J, Squires J, De Bondi P, Morris M, Piekarski W: ARQuake: An Outdoor/Indoor
Augmented Reality First Person Application. In Wearable Computers, The Fourth International Symposium on:
16-17 October 2000. Atlanta, Georgia, USA; 2000:139146.
20. Van Krevelen DWF, Poelman R: A Survey of Augmented Reality Technologies, Applications and Limitations. The
Int J Virtual Reality 2010, 9(Suppl 2):120.
21. Azuma R, Baillot Y, Behringer R, Feiner S, Julier S, MacIntyre B: Recent Advances in Augmented Reality.
IEEE Computer Graphics Applications November 2001, 21(Suppl 6):3447.
22. Sutherland C, Hashtrudi-Zaad K, Sellens R, Abolmaesumi P, Mousavi P: An augmented reality haptic training
simulator for spinal needle procedures. IEEE Trans Biomed Eng November 2013, 60(Suppl 11):30093018.
23. Alberto Borghese N, Frosio I: Compact tracking of surgical instruments through structured markers. Med Biol
Eng Comput July 2013, 51(Suppl 7):823833.
24. State A, Livingston MA, Hirota G, Garrett WF, Whitton MC, Fuchs H, Pisano ED: Technologies for augmented
reality systems: realizing ultrasound-guided needle biopsies. In Proceedings of the 23rd annual conference on
Computer graphics and interactive techniques (SIGGRAPH 96); 1996:439446.
25. Fuchs H, Livingston MA, Raskar R, State A, Crawford JR, Rademacher P, Drake SH, Meyer AA: Augmented Reality
Visualization for Laparoscopic Surgery. In Proceedings of the First International Conference on Medical Image
Computing and Computer-Assisted Intervention (MICCAI 98). Heidelberg, Germany: Springer; 1998:934943.
26. Navab N, Traub J, Sielhorst T, Feuerstein M, Bichlmeier C: Action- and workflow-driven augmented reality for
computer-aided medical procedures. IEEE Comput Graph Appl 2007, 27(Suppl 5):1014.
27. Feuerstein M, Mussack T, Heining SM, Navab N: Intraoperative laparoscope augmentation for port placement
and resection planning in minimally invasive liver resection. IEEE Trans Med Imaging 2008, 27(Suppl 1):355369.
28. Vogt S, Khamene A, Sauer F: Reality augmentation for medical procedures: System architecture, single camera
marker tracking, and system evaluation. Int J Computer Vision 2006, 70(Suppl 2):179190.
29. Burdea GC: Virtual rehabilitation - benefits and challenges. Methods Inf Med 2003, 42(Suppl 5):519523.
30. Villiger M, Bohli D, Kiper D, Pyk P, Spillmann J, Meilick B, Curt A, Hepp-Reymond MC, Hotz-Boendermaker S, Eng K:
Virtual reality-augmented neurorehabilitation improves motor function and reduces neuropathic pain in
patients with incomplete spinal cord injury. Neurorehabil Neural Repair 2013, 27(Suppl8):675683.
31. Kotranza A, Lok B: Virtual human + tangible interface = mixed reality human an initial exploration with a
virtual breast exam patient. In Proceedings of Virtual Reality Conference, 2008. VR08. IEEE. Reno, Nevada, USA;
Bifulco et al. BioMedical Engineering OnLine 2014, 13:153 Page 15 of 16
32. State A, Chen DT, Tector C, Brandt A, Chen H, Ohbuchi R, Bajura M, Fuchs H: Case Study: Observing a Volume
Rendered Fetus within a Pregnant Patient. In Proceedings of IEEE Conference on Visualization94: 17-21 October
1994. Washington DC, USA; 1994:364368.
33. Sielhorst T, Obst T, Burgkart R, Riener R, Navab N: An Augmented Reality Delivery Simulator for medical
training. In Proceedings of the Workshop AMI-ARCS 2004 Augmented Environments for Medical Imaging:
30 September 2004. Rennes, France: Rennes, France; 2004:1120.
34. Masero V, Sanchez FM, Uson J: A telemedicine system for enabling teaching activities. J Telemed Telecare 2000,
35. Graschew G, Rakowsky S, Balanou P, Schlag PM: Interactive telemedicine in the operating theatre of the future.
J Telemed Telecare 2000, 6:2024.
36. Green M, White L: The Cave-let: a low-cost projective immersive display. J Telemed Telecare 2000, 6:2426.
37. Van Dijk H, Hermens HJ: Distance training for the restoration of motor function. J Telemed Telecare 2004, 10:6371.
38. Henderson S, Feiner S: Exploring the benefits of augmented reality documentation for maintenance and
repair. Visualization Computer Graphics, IEEE Transactions on 2011, 17(10):13551368.
39. McCann K, Holdgate A, Mahammad R, Waddington A: Accuracy of ECG electrode placement by emergency
department clinicians. Emerg Med Australas 2007, 19(5):442448.
40. Rudiger A, Hellermann JP, Mukherjeec R, Follath F, Turina J: Electrocardiographic artifacs due to electrode
misplacement and their frequency in different clinical settings. Am J Emer Med 2007, 25:174178.
41. Wenger W, Kligfield P: Variability of precordial electrode placement during routine electrocardiography.
J Electrocardiol 1996, 29:179184.
42. Harrigan RA, Chan TC, Brady WJ: Electrocardiographic electrode misplacement, misconnection, and artifact.
J Emerg Med December 2012, 43(Suppl 6):10381044.
43. Rudiger A, Hellermann JP, Mukherjee R, Follath F, Turina J: Electrocardiographic artifacts due to electrode
misplacement and their frequency in different clinical settings. Am J Emerg Med 2007, 25(Suppl 2):174178.
44. ARtoolkit official website. In [].
45. Abate AF, Narducci F, Ricciardi S: Mixed reality environment for mission critical systems servicing and repair. In
5th International Conference, VAMR 2013, Held as Part of HCI International 2013, Proceedings, Part II: 21-26 July 2013.
Las Vegas, Nevada, USA: Springer Berlin Heidelberg, 2013; 2013:201210.
46. Yoon JH, Park JS, Kim C: Increasing camera pose estimation accuracy using multiple markers.InAdvances in
Artificial Reality and Tele-Existence. Berlin Heidelberg: Springer; 2006:239248.
47. Abawi DF, Bienwald J, Dorner R: Accuracy in optical tracking with fiducial markers: An accuracy function for
ARToolKit. In Proceedings of the 3rd IEEE/ACM International Symposium on Mixed and Augmented Reality 2004.
IEEE Computer Society; 260261.
48. Kato H, Billinghurst M: Marker tracking and HDM calibration for a video-based augmented reality conferencing
system. In Augmented Reality, 1999. (IWAR99) Proceedings. 2nd IEEE and ACM International Workshop. IEEE:
IEEE; 1999:8594.
49. Lundsgaard C, Van Slyke DD: Studies of lung volume. I relation between thorax size and lung volume in
normal adults. J Exper Med 1918, 27(1):6586.
50. Wilson F, Kossmann C, Burch G, Goldberger E, Graybiel A, Hecht H, Johnston F, Lepeschkin E, Myers G:
Recommendations for standardization of electrocardiographic and vector cardiographic leads. Circulation
1954, 10:564573.
51. Kania M, Rix H, Fereniec M, Zavala-Fernandez H, Janusek D, Mroczka T, Maniewski R: The effect of precordial lead
displacement on ECG morphology. Med Biol Eng Comput 2014, 52(2):109119.
52. Rajaganeshan R, Ludlam CL, Francis DP, Parasramka SV, Sutton R: Accuracy in ECG lead placement among
technicians, nurses, general physicians and cardiologists. Int J Clin Pract 2008, 62(Supp1):6570.
53. Farbiz F, Cheok AD, Liu W, Zhou ZY, Xu K, Prince S, Billinghurst M, Live KH: Three-dimensional content for
augmented reality. IEEE Trans Multimedia 2005, 7(3):514523. doi:10.1109/TMM.2005.846787.
54. Pati S, Erat O, Wang L, Weidert S, Euler E, Navab N, Fallavollita P: Accurate pose estimation using single marker
single camera calibration system. In Proc. SPIE 8671, Medical Imaging 2013: Image-Guided Procedures, Robotic
Interventions, and Modeling, 867126; 2013. doi:10.1117/12.2006776.
Cite this article as: Bifulco et al.:Telemedicine supported by Augmented Reality: an interactive guide for
untrained people in performing an ECG test. BioMedical Engineering OnLine 2014 13:153.
Submit your next manuscript to BioMed Central
and take full advantage of:
Convenient online submission
Thorough peer review
No space constraints or color figure charges
Immediate publication on acceptance
Inclusion in PubMed, CAS, Scopus and Google Scholar
Research which is freely available for redistribution
Submit your manuscript at
Bifulco et al. BioMedical Engineering OnLine 2014, 13:153 Page 16 of 16
... However, the use of an electrocardiograph by untrained users is a problem due to the difficulties with setting up channels. An incorrect electrode placement can result in incorrect ECG data, which may affect clinical decisions [30]; • Electromyography (EMG): These devices are presented in the form of a bracelet with sensors that record data on a person's muscle activity. The obtained data are sources of important information about the person's condition in many areas, such as musculoskeletal rehabilitation, prosthetics, sports, etc. ...
Full-text available
In the organizing of professional training, the assessment of the trainee’s reaction and state in stressful situations is of great importance. Phobic reactions are a specific type of stress reaction that, however, is rarely taken into account when developing virtual simulators, and are a risk factor in the workplace. A method for evaluating the impact of various phobic stimuli on the quality of training is considered, which takes into account the time, accuracy, and speed of performing professional tasks, as well as the characteristics of electroencephalograms (the amplitude, power, coherence, Hurst exponent, and degree of interhemispheric asymmetry). To evaluate the impact of phobias during experimental research, participants in the experimental group performed exercises in different environments: under normal conditions and under the influence of acrophobic and arachnophobic stimuli. The participants were divided into subgroups using clustering algorithms and an expert neurologist. After that, a comparison of the subgroup metrics was carried out. The research conducted makes it possible to partially confirm our hypotheses about the negative impact of phobic effects on some participants in the experimental group. The relationship between the reaction to a phobia and the characteristics of brain activity was revealed, and the characteristics of the electroencephalogram signal were considered as the metrics for detecting a phobic reaction.
... Notably, VR is experiencing growing traction in health sciences due to adequate visualization provisions for procedural simulation, surgical skill refinement, surgical planning, and in-depth gross anatomy education [82]. Bifulco et al. [83] examined AR's effectiveness as an educational aid for untrained individuals lacking medical expertise. The researchers devised an AR system that enabled individuals to perform ECG assessments using a head-mounted display with pointers, text boxes, and audio. ...
Full-text available
Augmented Reality (AR) and Virtual Reality (VR) are poised to revolutionize education by offering immersive and interactive learning experiences. This research comprehensively evaluates the educational applications of AR and VR, specifically emphasizing their impact on student motivation, learning outcomes, engagement, and overall learning experiences. The analysis explores how AR and VR can improve student learning, knowledge retention, and skill acquisition by systematically reviewing existing the literature from diverse educational domains, including K-12 education, higher education, STEM education, professional training, and lifelong learning. Additionally, the research investigates the pivotal role of AR and VR in fostering immersive and interactive learning environments, unveiling how these technologies promote active learning, collaboration, and critical thinking through simulations and interactive experiences. The evaluation considers the potential of AR and VR beyond traditional classroom settings in distance education and assesses the feasibility of virtual classrooms, web-based learning environments, and Massive Open Online Courses (MOOCs). A significant aspect of the study involves understanding student attitudes toward AR and VR technologies and their influence on intrinsic motivation, interest, and enthusiasm for the learning material. Based on a thorough analysis of relevant literature, the research aims to provide practical recommendations for educators to effectively incorporate AR and VR into education practices. The recommendations prioritize a pedagogically sound design, educator training, and accessibility consideration to ensure equitable access for all learners. In summary, this extensive research reveals the significant impact of AR and VR on education by understanding the strengths, limitations, and challenges of making informed decisions on utilizing these technologies to create engaging, impactful learning experiences, fostering a generation of technologically proficient and knowledge-driven learners.
... On the other hand, in less developed or remote coastal regions, the availability of telemedicine services may be limited. Telemedicine may not be feasible in areas lacking reliable internet connectivity or advanced medical equipment [45]. Telemedicine for seafarers would require significant infrastructure development and capacity building. ...
Full-text available
Objective: From medicine via radio to telemedicine, personalized medical care at sea has improved significantly over the years. Currently, very little research has been conducted on telemedicine services and tools at sea. This study aims to review real-time case studies of seafarers’ personalized treatment via telemedical devices published in medical journals. Methods: A literature search was conducted using three libraries such as PubMed (Medline), Cumulative Index to Nursing and Allied Health Literature (CINAHL), BioMed Central, and Google Scholar. The Medical Subject Headings (MeSH) were used for information retrieval and document selection was conducted based on the guidelines of preferred reporting items for systematic reviews and meta-analyses (PRISMA) 2020 flowchart. Selected articles were subjected to quality checks using the Newcastle–Ottawa scale (NOS). Results: The literature search produced 785 papers and documents. The selection was conducted in three stages such as selection, screening, and inclusion. After applying predefined inclusion and exclusion criteria, only three articles on real-time medical assistance with telemedical tools were identified. It is reported that medical attention is delivered to seafarers in real time thanks to advancements in telemedicine, satellite technology, and video conferencing. Conclusions: By improving the quality of medical care and reducing response times for medical emergencies at sea, lives have been saved. There are still several gaps despite these advancements. Medical assistance at sea should therefore be improved to address many of the still unsolved issues.
... From the nonsurgical studies included in this review, there is an interest in AR for tele-sonography, EM, and hospital medicine, with many nonsurgical fields yet to be represented. In the realm of patient evaluation, AR's role is still being separately investigated for electrograms [71], diagnostic procedures related to endoscopy [72], biopsies [73], and urology [74], as well as specialty-specific examinations pertinent to dentistry [75], ophthalmology [76], and dermatology [9], to name a few. ...
Full-text available
Background: Over the last decade, augmented reality (AR) has emerged in health care as a tool for visualizing data and enhancing simulation learning. AR, which has largely been explored for communication and collaboration in nonhealth contexts, could play a role in shaping future remote medical services and training. This review summarized existing studies implementing AR in real-time telemedicine and telementoring to create a foundation for health care providers and technology developers to understand future opportunities in remote care and education. Objective: This review described devices and platforms that use AR for real-time telemedicine and telementoring, the tasks for which AR was implemented, and the ways in which these implementations were evaluated to identify gaps in research that provide opportunities for further study. Methods: We searched PubMed, Scopus, Embase, and MEDLINE to identify English-language studies published between January 1, 2012, and October 18, 2022, implementing AR technology in a real-time interaction related to telemedicine or telementoring. The search terms were “augmented reality” OR “AR” AND “remote” OR “telemedicine” OR “telehealth” OR “telementoring.” Systematic reviews, meta-analyses, and discussion-based articles were excluded from analysis. Results: A total of 39 articles met the inclusion criteria and were categorized into themes of patient evaluation, medical intervention, and education. In total, 20 devices and platforms using AR were identified, with common features being the ability for remote users to annotate, display graphics, and display their hands or tools in the local user’s view. Common themes across the studies included consultation and procedural education, with surgery, emergency, and hospital medicine being the most represented specialties. Outcomes were most often measured using feedback surveys and interviews. The most common objective measures were time to task completion and performance. Long-term outcome and resource cost measurements were rare. Across the studies, user feedback was consistently positive for perceived efficacy, feasibility, and acceptability. Comparative trials demonstrated that AR-assisted conditions had noninferior reliability and performance and did not consistently extend procedure times compared with in-person controls. Conclusions: Studies implementing AR in telemedicine and telementoring demonstrated the technology’s ability to enhance access to information and facilitate guidance in multiple health care settings. However, AR’s role as an alternative to current telecommunication platforms or even in-person interactions remains to be validated, with many disciplines and provider-to-nonprovider uses still lacking robust investigation. Additional studies comparing existing methods may offer more insight into this intersection, but the early stage of technical development and the lack of standardized tools and adoption have hindered the conduct of larger longitudinal and randomized controlled trials. Overall, AR has the potential to complement and advance the capabilities of remote medical care and learning, creating unique opportunities for innovator, provider, and patient involvement.
... (Billinghurst, 2002;Yip et al., 2019). In medicine, the technology has been used e.g. in on-the-fly training of technicians to perform ultra-sonic evaluations (Wang et al., 2017), in teaching untrained users to place electrocardiogram electrodes (Bifulco et al., 2014) or as a visual aid during surgeries (Haavik, 2016;Orring, 2017). ...
Full-text available
Demand for locally produced meat from small-scale slaughterhouses and game�handling facilities brings severe logistical challenges in meat inspection, since such facilities are often situated in rural areas and the costs and time associated with inspection visits to remote establishments are significant. With advances in technology, it might be possible to conduct meat inspections remotely via video link. This thesis aims to determine the performance of remote inspections, both ante mortem and post mortem, where a guided, untrained assistant present at the slaughterhouse relays video and sensory impressions to a veterinarian off-site. Performance was evaluated based on agreement between remote and on-site inspections. As part of this, a practical technological solution, with emphasis on a less-is-more approach, was conceived, assembled and tested in practical use. Comparisons between methods were conducted using Cohen’s kappa-based statistics. On directly comparing findings under 26 different inspection codes or classifications recorded by two veterinarians conducting inspections on 400 carcasses and organs of pigs at a Swedish large-scale slaughterhouse, it was found that the level of agreement between the two methods was generally high, with most findings scoring ‘almost perfect agreement’. When ante mortem inspections were evaluated in a similar way, it was concluded that, due to extremely low prevalence of findings, ante mortem inspections are difficult to perform remotely under current conditions. In conclusion, remote post mortem veterinary inspections appear very promising, but the method needs to be evaluated further.
... -Remote assistance to rescue teams, including through augmented reality tools (e.g., glasses allowing the same view as the rescuers, and voice and sound transmission from the scene) [48][49][50][51]; -Triage support at the operations center to enable the most correct intervention [52,53]; -Support for non-specialist Department of Emergencies in setting up the diagnostictherapeutic programme, favoring the timely centralization of serious cases with high developmental potential, but limiting the avoidable transfer of patients who can be effectively treated locally [54,55] -Clinical follow-up of patients, either at home, at the Department of Emergency or at non-specialist pediatric facilities [56][57][58]; -In the event of a large influx of pediatric patients, it may happen that non-specialist personnel may have to intervene in the children's case. Telemedicine can allow the remote specialist coordination of those dealing with the emergency. ...
Full-text available
Technological innovation can contribute to a reorganization of healthcare, particularly by supporting the shift in the focus of care from the hospital to the territory, through innovative citizen-centered models, and facilitating access to services in the territory. Health and social care delivery modalities, enabled by telemedicine, are crucial in this regard. The objective of his Consensus document, written by the main Italian Scientific Societies involved in the use of telemedicine in pediatrics, is to define a standard for its use at the territorial level in various declinations in the pediatric field; this paper also identifies priority areas for its application and the types of services that most require intervention and investment. The changes that are underway in digital transformation in all sectors are unstoppable, and for the digital transformation to take place in a productive sense, the contribution of not only all health professionals, but also of patients, is necessary. From this perspective, authors from different backgrounds were involved in the drafting of this Consensus and, in the future, other figures, primarily patients, are expected to be involved. In fact, this belongs to the vision of connected care, in which the citizen/patient actively participates in the treatment path so that they are assisted in a personalized, predictive and preventive way. The future scenario must be able to provide for the involvement of patients from the initial stages of planning any treatment path, even in the pediatric age, and increasing, where possible, the proximity of the health service to the families.
... However, pharmacy education is undergoing changes, reflecting the evolution of pharmacy practice and the increasing adoption of new technologies for teaching and learning [35], anatomy [36] and pathology education [37] as well as administrating local anaesthesia in paediatric patients [38]. AR has been proven its feasibility also in prehospital care by assisting in electrocardiography [39], ultrasound [40] and triage of patients [41]. ...
Full-text available
An essential feature of pharmacy education is the teaching of theoretical knowledge with the support of practical work in the laboratory. When properly utilized, laboratory activities have the potential to enhance students’ achievement, conceptual understanding, and positive attitudes towards learning. In this pilot study, an augmented reality (AR) environment was designed and introduced for teaching laboratory skills in pharmacy education at the university level. The AR environment was used by pharmacy students (n = 36), featuring gate questions, information screens, Quick Response codes, think-aloud questions, and instant feedback. The environment was utilized with smart glasses and mobile devices with the aim of comparing the support to students’ performance. User experience was evaluated through self-efficacy beliefs and anxiety towards the technology. As a result, students found the environment a useful supplement to traditional laboratory teaching. Smart glasses and mobile devices were both accepted with great positivity but neither being clearly preferred over the other. Smart glasses were noted to provide sufficient feedback in the right stages of work. In contrast, mobile devices promoted the learning process more than the smart glasses. The self-efficacy results for mobile device use were higher, especially related to device handling and operating the AR environment. The pilot study gives educators valuable insights on the usability of AR technology in guiding laboratory tasks, although future work should involve larger and more diverse samples, as well as different learning tasks.
On average, annual building maintenance and repair cost 1–5% of the initial building cost, which, accumulated over the building life, could even exceed the initial construction cost. New advancements in architecture, engineering, construction, and operation (AECO) are transforming the current practice of building maintenance operations using visualization and sensing technologies. This chapter will describe a use case for the application of augmented reality (AR) visualization in building maintenance and repair. AR enhances user’s perception of the surroundings by overlaying virtual objects on real-world views and can lead to new forms of user interaction. For instance, AR visualization embedded in the building maintenance instruction manual (BMIM) can be used to guide facility managers and repair personnel. We present the design and evaluation of an AR-integrated BMIM to improve the quality of building operation and maintenance. We adopt the design science research (DSR) methodology to carry out a systematic literature review, characterization of AR features that can be applied to BMIM, development of AR artifacts for incorporating into BMIM, and assessment of user performance through experiments with measurements taken with the NASA TLX protocol. Results are implemented in two applications, namely, Living Augmented Reality (LAR) and Manual Augmented Reality (MAR), with different visualization scales. Analysis of users’ workload data indicates that a majority agree that BMIM can be effectively enhanced with a high degree of acceptability using AR on mobile devices. It is also found that this integration will be helpful to future evolution of the BMIM and integration with the Internet of Things paradigm.
Immersive technology refers to technology that enhances reality by blending the physical environment with virtual content or by completely taking the user to a virtual world far away from reality. Different immersive technologies are augmented reality (AR), virtual reality (VR), and mixed reality (MR). As immersive technology is becoming more affordable, user-friendly, pervasive, and ubiquitous, it's been adopted and embraced by several industries. Though its early adopters were from the gaming industry, now it's explored and used by many other industries such as mining, healthcare, and medicine, retail, education, automotive, manufacturing, etc. Using these technologies, medical professionals can improve their competencies, and they will be able to effectively transfer the skill acquired through simulations to the operation theatre. This chapter focuses on uses, benefits, and adoption challenges of Immersive technologies with specific reference to healthcare training.
Full-text available
We are on the verge of ubiquitously adopting Augmented Reality (AR) technologies to enhance our perception and help us see, hear, and feel our environments in new and enriched ways. AR will support us in fields such as education, maintenance, design and reconnaissance, to name but a few. This paper describes the field of AR, including a brief definition and development history, the enabling technologies and their characteristics. It surveys the state of the art by reviewing some recent applications of AR technology as well as some known limitations regarding human factors in the use of AR systems that developers will need to overcome.
Full-text available
Inaccurate electrode placement and differences in inter-individual human anatomies can lead to misinterpretation of ECG examination. The aim of the study was to investigate the effect of precordial electrodes displacement on morphology of the ECG signal in a group of 60 patients with diagnosed cardiac disease. Shapes of ECG signals recorded from precordial leads were compared with signals interpolated at the points located at a distance up to 5 cm from lead location. Shape differences of the QRS and ST-T-U complexes were quantified using the distribution function method, correlation coefficient, root-mean-square error (RMSE), and normalized RMSE. The relative variability (RV) index was calculated to quantify inter-individual variability. ECG morphology changes were prominent in all shape parameters beyond 2 cm distance to precordial leads. Lead V2 was the most sensitive to displacement errors, followed by leads V3, V1, and V4, for which the direction of electrodes displacement plays a key role. No visible changes in ECG morphology were observed in leads V5 and V6, only scaling effect of signal amplitude. The RV ranged from 0.639 to 0.989. Distortions in ECG tracings increase with the distance from precordial lead, which are specific to chosen electrode, direction of displacement, and for ECG segment selected for calculations.
Full-text available
This study was to extrapolate potential roles of augmented reality goggles as a clinical support tool assisting in the reduction of preventable causes of death on the battlefield. Our pilot study was designed to improve medic performance in accurately placing a large bore catheter to release tension pneumothorax (prehospital setting) while using augmented reality goggles. Thirty-four preclinical medical students recruited from Morehouse School of Medicine performed needle decompressions on human cadaver models after hearing a brief training lecture on tension pneumothorax management. Clinical vignettes identifying cadavers as having life-threatening tension pneumothoraces as a consequence of improvised explosive device attacks were used. Study group (n = 13) performed needle decompression using augmented reality goggles whereas the control group (n = 21) relied solely on memory from the lecture. The two groups were compared according to their ability to accurately complete the steps required to decompress a tension pneumothorax. The medical students using augmented reality goggle support were able to treat the tension pneumothorax on the human cadaver models more accurately than the students relying on their memory (p < 0.008). Although the augmented reality group required more time to complete the needle decompression intervention (p = 0.0684), this did not reach statistical significance.
Objectives: To discuss the advantages and disadvantages of rehabilitation applications of virtual reality. Methods: VR can be used as an enhancement to conventional therapy for patients with conditions ranging from musculo-skeletal problems, to stroke-induced paralysis, to cognitive deficits. This approach is called “VR-augmented rehabilitation.” Alternately, VR can replace conventional interventions altogether, in which case the rehabilitation is “VR-based.” If the intervention is done at a distance, then it is called “telerehabilitation.” Simulation exercises for post-stroke patients have been developed using a “teacher object” approach or a video game approach. Simulations for musculo-skeletal patients use virtual replicas of rehabilitation devices (such as rubber ball, power putty, peg board). Phobia-inducing virtual environments are prescribed for patients with cognitive deficits. Results: VR-augmented rehabilitation has been shown effective for stroke patients in the chronic phase of the disease. VR-based rehabilitation has been improving patients with fear of flying, Vietnam syndrome, fear of heights, and chronic stroke patients. Telerehabilitation interventions using VR have improved musculo-skeletal and post-stroke patients, however less data is available at this time. Conclusions: Virtual reality presents significant advantages when applied to rehabilitation of patients with varied conditions. These advantages include patient motivation, adaptability and variability based on patient baseline, transparent data storage, online remote data access, economy of scale, reduced medical costs. Challenges in VR use for rehabilitation relate to lack of computer skills on the part of therapists, lack of support infrastructure, expensive equipment (initially), inadequate communication infrastructure (for telerehabilitation in rural areas), and patient safety concerns.
Conference Paper
Mixed Reality (MR) technologies may play an important role in as-sisting on-site operators during maintenance and repair activities. Nevertheless, industrial equipment augmentation requires a high level of precision when co-registering virtual objects to the corresponding real counterparts. In this paper we describe a comprehensive proposal for a mixed reality environment aimed to improve the effectiveness of servicing and repair procedures in mission critical systems, while reducing the time required for the intervention. The tracking of the user’s point of view exploits a multi-marker based solution for robust and precise augmentation of the operating field. The architecture also features a di-minishing visualization strategy allowing the user to see only the fraction of re-al equipment that is relevant for the maintenance task. A finger color-based tracking provides powerful interaction capabilities by means of a not-instrumented interface exploiting colored fingertips caps. An evaluation study of the proposed MR environment, performed by technicians with no previous experience of MR systems, highlights the potential of the approach.
Visual marker based tracking is one of the most widely used tracking techniques in Augmented Reality (AR) applications. Generally, multiple square markers are needed to perform robust and accurate tracking. Various marker based methods for calibrating relative marker poses have already been proposed. However, the calibration accuracy of these methods relies on the order of the image sequence and pre-evaluation of pose-estimation errors, making the method offline. Several studies have shown that the accuracy of pose estimation for an individual square marker depends on camera distance and viewing angle. We propose a method to accurately model the error in the estimated pose and translation of a camera using a single marker via an online method based on the Scaled Unscented Transform (SUT). Thus, the pose estimation for each marker can be estimated with highly accurate calibration results independent of the order of image sequences compared to cases when this knowledge is not used. This removes the need for having multiple markers and an offline estimation system to calculate camera pose in an AR application.
Background Electrocardiograms (ECGs) are performed by humans, and thus are subject to human error. An underappreciated source of electrocardiographic abnormality is electrode misconnection, both limb and precordial, and improper placement, which is principally an issue with the precordial electrodes due to anatomic variation. Patterns of abnormality exist; recognition allows the emergency physician to avoid mistaking the resulting electrocardiographic findings for true pathology.Objectives The purpose of this clinical review is to describe the patterns of electrocardiographic electrode reversal, misplacement, and artifact and thus make them recognizable to the Emergency Physician.DiscussionCommon limb electrode reversals feature distinctive patterns manifesting as unexpected morphologic and frontal plane axis changes in the QRS complexes in the limb and augmented leads. Precordial electrode misplacement (improper positioning of the electrodes on the chest) is common and may mimic a pseudoinfarction pattern, or ST-segment/T-wave changes, which must be recognized as the result of the misplacement rather than true cardiac ischemia. Precordial electrode reversal should be suspected when the normal R/S wave amplitude transition is violated. Electrocardiographic artifact must be distinguished from dysrhythmia to avoid a potentially hazardous progression to unnecessary diagnostics and therapeutics.Conclusions The hallmarks of electrode misconnection, misplacement, and electrocardiographic artifact can be easily mastered by the Emergency Physician; recognition of these findings can positively impact patient care by avoiding unnecessary intervention secondary to misattribution of findings on the 12-lead ECG to cardiac pathology.