Content uploaded by Wilbert Tabone
Author content
All content in this area was uploaded by Wilbert Tabone on Jul 25, 2019
Content may be subject to copyright.
Wilbert Tabone
47
Digitised maps of the Ħal Saieni Hypogeum
and St Paul’s Catacombs, courtesy: Steven Psaila
Cultural Heritage (CH) has been dened as an
expression of the ways of living developed
by a community and passed on from
generation to generation, including customs,
practices, places, objects, artistic expressions and
values.1 CH is split into two categories, namely
tangible and intangible CH. e former refers to
the physical artefacts of a society including built
heritage, artistic creations and other physical
products that are imbued with cultural elements
of society. Conversely, intangible CH refers to
non-physical practices, expressions, knowledge,
representations and skills which are recognised as
a vital component of CH2 and that are constantly
being re-evolved by communities.3
As technology advanced in the last decades, it
has seen gradual permeation in the CH domain,
namely in museums and archaeological sites. is
has seen the emergence of the eld of Cultural
Computing (CC) which applies computer
technology and scientic methods to culture,
arts, and the social sciences to represent, enhance,
extend, and transform creative products and
processes.4 is has allowed unprecedented ease
of access to both visitors and researchers to CH
which was previously inaccessible or in danger
of being lost forever. Some of the most popular
domains have been digitisation processes and
the application of their outputs to develop mixed
reality applications, together with the various
projects involving articial intelligence and
video games. is article shall attempt to give a
brief overview of the diverse number of major
technologies enhancing the researcher’s work
and the user experience of CH sites.
Digitisation
e need for dierent forms of CH digitisation
dates back several decades and has been made
explicit in various EU Commission policies.
ese stress the need for the democratisation of
goods that have value for all humanity, through
digitisation, accessibility and interoperability
in order to enable sharing of information and
responsibilities aimed at conserving cultural
identity.5 is drive has led to projects such as
Europeana, which is a digital cultural heritage
platform for Europe with digitised cultural
material, including a digital library, archive and
museum. Europeana now provides access to
some 30 million cultural objects from more than
2,500 organisations.
Digitisation has also been widely adopted
in archaeology, where discovered artefacts
are digitised for further study or application,
for example to be used in extended reality
applications. Entire sites are also scanned using
photogrammetric techniques to create maps6
Technology for
Cultural Heritage
48
Technology for Cultural Heritage
49
and the generating of GIS data for further study
on the topology of the site.7
e use of extended reality
With the creation of large digitised data sets,
it is no surprise that mixed reality systems have
become increasingly popular in museums and CH
sites. Augmented Reality (AR), Virtual Reality
(VR) and Mixed Reality (MR) technologies enable
a user-centred experience that makes cultural
heritage digitally accessible anywhere, especially
when physical access is constrained. VR enhances
the users’ presence through a computer generated
environment which cannot be interacted with
and oers no opportunity to see the real-world.
On the other hand, AR superimposes virtual
elements on the real world layer whilst MR blends
both real and virtual environments.8 Extended
Reality (XR) refers to all real and virtual combined
environments and human-machine interactions
generated by computer technology and wearables.
Hence, it can be seen as the umbrella term
encompassing all others.
It is generally understood that museum
installations that do not introduce new
technologies are regarded to be less interesting
and thus attract fewer visitors.9 Experiences
relying simply on textual panels and descriptions
may be informative but they are not interactive.10
erefore, the creation of an intelligent
environment responsive to the presence of
visitors which adapts dynamically makes the
visiting experience far more appealing.11
is is further attested by an experiment
conducted in an art museum which involved
three groups, each assigned a dierent guiding
mode (non-guided, audio-guided and AR-
guided). Results indicated that overall the
learning performance of the AR-guided group
was superior to that of the other two groups.
Similarly, the time spent observing a painting
was highest for the AR-guided group followed
by the audio-guided and non-guided group with
text panels only.12 e AR-guided group felt that
the guide device provided a greater motivation
to learn and was more eective in delivering the
message in a “livelier and less-boring” manner,
due to its extra visual commentary. In fact, the
non-guided group expressed their desire to have
a guide or commentary to aid them understand
and appreciate the paintings, as they felt that
attempting to appreciate the artworks on their
own proved to be ineective. is concluded
that the AR-guiding system eliminated certain
limitations of other systems such as interactive
media kiosks or video players which do not oer
instant comparisons with what is being exhibited,
leading to a reduction in the visitor’s interest and
subsequently the viewing time.
erefore, it is no surprise that mixed reality
has seen a surge in applications for the CH domain.
Where this is concerned, augmented, virtual and
mixed reality technologies have been utilised
in CH for a number of diverse applications,
including education, exhibition enhancement,
reconstruction of lost experiences, virtual
museums and exploration. ese categories may
overlap depending on the application, such as a
virtual museum which may be utilised as part of
an exhibition.
Education is concerned with the
dissemination and diusion of the historical
aspects of both tangible and intangible CH. Such
playful learning improves learning outcomes
since it appeals to students’ curiosity and
willingness to share their experiences through
their eagerness to use new technologies. It also
encourages a deeper immersion and interaction
with the exhibition, achieved through the use of
mixed reality technology.13
e use of mixed reality for exhibition
enhancement is intended to improve the visitor’s
experience at a physical museum or heritage site
both indoors and outdoors. A virtual element
such as guide maps, descriptions or virtual-human
characters are superimposed on the users’ real-
world layer as they explore the exhibition space.14
Exploration supports the users in visualising
and exploring historical as well as current views
of CH sites in order to discover, visualise and
manipulate content leading to new knowledge
creation. In such contexts, the users are usually
experts in their eld and hence, the designers of
the application can assume prior knowledge of
domain-specic visualisation when developing
the system. In the last two decades, archaeology
has beneted greatly with the widespread
availability of digitised 3D content, allowing
developers to represent dicult-to-reach
environments using such exploration systems.15
In the context of Malta, imagine
utilising such systems in order to access the
unreachable parts of St Paul’s Catacombs,
the Ħal Saflieni Hypogeum or Għar Dalam,
In 2017 Heritage Malta, with the cooperation
of Louvre Museum-Paris, brought back the
famous dagger of Grand Master Jean de Valette
for a historic visit to be exhibited in Malta, after
Bonaparte had kept the dagger for himself in
1798 – since the accompanying ceremonial
sword was too fragile for travel, it was virtually
displayed next to the dagger, courtesy:
Stefania Zuccarello
Left: A visitor experiencing VR during the launch of Wonders of Malta
by Heritage Malta and Google in 2017, courtesy Steven Psaila;
Above: the Guva or oubliette, a prison originally intended as a water
cistern beneath Fort St Angelo, only accessible through a trapdoor,
can now be experienced through Wonders of Malta with a 360o view
of the grati carved by imprisoned knights – allegedly the most
notable prisoner was Caravaggio
NB: Scan QR code to view grati
50 51
Technology for Cultural Heritage
paving the way for further studies and new
discoveries to be made possible. Similarly,
reconstruction enables users to visualise and
interact with reconstructed historical views of
both tangible and intangible CH that existed
only in the past, or only partially exist today.
However, contrary to exploration, it does not
solely cater for experts, and interacting with
such systems does not necessarily lead to
knowledge creation. AR and MR technologies
can superimpose the reconstructed views over
their historical location, together with an
additional information overlay,16 making them
more suitable for tangible CH and a blend of
both tangible and intangible CH applications.
Virtual museums simulate physical museums
and CH sites with their tangible and intangible
artefacts. Most of the time these artefacts are
too fragile or inaccessible to the public, and
therefore these are realistically simulated with
great detail so that users cannot easily discern
the dierences between the originals and their
digital replicas. Such simulations allow users
to feel as if they are physically present in the
actual museum or CH site. Some simulations
may also go as far as including avatars of other
visitors which mimic their movements and
actions through sensors. is creates a hyper-
realistic real-life scenario for all users who
can also interact via gesture, gaze, speech, and
movement.17
Articial Intelligence
Articial Intelligence (AI) has been applied to
various areas of CH and has addressed topics
such as the linking of museum narratives
with online data sources, or other narratives
and museum artefacts which may aid with
interpretation and storytelling. Semantic digital
archives have also been created in order to
provide user-friendly, eective and ecient
search with retrieval capabilities. ere are also
ventures in creating Natural User Interfaces
(NUI) for CH in a semi-virtual environment
(augmented reality) and autonomous mobile
robots for monitoring archaeological sites
faster, cheaper and safer.18
Another area of AI is computer vision (CV)
which is the study of enabling machines to
interpret and analyse images.19 A particular
application utilising computer vision was
the linking of portraits according to similar
facial characteristics20 of works by Mattia
Preti (1613-1699) and Francesco Zahra (1710-
1773). Hence, portraits housed in different
museums or galleries and created by the same
or different artists, may be linked through a
common model sitting or inspiration.
This gave rise to the idea of visitors taking
selfies upon entering a museum or gallery, and
discovering which portrait resembles their
own face.
In 2013 Heritage Malta launched a 3D
virtual facial reconstruction of an over
5,000-year-old prehistoric skull found
at the Xagħra Stone Circle in Gozo, now
permanently exhibited at the Ġgantija
visitor centre close to the site where it was
excavated
Two artworks depicting the same subject are
correctly linked using a facial recognition
system, source: Tabone et al., 2015
52 53
Technology for Cultural Heritage
Digital Games for Cultural Heritage
In the last decades, video gaming has become a
major component of Western lifestyles. Video
games have provided CH researchers and
archaeologists with a perfect way to present their
research in a media format that incorporates
multiple perspectives and alternative narratives
to audiences which are not normally engaged
with other forms of academic literature or media
regarding CH or archaeology.21
is is no surprise, as unlike traditional
media, video games are capable to deepen
the understanding of cultural heritage in a
very interactive way. A number of existing
commercial entertainment video games can
also be used for non-leisure purposes including
for presenting CH,22 especially the so-called
documentary games which realistically depict
past historical events such as battles and wars.23
Examples include Total War: ree Kingdoms
(Creative Assembly, 2019) and Assassin’s Creed:
Unity (Ubiso Montreal, 2014) which can be
used for educational purposes through their
historical accuracy and realism. For example,
the latter is a 3D action adventure game set in
Paris during the French Revolution (late 1700s)
which has great visual and behavioural accuracy.
Famous historical characters such as Napoleon
Bonaparte, Louis XVI of France and the Marquis
de Sade are recreated and mixed with ctitious
characters. e essence and historical events
of the epoch are transmitted very realistically
together with recreations of locations and
buildings of the real world. It also oers a great
insight into the behaviours, customs and general
ambiance of the era for the player.
Moreover, there are a number of serious
games (SGs) also known as applied games or
‘games with a purpose’ which are designed with a
special intent separate from pure entertainment24.
Examples of their use in CH includes interactive
virtual museums, where gaming technology
is utilised for the purpose of entertaining and
educating visitors25 and usually incorporates
exploration and quizzes about various elements.
Examples include Virtual Egyptian Temple and
Walk through Ancient Olympia. ere are also
prototypes and demonstrators which are games
based on precise 3D virtual reconstructions
from geographic data of ancient historical sites
that usually provide realistic archaeological
exploration. ese include political, religious
and artistic walk-throughs utilising historically
accurate procedurally generated non-player
characters (NPCs). Such examples include
Pompei: e Legend of Vesuvius edutainment
game and Roma Nova. An extension of these,
are games specically used for the acquisition
of cultural knowledge and intangible heritage
The historical character of
Napoleon Bonaparte as
depicted in Assassin’s Creed:
Unity, source: Ubisoft Montreal,
2014 Heritage Malta sites available on Google:
Wonders of Malta with street view technology to
navigate various museums and sites almost as if
the viewer is present e.g. clockwise starting from
Top left: Tarxien Temples, Ta’ Kola Windmill-Gozo
and St Paul’s Catacombs-Rabat
which include intercultural skills and language
training through high-delity 3D simulation of
cultural settings. e nal categories are social
tagging and knowledge acquisition games. ese
encourage players to submit accurate information
on CH artefacts with the purpose of knowledge
verication and data mining so as to enhance the
corpus of knowledge relating to various tangible
and intangible CH elements.26
Looking ahead
is article demonstrates that technology has
truly changed and shied the area of CH. In
this day and age, cultural artefacts from across
the world are accessible to researchers and
enthusiasts at the click of a button. Museums can
be accessed virtually through mobile devices and
carried in one’s pocket. Inaccessible CH locations
can now be easily explored through the use of
virtuality. Articial intelligence is paving the
way to an exciting future, where breakthroughs
in CH research would be possible through new
tools and techniques that can handle the massive
data generated. As technological advancement
is fully embraced by CH researchers, a truly
exciting future awaits.
Notes & References
1 ICOMOS 2002.
2 Ahmad, 2006.
3 Lenzerini, 2011.
4 Haydar, et al. 2011; Wang 2009.
5 Bekele, et al. 2018.
6 Georgopoulos, 2017.
7 Costantino and Angelini, 2010.
8 Bekele, et al. 2018.
9 Gerval and Le Ru, 2015.
10 Lu, et al. 2014.
11 Manovich, 2006.
12 Chang, et al. 2014.
13 Gargalakos, et al. 2011.
14 Choi, 2014.
15 Comes, et al. 2014; Haydar, et al. 2011.
16 Saggio and Borra, 2011.
17 Bekele, et al. 2018.
18 Bordoni and Mele, 2016.
19 Szeliski, 2010.
20 Tabone and Seychell, 2015.
21 Morgan, 2016.
22 Anderson, et al. 2009.
23 Burton, 2005.
24 Gee, 2003.
54 55
Technology for Cultural Heritage
25 Sylaiou, et al. 2009.
26 Bontchev, 2015.
Bibliography
LU, WEIQUAN, Linh-Chi NGUYEN, Teong Leong CHUAH,
and Ellen Yi-Luen DO, 2014, “Eects of mobile AR-
enabled interactions on retention and transfer for
learning in art museum contexts”, Proceedings of
the 2014 IEEE International Symposium on Mixed
and Augmented Reality-Media, Art, Social Science,
Humanities and Design (ISMAR-MASH’D’14), pp.3-11.
AHMAD, Yahaya, 2006, “The scope and denitions of
heritage: from tangible to intangible.” International
journal of heritage studies 12 (3): pp.292-300.
ANDERSON, E. F., L. MCLOUGHLIN, F. LIAROKAPIS, C.
PETERS, P. PETRIDIS, and S. D. FREITAS, 2009, “Serious
games in cultural heritage”, Proc. of 10th Int. Symp. on
Virtual Reality, Archaeology and Cultural Heritage VAST.
BEKELE, K. M., R. PIERDICCA, E. FRONTONI, E.S.
MALINVERNI, and J. GAIN, 2018, “A survey of
augmented, virtual, and mixed reality for cultural
heritage.” Journal on Computing and Cultural Heritage
(JOCCH) 11 (2): p.7.
BONTCHEV, Boyan, 2015, “Serious Games for and as
Cultural Heritage.” Bontchev, Boyan “Serious games
for and as cultural heritage.” The 5th International
conference on digital presentation and preservation of
cultural and scientic Heritage, Tarnovo.
BORDONI, Luciana, and Francesco MELE, 2016, Articial
Intelligence for Cultural Heritage, Cambridge:
Cambridge Scholars Publishing.
BURTON, J., 2005, “News-game journalism: History,
current use and possible futures”, Australian Journal
of Emerging Technologies and Society 3 (2): pp.87–99.
CHANG, Kuo-En, Chia-Tzu CHANG, Chia-Tzu HOU, Yao-
Ting SUNG, Huei-Lin CHAO, and Cheng-Ming LEE,
2014, “Development and behavioral pattern analysis
of a mobile guide system with augmented reality for
painting appreciation instruction in an art museum”,
Computers and Education (Elsevier) 71: pp.185-197.
CHOI, Hee-soo, 2014, “The conjugation method of
augmented reality in museum exhibition”, Int. J. Smart
Home 8 (1): pp.217-228.
COMES, Radu, Călin NEAMŢU, Zsolt BUNA, Ionuţ BADIU,
and Paul PUPEZĂ, 2014, “Methodology to create 3d
models for augmented reality appli- cations using
scanned point clouds”, Mediter. Archaeol. Archaeom,
14 (4): pp.35-44.
COSTANTINO, Domenica, and Maria Giuseppa ANGELINI,
2010, “Realization of a Cartographic GIS for the Filing
and Management of the Archaelogical Excavations in
the Nelson’s Island”, Euro-Mediterranean Conference,
Berlin, Heidelberg: Springer. pp.513-527.
GARGALAKOS, Michael, Elpida GIALLOURI, Aggelos
LAZOUDIS, Sofoklis SOTIRIOU, and Franz X BOGNER,
2011, “Assessing the impact of technology enhanced
eld trips in science centers and museums”, Adv. Sci.
Lett. 4: pp.11-12.
GEE, James Paul, 2003, “What video games have to
teach us about learning and literacy”, Computers
in Entertainment (CIE) - Theoretical and Practical
Computer Applications in Entertainment, October.
GEORGOPOULOS, Andreas, 2017, “Data acquisition for
the geometric documentation of cultural heritage”,
Mixed Reality and Gamication for Cultural Heritage
(Springer) pp.29-73.
GERVAL, Jean-Pierre, and Yann LE RU, 2015, “Fusion of
multimedia and mobile technology in audioguides
for museums and exhibitions”, Fusion of Smart,
Multimedia and Computer Gaming Technologies,
(Springer) 173–205.
HAYDAR, Mahmoud, David ROUSSEL, Madjid MAÏdi,
Samir OTMANE, and Malik MALLEM, 2011, “Virtual
and augmented reality for cultural com- puting
and heritage: A case study of virtual exploration of
underwater archaeological sites (preprint)”, Virtual
Real 15 (4): pp.311-327.
ICOMOS, 2002, International Cultural Tourism Charter.
Principles And Guidelines For Managing Tourism At
Places Of Cultural And Heritage Signicance, ICOMOS
International Cultural Tourism Committee.
LENZERINI, Federico, 2011, “Intangible cultural Training:
The living culture of peoples”, European Journal of
International Law 22 (1): pp.101-120.
MANOVICH, Lev, 2006, “The poetics of augmented space”,
Visual communication 5 (2): pp.219-240.
MORGAN, Colleen, 2016, “Video games and archaeology”,
The SAA archaeological record, November: pp.9-10.
SAGGIO, Giovanni, and Davide BORRA, 2011, “Augmented
reality for restoration/reconstruction of artefacts with
artistic or historical value”, Some Emerging Application
Area.
SYLAIOU, S., F. Liarokapis, K. KOTSAKIS, and P. PATIAS,
2009, “Virtual museums, a survey on methods and
tools”, Journal of Cultural Heritage 10 (4): pp.520–528.
SZELISKI, Richard, 2010, Computer vision: algorithms and
applications. Springer Science & Business Media.
TABONE, Wilbert, and Dylan SEYCHELL, 2015,
“Recognising Familiar Facial Features in Paintings
Belonging to Separate Domains”, International
Conference on Computer Analysis of Images and
Patterns, Springer, pp.124-136.
WANG, Fei-Yue, 2009, “Is culture computable?” IEEE Intell.
Syst. 24 (2): pp.2-3.