Content uploaded by Stella Sylaiou
Author content
All content in this area was uploaded by Stella Sylaiou on Feb 03, 2016
Content may be subject to copyright.
Cartographies of Mind, Soul and Knowledge | Special issue for Professor Emeritus Myron Myridis
School of Rural and Surveying Engineers, AUTH, 2015
Evaluation of a Cultural Heritage Augmented Reality Game
S. Sylaiou
1
*
, K. Mania
2
, F. Liarokapis
3
, M. White
4
, K. Walczak
5
,
R. Wojciechowski
5
, W. Wiza
5
, P. Patias
1
1 Aristotle University of Thessaloniki, Laboratory of Photogrammetry
and Remote Sensing, Greece
2 Technical University of Crete, Greece
3 Masaryk University, Brno, Czech Republic
4 University of Sussex, U.K.
5 Poznań University of Economics, Poland
Abstract:
The ability to develop sophisticated and flexible web and mobile applications
based on augmented reality (AR) and gaming engine technologies is incredible taking into
account the plethora of powerful mobile computing devices for displaying such applica-
tions. In this respect, the research community is challenged to investigate the factors that
make such technologies effective, productive and engaging. In this paper, we investigate
the efficacy of augmented reality in a simple cultural heritage game, by evaluating real-time
feedback from users. The game has been implemented within the ARCO (Augmented Rep-
resentation of Cultural Objects) system using its flexible AR scenario authoring tools and
dynamic content composition and delivery technology. The game has been presented within
an Augmented Reality Interface—a specialized application enabling AR on a user’s com-
puter. The ARCO system, a result of the ARCO project, allows museums to create their
own virtual museum exhibitions based around a collection of cultural objects. A typical
virtual museum would be composed of exhibition spaces that present these digital objects to
a user, for example, on a web page, on a mobile device or in a virtual environment. A vir-
tual museum can present these digital objects in the form of an interactive AR game or
quiz. Through the use of interviews and structured questionnaires, user feedback has been
collected for evaluation to enable us to improve the game characteristics, but more impor-
tantly to understand whether AR based games are educationally useful, meaningful and
appealing to users. In this respect, our study revealed that AR based games are appealing
and do attract user engagement and interest, however, because of the integration of real-
world and 3D elements, AR games also have problems and deficiencies.
*
sylaiou@photo.topo.auth.gr
154 S. Sylaiou, K. Mania, F. Liarokapis, M. White, K. Walczak, R. Wojciechowski,
W. Wiza, P. Patias
1. Introduction
Anything that we have to learn to do, we learn by the actual doing of it...
Aristotle icomachean Ethics, Book II, p .91
Museums can be characterized as places for learning. However, as a consequence
of the limited capabilities of displaying physical artefacts, museums often uninten-
tionally communicate views about what they consider worth learning through the
way that their artworks, objects and historical material are presented. To escape
this criticism museums must contribute to a variety of learning aptitudes, and ac-
count for visitor prior experience (e.g., with technology) when they present their
collections to the public. It is perhaps naïve to think any museum can take or even
afford this approach for every object, but they can implement limited digital re-
source strategies that alleviate such perceptions. For example, at one end of the
spectrum providing a simple on-line browse and search interface for digitised ar-
chives are not beyond a museum’s basic web presence, while the addition of social
media to allow users to comment on collections is also possible. At the other end of
the spectrum, the application of sophisticated 3D and augmented reality to repre-
sent a museum’s digital objects provides a great level of user interaction possibili-
ties, particularly when combined in a game based scenario—affordability for the
museum is another question. Nevertheless, museums have more recently made
greater efforts to discover new technological ways to present their artefacts and
enhance learning experiences and visitors’ engagement
1
. Further, because a visi-
tor’s activities related to exploration of an object can be experiential involving
learning by doing, a museum’s digital objects and collections can also be personal-
ized and configured to respond easily to visitors’ feedback interactively.
Quite early on, MacDonald and Alsford stated “… museums cannot remain aloof
from technological trends if they wish to attract 21st century audiences” (Mac-
Donald and Alsford, 1997). Museum visitors are becoming more accustomed to
technology in this setting and expect exhibits and information to be technologically
aware (Falk and Dierking, 2000). Lately, emerging technologies, such as 3D web
standards and augmented reality (AR), are employed to transform, enhance and
‘augment’ a museum learning experience by employing in-house installations (e.g.,
multimedia based kiosks) or web-based material involving some form of computer
graphics, thereby effectively creating a ‘virtual museum’ premise. AR enables an
enriched experience by superimposing virtual objects on a view of the real world or
on a video of the real world captured in real-time. Such emerging technologies can
1 “Engagement is defined as the quality of user experience that facilitates more enriching interac-
tions with computer applications and is defined by a core set of attributes: aesthetic appeal, nov-
elty, involvement, focused attention, perceived usability, and endurability” (O’Brien and McLean,
2009).
Evaluation of a Cultural Heritage Augmented Reality Game 155
become a powerful communication channel offering a virtual ‘direct’ and personal-
ized experience. These technologies transform users who were passive viewers of
static exhibits into more active users of a museum experience through the use of
interactive interfaces (Liarokapis et al., 2008). Sylaiou et al. suggest “Museum visi-
tors use and interact with a virtual museum environment via a constructive dia-
logue that provides them with access to thematic information and explanations
about the museum objects’ context selecting an appropriate level of information
and amount of detail they prefer” (Sylaiou et al., 2009). Museums shift their focus
from the high-quality presentation of physical collections to the making of mean-
ingful digital presentations related to the artefacts and their interpretation. There is
the constant need for virtual museums to reach out and attract larger and more di-
verse audiences and find ways to understand visitor expectations and experiences
in order to address the needs of diverse user groups and be responsive to various
communities’ interests. Despite the number of AR applications in museums, the
impact such ‘virtual’ exhibits have on the social ecology of exhibitions is largely
unexplored (Reeves, 2004; Schmalstieg, 2005).
More and more, AR technology has started to be recognized as an indispensable
tool that can improve, on a daily basis, human activities in gaming, communica-
tion, medicine, education, design and many other domains. AR, in this sense, has
been acknowledged as a potential strategy for education. If AR technologies are
combined with the so-called ‘serious games’, they can enrich educational experi-
ences either in situ in a museum, via a mobile phone or by remote access to a mu-
seum’s website. In their paper Belotti et al. state “designing games that support
knowledge and skill acquisition has become a promising frontier for training, since
games are able to capture concentration for long periods and can present users with
compelling challenges” (Belloti et al., 2009). It is widely recognized that games
can be devised not only for fun, but also as a means to provide effective learning
experiences in museum settings (Anderson et al., 2010). The main challenge con-
cerning serious gaming technology is when, how and why the combination of AR
and games promote learning. Realizing the goals of AR and gaming and harnessing
them to successful digital heritage applications could be accomplished by human-
centred design, employing robust evaluation metrics and human-centred experi-
mentation. In this paper, we investigate the efficacy of using augmented reality
technology in a cultural heritage game, by evaluating real-time feedback from its
users.
The AR game evaluated in this study has been created with the help of the ARCO
system (White et al., 2004; Walczak et al., 2006). The ARCO system, built within
the 5
th
EU Framework Program project entitled ARCO (Augmented Representation
of Cultural Objects) is a comprehensive solution for creating and managing virtual
museums. The ARCO system users store and manage cultural objects and associ-
ated multimedia objects together with corresponding metadata and present the col-
156 S. Sylaiou, K. Mania, F. Liarokapis, M. White, K. Walczak, R. Wojciechowski,
W. Wiza, P. Patias
lected objects in variety of interfaces, including web pages, virtual reality environ-
ments, mobile devices, and AR interfaces. These operations are supported by the
ARCO system through specialized built-in managers available in the ARCO Con-
tent Management Application (ACMA) interface, which enables the creation of
interactive game and educational scenarios (Wojciechowski et al., 2004). A user —
teacher or museum creator — may build game scenarios, parameterize them, and
manipulate visualized objects. A prepared game is then delivered dynamically on-
demand to an end-user (a pupil or a museum visitor) through the ARCO Exhibition
Server and visualized by the Augmented Reality Interface (ARIF) — an AR appli-
cation installed on a recipient’s computer.
2. Background
2.1 Overview of the ARCO evaluation research
Evaluation of the ARCO system was largely focused around evaluating the user
interfaces (ACMA and ARIF) through a formative and summative approach using
participatory design principles (Banathy, 1992; Carr, 1997; Kensing, 2003; Muller,
2007). In particular, the ARCO evaluation focused on the ACMA interface, which
is a content management system for use by museum professionals to manage digi-
tal objects in the database together with their associated media objects (3D, images,
video, etc.) and relevant metadata allowing online publishing of virtual museums,
and the ARIF interface, which is a tool used to present 3D objects in a virtual mu-
seum through an online web browser that can switch to an augmented reality view
of the same 3D objects. Utilising both empirical and expert based approaches, Sy-
laiou et al., conducted evaluations of ACMA and ARIF in collaboration with Vic-
toria and Albert Museum curators and domain level experts (Karoulis et al., 2006,
Sylaiou et al., 2008, Sylaiou S., et al., 2010). These evaluations adopted question-
naire surveys based on (a) the QUIS (Questionnaire for User Interaction Satisfac-
tion) to assess user satisfaction (Chin et al., 1988), and (b) a Cognitive Walk-
through session with domain and usability experts (Polson et al., 1992). Of particu-
lar interest were the findings related to the evaluation of the ARIF interface. The
ARIF evaluation showed that both museum curators and visitors responded posi-
tively to this type of augmented reality interface, and indicated presence was asso-
ciated with satisfaction and gratification (Sylaiou et al., 2009). The experimental
AR game scenario included cultural information and required users to interactively
answer a series of questions as well as resolve tasks employing an AR interface.
2.2 AR game-based learning
Many studies have explored the potential and contribution of computer games to
the learning process (Kafai and Ching, 1996; Ricci et al., 1996; Rieber, 1996; Pren-
Evaluation of a Cultural Heritage Augmented Reality Game 157
sky, 2001; Gee, 2003; Squire and Barab, 2004). AR lends itself very well to the
concept of creating interesting and engaging games for different areas including
education, learning, training and entertainment; in this respect it is an ideal tech-
nology for integration into interactive museum exhibitions or kiosks. Billingshurst
et al. demonstrate an excellent example of the use of AR with a high degree of user
interaction with their MagicBook (Billinghurst et al., 2001). The Magic book con-
cept is used like any book, where the user simply turns the pages. However, in this
case a page usually contains some form of fiducial marker, which is recognised
through the AR application’s image software, to effect the placement of a 3D ob-
ject on the marker. When viewed through a hand held device, e.g. a smartphone or
the desktop PC screen it gives the appearance of the object popping out of the
book, hence the term Magic Book. The BBC identified in recent research that
young children (around 5 years old) respond positively to AR based learning sce-
narios, simulating their imagination, and enabling them to learn through play in a
natural way (Thomas, 2006). Slightly older children, aged 10+, also demonstrated
an ability to learn and understand, for example, how the Earth, Sun and Moon in-
teract through the use of AR as a teaching tool (Kerawalla et al., 2006). Many other
studies also illustrate the validity of using AR for educational purposes: medicine
(Nischelwitzer et al., 2007), teaching and learning (Liarokapis, 2007), and learning
and performance (Holzinger et al., 2008). Further, AR implicitly involves the user
being ‘immersed’ in the educational activity; therefore immersion should be de-
signed into the educational AR game from the perspective of engagement and mo-
tivation, including interactivity, narrative, ‘flow’ and fidelity (Csikszentmihalyi,
1990; de Freitas and Liarokapis, 2011).
Several AR evaluation studies show that this technology implemented in a gaming
or quiz scenario is an effective educational tool (Andersen et al., 2004; Nilsen et
al., 2004; Liarokapis, 2006). Recent studies concerning an AR workspace in terms
of interaction and pedagogical concepts, show that AR increases students’ motiva-
tion to learn, could be an effective educational instrument, increases vividness of
complex or abstract subjects and users confirmed a high perception of usefulness
and enjoyment (Krauß and Bogen, 2010). The use of usability questionnaires com-
bined with semi-structured, qualitative interviews was illustrated during the usabil-
ity testing of the Virtual Showcases system (Virtual Showcases, 2004). Moreover,
Seagram and Amory (2005) used qualitative and quantitative methods investigating
game-based learning about serious diseases. Another system based on a competi-
tive gaming-learning environment built to stimulate students’ motivation to learn
has also been evaluated (Chang et al., 2003). Other AR game-based learning envi-
ronments have been evaluated to assess their efficacy from a user perspective (Nil-
sen and Looser, 2005, Liarokapis, 2007). The study presented in this paper ex-
plores the system usability of a simple cultural AR game developed with the
ARCO system to examine the efficacy of AR in a museum learning context, dis-
158 S. Sylaiou, K. Mania, F. Liarokapis, M. White, K. Walczak, R. Wojciechowski,
W. Wiza, P. Patias
cover its appealing and pleasant attributes as well as any problematic elements in
an AR game, and see whether the educational usefulness of an AR learning sce-
nario influenced the individuals’ subjective impressions concerning the AR mu-
seum game presentation. The evaluation is conducted both qualitatively through
observational feedback as well as quantitatively through structured questionnaires.
3. The ARCO system
Before we discuss the game methods, materials and evaluation of the simple AR
game, we outline the ARCO system, and its augmented reality capabilities
.
3.1 General information
The ARCO system provides a set of tools that museums can use to create, manage,
and present digital artefacts within interactive virtual museums online (White et al.,
2003; White et al., 2004). The overall architecture of ARCO and the dataflow
within the system are presented in Figure 1.
Figure 1: ARCO system architecture and dataflow
The latest version of the ARCO content pipeline utilises Flex-VR (Walczak, 2009,
2012) to build 3D/VR/AR applications. With Flex-VR, complex interactive 3D
application content can be relatively easy created by museum staff by configuring
predesigned geometrical, logical and behavioural components. Three main phases
can be distinguished in the ARCO content pipeline (Figure 1): content production,
Evaluation of a Cultural Heritage Augmented Reality Game 159
content management and content presentation. Content production includes proc-
esses required for the creation of a museum’s digital representations of its artefacts.
Digital representations may take the form of 3D models, images, sounds and videos
as well as other multimedia objects. Museum artefacts with simple geometry can
be modelled in 3D with classical 3D authoring software such as 3ds Max, Maya,
etc., which can also be augmented with a set of additional plug-ins that simplify the
process of creating 3D models More complex objects can be modelled using one of
several scanning techniques, such as laser or photogrammetry scanning. Such 3D
modelling techniques were demonstrated in the ARCO project (White et al., 2003;
White et al., 2004, ARCO, 2005).
Content management starts with a museum’s digital representations being acquired
by ARCO’s database using the Cultural Object Manager, which is one of the
ARCO Content Management Applications (ACMA) manager tools. Once digitised
and captured in the ARCO database a digital representation is composed of a set of
media objects — each media object represents a different view of the artefact (cul-
tural object), e.g. 3D view, image view, movie view, etc. — and associated meta-
data (Mourkoussis et al., 2003; Patel et al., 2005). After a collection of digital rep-
resentations is acquired, a museum can then build a virtual museum exhibition,
which is configured with ACMA’s Presentation Manager. ARCO enables creation
of different kinds of virtual museum exhibitions, from 2D multimedia web pages,
through interactive 3D web presentations, to complex AR games. ARCO employs
the notion of presentation templates that separate the design and programming of
the virtual museum’s presentation code (e.g. 3D and web code) from the actual
process of creating the virtual museum exhibition (the creativity process). Museum
staff can then perform the creativity process without experience in 3D modelling
and computer programming. Presentation templates are created by an IT specialist
and contain all program code necessary to build virtual museum exhibitions.
The structure of an ARCO based virtual museum or exhibition is based around
presentation spaces in the database. Each presentation space can represent a com-
plete virtual museum, an exhibition in the virtual museum or even part of an exhi-
bition. Thus, a virtual museum can be sub-divided into several smaller exhibitions
with sub-spaces representing geometrical (e.g., different museum rooms) or logical
(e.g., different stages of a game) elements of a virtual museum or exhibition. Pres-
entation spaces are akin to folders, which may contain instances of cultural objects
(digital representations of the museum’s artefacts) and instances of presentation
templates — content templates and behaviour templates. Presentation properties
can describe presentation spaces, cultural objects within presentation spaces, or
media objects within cultural objects within particular presentation spaces. In addi-
tion to fixed lists of cultural objects, ARCO enables assignment of cultural object
selection rules to presentation spaces, enabling creation of virtual galleries with
content selected dynamically based on metadata records. A virtual museum or ex-
160 S. Sylaiou, K. Mania, F. Liarokapis, M. White, K. Walczak, R. Wojciechowski,
W. Wiza, P. Patias
hibition designer (which could be the curator) can build a virtual museum exhibi-
tion by creating a structure of presentation spaces, then creating instances of con-
tent templates and behaviour templates in these spaces, then assigning cultural ob-
jects and cultural object selection rules, and finally setting presentation properties.
In ARCO, the concept of presentation domains is used to differentiate content
presentation in different environments, on different platforms or for different
groups of users. For this purpose, multiple instances of presentation templates can
be assigned to presentation spaces. Typically, one domain is used for internal mu-
seum use, e.g. a museum kiosk, and one for Internet based exhibitions. Other do-
mains may be used for specific purposes, e.g. to distinguish touch screen interfaces
and standard keyboard/mouse interfaces or to use other forms of presentation such
as AR games (Wojciechowski et al., 2003, 2004). Virtual museums or exhibitions,
accessible to end-users, e.g. in the form of 3D web presentations or AR games, are
dynamically generated on-demand based on the presentation structures stored in
the ARCO database. The use of different templates, different template parameter
values and different presentation properties permits different forms of presentation
of the same content. Since the presentations are created on-demand, they can be
personalized for different user groups or created in response to specific user que-
ries.
3.2 ARCO AR tools and methods
The ARCO virtual exhibitions can be visualized using standard web browsers, but
the web browser must be enabled with a VRML/X3D plug-in, or a specifically
designed Augmented Reality Interface (ARIF) application (Wojciechowski et al.,
2003) based on the ARToolKit library (Kato et al., 2000; ARToolWorks, 2012)
must be used. In a web browser, a user can browse virtual museums or exhibitions
based on 3D VRML or 2D web pages (with embedded 3D, images, video, etc.).
Further, such exhibitions can be configured for local displays inside a museum, e.g.
a museum kiosk, and remotely over the Internet. ARCO provides an ARIF applica-
tion, which is used for displaying web-based presentations enhanced with visuali-
zation of cultural objects in AR environments. ARIF has two components: a web
browser and an AR browser. The web browser is used for navigating the virtual
exhibitions and selecting cultural objects for presentation in an AR environment
within the AR browser. To indicate locations of virtual objects in the AR environ-
ment, there are fiducial markers placed in the real environment. The real environ-
ment is observed by a camera, which streams the captured video to the AR
browser. The AR browser then overlays cultural objects (their 3D model) in to the
video image corresponding with the position and orientation of the markers placed
in the physical environment. A camera and a screen for displaying the ARCO vir-
tual exhibitions can be integrated into a kiosk, which can be located within the mu-
seum premises. Also, it is important to provide users with enough space for the
Evaluation of a Cultural Heritage Augmented Reality Game 161
manipulation of markers in front of the camera.
AR learning environments are based on ARCO virtual exhibitions, which contain
behaviour templates, encoded in XML, specifying the course of learning scenarios.
Different learning scenarios can be implemented based on the same virtual exhibi-
tion when supplied with different behaviour templates, e.g. a scenario designer can
adapt the presentation to different age groups and various expectations of users.
The key element of the scenarios is learning metadata, which are specified as pa-
rameters of the behaviour templates. Each of the learning scenarios designed for
presenting cultural objects from a given virtual exhibition can be characterized by
different metadata. The metadata can be specified in the ACMA tool with an easy-
to-use editor, which enables users to set up the metadata values without requiring
manual coding in XML. In the AR game scenario presented in this paper, the learn-
ing metadata for each cultural object consists of a list of questions, answers, and
object description. These metadata values are used to generate content presented in
the AR game at runtime.
4. Materials and methods
In order to perform the evaluation of the cultural heritage AR game, first we had to
deploy and test the ARCO system, select a user group, develop some simple AR
content in the context of a simple game, and define the experimental procedure and
evaluation metrics.
4.1 Apparatus and visual content
The ARCO system is set up in a Windows PC environment with suitable lighting
conditions for the ARIF component, which also includes a web camera. The ARIF
architecture and rendering of digital objects is built around the ARToolKit (Bill-
inghurst et al., 2001; Wojciechowski et al., 2004; Liarokapis, 2007) as discussed
above.
4.2 Participants
Twenty-nine subjects from the Departments of Computer Science and Psychology
at the University of Sussex (16 males, 13 females, age range: 19–33) participated
in the evaluation of the AR cultural heritage game. The volunteers employed in this
study were not involved in any of the technical development stages of the ARCO
system.
4.3 Visual content and experimental procedure
The AR game or quiz was based on an historical and archaeological context pre-
sented in an interactive scenario focused on question and answer AR scenario. We
162 S. Sylaiou, K. Mania, F. Liarokapis, M. White, K. Walczak, R. Wojciechowski,
W. Wiza, P. Patias
designed with our ARCO ARIF system a simple AR game structure that supports
learning through visualisation, creativity and experimentation (Prensky, 2005).
Archaeological artefacts from Fishbourne Roman Palace, UK were modelled in
3D, including a partial representation of the Palace itself for use as the 3D content
in the AR game. At the beginning of the game or quiz scenario, a welcome web
page was displayed in the web browser component of the ARIF application. The
web browser’s opening page includes a brief story about Fishbourne Roman Palace
and an introduction to the quiz, which is about the archaeological artefacts. The
goal was to test the users’ ability to discover information about the artefacts and the
Palace in the context of an AR game.
The AR game environment included setting up the markers on the experimental
table top, after which a 3D model of an artefact appeared on a marker with a corre-
sponding question. The possible answers to the question were displayed on the
available markers (Figure 2a). The 3D model was rotated around to enable users to
observe it from different angles. Turning each of the three markers to the other side
revealed whether the answer selected by a user was correct or not (Figure 2b). A
correct answer was indicated by a 3D green smiling face, whereas a 3D red sad
face displayed on the other side of the marker indicated an incorrect answer. In this
way, users participated in the AR game experience and discovered the correct an-
swers by exploration and via a dialogue with the system.
Figure 2a: Visualization of an example
cultural object
Figure 2b: Selection of the correct answer
(White et al., 2004)
During the game, each correct answer was awarded a number of points, whereas a
wrong answer decreased score for a given question. The visual feedback on the
answer selected by a user was also accompanied with appropriate audible feedback
expressing approval or disapproval. When all questions regarding a cultural object
were answered correctly, the ARIF application was switched to the web browser
for presenting more detailed information on the cultural object, as shown in Figure
3. Next, the user could carry on the game experience and explore the interactive
presentation of the remaining cultural objects.
Evaluation of a Cultural Heritage Augmented Reality Game 163
Figure 3: The detailed information on a part of an ancient mosaic floor
At the end of the AR game, the final results were presented in the web browser
component. If a user earned enough points, he/she could see a 3D reconstruction of
one of the Palace wings, as depicted in Figure 4. Clearly, this is only a simple sce-
nario, which could be extensively elaborated particularly with the new plethora of
commercial AR SDKs now available in mobile formats, with specific web and mo-
bile 3D standards such as OpenGL ES and WebGL.
Figure 4: The game results — 3D reconstruction
of a part of the Fishbourne Roman Palace
The user evaluation and associated interviews took place in the Computer Graphics
Centre, at the University of Sussex, UK. It involved only one participant at a time,
while research assistants guided the participants on how to use the system.
During the evaluation process all “users were provided with written instructions
concerning sets of pre-determined tasks while navigating through the ARIF inter-
face. The evaluation used cued testing, which involves guiding users while exposed
164 S. Sylaiou, K. Mania, F. Liarokapis, M. White, K. Walczak, R. Wojciechowski,
W. Wiza, P. Patias
to the system and asking them to perform specific tasks or to answer questions”
(Sylaiou et al., 2010). It was interesting to explore the relationship between the
educational usefulness of the learning scenario and previous user experience with
virtual reality, augmented reality and computer games. Users’ willingness to play
the AR game again offered a measure of engagement.
Initially, the users were informed about the game tasks. During the next stage, us-
ers moved through the contents of the AR museum game. They explored the inter-
face and made decisions in relation to defined tasks. Finally, the system usability
was assessed without keeping track of users’ errors, or the time needed to complete
the tasks, because the evaluation was focused on assessing system performance
rather than users’ performance.
4.4 Evaluation metrics
The experimental scenario included cultural heritage information and required us-
ers to interactively answer a series of questions as well as resolve tasks employing
an experimental AR interface. The AR cultural heritage game relates to learning by
doing and to discovery learning (Hein, 1998). According to the ISO-standard 9241
(ISO, 1998), “usability of a system is its ability to function effectively and effi-
ciently, while providing subjective satisfaction to its users”. Nielsen defines the
usability of an interface as being usually associated with five parameters:
(1) Easy to learn: a user can get work done quickly with the system, (2) efficient to
use: once a user has learnt the system, a high level of productivity is possible, (3)
easy to remember: a casual user is able to return to using the system after some
period without having to learn everything all over again, (4) few errors: users do
not make many errors during the use of the system or if they do so they can easily
recover them, and (5) pleasant to use: users are subjectively satisfied by using the
system; they like it (Nielsen, 1993).
In order to address these parameters, a qualitative as well as a quantitative evalua-
tion was conducted. The qualitative evaluation was based on observation of prob-
lems encountered while users were playing the AR game. The main quantitative
evaluation instrument used for this study was the ACMA-ARIF Tutorial Question-
naire (ARCO, 2005), which included six main questions for subsequent analysis:
(1) The educational usefulness of the learning scenario within a museum/class
room is (poor/excellent), (2) The presentation of questions in the AR environment
is (poor/excellent), (3) Answering questions using double-sided markers is (very
difficult/very easy), (4) The integration of the web and AR presentation is
(poor/excellent), (5) The scoring mechanism is (nonsense/essential), and (6) The
sounds accompanying the learning scenario are (nonsense/essential) (Karoulis,
Sylaiou and White, 2006)
There was an additional question exploring the users’ intention to try AR technolo-
gies again scored on a 7-point Likert scale (not at all/definitely).
Evaluation of a Cultural Heritage Augmented Reality Game 165
5. Results and discussion
5.1 Qualitative analysis
Two important concepts regarding the usability of an interface are ‘transparency’
and ‘intuitiveness’ (Nielsen, 1993; Preece et al., 1994). Transparency refers to the
ability of the interface to fade out in the background, allowing the user to concen-
trate during his work on what needs to be done and not on how to do it. In our case
this means not interfering with the learning procedure, while intuitiveness refers to
its ability to guide the user through it by the use of proper metaphors and success-
ful mapping to the real world. For example, by providing the user with the appro-
priate icons, correct labelling, exact phrasing, constructive feedback, etc. (Karoulis,
Sylaiou and White, 2006). Transparency and intuitiveness were assessed by ini-
tially gathering observational information in relation to the problems encountered
while users resolved the tasks of the AR game, forming the qualitative part of the
evaluation.
The interviewer took careful notes of the participants’ responses and then the inter-
views were transcribed. The results were elaborated using manual content analysis
coding and were categorised according to words and phrases that were repeated
often creating patterns. For the data processing, a cutting and sorting technique was
used. Phrases and expressions that were considered as important were arranged in
categories with common characteristics and the common themes concerning the
problems encountered and the suggestions for the system improvement were dis-
covered. This information together with users’ suggestions for system improve-
ment was grouped and is documented in an ARCO project deliverable D16 As-
sessment and Evaluation Report on the ARCO System. Table 1 presents a sum-
mary of user feedback responses specific to the AR cultural heritage game used as
a scenario for the ARIF part of that evaluation study documented in D16.
The problems encountered and issues raised were grouped according to the func-
tionality of the graphic elements, the quality of the graphics, the ease of navigation,
the functionality of the AR elements, the aesthetic issues in relation to the look and
feel of the application as well as any general technological problems that need to be
addressed. Most notable observations in relation to the AR game functionality pro-
posed the improvement of the navigation interface as well as 3D interface in order
to intuitively manipulate the 3D models embedded in the AR quiz as well as the
provision of on-line help concerning the scoring mechanism while playing. More-
over, the contrast between background and AR graphical elements when superim-
posed is considered crucial for successful AR applications because the integration
of the real-world view as well as the graphics elements in one environment should
be visually clear. Higher resolution could help towards visual clarity too.
166 S. Sylaiou, K. Mania, F. Liarokapis, M. White, K. Walczak, R. Wojciechowski,
W. Wiza, P. Patias
Table 1: Qualitative evaluation of the AR game (Giorgini et al, 2004 Karoulis, Sylaiou,
and White, 2006, Sylaiou et al, 2008)
Problems
encountered
Description Suggestions for system improvement
1. Functionality
of Graphic
Elements
Background,
colour and
size of fonts
The fonts should be more obvious and legible
Change the contrast of the button in relation to back-
ground
Consider higher contrast between interface elements
and background
2. Quality
of the VRML
models
The quality
of the VRML
model should
be improved
More details about the VRML model should be pro-
vided/ Higher resolution of the VRML models’ tex-
tures
3. avigation Sense of
control on
the naviga-
tion
Add a quit option
4. Help Missing
explanatory
information
Provide better explanation about the scoring mecha-
nism. For example, an indication of the number of
correct and wrong answers
Provide hints when answering questions
Provide a brief introduction to the quiz. For example, a
clip or a movie
5. AR game
elements
functionality
On the web browser certain icons could be added such
as navigation buttons, zoom in, rotate, pan buttons, etc.
Minimize text
Blocking the correct answer instead of turning over the
markers
The game rule signifying that when a player turns the
same marker twice, two points are lost should be abol-
ished
6. Aesthetic
issues
Poor quality
of graphic
elements
Use different emotion icons
Use a clapping hand instead of a smiley face to indicate
a correct answer
Depending on the score a bigger reconstruction should
be built
Omit the sounds of applause and the smiling faces
when the user selects a correct answer, as well as the
sounds of disappointment and the sad faces used when
the wrong choice is selected
7. Content
enrichment
Addition of more questions
8. Technological
problems
The smiley
face did not
work
Correction of bugs
Evaluation of a Cultural Heritage Augmented Reality Game 167
While the users played the AR game, navigation was challenging. Two users noted
that they had lost control of the navigation between the elements of the game pro-
posing a ‘quit’ option to be added. More detailed feedback includes: Four users
required explanations about the game and the scoring mechanism and proposed the
addition of a help file that explains the scoring mechanism more clearly. Four par-
ticipants stated that the navigation to the system would be ‘easier with instructions’
whereas, five participants said that ‘you need someone to guide you’. The inter-
viewers also proposed to add a brief introduction to the AR quiz ‘for example a
clip or a movie’ and provide more explanation about the scoring mechanism. The
use of a clapping hand instead of a smiley face to indicate a correct answer was
preferred. The participants suggested that the application developers should omit
the sounds of applause and the smiling faces when the user selects a correct answer
as well as the sounds of disappointment and the sad faces used when the wrong
choice is selected. (Karoulis, Sylaiou, White, 2006). Feedback, though, indicating
whether in an AR game a player has completed a task successfully or not is consid-
ered essential, however, it could be adapted according to age group and type of
user. Issues raised in relation to the functionality problems of the AR game ele-
ments could be resolved by adding explanatory graphical icons on the web
browser. Usability issues such as the ones derived from the qualitative analysis, if
not resolved, would have been a severe obstacle towards completing an educational
task with concentration and engagement. The qualitative evaluation based mostly
on observational data proved to be invaluable proposing a list of guidelines for
future development of any AR game.
5.2 Quantitative analysis
The quantitative elaboration was based on calculating the group mean response to
the six main questions posed to users answered by selecting a rating out of a 7-
point Likert scale as detailed in Section 4.4. A seventh question was designed to
assess the users’ future plans to explore augmented reality again. All questions
were answered by selecting a rating out of a 7-point Likert scale.
Figure 5 illustrates the participant’s responses to the questions based on a 7-point
Likert scale. At the time of this study, AR as a methodology was very new with
available library in need of much more functionality; nevertheless the results show
an overall positive response with a great interest to explore AR further in the con-
text of cultural heritage AR games. This provides a promising motivation for de-
veloping more sophisticated AR gaming installations for museums. The data from
which the graph of Figure 1 is derived is detailed in Table 2.
Descriptive statistics in Table 2 show that the participants consider that the pro-
posed learning scenario is useful reflected by a mean value 4.17 on a 7-point Likert
scale (SD: 0,711). The presentation of the questions on the AR interface could be
improved resulting in a mean quality response value of 3,93 out of 7 (SD: 0,615).
168 S. Sylaiou, K. Mania, F. Liarokapis, M. White, K. Walczak, R. Wojciechowski,
W. Wiza, P. Patias
Figure 5: Quantitative analysis
Table 2: Descriptive statistics (29 Users)
Minimum Maximum Mean
Std.
Deviation
Main User Questions
Statistic Statistic Statistic
Std.
Error
Statistic
1 Educational usefulness learning
scenario
3 5 4,17 ,132 ,711
2 Presentation questions in AR 3 5 3,93 ,121 ,651
3 Answering questions using double
sided markers
2 5 4,07 ,156 ,842
4 Integration of AR web
presentation
3 5 4,10 ,152 ,817
5 Scoring mechanism 1 5 3,52 ,196 1,056
6 Sounds accompanying learning
scenario
2 5 3,68 ,155 ,819
7 Intention to try virtual or aug-
mented reality in the future
6 7 6,72 ,084 ,455
This result was expected, because according to the qualitative data a major area of
usability problems was related to the functionality of the graphic elements of the
AR game. Complaints concerning the clarity of the fonts’ size and colour, which to
some appeared blurred and ‘not obvious’ resulted in lower ratings. A design guide-
line for AR applications involving text and imagery would be that existing fonts
and superimposed elements should be of high contrast in relation to the back-
ground. Further, newer more sophisticated AR SDKs, e.g. Qualcomm, will take
care of this problem.
When assessing the ease of use of the AR double-sided markers and the seamless
integration of the AR game and the web presentation, means of 4,07 out of 7 (SD:
0,842) and 4,1 out of 7 (SD: 0,817) were received respectively. Two of the partici-
Evaluation of a Cultural Heritage Augmented Reality Game 169
pants noted that the quality of the 3D models was not satisfactory and that it may
be useful in the future to include textures of higher resolution, again future AR
development with more sophisticated AR SDKs, and gaming engine graphics and
functionality will alleviate this problem. Textures applied to the 3D models and
images of the cultural objects were of medium resolution so as for the AR game to
be computationally efficient for storage and transmission over the Internet. A de-
sign guideline for AR applications would be to include as high-resolution textures
and images as possible taking into account computational load. Whether the scor-
ing mechanism was essential and understandable resulted in a mean rating of 3,52
out of 7 (SD: 1,056). Similarly, assessing whether the sounds accompanying the
learning scenario were essential resulted in a mean rating of 3,68 out of 7 (SD:
0,819). Both questions communicate to the designer of AR educational games that
it is essential that the scoring of an AR game should be well explained. Moreover,
the sounds may be vital when they accompany an intuitively designed gaming sce-
nario.
A mean value of 6.72 out of 7 (SD: 0,455) indicated a very positive response by
users with an intention to try virtual or augmented reality in the future.
A correlation conducted between the ratings of usefulness of the educational sce-
nario and the ease of use of the AR game questions resulting in a significant posi-
tive correlation (Pearson-r correlation coefficient r = 0,490, p < 0.01) signifying
that the educational goals of any AR game may be accomplished when the AR
game narrative is clear.
A significant correlation was not observed in relation to the relationship between
the educational usefulness of the learning scenario and previous user experience
with virtual reality, augmented reality and computer games. Therefore, such tech-
nologies are open to the public without the need of previous user experience with
similar environments.
The AR cultural heritage game being evaluated was based on a very simple sce-
nario utilising an early prototype augmented reality application, as such techno-
logical problems and system bugs were apparent. These were addressed in the im-
proved version of the ARCO ARIF system now open for licensing.
6. Conclusions
The study revealed that AR games are appealing and attract user engagement and
interest, however, because of the complexity of AR technologies, AR games may
also have problems and deficiencies. The evaluation provided invaluable recom-
mendations concerning factors that were considered an important determinant of
AR engaging experiences. Summarizing the research results, the following main
outcomes are apparent:
170 S. Sylaiou, K. Mania, F. Liarokapis, M. White, K. Walczak, R. Wojciechowski,
W. Wiza, P. Patias
• The users must use intuitively the graphic elements of the user interface. The
background must have high contrast colour combinations, in order to avoid con-
fusion with interface elements, like buttons. Also, the colour and the size of the
fonts must be legible.
• The 3D models must be of high quality, as photorealistic as possible.
• The users must have obvious exit and a sense of control over the Virtual Envi-
ronment of the AR game.
• A help file that explains the functions of the AR game to the users must be pro-
vided.
• The game must be challenging, so as to help testing users’ skills, increasing
their interest and enhancing learning.
The evaluation suggested a series of system refinements to ARCO’s ARIF compo-
nent. Changes applied based on feedback received during the qualitative and quan-
titative evaluation detailed above improved the problematic features of the game.
The evaluation framework proposed combining both qualitative feedback as well
as quantitative elaboration of findings in relation to ease of use and efficiency of
interface elements was considered complementary and useful. Such a framework
would be applicable in the broader context of educational interfaces (whether web,
VR or AR based) providing a fair methodology with the prospect of eliciting reli-
able and valid evaluation results.
The development of AR based education tools or museum interactive systems
based on AR technology is fraught with problems that can lead to mixed feelings
from visitors. When it works, it is outstanding as an educational tool that captivates
and motivates the user. But when an AR object fails to materialise (pop out of the
book, for example) due to poor lighting conditions or bad choice of fidu-
cial/marker, etc., it can lead to disappointment. So as long as the AR system and
environment is carefully designed to avoid these common pitfalls, the interactive
experience will engage the user increasing their motivation to learn. Mobile com-
puting devices and the latest AR SDKs that are geared for mobile devices will see a
greater uptake of AR and gaming technologies built into mobile applications for
museums based learning scenarios. Such scenarios, museum games or virtual mu-
seums deploying AR could also exploit the human perceptual system to better un-
derstand how we perceive AR across different display media, smartphone, tablet,
screen, etc. particularly in terms of immersion. Assessing the efficacy of AR in
such environments would be beneficial.
Acknowledgments
This work has been funded by the Marie Curie Action on Human resources and
Mobility, Marie Curie training site: Virtual Reality and Computer Graphics, project
HPMTCT-2001-00326 and FP5 IST project ARCO—Augmented Representation
of Cultural Objects.
Evaluation of a Cultural Heritage Augmented Reality Game 171
References
Andersen, T.L., Kristensen S., Nielsen B.W., Grønbæk, K. (2004) Designing an augmented
reality board game with children: the battleboard 3D experience. Proc. Interaction de-
sign and children, pp. 137-138. ACM Press.
Anderson, E. F., McLoughlin, L., Liarokapis, F., Peters C., Petridis, P., de Freitas, S., 2010.
Developing serious games for cultural heritage: a state-of-the-art review. Virtual Re-
ality 14(4): 255-275.
ARCO, 2005. Evaluation Report, D16: Assessment and Evaluation report on the ARCO
system and its components,
http://www.arco-web.org/TextVersion/Documents/Deliverables/ARCO-D16-R-1.0-
170904.pdf (Retrieved 20 June 2014)
Aristotle, The Nicomachean Ethics, London: Penguin.
Arnab, S., Protopsaltis, A., Minoi, J.L., Dunwell, I., de Freitas, S., 2010. Promoting cross-
cultural awareness through exposure in Game-based Learning. IEEE Learning Tech-
nology Newsletter.
ARToolWorks, 2012. ARToolKit; http://www.artoolworks.com/
Banathy, B.H., 1992. Comprehensive systems design in education: building a design cul-
ture in education. Educational Technology 22(3): 33-35.
Belottie, F., Berta, R., De Gloria, A., Primavera, L., 2009. Adaptive Experience Engine for
Serious Games, IEEE Transactions on Computational Intelligence and AI in Games,
Vol. 1, (4), pp. 264-280.
Billinghurst, M., Kato, H., Poupyrev, I., 2001. The MagicBook: A transitional AR interface.
Computers & Graphics 25(5): 745-753.
Carr, A.A., 1997. User-design in the creation of human learning systems. Educational
Technology Research and Development 45 (3): 5-22.
Chang, L.J., Yang, J.C., Chan, T.W., Yu, F.Y., 2003. Development and evaluation of multi-
ple competitive activities in a synchronous quiz game system. Innovations in Education
& Teaching International 40(1): pp. 16-26.
Chin, J.P., Diehl, V.A., Norman K.L., 1988. Development of an instrument measuring user
satisfaction of the human-computer interface. Proc. of SIGCHI '88. ACM/SIGCHI,
New York, pp. 213-218.
Csikszentmihalyi, M., 1990. Flow—The psychology of optimal experience. New York:
Harper Collins.
de Freitas, S., Liarokapis, F., 2011. Serious Games: A New Paradigm for Education?, Seri-
ous Games and Edutainment Applications, Springer.
Falk, J.H., Dierking, L.D., 2000. The museum experience, Walnut Creek, CA: Alta Mira
Press.
Gee, J.P., 2003. What video games have to teach us about learning and literacy. New York:
Palgrave Macmillan.
Giorgini, F., Gaspard, F., Sayd, P., White, M., Almosawi, A., Mourkoussis, N., 2004.
ARCO — D16 Assessment and Evaluation Report on the ARCO System, Ed. White
172 S. Sylaiou, K. Mania, F. Liarokapis, M. White, K. Walczak, R. Wojciechowski,
W. Wiza, P. Patias
M, University of Sussex, Brighton, UK.
Hein, G.E., 1998. Learning in the museum. London: Routledge.
Holzinger, A., Kickmeier-Rust, M., Albert, D., 2008. Dynamic media in computer science
education; content complexity and learning performance: Is less more? Educational
Technology and Society 11(1): 279-290.
ISO 9241 – International Standarization Organization, 1998. Ergonomic Requirements for
Office Work with Visual Display Terminals (VDT’s).
Kafai, Y.B. and Ching, C.C., 1996. Meaningful contexts for mathematical learning: The
potential of game making activities. Proc. of the International Conference on Learning
Sciences, pp. 164-171.
Karoulis, A., Sylaiou, S., White, M., 2006. Usability evaluation of a virtual museum inter-
face. Journal Informatica 17(3): 363-380.
Karoulis, A., Sylaiou, S., White, M., 2006. Combinatory Usability Evaluation of an Educa-
tional Virtual Museum Interface. Proc. of the Sixth International Conference on Ad-
vanced Learning Technologies (ICALT ’06), IEEE Computer Society.
Kato, H., Billinghurst, M., Popyrev, I., Imamoto, K., Tachibana, K., 2000. Virtual object
manipulation on a table-top AR environments, Proc. of International Symposium on
Augmented Reality ISAR’00, Munich, Germany.
Kensing F., 2003. Methods and Practices in Participatory Design. ITU Press, Copenhagen,
Denmark.
Kerawalla, L., Luckin, R., Seljeflot, S., Woolard A., 2006. Making it real: exploring the
potential of augmented reality for teaching primary school science. Virtual Reality
10(3): 163-174.
Krauß, M., Bogen, M., 2010. Conveying Cultural Heritage and Legacy with Innovative
AR-based Solutions. Proc. Museums and the Web 2010, Toronto: Archives & Mu-
seum Informatics. http://www.archimuse.com/mw2010/papers/krauss/krauss.html (Re-
trieved 20 June 2014).
Liarokapis, F., 2006. An exploration from virtual to augmented reality gaming. Simulation
and Gaming, Symposium: Virtual Reality Simulation 37(4): 507-533.
Liarokapis, F., 2007. An Augmented Reality Interface for Visualizing and Interacting with
Virtual Content. Virtual Reality 11(1): 23-43.
Liarokapis, F., Sylaiou, S., Mountain D., 2008. Personalizing Virtual and Augmented
Reality for Cultural Heritage Indoor and Outdoor Experiences. Proc. of the 9
th
International Symposium on Virtual Reality, Archaeology and Cultural Heritage
(VAST '08), Eurographics, Braga, Portugal, 2-5 December 2008, pp. 55-62.
MacDonald, G.F., Alsford, S., 1997. Towards the Meta Museum, The Wired Museum:
Emerging technology and changing paradigms. In: Jones-Garmil K. (ed.), Washington
DC: American Association of Museums.
Mourkoussis, N., White, M., Patel, M., Chmielewski, J., Walczak, K., 2003. AMS - Meta-
data for Cultural Exhibitions using Virtual Reality, Dublin Core International Confer-
ence DC 2003, Seattle, WA.
Muller, M.J., 2007. Participatory design: The third space in HCI (revised). In: J. Jacko and
Evaluation of a Cultural Heritage Augmented Reality Game 173
A. Sears (eds.), Handbook of HCI 2
nd
Edition. Mahway N.J. USA: Erlbaum.
Nielsen, J., 1993. Usability Engineering. Academic Press, San Diego.
Nilsen, T., Linton, S., and Looser, J., 2004. Motivations for Augmented Reality Gaming.
Proc. of the Fuse ’04 New Zealand Game Developers Conference. 26-29 June. Dune-
din. New Zealand, pp. 86-93.
Nilsen, T. and Looser, J., 2005. Tankwar Tabletop war gaming in augmented reality. Proc.
2
nd
Workshop on Pervasive Gaming Applications. Munich, Germany.
Nischelwitzer, A., Lenz, F.-J., Searle, G., Holzinger, A., 2007. Some aspects of the devel-
opment of low-cost augmented reality learning environments as examples for future
interfaces in technology enhanced learning. In: Universal Access to Applications and
Services (LNCS 4556), pp. 728-737. New York: Springer.
O’Brien, H.L. and McLean, K., 2009. Measuring the User Engagement Experience. En-
gagement by Design Pre-Conference Workshop, CHI 2009 Digital Life New World,
Boston, MA, April 5, 2009.
Patel, M., White, M., Walczak, K., Sayd, P., 2003. Digitisation to Presentation - Building
Virtual Museum Exhibitions, Proc. International Conference on Vision, Video and
Graphics, Peter Hall, Phil Willis (Eds.), Bath, UK.
Patel, M., et al., 2005. Metadata Requirements for Digital Museum Environments, Int.
Journal on Digital Libraries, Special Issue on Digital Museum, Vol. 5, Issue 3, May
2005; Springer-Verlag.
Polson, P., Lewis, C., Rieman, J., Wharton, C., 1992. Cognitive walkthroughs: A method
for theory-based evaluation of user interfaces. International Journal of Man-Machine
Studies 36(5): 741-773.
Preece, J., Rogers, Y., Sharp, H., Benyon, D., Holland, S., Carey, T., 1994. Human-
Computer Interaction. Addison-Wesley Publ., London
Prensky, M., 2001. Digital game-based learning. New York: McGraw-Hill
Prensky, M., 2005. In educational games, complexity matters. Mini-games are trivial – but
“complex” games are not. An important way for teachers, parents and others to look at
educational computer and videogames, http://www.marcprensky.com/writing/Prensky-
Complexity_Matters.pdf (Retrieved 20 June 2011)
Reeves S., 2004. Research Techniques for Augmented Reality Experiences,
http://www.mrl.nott.ac.uk/~str/doc/methods.pdf (Retrieved 20 June 2011)
Rieber, L.P., 1996. Seriously considering play: Designing interactive learning environ-
ments based on the blending of microworlds, simulations, and games. Educational
Technology, Research, and Development, 44(1): 43-58.
Ricci, K.E., Salas, E., Cannon-Bowers J.A., 1996. Do computer-based games facilitate
knowledge acquisition and retention? Military Psychology 8(4): 295-307.
Schmalstieg D., 2005. Augmented Reality Techniques in Games, 4
th
IEEE and ACM Inter-
national Symposium on Mixed and Augmented Reality ISMAR05.
Seagram, R. and Amory, A., 2006. An assessment of learning through the use of a con-
structivist learning environment. In: E. Pearson & P. Bohman (Eds.), World Confer-
174 S. Sylaiou, K. Mania, F. Liarokapis, M. White, K. Walczak, R. Wojciechowski,
W. Wiza, P. Patias
ence on Educational Multimedia, Hypermedia and Telecommunications 2006: 165-
2172. AACE.
Squire, K. and Barab S., 2004. Replaying history: Engaging urban underserved students in
learning world history through computer simulation games. Proc. of the 6th Interna-
tional Conference on Learning Sciences, pp 505-512. Santa Monica, CA: International
Society of the Learning Sciences.
Sylaiou, S., Economou, M., Karoulis, A., White, M., 2008. The evaluation of ARCO: a
lesson in curatorial competence and intuition with new technology. ACM Computers
in Entertainment, 6(2) (April/June), New York: ACM Press.
Sylaiou, S., Mania K., Karoulis A., White M., 2009. Presence-centred Usability Evaluation
of a Virtual Museum: Exploring the Relationship between Presence, Previous User
Experience and Enjoyment, International Journal of Human-Computer Studies
(IJHCS) 68(5): 243-253, Amsterdam: Elsevier, Academic Press.
Sylaiou, S., Liarokapis F., Kotsakis K., Patias P., 2009. Virtual museums, a survey and
some issues for consideration, Journal of Cultural Heritage, Volume 10, Issue 4, pp.
520-528.
Sylaiou, S., Mania K., Karoulis A., White M., 2010. Exploring the relationship between
presence and enjoyment in a virtual museum, International Journal of Human-
Computer Studies, Volume 68, Issue 5, pp. 243-253.
Walczak, K., 2009. Configurable Virtual Reality Applications, Poznań University of Eco-
nomics Press, Poznań, pp. 240.
Walczak, K., 2012. Building Configurable 3D Web Applications with Flex-VR. In: Interac-
tive 3D Multimedia Content - Models for Creation, Management, Search and Presenta-
tion, ed. Cellary, W., and K. Walczak, Springer, London, Dordrecht, Heidelberg, New
York, 2012, pp. 103-135.
Walczak K., Cellary W. and White M., 2006. Virtual Museum Exhibitions, IEEE Computer
39, Issue 3: March 2006. IEEE Computer Society Press, 93-95.
White M., Mourkoussis N., Patel M., Chmielewski J., Walczak K., 2003. AMS – metadata
for cultural exhibitions using virtual reality. Proc. of International Conference on Dub-
lin Core and Metadata Applications, Dublin Core Metadata Initiative, pp. 1–10.
White, M., et al., 2004. ARCO-An Architecture for Digitization, Management and Presen-
tation of Virtual Exhibitions. Proc. of the 22
nd
International Conference on Computer
Graphics (CGI'2004), IEEE Computer Society, Hersonissos, Crete, June 16-19, pp.
622-625.
Wojciechowski, R., Walczak, K, White M., 2003. Augmented Reality Interface for Museum
Artifact Visualization, Proc. IASTED International Conference on Visualization, Im-
aging and Image Processing VIIP 2003, Benalmadena, Spain.
Wojciechowski, R., Walczak, K., White, M., Cellary, W., 2004. Building Virtual and Aug-
mented Reality Museum Exhibitions. Proc. of the Web3D 2004 Symposium—the 9th
International Conference on 3D Web Technology, ACM SIGGRAPH, Monterey, Cali-
fornia (USA), pp. 135–144.
Περιεχόμενα – Contents
G. Gartner
Honory Note for Prof Dr Myron Myridis ................................................ 1
Χαρτογραφίες Νου
Γ. Αλαβάνος
Χαρτογραφώντας τον λόγο των επιστημόνων στο δημόσιο τοπίο .......... 3
Α. Πασχαλίδης
Οι σχέσεις του γεωγραφικού χώρου - χρόνου στον πρώτο τόμο
του Κεφαλαίου του Μαρξ, από το βιβλίο του D. Harvey (2011) .......... 19
Ν. Βαΐου
Άτλας των Γυναικών στον Κόσμο. Χαρτογραφώντας την πατριαρχία ... 27
Ελένη Βαροπούλου
Θεατροποιημένα Τοπία ......................................................................... 38
Alain Guez
Réflexions autour de l’expérience spatialisée du temps ........................ 48
E. Andrikopoulou, G. Kafkalas
Territorial cohesion and spatial planning in the context of European
integration .............................................................................................. 55
Β. Νάκος
Χάρτης και αλληλεπίδραση ................................................................... 67
Β. Παππάς
Eκδοχές ανάγνωσης και οπτικοποίησης της αστικότητας ..................... 79
Β. Φιλιππακοπούλου
Μια Σημειωτική Προσέγγιση στον Χαρτογραφικό Συμβολισμό .......... 92
A. Dermanis
Problems in parameter estimation with nonlinear models ................... 116
Δ. Ρωσσικόπουλος
Ζάρια, μεσημβρινοί, γονίδια και γεωγραφίες. Μια σύντομη ιστορία
της στατιστικής .................................................................................... 134
S. Sylaiou, K. Mania, F. Liarokapis, M. White, K. Walczak,
R. Wojciechowski, W. Wiza, P. Patias
Evaluation of a Cultural Heritage Augmented Reality Game ............. 153
Α. Τσάτσαρης, Γ. Γρηγορακάκης
Τεγεάτις: Ένα Χαρτογραφικό Ταξίδι στη Γη του Μύθου
και της Ιστορίας ................................................................................... 175
Α. Φωτίου
Η πρώτη ακριβής μέτρηση ύψους βουνού από τον Ξεναγόρα
στο αρχαίο Πύθιο Ολύμπου της Περραιβικής Τρίπολης ..................... 190
Κ. Πατραμάνης
Διαίρεση τυχαίας γωνίας με κανόνα και διαβήτη κατά μεγάλη
προσέγγιση [με την χρήση / συμβολή του λόγου της χρυσής τομής
(φ)] ....................................................................................................... 211
Ν. Μυρίδης
Τοπογραφία Διαδικτύου ...................................................................... 224
Ε. Μανούτσογλου, Ν. Ασημάκης, Ε. Καραμπούζη, Ν. Σκουτέλης
Σπήλαια και Αρχιτεκτονική – Σχεδιασμός του Σπηλαιολογικού
Μουσείου Κρήτης στο Χορδάκι Ακρωτηρίου ..................................... 236
Χαρτογραφίες Ψυχής
M. Borgel
Une lettre à Myron, ami de longue date .............................................. 247
Ε. Κούκη
Γραμμές πορείας/περιπλάνησης .......................................................... 251
M. Bruneau
La sensibilité géographique de Photis Kondoglou .............................. 262
Κ. Χατζημιχάλης
Γεωγραφίες και Χάρτες: Περιήγηση με τον Μύρωνα Μυρίδη ........... 281
L. Bensasson
Des recherches menées au Théâtre du Soleil à la fondation d’ARTA:
De l’exploration des formes à la rencontre des traditions ................... 293
Κ. Δασκαλάκης
Μικρά νησιά – Τοπίο – Βιώσιμη Ανάπτυξη ........................................ 305
Κ. Μίχα
Το σπίτι με το σαλχίμι ή η αποκατάσταση διατηρητέας αγροτικής
κατοικίας στην περιοχή Κορθίου Άνδρου –
Μια σύντομη εξιστόρηση .................................................................... 310
Λ. Στάμου
Μαθήματα … ζωής, οικονομίας και … οικολογίας
από τα Τζουμέρκα ............................................................................... 318
Κ. Τοκμακίδης
Ενθύμηση: Μία Φιλοσοφική Προσέγγιση ........................................... 331
Χαρτογραφίες Γνώσης
E. Λιβιεράτος
Χαρτογραφία στην Ελλάδα του 19
ου
αιώνα ......................................... 342
F. Ormeling
The rendering of Greece in Dutch school Atlases ............................... 362
Λ. Λεοντίδου
Η δική μας Αγεωγράφητος Χώρα: Ετεροτοπίες της ελληνικής
πανεπιστημιακής εκπαίδευσης ............................................................. 376
Ν. Μπεόπουλος
«… φάγαμε τα χέρια μας, σπάσαμε τη μέση μας, χάσαμε τα νιάτα μας
για να βγάλουμε τα πουρνάρια και τα βάτα και τώρα μας ζητάνε
να τα ξαναφτιάσουμε ...» ..................................................................... 397
Α. Βαρθολομαίος, Ε. Καρνάβου
«Άπαξ δάσος πάντα δάσος»; Οι πολιτικές της αποδάσωσης
στην Ελλάδα ........................................................................................ 411
Π. Λαφαζάνη, Μ. Παπαδοπούλου,
Σ. Γιαννακοπούλου,
Α. Χριστοδούλου
Η προσέγγιση των Περιφερειακών Ανισοτήτων στην Παραγωγική
και Κοινωνική Διάρθρωση του Ορεινού Πληθυσμού της χώρας
στη 10ετία 1991-2001 .......................................................................... 428
Θ. Μαλούτας
Κοινωνικός διαχωρισμός στην Αθήνα των αρχών του 21
ου
αιώνα ..... 458
S.-2. Spyrellis
Mutations socio-économiques dans la Région Métropolitaine
d’Athènes :
Localisation entrepreneurial et pôles métropolitains ........................... 470
M. Sivignon
Un géographe devant la crise grecque ................................................. 488
Ι. Μπεοπούλου
The open and the domesticated sea ...................................................... 498
Δ. Ρόκος
Η Ολοκληρωμένη Ανάπτυξη στην Ελλάδα στα Χρόνια
της Πολυδιάστατης Κρίσης. Αιτίες, Ευθύνες, Προτάσεις, Μέτρα,
Δράσεις και Προοπτικές ...................................................................... 507
Κ. Πετροπούλου
Για το ζήτημα του αυθόρμητου στην καπιταλιστική ημιπεριφέρεια:
Παραδείγματα από την Ελλάδα και το Μεξικό ................................... 526
Σ. Γκιάλης
Εργασιακή Γεωγραφία: Χώροι και τόποι της εργασίας
και της δράσης ..................................................................................... 539
Α. Πασχαλίδης
Η αντίθεση και ο ανταγωνισμός Πόλης - Υπαίθρου
(από τα έργα των Μαρξ και Ένγκελς) ................................................. 551
Α. Καραδήμου - Γερόλυμπου
Εποικισμός και πολεοδομία στα πρώτα μετεπαναστατικά χρόνια
στη νεώτερη Ελλάδα ........................................................................... 563
G. 2adalini
La colonisation rurale de la Cyrénaïque entre les deux Guerres ......... 574
Ν. Τσελεπή, Η. Πετράκου
Νέες Χαρτογραφήσεις Συνόρων & Μεταναστεύσεων ........................ 590
Α. Αναστασιάδης
Η Ελκτικότητα και Προβολή της πόλης μέσα από Πολεοδομικές
Επεμβάσεις στον Αστικό Χώρο .......................................................... 604
Ε. Άγα
Η διαχρονικότητα της ποιοτικής απόδοσης των πλατειών
της Χώρας Φολεγάνδρου ..................................................................... 619
Ι. Πισσούριος
Η μοντελοποίηση της περιοχής εξυπηρέτησης δημόσιων χρήσεων:
Μια ποσοτική προσέγγιση της πολεοδομικής ανάλυσης
και του προγραμματισμού τους ........................................................... 629
Α. Βοϊβόνδα, Χ. Πετρίτση
Χαρτογραφική Ανάλυση και η χρήση της στη διερεύνηση
εναλλακτικών προτάσεων πολεοδομικής οργάνωσης:
Η περίπτωση του οικισμού Γουρνών στο Ηράκλειο Κρήτης .............. 647
Π. Αρβανιτίδης, Φ. Νασιώκα
Οι χώροι αστικού πρασίνου ως «κοινοί πόροι» ................................... 662
Ε. Γαβρά, Σ. Κασίδου
Διδακτήρια Δημοτικής Εκπαίδευσης στην Εκπαιδευτική Περιφέρεια
Φλωρίνης (1918 - 1923): Η περίπτωση της περιοχής των Πρεσπών ... 681
Ι. Φραγκόπουλος, Χ. Παπασυμεών, Δ. Κουρκουρίδης
Κοινωνικοχωρική διαλεκτική, χώρος και ετεροτοπία στα σύγχρονα
εμπορικά κέντρα: Mια προσέγγιση με αφορμή την εμπειρική έρευνα
στο Mediterranean Cosmos στη Θεσσαλονίκη .................................... 698
Ε. Γαβρά, Χ. Μπαρδάκας
Σχεδιασμός και πολιτικές ανάπτυξης του Δήμου Φλώρινας:
Το δομημένο περιβάλλον μέσα από τα Πρακτικά του Δημοτικού
Συμβουλίου (1912 - 1916) ................................................................... 716
Α. Αρβανίτης
Το Κτηματολόγιο ως Σύγχρονο Σύστημα Διοίκησης της Γης
και Διαχείρισης της Γεωπληροφορίας ................................................. 732
M. Aza, M. Papadopoulou, D. Voutsa, P. Lafazani
Creation of a GIS tool for the selection of a classification method,
the statistical assessment of the method’s class intervals
and its implementation in choroplethic mapping ................................. 747
Ν. Καρανικόλας
Επαναδιατυπώνοντας την σχέση των δεδομένων και των θεματικών
συμβόλων ενός χάρτη. Εφαρμογή σε έναν κλασσικό Ελληνικό
Άτλαντα του 2
ου
παγκοσμίου πολέμου ................................................ 758
Δ. Καϊμάρης, Ο. Γεωργούλα, Π. Πατιάς
Google Earth επανεξέταση: Εφαρμογή στην πεδιάδα της Λάρισας .... 768
Ν. Λαμπρινός
Οι τεχνολογίες των Γεωγραφικών Συστημάτων Πληροφοριών (ΓΣΠ)
ως εργαλεία υποστήριξης της χωρικής σκέψης
στο Δημοτικό Σχολείο ......................................................................... 777
I. Doukas, S. Demoula
Historical GIS (HGIS): An amply mature high-tech tool,
to the decisive and effective help in the historical research ................ 791
Α.Μ. Αγατζά - Μπαλοδήμου, Χ. Μητσακάκη, Κ. Παπαζήση
Επιλογή Μετασχηματισμού μεταξύ Γεωδαιτικών Συστημάτων
Αναφοράς ............................................................................................ 813
Β. Γρηγοριάδης, Η. Τζιαβός, Γ. Βέργος, Δ. Νατσιόπουλος
Μελέτη παγκόσμιων μοντέλων βαθυμετρίας και δεδομένων
ηχοβολίσεων στην περιοχή του δέλτα Αξιού - Λουδία - Αλιάκμονα .. 829
Δ. Δεληκαράογλου, Χ. Χατζηκυριάκου
Αξιοποίηση δεδομένων Δορυφορικής Αλτιμετρίας για τυπικές
χαρτογραφικές εφαρμογές στον τομέα της θαλάσσιας Γεωδαισίας
και Ωκεανογραφίας ............................................................................. 842
D.G. Panagiotopoulos, C. Papazachos, G. Vougioukalakis, S.C. Stiros,
Th. Laopoulos, M. Fytikas, E. Karagianni, D. Vamvakaris, F. Mo-
schas,
V. Saltogianni, K. Albanakis
Santorini Volcano: The intra-caldera unrest of the period
of 2011 - 2012, as revealed by seismicity, temperature, sea-level,
geochemical and GPS data .................................................................. 854
Ch.Z. Pasiakou, M.D. Tsakiri - Strati, S. Siachalou, G. Doxani
Integrated IR-MAD and OBIA for supervised change detection:
The case study of Polifitos Dam in Greece ......................................... 871
Μ. Λαζαρίδου
Το δορυφορικό πρόγραμμα Landsat .................................................... 887
P. Stournara, Μ. Tsakiri - Strati, P. Patias
Hyperspectral analysis of EO-1 Hyperion imagery: A case study
in the geographical area of Greece ....................................................... 894
Χ. Χαλκιάς, Δ. Κλάδης
Χαρτογράφηση του Τεχνητού Νυχτερινού Φωτισμού στον Ελληνικό
Χώρο. Διαχρονική μελέτη με την αξιοποίηση δορυφορικών
δεδομένων DMSP-OLS ....................................................................... 907
Α. Κοκκάλης, Γ. Μίντσης, Χ. Ταξιλτάρης, Σ. Μπάσμπας, Ι. Δημητρό-
πουλος
Σπουδή του διαγράμματος κατανομής λειτουργικών ταχυτήτων
σε σχέση με την οδική ασφάλεια ......................................................... 922
Ι. Γιαννοπούλου, Σ. Γιαννόπουλος
Η τιμολόγηση του αρδευτικού νερού στις χώρες
του ΟΟΣΑ – Σύγχρονες Τάσεις και Απόψεις ...................................... 934
Σ. Σεβαστάς, Ι. Υφαντής
Εφαρμογή των Γεωγραφικών Συστημάτων Πληροφοριών
στη προσομοίωση συνδυασμένης λειτουργίας επιφανειακών
και υπόγειων υδάτων σε λεκάνη απορροής.
Η περίπτωση του Άνω Ανθεμούντα ....................................................... 959
Κ. Τσιόκανος
Ο χωροταξικός σχεδιασμός σε λεκάνες απορροής λιμνών
στον ελλαδικό χώρο ............................................................................. 982
Ευρετήριο συγγραφέων – Authors’ index ................................................. 995
iv
Εξώφυλλο:
φωτογραφία αντιγράφου ΜΕΤ /Νέα Υόρκη (συλλογή Μύρωνα Μυρίδη)
Χάρτης Κλαύδιου Πτολεμαίου Tabula Asia I (Asia Minor), έκδοση Πάντοβας, 1621
Φωτογράφιση:
Ελισάβετ Κούκη
Ψηφιακή επεξεργασία φωτογραφίας:
Χρύσα Πετρίτση
Σύνθεση:
Μύρων Μυρίδης
Ψηφιακή επεξεργασία, σύνθεση και συνολική επιμέλεια εξωφύλλου:
Άννα Παναγοπούλου
ISBN 978-960-89320-7-4
© Copyright, Νοέμβριος 2015
18
ο
χλμ Θεσσαλονίκης - Περαίας
T.Θ. 4171 • Περαία Θεσσαλονίκης • T.K. 570 19
Tηλ.: 2392.072.222 - Fax: 2392.072.229 • e-mail: info@ziti.gr
Π. ZHTH & Σια OE
Φωτοστοιχειοθεσία
Eκτύπωση
Βιβλιοδεσία
ww w.ziti.gr
BIBΛIOΠΩΛEIO ΘΕΣΣΑΛΟΝΙΚΗΣ - KENTPIKH ΔIAΘEΣH:
Aρμενοπούλου 27 - 546 35 Θεσσαλονίκη • Tηλ.: 2310-203.720 • Fax 2310-211.305
e-mail: sales@ziti.gr
BIBΛIOΠΩΛEIO - AΠOΘHKH AΘHNΩN:
Χαριλάου Τρικούπη 22 - Τ.Κ. 106 79, Aθήνα • Tηλ.-Fax: 210-3816.650
e-mail: athina@ziti.gr
ΗΛΕΚΤΡΟΝΙΚΟ ΒΙΒΛΙΟΠΩΛΕΙΟ: www.ziti.gr