Content uploaded by Noha Saleeb
Author content
All content in this area was uploaded by Noha Saleeb on Apr 14, 2015
Content may be subject to copyright.
Saleeb, N. & Dafoulas, G. (2013), ‘Artificial Intelligence in 3D Virtual Environments as
Technological Support for Pedagogy’, Proceedings of the 2nd Workshop on Future
Intelligent Educational Environments Conference (WOFIEE’13), 16-17 July 2013 Athens,
Greece.
Artificial Intelligence in 3D Virtual
Environments as Technological Support for
Pedagogy
Noha SALEEB and Georgios DAFOULAS
Middlesex University, London, UK
N.Saleeb, G.Dafoulas{@mdx.ac.uk}
Abstract. This paper researches the opportunities available to use 3D Virtual learning
Environments (3D VLEs), such as Second Life, to create e-learning Project Innovation
for students using 3D Virtual Design concepts and programming. This entails the use of
programming and coding to create bots (artificial intelligence robotic avatars) that can
be used to direct interactive teaching and learning activities inside a 3D VLE.
Moreover, through the creation and coding of holographic platforms (holodecks) inside
3D VLEs, diverse classroom and environmental settings can be created to aid in the e-
learning process and help the students themselves to use this technique to create
immersive 3D projects e.g. 3D catalogues and exhibitions. This is in addition to the
prospects of using these holodecks for educational role-play activities, modelling
activities and interactive discussions and seminars.
Keywords. Artificial intelligence, E-Learning, Blended Learning, 3D Virtual Worlds,
Teaching Learning Innovation, Second Life, Bots, Psychology
Introduction
3D Virtual Learning Environments (3DVLEs) have been a host for many virtual campuses
of universities, e.g. Harvard and Cambridge, since their offset more than a decade ago [1].
These virtual media offer innovative opportunities for technologically supported pedagogy
and e-learning for many fields of sciences and arts which has reaped noticeable
participation, satisfaction and hence achievement from students [2]. Through 3D VLE
online courses, online avatars allow students and their instructors to interact synchronously
by audio, text chat and other media presentation techniques [3]. It thus becomes imperative
to investigate the merits of migrating with delivery of e-learning to these environments.
The emergence of digitally influenced generations of students, whom Prensky [4] and
Oblinger and Oblinger [5] referred to as “Digital Natives, “Games Generation” and
“Millenials”, deems it logical to anticipate why in order to enhance future learning, students
are currently being encouraged to utilise game-like 3D virtual worlds, or VLEs like Second
Life, Active Worlds and others to accommodate for new cognitive style changes. These
play an essential role in shaping future e-learning as suggested by Wang et al. [6],
specifically with the potential to bridge the gap between simple knowledge of a topic and
hands-on experience with it i.e. “learning by doing” [7].
Furthermore, a paradigm shift in education also emerged called “Animated
Pedagogical Agents [8]. This uses lifelike autonomous 3D characters or avatars that
cohabitate the learning environment to provide a rich interactive face-to-face interface and
activities with students who are also embodied in the learning environment as avatars [9].
These recently can be coded / programmed to provide an intelligent tutoring system as will
be explained subsequently as part of the pedagogical practices presented in this research.
The previously described technological advances and practices support the
developmental perspective of teaching and transmission of knowledge by adopting the
constructivist paradigm/approach to teaching and learning. As indicated by Mikropoulos
and Natsis [10] Constructivism seems to be the theoretical model the majority of the 3D
VLEs are based on. This can be explained by Dalgarno’s and Lee’s [11] conception that
“technologies themselves do not directly cause learning to occur but can afford certain
tasks that themselves may result in learning”. Thus examples of supporting the
constructivist paradigm will be seen in the following sections that fulfil the seven principles
of constructivism as presented by Jonassen [12]:
1. Provide multiple depictions of reality
2. Focus on knowledge construction not reproduction
3. Produce genuine tasks
4. Provide case-based learning environments
5. Promote reflective activities
6. Enable context and content dependent knowledge construction
7. Support collaborative negotiation
The teaching/learning examples adopted and suggested for creation in this research
encourage, through project work, constructing new subjective knowledge in students that is
influenced by their prior experiences. Hence the constructivist approach to
teaching/learning rather than objectively and passively acquiring knowledge as is the case
with behaviourism [13]. Students learn as they work to understand their experiences and
create meaning from it. Therefore, teachers are facilitators who create a curriculum to
support a self-directed, collaborative search for meanings [14]. In this case the curriculum
would encompass programming and coding bots (artificial intelligence automated avatars)
inside 3D VLEs to offer interactive activities for students. As a result since students have
diverse perspectives, backgrounds, learning styles and experiences, this collaborative
learning environment would provide an abundance of benefits [15]. This has the possibility
of increasing even more with the technological capabilities suggested above. Additionally
this aligns with the developmental perspective for teaching and learning which relies on
encouraging self-exploration and inquiry, by “cultivating ways of thinking” beyond the
tutor’s supervision [16]. Hence, with the developmental perspective students are guided
towards deriving problem solutions but not provided with them.
There is a challenge to integrate contributions from a number of different disciplines
into a single learning support offering that will (i) take under consideration the pedagogic
needs associated with the use of 3D VLEs, (ii) address usability and web 2.0 issues from
the use of a social learning network and (iii) investigate 3D VLE interactions with the
mediums used to access learning platforms. So far the creation of intelligent 3D VLEs is
primarily concerned with the design of content for virtual learning tasks.
The next section will demonstrate several educational scenarios through the creation
and use of bots (artificial intelligence automated avatars) inside Second Life, an example of
3D Virtual Learning Environments, to create different interactive projects and activities to
enhance students’ e-learning.
1. Pedagogical Scenarios!!
Second Life is primarily crafted to be built and populated with content generated by its
users–hence is a rich environment for content creation [17]. Along with creating solid or
hollow inanimate objects, it is possible to place programmable scripts on these created
objects to give them specific repetitive animations, or provide the objects with Artificial
Intelligence (AI) awareness of the surrounding actions and events and hence react
according to different situations and stimuli [18]. Inside Second Life these scripts are
created using the Linden Scripting Language (LSL) and attached to inanimate objects or
representations of avatars to animate them. These programmed avatars are called bots. This
is done by placing the scripts on an object attached to the avatar, and it this object that runs
the script and controls the avatar to appear to be walking, talking etc. [18].
Bots can perform many simple interaction tasks such as to recognize approach of other
avatars, ask questions, provide pre-prepared answers to questions, follow, lead or locate
other avatars, turn on or off other objects, play pre-recorded animations as responses to
different stimuli, collect data or information, simulate roles e.g. patient, waiter etc. The
advantage of using bots is that they appear as realistic as “real” avatars, which belong to
real life users, thus as indicated by Varvello and Voelker [19] can be used to conduct
endless activities, social interactions and experiments at any time of day with calculated
precision and efficiency. For example, Kemp and Livingstone [20] suggested setting up
“tour bot” agents inside museums to greet guests and take them on a pre-determined route
with descriptions of the exhibits. The stopping points and text for the descriptions sit inside
the “bots” as notecards and the Logic is implemented using LSL. Bots are being
increasingly used in virtual environments [21] e.g. Second Life, for their convenience as
simulation platforms for testing multi-agent systems and other AI concepts that are more
cost effective to use than physical ones [22].
For the purpose of this research 2 different kinds of bots were experimented with:
“Pandorabots” and “Pikkubots”. These were used in association with a simulation
“holodeck” as explained henceforth:
• Pandorabots are AI “minds” or “chatbots” which can be created or customised
using a free open-source-based website enabling development and publishing of
these chatbots anywhere on the web, including 3D VLEs like Second life.
Pandorabots support the new AIML 2.0 as their knowledge content markup
language. They are used due to their ease of programming and adaptability to
work in any virtual reality program, for a Pandorabot mind can be easily trained to
provide certain sets of answers when asked certain combinations of questions or
keywords.
• PikkuBots are bots or avatar entities created for Second Life which can be
operated automatically. PikkuBot is actually a program that is usually installed on
a dedicated server to automatically run inanimate avatars in "Second Life" even
when the user is not at the computer. The PikkuBot can be configured to do many
tasks. After installing and configuring, the bot is controlled using “commands”.
These are short words sent to it either using the instant messaging chat inworld
(inside Second Life), typed in the command line at the bottom of the bots’ GUI, or
using commands sent to it directly using an inworld scripting engine placed in a
concealed or visible object which triggers the command when the bot steps on it.
The bots’ server feeds sensory information for the characters/avatars over network
connections containing the current state of the virtual world. The bots interact in
the environment by sending action commands back to the server and the character
moves, talks etc. [23].
• Holodecks are virtual reality platforms which can take the form of any object
inside a 3D VLE but contain scripts to “rez” or materialize/create an immersive
new environment around the avatar. This can be used to provide multiple alternate
environments or realities, through choice from a menu, which students can engage
with.
A technique to combine all 3 separate technological AI components mentioned above
was used for the pedagogical scenarios presented in this research. This main merging
concept for Pandorabots, Pikkubots and Holodecks was used to create multiple projects as
explained henceforth. The technique used was i) Create a Pandorabot mind and train it to
recognise a series of questions using combinations of keywords, then provide groups of
specific answers for the bot to reply with. ii) Create/purchase an inanimate Pikkubot in
Second life, customise its appearance and place the Pandorabot AI mind on it (attach it to
it) to give the Pikkubot the life-like interactive conversational abilities to communicate with
other real users’ avatars. iii) Program a scripting engine in Second Life, a commercial
example of which is “ImagiLearning Platform”, which when stepped on will animate the
Pikkubot’s physical actions e.g. move, point etc. iv) Create/customise/script a Holodeck,
build and compress all the environments/spaces/buildings which are to be rezzed from it,
then place these environments inside the holodeck and script its menu to materialize them
on demand. v) Adjust the scripting engine controlling the Pikkubot so that one of its
commands would make the Pikkubot trigger the holodeck and rez a specific environment
based on the questions and answers dialogue with the real avatar users. vi) Devise the
different project scenarios to be used with the students, utilising the above created
comprehensive AI system comprising of Pandorabot, Pikkubot, scripting engine, built
spaces and Holodeck. Examples of this are demonstrated in the next section.
2. Project Examples!!
Figure 1. Pikkubots interacting with student avatars
An example related to digital creativity and design modelling was a project called “Dream
Environment”. The purpose of this was to allow the students in a 3D environment to
change the building style they are in to study different elements of architecture related to a
certain era e.g. in an Egyptian, Chinese, Indian, Roman, Classic style temple or building.
The building prototypes would be created then loaded inside a Holodeck (a commercial
example of which is “Horizon Holodeck”). The student or tutor can choose whatever
environment he wishes for to open up around him from a menu that appears for him inside
Second Life. A Pikkubot would then appear, as shown in Figure 1, dressed appropriate to
the era chosen and provide information about the architecture and design, asking questions
interactively from the student. Other applications of this system can be e.g. to rez a
courthouse to conduct forensic studies investigation and role-play.
“Obedient Patient” is another example of a project where bots can be trained as virtual
patients, as shown in Figure2, to give certain responses on being examined in different
ways by avatars who belong to medical students training in SL on dealing with patients.
This can be held inside an emergency room in a hospital rezzed from a holodeck.
Furthermore 3D voice recognition can be used to provide different personalities for the
replying bot.
Figure 2. Pikkubot posing as virtual patient
Figure 3. Holodeck and different virtual environments rezzed from inside it using a choice menu
Another example was a project called “Virtual Tourist” using the same technology to
teach students about different touristic places on Earth, dangerous places or historical
extinct places (could be used by any tourist unable to visit these places due to disability or
time) by modeling (simulating) these places e.g. Pyramids, Eiffel tower, Everest Mountain,
North Pole, Pacific Ocean, Solar System, placing them in the Holodeck (Figure 3 image 1)
then rezzing them at will, with a Pikkubot to explain, provide a virtual tour and question the
students. Not only can one build an environment to rez, as can be seen in environments 2
and 3 in Figure 3, but also a real-life panoramic view can be placed in the Holodeck, which
would rez around the avatar and create a feeling of immersion inside it (Figure 3
environments 4-8).
“3D Catalogue” is another interesting project, which appealed to students. It involves
creating a complete application for use by a real estate company (houses to buy or rent /
hotels to choose from), where the user talks to a Pikkubot (representing an agent). The user
specifies the house size he wants, number of rooms, price range etc., and automatically
samples of model house appear before him to choose from (from a holodeck) as can be
seen in Figure 4. This has potential of being an online service for a real-life business.
Figure 4. Holodeck rezzing complete buildings for demonstration
“3D Exhibition” is a project similar to the above but displaying a gallery of renowned
images for e.g. Monet, Renoir in the Louvre with spoken info by a Pikkubot on each
(Figure 5). Another project “3D Interactive Environment” uses the Holodeck to create
interactive environments e.g. how to set up alarm system, piping system, precautions in
house. Figure 6 illustrates a rezzed kitchen using Holodeck. The frying pan is the Pikkubot
asking questions like what to do if a pan was on fire. If you answer correctly it makes a
flaring sound, if not the kitchen goes on fire, then you can reset the system again.
Figure 5. Holodeck rezzing an art gallery
Figure 6. Holodeck rezzing an interactive kitchen environment
“Virtual Sensor Simulation” is a final project example, which uses reflexive
architecture techniques in SL, which use sensors to identify approach of avatars. This can
be used to simulate robotic movement, car crash etc. to help with real-life design of these
devices.
The 7 projects described above were piloted with samples of students at random inside
Second Life. However their actual impact on under-graduate and post-graduate course
remain to be investigated. One point of interest is that According to Maher and Giro [24],
agents or bots can function in three modes based on their internal processes: reflexive,
reactive, and reflective. Reflexive mode is where the bot responds to sensory data from the
environment with a pre-programmed response or reflex without any reasoning. In this mode
the bot behaves automatically with no apparent intelligence. Reactive mode is where the
agent displays the ability to reason according to the input data such that the bot appears to
behave with a limited form of intelligence giving different responses for different
situations. Reflective mode is where the bot exhibits capacity to “reflect” on input and
propose alternate actions or decisions, i.e. not simply to react but to hypothesize [24]. The
projects suggested in this research demonstrate both reflexive and reactive behaviour from
the bots, but not reflective. The reflexive aspect can be seen through the Pikkubots’
automatically induced reactions in response to a student’s action. The reactive behaviour
can be seen in the interactive solutions or answers offered by the Pandorabot mind attached
to the Pikkubot in reaction to a user’s choices or questions. This is reactive because the bot
chooses answers or actions from a database, based on its previous training by its
programmer. However the presented scenarios here still need to investigate the possibility
of creating reflective decision-making AI within the bots.
During their engagement in each project with the bots and holodeck, students were
asked to fill in questionnaires in the form of note cards in Second Life to comment and
reflect on their experience and interaction. These note cards were then shared with the
researcher through a note card giver. An automatic chat log of the participants’ interactions
with the bots served as the observation of the interaction that participants had with the bots.
This technique was employed as previously utilised by Beaumont et al. [21]. The automatic
log of the interaction that every participant had with the bots was analysed, and merits and
difficulties recorded to better inform the researcher of the effectiveness of the system and
ways to enhance it for future testing with under-graduate and post-graduate courses.
The main merit recognised by the students, as also identified by Muir et al. [25], was
that these projects allowed the student avatars to participate in an interactive, engaging
"lived experience" that would not be possible in the physical world. They could embody
their character, cooperate with others and submerge themselves in an experience that could
not be replicated as fully in real-life. Additionally, the free form nature of the Second Life
environment meant that each session/lesson could be different, allowing for different
situations to be played out depending upon the contributions of the participants. The main
drawback however was that students recommended that the system needs some training or
orientation before usage as it is not easy or straight forward to use especially for users who
are not technologically savvy.
Finally, as claimed earlier, there is sufficient evidence to confirm that the 3D AI virtual
projects created in this research satisfy the 7 conditions of constructivism previously
mentioned, thus demonstrate usage of constructivism as follows:
1. Providing multiple depictions of reality – through the diverse rezzed
environments
2. Focussing on knowledge construction not reproduction – by formation of
knowledge through the interaction between bots and users
3. Production of genuine tasks – through innovative ideas created using the
bots/holodeck system
4. Providing case-based learning environments – since each project presents a
unique case study, situation or environment
5. Promoting reflective activities – through providing critical analysis and
reflection on the experience in the form of questionnaires filled by the users
6. Enabling context and content dependent knowledge construction – by
programming content specific scripts in the system.
7. Supporting collaborative negotiation – through the engagement of all the
students in class in the experience together and contributing to the discussion
with the bots
Conclusion
The Artificial Intelligence bot and holodeck system developed in this research shows how
3D virtual worlds can provide environments that can respond automatically and
interactively with their users. The diversity of projects created using this system opens
endless frontiers for creating student-centred and engaging educational activities to enhance
a student’s learning experience. As mentioned previously future research involves
enhancing the system, simplifying it and testing it with under-graduate and post-graduate
students in Higher Education courses. This is in addition to investigating the possibility of
adding reflective behaviour in bot actions to enable decision making to reap most value
from technologically supported pedagogy.
References
[1] J. Kay, Educational Uses of Second Life (2009). Retrieved September 3, 2012, from
http://sleducation.wikispaces.com/educationaluses
[2] C. M. Calongne, Educational Frontiers: Learning in a Virtual World, EDUCAUSE Review, 43(5),
(2008), pp. 36-38.
[3] D. Butler, and J. White, A slice of Second Life: Academics, support staff and students navigating a
changing landscape, Hello! Where are you in the landscape of educational technology? Proceedings
ascilite Melbourne. (2008).
[4] M. Prenksy, Digital natives, digital immigrants, On the Horizon, 9(5), (2001), pp.1–6.
[5] D. Oblinger and J. Oblinger, Is it age or IT: first steps towards understanding the net generation, In D.
Oblinger & J. Oblinger (Eds), Educating the Net generation Boulder, CO: EDUCAUSE, (2005), pp.
2.1–2.20.
[6] X., Wang, P. E., Love, R., Klinc, M. J., Kim, and P. R. Davis, Integration of E–learning 2.0 with Web
2.0. Journal of Information Technology in Construction (ITcon), 17, (2012), pp. 387-396.
[7] K. Ku and P. S. Mahabaleshwarkar, Building Interactive Modeling for Construction Education in
Virtual Worlds, Journal of Information Technology in Construction,16, (2011), pp. 189-208.
[8] W. L., Johnson, J. W., Rickel, and J. C. Lester, Animated pedagogical agents: Face-to-face interaction
in interactive learning environments. International Journal of Artificial intelligence in education, 11(1),
(2000), pp. 47-78
[9] M. A. S., Nunes, L. L., Dihl, L. M., Fraga, C. R., Woszezenki, L., Oliveira, D. J. Francisco, and M. D.
G. Notargiacomo, Animated pedagogical agent in the intelligent virtual teaching environment. Digital
Education Review, 4, (2010), pp. 53-61
[10] T. A., Mikropoulos, and A. Natsis, Educational virtual environments: A ten-year review of empirical
research (1999–2009). Computers & Education, 56(3), (2011), pp. 769-780.
[11] B. Dalgarno, and M. J. W. Lee, What are the learning affordances of 3-D virtual environments?. British
Journal of Educational Technology, 41, (2010), pp. 10–32.
[12] D. H. Jonassen, Thinking technology: toward a constructivist design model. Educational Technology,
34(4), (1994), pp. 34–37
[13] M. A. Boudourides, Constructivism, Education, Science, and Technology. Canadian Journal of
Learning and Technology, 29(3). (2003).
[14] G. Siemens, Connectivism: A learning theory for the digital age. International Journal of Instructional
Technology and Distance Learning, 2 (1), (2005).
[15] J. J. Ham and M. A. Schnabel, Web 2.0 virtual design studio: social networking as facilitator of design
education, Architectural Science Review, 54(2), (2011), pp. 108-116.
[16] D. D. Pratt, and Associates, Five Perspectives on Teaching in Adult and Higher Education. Malabar,
FL: Krieger, Publishers (1998).
[17] M., Liu, D., Kalk, L. Kinney, and G. Orr, Web 2.0 and its use in higher education from 2007-2009: A
review of literature. International Journal on E-Learning, 11(2), (2012), pp. 153-179.
[18] D., Friedman, A., Steed, and M., Slater, Spatial social behavior in second life. In Intelligent virtual
agents, pp. 252-263. Springer Berlin Heidelberg, (2007).
[19] M., Varvello, and G. M. Voelker, Second life: a social network of humans and bots. In Proceedings of
the 20th international workshop on Network and operating systems support for digital audio and video,
pp. 9-14, ACM, (2010).
[20] J., Kemp, and D. Livingstone, Putting a Second Life “metaverse” skin on learning management
systems. In Proceedings of the Second Life education workshop at the Second Life community
convention, pp. 13-18. CA, San Francisco: The University Of Paisley, (2006).
[21] C., Beaumont, M., Savin-Baden, E., Conradi, and T. Poulton, Evaluating a Second Life Problem-Based
Learning (PBL) demonstrator project: what can we learn? Interactive Learning Environments, pp.1-17,
(2012).
[22] S., Ranathunga, S., Cranefield, and M. Purvis, Interfacing a cognitive agent platform with Second
Life, pp. 1-21. Springer Berlin Heidelberg, (2012).
[23] R., Adobbati, A. N., Marshall, A., Scholer, S., Tejada, G. A., Kaminka, S., Schaffer, and C., Sollitto,
Gamebots: A 3D virtual world test-bed for multi-agent research. In Proceedings of the Second
International Workshop on Infrastructure for Agents, MAS, and Scalable MAS, vol. 5, Montreal,
Canada, (2001).
[24] M. L., Maher, and J. S. Gero, Agent models of 3D virtual worlds. Proceedings of ACADIA 2002,
(2002).
[25] T., Muir, J. M., Allen, C. S., Rayner, and B. Cleland, Preparing pre-service teachers for classroom
practice in a virtual world: A pilot study using Second Life. Journal of Interactive Media in Education,
(2013).
