Shih, J. L. et al. (Eds.) (2019). Proceedings of the 27th International Conference on Computers in Education.
Taiwan: Asia-Pacific Society for Computers in Education
Study of Augmented Reality Interaction
Mediums (AIMs) towards Collaboratively
Solving Open-Ended Problems
Pratiti SARKARa*₸, Prabodh SAKHARDANDEa₸, Utsav OZAb & Jayesh PILLAIa
aIDC School of Design, Indian Institute of Technology Bombay, India
bAhmedabad University, India
₸These authors contributed equally
Abstract: Open-ended problem solving involves multiple approaches in solving a problem.
This can help students to think divergently to relate and apply their classroom learnings in
real-life examples. At the same time, through active collaboration, students get to exchange and
enhance their knowledge, thus increasing productivity beyond that of an individual. The aim of
our study was to develop a collaborative open-ended learning environment using Augmented
Reality Interaction Mediums (AIMs) as scaffolds. We conducted a study in a classroom with 12
students of 7th grade who collaboratively used different AIMs to solve certain open-ended
problems based on their Mathematics syllabus. We observed their interactions and performance
with AIMs as compared to the controlled treatment for each task. Further, we evaluated the
creativity through divergent thinking scores using the parameters of fluency, flexibility and
originality, where the experimental groups using different AIMs had better creativity score
(M=86.3) as compared to the control group (M=79). Thus, a collaborative open-ended approach
using AIMs as scaffold can be explored further in improving creative problem solving.
Keywords: Divergent Thinking, Creativity, Augmented Reality, Collaborative Classroom
The 21st century K-12 education involves a deep understanding of complex concepts, to further
creatively generate new knowledge and enhance the Science, Technology, Engineering, Arts, and
Mathematics (STEAM) skills. A critical part of STEAM education involves experiential learning,
where the learners learn from their experiences and reflect on those with minimal help of the adults. One
such approach involves using the learning theory of Constructivism, where students tend to construct
their own knowledge (Mughal, & Zafar, 2011). The students can be made to solve a real-life problem by
constructing their knowledge on top of prior experiences. This learning in classrooms can be enhanced
while exploring the multiple solution approaches in the Open-Ended learning environments with few
resources and tools as the scaffold (Biswas, Segedy, & Bunchongchit, 2016). Collaboration among
students in this process can further help in exchanging knowledge and developing social skills, critical
thinking and creative problem solving ability (Laal, & Ghodsi, 2012).
With the advent of technology, the learnings are now being imparted with one of the emerging
technology called Augmented Reality (AR), which helps in superimposing virtual objects in the real
world in real time (Azuma, 1997). These virtual graphics can be in the form of images, 3D models,
textual information, audio, video, animation, etc. Thus, when it comes to classroom education, AR can
be useful as a scaffold in providing affordances that are not readily available in classroom
environments. We have attempted to provide such an experience in the school curriculum where
students collaboratively explore different ways of using AR in creatively solving open-ended problems.
In this paper, we have discussed an experiential learning study which involved collaboratively
solving open-ended problems using different AR Interaction Mediums (AIMs) on a tablet. The broad
goal of the study was to understand the interaction of the students with the AIMs and the effect on their
creativity while solving open-ended tasks.
2. Background Work
Experiential learning emphasizes learning to be a process of gaining and constructing knowledge
through reflection on prior experiences. As per Kolb’s Theory of Experiential Learning (Kolb, 1984),
the knowledge is constructed in a cyclical manner involving the transformation of experience in each
stage. With the benefit of active participation of students, classroom-based experiential learning is thus
being highly adopted and implemented (Huang, 2019). Kolb (1984), Piaget (1966) and Dewey (1938)
have explored experiential learning through constructivism. Among the pioneers of constructivism,
Jean Piaget in his Constructivism Theory states that people generate knowledge and form meanings
based upon their experiences (Ackermann, 2001). The theory also states that by the age of 10-14 years,
middle school students reach the stage of formal operation with the ability to think logically and
conceptualize the things not seen in the actual surroundings (Ojose, 2008). Thus at this age, the students
can be guided towards building up their creative imaginary skills.
Vygotsky hypothesized creativity as any human act that produces something new (Vygotsky,
2004), and calling imagination as the basis of all creative activities. Describing it as a complex process
of dissociation and association of various elements in new ways towards creation of a new entity,
imagination builds upon material supplied by reality (Vygotsky, 2004). There are tests (Guilford
(1967), Wallach & Kogan (1965), Torrance (1962)) which evaluate creativity as a measure of divergent
thinking. Also, in a study with 7th grade students, it was found that open-ended problems in
Mathematics led to an increase in creativity through divergent thinking (Kwon, Park, & Park, 2006).
Similarly, our study involves the students to participate collaboratively to help in developing problem
solving ability, creativity, critical thinking and social skills ( Laal, & Ghodsi, 2012).
This creativity is brought forth in our study using Augmented Reality (AR). Experiential
learning theory in AR suggests that gaining personal experience from AR activities, can enhance the
learning achievement of the students (Hung, Chen, & Huang, 2017). Thus, in the classroom, AR as a
scaffold can provide an interactive, engaging experience by helping students visualize the concepts
which are otherwise difficult to imagine. Our previous work explored the use of AR medium in middle
school classroom to collaboratively solve closed problem, showing enhancement in spatial
visualization skills (Sarkar, Pillai, & Gupta, 2018). However, along with visualization skills, this study
focuses on enhancing the imagination and creativity skills by collaboratively solving certain
open-ended problems using different AR Interaction Mediums in the classroom.
3. Design and Implementation
In the study, open-ended tasks were designed (Table 1) as per the 7th grade Mathematics syllabus.
Designing the Tasks: The four open-ended tasks given to students to solve
Field on a 9x9 grid
To think of ways in
which its area could be
To understand what a square unit is.
To provoke discussion on the ways in
which the area for irregular shapes could be
Walls with 120° angle
To think of methods to
find the internal angle
between the two walls.
To understand the basis of formation of
Lines & Angles and their measurement.
To leverage concepts and laws of geometry
like parallel lines and adjacent angles.
To think of ways to fill
this structure (leaving
no space) with objects of
any shape and size.
To evaluate the ways in which different
shapes of different sizes fit together and
their fitting gets affected by the scale.
To find ways of
climbing the mountain
in the fastest way
To be able to relate the model to an actual
mountain and develop a thorough
understanding of the shape as this was a
major factor in path and method planning.
In our study, the experimental groups used one of the three different AIMs (Table 2) in a task to
solve the given open-ended problems. Each group was also once the control group where the task had to
be done seeing a 2D isometric image of the 3D objects, shown to other groups as 3D models in AR.
Defining the Augmented Reality Interaction Mediums (AIMs)
It emphasized creative use of
imagination as a method of problem
solving. The students could move
around this model, viewing it through
different perspectives and level of detail
by zooming in and out. A three finger
tap cleared all elements on the screen.
One could draw in the complete 3D
space by moving the phone around
and drawing anywhere on the screen
with one finger. A three-finger tap
completely cleared the screen and a
two-finger tap was used to place the
task object on a detected plane.
Two tangible marker cubes were provided
which overlaid different 3D models, based
on the task being performed:
Task 1: a scale and a protractor
Task 2: a protractor and a ladder
Task 3: a round table and a cupboard
Task 4: a flagpole (post) and a rope.
The applications were designed in Unity and exported as Android packages on Samsung
Galaxy S4 Tablets. Google AR Core SDK was used in the draw and imagine AIMs to enable use of AR.
Plane detection and raycasting for placing objects on the plane were the primary AR Core facilities
used. In the cube AIM, Vuforia was used to enable tracking. Multi image tracking (cuboidal) was used
for the cubes and single image target (QR Code) was used for placing the 3D objects on the given cubes.
Our study addresses the following Research Questions:
1. How do students collaboratively interact with AIMs to solve open-ended problems?
2. What is the effect of AIMs on students’ creativity as compared to a traditional medium?
The participants were 7th grade students of a sub-urban Indian school. Through convenience sampling,
study was conducted with 12 students (5 boys and 7 girls) of age group 12-14. They were randomly
divided into 4 groups of 3 students each. The study was conducted a few days after their end-semester
examinations, to ensure they all are familiar with the concepts covered in the AR tasks. Each group was
assisted by a researcher to guide them about the tasks and observe their actions. The task and its
corresponding AIM for a group was selected using the balanced Latin square design (Figure 1).
Figure 1. The balanced Latin square design for task and AIM distribution to a group
The students were encouraged to collaboratively think of multiple answers for a given problem.
The interactions were captured using video recording. Observation Logs were used to note group
behaviour, involvement and interaction with each other and the AR interface. Further, the students
wrote their answers on a sheet, in forms of writing sentences, sketches, diagrams etc. At the end of each
task, the students were interviewed about their approach to solve the problem.
The answers of each task written by the groups, were digitized. To answer RQ1, their answers
were evaluated, video data was observed, the recorded interview were transcribed and the observation
log was assessed to determine the behaviour and approach of the groups in solving the open-ended tasks
in each of the different AIMs provided. To answer RQ2, the answers that students gave were evaluated
for creativity score by taking inspiration from Guilford’s test of divergent thinking (Guilford, 1967).
5. Results and Discussions
5.1 Collaborative Interaction with AIMs to Solve Open-Ended Problems
5.1.1 Approaches in Solving Tasks
AR has previously helped learners to collaboratively play and visualize mathematics with everyday
objects (Khan, Trujano, & Maes, 2018). While solving the Area problem, students suggested materials
and objects from their surroundings as the measurement tools. The square grid placed around the field
helped them calculate the area in terms of units. All AIMs using groups used this grid in some form of
measurement. The Imagine and Draw AIMs groups, split the given field shape into its individual
rectangular components to calculate their respective areas as per the learned formulae. However, all
students neglected the semicircular shape of the field, for not being able to understand the way of
calculating it using the learned formulae.
In the Lines & Angles task, the Imagine and Draw AIMs using groups, initially suggested
multiple angle degrees between the walls as per their perspective of looking at the walls using tablets.
On viewing bit longer, they realized that the angle was constant and the angles looked different because
of different perspective views. The Draw group placed the augmented model of the wall against the
actual wall of the classroom. They then hypothesized that the augmented wall model had an obtuse
angle. Even in this task, students used objects from their immediate surroundings to compare and
measure the angles between the walls.
All the groups related the floor plan to that of their homes in the Symmetry & Congruence task.
They thought of ways to place at least 20 household items while considering their shape and space
available. The Cube AIM group partitioned the entire floor plan based on size and classified the objects
to be placed based on a size proportional to these partitions. This group also suggested organic materials
like salt and powdered spices to fill in minute empty spaces. The control group proposed filling up the
remaining space by increasing the number of smaller sized objects. The group using Imagine AIM
proposed scaling up objects to fill space.
The mountain climbing problem in Visualizing 3D Solids task required students to apply
problem solving skills learnt in their syllabus towards the real life scenario of a macro problem. As seen
in previous studies (Schneider, Weinmann, Roth, Knop, & Vorderer, 2016), influence of entertainment
media was observed where the Control group took an aerial approach to suggest jumping off a
helicopter as the fastest way as seen in television. The Imagine AIM group categorized their answers
based on risk and speed involved to climb .
5.1.2 Use of Augmented Reality Interaction Mediums (AIMs)
Using the Draw AIM, the group with the Area task communicated their ideas instead of drawing the
solutions. In Visualizing 3D Solids, the group precisely drew a ladder and a rope, realizing the need of
an anchor with it. In the Lines & Angles task a triangle was drawn on top of the augmented wall. It was
then replicated on a paper to calculate the angle. Thus, students identified their own effective and
unique ways of using the draw AIM.
With the Imagine AIM, the group of Lines & Angles task on collaboratively discussing and
using hand gestures to while viewing the wall from different sides, deduced that it was indeed the same
angle which was obtuse. In the Area task, the group calculated area by using objects which were around
them, e.g. the field was approximately 10 lunch boxes or 15 pencil boxes large. Thus, they were able to
associate and dissociate the meaning of square units.
The cubes and the respective overlaid objects were used as a stimulus for students to answer.
The use of cubes was less effective for Visualizing 3D Solids task as they used the rope but not the
flagpole in the solutions. In Lines & Angles and Area tasks, students attempted to calculate the exact
values using the objects of cubes, considering them to be realistic measuring tools. For the cubes to be
within camera view to get tracked one student held the tablet and the others held the cubes. One group,
however, placed both cubes on the floor and changed their perspectives by moving the tablet around.
The control group in Visualizing 3D Solids task sketched to communicate complex ideas. They
numbered the multiple drawn paths on the hill to discuss the ways of using these paths. In the Area task,
they drew a top view of the field from the given isometric image and used a protractor to measure the
drawn semi-circular area. In the Symmetry & Congruence task, the group noted the approximate
number of items to cover the floor, e.g. 40 cellphones, 14 newspapers etc. However, this group
restricted themselves to their household scenarios and did not think further. Overall, the control group
had limited responses due to the inability to visualize the hypothetical scenario on seeing a 2D image.
5.2 Effect of AIMs on Students’ Creativity as Compared to a Traditional Medium
The students’ answers were evaluated for creativity inspired from Guilford’s test of divergent thinking
(Guilford, 1967). The answers were evaluated based on fluency, originality and flexibility. The fluency
score was the total number of answers given by a group. The flexibility score was the number of
categories or different ways of thinking answer. The originality score was calculated as a percentage of
uniqueness of the answers. If the answers were rarer than 10%, 20 % or more than 20% of all the
answers for a particular task, it was given a score of 2, 1 or 0 respectively. The inter rater agreement of
the two raters on the scores of flexibility and originality was 93.15%. The scores were categorized into
groups, tasks and AIMs. A creativity score was calculated by adding the scores for fluency, flexibility
and originality for a particular category. As shown in Figure 2, Group 2 had the highest creativity score
of 155, the task of Symmetry & Congruence had the highest creativity score of 150 and the creativity
score of 99 was the highest for AIM Imagine.
Figure 2. Creativity evaluated across groups, tasks and AIMs
The mean creativity score of the experimental groups (AIMs) was higher (M=86.3) than the
control group (M=79) across all tasks. In terms of creativity, comparatively the Cube AIM lacked. This
AIM by its design, provided students with a prompt of two tools to stimulate their thinking ability to
find solutions. However, it was observed that their thoughts were limited to the two tools. Thus, even
though students liked the Cube AIM’s interactive environment, such AIM might be more beneficial for
closed problems or convergent thinking tasks. For example, an AR Mathematical education game was
developed using three tangible marker cubes to teach certain defined operations (Lee, and Lee, 2008).
In our study, the AIMs provided to experimental groups, helped in visualizing the problem and
generating a higher number of creative ideas, as similarly seen in the study by Huang (2019) to enhance
students’ creativity using AR. The control group had a high flexibility and originality score but a lower
fluency score. Group interactions and dynamics were essential in shaping the ideas of students. The
discussions were overall positive and helped in the formation of finished solutions, as was seen in our
earlier study as well (Sarkar, Pillai, & Gupta, 2018). AIMs provided a stimulus to discussion. All the
groups had a positive response to the AIMs, one of the groups, while being the control group, did not
want to do the task on paper but wanted to use one of the AIMs. Like observed by Huang (2019), it was
seen in our study that prior knowledge and experiences played a major role in the generation of ideas by
students. Most of the solutions were directly inspired from either school or household scenarios along
with media like TV shows and online videos. Thus, we could claim that AIMs have the potential to
provide experiences that are otherwise not possible. The design of AIMs ensures that students use AR
not only as a visual tool but also as an immersive experience to think beyond the screen.
6. Conclusion and Limitations
We explored students’ approach towards solving open-ended problems in a collaborative environment
through Augmented Reality (AR) mediums as scaffolds. We developed three AR Interaction Mediums
(AIMs) on tablets providing immersive experiences. These were applied through four different
open-ended tasks based on 7th grade Mathematics syllabus. The results of RQ1 report qualitative
differences on the ways in which participants approached various tasks as well as their positive
experience using AIMs. The results of RQ2 evaluated creativity based on the multiple solutions that
students gave for the problems. We found that the groups which solved the problems using the AIMs
had higher creativity score (M=86.3) than the control group (M=79) using traditional pen and paper.
While these results are promising towards a direction in the way AIMs can be used in
classrooms, there are certain limitations to our study. The study was conducted with a small sample size
(N=12) over a day. Thus, a larger sample size with the study conducted over a longer time would give
more in-depth insights. Another limitation pertaining to plane detection and occlusion using Cube
AIMs requires the improvement of the technical aspect of the application.
The project was funded by Tata Centre for Technology and Design, IIT Bombay. We thank the school
authorities, teachers, students and their parents for their consent and valuable time to conduct the study.
Thanks to the volunteers at IIT Bombay – Amarnath Murugan, Jonathon Mathew and Amal Dev.
Ackerman, E. (2017). Piaget’s constructivism, Papert’s constructionism: what’s the difference?, 2001. URL
http://learning. media. mit. edu/content/publications/EA. Piaget_Papert. pdf.–(URL geprüft: 05/2009).
Azuma, R. T. (2018). A survey of augmented reality. 1997. Disponível in: http://www.cs.unc.edu/~
Biswas, G., Segedy, J. R., & Bunchongchit, K. (2016). From design to implementation to practice a learning by
teaching system: Betty’s Brain. International Journal of Artificial Intelligence in Education, 26(1), 350-364.
Dewey, J. (1938). Experience and Education. New York: Collier Books
Guilford, J. P. (1967). Creativity: Yesterday, today and tomorrow. The Journal of Creative Behavior, 1(1), 3-14.
Huang, T. C. (2019). Seeing creativity in an augmented experiential learning environment. Universal Access in
the Information Society, 1-13.
Hung, Y. H., Chen, C. H., & Huang, S. W. (2017). Applying augmented reality to enhance learning: a study of
different teaching materials. Journal of Computer Assisted Learning, 33(3), 252-266.
Khan, M., Trujano, F., & Maes, P. (2018, June). Mathland: Constructionist Mathematical Learning in the Real
World Using Immersive Mixed Reality. In International Conference on Immersive Learning (pp. 133-147).
Kolb, D. A. (1984). Experiential Learning: Experiences as a source of learning and development, Englewood
Cliffs, NJ: Prentice-Hall
Kwon, O. N., Park, J. H., & Park, J. S. (2006). Cultivating divergent thinking in mathematics through an
open-ended approach. Asia Pacific Education Review, 7(1), 51-61.
Laal, M. & Ghodsi, S. M. (2012). Benefits of collaborative learning. In Procedia-social and behavioral sciences,
Lee, H. S., & Lee, J. W. (2008, June). Mathematical education game based on augmented reality. In International
Conference on Technologies for E-Learning and Digital Entertainment (pp. 442-450). Springer, Berlin,
Mughal, F. & Zafar, A. (2011). Experiential Learning from a Constructivist Perspective: Reconceptualizing the
Kolbian Cycle. International Journal of Learning and Development. 1. 10.5296/ijld.v1i2.1179.
Ojose, B. (2008). Applying Piaget's theory of cognitive development to mathematics instruction. The
Mathematics Educator, 18(1).
Piaget, J. (2005). The psychology of intelligence. Routledge.
Sarkar, P., Pillai, J. S., & Gupta, A. (2018, December). ScholAR: a collaborative learning experience for rural
schools using Augmented Reality application. In 2018 IEEE Tenth International Conference on Technology
for Education (T4E)(pp. 8-15). IEEE.
Schneider, F. M., Weinmann, C., Roth, F. S., Knop, K., & Vorderer, P. (2016). Learning from entertaining online
video clips? Enjoyment and appreciation and their differential relationships with knowledge and behavioral
intentions. Computers in Human Behavior, 54, 475-482.
Torrance, E. P. (1962). Guiding creative talent.
Vygotsky, L. S. (2004). Imagination and creativity in childhood. Journal of Russian & East European Psychology,
Wallach, M. A., & Kogan, N. (1965). Modes of thinking in young children.