Content uploaded by André Henriksen
Author content
All content in this area was uploaded by André Henriksen on Aug 22, 2022
Content may be subject to copyright.
The 18th Scandinavian Conference on Health informatics, Tromsø, Norway, August 22-24, 2022. Organized by UiT The Arctic
University of Norway. Conference Proceedings published by Linköping University Electronic Press at
https://doi.org/10.3384/ecp187. © The Author(s). This work is licensed under the Creative Commons Attribution-
NonCommercial 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by-
nc/4.0/
Sorterius - An Augmented Reality App for Encouraging Outdoor Physical Activity for
People with Intellectual Disabilities
Magnus Stellander1, André Henriksen1, Henriette Michalsen1,2, Audny Anke1,2, Daniel Ursin1, Santiago Martinez3, Susanna
Pelagatti4, Keiichi Sato1,5, Vebjørn Haugland1, Erlend Johannessen1, Juan Carlos Torrado6, and Gunnar Hartvigsen1,3
1UiT The Arctic University of Norway, Tromsø, Norway, andre.henriksen@uit.no
2University Hospital of North Norway, Tromsø, Norway
3University of Agder, Grimstad, Norway
4Università di Pisa, Pisa, Italy
5llinois Institute of Technology, Chicago, IL 60616, USA
6University of Bergen, Bergen, Norway
Abstract
Many with intellectual disabilities (ID) have difficulties adhering to current physical activity guidelines. The goal
of this study was to develop a mobile app for assisting people with ID to be more physically active. We
implemented a solution that combines the digital and real world using augmented reality (AR). Eight people
working with people with ID (special education teachers, social workers, psychologists, and researchers) tested
the app and completed a usability test. Results indicate that a mobile app focusing on everyday life scenarios
can have a potential value for the targeted user group, but AR solutions can be challenging.
Keywords
Motivation, mHealth, mobile health, exergames, steps, smartphone
1 INTRODUCTION
Physical activity (PA) provides significant health benefits
[1]
, and the World Health Organization (WHO)
recommends young adults to perform at least 150 minutes
of moderate PA every week
[2]
. However, people with
intellectual disabilities (ID) are known to have difficulties
achieving these recommendations
[3]
. Compared to the
general population, they have lower PA levels and worse
health
[4; 3]
. One of the barriers for participation in PA for
individuals with ID is lack of interest and low PA related
self-efficacy
[5]
. However, motivation for PA could be
triggered by fun, use of rewards, and technology [6].
The availability of mobile applications (apps) for improving
and motivating PA has greatly improved the last decades
[7]
. However, apps tend to generally be too complex for
people with ID. It follows that people with ID need tailored
apps that can motivate them and increase their PA [8]. The
goal of this study was to develop an app for assisting
people with ID to be more physically active. The app is part
of an ongoing intervention with people with ID and is
currently being tested and evaluated [9; 10].
2 METHOD
We implemented Sorterius, a Pokémon Go-inspired cross-
platform app using the Unity game engine (Unity
Technologies, SF, US, v2019.4).
The game uses augmented reality (AR), where players
observe the real world through the smart phone camera.
Digital content, in the form of 3D garbage objects, appears
on the screen as the player walks around. Sorterius is
based on previous work by Haugland et al. [11; 12]. A
thorough description of the design and implementation of
Sorterius is described by Stellander [13].
The player’s goal is to help the game mascot, Sorterius, to
clean the world. This is achieved by picking up virtual
garbage as it appears on the screen (by tapping them).
Players must correctly choose between several containers
to throw the garbage in, depending on difficult level. The
player can choose between three difficulty levels: easy
(one garbage container), medium (two containers), and
hard (four containers). An internal step counter is used to
determine how often new garbage objects should appear.
Players are rewarded with daily virtual rewards, in the
form of stars and positive feedback. A caretaker menu
allows customization of level difficulty and to define the
daily step goal and weekly star goal. Completing the daily
step goal awards three stars. In addition, to further
motivate usage, physical rewards can be defined (e.g.,
movie tickets), which will be awarded by the caretaker
upon achieving the weekly star goal. Text-to-speech
options can be enabled for players with limited reading
skills.
The 18th Scandinavian Conference on Health informatics, Tromsø, Norway, August 22-24, 2022. 88
Due to restrictive infection control measures related to the
Covid-19 pandemic, we were unable to test the app in the
target group. However, eight people with relevant
background, i.e., special education teachers, social
workers, psychologists, and researchers working with
people with ID, tested the app, completed a System
Usability Scale (SUS) questionnaire, and gave general
open-ended comments to the solution. A SUS is used for
collecting feedback on subjective aspects of usability of a
system [14]. The scale is a 10-question questionnaire,
where answers are given on a 5-point liker scale, ranging
from strongly disagree to strongly agree. We customized
the SUS questionnaire to address the limitation of who
completed the questionnaire. Table 1 gives a list of the 10
customized questions.
#
Question
Q1
I think the application would be regularly used by
people with an intellectual disability
Q2
I think the application is too complicated for
people with an intellectual disability
Q3
I think the application is easy to use for people
with an intellectual disability
Q4
I think a person with an intellectual disability
would need support to use the application
Q5
I think a person with an intellectual disability
would think the different parts of the application
are well connected
Q6
I think a person with an intellectual disability
would think there are too many inconsistencies
in the application
Q7
I think a person with an intellectual disability
would be able to learn to how to use the
application
Q8
I think the application is too difficult to use for a
person with an intellectual disability
Q9
I think a person with an intellectual disability
would be comfortable using the application
alone
Q10
I think it will require extensive training before
this application can be used
Individual SUS scores for each question were calculated by
subtracting one point (i.e., score-1) from all odd questions
(positive polarity), whereas for even questions (negative
polarity), participant responses were subtracted from five
(i.e., 5-score) This gives 0-4 points for each question.
Scores for all participants were added and multiplied by
2.5 to create a scale from 0-100 for each question.
According to Sauro et al. [15], a score of 68 is an average
score when analysed as a percentile rank and can be
considered as a “Satisfactory” system. Alternatively,
Bangor et al. [16] defined an acceptability scale, where a
SUS score above 70 is considered “Acceptable”.
3 RESULTS
Figure 1 shows two game play screenshots, showing the
visualization of the garbage on screen using AR. The app
achieved an overall SUS score of 61. This is somewhat
lower than the average score mentioned above of 68 [15]
and corresponds to a “marginally acceptable” [16] system.
Participant’s individual SUS scores ranged from 42.5 to
80.0, where half of participants gave a SUS score of 65 or
higher. Individual raw (i.e., before inverting negatively
framed questions and before multiplying with 2.5 to
create 0-100 scale) SUS scores ranged from 2.1 to 3.9. An
overview of SUS (range 1-5) mean and standard deviation
for each question is given in Table 2. For questions Q1, Q3,
Q5, Q7, and Q9 (banded in Table 1), higher score is better.
For questions Q2, Q4, Q6, Q8, and Q10 (non-banded in
Table 1), lower score is better.
Question
Q1
Q2
Q3
Q4
Q5
Mean
3.4
2.4
3.3
3.6
3.5
SD
0.48
0.99
0.66
0.99
0.87
Question
Q6
Q7
Q8
Q9
Q10
Mean
2.1
3.9
2.3
3.1
2.6
SD
0.60
0.33
0.83
1.05
1.22
Regarding open-ended comments, a device with a larger
screen (e.g., tablets) was suggested by several participants
who reported difficulties when trying to observe objects in
detail. Other participants also experienced glitches during
testing and emphasized the need to resolve such issues
before exposing the app to the target user group, to
prevent irritation. One participant suggested to log failed
sorting attempts and use this data to increase learning
potential when sorting garbage. The same participant also
Figure 1. Example gameplay screenshots. Plastic garbage
on medium difficulty (left). Food garbage on easy
difficulty (right).
Table 1. Customized System Usability Scale (SUS)
questions
Table 2. SUS Mean score and standard deviation (SD) for
each question. Score range 1 to 5.
89 The 18th Scandinavian Conference on Health informatics, Tromsø, Norway, August 22-24, 2022.
suggested to expand Sorterius and use it as a learning tool
for sorting other item categories. Overall, participants
gave positive comments of the general usability of the
game.
4 DISCUSSION
The mobile app Sorterius show promising potential for
being used by individuals with ID, and hopefully influence
PA levels. Participants in this study believed people with ID
would be able to learn how to use the app (Q7) and that
they would use it regularly (Q1). However, one question
(Q4) affected the overall score negatively, showing a high
score for thinking the target group would need support to
use the app. For this user group, relying on support from
caretaker is common. Because of this, scoring high in this
question may therefore not necessarily substantially affect
the usability. In fact, engagement of support people may
increase motivation for PA
[6].
In the previous work by
Haugland [12], this question was scored with a similar high
SUS score. Haugland also implemented a Pokémon Go-
inspired AR game. One possible explanation to this score,
may be that people with ID can struggle with abstract
concepts, and AR games may therefore be challenging for
the target group.
There was also a large score difference between the
individual with the highest score (SUS=80) and the
individual with the lowest score (SUS=42.5). When
completing the questionnaire, several participants
indicated that it was difficult to generalize, because they
worked with people with different levels of ID. Although
one participant thought the app was usable for people
with moderate to severe ID (with support), others thought
that it would be too difficult for people with a moderate
severity level. Variety in SUS-questionnaires is not
unusual, but the SUS range was nonetheless wide in this
usability test. This SUS-questionnaire may be hard to
evaluate precisely without testing the app on the target
user group.
This research has some limitations. The SUS questionnaire
was translated to Norwegian and modified to target
people working with ID. This could potentially affect the
meaning of the questions. A validation of the SUS
questionnaire in the Norwegian language is thus required
as further research. In addition, due to the restrictions
caused by the COVID-19 pandemic, we could not invite
people from the actual target user group to test the app.
These issues may affect the validity of results and have
been specifically addressed in the ongoing pilot and
feasibility study [9; 10].
5 CONCLUSION
The benefits of PA could also be achieved by people with
ID. Using technology is one way to provide incentives to a
target group outside the traditional marketing campaigns
of technology development. The apps available in the
market tend to be too complex for people with ID. This
research has presented a tailored app aiming to motivate
the target user group to do PA. The main contribution of
this project is a cross-platform AR app combining a
motivational tool for PA participation and learning (e.g.,
how to sort garbage), for a group that is often neglected in
technology intervention in society. The app includes goal
setting, involving the support people around the users
with ID, which previously is shown to improve motivation
towards PA in this user group. Future testing of the app
should include users with ID to ensure the generalizability
of this research and usability of the app. The evaluation of
the test results shows that, although some of the test
scores were low, we have created a “marginally
acceptable” user interface for the targeted user group.
The developed solution can be expanded to target a
broader range of users and projects. Using AR to address
meaningful societal aspects, such as the environment, can
be a helpful tool in raising awareness while at the same
time strengthening the knowledge towards serious topics
in an engaging digital environment. The final product has
received exciting feedback from experts and testing
participants.
Sorterius is currently being used in a pilot and feasibility
study for a randomized control trial (RCT) intervention
where one of the goals is to investigate whether tailored
mHealth support can stimulate PA for individuals with ID
[9; 10]. A Norwegian version is available in for free
download in Apple’s AppStore [17] and in Google Play [18].
English, Portuguese, Italian, and Spanish versions are
currently under development as part of the MOVE-IT
project [19].
6 REFERENCES
[1] Warburton, D. E., Nicol, C. W., and Bredin, S. S.,
"Health benefits of physical activity: the evidence",
CMAJ, 174 (6), 801-9, 2006
[2] WHO 'Physical activity',
<https://www.who.int/news-room/fact-
sheets/detail/physical-activity>, accessed 2022-06-
20.
[3] Wouters, M., Evenhuis, H. M., and Hilgenkamp, T. I.
M., "Physical activity levels of children and
adolescents with moderate-to-severe intellectual
disability", J Appl Res Intellect Disabil, 32 (1), 131-42,
2019
[4] Walsh, P. N., "Health indicators and intellectual
disability", Curr Opin Psychiatry, 21 (5), 474-8, 2008
[5] Dixon-Ibarra, A., Driver, S., Vanderbom, K., and
Humphries, K., "Understanding physical activity in
the group home setting: a qualitative inquiry",
Disabil Rehabil, 39 (7), 653-62, 2017
[6] Michalsen, H., Wangberg, S. C., Anke, A., Hartvigsen,
G., Jaccheri, L., and Arntzen, C., "Family members
and health care workers' perspectives on
motivational factors of participation in physical
activity for people with intellectual disability: A
qualitative study", J Intellect Disabil Res, 64 (4), 259-
70, 2020
[7] Gal, R., May, A. M., van Overmeeren, E. J., Simons,
M., and Monninkhof, E. M., "The Effect of Physical
Activity Interventions Comprising Wearables and
Smartphone Applications on Physical Activity: a
The 18th Scandinavian Conference on Health informatics, Tromsø, Norway, August 22-24, 2022. 90
Systematic Review and Meta-analysis", Sports Med
Open, 4 (1), 42, 2018
[8] Torrado, J. C., Jaccheri, L., Pelagatti, S., and Wold, I.,
"HikePal: A mobile exergame to motivate people
with intellectual disabilities to do outdoor physical
activities", Entertainment Computing, 42, 2022
[9] Michalsen, H., Wangberg, S. C., Hartvigsen, G.,
Jaccheri, L., Muzny, M., Henriksen, A., Olsen, M. I.,
Thrane, G., Jahnsen, R. B., Pettersen, G., Arntzen, C.,
and Anke, A., "Physical Activity With Tailored
mHealth Support for Individuals With Intellectual
Disabilities: Protocol for a Randomized Controlled
Trial", JMIR Res Protoc, 9 (6), e19213, 2020
[10] Michalsen, H., Wangberg, S. C., Hartvigsen, G.,
Henriksen, A., Pettersen, G., Jaccheri, L., Jahnsen, R.,
Thrane, G., Arntzen, C., and Anke, A., "Mobile health
support to stimulate physical activity in individuals
with intellectual disability: Protocol for mixed
methods pilot study", JMIR Preprints, 2022
[11] Berg, V., Haugland, V., Wiik, M. F., Michalsen, H.,
Anke, A., Muzny, M., Gomez, J., Martinez, S. G.,
Martinez-Millana, A., Henriksen, A., Sato, K., and
Hartvigsen, G. (2020), 'eHealth Approach for
Motivating Physical Activities of People with
Intellectual Disabilities', in Ilias O. Pappas, et al.
(eds.), Digital Transformation for a Sustainable
Society in the 21st Century (Cham: Springer
International Publishing), 31-41.
[12] Haugland, V. (2019), 'CorpOperatio: Game-inspired
App for Encouraging Outdoor Physical Activity for
People with Intellectual Disabilities'.
[13] Stellander, M. (2021), 'Sorterius: Game-inspired App
for Encouraging Outdoor Physical Activity for People
with Intellectual Disabilities'.
[14] Brooke, J. (1995), 'SUS - A quick and dirty usability
scale', Usability Evaluation in Industry.
[15] Sauro, J. (2011), A Practical Guide to the System
Usability Scale: Background, Benchmarks & Best
Practices (Denver, CO: Measuring Usability LLC).
[16] Bangor, A., Kortum, P. T., and Miller, J. T., "An
empirical evaluation of the System Usability Scale",
International Journal of Human-Computer
Interaction, 24 (6), 574-94, 2008
[17] UiT The Arctic University of Norway (2022),
'Sorterius (AppStore)',
<https://apps.apple.com/ng/app/sorterius/id16101
30479>, accessed 04-01.
[18] UiT The Arctic University of Norway (2022),
'Sorterius (Google Play)',
<https://play.google.com/store/apps/details?id=no.
uit.ifi.sorterius>, accessed 06-25.
[19] moveIT project (2022), 'moveIT',
<https://moveit.webs.upv.es>, accessed 06-10.
91 The 18th Scandinavian Conference on Health informatics, Tromsø, Norway, August 22-24, 2022.
