Cognitive Rehabilitation for Autism Children Mental Status Observation Using Virtual Reality Based Interactive Environment

Abstract

Proposed Cognitive Rehabilitation Therapy is to observe autism children mental status using virtual reality based interactive environment. Interactive environment is designed to analyze three levels of cognitive impairment decreasing functioning such as attention, reasoning, emotion, social behavior, decision making and language understanding. All the three levels were designed for mild autism children to have virtual walk through using HMD (Head Mount Device). We modeled a house and the behavior of the autism children were analyzed with day to day activities with the simple query to observe children language understanding. Observing the 5 autism children (3 boys and 2 girls in the age between 7 to 11) with the support of therapist gives promising improvement over the behavior in the virtual interactive environment and enhance their cognitive rehabilitation. Each level the therapist repeat the level, wherever children not responding. Observation results gives the children interact with virtual world with happy and positive emotions. In all the levels we incorporated children native language (Tamil Language) for the instruction, query and appreciation while they interact well. In future it is proposed to track their face, eye ball movement and predict the mental status of the children in detail to improve their overall mental status of autism children.

Full text

Cognitive Rehabilitation for Autism Children
Mental Status Observation Using Virtual
Reality Based Interactive Environment
Tamilselvi Dhamodharan
1
, Manju Thomas
1
,
Sathiyaprakash Ramdoss
2
, Karthikeyan JothiKumar
1
,
SaiNaveenaSri SaravanaSundharam
1(&)
,
BhavaniDevi Muthuramalingam
1
, NilofarNisa Hussainalikhan
1
,
Sugirtha Ravichandran
1
, VaibhavaShivani Vadivel
1
, Pavika Suresh
1
,
Sasikumar Buddhan
3
, and Ajith Madhusudanan
3
1
Cognitive Science Research Group, Department of Information Technology,
Thiagarajar College of Engineering, Madurai 625 015, India
dtamilselvi@tce.edu,
SaiNaveenaSri59364@student.tce.edu
2
Department of Doctoral Studies and Advance Programs, College of Education,
Ashland University, Ashland, OH 44805, USA
3
Kewaunee Labway India Pvt. Ltd., Bengaluru, India
Abstract. Proposed Cognitive Rehabilitation Therapy is to observe autism
children mental status using virtual reality based interactive environment.
Interactive environment is designed to analyze three levels of cognitive
impairment decreasing functioning such as attention, reasoning, emotion, social
behavior, decision making and language understanding. All the three levels
were designed for mild autism children to have virtual walk through using HMD
(Head Mount Device). We modeled a house and the behavior of the autism
children were analyzed with day to day activities with the simple query to
observe children language understanding. Observing the 5 autism children (3
boys and 2 girls in the age between 7 to 11) with the support of therapist gives
promising improvement over the behavior in the virtual interactive environment
and enhance their cognitive rehabilitation. Each level the therapist repeat the
level, wherever children not responding. Observation results gives the children
interact with virtual world with happy and positive emotions. In all the levels we
incorporated children native language (Tamil Language) for the instruction,
query and appreciation while they interact well. In future it is proposed to track
their face, eye ball movement and predict the mental status of the children in
detail to improve their overall mental status of autism children.
Keywords: Autism children Rehabilitation Virtual reality Mental state
observation Assessment
©Springer Nature Switzerland AG 2020
T. Ahram et al. (Eds.): IHSI 2020, AISC 1131, pp. 1213–1218, 2020.
https://doi.org/10.1007/978-3-030-39512-4_185

1 Introduction
Autism is diagnosed when a child or adult has abnormalities in a ‘triad’of behavioural
domains: social development, communication, and repetitive behaviour/obsessive
interests. Autism is a spectrum disorder and children have very different abilities and
skills [1,2]. Theoretical backbone of cognitive rehabilitation originates from Luria’s
theory of brain organization, which emphasizes integrated brain function [3]. In the
rehabilitation of children with autism therapeutic issues (e.g. eye contact, joint atten-
tion, turn taking, reading mental states and emotions) are usually addressed in con-
strained teaching sessions [4]. Autism children feel comfortable in predictable
environment, and enjoy interacting with computers [5,6]. Computer-based interactive
learning used in therapy and education have improve the students learning as well as
skills [7,8]. Virtual Environment research in autism includes some of the most
quintessential aspects of the human experience: from understanding others’intentions
(theory of mind) to managing everyday tasks (multitasking).
Children with autism fail to develop the ability to think about mental states in the
normal way, and thus fail to understand behavior in terms of mental state [9]. Theory of
mind, executive function of autism children mind have direct affinity with VR Tech-
nologies [10]. The immersive environment allows the student to train and develop
different social situations in a structured, visual and continuous manner [11]. ToMM
(Theory of Mind Machines) develops between 2 and 4 years of age, and allows pretend
play [12], understanding of false belief [13], and understanding of the relationships
between mental states [14]. Three commonly found traits involve abnormal response
for autism children are input stimuli, lack of human interaction, and the inability to
interact with people and objects for their needs [15].
Virtual environment are interactive, virtual image displays enhanced by special
processing such as auditory and haptic, to convince users that they are immersed in a
synthetic space [16–19]. Virtual environments can be simplified to the level of input
stimuli tolerable by autism children [20]. Visual and auditory effects make the autism
children to interact emotionally [21,22] . Virtual Reality environment immerses autism
children in synthetic world will exhibit different behaviors [23]. Computers play a vital
role in children language therapy [24,25]. VR environment, haptic technology added
with computers in the imaginary world plays a major role in the rehabilitation therapy
such cognitive, behavioral and social interaction for the autism children [26]. Tech-
nologies can support for human therapist and replace the therapy also.
Autism children immersive into virtual environments and the task created will
stimulate the visual sensory information and create the movement automatically and
change the behavior of children [27,28]. Fear, anxiety and basic day to day activities
were handled with the help of VR to improve the psychology based activities designed
to improve the therapy, learning and playing activity of children [29–31]. VR empower
the autism children cognitive disabilities [32]. HMD (Head Mount Device) along with
VR changes in real time in response to the children movement. Iwata says, bodily
movement is coordinated with visual images [33]. A more lifelike model can be
experienced along with the walking movements [34]. Additional devices like HMD
used should be evaluated for physiological effects of patients [35].
1214 T. Dhamodharan et al.

The proposed VR environment for autism children rehabilitation for the mental
state observation was designed with 3 levels. First level designed for Attention and
Reasoning. Second level with emotion and applied behavior analysis. Third level with
decision making and language understanding. Autism children observed for the mental
state were assessed by the therapist. Mild and Medium level autism children were taken
for pilot study and based on the cognitive and behavior state of the mental state, trials in
the VR environment was given and improvement is assessed by the therapist for the
children cognitive rehabilitation.
2 Proposed Methodology
Figure 1shows the proposed method to observe the mental state of autism children for
3 levels. Mild and medium level autism children were considered for the rehabilitation
trial. The virtual environment is created with 3 levels. Autism children interact with
Virtual Environment through HMD or Leap motion device based on the convenient to
interact for the 3 levels.
2.1 First Level –Cognitive Behavioral Therapy
The proposed first level concentrates on the autism children observation. The lack of
communication skills and difficulty in handling day to day routine tasks were designed
in the VR environment as shown in Fig. 2. The above tasks were created in VR and
interaction through the environment by the children using HMD/Leap motion device.
Based on the children behavior the mental state of the children were observed.
Attention, Recognition and Reasoning during interaction act as a measure to identify
the levels of cognitive therapy. Based on the levels, the therapy was repeated as per the
therapist recommendations. 3 mild level and 2 medium level children were considered
for the 3 levels of therapy.
Once the children performance in the first level shown the remarkable results and
enhancement in learning while going of cognitive behavior therapy, we can make the
children to enter in to the second level of Therapy. Mild autism children interact with
Fig. 1. VR environment for cognitive, occupational and evidence based therapy
Cognitive Rehabilitation for Autism Children Mental Status Observation 1215

VR using HMD. Medium level children find difficult to coordinate with HMD. Leap
motion device with hand free movement is given for the medium level autism children.
2.2 Second Level –Occupational and Music Therapy
Second level music therapy will stimulate the children emotions and interest to deal
with musical instruments and occupational therapy for the children to apply their
knowledge. While the children walkthrough using leap motion device, wire free
environment makes them to touch the various instruments designed in VR environ-
ment. While hearing the music they can touch and create the sounds in different forms.
Occupational Therapy various color balls in the environment are randomly distributed
and children can apply the color, shape identified in the first level and apply the
learning in second level and make the similar group of color balls or shapes. Mild and
medium children struggles and takes time to do the task in their own. Repeated trials
will shows the significant improvement over the second level.
2.3 Third Level Decision Making and Language Understanding
Third level VR environment is designed to make the autism children to take decision
and language understanding in Tamil and English Language. A house model was
created in VR and make the child to explore their day to day life things used and the
purpose with the voice added. It improves the language understanding and do the task
as per the instructions given in the VR environment.
3 Conclusion and Future Works
Based on the observation of 5 autism children at the 3 levels, mild level children are
interested in all 3 levels and do well the activities designed in the VR environment.
Medium level autism children one child hesitate to use HMD Device and interact only
through Leap motion device. In the second level, music therapy some children play the
Fig. 2. Cognitive behavior therapy for autism children
1216 T. Dhamodharan et al.

instruments randomly and one mild level children only hear the music and doesn’t
initiate interaction in the VR environment. Third level language description medium
level autism children find difficulty in English language. All the observations are made
in the presence of therapist. In future, the proposed work will be integrated in the
Avatar character in Augmented Reality for all the three levels to initiate the interaction,
mimic the activates and encourage the children to do the task. If the children hesitate,
Avatar will engage them with common songs and actions and decrease the distraction
and create interest for the student to interact with Avatar and perform the above 3 levels
of activities. This will promisingly increase the mental state of autism children for all
three levels and improve their life skills.
Acknowledgement. Work is funded by DST –CSRI (Department of Science and Technology –
Cognitive Research Initiative) Project (Sanction No. DST/CSRI/2017/304 dated 12.4.2018). Our
sincere thanks to Sparks Vidhyalaya Autism Special School for observing the Autism Children
observation during observation studies.
References
1. American Psychiatric Association: DSM-IV Diagnostic and Statistical Manual of Mental
Disorders, 4th edn. American Psychiatric Association, Washington DC (1994)
2. I.C.D-10: International classification of diseases, 10th edn. World Health Organization,
Geneva (1994)
3. Luria, A.R., Yudovich, F.Ia.: Speech and the development of mental processes in the child.
In: Kovasc, O., Simon, J. (eds.) Penguin, Harmondsworth (1971)
4. Howlin, P., Baron-Cohen, S., Hadwin, J.: Teaching Children With Autism to Mind-Read.
Wiley, Hoboken (1999)
5. Moor, D.: Computers and people with autism. Communication 20–21 (1998)
6. Powell, S.: The use of computers in teaching people with autism. Autism on the agenda.
Papers from a National Autistic Society Conference, London (1996)
7. Hershkowitz, V.: How adults with autism utilized their computers. Advocate - Newsletter of
the Autism Society of America Inc., November–December 1997
8. Hershkowitz, V.: Computer based therapy for individuals with autism. Adv. Mag. (2000)
9. Baron-Cohen, S., Tager-Flusberg, H., Cohen, D.J. (eds.): Understanding Other Minds:
Perspectives from Autism. Oxford University Press, New York (1994)
10. Rajendran, G.: Virtual environments and autism: a developmental psychopathological
approach. J. Comput. Assist. Learn. 29(4), 334–347 (2013)
11. Lorenzo, G., Lledó, A., Pomares, J., Roig, R.: Design and application of an immersive
virtual reality system to enhance emotional skills for children with autism spectrum
disorders. Comput. Educ. 98, 192–205 (2016)
12. Leslie, A.M.: Pretence and representation: the origins of theory of mind. Psychol. Rev. 94,
412–426 (1987)
13. Wimmer, H., Perner, J.: Beliefs about beliefs: representation and constraining function of
wrong beliefs in young children’s understanding of deception. Cognition 13, 103–128
(1983)
14. Wellman, H.: Children’s Theories of Mind. Bradford/MIT Press, Cambridge (1990)
15. Rutter, M.: Diagnosis and definition of childhood autism. J. Autistic Child. Schizophr. 8,
1938–1961 (1978)
Cognitive Rehabilitation for Autism Children Mental Status Observation 1217

16. Hodges, L.F., Kooper, R., Meyer, T.C., Rothbaum, B.O., Opdyke, D., De Graaff, J.J.,
Willford, J.S., North, M.M.: Virtual environments for treating the fear of heights. IEEE
Comput. 28(7), 27–34 (1995)
17. Durlach, N.I., Mavor, A.S. (eds.): Virtual Reality: Scientific and Technological Challanges.
National Academy Press, Washington DC (1995)
18. Burdea, G.C., Coiffet, P.: Virtual Reality Technology. Wiley, Hoboken (2003)
19. Stanney, K.M. (ed.): Handbook of Virtual Environments: Design, Implementation and
Applications. Lawrence Erlbaum, Mahwah (2002)
20. Strickland, D.: Virtual reality for the treatment of autism. In: Riva, G. (ed.) Virtual Reality in
Neuro-Psycho-Physiology, pp. 81–86 (1997)
21. Park, D., Youderian, P.: Light and number: ordering principles in the world of an autistic
child. J. Autism Child. Schizophr. 4, 313–323 (1974)
22. Grandin, T., Acariano, M.: Emergence: Labeled Autistic. Arena, Novata (1986)
23. Gregory, R.L.: Seeing by exploring. In: Ellis, S.R. (ed.) Pictorial Communications in Virtual
and Real Environments, pp. 328–337. Taylor and Francil, London (1991)
24. Colby, K.M.: Computer aided language development in non-speaking children. Arch. Gen.
Psychiatry 19, 641–652 (1968)
25. Zarr, M.: Computer-aided psychotherapy: machine helping therapist. Psychiatr. Ann. 24(1),
42–46 (1994)
26. Barfield, W., Hendrix, C.: Factors affecting presence and performance in virtual environ-
ments. In: Interactive Technology and the New Paradigm for Healthcare, pp. 21–28. IOS
Press, Washington D.C. (1995)
27. Wann, J.P., Turnbull, J.D.: Motor skill learning in cerebral palsy: movement, action, and
computer-enhanced therapy. Bailliere’s Clin. Neurol. 2(1), 15–28 (1993)
28. VREPAR Project. http://www.ehto.be/ht_projects/vrepar/movement.html
29. Strickland, D.: A virtual reality application with autistic children. Presence: Teleoperators
Virtual Environ. 5(3 Summer), 319–329 (1996)
30. Buckert-Donelson, A.: Heads up products: virtual worlds ease dental patients. VR World
3(3), 6 (1995)
31. Gourlay, D., Lun, K.C., Lee, Y.N., et al.: Virtual reality for relearning daily living skills. Int.
J. Med. Infomatics 60, 225–261 (2000)
32. Greenleaf, W.J.: Rehabilitation, ergonomics, and disability solutions using virtual reality
technology. In: Interactive Technology and the New Paradigm for Healthcare, pp. 415–422.
IOS Press, Washington D.C. (1995)
33. Kenyon, R.V., DeFanti, T.A., Sandin, D.J.: Visual requirements for virtual environment
generation. J. Soc. Inform. Disp. 3, 211–214 (1995)
34. Rowley, T.W.: Problems of head mounted display. In: IEE Colloquium on Real World
Visualization –Virtual World –Virtual Reality, pp. 12/1–4. IEE, London (1991)
35. Nakajima, R., Nomura, J., Fukushima, S., Ojika, T.: A relax/refresh system applying virtual
reality. In: Proceedings: 3rd IEEE International Workshop on Robot and Human
Communication, pp. 145–150. IEEE, NewYork (1994)
1218 T. Dhamodharan et al.