Available via license: CC BY 4.0
Content may be subject to copyright.
DISCOVERING BEHAVIORAL PREDISPOSITIONS IN DATA TO
IMPROVE HUMAN ACTIVITY RECOGNITION
Maximilian Popko
University of Rostock, Germany
maximilian.popko@uni-rostock.de
Sebastian Bader
University of Rostock, Germany
sebastian.bader@uni-rostock.de
Stefan Lüdtke
University of Mannheim, Germany
luedtke@es.uni-mannheim.de
Thomas Kirste
University of Rostock, Germany
thomas.kirste@uni-rostock.de
ABS TRAC T
The automatic, sensor-based assessment of challenging behavior of persons with dementia is an
important task to support the selection of interventions. However, predicting behaviors like apathy
and agitation is challenging due to the large inter- and intra-patient variability. Goal of this paper is to
improve the recognition performance by making use of the observation that patients tend to show
specific behaviors at certain times of the day or week. We propose to identify such segments of similar
behavior via clustering the distributions of annotations of the time segments. All time segments
within a cluster then consist of similar behaviors and thus indicate a behavioral predisposition (BPD).
We utilize BPDs by training a classifier for each BPD. Empirically, we demonstrate that when the
BPD per time segment is known, activity recognition performance can be substantially improved.
Keywords Human activity recognition ·Wearable sensors ·Machine learning ·Clustering
1 Introduction
The recognition of human behavior is important for many domains, like sports, the analysis of manual work processes,
or as a component for situation-aware assistants. As another example, sensor-based Human Activity Recognition (HAR)
can also be used for the automatic assessment of behavioral symptoms of people with dementia, like apathy or agitation.
In contrast to questionnaires like the Cohen-Mansfield Agitation Inventory [
1
], HAR allows the real-time, objective
assessment of symptoms, supporting caregivers in selecting appropriate interventions. However, accurate HAR is
challenging due to the high inter- and intra-subject variability of movement. Specifically, the automatic recognition of
challenging behavior from wearable sensor data has been of limited success so far [2].
Multiple approaches have been proposed to improve HAR accuracy, by making use of additional external knowledge
[
3
,
4
,
5
,
6
,
7
,
8
]. In this paper, we investigate how HAR performance can be improved by making use of the fact that
there is a tendency of specific subjects to show specific behaviors at certain times of the day or week. For example,
sundowning is the increase in restlessness of some people with dementia in the evening [
9
]. The knowledge of such
behavioral predispositions (BPDs) could serve as a useful prior to increase HAR accuracy.
Previous work relied on external knowledge to define BPDs. For example, Lüdtke et al. [
7
] partitioned warehouse order
picking into distinct process steps, the knowledge of which can inform HAR. However, in the context of dementia
care, there are no established BPDs that are externally given. The contribution of this paper is to identify such BPDs
in a data-driven way: We calculate the distribution of annotations in time segments of a day, and then cluster these
histograms. This results in clusters of time frames in which the subjects behaved similarly. Each cluster forms a BPD.
Given the BPD, we then train a classifier specifically for data recorded within the BPD, i.e., there is one classifier for
each cluster. This classifier is thus biased towards the more frequent classes in the BPD. An overview of this process
can be found in figure 1. In summary, this paper provides the following contributions:
arXiv:2207.08816v1 [cs.LG] 18 Jul 2022
Discovering Behavioral Predispositions in Data to Improve Human Activity Recognition
Motion Sensor Data
Annotations Calculating Histograms
Classifier for
BPD 1
Classifier for
BPD 2
Classifier for
BPD 3
Classifier for
BPD 4
Features
Clustering to BPDs
Select Classifier based on
BPD
Predicted Behavior
Figure 1: Overview of the concept. We calculate the distribution of annotations for a given segment of the day
(histograms). We then cluster these distributions. Next, we use the cluster label (BPD) to select the classifier (color
indicates the corresponding classifier). The classifier predicts the behavior from the features of the motion sensor data.
• A clustering-based method for the data-driven construction of behavioral predispositions (BPDs)
• An approach for utilizing these BPDs in HAR
•
An evaluation of this approach for the sensor-based recognition of behavioral symptom of persons with
dementia
Empirically, our approach leads to a substantial increase of HAR performance, when assuming that the BPD for each
time segment is given.
2 Data and Observations
This section presents the insideDem framework, as well as the data set. We make observations about the distribution of
annotations given time frames, which will then lead to the construction of the BPDs and the utilization of them in the
methodology section 3.
2.1 insideDem Project
Part of the insideDem framework was the recording of the behaviours of dementia diagnozed patients [
10
]. In two
nursing homes, real-time observations and sensor data of a total of 17 residents (11 women and 6 men) with moderate
to severe dementia were recorded. The behaviour of persons in one nursing home was also recorded on video. The age
range was 73 to 94 years. All persons were under psycho-pharmacological treatment. Dementia medication was taken
by 8 residents of the nursing homes. The behaviour of the persons was recorded in real time by an expert in the sitting
room of the nursing home. This room was accessible to all residents and their visitors as well as the staff of the nursing
home. The behaviour was annotated every five minutes following the annotation scheme which was developed for the
insideDem project. This scheme is based on the neuropsychiatric inventory [
11
] and the Cohens-Mansfield agitation
inventory [
1
]. It consists of seven different behaviours: apathy, general restlessness, mannerisms, pacing, aggression,
trying to get to a different place, normal behaviour.
During the day, between 8 a.m. and 6 p.m., the subjects wore two wristbands, which were attached to the dominant
hand and the ankle. During the night, a band was worn on the ankle. The sensors recorded accelerations and rotations
as well as the sound level of the environment, the light conditions and the sound pressure. In this work, we only use the
acceleration sensor data.
2.2 Observations
Figure 2 shows the distribution of annotations for each subject. The behaviour of the eight patients were observed
over the course of 29 days and accumulate around 17,000 annotated five-minute intervals. For every subject the the
behaviours occur in different portions. As example, the patient X111 showed 9% apathy where as X126 showed nearly
50% apathy during the time span of 29 days. However, for X111, pacing was mostly annotated but X126 nearly showed
no pacing.
In the literature events for triggering agitated behaviors were observed. The sunset phenomenon as described in [
9
] may
be such a trigger for one BPD. At sunset people with dementia often show more agitated behavior than at other times of
2
Discovering Behavioral Predispositions in Data to Improve Human Activity Recognition
X110 X111 X113 X114 X121 X122 X124 X126
0
20
40
60
80
100
Distribution
25.1%
9.0%
33.0%
3.3%
36.1%
20.7% 27.7%
49.7%
2.1%
13.0%
3.7%
3.6%
19.8%
14.3%
12.2%
20.9%
18.3% 24.4%
12.1%
40.9%
24.7%
27.3% 47.9%
44.2%
50.0%
39.9%
37.1%
8.7%
2.3%
2.0%
11.8%
42.0%
21.8% 12.8% 22.4%
4.2%
Apathy
Aggression
General Restlessness
Mannerisms
Getting to different place
Pacing
Normal
Figure 2: Distribution of annotations per person over all days of recordings.
the day. Also, the fact of boredom can trigger agitated behavior. If there is nothing planned for an afternoon, dementia
diagnosed persons might show agitated behavior because of the boredom [
12
]. As a result, it is reasonable to look into
noticeable time frames of the day which show one person behaving agitated more often.
The figure 3 (a) now shows the average occurrences of the annotations for one-hour segments of the day. The represented
subject behaves in the morning more apathetic than in the later course of the day. Also at noon, the patient shows more
normal behavior. Pacing is mostly happening in the afternoon. Figure 3 (b) shows the variances of the annotations
between each day. This plot represents the distribution of the proportion each behavior has on the time segment every
day. Between 10 and 11 a.m. we saw the patient behaves apathetically to 46% in plot (a). However, in plot (b) we see
that the apathy part can differ from 0% to 100% with equal probability from day to day.
3 Methods
Behavioural predispositions (BPD) describe the tendency to show specific behaviours. In this section, we propose a
method to form such BPDs from the annotations of behaviors. In the following, we present a simple model that can use
these newly found BPDs to simplify the classification.
3.1 Extracting behavioural predispositions
BPD describe the urge to show specific kinds of behaviors. Knowing such would intuitively mean that the prediction
of the current actual behavior becomes easier since it provides an informative prior on the behavior of the subject.
Unfortunately an expert model describing BPD for patients with dementia does not exist and thus the aim is to derive
them from the data of the insideDem project.
3.1.1 Time-based BPD
Following the observations of figure 3 (a), one can argue to use a similar approach to [
8
] or define that every BPD
is one segment of the day. This means from 10 to 11 a.m. the subject always behaves similarly and by knowing the
current time, we can use this information to improve the classification. However, this approach has the drawback of
high variances of the annotations between each day which lead to BPDs that do not carry a lot of information about the
actual behavior.
3.1.2 Clustering-based BPD
To overcome the problem of high variances, we propose to find similar distributions globally over all days and time
segments instead of grouping them by the time of the day. Let
p(At)
be the probability distribution of the annotation
for the day and time segment
t
. Note that
t
describes a specific time frame e.g. 06–15 10:00 to 06–15 11:00. This
categorical distribution can be obtained by counting the relative frequencies of the annotations in the segment. This
3
Discovering Behavioral Predispositions in Data to Improve Human Activity Recognition
(a) Average of annotations in time segment
(b) Distribution of annotation in time segment
Figure 3: Percentual distribution of annotations for one subject and one-hour intervals of the day. (a) shows the average
of the annotation and (b) describes the distribution (variance).
counted distribution is called a histogram. The BPD represents the tendency to behave in a certain way. Given a set of
histograms, the task is to divide this set into groups. These groups should contain similar histograms. With the help of
the k-means clustering algorithm, clusters of histograms can be generated which are grouped according to their spatial
similarity (Euclidean distance). Each of these clusters describes a behavioral disposition. The histograms of the cluster
D=dtherefore represent the distribution p(A|D=d).
3.2 Classification of Sensor Data
In the scope of this work, we want to show what impact the complete knowledge of the clustered BPD can have on
the classification. Therefore, we assume the BPD, i.e. the cluster, as given. Note that this would not be the case in
a real world environment. Given a BPD
d
, we then have a prior knowledge of the annotation
p(A|d)
. To make use
of this knowledge, we train a different classifier for each BPD. Thus, each classifier is trained with biased data and is
therefore biased to more likely predict the classes that are most presently in this BPD. The classifier can be seen as
expert for its BPD. The model
m
maps sensor data
s
and the BPD
d
to an annotation
a
by choosing the classifier
cd
that
was specifically trained for d. The classifier then maps sto a:
m(s, d) = cd(s) = a .
4 Experimental Evaluation
The overall goal of the experiments are the evaluation of the impact of the BPD on the HAR performance. In the
following, we describe the experimental design in more detail.
4.1 Experimental Design
To evaluate the impact of the knowledge of the BPDs on HAR performance, we use a factorial design. The factors and
levels are presented in table 1. For each subject, a model with a different clustering method, number of clusters (i.e.
BPDs), segment length, and classifier is trained. We randomly select 70% of the data for training and 30% for testing
4
Discovering Behavioral Predispositions in Data to Improve Human Activity Recognition
Table 1: Factors and levels of the experimental design
Factors Levels Description
Clustering K-means, time-based Methods for generating the BPDs
#-clusters 1, . . . , 20 Number of BPDs
Segment
lengths
0, . . . , 120 minutes (k-means)
Length of the segment over which the his-
togram is calculated, only for k-means ap-
proach
Classifiers Majority, naive Bayes, SVM
Classifiers to predict the annotation from
sensor data
Subjects
X110, X111, X113, X114, X121,
X122, X124, X126
One model for each subject
the model. Every combination of the levels is evaluated 10 times. Different clusters and segment lengths are used to test
the expressiveness of the BPD. The segment length plays an important role for the k-means clustering strategy as the
length is used to define the size over which the histogram is calculated and then clustered. At a segment length of 30
minutes, we obtain around 400 histograms per subject. We then cluster the histograms of a single subject, resulting
in BPDs specifically for this subject. High number of clusters and small segment lengths yield finer BPDs. Note that
one cluster means we have no further information about the BPD and thus gives us a baseline of performance. The
time-based clustering method describes that a BPD only depends on the time. E.g., having 10 clusters and time-based
clustering this method would split the day into 10 parts and for every part, one classifier is trained. We use three
different methods for classifying the sensor data. The majority method functions as a baseline by always outputting the
most frequent annotation of the BPD. The naive Bayes (NB) method shows the performance of a simplistic classifier.
The NB uses for every BPD a different prior and observation model. Thirdly, a support vector machine (SVM) is
trained. For preprocessing the accelerator data, features were calculated following the work of [
13
]. The original data is
sampled in 50 Hz. Feature were calculated over a window size of one minute and an overlap of 50%. Statistical features,
spatial relations between the three axis and frequency features were used. As measure of accuracy, we use F1score.
4.2 Results and Discussion
Figure 4 shows the average
F1
scores for each classifier and the strategies used for generating the BPDs. Firstly, we
want to investigate only the k-means approach. The SVM shows an increase of
F1
score up to a number of about 10
clusters. From this point on, only smaller segment sizes benefit. For the majority classifier, the
F1
score no longer
increases significantly above a number of 6 clusters. The naive Bayes classifier shows a nearly linear increase. Thus, for
all classifiers, the
F1
measure increases with a larger number of clusters, as well as a decreasing segment length. The
fact that the majority classifier benefits from this approach comes as no surprise. With an increasing number of clusters
and finer segment lengths, the most frequent annotation occupies a higher portion of data within the cluster. Predicting
only the most frequent annotation yields an
F1
score of around 0.1 whereas dividing the data by the BPD yields a score
of around 0.5 at 20 BPDs and a segment length of 30 minutes. Note that some BPDs only describe one annotation and
thus predicting can achieve 100% accuracy by always outputting that one same class. According to this observation, the
classification task should become simpler. Given such BPD with only one class, all classifiers used here can benefit and
improve with knowledge of the BPD. However, the
F1
score of the majority classifier proposes that the knowledge of
the most frequent annotation in the training data does not give us further advantages when we have more than 6 clusters.
Yet the
F1
score of the naive Bayes classifier increases nearly linearly. Also, the SVM has a still increasing
F1
score at
20 BPDs, which underlines the fact that knowing the BPD simplifies the classification task by reducing the complexity
of possible behaviors or even ruling them out. Note that the knowledge presented by the BPD converges with the BPD
describing the exact underlying histogram, i.e. we have the same number of BPDs (clusters) as histograms.
We now investigate the time-based approach. The time-based approach shows an increase of
F1
score as the number of
clusters rises. This can be observed for every classifier evaluated. Although the increase is not high, the knowledge
of the clock time can help the classification. Thus dividing the day into segments and training a classifier for every
segment each, increases the
F1
score. Comparing this approach with the k-means approach can be done by comparing
the points where the number of BPDs and the segment length are approximately equal for both. E.g. splitting the day,
where the subjects were recorded, into 20 segments yields a segment length of around 30 minutes and 20 BPDs. These
points are denoted with rectangles in figure 4. The plot yields a better performance of the k-means approach for all 4
points. The inferiority of the time-based approach can be explained through the high variances of annotations among
the days (shown in figure 3). The k-means approach, on the other hand, describes the histograms by their similarity and
thus achieving less variance within the cluster. Having less variance means that the classifier has an easier task to solve.
5
Discovering Behavioral Predispositions in Data to Improve Human Activity Recognition
0 2 4 6 8 10 12 14 16 18 20
0.0
0.1
0.2
0.3
0.4
0.5
0.6 K-Means BPD - NB
30
45
60
90
120
0 2 4 6 8 10 12 14 16 18 20
0.0
0.1
0.2
0.3
0.4
0.5
0.6 K-Means BPD - Majority
30
45
60
90
120
0 2 4 6 8 10 12 14 16 18 20
0.0
0.1
0.2
0.3
0.4
0.5
0.6 K-Means BPD - SVM
30
45
60
90
120
0 2 4 6 8 10 12 14 16 18 20
0.0
0.1
0.2
0.3
0.4
0.5
0.6 Time-based BPD
SVM
NB
Majority
#-Cluster
Average
F
1
Figure 4: Averaged
F1
score for varying number of clusters and segment lengths. For time-based clustering, the segment
length is implicitly given through the number of clusters. For k-means clustering, the segment lengths of the histograms
are represented by the color. The segment length is in minutes. Rectangles denote similar BPD structures because of
the same number of clusters and similar segment lengths.
Overall, the
F1
score reaches a maximum of around 0.5. This score is produced by using 20 BPDs, a segment length of
30 minutes, and an SVM as a classifier. The same results can be achieved by using the majority method. The SVM
outperforms the majority classifier for larger segment lengths. Figure 5 shows the confusion matrix of the SVMs for the
classification with 20 BPDs and a segment length of 30 minutes. We can see that the classes mannerisms, apathy, and
normal behavior represent the majority of the annotations. Furthermore, there is great uncertainty between these classes.
This can be explained by the similarity of these behaviors in this scenario. Intuitively, normal behavior is similar to
apathy, as dementia patients in the nursing home live more secluded. Mannerisms are difficult to recognize when, for
example, they are performed with the non-dominant hand (sensor is worn at the dominant hand). Another uncertainty is
between the annotations pacing, getting to a different place, and normal behavior. This can be explained by the fact that
these three annotations can be understood as the activity of walking so a misclassification can occur between these
classes. The behavior is also annotated in five-minute intervals, but the classification takes place in 30-second intervals.
This means that the person may have walked at one point in time, but the interval was annotated differently. Another
reason is the exclusive use of the motion sensor on the wrist.
5 Future work
The evaluation showed that the knowledge of the BPD can yield better classification and thus further investigations
are of interest. There is the question of whether finding the BPD with k-means clustering is the best choice. Since
similar histograms and thus categorical probability distributions are of interest, clustering methods with entropy-based
distances, such as the Jensen-Shannon divergence, are of interest. Furthermore, Latent Dirichlet Allocation (LDA) [
14
]
can be investigated as a method for determining behavioral dispositions. The LDA is a generative probabilistic model
and is mainly used in applications to assign a topic to documents based on their words. In this scenario, a segment of
6
Discovering Behavioral Predispositions in Data to Improve Human Activity Recognition
Apathy
Restlessness
Mannerisms
Normal
Get to diff. Place
Pacing
Predicted Annotation
Apathy
Restlessness
Mannerisms
Normal
Get to diff. Place
Pacing
True Annotation
8850 48 1579 1683 48 203
22 96 41 5 0 26
1328 66 4417 1517 22 508
1654 7 2465 9310 84 902
22 0 7 44 24 17
261 11 526 713 122 1988
SVM, 20 BPDs, 30 min segments
Figure 5: Confusion matrix of the predicted behaviors using 20 BPDs and thus 20 SVMs. A segment length of 30
minutes were used to calculate the histrogams.
the day can be regarded as a document and its words are the annotations in this segment so that the resulting topics
describe the BPD.
Currently, the BPD is only constructed by the annotations. A next step could be to determine the BPD based on the
annotation as well as the sensor data. This would lead to BPDs that also hold information about the current motion data
and its connection to the annotation. In this work, we created BPDs for a single person. As a next step, the evaluation
of global BPDs across the patients is needed.
We also saw that the overall accuracy of the classification was low and thus looking into better options for classification
is of interest. In this work, we only presented the approach of training a classifier for each BPD. However, this reduces
the number of samples for the classifier. Another approach can be the feeding of the BPD label as input to the classifier.
Also, classifiers that can make use of distributions over BPDs are of interest. Methods like mixed-effects neural
networks [15] can model BPDs as another feature for activity recognition.
In reality, the BPD is not available and has to be found from sensor data. A good estimation of the BPD is therefore
needed to improve the classification. With an increasing number of BPDs, the error of classification will also rise. Thus
finding a good number of BPDs and a model to predict them is a crucial point for practical use. This can be done by
using hierarchical models which first infer the BPD from the sensor data and then make use of the relations between
behavior and BPD. Also, hidden Markov models can be of interest to exploit possible temporal properties of the BPDs.
6 Related work
To improve the classification of sensor data using additional information, firstly, the knowledge has to be modeled
and perhaps has to be generated beforehand. Depending on the domain, the kind of knowledge can differ and thus
arise different ways of incorporation into human activity recognition. Mostly the improvement of accuracy is the main
purpose. However, the simplification of the recognition model can also be of interest. The authors of [
3
] are aiming
to create a model that still maintains the same level of accuracy under less training data than state-of-the-art methods.
They use region connection calculus (RCC) [
16
] to describe the spatial relations between objects and formalize the
activities and their causal properties within the domain using the planning domain definition language (PDDL) [
17
]. A
probabilistic concurrent constraint automaton (PCCA) [
18
] is generated from the PDDL model and reasons about the
7
Discovering Behavioral Predispositions in Data to Improve Human Activity Recognition
predicates from the RCC model. The RCC model calculates these predicates from high-level observations. The authors
show that their approach can yield similar results in comparison to convolutional neural networks (CNN) but avoid the
training data hungry nature of neural networks. On the other hand, if enough training data and further information about
the domain is available, the work of Xie et al. [
4
] aims to improve the accuracy of classification by incorporating prior
knowledge in the training process of a neural network. They achieve this goal by modeling rules about the practicability
of actions using linear temporal logics (LTL) [
19
]. To overcome the problem of the multi-interpretable nature of LTL,
the author convert them into deterministic finite automatons such that they can be embedded as hierarchical graph into
the loss calculation of the neural network’s training process and achieve learning consistent to the LTL rules and better
accuracy at classification. The works by Yordanaova et al. [
5
] and Rueda et al. [
6
] use computational causal behavioral
models (CCBM) [
20
] to model domain knowledge and reason about the current state of the environment. CCBM also
uses PPDL-like rules to describe the domain-specific properties of the activities. Also, the relations between user and
object as well as durations of activities can be modeled in CCBM. As an observation model, a decision tree (DT) in [
5
]
and a convolutional neural network in [
6
] are used to weight the predicted state based on the sensor data. In contrast to
these works, the prior knowledge does not need to be embedded directly in the model. Lüdtke et al. [
7
] demonstrate
how providing external context information impacts classification performance. Specifically, they show that providing
the current process step of structured activities can improve HAR performance, even when that information is noisy.
Even if no external information is available, the authors of [
8
] made the observation that activities feature different
quantities in times of a day. Thus segmenting the day and reasoning about the activities in one segment yields better
accuracy than reasoning about the activities of the whole day.
7 Conclusion
In this paper, we proposed a data-driven method to construct behavioral predispositions and use them as informative
prior to improve the performance of sensor-based human activity recognition. We presented a clustering-based approach
that can determine time segments that display similar behaviors of a dementia-diagnosed person. We found such
groups by first calculating the distribution of the annotations in the time segment and then clustering these distributions.
The resulting clusters explain behavioral predispositions (BPDs). With complete knowledge of the current BPD, we
demonstrated empirically that a substantial increase of the
F1
score can be achieved by using classifiers that were
specifically trained for each BPD. For practical usage, the investigation of predicting the BPDs from sensor data is of
high interest, as the BPD is not given a priori and instead must be inferred.
Furthermore, the presented approach is not only applicable to predicting the behavior of persons with dementia, but also
to any continuous HAR task. Whenever the prior for occurrence of activities varies over time, the method described in
this paper offers a viable option to improve HAR performance.
References
[1]
Jiska Cohen-Mansfield. Assessment of agitation. International Psychogeriatrics, 8(2):233–245, 1996.
doi:10.1017/S104161029600261X.
[2]
Doreen Goerss, Albert Hein, Sebastian Bader, Margareta Halek, Sven Kernebeck, Andreas Kutschke, Christina
Heine, Frank Krueger, Thomas Kirste, and Stefan Teipel. Automated sensor-based detection of challenging
behaviors in advanced stages of dementia in nursing homes. Alzheimer’s & Dementia, 16(4):672–680, 2020.
doi:https://doi.org/10.1016/j.jalz.2019.08.193. URL
https://alz-journals.onlinelibrary.wiley.com/
doi/abs/10.1016/j.jalz.2019.08.193.
[3]
Sang Uk Lee, Andreas Hofmann, and Brian Williams. A Model-Based Human Activity Recognition
for Human–Robot Collaboration. In 2019 IEEE/RSJ International Conference on Intelligent Robots
and Systems (IROS), pages 736–743, Macau, China, November 2019. IEEE. ISBN 978-1-72814-004-9.
doi:10.1109/IROS40897.2019.8967650. URL https://ieeexplore.ieee.org/document/8967650/.
[4]
Yaqi Xie, Fan Zhou, and Harold Soh. Embedding Symbolic Temporal Knowledge into Deep Sequential Models. In
2021 IEEE International Conference on Robotics and Automation (ICRA), pages 4267–4273, Xi’an, China, May
2021. IEEE. ISBN 978-1-72819-077-8. doi:10.1109/ICRA48506.2021.9561952. URL
https://ieeexplore.
ieee.org/document/9561952/.
[5] Kristina Yordanova, Stefan Lüdtke, Samuel Whitehouse, Frank Krüger, Adeline Paiement, Majid Mirmehdi, Ian
Craddock, and Thomas Kirste. Analysing Cooking Behaviour in Home Settings: Towards Health Monitoring.
Sensors, 19(3):646, February 2019. ISSN 1424-8220. doi:10.3390/s19030646. URL
http://www.mdpi.com/
1424-8220/19/3/646.
8
Discovering Behavioral Predispositions in Data to Improve Human Activity Recognition
[6]
Fernando Moya Rueda, Stefan Lüdtke, Max Schroder, Kristina Yordanova, Thomas Kirste, and Gernot A.
Fink. Combining Symbolic Reasoning and Deep Learning for Human Activity Recognition. In 2019 IEEE
International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), pages
22–27, Kyoto, Japan, March 2019. IEEE. ISBN 978-1-5386-9151-9. doi:10.1109/PERCOMW.2019.8730792.
URL https://ieeexplore.ieee.org/document/8730792/.
[7]
Stefan Lüdtke, Fernando Moya Rueda, Waqas Ahmed, Gernot A. Fink, and Thomas Kirste. Human Activity
Recognition using Attribute-Based Neural Networks and Context Information. In 3rd International Workshop on
Deep Learning for Human Activity Recognition, 2021.
[8]
Sozo Inoue, Naonori Ueda, Yasunobu Nohara, and Naoki Nakashima. Recognizing and Understanding Nursing
Activities for a Whole Day with a Big Dataset. Journal of Information Processing, 24(6):853–866, 2016. ISSN
1882-6652. doi:10.2197/ipsjjip.24.853. URL
https://www.jstage.jst.go.jp/article/ipsjjip/24/6/
24_853/_article.
[9]
Marco Canevelli, Martina Valletta, Alessandro Trebbastoni, Giuseppe Sarli, Fabrizia D’Antonio, Leonardo
Tariciotti, Carlo de Lena, and Giuseppe Bruno. Sundowning in Dementia: Clinical Relevance, Pathophysiological
Determinants, and Therapeutic Approaches. Frontiers in Medicine, 3:73, December 2016. ISSN 2296-858X.
doi:10.3389/fmed.2016.00073. URL
http://journal.frontiersin.org/article/10.3389/fmed.2016.
00073/full.
[10]
Stefan Teipel, Christina Heine, Albert Hein, Frank Krüger, Andreas Kutschke, Sven Kernebeck, Margareta Halek,
Sebastian Bader, and Thomas Kirste. Multidimensional assessment of challenging behaviors in advanced stages
of dementia in nursing homes—The insideDEM framework. Alzheimer’s & Dementia: Diagnosis, Assessment &
Disease Monitoring, 8(1):36–44, January 2017. ISSN 2352-8729, 2352-8729. doi:10.1016/j.dadm.2017.03.006.
URL https://onlinelibrary.wiley.com/doi/abs/10.1016/j.dadm.2017.03.006.
[11]
Jeffrey L Cummings. The neuropsychiatric inventory: assessing psychopathology in dementia patients. Neurology,
48(5 Suppl 6):10S–16S, 1997.
[12]
Jiska Cohen-Mansfield and Perla Werner. Environmental influences on agitation: An integrative summary of an
observational study. American Journal of Alzheimer’s Care and Related Disorders & Research, 10(1):32–39,
January 1995. ISSN 0895-5336. doi:10.1177/153331759501000108. URL
http://journals.sagepub.com/
doi/10.1177/153331759501000108.
[13]
Ridwan Alam, Azziza Bankole, Martha Anderson, and John Lach. Multiple-Instance Learning for Sparse
Behavior Modeling from Wearables: Toward Dementia-Related Agitation Prediction. In 2019 41st Annual
International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pages 1330–
1333, Berlin, Germany, July 2019. IEEE. ISBN 978-1-5386-1311-5. doi:10.1109/EMBC.2019.8856502. URL
https://ieeexplore.ieee.org/document/8856502/.
[14]
David M Blei, Andrew Y Ng, and Michael I Jordan. Latent dirichlet allocation. Journal of machine Learning
research, 3(Jan):993–1022, 2003.
[15]
Yunyang Xiong, Hyunwoo J. Kim, and Vikas Singh. Mixed Effects Neural Networks (MeNets) With Applications
to Gaze Estimation. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages
7735–7744, Long Beach, CA, USA, June 2019. IEEE. ISBN 978-1-72813-293-8. doi:10.1109/CVPR.2019.00793.
URL https://ieeexplore.ieee.org/document/8954429/.
[16]
Anthony G. Cohn, Brandon Bennett, John Gooday, and Nicholas Mark Gotts. Qualitative Spatial Representation
and Reasoning with the Region Connection Calculus. GeoInformatica, 1(3):275–316, October 1997. ISSN
1573-7624. doi:10.1023/A:1009712514511. URL https://doi.org/10.1023/A:1009712514511.
[17]
M. Fox and D. Long. PDDL2.1: An Extension to PDDL for Expressing Temporal Planning Domains. Journal
of Artificial Intelligence Research, 20:61–124, December 2003. ISSN 1076-9757. doi:10.1613/jair.1129. URL
https://jair.org/index.php/jair/article/view/10352.
[18]
B.C. Williams, M.D. Ingham, S.H. Chung, and P.H. Elliott. Model-based programming of intelli-
gent embedded systems and robotic space explorers. Proceedings of the IEEE, 91(1):212–237, 2003.
doi:10.1109/JPROC.2002.805828.
[19]
Amir Pnueli. The temporal logic of programs. In Proceedings of the 18th Annual Symposium on Foundations
of Computer Science, SFCS ’77, page 46–57, USA, 1977. IEEE Computer Society. doi:10.1109/SFCS.1977.32.
URL https://doi.org/10.1109/SFCS.1977.32.
[20]
Frank Krüger, Alexander Steiniger, Sebastian Bader, and Thomas Kirste. Evaluating the robustness of activity
recognition using computational causal behavior models. In Proceedings of the 2012 ACM Conference on
9
