ArticlePDF Available

Abstract and Figures

In the last two decades, studies about using technology for automatic detection of human fall increased considerably. The automatic detection of falls allows for quicker aid that is key to increasing the chances of treatment and mitigating the consequences of falls. However, each type of fall has its specificities, and determining the correct type of fall can help treat the person who has fallen. Although it is essential to use computational methods to classify falls, there are few studies about that in the literature, especially compared to the studies that propose solutions for fall detection. In this sense, we execute a systematic literature review (SLR) using the Kitchenham (2009) [1] method to investigate the computational solutions used to classify the different types of falls. We performed a search on Scopus, Web of Science and PubMed scientific databases looking for computational methods to falls classification in their papers. We use the grounded theory methodology for a more detailed qualitative analysis of the papers. As a result of our search, we selected a total of 36 studies for our review and found two different computational methods for classifying falls. Related to the steps used in each method, we found fourteen different types of sensors, four different techniques for background and foreground extraction of videos, twenty-one techniques for feature extraction, and seven different fall classification strategies. Finally, we also identified fifty-one different types of falls. In conclusion, we believe that the methods and techniques analyzed in our study can help developers to create new and better systems for classification, detection, and prevention of falls and falls database. Besides, we identified gaps that can be explored in future research related to the automatic classification of falls.
Content may be subject to copyright.
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3132796, IEEE Access
Digital Object Identifier
Computational Solutions for Human
Falls Classification
EVILASIO C. JUNIOR1, ROSSANA M. C. ANDRADE1, LEONARDO S. ROCHA2, CARLA
TARAMASCO3AND LEONARDO FERREIRA.2
1Federal University of Ceará, Ceará, Brazil (e-mails: evilasiojunior@great.ufc.br, rossana@ufc.br)
2Ceará State University, Ceará, Brazil (e-mail: leonardo.sampaio@uece.br, leonardo.costa@aluno.uece.br)
3Universidad de Valparaíso, Valparaíso, Chile (e-mail: carla.taramasco@uv.cl)
Corresponding author: Evilasio C. Junior (e-mail: evilasiojunior@great.ufc.br). ORCID: https://orcid.org/0000-0002-0281-2964
This reasearch was funded by the Coordination of Improvement of Higher Level Personnel - Brazil (CAPES) that provided to Evilasio
Costa Junior a Ph.D. scholarship, and by the Conselho Nacional de Desenvolvimento Tecnológico - Brasil (CNPQ) that provided to
Rossana Maria de Castro Andrade a DT-2 Productivity Scholarship (Case No. 315543/2018-3). This research was also partially supported
by FONDECYT Regular 1201787 (“Multimodal Machine Learning approach for detecting pathological activity patterns in elderlies”).
ABSTRACT In the last two decades, studies about using technology for automatic detection of human fall
increased considerably. The automatic detection of falls allows for quicker aid that is key to increasing the
chances of treatment and mitigating the consequences of falls. However, each type of fall has its specificities,
and determining the correct type of fall can help treat the person who has fallen. Although it is essential to
use computational methods to classify falls, there are few studies about that in the literature, especially
compared to the studies that propose solutions for fall detection. In this sense, we execute a systematic
literature review (SLR) using the Kitchenham (2009) [1] method to investigate the computational solutions
used to classify the different types of falls. We performed a search on Scopus, Web of Science and PubMed
scientific databases looking for computational methods to falls classification in their papers. We use the
grounded theory methodology for a more detailed qualitative analysis of the papers. As a result of our
search, we selected a total of 36 studies for our review and found two different computational methods for
classifying falls. Related to the steps used in each method, we found fourteen different types of sensors,
four different techniques for background and foreground extraction of videos, twenty-one techniques for
feature extraction, and seven different fall classification strategies. Finally, we also identified fifty-one
different types of falls. In conclusion, we believe that the methods and techniques analyzed in our study can
help developers to create new and better systems for classification, detection, and prevention of falls and
falls database. Besides, we identified gaps that can be explored in future research related to the automatic
classification of falls.
INDEX TERMS Automated falls, Classification algorithms, e-Health, Falls, Falls classification, Types of
falls
I. INTRODUCTION
Falls are the main cause of morbidity, disability, and in-
creased utilization of health care among the older adults
[2] population. According to the World Health Organization
(WHO) [3], falls are the leading cause of serious injury in
the elderly, reaching as much as 28-35% of people over the
age of 65 and over 32-42% of people over 70 years of age.
Fall is defined as “an event in which a person inadvertently
comes to rest on the ground, floor, or lower-level” [4]. When
a fall occurs, it is crucial to immediately detect the situation,
because these accidents usually lead to more severe illness or
even death. Early detection of falls is essential for rescuing
injured people from danger and getting help as quickly as
possible [5]. For Mubashir and Shao (2013) [6], the demand
for surveillance systems, especially for fall detection, has
increased in the health sector with the rapid growth of the
older adult population in the world. It has become relevant
then to develop intelligent surveillance systems that can
automatically monitor and detect falls.
Several fall detection devices and fall risk assessment and
prevention systems have been developed to enable older
adults or those with chronic diseases to live safely and inde-
pendently at home. According to Abdelhedi et al. (2016) [7],
a fall detection system is one or more system that sends an
alert in response to a fall. A miniaturized fall detection device
seeks to improve the accuracy of fall detection, having a
2021 1
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3132796, IEEE Access
Evilasio C. Junior et al.: Computational Solutions for Human Falls Classification
minimal impact on the daily life of the user (e.g., apple watch
series 4). Moreover, a fall risk assessment system is one or
more systems capable of identifying the risk of a person
falling based on sensory data and well-defined measures [8]
[9].
Falls may be due to intrinsic causes (such as pre-existing
diseases) or extrinsic causes (such as slippery environments)
and may have specific characteristics that impact the reliabil-
ity of fall prevention and detection solutions [9]. Therefore,
works that seek to provide these computational solutions
usually classify or categorize types of falls according to the
characteristics observed about it, for example, the direction
of the fall, the place where the fall occurred, the speed of
the fall, the final position, or even the post-fall movement.
According to Mubashir and Shao (2013) [6], we should be
considering different scenarios when identifying different
types of falls: walking or standing falls, falls with supports
(e.g., stairs), falls during sleep or lying in bed, and falls when
sitting in a chair.
It is also interesting to note that some fall characteristics
also exist in daily actions, for example, a squat also demon-
strates a rapid downward movement. Moreover, each fall has
specificities that may be related to the profile of the person
[10] [11] and to the health status of the patient when the fall
occurred, for example, some falls may correlate with specific
diseases [12]. Besides, there are types of falls that are more
dangerous and deserve more attention [13]. For example,
falls to the sides may be more likely to cause fractures in
frail older adults [14] [15].
Thus, it is important to not only develop solutions for fall
prevention and detection but also to classify its types accord-
ing to characteristics observed for each fall. Using known
computational methods to classify human falls may be ad-
vantageous for developing better fall detection applications,
fall risk assessment systems, and fall prevention solutions
capable of identifying specificities and even possible causes
of falls, as in Makhlouf et al. (2018) [16]. These methods
should have steps and techniques for each of these steps well-
defined to allow replicability. These methods can also aid
in building fall databases to be used in experiments aimed
at new automatic fall detection and prevention solutions and
assist in the faster identification of better treatment for each
specific type of fall.
Therefore, we execute a Systematic Literature Review
(SLR) and find studies from 2006 to 2021 with methods for
the classification of human fall aided by computational tech-
nologies. Moreover, we analyze how these methods work. As
a result, we found thirty six studies that use fall classification
methods. Based on these studies, two different types of
methods with three or four activities are identified. These
methods have as main activities: Sensing, Background and
Foreground Extraction (exclusively for methods based on
Video Technologies), Feature Extraction, and Execution of
the Fall Classification Strategy. Also, we found three types
of technologies used by these studies and 51 different types
of falls covered by the selected studies. Each kind of fall is
related to an observed characteristic of each fall. Finally, we
find out open questions about fall classification not treated
by these studies as well as challenges that require further
research.
II. RESEARCH METHODOLOGY
We based our Systematic Literature Review (SLR) on the
method proposed by Brereton et al. (2007) [17] and Kitchen-
ham et al. (2009) [1]. This is the most used method for
developing SLRs in the software engineering area and has
three activities: Planning, Execution (or conducting), and
Presentation (or documentation). Each activity has a series of
specific tasks for the SLR development. Figure 1 illustrates
the process adopted in this study.
During the SLR planning, we define the research questions
and the search strategy, and generate the protocol that guides
the execution. This protocol is constructed and validated
interactively. In our case, we created several versions of this
protocol and submitted it to the evaluation of specialists
until obtaining the final version. This document contains
the general objective of the review, the search strategy, the
research questions, the papers’ eligibility criteria, the quality
assessment criteria of the selected literature, and the list of
data that we want to extract of the selected literature.
In the conducting phase, we execute the search strategy
and apply the eligibility criteria for selecting the papers. After
this, we verify the quality criteria of the selected studies and
extract and synthesize the data.
Finally, in the presentation phase, we generate the report
and discuss the results. This paper presents our report, and
it contains the results of the SLR and the discussion about
them. This work follows the model of the Preferred Reporting
Items for Systematic Reviews and Meta-Analyzes (PRISMA)
[19] that suggests the discussion of the results based on the
research questions.
A. PLANNING
This section presents the research questions, the search strat-
egy, the query string, and the eligibility criteria.
First, we specified four research questions for this SLR,
as follows:
1) What are the computational methods used to classify
falls?
2) What are the techniques used in each activity of these
methods?
3) What are the advantages of using fall classification
methods?
4) Which types of falls are classified by these methods?
We analyzed and discussed the answers to these questions
in Section IV.
The search strategy of this SLR consists of two phases. In
the first phase, we utilized a query string to search papers in
public scientific studies databases, and, in the second phase,
we performed a manual procedure, known as snowballing, to
analyze the citations (snowballing forward) and references
22021
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3132796, IEEE Access
Evilasio C. Junior et al.: Computational Solutions for Human Falls Classification
Systematic Review
Plan Review Conduct Review Document Review
Specify Research
Questions
Develop Review
Protocol
Validate Review
Protocol
Write Review Report
Systematic Review
Identify Relevant
Researches
Select Primary Studies
(applying selection
criteria)
Assess Study
Quality
Snowballing
(Backward and Foward
- 1st level)
Select Primary Studies
(applying selection
criteria) Validade Review
Systematic
Review Protocol
Extract Required
Data
Assess Study
Quality
Extract Required
Data
Synthesis
Primary
Studies Primary
Studies
FIGURE 1: Systematic Review Process (Adapted from Brereton (2007) [17] and Wohlin et al. (2014) [18]).
(snowballing backward) of the articles previously selected
in the first phase. Snowballing is used to complement the
search procedure in the public databases, making the litera-
ture search coverage more complete. These two initial phases
were executed from April to May 2018.
We chose the databases SCOPUS and Web of Science for
the first phase of the literature search. According to Archam-
bault et al. (2009) [20] and Aghaei et al. (2013) [21], which
are the most relevant search databases for Computer Science,
aggregating works of several other relevant databases for the
area of Computing and related.
In April 2021, we executed a new search phase. In this
phase, we made a new search on the Scopus database, con-
sidering articles after 2018, and we added a new database,
PubMed [22], a well-known literature database for research
in the medical literature. In the PubMed, we do not restrict
the search date.
For the generation of the query string, we used the PICO
approach that was created for systematic reviews in medical
research areas, but which is also widely used in Software
Engineering research [23] [19]. This method separates the
question into four aspects: Population of interest (Popula-
tion), Intervention, Comparison, and Outcome of interest.
The Population represents the types of studies we want to
address in the research. The Intervention corresponds to what
characteristic we want to find in studies on our Population.
The Comparison is related to the control group used in the
experiments carried out in our population studies. Finally, the
Outcome of interest corresponds to the information we want
to find in our population studies. Table 1 shows the elements
identified for each component of the PICO approach, accord-
ing to the research questions presented previously.
TABLE 1: Identified elements of the PICO approach.
Aspect Identified Element
Population Papers about human fall in the e-health research
area and related (e.g., Telemedicine)
Intervention Classification indicators
Comparison Not applied in this research
Outcome interest Techniques, Methods and Technologies
In general, systematic literature reviews in the Software
Engineering area are exploratory studies designed to charac-
terize a specific research line. In this case, these SLRs do
not use a control group and we do not use any term for
Comparison. However, some authors consider that the lack of
this item of the PICO approach is a quasi-systematic review
[24] [25].
We evaluated several query strings with the help of three
experts until we obtained the final version presented in
Textbox 1. These specialists also evaluated the protocol gen-
erated during the planning phase.
Textbox 1. Query String
2021 3
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3132796, IEEE Access
Evilasio C. Junior et al.: Computational Solutions for Human Falls Classification
(“Fall” OR “Falls” OR “Human Falling” OR “Falling Hu-
man” OR “Falls in*” OR “Accidental falls”) AND ("Smart
Health" OR "E-health" OR "Ambient Assisted Living"
OR "AAL" OR "Tele-healthcare" OR "Telemedicine" OR
“Healthcare”) AND (classifi* OR detect* OR identifi*
OR “recognition”) AND (“Technique” OR “Approach” OR
“Model” OR “Procedure” OR “Method” OR “Process” OR
“Technology”)
The papers resulting from our search had their bibli-
ographic references in .bibtex format extracted from the
databases. The data was then organized and stored as PDF
files by Mendeley1software, which was also used to manage
the execution of the selected activity.
For the selection of the most relevant studies, it is neces-
sary to define exclusion and inclusion criteria (called eligibil-
ity criteria) that can be replicated by other researchers [1].
In this SLR, the exclusion criteria operate in sequential order
similar to an Access Control List (ACL) as in Sanndhu and
Samarati (1994) [26]. Thus, when we found a match on the
list, we performed the exclusion action, and we did not check
any other criterion.
We defined the following exclusion criteria for this SLR:
Non-English papers (E1);
Non-articles, Non-conference papers, Non-book chap-
ters (E2);
Papers with less than five pages (short paper) (E3);
Secondary studies (e.g., literature review) (E4);
Papers that do not present the falls classification (E5);
and
Papers that do not use computational technology to
classification, detection or recognition of human falls
(E6).
We defined the following inclusion criterion for this SLR:
Studies with experiments that have more than one type
of fall (I1).
Studies with computational methods for falls classifica-
tion (I2).
B. CONDUCTING
In this phase, first, we executed a search with the query string
from April to May 2018 in databases of academic papers and
with the search filters referring to the exclusion criteria E1
and E2, which could be applied directly in the search engines
of the databases. We found 1163 articles for analysis and,
using the Mendeley tool, we identified 297 either duplicate
papers or we did not consider the papers because they did
not have a title, abstract, or author. From the remaining
866 articles, we excluded 817, according to the exclusion
criteria based on the dynamic reading of the papers, focusing
on the title, abstract, and the most relevant parts of these
papers.Then, from the 49 remaining papers, after evaluating
the first inclusion criterion, we select 45 papers.
1Mendeley - https://www.mendeley.com/
Following the Conducting phase steps, to correctly apply
the second inclusion criterion, a detailed reading of the arti-
cles was needed. However, to increase the research coverage,
we opted to use the 45 articles remaining from the application
of the exclusion criteria and the first inclusion criterion as the
source of the snowballing process. Just after the snowballing
process, we did the detailed reading of these papers and
evaluated the second inclusion criterion.
To apply the snowballing technique, we identified the
citations of the articles using Google Scholar, as suggested in
Wohlin et al. (2014) [18]. Altogether, we found 2819 papers
from citations of the 45 studies aforeselected and another
1249 papers from the references, totaling 4068 papers for
analysis. Using the Mendeley tool, we excluded 23 duplicate
articles. From the remaining 4045 studies, we excluded 4008
papers, according to the exclusion criteria based on the
dynamic reading of the papers, focusing on the title, abstract,
and the most relevant parts of these papers, obtaining 37
studies. From these, we selected 36 papers after the first
inclusion criterion assessment.
Finally, we read the 82 selected studies, and we found 30
articles that fulfill the second inclusion criterion.
In April 2021, we executed a new search in the academic
databases, including the PubMed Database, and we found a
new set of 1454 papers (552 from SCOPUS and 902 from
PubMED). Using the Mendeley tool, we identified 967 either
duplicate papers or we did not consider the papers because
they did not have a title, abstract, or author. We identified
that many studies found in PubMed had already been found
in the search performed until 2018 in the SCOPUS and Web
of Science databases. In PUbMed we did not use a time filter,
then, for this reason, we found a large number of duplicate
papers. From the remaining 487 articles, we excluded 474,
according to the exclusion criteria based on the dynamic
reading of the papers, focusing on the title, abstract, and the
most relevant parts of these papers. Finally, from the 13 left
we selected 6 papers after the first and the second inclusion
criterion evaluation.
To conclude the selection, we extracted data from the 36
selected articles (i.e., the 30 articles found in the literature
search carried out in 2018 and the other 6 articles added after
the complementary literature search carried out in 2021) and
assessed the quality of the papers. The quality assessment
was based on well-defined criteria, as suggested by Kitchen-
ham et al. (2009), [1]. Our goal is to evaluate the potential
of the selected studies to contribute to the answers to the
research questions. Then, for this SLR, we chose two quality
assessment criteria, that are:
A Level of detailing of the fall classification method from
the study; and
B Presence of different types of falls addressed in the
study results.
For our review, the data extraction and the quality assess-
ment were performed by two researchers who used an online
form generated in Google forms. The form containing the
42021
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3132796, IEEE Access
Evilasio C. Junior et al.: Computational Solutions for Human Falls Classification
TABLE 2: Quality assessment criteria scores.
# Criteria Score Weight
1
Level of details of the fall classification method from
the study
(+3) The paper presents a detailed method for falls classification.
2(+2) The paper has a method for falls classification, but does not detail it.
(+1) The paper use falls classification of another study.
2Presence of different types of falls
addressed in the study results
(+4) The paper evaluates all fall types separately.
1
(+3) The paper evaluates most types of falls separately.
(+2) The paper evaluates some fall types.
(+1) The paper evaluates only if there was a fall or not.
(+0) There is no evaluation in the study.
information to be extracted from each paper can be seen at
the link https://bityli.com/Y730w.
In Table 2, we show the scores for the answers of each
quality criterion specified for this SLR. The first criterion
indicates if the study presents a detailed fall classification
method, which is a set of replicable and sequential activities
that must be performed by the computational solution to
classify falls. This criterion is directly correlated to the first
and second research questions and has a higher weight in our
evaluation. The second criterion assesses if the evaluation
procedure results in each study consider the different types
of falls. By "different types of falls addressed in the study
results", we mean results of the studies (possibly from ex-
periments) that indicate not only that a fall has occurred but
also something that characterizes the fall. For example, the
direction of the fall (front, back, left or right), the place where
the fall occurred (kitchen, bathroom, living room), whether
the fall was due to a slide, whether the fall was slow or fast.
Figure 2 illustrates the distribution of the sum of the
quality assessment criteria values multiplied by their weights
for the 36 papers selected for this SLR.
FIGURE 2: Analysis of the quality assessment criteria
C. SYNTHESIS AND THE GROUNDED THEORY
We arranged the extracted data in a google sheet, and the data
were synthesized based on quantitative and qualitative ana-
lyzes to get at the results that we present in the next section.
For the qualitative analysis, we used the grounded theory
(GT) methodology [27]. According to Corbin and Strauss
(2008) [27], the GT is a specific methodology developed for
building theory from data, but the grounded theory can be
used in a more generic sense to denote theoretical constructs
derived from qualitative analysis of data.
FIGURE 3: Data analysis procedure of the grounded theory
method (adapted from Cho (2014) [28]).
In general, GT has the following steps: planning, data
collection, coding, and reporting [27]. In the planning step,
we identify the area of interest and the research question. In
our case, the area of interest is "Computational classification
of human falls" and the research question is: "What are the
computational methods used to classify falls? Furthermore,
how do these methods work?". After the planning step, we
did the data collection, which is necessary to answer the
research question. For our analysis, we used the data obtained
during the data extraction phase of the systematic review.
The coding step is the main stage of the GT. According
to Corbin and Strauss (2008), [27], in this step, we extract
concepts (codes) from the raw data and correlate them hier-
archically until we obtain a central concept (or code). In this
research, we would like to obtain and relate concepts that
characterize the methods used to classify falls. The coding
step involves three tasks: open, axial, and selective coding.
As presented in Figure 3, the coding step has two unique
characteristics: theoretical sampling and constant compara-
tive analysis [28]. Theoretical sampling is the step of col-
lecting data for comparative evaluation, which means insight
from initial data collection, and analysis leads to subsequent
data collection and analysis. Constant comparative is an
iterative activity of concurrent data collection and analysis.
The Results of the Coding phase are presented in Section III.
2021 5
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3132796, IEEE Access
Evilasio C. Junior et al.: Computational Solutions for Human Falls Classification
D. THREATS TO VALIDITY
This systematic literature review focused on identifying com-
putational solutions for the classification of human falls.
Therefore, it is possible to have studies in the medical lit-
erature about fall classification not selected by this review,
because they do not use computational technologies for clas-
sification. It would be then interesting for future work to
identify how the medical literature treats the classification of
falls and to use that to propose new computational methods.
It is also possible that there are relevant studies related
to this SLR that we could not find, because: (i) the study
sources are not indexed by the databases used in this review,
and (ii) the query string does not cover the studies that we
needed. However, to mitigate these threats, we used relevant
electronic databases [20] [21] similar to many systematic re-
search and reviews in the field covered by this SLR. Besides,
several attempts were made to construct the final version
of the query string. Moreover, we used the snowballing
strategy [18] to increase the coverage of articles and possible
inconsistency of the query string.
III. RESULTS
In this SLR, we selected 36 papers to answer the defined
research questions. These studies were published between
2006 and 2021. Table 3 shows the list of studies selected by
the type of hardware used in the studies.
TABLE 3: List of selected papers by technology.
Technology Paper selected
AAL [29] [30]
Video [31] [32] [33] [34] [35] [36] [37] [38]
[39] [40]
Wearable [41] [42] [43] [44] [45] [46] [47] [48] [49]
[50] [4] [51] [52] [53] [54] [55] [56] [57]
AAL and Wearable [58] [16]
Video and Wearable [59] [60] [61]
AAL, Video and Wearables [62]
A. FALL CLASSIFICATION METHODS
We use the codification process in the GT methodology to
analyze the fall classification methods and their techiniques.
Firstly, in open coding, we check the data to understand the
essence of "what is" expresses [27]. We inspect the data ex-
tracted from the papers using the extraction form, as done in
Carvalho et al. (2018) [63]. Then, a conceptual name (code)
is created to represent our understanding. Codes consist of
an entire word, phrase, or paragraph. Table 4 presents some
examples of codes. We use the QDA Miner Lite tool to aid
open coding, as done in [64].
We create 61 codes, which were divided into five cat-
egories: Sensors, Hardware limitations, Background and
Foreground Extraction (BFE) techniques, Feature extraction
techniques, and classification techniques. These categories
were extracted from the articles themselves while we refined
the codes. Table 5 presents the identified codes divided by
categories. To facilitate the analysis, we identified the types
of technology associated with each code.
TABLE 4: Examples of codes from open coding.
Code Text segments from the extracted data
Dynamic Time
Warping
"Dynamic Time Warping aligns two time series in
such a way that some distance measure is minimized
(usually the Euclidean distance is used). Optimal
alignment (minimum distance warp path) is obtained
by allowing the assignment of multiple successive
values of one time series to a single value of the other
time series...." [41]
"We also collect segmented data streams generated by
falls with various falling directions to build the
anchoring data streams for the later DTW distance
calculations. . . " [30]
Fall classification
strategy based on
thresholds
"The threshold was determined by considering
accelerations in SVM (Signal Magnitude Vector) and
in the x-, y-, and z-axes, whereas falls and stumbles
were simulated. . . " [46]
The sensors category contains the kind of hardware used
for the sensing of the raw data. The hardware limitation cat-
egory presents the hardware limitations related to the device
used to obtain the raw data. The BFE techniques category
comprises image preprocessing techniques to remove back-
ground and foreground to determine the form to be tracked
in the video, allowing feature extraction. These techniques
are exclusively related to video technologies. The feature
extraction techniques category contains the techniques used
to extract features from the raw data. Finally, the classifica-
tion techniques category contains the techniques used for fall
classification.
Next, we correlated the open coding categories with the
sequence of activities executed for falls classification in the
selected papers (Axial coding step). With this, we identified
that the fall classification solutions follow the method of
Figure 4a when using wearables or AAL sensors, and the
method of Figure 4b when using video sensors.
Figure 5 shows the representation of axial coding. It
presents the relationships between the code categories from
open coding and the activities of the fall classification meth-
ods. Lastly, according to Corbin and Strauss (2008), [27],
when all categories can be related to a core category, it
means the researcher is doing selective coding. Selective
coding is the final step of Grounded Theory and consists of
linking categories around a core category and refining the
resulting theoretical construction. This core category is the
"Falls Classification Methods" in our research, as shown in
the figure.
B. ACTIVITIES AND TECHNIQUES OF FALL
CLASSIFICATION METHODS
This section describes the activities of the fall classification
methods and the techniques used in the selected studies for
each activity.
The sensing activity involves obtaining and storing the raw
data that will be processed to generate the features. Associ-
ated with the sensing activity are the categories of sensors
and hardware limitations. Ambient assisted living (AAL)
environment sensors [16], [29], [30], [58], [62] obtain con-
62021
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3132796, IEEE Access
Evilasio C. Junior et al.: Computational Solutions for Human Falls Classification
TABLE 5: Codes identified in the open coding step.
Category Codes
Sensors
(AAL) Ambient Sensors; (AAL) Presence Sensors; (AAL) RFID Tags; (Video) ATC Video Sensors;(Video)
Infrared; (Video) Microsoft Kinect; (Video) Video Camera 2D and 3D; (Wearable) ECG; (Wearable) Smart IR
Tags; (Wearable) with accelerometer; (Wearable) with accelerometer and gyroscope; (Wearable) with accelerometer,
gyroscope and magnetometer; (Wearable) with accelerometer, gyroscope, magnetometer and altimeter; (Wearable)
with Accelerometer and Heart Rate Sensor.
Hardware limitations (Video) Human Posture Similarity; (Video) Occlusion; (Video) Video limited memory; (Wearable) Location of the
sensor; (Wearable) Poor processing power and limited battery.
BFE techniques Reference background; Gaussian mixture and weighted subtraction; Gaussian mixture; Window regression layer.
Feature extraction techniques
(Ambient Sensors) Gaussian-like probability density; (RFID Tags/Smart IR Tags) Dynamic Time Warping; (AAL)
Raw data from presence sensor; (ATC Video) Point-cloud compromised; (Microsoft Kinect) Region Proposal
Network and Fast Region-based Convolutional Network; (Microsoft Kinect) V-disparity; (Video 2D or 3D) α-β-
γfilter; (Video 2D or 3D) Bayesian Segmentation; (Video 2D or 3D) Change in the human shape; (Video 2D
or 3D) R-transform; (Video 2D or 3D) R-transform and Generalized discriminant analysis; (Video 2D or 3D)
R-transform and Principal component analysis and Independent component analysis; (Video) Raw data from 2D
or 3D video; (Wearable with Accelerometer and Gyroscope and Magnetometer and Altimeter) PDR algorithm;
(Wearable with Accelerometer) Discrete wavelet transform; (Wearable with Accelerometer) Median filter &Low
pass filter &Elliptical infinite impulse response filter; (Wearable) Raw data from accelerometer; (Wearable) Raw
data from accelerometer and gyroscope; (Wearable) Raw data from accelerometer, gyroscope and magnetometer;
(Wearable) Raw data from accelerometer and Heart Rate Sensor; (Wearable) Raw data from accelerometer,
gyroscope, magnetometer and altimeter.
Classification techniques
(AAL/Video/Wearable/Wearable and AAL/Wearable and Video) Based on thresholds; (Video/Wearable) Pattern
Recognition; (AAL/Video/Wearable) Based on thresholds and Pattern Recognition; (Video/Wearable) Based on
logic inferences; (Wearable and AAL) Based on a Specific Grammar-Feature-Based; (Wearable) Based on a Specific
Sequence of Classifiers; (Wearable) Multiple-Phases Features Pattern Recognition.
tinuous data from specific locations that vary when there is
movement within that space. The presence sensors are used
in conjunction with sensors of other types of technology and
fulfill the function of determining only the location of the
individual in a specific room within that AAL, while the other
AAL sensors obtain the data that will be used to determine
the type of movement, for example, the type of fall.
The video sensors [32]–[38], [40], [59]–[62], in general,
can be divided into four types of approaches, using video
2D, 3D, Infrared or based on the variation of luminosity or
colors. In all cases, the general idea is to identify a region of
interest of the video that contains the human body, and when
this region varies, we identify an occurrence of falls. Finally,
all wearable approaches [4], [16], [41]–[62], [65]–[69] uses
accelerometer to derive from the raw data that is used to iden-
tify and classify the fall. However, many of the works also
used other sensors like gyroscope, magnetometer, barometer,
which are used as an altimeter, ECG and even heart rate
sensors, used to identify the heart rate at the time of a fall.
We found some hardware limitations directly related to
the sensing of the approaches that use video or wearable.
The similarity between various human postures, the occlu-
sion caused by objects in front of the individual, and the
limited memory are the hardware limitations identified for
video approaches. Finally, the limited battery of the devices,
the low processing power, and the amount of storage of
the equipment are the most common restrictions for the
wearables. Besides, the location of the wearable in the body
also influences the measurement. Most papers that treats this
subject indicate that the results are best when the device is on
the chest or the waist of the person.
The BFE activity separates the region of interest from
the rest of the video. This activity is part of the video
preprocessing and later affects feature extraction. The BFE
techniques category is associated with this activity. Each BFE
technique represents the video as points with values that vary
among them. This variation may, for example, be obtained by
checking the variation of the pixel sets that delimit specific
regions of the image, as in the Gaussian mixing technique
used in [35], [36], [38].
The feature extraction activity involves features generation
from raw data or preprocessed data. These features will be
used to detect and classify falls. Each feature extraction
techniques category is associated with the feature extraction
activity. Each feature extraction technique combines raw or
preprocessed values to generate more representative (fea-
tures). For example, a feature extraction technique for a
solution using with accelerometer device can generate the
Signal Magnitude Vector (SMV) feature [16], [43], [46],
[47], [52]–[54], [69]. The SMV is generated by combining
the values obtained for each axis during an accelerometer
measurement and follows the formula:
SM V (ti) = qAx
2(ti) + Ay
2(ti) + Az
2(ti)(1)
Where tiindicates the measurement in time i, and Ax,Ay,
and Azare the accelerometer values from axis x,y, and z.
The SMV feature can be used to generate other features, like
standard deviation, or can be used alone by the classification
strategies. We found 67 different features, as presented in
Table 6, separated by the type of hardware. Note that some
features are associated with more than one kind of device.
In the last activity, the classification strategies are exe-
cuted, including the application of pattern recognition tech-
niques. Note in Figure 4 that the types of falls are inputs to
the activity, so they are predetermined.
We identify seven types of fall classification techniques.
The most common is the use of thresholds and, in these cases,
2021 7
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3132796, IEEE Access
Evilasio C. Junior et al.: Computational Solutions for Human Falls Classification
(a) Classification method for wearable and AAL approaches.
(b) Classification method for video approaches.
FIGURE 4: Fall classification methods.
FIGURE 5: Axial and selective coding representation.
characteristic values, known as thresholds, are defined for
certain phases of the movement of the fall. By exceeding
these thresholds, the fall can be identified and, more specifi-
cally, the type of fall.
These thresholds are drawn from previous studies or deter-
TABLE 6: Features used for fall classification strategy.
Type Features
AAL Mean value, Variance, Location, Euclidean distance.
Video
3D Silhouette, 2D Silhouette, ID position, Centroid, Color,
Binary map, 3D Depth image, Position, Speed, Acceleration
of the centre of gravity, Orientation, Length, Distance of
joint of human skeleton, Angle of joint of human skeleton,
Velocity of joint of human skeleton, Shape aspect ratio,
Height, Human motion velocity, Distance to object.
Wearable
Tilt angles, Signal magnitude vector, Acceleration
magnitude, Angular velocity, Direction, Euclidean norm,
Motion angle, Euclidean distance, Square root of the
square of the gyroscope values, Square root of the square
of the magnetometer values, Wavelet acceleration, Heart rate.
mined by applying a pattern recognition technique employed
to a training group. This training group consists of data
obtained from fall experiments, explicitly performed for a
study, or collected from public falls databases.
Another type of fall classification technique usually found
in the papers are pattern recognition algorithms, in one or
multiple phases [50], to classify falls based on a training
82021
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3132796, IEEE Access
Evilasio C. Junior et al.: Computational Solutions for Human Falls Classification
FIGURE 6: Recognition pattern algorithms.
set. With the algorithm trained, when a new fall occurs, this
event is classified according to the class whose values of the
features more closely resembles. Some approaches use both
thresholds and pattern recognition algorithms to detect and
classify falls, rather than pattern recognition algorithms used
only to identify thresholds.
Figure 6 presents the pattern recognition algorithms and
how many of the studies selected uses each algorithm. It is
worth mentioning that some studies contain more than one of
these algorithms. We can see that Artificial Neural Network
(ANN), k-Nearest Neighbors (KNN) and Support Vector
Machines (SVM) are the most used algorithms. We believe
this happens because they can sort data quickly and produce
better results than other algorithms. However, the average
training time of these algorithms is higher than others, like
tree-based algorithms. It is worth noting that there was a
similar prevalence of ANN, SVM, and KNN algorithms in
wearable-based and video-based systems studies. However,
most of the other algorithms were used by the studies from
video-based systems.
The studies [47] [40] and [62] use a set of rules of fuzzy
inferences to detect and classify falls. They apply inference
rules according to the value assumed by the features. This
strategy is similar to the use of thresholds, but, in their case,
some sets of values are related to the occurrence of the same
type of fall, depending on the rules of inference formulated.
In short, we observed that the thresholds strategy is more
common in systems that use wearable sensors and smart-
phones to obtain data. In contrast, there is a prevalence of
strategies based on logical inferences and pattern recognition
algorithms in video-based systems.
Some studies also use specific strategies to detect falls. Li
(2011) [58] proposes a specific grammar based on features.
This approach detects a particular type of fall by combining
the grammar elements in some ways. In He and Li (2013)
[54], classifiers are generated based on features extracted
from wearable data, which, when combined in specific se-
quences, correspond to particular types of falls.
C. TYPES OF FALLS
In our systematic review, we identified a total of 51 different
types of falls. According to Yu (2008) [70], falls are related
to movement performed and position, and are divided into
four major categories: falls from standing, falls from sitting,
falls from lying, and falls from standing on a support (e.g.,
a ladder). However, we found other categories of types of
falls in Makhlouf et al. (2018) [16], which classifies falls
into three different types of cardiac problems (bradycardia,
tachycardia, and cardiac arrest), and according to where they
occurred (e.g., bathroom, kitchen, room, living room). In
addition, Saha et al. (2018) [57] and Gulati and Kaur (2021)
[62] show falls related to cardiac and respiratory problems.
Therefore, we decided to categorize the types of falls into
four categories: falls related to health issues, location, the
position of the person, and the kind of motion. Figure 7 shows
the types of falls for the category Kind of Motion and Figure
8 presents the types of falls for another three categories. The
number next to each type of fall in the figure informs the
number of articles in which the type of fall was mentioned.
The categories kind of motion and the position include the
same types of falls presents by Yu (2008) [70], but they have
more examples of falls that use elements related to the move-
ment performed (direction of fall, rotation, speed, severity)
and the position before or after the fall. Finally, it is worth
noting that the most used falls in the studied literature are
related to the direction of movement (Forward, Backward,
Leftward, and Rightward), as can be seen in Figure 7.
D. PROFILE OF THE EXPERIMENT PARTICIPANTS
In general, to evaluate the proposed approaches for the clas-
sification of falls, the studies use falls from databases or
experiments generated by each research. Most of these papers
present a profile of the experiment participants and, with this,
it is possible to get more information about the approaches.
We identified that 19 of the articles present quantity and some
profile of the participants.
The papers [49] and [46] use falls or daily activities from
adults over 60 years old, the main risk group. The others
use experiments with adults, men, and women, between 19
and 57 years old, with most participants between 20 and 30
years old. Some of these authors (e.g., [49] [38] [52]) admit
that there could be variations when they use their proposals
with older adults, but, according to Karantonis et al. (2006)
[46], experiments without the presence of older adults do not
make the proposal unfeasible. Moreover, several studies have
also identified the participants’ height, weight, or body mass
index. According to these studies, these characteristics may
influence the measurements of the sensors, but they do not
show examples of how these characteristics affect the results.
IV. DISCUSSION
In this section, we discussed the SLR results and identified
research gaps and challenges. This SLR aims to discover
studies that present classifications of human falls supported
by computational methods and how and why these studies
2021 9
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3132796, IEEE Access
Evilasio C. Junior et al.: Computational Solutions for Human Falls Classification
FIGURE 7: Types of falls identified for the category: Kind of Motion.
use them. In this way, we found 36 studies that have a method
to classify falls. In general, to evaluate such fall classification
methods, the authors used experiments with data of different
types of falls performed. Table 7 presents a summary of the
answers to each Research Questions (RQs).
As shown in Table 7, we have identified two different types
of computational methods used by the studies to classify
falls, which differ mainly by the sensing technology used.
We also identify techniques used in each activity of these
methods. However, most of these methods are used only to
improve the accuracy and precision of fall detection systems
or systems to identify fall risk. However, they do not seek
to identify the severity of these falls, thus prioritizing falls
considered the most dangerous in the medical literature, such
as lateral falls [14] [15].
Makhlouf et al. (2018) [16], Saha et al. (2018) [57] and
Gulati and Kaur (2021) [62] are the exceptions that use fall
types associated with diseases. There are still few studies
that associate falls with specific health problems using com-
putational technologies. In this sense, we believe that this
type of relationship between falls and other health issues is
a challenge that can be explored in future research.
As we mentioned before, these studies classified the types
of falls in two categories based on the type of movement
or based on the person’s position before and after the fall.
However, most of them do not clarify why these are the
categories that should be considered. We believe that, to build
relevant databases, it is important to understand the nature
of the data and categorize it. Thus, another challenge that
could be explored in future works should be to understand
what makes the categories of the types of falls used in the
literature relevant and if other relevant characteristics allow a
better categorization of falls. In this sense, an exciting gap to
be explored in future research is to identify, together with the
literature of the health area and health professionals, if the
types of falls presented by the works selected in this SLR are
relevant to determine the severity of the fall event.
Moreover, the proposal of a classification method using
sensor data obtained from fall events to identify new types
of falls, for example, using grouping techniques such as
clustering, could generate interesting future research. Some
studies selected for this SLR utilize clustering techniques
(e.g., the k-means algorithm), but these techniques were used
to classify the falls according to the predefined types of falls.
Finally, only Ponce and Martínez-Villaseñor (2020) [60]
take into account how the falls database used is classified.
We believe that is advantageous to use classification methods
in existing falls datasets to classify them or assist the creation
of new fall databases.
10 2021
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3132796, IEEE Access
Evilasio C. Junior et al.: Computational Solutions for Human Falls Classification
FIGURE 8: Types of falls identified for the categories: Health Issues, Location and Position.
TABLE 7: Summary of the Answers to the RQs
Answers to the Research Questions
Research Question 1. What are the computational methods used to
classify falls?
We identify two different methods used for fall classification. We found
a method with three steps for solutions using AAL sensors or wearables:
sensing, feature extraction, and execution of the classification strategy.
Yet, for video approaches, the same activities are executed, but
before the feature extraction there is one more activity: Background and
Foreground Extraction.
Research Question 2. What are the techniques used in each activity of
these methods?
This SLR identified four different techniques for background and
foreground extraction of videos, twenty-one techniques for feature
extraction, and seven different fall classification strategies. Also, we
identify fourteen different types of sensors used by the selected studies,
and five hardware limitations. The list of techniques is presented in
Table 6 and detailed in the results section of this study.
Research Question 3. What are the advantages of using fall classification
methods?
The studies intend to improve the precision and accuracy of systems,
applications, or approaches of automatic detection of falls and recognize
falls risk. The authors argue that different types of falls may behave
considerably different from the data, and classifying each type of fall or
group of types of falls allows greater accuracy in detecting falls. Besides,
Makhlouf et al. (2018) [16], Saha et al. (2018) [57] and Gulati and
Kaur (2021) [62] explore the advantages that identifying the type of fall
can have in the best treatment of the patient.
Research Question 4. Which types of falls are classified by these
methods?
All the way, we identified 51 types of falls. According to the literature,
it is possible to categorize the types of falls as: falls related to the type
of movement and falls related to the person’s position before and after
the fall movement. However, in our research, we found some works
presenting types of falls that do not match into these categories.
Therefore, we divide these types of falls into two other categories: falls
location and falls related to health issues (See Figures 8 and 7).
V. CONCLUSION
Different types of falls can directly influence the quality and
accuracy of fall detection and fall risk identification systems.
Fall classification allows identifying particular problems and
risks of specific types of falls. Furthermore, according to the
medical literature, there is an inherent severity of each type
of fall that is also important to consider. The detection and
classification of falls can be done automatically using com-
puter devices equipped with sensors capable of monitoring
the movement of patients. Using a computational approach is
mainly due to the agility in identifying the fall and the risks
inherent to the type of fall that the person suffered. So, the
systematic literature review presented in this paper aimed to
find automatic methods of fall classification in the literature
as well as gaps for future research.
We utilized a two-step search strategy: a search using three
academic article databases, and a snowball strategy on the
selected papers after searching the databases. Then, we found
several computational fall classification solutions that, as we
concluded, followed these two strategies. The differences
between them are the sensors and activities employed. The
first method is three-step, which is executed by wearables and
AAl approaches with the following activities: sensing, fea-
ture extraction, and falls classification strategy. The second
method is four-step, which is executed by Video solutions
with the same activities of the previous method plus a BFE
activity. Besides, in this SLR, we also organized the types of
falls found in the selected studies.
Finally, as one of the results of this study, we identified
challenges and open questions in the SLR selected papers
that can be addressed in future work, which are summarized
as follows: (i) comparison of the techniques applied in each
step of the methods and generation of a catalog to assist
the development of new hardware and software solutions
to falls detection and classification; (ii) a new approach for
classifying falls that addresses the types of falls categorized
2021 11
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3132796, IEEE Access
Evilasio C. Junior et al.: Computational Solutions for Human Falls Classification
in the medical literature and their inherent severity; and
(iii) development of a solution, considering the methods and
techniques identified in this study, to help classify and build
new falls databases.
REFERENCES
[1] B. Kitchenham, O. P. Brereton, D. Budgen, M. Turner, J. Bailey, and
S. Linkman, “Systematic literature reviews in software engineering–a sys-
tematic literature review,” Information and software technology, vol. 51,
no. 1, pp. 7–15, 2009.
[2] M. E. Tinetti, “Prevention of falls and fall injuries in elderly persons: a
research agenda.” Preventive medicine, vol. 23, no. 5, pp. 756–762, 1994.
[3] W. H. Organization, W. H. O. Ageing, and L. C. Unit, WHO global report
on falls prevention in older age. World Health Organization, 2008.
[4] C.-Y. Hsieh, K.-C. Liu, C.-N. Huang, W.-C. Chu, and C.-T. Chan, “Novel
hierarchical fall detection algorithm using a multiphase fall model,” Sen-
sors, vol. 17, no. 2, p. 307, 2017.
[5] A.-y. Jeon, S.-y. Ye, J.-m. Park, K.-n. Kim, J.-h. Kim, D.-k. Jung, G.-r.
Jeon, and J.-h. Ro, “Emergency detection system using pda based on self-
response algorithm,” in 2007 International Conference on Convergence
Information Technology (ICCIT 2007). IEEE, 2007, pp. 1207–1212.
[6] M. Mubashir, L. Shao, and L. Seed, “A survey on fall detection: Principles
and approaches,” Neurocomputing, vol. 100, pp. 144–152, 2013.
[7] S. Abdelhedi, R. Bourguiba, J. Mouine, M. Baklouti, and A. Youssef,
“Fall detection fpga-based systems: A survey,” International Journal of
Automation and Smart Technology, vol. 6, no. 4, pp. 191–202, 2016.
[8] K. L. Perell, A. Nelson, R. L. Goldman, S. L. Luther, N. Prieto-Lewis,
and L. Z. Rubenstein, “Fall risk assessment measures: an analytic review,”
The Journals of Gerontology Series A: Biological Sciences and Medical
Sciences, vol. 56, no. 12, pp. M761–M766, 2001.
[9] C.-Y. Hsieh, W.-T. Shi, H.-Y. Huang, K.-C. Liu, S. J. Hsu, and C.-T.
Chan, “Machine learning-based fall characteristics monitoring system for
strategic plan of falls prevention,” in 2018 IEEE International Conference
on Applied System Invention (ICASI). IEEE, 2018, pp. 818–821.
[10] T. O’neill, J. Varlow, A. Silman, J. Reeve, D. Reid, C. Todd, and A. Woolf,
“Age and sex influences on fall characteristics.” Annals of the rheumatic
diseases, vol. 53, no. 11, p. 773, 1994.
[11] L. J. Lattimer, J. L. Lanovaz, J. P. Farthing, S. Madill, S. Y. Kim,
S. Robinovitch, and C. M. Arnold, “Biomechanical and physiological age
differences in a simulated forward fall on outstretched hands in women,
Clinical biomechanics, vol. 52, pp. 102–108, 2018.
[12] S. Buksman, A. Vilela, S. Pereira, V. Lino, and V. Santos, “Quedas em
idosos: prevenção,Brazil: Sociedade Brazileira de Geriatria e Geron-
tologia, 2008.
[13] A. A. Ellis and R. B. Trent, “Do the risks and consequences of hospitalized
fall injuries among older adults in california vary by type of fall?”
The Journals of Gerontology Series A: Biological Sciences and Medical
Sciences, vol. 56, no. 11, pp. M686–M692, 2001.
[14] S. L. Greenspan, E. R. Myers, D. P. Kiel, R. A. Parker, W. C. Hayes,
and N. M. Resnick, “Fall direction, bone mineral density, and function:
risk factors for hip fracture in frail nursing home elderly,The American
journal of medicine, vol. 104, no. 6, pp. 539–545, 1998.
[15] S. K. Gratza, P. O. Chocano-Bedoya, E. Orav, M. Fischbacher, G. Freystät-
ter, R. Theiler, A. Egli, R. Kressig, J. A. Kanis, and H. A. Bischoff-Ferrari,
“Influence of fall environment and fall direction on risk of injury among
pre-frail and frail adults,” Osteoporosis international, vol. 30, no. 11, pp.
2205–2215, 2019.
[16] A. Makhlouf, I. Boudouane, N. Saadia, and A. R. Cherif, “Ambient
assistance service for fall and heart problem detection,” Journal of Ambient
Intelligence and Humanized Computing, pp. 1–20, 2018.
[17] P. Brereton, B. A. Kitchenham, D. Budgen, M. Turner, and M. Khalil,
“Lessons from applying the systematic literature review process within the
software engineering domain,” Journal of systems and software, vol. 80,
no. 4, pp. 571–583, 2007.
[18] C. Wohlin, “Guidelines for snowballing in systematic literature studies and
a replication in software engineering,” in Proceedings of the 18th interna-
tional conference on evaluation and assessment in software engineering.
ACM, 2014, p. 38.
[19] D. Moher, A. Liberati, J. Tetzlaff, and D. G. Altman, “Preferred reporting
items for systematic reviews and meta-analyses: the prisma statement,
Annals of internal medicine, vol. 151, no. 4, pp. 264–269, 2009.
[20] É. Archambault, D. Campbell, Y. Gingras, and V. Larivière, “Comparing
bibliometric statistics obtained from the web of science and scopus,”
Journal of the American society for information science and technology,
vol. 60, no. 7, pp. 1320–1326, 2009.
[21] A. Aghaei Chadegani, H. Salehi, M. Yunus, H. Farhadi, M. Fooladi,
M. Farhadi, and N. Ale Ebrahim, “A comparison between two main
academic literature collections: Web of science and scopus databases,
Asian Social Science, vol. 9, no. 5, pp. 18–26, 2013.
[22] K. Canese and S. Weis, “Pubmed: the bibliographic database,” in The
NCBI Handbook [Internet]. 2nd edition. National Center for Biotech-
nology Information (US), 2013.
[23] P. Stone, “Popping the (pico) question in research and evidence-based
practice,” Nurs Res, vol. 15, no. 3, pp. 197–198, 2002.
[24] I. de Sousa Santos, R. M. de Castro Andrade, L. S. Rocha, S. Matalonga,
K. M. de Oliveira, and G. H. Travassos, “Test case design for context-
aware applications: Are we there yet?” Information and Software Technol-
ogy, vol. 88, pp. 1–16, 2017.
[25] G. H. Travassos, P. S. M. dos Santos, P. G. Mian, A. C. D. Neto, and J. Biol-
chini, “An environment to support large scale experimentation in software
engineering,” in 13th IEEE International Conference on Engineering of
(ICECCS). IEEE, 2008, pp. 193–202.
[26] R. S. Sandhu and P. Samarati, “Access control: principle and practice,”
IEEE communications magazine, vol. 32, no. 9, pp. 40–48, 1994.
[27] J. Corbin and A. Strauss, “Techniques and procedures for developing
grounded theory,Basics of Qualitative Research, 3rd ed.; Sage: Thousand
Oaks, CA, USA, p. ., 2008.
[28] J. Y. Cho and E.-H. Lee, “Reducing confusion about grounded theory and
qualitative content analysis: Similarities and differences,The qualitative
report, vol. 19, no. 32, pp. 1–20, 2014.
[29] L. Meng, C. Miao, and C. Leung, “Towards online and personalized daily
activity recognition, habit modeling, and anomaly detection for the solitary
elderly through unobtrusive sensing,Multimedia Tools and Applications,
vol. 76, no. 8, pp. 10 779–10 799, 2017.
[30] W. Ruan, L. Yao, Q. Z. Sheng, N. Falkner, X. Li, and T. Gu, “Tagfall:
Towards unobstructive fine-grained fall detection based on uhf passive rfid
tags,” in proceedings of the 12th EAI International Conference on Mobile
and Ubiquitous Systems. ICST (Institute for Computer Sciences, Social-
Informatics and Telecommunications Engineering), 2015, pp. 140–149.
[31] H. Foroughi, H. R. Pourreza et al., “Intelligent video surveillance for
monitoring fall detection of elderly in home environments,” in 11th Int.
Conf. on Computer and Information Technology, 2008.
[32] Á. M. Srp and F. Vajda, “Fall detection for independently living older
people utilizing machine learning,” IFAC Proceedings Volumes, vol. 45,
no. 18, pp. 79–84, 2012.
[33] T.-L. Le, J. Morel et al., “An analysis on human fall detection using
skeleton from microsoft kinect,” in Fifth International Conference on
Communications and Electronics (ICCE). IEEE, 2014, pp. 484–489.
[34] W. Min, H. Cui, H. Rao, Z. Li, and L. Yao, “Detection of human falls
on furniture using scene analysis based on deep learning and activity
characteristics,” IEEE Access, vol. 6, pp. 9324–9335, 2018.
[35] A. Leone, G. Diraco, and P. Siciliano, “Detecting falls with 3d range
camera in ambient assisted living applications: A preliminary study,
Medical engineering & physics, vol. 33, no. 6, pp. 770–781, 2011.
[36] Z. A. Khan and W. Sohn, “A model for abnormal activity recognition
and alert generation system for elderly care by hidden conditional random
fields using r-transform and generalized discriminant analysis features,”
Telemedicine and e-Health, vol. 18, no. 8, pp. 641–647, 2012.
[37] Y.-S. Lee and W.-Y. Chung, “Visual sensor based abnormal event detection
with moving shadow removal in home healthcare applications,Sensors,
vol. 12, no. 1, pp. 573–584, 2012.
[38] Z. A. Khan and W. Sohn, “Abnormal human activity recognition system
based on r-transform and independent component features for elderly
healthcare,” Journal of the Chinese Institute of Engineers, vol. 36, no. 4,
pp. 441–451, 2013.
[39] ——, “A hierarchical abnormal human activity recognition system based
on r-transform and kernel discriminant analysis for elderly health care,
Computing, vol. 95, no. 2, pp. 109–127, 2013.
[40] M. V. Sokolova and A. Fernández-Caballero, “Fuzzy sets for human
fall pattern recognition,” in Mexican Conference on Pattern Recognition.
Springer, 2012, pp. 117–126.
[41] B. Pogorelc and M. Gams, “Recognition of patterns of health problems
and falls in the elderly using data mining,” in Iberoamerican Congress on
Pattern Recognition. Springer, 2012, pp. 463–471.
12 2021
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3132796, IEEE Access
Evilasio C. Junior et al.: Computational Solutions for Human Falls Classification
[42] S. Bulotta, H. Mahmoud, F. Masulli, E. Palummeri, and S. Rovetta, “Fall
detection using an ensemble of learning machines,” in Neural Nets and
Surroundings. Springer, 2013, pp. 81–90.
[43] D. Genoud, V. Cuendet, and J. Torrent, “Soft fall detection using machine
learning in wearable devices,” in 30th International Conference on Ad-
vanced Information Networking and Applications (AINA). IEEE, 2016,
pp. 501–505.
[44] B. Andò, S. Baglio, C. O. Lombardo, and V. Marletta, “A multisensor
data-fusion approach for adl and fall classification,” IEEE Transactions on
Instrumentation and Measurement, vol. 65, no. 9, pp. 1960–1967, 2016.
[45] R.-J. Dzeng, Y.-C. Fang, and I.-C. Chen, “A feasibility study of using
smartphone built-in accelerometers to detect fall portents,Automation in
Construction, vol. 38, pp. 74–86, 2014.
[46] D. M. Karantonis, M. R. Narayanan, M. Mathie, N. H. Lovell, and B. G.
Celler, “Implementation of a real-time human movement classifier using a
triaxial accelerometer for ambulatory monitoring,” IEEE transactions on
information technology in biomedicine, vol. 10, no. 1, pp. 156–167, 2006.
[47] W. Putchana, S. Chivapreecha, and T. Limpiti, “Wireless intelligent fall
detection and movement classification using fuzzy logic,” in Biomedical
Engineering International Conference (BMEiCON). IEEE, 2012, pp. 1–
5.
[48] K. Ganapathy, V. Vaidehi, and D. Poorani, “Sensor based efficient decision
making framework for remote healthcare,Journal of Ambient Intelligence
and Smart Environments, vol. 7, no. 4, pp. 461–481, 2015.
[49] M. M. Baig, H. Gholamhosseini, and M. J. Connolly, “Falls risk assess-
ment for hospitalised older adults: a combination of motion data and vital
signs,” Aging clinical and experimental research, vol. 28, no. 6, pp. 1159–
1168, 2016.
[50] R. M. Gibson, A. Amira, N. Ramzan, P. Casaseca-de-la Higuera, and
Z. Pervez, “Multiple comparator classifier framework for accelerometer-
based fall detection and diagnostic,” Applied Soft Computing, vol. 39, pp.
94–103, 2016.
[51] C. Zhang, C.-F. Lai, Y.-H. Lai, Z.-W. Wu, and H.-C. Chao, “An inferential
real-time falling posture reconstruction for internet of healthcare things,”
Journal of Network and Computer Applications, vol. 89, pp. 86–95, 2017.
[52] G. Zhao, Z. Mei, D. Liang, K. Ivanov, Y. Guo, Y. Wang, and L. Wang,
“Exploration and implementation of a pre-impact fall recognition method
based on an inertial body sensor network,” Sensors, vol. 12, no. 11, pp.
15 338–15 355, 2012.
[53] D. Dziak, B. Jachimczyk, and W. J. Kulesza, “Iot-based information
system for healthcare application: design methodology approach,” Applied
Sciences, vol. 7, no. 6, p. 596, 2017.
[54] Y. He and Y. Li, “Physical activity recognition utilizing the built-in
kinematic sensors of a smartphone,” International Journal of Distributed
Sensor Networks, vol. 9, no. 4, p. 481580, 2013.
[55] T. Aktay and M. Ö. Efe, “A comparative study of ann tuning methods for
multiclass daily activity and fall recognition,” in Mediterranean Confer-
ence on Pattern Recognition and Artificial Intelligence. Springer, 2019,
pp. 24–38.
[56] D. Ajerla, S. Mahfuz, and F. Zulkernine, “A real-time patient monitoring
framework for fall detection,Wireless Communications and Mobile Com-
puting, vol. 2019, 2019.
[57] S. S. Saha, S. Rahman, M. J. Rasna, T. B. Zahid, A. M. Islam, and M. A. R.
Ahad, “Feature extraction, performance analysis and system design using
the du mobility dataset,” IEEE Access, vol. 6, pp. 44776–44 786, 2018.
[58] Q. Li and J. A. Stankovic, “Grammar-based, posture-and context-cognitive
detection for falls with different activity levels,” in Proceedings of the 2nd
Conference on Wireless Health. ACM, 2011, p. 6.
[59] G. Diraco, A. Leone, P. Siciliano, M. Grassi, and P. Malcovati, “A multi-
sensor system for fall detection in ambient assisted living contexts.” in
SENSORNETS, 2012, pp. 213–219.
[60] H. Ponce and L. Martínez-Villaseñor, “Approaching fall classification us-
ing the up-fall detection dataset: Analysis and results from an international
competition,” in Challenges and Trends in Multimodal Fall Detection for
Healthcare. Springer, 2020, pp. 121–133.
[61] R. Espinosa, H. Ponce, S. Gutiérrez, L. Martínez-Villaseñor, J. Brieva, and
E. Moya-Albor, “A vision-based approach for fall detection using multiple
cameras and convolutional neural networks: A case study using the up-
fall detection dataset,” Computers in biology and medicine, vol. 115, p.
103520, 2019.
[62] N. Gulati and P. D. Kaur, “An argumentation enabled decision making
approach for fall activity recognition in social iot based ambient assisted
living systems,” Future Generation Computer Systems, vol. 122, pp. 82–
97, 2021.
[63] R. Carvalho, R. Andrade, K. Oliveira, and C. Kolski, “Catalog of invisibil-
ity requirements for ubicomp and iot applications,” in 26th International
Requirements Engineering Conference (RE). IEEE, 2018, pp. 88–99.
[64] R. C. Motta, “On context-aware software systems and interoperability: A
discussion grounded in data,” Ph.D. dissertation, Universidade Federal do
Rio de Janeiro, 2016.
[65] M. Prado-Velasco, R. O. Marín, and G. del Rio Cidoncha, “Detection
of human impacts by an adaptive energy-based anisotropic algorithm,
International journal of environmental research and public health, vol. 10,
no. 10, pp. 4767–4789, 2013.
[66] P. Pierleoni, A. Belli, L. Palma, M. Pellegrini, L. Pernini, and S. Valenti, “A
high reliability wearable device for elderly fall detection,IEEE Sensors
Journal, vol. 15, no. 8, pp. 4544–4553, 2015.
[67] A. Mao, X. Ma, Y. He, and J. Luo, “Highly portable, sensor-based system
for human fall monitoring,” Sensors, vol. 17, no. 9, p. 2096, 2017.
[68] P. Pierleoni, A. Belli, L. Maurizi, L. Palma, L. Pernini, M. Paniccia, and
S. Valenti, “A wearable fall detector for elderly people based on ahrs and
barometric sensor,IEEE Sensors Journal, vol. 16, no. 17, pp. 6733–6744,
2016.
[69] J. Wang, Z. Zhang, B. Li, S. Lee, and R. Sherratt, “An enhanced fall
detection system for elderly person monitoring using consumer home
networks,” IEEE transactions on consumer electronics, vol. 60, no. 1, pp.
23–29, 2014.
[70] X. Yu, “Approaches and principles of fall detection for elderly and patient,”
in HealthCom 2008-10th International Conference on e-health Network-
ing, Applications and Services. IEEE, 2008, pp. 42–47.
2021 13
Preprint
Full-text available
I have received 1,527 citations from 1,430 documents to 48 publications in the SCOPUS database up to the end of the year 2022 (The list extracted from SCOPUS on January 5th, 2023). I know it is not a big number compared to others in the field, but it is a small milestone in my academic career. I am grateful because I could have never dreamed of being where I am today. Thanks to all my co-authors who have published with me over the years. It was great writing articles with you as an attempt to disseminate knowledge. Thank you to all respective researchers, scientists, and individuals for recognizing our research team's work. Once more, I would like to say huge thanks to all my fantastic co-authors and all of you who read and cited our papers. Thank you very much for your recognition, encouragement, and support. Your citations will provide more evidence and inspire more researchers in the field to continue conducting more research. The citation number can be considered as one of the indicators for the impacts of our studies on the academic community and I am happy to see the impacts. It has been a challenging, but rewarding, journey with its ups & its downs, which I have loved overall. Once again, I am thankful to all of those who have supported me on my academic journey. It is indeed a special milestone that I will never forget. I hope the new year brings more exciting research and more collaboration between researchers in the field.
Chapter
Full-text available
Smart phones and other sensor-enabled devices are very frequently used daily life devices. Movement data obtained by sensors from these devices can be interpreted by artificial intelligence algorithms and this may be critically helpful in some daily life issues. Such a daily activity and fall classification mechanism is particularly important for rapid and accurate medical intervention to the elderly people who live alone. In addition, the real time human activity recognition (HAR) is important for healthcare solutions and better assistance of intelligent personal assistants (IPAs). In this study, the dataset is obtained from 6 different wearable sensors. It contains 20 daily activities and 16 fall motions on the 3060 observations. To classify these movements separately, 3 different Artificial Neural Network (ANN) training algorithms were chosen as the basis. These are gradient descent, momentum with gradient descent and Adam algorithms. Dropout and L2 regularization techniques are used to obtain better results for the test data. The results have shown that the ANN based approach correctly recognizes the daily activities and falls with 94.58% accuracy score on the test set.
Article
Full-text available
Fall detection is a major problem in the healthcare department. Elderly people are more prone to fall than others. There are more than 50% of injury-related hospitalizations in people aged over 65. Commercial fall detection devices are expensive and charge a monthly fee for their services. A more affordable and adaptable system is necessary for retirement homes and clinics to build a smart city powered by IoT and artificial intelligence. An effective fall detection system would detect a fall and send an alarm to the appropriate authorities. We propose a framework that uses edge computing where instead of sending data to the cloud, wearable devices send data to a nearby edge device like a laptop or mobile device for real-time analysis. We use cheap wearable sensor devices from MbientLab, an open source streaming engine called Apache Flink for streaming data analytics, and a long short-term memory (LSTM) network model for fall classification. The model is trained using a published dataset called “MobiAct.” Using the trained model, we analyse optimal sampling rates, sensor placement, and multistream data correction. Our edge computing framework can perform real-time streaming data analytics to detect falls with an accuracy of 95.8%.
Article
Full-text available
The University of Dhaka mobility data set (DU-MD) is a human action recognition (HAR) data set consisting of 10 classes and 5000 observations from 50 subjects recorded using wrist-mounted sensors embracing accelerometry. The data set exhibits sufficient statistical diversity in physiological parameters and a noteworthy correlation between similar activities with coveted quantitative and qualitative features, suitable for training machine learning models. On the other hand, the wrist-mounted approach parallels the future commercial scenarios. In this paper, we explore how the quantitative features of the DU-MD have been extracted and selected. Existing machine learning models used in HAR, in particular, support vector machines, ensemble of classifi ers, and subspace K-nearest neighbours have been applied to our data set for activity and fall classi cation, with outcomes being compared with benchmark and similar data sets. With a HAR classi cation accuracy of 93%, fall detection accuracy of 97% and fall classi cation of 68.3%, quantitative performance metrics have either approached or outperformed other data sets, making this dataset suitable for application in hardware-independent healthcare monitoring systems. Finally, we construct an algorithm with our data set based on performance metrics, and suggest some strategies for large-scale commercial implementation.
Article
Full-text available
Continuous monitoring of vital signs and activity measures has the potential to provide remote health monitoring and rapid detection of critical events such as heart attacks and falls. This paper proposes a multimodal system for monitoring the elderly at their homes. The system proposed contains three ambient assistance services (Fall detection, Heart disorder detection and Location) and an emergency service. A three-axis accelerometer, pulse oximeter and eight photoelectric sensors are applied for fall detection, cardiac problems detection and location respectively. The emergency service provides data fusion of this sensors and sends detailed information about the statue of the followed person to the doctor. This multimodal system is modeled by Colored Timed and Stochastic Petri nets (CTSPN) simulated in CPNTools. Experimental tests for each service have been performed on 10 subjects. The results show that falls can be detected from walking or standing with 87% of accuracy, 82% of sensitivity and 92% of specificity, from a total data set of 50 emulates falls and 50 normal activities daily living. The results obtained during the tests validate the detection of tachycardia with 100% of success. The location was done with 94% of sensitivity. The proposed system minimizes the false positive and false negative.
Article
With the advancement in Information and Communication Technologies (ICTs), smart devices are becoming even more smart and intelligent with every passing day. Further, the evolution of speaking and hearing enabled devices in an IoT network is transforming the face of research in the Social IoT domain. However, the integration of argumentation enabled devices in Social IoT network has not been fully explored by researchers in the past. Therefore, this research work focuses on development of argument enabled Social IoT networks. In this paper, a fuzzy argument based classification scheme termed as Classifiication Enhaned with Fuzzy Argumentation (CleFAR) is proposed. The proposed scheme is deployed for classification of fall activities in fall prevention applications. A novel framework for fall prevention system using Fall Activity Recognition (FAR) is presented. The proposed system is designed for the purpose of fall activity recognition in smart home Ambient Assisted Living (AAL) systems. To experimentally evaluate the system’s performance, a smart home AAL environment is simulated and the inhabitant’s routine activity dataset is generated. The fall activities are simulated using wearable fall detection systems. The proposed scheme is trained and tested on generated datasets and its performance is compared with traditional classification algorithms such as Random Forest (RF), Support Vector Machines (SVM), Naive Bayes (NB), Decision Tree (DT) and Artificial Neural Networks (ANN) as well as existing argumentation based game theoretic Weighted Voting Scheme (WVS). Experimental results indicate that the proposed scheme outperforms the traditional classification schemes and WVS approach with prediction accuracy upto 91%. It turns out that the proposed approach achieves significant improvement over the existing schemes.
Chapter
This chapter presents the results of the Challenge UP – Multimodal Fall Detection competition that was held during the 2019 International Joint Conference on Neural Networks (IJCNN 2019). This competition lies on the fall classification problem, and it aims to classify eleven human activities (i.e. five types of falls and six simple daily activities) using the joint information from different wearables, ambient sensors and video recordings, stored in a given dataset. After five months of competition, three winners and one honorific mention were awarded during the conference event. The machine learning model from the first place scored \(82.47\%\) in \(F_1\)-score, outperforming the baseline of \(70.44\%\). After analyzing the implementations from the participants, we summarized the insights and trends of fall classification.
Article
The automatic recognition of human falls is currently an important topic of research for the computer vision and artificial intelligence communities. In image analysis, it is common to use a vision-based approach for fall detection and classification systems due to the recent exponential increase in the use of cameras. Moreover, deep learning techniques have revolutionized vision-based approaches. These techniques are considered robust and reliable solutions for detection and classification problems, mostly using convolutional neural networks (CNNs). Recently, our research group released a public multimodal dataset for fall detection called the UP-Fall Detection dataset, and studies on modality approaches for fall detection and classification are required. Focusing only on a vision-based approach, in this paper, we present a fall detection system based on a 2D CNN inference method and multiple cameras. This approach analyzes images in fixed time windows and extracts features using an optical flow method that obtains information on the relative motion between two consecutive images. We tested this approach on our public dataset, and the results showed that our proposed multi-vision-based approach detects human falls and achieves an accuracy of 95.64% compared to state-of-the-art methods with a simple CNN network architecture.
Article
In this prospective study, half of all falls resulted in injury. Pre-frail adults sustained more injuries, while more frail adults had injuries requiring hospitalization or fractures. Pre-frail adults fell more often when in movement compared with frail adults who fell more often when standing and in indoor public spaces. Purpose To assess prospectively how fall environment and direction are related to injury among pre-frail and frail adults. Methods We included 200 community-dwelling adults with a prior fall (pre-frail, mean age 77 years) and 173 adults with acute hip fracture (frail, mean age 84 years; 77% community-dwelling). Falls were prospectively recorded using standardized protocols in monthly intervals, including date, time, fall direction and environment, and injury. We used logistic regression to assess the odds of injury adjusting for age, body mass index (BMI), and gender. Results We recorded 513 falls and 331 fall-related injuries (64.5%) among the 373 participants. While the fall rate was similar between groups, pre-frail adults had more injuries (71% among pre-frail vs. 56% among frail, p = 0.0004) but a lower incidence of major injuries (9% among pre-frail vs. 27% among frail, p = 0.003). Pre-frail adults fell more often while in movement (84% among pre-frail vs. 55% among frail, p < 0.0001), and frail adults fell more often while standing (26% vs. 15% respectively, p = 0.01). The odds of injury among frail adults was increased 3.3-fold when falling sideways (OR = 3.29, 95% CI = 1.68–6.45) and 2.4-fold when falling in an indoor public space (OR = 2.35, 95% CI = 1.00–5.53), and was reduced when falling at home (OR = 0.55, 95% CI = 0.31–0.98). The odds of injury among pre-frail adults was not influenced by environment and was 53% lower when falling backwards (OR = 0.47, 95% CI = 0.26–0.82). Conclusion While pre-frail adults sustain more fall-related injuries, frail adults were more likely to sustain major injuries, especially when falling sideways or outside their home.
Conference Paper
A new set of Non-Functional Requirements (NFRs) have appeared with the advent of Ubiquitous Computing (UbiComp) and more recently Internet of Things (IoT). Invisibil-ity is one of these NFRs that means the ability to hide technology from users. Although invisibility is long seen as an essential characteristic for achieving the goals of UbiComp, it has not been cataloged regarding its subcharacteristics and solutions, making its design and requirements specification in such applications a challenging task. Considering the Softgoal Interdependency Graph (SIG), which is a well-known format to catalog NFRs, this work aims at capturing subcharacteristics and solutions for In-visibility and cataloguing them in a SIG. Since there is no systematic approach on how to build SIGs, we also propose to systematize the definition of Invisibility SIG using the following well-defined research methods: snowballing, database search, grounded theory and questionnaires. As a result, we got an Invis-ibility SIG composed of two main subcharacteristics, twelve sub-subcharacteristics, ten general solutions and fifty-six specific solutions. This organized body of knowledge is useful for supporting software engineers to specify requirements and practical solutions for UbiComp and IoT applications. Furthermore, the proposed methodology used to capture and catalog requirements in a SIG can be reused for other NFRs.