ArticlePDF Available

Cognitive Workload Assessment of Prosthetic Devices: A Review of Literature and Meta-Analysis

Authors:

Abstract and Figures

Abstract—Limb amputation can cause severe functional disability for the performance of activities of daily living. Previous studies have found differences in cognitive demands imposed by prosthetic devices due to variations in their design. The objectives of this article were to 1) identify the range of cognitive workload (CW)assessment techniques used in prior studies comparing different prosthetic devices, 2) identify the device configurations or features that reduced CW of users, and 3) provide guidelines for designing future prosthetic devices to reduce CW. A literature search was conducted using Compendex, Inspec, Web of Science, Proquest, IEEE, Engineering Research Database, PubMed, Cochrane, andGoogle Scholar. Forty-three studies met the inclusion criteria. Findings suggested that CW of prosthetic devices was assesse dusing physiological, task performance, and subjective measures. However, due to the limitations of these methods, there is a need for more theoretical and model-based approaches to quantify CW. Device configurations such as hybrid input signals and use of multiodal feedback can reduce CW of prosthetic devices. Furthermore, to evaluate the effectiveness of a training strategy for reducingCW and improving device usability, both task performance and subjective measures should be considered. Based on the literature review, a set of guidelines was provided to improve the usability of future prosthetic devices and reduce CW.
summarizes the findings of meta-analysis. Black circles represent the difference (D) in CW between the two conditions. A negative D indicates that the CW of the former condition was lower than the latter. The red square represents the average D in each category, k represents the number of articles in each category, and p indicates the p-value. If the red square is on the left side of the vertical blue dotted bar, it means that the former condition is better than the latter in terms of CW. For example, in the first comparison, the red square is on the left side, meaning that the CW of using a hybrid approach is significantly lower than that of a nonhybrid configuration. 1) Comparison of Different Prosthetic Device Configurations in Terms of Input Signals: Based on the meta-analysis, it was found that CW was significantly lower when hybrid signals were used as an input as compared to the nonhybrid approach (p < 0.05). Hybrid configurations include a combination of EMG or brain signals with other input signals such as EMG + IMU [64], [65] and motion tracking + brain signal [78]. Hybrid input signals were found to have lower CW than single input signals (i.e., brain signal, EMG, and motion tracking only). Within the EMG-based interfaces, CW was significantly lower when the PR controller was used as compared to the DC in performing simulations of ADLs such as the clothespin relocation task, Jebsen hand function test, and cubbies task (p < 0.05) [19]-[21], [39], [41], [100]. 2) Comparison of Feedback Modalities in Prosthetic Devices: The analysis on feedback modalities indicated that using auditory feedback was less cognitively demanding as compared to the visual feedback (p < 0.05) [35], [46], [47]. However, using multimodal feedback (i.e., visual and auditory) was more beneficial than using visual feedback alone (p < 0.05) [35], [46], [47]. The feedback was provided on the task performance. For
… 
Content may be subject to copyright.
This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS 1
Cognitive Workload Assessment of Prosthetic
Devices: A Review of Literature and Meta-Analysis
Junho Park , Student Member, IEEE, and Maryam Zahabi , Member, IEEE
Abstract—Limb amputation can cause severe functional disabil-
ity for the performance of activities of daily living. Previous studies
have found differences in cognitive demands imposed by prosthetic
devices due to variations in their design. The objectives of this
article were to 1) identify the range of cognitive workload (CW)
assessment techniques used in prior studies comparing different
prosthetic devices, 2) identify the device configurations or features
that reduced CW of users, and 3) provide guidelines for designing
future prosthetic devices to reduce CW. A literature search was
conducted using Compendex, Inspec, Web of Science, Proquest,
IEEE, Engineering Research Database, PubMed, Cochrane, and
Google Scholar. Forty-three studies met the inclusion criteria.
Findings suggested that CW of prosthetic devices was assessed
using physiological, task performance, and subjective measures.
However, due to the limitations of these methods, there is a need
for more theoretical and model-based approaches to quantify CW.
Device configurations such as hybrid input signals and use of multi-
modal feedback can reduce CW of prosthetic devices. Furthermore,
to evaluate the effectiveness of a training strategy for reducing
CW and improving device usability, both task performance and
subjective measures should be considered. Based on the literature
review, a set of guidelines was provided to improve the usability of
future prosthetic devices and reduce CW.
Index Terms—Human–machine interface, literature review,
mental workload, meta-analysis, prosthesis.
I. INTRODUCTION
MORE than 2.1 million people with amputations live in the
USA and about 185 000 amputations occur each year [1],
[2]. Limb amputation can cause severe functional disability for
the performance of activities of daily living (ADLs).
Amputees use prosthetic devices on a regular basis to perform
ADLs. Without these devices, ADLs may not be possible or may
require additional effort and time [3], [4]. However, existing de-
vices are often reported to be challenging to use, leading to poor
utilization and device rejection [5], [6]. In an article assessing
the usability of different prosthetic devices, it was found that
53% of passive hand users, 50% of body-powered hook users,
and 39% of myoelectric hand users rejected prosthetic hands.
Manuscript received April 1, 2020; revised December 17, 2020, February 21,
2021, and November 19, 2021; accepted January 11, 2022. This work was sup-
ported by the National Science Foundation under Grant IIS-1856676. The views
and opinions expressed are those of authors. This article was recommended
by Associate Editor J. L. Contreras-Vidal. (Corresponding author: Maryam
Zahabi.)
The authors are with the Wm Michael Barnes ’64 Department of Indus-
trial and Systems Engineering, College Station, TX 77843 USA (e-mail:
junho.park@tamu.edu; mzahabi@tamu.edu).
Digital Object Identifier 10.1109/THMS.2022.3143998
The main reasons for rejection were identified as poor dexterity,
glove durability, and lack of sensory feedback [7]–[9]. Other
articles indicated that 40% to 60% of amputees were not satisfied
with their lower-limb prostheses mainly due to issues such as
discomfort, excessive weight, difficulty of use, and pain [10],
[11]. Related to that, 31% of armed forces service members
with lower-limb amputations rejected their prosthesis due to the
lack of satisfaction and usability issues [12], [13].
Using prosthetic devices requires substantial amount of cog-
nitive resources [14]–[18], which can be an underlying cause
for device rejection. Previous articles have found that devices
that impose high cognitive workload (CW) can reduce task per-
formance, which can negatively affect user satisfaction, reduce
device usability, and ultimately might lead to frustration and
prosthetic device rejection [19]–[21]. These cognitive resources
are used to compensate for the loss of motor control and to
mitigate the loss of somatosensory feedback from the amputated
limb [16], [18], [22]–[25]. Therefore, using prostheses can cause
a lack of cognitive capacity available to conduct other mental
activities [16], [18]. High CW can also reduce primary task per-
formance [26]. For example, an amputee may find it difficult to
avoid obstacles or walk in uneven terrain. In case of upper-limb
amputation, most of the current control strategies use limited
information [i.e., shoulder movements or recorded electromyo-
graphy (EMG) signals] for activating several degrees of freedom
of the prosthetic devices, which is not intuitive and results in high
CW [27]. Assessment of CW can provide an understanding of
the underlying attentional resources that are engaged during task
execution and support the evaluation/development of prosthetic
devices [4].
A. Cognitive Workload Assessment Techniques
CW assessment techniques are typically categorized into three
broad categories including physiological measures, subjective
rating scales, and performance measures [28]. Physiological
measures (e.g., heart rate variability) allow the understanding of
physiological processes through their effect on body, rather than
through task performance or perceptual ratings [29]. Subjective
ratings quantify humans’ understanding and judgments of their
experienced demand. Performance measures are classified into
two major categories including primary and secondary task mea-
sures. Primary task measures evaluate operator’s performance
on the task of interest. Examples of primary task performance
measures of CW include speed, accuracy, reaction or response
times, and error rate. Secondary task measures provide an index
2168-2291 © 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Texas A M University. Downloaded on February 11,2022 at 16:28:01 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
2IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS
of the remaining operator capacity while performing primary
tasks, and are more diagnostic than primary task performance
measures [29]. Examples of secondary tasks performance mea-
sures include n-back, verbal shadowing, and pursuit tracking
task.
Besides the conventional measures of CW, cognitive perfor-
mance modeling (CPM) technique is another approach to assess
or predict CW. CPM methods such as goals, operators, methods,
and selection rules (GOMS) can provide representations of
human performance, including learning time, execution time,
number of cognitive/perceptual/motor operators, and task errors
[30]. These models can predict the amount of time that an
expert need to retrieve information from memory, select from
decision options, and execute motor movements. These features
enhance the interpretability of CPM approaches as compared to
other CW measurement techniques such as physiological and
subjective measures [20]. Furthermore, as compared to some of
the physiological measures of CW [e.g., electroencephalogram
(EEG)], which are intrusive and can be contaminated by body
motion, CPM approaches are not intrusive and the models can
be modified and applied in different applications. CPMs can be
coded, compiled, and run using software applications such as
Cogulator and CogTool [31], [32].
B. Cognitive Workload Assessment of Prosthetic Devices
Prior articles have investigated CW of different prosthetic
devices using various physiological (e.g., EEG, heart rate, res-
piratory rate, and skin conductance) [33]–[35] or subjective
measures [e.g., NASA-Task Load Index (NASA-TLX)] [36],
[37]. However, there is a lack of a systematic review that
identifies what prosthetic design features or configurations lead
to the differences in CW. In addition, prior articles were only
focused on specific prosthetic device configurations. For exam-
ple, some articles were focused on direct control (DC) or pattern
recognition (PR) controllers [19]–[21], [38]–[43], whereas other
investigations were focused on assessing workload in hybrid
methods such as surface EMG with force myography (FMG)
[44], [45]. Other investigations compared different feedback
modalities in prosthetic devices such as auditory, visual, or
vibrotactile feedback [34], [35], [46]–[48]. Furthermore, some
articles assessed the impact of training on CW of prosthetic
users. However, these assessments were limited to specific
prosthetic configurations (e.g., body-powered or EMG-based
devices) [34], [49]–[51] or training duration [52], [53]. Thus,
there is a need for an integrated analysis on the impact of training
on reducing CW of prosthetic users. Furthermore, only one
article used CPM approach to assess CW of prosthetic devices
although the method has been used extensively in other domains
such as driver workload assessment and usability evaluations
[20], [54].
The objectives of this review were to 1) identify the range
of CW assessment techniques used in prior articles comparing
different prosthetic devices, 2) identify the device configurations
or features that reduce CW of users, and 3) provide a set of
guidelines for designing future prosthetic devices to reduce CW.
II. METHOD
The literature review was performed in accordance with the
Preferred Reporting Items for Systematic Reviews and Meta-
Analyses (PRISMA) guidelines [55]. The detailed description
of our method including search strategy, eligibility and selection
criteria, and data analysis is described below.
A. Information Sources and Search Strategy
A systematic literature search was conducted using Com-
pendex, Inspec, Web of Science, Proquest, IEEE, Engineering
Research Database, PubMed, Cochrane, and Google Scholar
databases. The search was completed in November, 2020. Man-
ual search was also conducted in Google Scholar, as it is the
most comprehensive search engine [56]–[59]. The search termi-
nologies included “workload AND prosthAND limb.”Inthe
advanced setting, we limited the search to include only journal
articles, conference proceedings, and theses/dissertations.
B. Eligibility Criteria
To be included in this review, articles had to fulfill the follow-
ing criteria:
1) The article had to include quantitative information on
cognitive (or mental) workload.
2) The article had to meet the definition of prosthesis for this
review, i.e., “A device to replace missing upper or lower
limb functionality” [60], [61]. Limb functionality refers
to activities such as reaching, grasping, or walking (for
lower limb).
3) Study participants could be either amputees using actual
prosthetic devices or able-bodied participants using by-
pass prostheses.
4) The article had to be written in English.
5) The article should be published in or after 2005 for upper
limb (the year that Defense Advanced Research Projects
Agency started the Revolutionizing Prosthetics Program)
[62] and in or after 1999 for lower limb (the year that
fully microprocessor-controlled prosthesis made walking
with the prosthesis feel and look more natural and pro-
vided lower-limb amputees with a solution that was more
responsive to changes in walking speed) [63].
As shown in Table I, about 60% (26/43) of the articles
were conducted with able-bodied participants using bypass de-
vices. Approximately 12% (5/43) of the articles involved both
able-bodied and amputee participants. About 28% (12/43) of
the articles were conducted with amputees. For those articles
with able-bodied participants, bypass devices were developed
using various input signals such as EMG, inertial measurement
unit (IMU), FMG, and motion tracking. Also, bypass devices
were used to study effects of feedback modality and training
schedule on CW. Therefore, bypass devices with able-bodied
participants were included in the article as they are devices
that allow an able-bodied user to activate a terminal device
with similar controls that an amputee would use to operate a
custom-made prosthesis [90]. Furthermore, based on our review
and previous articles on prosthetic devices, recruiting amputee
Authorized licensed use limited to: Texas A M University. Downloaded on February 11,2022 at 16:28:01 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
PARK AND ZAHABI: COGNITIVE WORKLOAD ASSESSMENT OF PROSTHETIC DEVICES 3
TAB LE I
OVERVIEW OF REVIEWED STUDIES
Authorized licensed use limited to: Texas A M University. Downloaded on February 11,2022 at 16:28:01 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
4IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS
TAB LE I
(CONTINUED.)
(Alphabetical order) ACMC: assessment for capacity of myoelectric control, AM-ULA: activities measure for upper limb amputees, B&B: box and block, BF: breathing
frequency, CRT: clothespin relocation task, EEG: electroencephalography, DC: direct control, FMG: force myography, HR: heart rate, HRV: heart rate variability, IMU: inertial
measurement unit, JHFT: Jebsen hand function test, L: lower limb, NT: NASA-TLX, N/A: descriptive data only, PR: pattern recognition, RFID: radio frequency identification,
RR: respiratory rate, S: significant, SC: skin conductance, SHAP: Southampton hand assessment procedure, ST: skin temperature, U: upper limb.(Alphabetical order) EEG:
electroencephalography, IMU: inertial measurement unit, L: lower limb, NT: NASA-TLX, N/A: descriptive data only, PR: pattern recognition, S: significant, U: upper limb.
participants for human subject experiments are challenging and
therefore several articles used able-bodied participants to assess
the usability and CW of prosthetic devices. Thus, inclusion of
bypass devices in our article might be helpful in providing a more
comprehensive design guideline for assessing CW of prosthetic
devices. All publications including journal articles, conference
proceedings, and theses/dissertations were eligible.
C. Study Selection
Initially, the relevance of literature returned through the
searches was evaluated via a review of the titles. Literature
deemed relevant via title review was assessed for relevance again
via abstract review. Finally, the full text of literature deemed
relevant by both title and abstract were reviewed by the authors
(n=159). Of the initial records found (n=9261), 43 articles
were found to meet the eligibility criteria and were included in
this review per PRISMA methodology shown in Fig. 1. Among
the 43 articles, there were 32 journal articles, 7 conference
proceedings, and 4 Ph.D. or master theses.
The relevant articles were reviewed by the authors in order
to confirm relevance to the present article and to summarize the
findings. For each article, a structured summary was developed
including study citation, objective, methodology, findings, and
conclusions.
D. Data Extraction
The data extraction and variable coding for meta-analysis
were conducted based on the information provided in each article
(e.g., numbers, tables, or figures). Among the 43 articles, 25
articles were excluded due to lack of sufficient number of data
points (i.e., three data points per recommendations from [91])
for each comparison category such as PR vs. DC prosthetic
configurations. Therefore, the meta-analysis was conducted on
the remaining 18 articles.
Authorized licensed use limited to: Texas A M University. Downloaded on February 11,2022 at 16:28:01 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
PARK AND ZAHABI: COGNITIVE WORKLOAD ASSESSMENT OF PROSTHETIC DEVICES 5
TAB LE I I
HYPOTHESES (HYPOTHESIS NUMBER IN PARENTHESES)
Fig. 1. Literature review process based on PRISMA methodology.
For the overall NASA-TLX score, the average of six subdi-
mensions including mental demand, physical demand, temporal
demand, effort, performance, and frustration was calculated due
to the lack of reporting weights of each category in prior articles.
However, this procedure has been used and validated in previous
articles [92].
E. Hypotheses
Based on the literature review results, a list of hypotheses (H)
was formulated which is presented in Table II. For input signals,
it was hypothesized that the hybrid approach will generate lower
CW than the nonhybrid approach (H1) due to its versatility in
terms of sources of input. Hypothesis 2 (H2) was formulated
based on the findings of our prior article [19]. Multiple resource
theory (MRT) [93] was applied to formulate hypotheses 3–5.
Since virtual environments (VEs) can provide visual and/or au-
ditory feedback to participants, they might reduce CW compared
to physical devices. In addition, since prosthetic device users
use visual resources to interact with the device, using another
modality (i.e., auditory) could generate lower CW than visual
feedback. Hypotheses 6 and 7 were formulated to capture the
effect of training and posture in lower-limb prosthetic devices,
respectively.
F. Data Analysis Approach
The meta-analysis was conducted using Revman 5.3, a meta-
analysis software where we calculated heterogeneity (I2)for
each group of comparison (both within-subject design and
between-subject design). On average, I2was calculated as 38%,
which based on Cochrane’s guide [94], indicates that there was
no heterogeneity issue. The statistical test and procedures used
were based on [95] and were similar to the procedure used in
other meta-analysis articles [96]. The meta-analysis plot was
generated using MATLAB 2020.
A regression analysis was conducted on NASA-TLX scores of
20 upper-limb articles using JMP Pro 15.0.3. Box-Cox transfor-
mation on the dependent variable satisfied both normality and
equal variance assumption. To verify differences in levels of
significant effects, Tukey’s honest significant difference post-
hoc multiple comparison was applied. A significance level of
α=0.05 was set as a criterion for the article.
III. RESULTS
A. Cognitive Workload Assessment Techniques in Previous
Articles
A review of literature on upper-limb articles revealed that
CW of prosthetic devices was assessed using a combination of
physiological, subjective, or task performance measures (Fig. 2).
Physiological measures included various types of brain activity
measures such as P200 (which represents some aspect of higher
order perceptual processing, modulated by attention), P300 (an
event-related potential component elicited in the process of
decision making), late positive potential (LPP, an event related
potential that reflects facilitated attention to emotional stimuli),
and frontal theta/parietal alpha (FT/PA) [35], [38], [43], [66]–
[68], [70], [71]. A few articles used cardiac [35], respiratory
[35], skin [35], and eye-tracking measurements [19]–[21], [35].
Skin measurements included skin conductance and temperature.
Eye-tracking measures included blink rate and pupillometry
measures such as pupil diameter [19]–[21]. Among all the CW
measures, NASA-TLX was the most frequently used method
(28 out of 43 articles) [34], [35], [43]–[48], [53], [64]–[66],
[69], [70], [72]–[82], [86]–[88]. The main reason for frequent
Authorized licensed use limited to: Texas A M University. Downloaded on February 11,2022 at 16:28:01 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
6IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS
Fig. 2. Cognitive workload assessment techniques in prosthesis studies.
use of NASA-TLX was determined as its capability to as-
sess CW in motor tasks [58], [66], [92] and consideration
of overall workload as well as the magnitude of each factor
[49], [50].
Primary and secondary task performance were used as CW
measures in 16 articles [19]–[21], [34], [39]–[42], [69], [72],
[78], [79], [83]–[85], [89]. Primary task measures were mainly
used when the participants performed ADL tasks and were
defined in terms of task completion time and the number of trans-
ported items [39]–[41], [79], [83], [85]. CW was also assessed
using secondary task performance measures when participants
were asked to perform verbal, semantic, or numerical cognitive
tasks along with the ADL tasks or other primary tasks during
the experiment (e.g., participants counted backward from 100 to
1 with three steps while they were moving an object with their
prosthetic device [42]).
Only one article used CPM approach to assess CW of pros-
thetic devices [20]. The finding of this article comparing DC
and PR control modes suggested that CPM approaches such as
GOMS models can be used to assess cognitive demands of using
upper-limb prostheses [20].
Six lower-limb articles met the criteria of this review [34],
[69], [72], [73], [84], [89]. Heart rate, skin conductance, skin
temperature [34], and EEG signals [69], [72], [73], [89] were
used as physiological measurement techniques in these articles.
In addition, some articles used NASA-TLX as a subjective
measure of CW [34], [69], [72], [73]. Secondary tasks or dual
tasks under walking or seating condition were also used in
few studies to assess CW [34], [69], [72], [84]. It was found
that participants prioritized between the difficulty of walking
(primary task) and the secondary cognitive tasks in different
environmental conditions (e.g., walking on uneven terrain).
Therefore, to reduce CW, not only secondary tasks but also
primary tasks and environments should be carefully designed
[34], [69], [84].
B. Prosthetic Device Configurations
We made a structure of human–machine interfaces for pros-
thetic devices based on reviewed articles. These interfaces can
be categorized in terms of input signals (signals captured from
human to control the device) and outputs controls (Fig. 3). In
terms of input signals, prosthetic devices can be controlled using
EMG signals, brain signals, and/or other methods. There are
various methods in “others” category such as body-powered
devices or devices controlled by FMG, IMU, or motion tracking.
Outputs can be generally categorized in two groups of physical
devices and VE. Most of the upper-limb articles used EMG as
an input signal for controlling the physical device. Lower-limb
prosthetic devices included passive, energy-storing prosthetic
feet, and microprocessor-controlled devices. In addition, some
articles used bypass input signals with sensors and displayed
virtual prosthesis motions on two-dimensional (2-D) or (3-D)
screens.
1) Inputs:
a) EMG signals: About 60% (26 out of 43 articles) of
the articles used EMG as an input signal, which was the most
frequently used input signal (Table II). This finding is consistent
with other articles [49], [97]–[99] that found myoelectric control
is an appropriate technique concerned with the detection, pro-
cessing, classification, and application of input signals to control
human–machine interfaces in rehabilitation. Most comparisons
of CW have been made between the DC and PR controllers. It
was found that the PR controller imposed less CW on the user as
compared to the DC mode due to intuitive muscle contractions
[20].
Various feedback modalities were combined with EMG-based
configurations to improve task performance and reduce CW. The
most heavily used feedback modality was vibrotactile feedback
[45], [48], [64], [65]. For example, participants received vibro-
tactile feedback that came from uniformly placed vibrotactors
providing information on contact, prosthesis state (active func-
tion), and grasping force. Auditory and visual modalities were
also used in a few articles [35], [46], [47]. For example, the
flexion of the fingers was divided into eight different positions,
which were identified by different piano major triads for the
palmar grasp. As a visual feedback, a green LED was used to
indicate when to start and finish each trial. The participant was
asked to open his hand completely and wait for the LED to turn
ON, then, start closing the hand until the bottle was fully grasped.
Authorized licensed use limited to: Texas A M University. Downloaded on February 11,2022 at 16:28:01 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
PARK AND ZAHABI: COGNITIVE WORKLOAD ASSESSMENT OF PROSTHETIC DEVICES 7
Fig. 3. Configurations of human–machine interfaces for prosthetic devices.
b) Brain signal: EEG [43], [78] was used to provide input
signals to control non-EMG-based devices in order to perform
ADLs. However, using EEG as input signals was more cog-
nitively demanding and required higher effort (attention) than
manual control [100]. Furthermore, EEG signals are highly
susceptible to interference from skeletal muscle activity and
often require the application of elaborate filtering methods that
may result in loss of meaningful signal information. However,
articles that used the hybrid approach (i.e., combination of EEG
and EMG signals) found reduction in CW as compared to the
EEG or EMG signals alone [78].
c) Others: Body-powered prosthetic devices are con-
trolled by body movements. For example, upper-limb body-
powered devices, or cable-operated limbs, work with a harness
and a cable around the opposite shoulder of the injured arm.
The participant can pull the cable to open the prosthetic hand
(hook) by shoulder movements. Passive devices (for lower limb)
[34], [69], [72], [84], [89] were used in five articles to compare
their capabilities with other prosthetic devices or to compare
task performance or CW in various feedback modalities. Force
measurement unit and IMU were used with EMG signals as a
hybrid approach and generated lower CW as compared to EMG
control inputs [44], [45], [75].
Motion tracking was also used to control prosthesis and 2-D
or 3-D displays [53], [77], [78], [80]–[82]. Head movements
were classified as a gesture or pattern for controlling the device’s
movements and animations in the VE. In addition, using motion
capture systems was found to be a more appropriate approach
for capturing head movements instead of using skin attachments
[53], [81], [82]. Lower-limb articles used passive controllers
(i.e., prosthetic devices that look like a natural lower limb) [34],
[72], [73], energy storing prosthetic feet (i.e., devices that use
body weight and force as inputs) [69], [89], and microprocessor-
controlled devices (i.e., devices that sense users’ movement)
[84].
2) Outputs:
a) Physical device: About 77% of the articles (33/43)
used physical devices as an output (Table I). Major EMG-based
devices included DC or PR controllers. A few articles also used
ON/OFF threshold setup (i.e., a prosthetic hand that is operated
with a constant speed in clockwise and counterclockwise direc-
tions with a full stop). Some EMG-based devices used EMG
signals in combination with FMG [44], [45], RFID [75], IMU
[64], [65], [75], and vision module [75] as a hybrid method. The
findings indicated that CW in the hybrid approach was lower
than devices using EMG signals alone.
b) Virtual environment: About 35% of the articles (15/43)
used VE as an output. The articles with VE setup used nonim-
mersive (i.e., 2-D displays) [43], [44], [53], [76]–[78], [80]–[83]
or immersive systems (i.e., 3-D displays) [38], [67], [70], [71],
[75]. None of the articles used head-mounted displays. VE
could be used for testing the capability of human, through
practice, to acquire new sensorimotor mappings to adapt to novel
kinematics or dynamics as well as to learn how to manipulate
a device [82].
However, VE can be challenging for participants in that they
might be able to intentionally control the movements in pros-
thetic devices while allocating less attention to task performance,
which leads to high stability in prosthesis movements and low
task performance [68]. No lower-limb article used 2-D or 3-D
Authorized licensed use limited to: Texas A M University. Downloaded on February 11,2022 at 16:28:01 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
8IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS
Fig. 4. Meta-analysis results.
VE as an output. Instead, these investigations used passive or
active transfemoral or transtibial prosthesis devices.
C. Meta-Analysis
Fig. 4 summarizes the findings of meta-analysis. Black circles
represent the difference (D) in CW between the two conditions.
AnegativeDindicates that the CW of the former condition was
lower than the latter. The red square represents the average D
in each category, krepresents the number of articles in each
category, and pindicates the p-value.
If the red square is on the left side of the vertical blue dotted
bar, it means that the former condition is better than the latter
in terms of CW. For example, in the first comparison, the red
square is on the left side, meaning that the CW of using a
hybrid approach is significantly lower than that of a nonhybrid
configuration.
1) Comparison of Different Prosthetic Device Configurations
in Terms of Input Signals: Based on the meta-analysis, it was
found that CW was significantly lower when hybrid signals were
used as an input as compared to the nonhybrid approach (p<
0.05). Hybrid configurations include a combination of EMG or
brain signals with other input signals such as EMG +IMU
[64], [65] and motion tracking +brain signal [78]. Hybrid input
signals were found to have lower CW than single input signals
(i.e., brain signal, EMG, and motion tracking only). Within the
EMG-based interfaces, CW was significantly lower when the
PR controller was used as compared to the DC in performing
simulations of ADLs such as the clothespin relocation task,
Jebsen hand function test, and cubbies task (p<0.05) [19]–[21],
[39], [41], [100].
2) Comparison of Feedback Modalities in Prosthetic De-
vices: The analysis on feedback modalities indicated that using
auditory feedback was less cognitively demanding as compared
to the visual feedback (p<0.05) [35], [46], [47]. However,
using multimodal feedback (i.e., visual and auditory) was more
beneficial than using visual feedback alone (p<0.05) [35], [46],
[47]. The feedback was provided on the task performance. For
TABLE III
STUDY VARIABLES AND LEVELS FOR REGRESSION ANALYSIS
example, auditory feedback was provided to let the user know
whether he/she grasped the object with sufficient force [35].
Altogether, the comparison between different feedback modali-
ties suggests using multimodal visual and auditory feedback to
reduce CW.
3) Effect of Training on Cognitive Workload: Findings sug-
gested that upper-limb amputees experienced less CW after
continuous training for 4–6 weeks at home (p<0.05) [39],
[41], [83] (Fig. 4). These articles assessed participants’ cognitive
load before and after 4–6 weeks of training. Under 6 weeks of
training, participants were exposed to prosthetic devices for at
least 124 h. During four weeks of training, participants received
at least 38 h training on the devices.
4) Cognitive Workload Assessment in Lower-Limb Articles:
CW was higher in the walking condition than the seated condi-
tion (p<0.05) [34], [69], [72]. However, there was no significant
difference in CW between the seated condition and seated con-
dition with secondary task (p>0.05). The findings suggest that
walking requires substantial cognitive resources for amputees
with lower-limb prosthetic devices and performing difficult sec-
ondary tasks during this activity significantly increases CW.
D. Regression Analysis
The objective of this analysis was to identify the effect of
prosthetic device configurations on CW. Data coding for the
regression analysis is shown in Table III. Two variables were
identified based on the literature review including: 1) input
signals and 2) output controls. It is important to note that the
Authorized licensed use limited to: Texas A M University. Downloaded on February 11,2022 at 16:28:01 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
PARK AND ZAHABI: COGNITIVE WORKLOAD ASSESSMENT OF PROSTHETIC DEVICES 9
TAB LE I V
RESULTS OF REGRESSION ANALYSIS
Input signals: Signals captured from human to control the device, Output control: including physical devices or virtual environment.∗∗An αlevel of 0.05 was used to
determine statistical significance for the regression analysis.
TAB LE V
SUMMARY OF HYPOTHESIS TESTS
TAB LE V I
COMPARISON OF CW ASSESSMENT TECHNIQUES
feedback modality was initially included in the model; however,
it was removed due to the lack of sufficient number of obser-
vations. The analysis was focused only on upper-limb articles
as there were only six relevant lower-limb articles and their
workload measurements were heterogeneous. The overall CW
was determined based on the overall NASA-TLX score reported
in prior articles.
Based on Table IV, the type of input signal had a significant
impact on overall CW (p<0.05). Among the input signals, the
hybrid approach significantly reduced CW (parameter estimate
=10.82), while others significantly increased CW (parameter
estimate =10.11). However, there was no effect of output control
on CW (p>0.05). The model predicting the overall CW was
specified according to (1):
Overall cognitive workload = α+βInput_Signal ·InputSignal
+βOutputControl ·OutputControl (1)
where αis the intercept, βinput_Signal and βoutput_Control are the
parameters associated with the study variables listed in Table IV.
E. Summary of Hypothesis Tests
Table V illustrates the summary of hypothesis tests based
on meta-analysis and regression results. The results are further
discussed in the following section.
IV. DISCUSSION
This article identified the determinants of CW associated with
using prosthetic devices using two approaches including a meta-
analysis and regression analysis. The meta-analysis provided a
comparison between different variables in terms of CW. Fur-
thermore, the regression analysis reinforced the meta-analysis
results and revealed the impact of device configurations (input
and output) on CW. From this discussion, a set of guidelines was
formulated in Table VII.
A. CW Assessment Techniques
This review revealed that a majority of articles used EEG,
NASA-TLX, and task performance measures to assess CW of
prosthetic devices. EEG signals were used frequently to measure
CW since in most of the articles conducted with upper-limb
prosthetic devices; the participant was in a static posture without
any head or body movement. This static posture would be helpful
to gather and analyze EEG signals from participants as these
signals can be easily contaminated by head or body motion.
In terms of statistical analysis, it was hard to generalize or
elicit insights on CW due to various measurement techniques
such as P200, P300, LPP, FT/PA, FT/frontal alpha ratio, and
insufficient number of articles per each category to conduct
statistical analysis.
Authorized licensed use limited to: Texas A M University. Downloaded on February 11,2022 at 16:28:01 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
10 IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS
TAB LE V II
SUMMARY OF GUIDELINES FOR DESIGNING PROSTHETIC DEVICES WITH LOW COGNITIVE WORKLOAD
Secondary tasks were frequently used in lower-limb articles
to assess CW of prosthetic devices while walking (i.e., primary
task). Use of secondary tasks in this condition is preferred over
physiological measures of CW since bio sensor signals might be
contaminated by whole body movement while walking. How-
ever, complexity of the secondary tasks or environmental condi-
tions can reduce the performance of walking, or the primary task
[84]. In addition, if the amputees are acclimated to the prosthesis
and the environment is stable, the impact of cognitive burden
can be limited, and therefore physiological measurements can
be used instead of task performance measurements to capture
subtle changes in CW under these conditions [101].
One possible alternative to assess CW is the CPM method.
Although CPM was used only in a case study with one amputee
participant under DC and PR conditions, the method has the
potential to be applied to other configurations and experimental
conditions, considering its capability to predict task performance
and calculate memory chunks [20]. The models can calculate
task performance, the number of cognitive/perceptual/motor
operators, and memory chunks to identify bottlenecks in the
task. These models have been widely applied to other domains
such as human–computer interaction research, aviation, health
care, usability testing, and cybersecurity [102]–[108]. However,
it is important to note that CPM approaches assume expert
performance, and therefore the methods might have limited
application to novice prosthetic users.
Advantages and limitations of each CW assessment technique
were summarized in Table VI. A detailed comparison of these
techniques based on sensitivity, intrusiveness, cost, and accu-
racy can be found in [109]. Physiological measures allow the
understanding of psychological processes through their effect
on the body, rather than through task performance or perceptual
ratings [29]. Therefore, the principal advantage of physiological
measures is that these measures are continuous and objective.
However, some signals can be contaminated by head or body
movements (e.g., neuroimaging or EEG measures), especially
in experiments using prosthetic devices or electrode caps [20].
Majority of articles used NASA-TLX to measure CW since
the method is unobtrusive and can be easily collected after the
experiment sessions. Subjective measurement techniques such
as NASA-TLX quantify humans’ understanding and judgments
of their experienced demand. While these methods have high
face validity, their interpretation, and ability to predict perfor-
mance is uncertain [29]. These measures also provide discrete
rather than continuous values, and prior articles have found dis-
sociation between subjective and performance measures [111].
Furthermore, subjective measures are limited due to recall bias
and substantial individual differences [92].
Performance measures are classified into two major cate-
gories, including primary and secondary task measures. Primary
task measures evaluate the operator’s performance on the task
of interest. Examples of primary task measures of workload
include speed, accuracy, reaction or response time, and error
rate. Secondary task measures provide an index of the remain-
ing operator capacity while performing primary tasks and are
more diagnostic than primary task measures [29]. Examples of
secondary tasks include n-back, verbal shadowing, and pursuit
tracking task. Performance measures have advantages in that
they evaluate participants’ performance on the task of interest
directly, and this is useful where the demands exceed operators’
capacity such that performance degrades from baseline or ideal
level [29]. However, they often lack scientific rigor, making
interpretation of the results difficult. Unknown or uncontrolled
factors may affect results rather than the intended manipulations
Authorized licensed use limited to: Texas A M University. Downloaded on February 11,2022 at 16:28:01 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
PARK AND ZAHABI: COGNITIVE WORKLOAD ASSESSMENT OF PROSTHETIC DEVICES 11
in the article. Also, due to the protective (compensatory) effect
of increased effort in the task, measuring performance might
not be sufficient to evaluate the participant’s state. For example,
the performance does not reflect information about the costs
involved in the adaptive response to stress [29].
B. Device Configuration: Input Signals
Hybrid input configurations such as EMG signals in combi-
nation with IMU or FMG were less cognitively demanding as
compared to EMG or EEG input signals alone [64], [65], [75].
Combination of EMG and IMU reduced task completion time
since they were more intuitive than myocontrol configuration
alone. This active support can help the user perform a larger
set of tasks including easy (e.g., reaching or grasping an object)
and complex tasks (e.g., clothespin and cups relocation) while
decreasing the rejection rate of the device [113], [114]. Thus,
the combination of EMG and IMU can improve prosthetic
device usability in terms of effectiveness, satisfaction, and ef-
ficiency [115]. In addition, a combination of FMG and EMG
signals showed higher overall stability (i.e., lower variance)
over time than the EMG signals alone [44], [45]. Using the
hybrid configuration, participants did not have to memorize so
many different movement patterns in order to perform simple
tasks. Therefore, use of hybrid input methods can improve
prosthetic device usability and frequency of use by amputees to
perform ADLs [75]. Among the EMG-based controllers, the PR
mode was the least cognitively demanding prosthetic controller
[39]–[41], [83]. This was mainly because patterns are generated
from the user attempting an actual movement (e.g., forming a
fist). Thus, this identification of user intention can reduce CW,
which is not possible in other EMG-based controllers such as
DC. In terms of device usability, the PR mode is better than
the DC since it requires more natural arm gestures to control
the device, which requires PR rather than recall from memory
[116]. However, even in PR, unintentional hand movement can
occur [19]. Articles have identified errors related to the PR con-
trol, which resulted in unintentional prosthesis movement and
reduced device reliability [41], [83]. However, it was found that
these errors reduced by additional training and as the participants
learned to make more distinguishable movements [117].
C. Device Configuration: Output Control
Some articles found that non-EMG-based devices were signif-
icantly more cognitively demanding as compared to EMG-based
prosthesis. This was mainly due to the difference in eye-hand
coordination performance. The eyes fixate on a target to provide
spatial information before the hands are engaged in a movement
[118]. In EMG-based prosthetic devices, the output control is
attached close to the upper limb and its shape is similar to human
arm, which makes the movement more natural (less cognitively
demanding) for the participant. However, in non-EMG-based
devices such as EEG-based prostheses, the movement of the
device is not always similar to the actual limb because the input
signal location (head) and control location (hand) are separate
[43], [100]. Thus, the operator needs more attention (higher CW)
to move, control, and check the status of the arm. Therefore,
from the usability point of view, non-EMG-based devices need
to be more advanced to improve effectiveness, efficiency, and
satisfaction.
According to recent articles [119], [120], VE can provide
adaptable and rich media to create conditions for the assessment
and training of motor deficits. VEs can mimic arm extension,
flexion, supination, and pronation based on the input commands.
Furthermore, objects like a virtual ball and basket can be created
and placed within the VE [70]. The main advantage of the VE is
its rich and adaptive versatility in training and assessment [71].
Enhanced technology is currently being explored in prosthetic
limb training. It has been shown that VEs help users to form
a robust mental model to perform the ADLs [121], which are
especially important in the initial training phase of using a
myoelectric prosthesis. Since VEs can provide visual and/or
auditory feedback to participants, they can also help to reduce
CW [38], [50], [51], [67], [70], [71].
D. Device Configuration: Feedback Modality
Findings suggested that using combined auditory and visual
feedback modalities could reduce CW. Participants mainly rely
on their visual attention to control their prosthetic device [35],
[46], [47] and to perform ADLs. Based on MRT [93], additional
visual feedback in this situation can cause attentional overload.
For example, if only visual feedback is provided, participants
should visually focus on the task environment to perform the task
while continuously monitoring the visual display to check the
status of their performance. Thus, to avoid attentional resource
competition in the same modality, it is recommended to use
other modalities of information presentation (such as auditory or
vibrotactile [34], [50], [64], [65]) as task performance feedback
modality. Use of auditory feedback can also improve task perfor-
mance while reducing CW, which can increase device usability
[115], [122]. However, vibrotactile feedback should be used with
caution. Upper-limb prosthetic users slightly agreed with the
positive effect of vibrotactile feedback to improve performance
accuracy [50]. However, if the task required continuous visual
attention, there was no significant difference in CW (based on
eye-tracking measures) with or without the vibrotactile feedback
[85]. Some lower-limb articles found that using a combination of
vibrotactile and visual feedback was less cognitively demanding
than a single feedback modality [34]. Furthermore, a majority of
participants preferred mixed feedback modality instead of visual
feedback alone [50].
Another consideration is related to task complexity and train-
ing [113]. The feedback was beneficial only for complex tasks
such as clothespin and cups relocation task. After a participant
was sufficiently trained, he or she could develop a feedforward
strategy, thus, the feedback became redundant. This shows a
need for adaptive feedback based on the skill level of a prosthetic
device user.
E. Training
Participants perceived less CW after 4–6 weeks of training
with the prosthetic device. This might have been due to the
improvement in quality of control over the device and execution
Authorized licensed use limited to: Texas A M University. Downloaded on February 11,2022 at 16:28:01 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
12 IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS
of movements, which led to less mental effort to perform the
ADLs [39], [42], [83]. However, this recommendation should
be stated with caution and requires additional context. First, our
meta-analysis was based on subjective measures of workload
(i.e., NASA-TLX) and did not include the findings of usability
evaluations. In some upper-limb articles, task performance and
error rate, and therefore device usability improved after four and
six weeks of training [41]. Other articles recommended training
duration of 30 days to determine device rejection rate [123]. Sec-
ond, it should be noted that there might be discrepancies between
participants’ self-report and objective task performance. For
example, although they perceived low CW based on subjective
measures after 4–6 weeks of training, they might not achieve
optimal task performance [39], [41], [49], [83].
F. Limitations
This article had some limitations. First, some reviewed articles
did not provide the exact descriptive statistics values (i.e., mean
and standard deviation). To resolve this issue, we contacted
the authors and could obtain some of those exact values [4],
[33], [39], [48], [49], [51], [52], [64], [65], [83], [100]. For the
articles that we could not find the exact values, the mean and
standard deviation of different conditions were extracted from
the figures [4], [43]–[45], [70], [101]. However, to ensure the
quality of data extraction based on graphs, we compared the
estimates with the exact values (for those articles that we had
the exact values) and the results were similar, which validated
our approach. The second limitation was that the meta-analysis
could not be performed on physiological measures due to the
lack of the number of articles with matched independent and
dependent variables.
Finally, this article included investigations which used bypass
prosthetic devices since a majority of prior articles used able-
bodied participants to assess the usability and CW of prosthetic
devices. However, there are many factors (e.g., device weight,
pain, motivation, perceived functional ability, and satisfaction
with device) that might influence amputees’ mental demands
on any given day. These factors cannot be replicated in bypass
device users, so the results obtained from this article should be
interpreted with caution to the actual population of interest.
V. C ONCLUSION
The first objective of this article was to identify the range
of CW measures used in prior articles comparing different pros-
thetic devices. It was found that CW measures can be categorized
into physiological (e.g., heart rate and pupillometry data), and
subjective and task performance (i.e., primary and secondary
task). Among all the measures, the NASA-TLX questionnaire
was the most frequently used method to measure CW in using
prosthetic devices.
The second objective of the article was to identify the device
configurations or features that led to the lowest CW for the
users. It was found that the hybrid approach (e.g., EMG signals
in combination with other input signals) resulted in the lowest
CW as compared to other configurations (e.g., motion tracking)
or nonhybrid approaches. Regarding the feedback modality,
multimodal feedback was the most effective feedback to reduce
CW, as compared to visual and auditory feedback modalities
alone. Although the use of vibrotactile feedback was found to
be effective in reducing CW of lower-limb prosthetic devices,
its advantages were not clear for upper limbs. In addition, to
evaluate the effectiveness of a training strategy for reducing
CW and improving device usability, both task performance and
subjective measures should be considered.
The final objective of the article was to provide a set of
guidelines for designing future prosthetic devices to reduce CW.
A set of guidelines (Table VII) was established based on the
findings to reduce CW with appropriate input signal, output
control, feedback modality, and training schedule.
The results of this article can be beneficial in design and
development process of future prosthetic devices in order to
reduce CW. By recognizing the identified issues in prosthetic
device configurations, feedback, and training, developers may
be better able to make appropriate design changes toward more
effective, efficient, and satisfying prosthesis use. In addition, the
guidelines might be beneficial in providing design recommen-
dations to improve the usability of prosthetic devices.
REFERENCES
[1] M. Owings and L. J. Kozak, “Ambulatory and inpatient procedures in
the United States, 1996 (No. 139.),” US Dept. Health Human Services,
Centers Disease Control Prevention, Nat. Center Health Statist., 1998.
[2] K. Ziegler-Graham, E. J. MacKenzie, P. L. Ephraim, T. G. Travison,
and R. Brookmeyer, “Estimating the prevalence of limb loss in the
United States: 2005 to 2050,” Arch. Phys. Med. Rehabil., vol. 89, no. 3,
pp. 422–429, 2008.
[3] M. M. Lusardi, M. Jorge, and C. C. Nielsen, Orthotics and Prosthetics in
Rehabilitation-E-Book. St. Louis, MO, USA: Elsevier Health Sciences,
2013.
[4] C. Gaskins et al., “Mental workload assessment during simulated upper
extremity prosthetic performance,” Arch. Phys. Med. Rehabil., vol. 99,
no. 10, 2018, Art. no. e33.
[5] S. M. Engdahl, B. P. Christie, B. Kelly, A. Davis, C. A. Chestek, and
D. H. Gates, “Surveying the interest of individuals with upper limb loss
in novel prosthetic control techniques,” J. Neuroeng. Rehabil., vol. 12,
no. 1, 2015, Art. no. 53.
[6] A. Kannenberg and B. Zacharias, “Difficulty of performing activities of
daily living with the michelangelo multigrip and traditional myoelectric
hands,” in Proc. Amer. Acad. Orthotists Prosthetists 40th Acad. Annu.
Meeting Sci. Symp., 2014.
[7] E.Biddiss, D. Beaton, and T. Chau, “Consumer design priorities for upper
limb prosthetics,” Disabil. Rehabil.: Assistive Technol., vol. 2, no. 6,
pp. 346–357, 2007.
[8] J. Bowker, “The art of prosthesis prescription,” in American Academy
of Orthopaedic Surgeons. Atlas of Amputations and Limb Deficienciens
Surgical, Prosthetic and Rehabilitation Principles,D.G.Smith,J.W.
Michael, and J. H. Bowker, Eds., 3rd ed. Rosemont, IL, USA: Bone and
Joint Decade, 2004, pp. 739–744.
[9] F. Montagnani, M. Controzzi, and C. Cipriani, “Is it finger or wrist
dexterity that is missing in current hand prostheses?,”IEEE Trans. Neural
Syst. Rehabil. Eng., vol. 23, no. 4, pp. 600–609, Jul. 2015.
[10] G. M. Berke et al., “Comparison of satisfaction with current prosthetic
care in veterans and servicemembers from vietnam and OIF/OEF con-
flicts with major traumatic limb loss,” J. Rehabil. Res. Develop., vol. 47,
no. 4, pp. 361–371, 2010.
[11] T. R. Dillingham, L. E. Pezzin, E. J. MacKenzie, and A. R. Burgess,
“Use and satisfaction with prosthetic devices among persons with trauma-
related amputations: A long-term outcome study,” Amer.J.Phys.Med.
Rehabil., vol. 80, no. 8, pp. 563–571, 2001.
[12] E. C. Baars, E. Schrier, P. U. Dijkstra, and J. H. Geertzen, “Prosthesis sat-
isfaction in lower limb amputees: A systematic review of associated fac-
tors and questionnaires,” Medicine, vol.97, no. 39, 2018, Art. no. e12296.
Authorized licensed use limited to: Texas A M University. Downloaded on February 11,2022 at 16:28:01 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
PARK AND ZAHABI: COGNITIVE WORKLOAD ASSESSMENT OF PROSTHETIC DEVICES 13
[13] R. Gailey et al., “Unilateral lower-limb loss: Prosthetic device use and
functional outcomes in servicemembers from Vietnam war and OIF/OEF
conflicts,” J. Rehabil. Res. Develop., vol. 47, no. 4, pp. 317–332, 2010.
[14] T. W. M. Geurts, B. Nienhuis, and R. Rijken, “Dual-task assessment of re-
organization of postural control in persons with lower limb amputation,
Arch. Phys. Med. Rehabil., vol. 72, no. 13, pp. 1059–1064, 1991.
[15] A. C. Geurts and T. W. Mulder, “Attention demands in balance recovery
following lower limb amputation,J. Motor Behav., vol. 26, no. 2,
pp. 162–170, 1994.
[16] B. W. Heller, D. Datta, and J. Howitt, “A pilot study comparing the cog-
nitive demand of walking for transfemoral amputees using the intelligent
prosthesis with that using conventionally damped knees,Clin. Rehabil.,
vol. 14, no. 5, pp. 518–522, 2000.
[17] C. J. Hofstad, V. Weerdesteyn, H. van der Linde, B. Nienhuis, A.
C. Geurts, and J. Duysens, “Evidence for bilaterally delayed and
decreased obstacle avoidance responses while walking with a lower
limb prosthesis,” Clin. Neurophysiol., vol. 120, no. 5, pp. 1009–1015,
2009.
[18] R. M. Williams et al., “Does having a computerized prosthetic knee
influence cognitive performance during amputee walking?,Arch. Phys.
Med. Rehabil., vol. 87, no. 7, pp. 989–994, 2006.
[19] M. M. White et al., “Usability comparison of conventional direct control
versus pattern recognition control of transradial prostheses,” IEEE Trans.
Human-Mach. Syst., vol. 47, no. 6, pp. 1146–1157, Dec. 2017.
[20] M.Zahabi et al., “Application of cognitive task performance modeling for
assessing usability of transradial prostheses,” IEEE Trans. Human-Mach.
Syst., vol. 49, no. 4, pp. 381–387, Aug. 2019.
[21] W. Zhang et al., “Cognitive workload in conventional direct control vs.
pattern recognition control of an upper-limb prosthesis,” in Proc. IEEE
Int. Conf. Syst., Man, Cybern., 2016, pp. 002335–002340.
[22] C. Krewer,F. Müller, B. Husemann, S. Heller, J. Quintern, and E. Koenig,
“The influence of different lokomat walking conditions on the energy
expenditure of hemiparetic patients and healthy subjects,” Gait Posture,
vol. 26, no. 3, pp. 372–377, 2007.
[23] H.Witteveen, L. de Rond, J. S. Rietman, and P. H. Veltink, “Hand-opening
feedback for myoelectric forearm prostheses: Performance in virtual
grasping tasks influenced by different levels of distraction,J. Rehabil.
Res. Dev., vol. 49, no. 10, pp. 1517–1526, 2012.
[24] D. S. Childress, “Closed-loop control in prosthetic systems: Historical
perspective,Ann. Biomed. Eng., vol. 8, no. 4, pp. 293–303, 1980.
[25] P. Herberts and L. Körner, “Ideas on sensory feedback in hand prosthe-
ses,” Prosthetics Orthotics Int., vol. 3, no. 3, pp. 157–162, 1979.
[26] J. Duysens, Z. Potocanac, J. Hegeman, S. Verschueren, and B. J. Mc-
Fadyen, “Split-second decisions on a split belt: Does simulated limping
affect obstacle avoidance?,Exp. Brain Res., vol. 223, no. 1, pp. 33–42,
2012.
[27] F. Cordella et al., “Literature review on needs of upper limb prosthesis
users,” Front. Neurosci., vol. 10, pp. 209–209, 2016.
[28] F. Eggemeier and G. Wilson, “Workload assessment in multi-task en-
vironments. Multiple task performance,” in DL Damos. London, U.K.:
Taylor & Francis, 1991.
[29] B. Cain, A Review of the Mental Workload Literature. Toronto, ON,
Canada: Defence Research And Development Toronto (Canada), 2007.
[30] S. K. Card, The Psychology of Human–Computer Interaction.Boca
Raton, FL, USA: CRC Press, 2018.
[31] S. Estes, J. Helleberg, K. Long, M. Pollack, and M. Quezada, “Guidelines
for speech interactions between pilot and cognitive assistant,” in Proc.
Integr. Commun., Navigation, Surveill. Conf., 2018, pp. 3H2–3H1–3H2–
10.
[32] L.-H. Teo, B. John, and M. Blackmon, “CogTool-explorer: A model of
goal-directed user exploration that considers information layout,”in Proc.
SIGCHI Conf. Hum. Factors Comput. Syst., 2012, pp. 2479–2488.
[33] J. M. Bradley, Enhancing Human–Machine Interaction With Wearable
Haptic Devices. Houston, TX, USA: Rice Univ., 2018.
[34] S. Crea, B. B. Edin, K. Knaepen, R. Meeusen, and N. Vitiello, “Time-
discrete vibrotactile feedback contributes to improved gait symmetry in
patients with lower limb amputations: Case series,” Phys. Ther., vol. 97,
no. 2, pp. 198–207, 2017.
[35] J. Gonzalez, H. Soma, M. Sekine, and W. Yu, “Psycho-physiological
assessment of a prosthetic hand sensory feedback system based on an
auditory display: A preliminary study,” J. Neuroeng. Rehabil.,vol.9,
no. 1, 2012, Art. no. 33.
[36] U. Khalid, “Development and human performance evaluation of control
modes of an exo-skeletal assistiverobotic arm (esara),” Ph.D. dissertation,
Dept. Elect. Comput. Eng., Wayne State Univ., Detroit, MI, USA, 2014.
[37] S. M. O’Meara, M. C. Shyr, K. R. Lyons, and S. S. Joshi, “Comparing
two different cursor control methods which use single-site surface elec-
tromyography,” in Proc. 9th Int. IEEE/EMBS Conf. Neural Eng., 2019,
pp. 1163–1166.
[38] S. Deeny, C. Chicoine, L. Hargrove, T. Parrish, and A. Jayaraman, “A
simple ERP method for quantitative analysis of cognitive workload in
myoelectric prosthesis control and human-machine interaction,” PLoS
One, vol. 9, no. 11, 2014, Art. no. e112091.
[39] L. J. Hargrove, L. A. Miller, K. Turner, and T. A. Kuiken, “Myoelectric
pattern recognition outperforms direct control for transhumeral amputees
with targeted muscle reinnervation: A randomized clinical trial,Sci.
Rep., vol. 7, no. 1, 2017, Art. no. 13840.
[40] T. Kuiken, L. Miller, and K. Turner, “A comparison of direct control
and pattern recognition control in transhumeral TMR subjects,” in Proc.
ISPO World Congr., 2015, Art. no. 129.
[41] T. A. Kuiken, L. A. Miller, K. Turner, and L. J. Hargrove, “A comparison
of pattern recognition control and direct control of a multiple degree-of-
freedom transradial prosthesis,” IEEE J. Transl. Eng. Health Med.,vol.4,
Nov. 2016, Art. no. 2100508.
[42] L. Resnik, H. H. Huang, A. Winslow, D. L. Crouch, F. Zhang, and N.
Wolk, “Evaluation of EMG pattern recognition for upper limb prosthesis
control: A case study in comparison with direct myoelectric control,” J.
Neuroeng. Rehabil., vol. 15, no. 1, 2018, Art. no. 23.
[43] R.Rupp et al., “Think2grasp-bci-controlled neuroprosthesis for the upper
extremity,” Biomed. Eng./Biomedizinische Technik, vol. 58, 2013, doi:
10.1515/bmt-2013-4440.
[44] M. Connan, E. Ruiz Ramírez, B. Vodermayer, and C. Castellini, “Assess-
ment of a wearable force-and electromyography device and comparison
of the related signals for myocontrol,” Front. Neurorobot., vol. 10, 2016,
Art. no. 17.
[45] E. Ruiz Ramírez, Control of a Hand Prosthesis Using Mixed Electromyo-
graphy and Pressure Sensing. Barcelona, Spain: Universitat Politècnica
de Catalunya, 2016.
[46] J. Gonzalez, H. Suzuki, N. Natsumi, M. Sekine, and W. Yu, “Auditory
display as a prosthetic hand sensory feedback for reaching and grasping
tasks,” in Proc. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc., 2012,
pp. 1789–1792.
[47] J. Gonzalez and W. Yu, “Auditory display as an aid for prosthetic
hand manipulation: Preliminary results,” International Community for
Auditory Display, 2011.
[48] M.Markovic et al., “The clinical relevance of advanced artificial feedback
in the control of a multi-functional myoelectric prosthesis,” J. Neuroeng.
Rehabil., vol. 15, no. 1, 2018, Art. no. 28.
[49] J. A. Arenas, “Evaluation of a novel myoelectric training device,” M.Sc.
thesis, Dept. Biomed. Eng., Virginia Commonwealth Univ., Richmond,
VA, USA, 2015.
[50] K. Bark et al., “Effects of vibrotactile feedback on human learning of
arm motions,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 23, no. 1,
pp. 51–63, Jan. 2015.
[51] G. Chai, D. Zhang, and X. Zhu, “Developing non-somatotopic phantom
finger sensation to comparable levels of somatotopic sensation through
user training with electrotactile stimulation,” IEEE Trans. Neural Syst.
Rehabil. Eng., vol. 25, no. 5, pp. 469–480, May 2017.
[52] M. Saraiji, T. Sasaki, K. Kunze, K. Minamizawa, and M. Inami,
“MetaArmS: Body remapping using feet-controlled artificial arms,” in
Proc. 31st Annu. ACM Symp. User Interface Softw. Technol., 2018,
pp. 65–74.
[53] I. M. Shuggi et al., “Motor performance, mental workload and self-
efficacy dynamics during learning of reaching movements through-
out multiple practice sessions,” Neuroscience, vol. 423, pp. 232–248,
2019.
[54] M. Zahabi and D. Kaber, “Identification of task demands and usability
issues in police use of mobile computing terminals,” Appl. Ergonom.,
vol. 66, pp. 161–171, 2018.
[55] D. Moher, A. Liberati, J. Tetzlaff, and D. G. Altman, “Preferred reporting
items for systematic reviews and meta-analyses: The PRISMA state-
ment,” Ann. Int. Med., vol. 151, no. 4, pp. 264–269, 2009.
[56] J. C. F. de Winter, R. Happee, M. H. Martens, and N. A. Stanton,
“Effects of adaptive cruise control and highly automated driving on
workload and situation awareness: A review of the empirical evidence,”
Transp. Res. Part F: Traffic Psychol. Behav., vol. 27, pp. 196–217,
2014.
[57] J.-F. Gehanno, L. Rollin, and S. Darmoni, “Is the coverage of google
scholar enough to be used alone for systematic reviews,BMC Med.
Inform. Decis. Mak., vol. 13, no. 1, 2013, Art. no. 7.
Authorized licensed use limited to: Texas A M University. Downloaded on February 11,2022 at 16:28:01 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
14 IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS
[58] S. G. Hart and L. E. Staveland, Development of NASA-TLX (Task Load
Index): Results of Empirical and Theoretical Research, in Advances
in Psychology. Amsterdam, The Netherlands; New York, NY, USA:
Elsevier, 1988, pp. 139–183.
[59] S. Z. Shariff et al., “Retrieving clinical evidence: A comparison of
pubmed and google scholar for quick clinical searches,” J. Med. Internet
Res., vol. 15, no. 8, 2013, Art. no. e164.
[60] I. V. D. Ploeg, Prosthetic Bodies: The Construction of the Fetus and
the Couple As Patients in Reproductive Technologies. Berlin, Germany:
Springer Science & Business Media, 2001.
[61] Farlex, The free dictionary by Farlex. Accessed: May 20, 2020. [Online].
Available: https://www.thefreedictionary.com/prosthesis
[62] R. A. Miranda et al., “DARPA-funded efforts in the development of novel
brain–computer interface technologies,” J. Neurosci. Methods, vol. 244,
pp. 52–67, 2015.
[63] M. Burner, “The C-Leg and proprioception: How a microprocessor-
controlled knee prosthesis mimics the human body’s system of awareness
of position and production of movement,” Unpublished thesis, Malone
Univ. Honors Program, 2011.
[64] R. Volkmar, Semi-Autonomous Bimanual Prosthesis Control. Göttingen,
Germany: Georg-August-Universität Göttingen, 2017.
[65] R. Volkmar, S. Dosen, J. Gonzalez-Vargas, M. Baum, and M.
Markovic, “Improving bimanual interaction with a prosthesis using
semi-autonomous control,” J. Neuroeng. Rehabil., vol. 16, no. 1, 2019,
Art. no. 140.
[66] L. Berntsson, “Evaluation of cognitive workload using EEG: Investiga-
tion of how sensory feedback improves function of osseo-neuromuscular
upper limb prostheses,” 2019. [Online]. Available: http://urn.kb.se/
resolve?urn=urn:nbn:se:liu:diva-160183 DiVA database
[67] S. Deeny, M. Barstead, C. Chicoine, L. Hargrove, T. Parrish,
and A. Jayaraman, “EEG as an outcome measure for cognitive
workload during prosthetic use,” 2014. [Online]. Available: http:
//media.mycrowdwisdom.com.s3.amazonaws.com/aaop/Resources/
JOP/2014/2014- 15.pdf
[68] J. Parr, S. J. Vine, M. R. Wilson, N. Harrison, and G. Wood, “Visual
attention, EEG alpha power and T7-Fz connectivity are implicated in
prosthetic hand control and can be optimized through gaze training,” J.
Neuroeng. Rehabil., vol. 16, no. 1, 2019, Art. no. 52.
[69] A. L. Pruziner et al., “Biomechanical and neurocognitive performance
outcomes of walking with transtibial limb loss while challenged by a
concurrent task,” Exp. Brain Res., vol. 237, no. 2, pp. 477–491, 2019.
[70] I. M. Rezazadeh, M. Firoozabadi, H. Hu, and S. M. R. H. Golpayegani,
“Co-adaptive and affective human-machine interface for improving train-
ing performances of virtual myoelectric forearm prosthesis,” IEEE Trans.
Affect. Comput., vol. 3, no. 3, pp. 285–297, Jul.–Sep. 2012.
[71] I. M. Rezazadeh, S. Firoozabadi, S. H. Golpayegani, and H. Hu, “Con-
trolling a virtual forehand prosthesis using an adaptive and affective
human-machine interface,” in Proc. Annu. Int. Conf. IEEE Eng. Med.
Biol. Soc., 2011, pp. 4128–4131.
[72] E. P. Shaw et al., “A comparison of mental workload in individuals
with transtibial and transfemoral lower limb loss during dual-task walk-
ing under varying demand,J. Int. Neuropsychol. Soc., vol. 25, no. 9,
pp. 985–997, 2019.
[73] E. P. Shaw et al., “Cerebral cortical networking for mental workload
assessment under various demands during dual-task walking,Exp. Brain
Res., vol. 237, no. 9, pp. 2279–2295, 2019.
[74] M. L. Davidson, Development of a Novel Prosthetic Wrist Device Incor-
porating the Dart Thrower’s Motion. Denver, CO, USA: Univ. Colorado
Denver, 2017.
[75] G. de Andrade and D. Thamilys, “Development and evaluation of
human-machine interfaces for the control and actuation of upper-limb
prostheses: Desenvolvimento e avaliação de interfaces homem-máquina
para o controle e atuação de próteses de membros superiores,” 2018.
[76] N. Garcia-Hernandez, K. Garza-Martinez, V. Parra-Vega, A. Alvarez-
Sanchez, and L. Conchas-Arteaga, “Development of an EMG-based
exergaming system for isometric muscle training and its effectiveness to
enhance motivation, performance and muscle strength,Int. J. Human-
Comput. Stud., vol. 124, pp. 44–55, 2019.
[77] R. J. Gentili, I. M. Shuggi, K. M. King, H. Oh, and P. A. Shewokis,
“Cognitive-motor processes during arm reaching performance through a
human body-machine interface,” in Proc. Int. Conf. Augmented Cogni-
tion, 2015, pp. 381–392.
[78] R. J. Gentili et al., “Human-robotic collaborative intelligent control for
reaching performance,” in Proc. Int. Conf. Augmented Cogn., 2013, pp.
666–675.
[79] N. R. Olsen et al., “An adaptable prosthetic wrist reduces subjective
workload,” bioRxiv, 2019.
[80] I. M. Shuggi, “The effect of a safety controller on user performance
through a prosthetic interface,” Ph.D. dissertation, Univ. Maryland, Col-
lege Park, 2014.
[81] I. M. Shuggi, H. Oh, P. A. Shewokis, and R. J. Gentili, “Mental workload
and motor performance dynamics during practice of reaching move-
ments under various levels of task difficulty,” Neuroscience, vol. 360,
pp. 166–179, 2017.
[82] I. M. Shuggi, P. A. Shewokis, J. W. Herrmann, and R. J. Gentili, “Changes
in motor performance and mental workload during practice of reaching
movements: A team dynamics perspective,Exp. Brain Res., vol. 236,
no. 2, pp. 433–451, 2018.
[83] L.Hargrove, L. Miller, K. Turner, and T.Kuiken, “Control within a virtual
environment is correlated to functional outcomes when using a physical
prosthesis,” J. Neuroeng. Rehabil., vol. 15, 2018, Art. no. 60.
[84] S. Morgan, V. Kelly, and B. Hafner, “The impact of transfemoral
amputation on the cognitive load associated with level walking,” in
Proc. 40th Annu. Acad. Meeting AAOP, Chicago, IL, USA, Feb. 2014.
[Online]. Available: http://media.mycrowdwisdom.com.s3.amazonaws.
com/aaop/Resources/ JOP/2014/2014-40.pdf
[85] E.Raveh, J. Friedman, and S. Portnoy, “Evaluation of the effects of adding
vibrotactile feedback to myoelectric prosthesis users on performance and
visual attention in a dual-task paradigm,” Clin. Rehabil., vol. 32, no. 10,
pp. 1308–1316, 2018.
[86] D.Cardona, G. Maldonado, V. Ferman, A. Lemus, and J. Fajardo,“Impact
of diverse aspects in user-prosthesis interfaces for myoelectric Upper-
limb prostheses,” in Proc. 8th IEEE RAS/EMBS Int. Conf. Biomed. Robot.
Biomechatron., 2020, pp. 954–960.
[87] J. Fajardo, V. Ferman, D. Cardona, G. Maldonado, A. Lemus, and E.
Rohmer, “Galileo hand: An anthropomorphic and affordable upper-limb
prosthesis,” IEEE Access, vol. 8, pp. 81365–81377, 2020.
[88] M. D. Paskett et al., “Activities of daily living with bionic arm improved
by combination training and latching filter in prosthesis control compar-
ison,” J. Neuroeng. Rehabil., vol. 18, 2020, Art. no. 45.
[89] J. Schack, A. H. Pripp, P. Mirtaheri, H. Steen, E. Güler, and T. Gjøvaag,
“Increased prefrontal cortical activation during challenging walking con-
ditions in persons with lower limb amputation—An fNIRS observational
study,” Physiotherapy Theory Pract., vol. 38, pp. 255–265, 2022.
[90] C. Bloomer, S. Wang, and K. Kontson, “Kinematic analysis of motor
learning in upper limb body-powered bypass prosthesis training,PLoS
One, vol. 15, no. 1, 2020, Art. no. e0226563.
[91] J. C. Valentine, T. D. Pigott, and H. R. Rothstein, “How many studies
do you need? A primer on statistical power for meta-analysis,” J. Educ.
Behav. Statist., vol. 35, no. 2, pp. 215–247, 2010.
[92] S. G. Hart, “NASA-task load index (NASA-TLX); 20 years later,” in
Proc. Hum. Factors Ergonom. Soc. Annu. Meeting, 2006, pp. 904–908.
[93] C. D. Wickens, “Multiple resources and mental workload,”Hum. Factors,
vol. 50, no. 3, pp. 449–455, 2008.
[94] J. P. Higgins et al.,Cochrane Handbook for Systematic Reviews of
Interventions. Chichester, U.K.: Wiley, 2019.
[95] J. J. Deeks and J. P. Higgins, “Statistical algorithms in review manager
5,” Statist. Methods Group Cochrane Collaboration, vol. 1, no. 11, 2010.
[96] B. Zhang, J. de Winter, S. Varotto, R. Happee, and M. Martens, “De-
terminants of take-over time from automated driving: A meta-analysis
of 129 studies,” Transp. Res. Part F: Traffic Psychol. Behav., vol. 64,
pp. 285–307, 2019.
[97] J. T. Belter, “Mechanical design and performance specifications of an-
thropomorphic prosthetic hands: A review,” J. Rehabil. Res. Develop.,
vol. 50, no. 5, 2016, Art. no. 599.
[98] M. A. Oskoei and H. Hu, “Myoelectric control systems—A survey,”
Biomed. Signal Process. Control, vol. 2, no. 4, pp. 275–294, 2007.
[99] M. Zecca, S. Micera, M. C. Carrozza, and P. Dario, “Control of multi-
functional prosthetic hands by processing the electromyographic signal,”
Crit. Rev. Biomed. Eng., vol. 30, no. 4–6, pp. 459–485, 2002.
[100] R. Leeb, L. Tonin, M. Rohm, L. Desideri, T. Carlson, and J. D. R. Millán,
“Towards independence: A BCI telepresence robot for people with severe
motor disabilities,” Proc. IEEE, vol. 103, no. 6, pp. 969–982, Jun. 2015.
[101] K. Knaepen et al., “Psychophysiological response to cognitive workload
during symmetrical, asymmetrical and dual-task walking,” Hum. Move-
ment Sci., vol. 40, pp. 248–263, 2015.
[102] M. Zahabia, T. Shupskyb, and A. Lymanb, “Analysis of law enforcement
mobile computer terminal interface,” in Proc. IISE Annu. Conf., Orlando,
FL, USA, 2019.
Authorized licensed use limited to: Texas A M University. Downloaded on February 11,2022 at 16:28:01 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
PARK AND ZAHABI: COGNITIVE WORKLOAD ASSESSMENT OF PROSTHETIC DEVICES 15
[103] M. Zahabi and E. McCollum, “An application of machine learning for
police mobile computer terminal usability evaluation,” in Proc. Hum.
Factors Ergonom. Soc. Annu. Meeting, 2019, pp. 1194–1198.
[104] M. Zahabi and A. Lyman, “Impact of electronic medical records on
patient-provider communication,” in Proc. Int. Symp. Hum. Factors
Ergonom. Health Care, 2019, pp. 23–27.
[105] R. M. Stanley, B. G. Hall, W. A. Baden, and M. Exum, “Analyzing air
traffic controller task times and capacity benefits from new automation
capabilities,” in Proc. Integr. Commun., Navigation Surveill. Conf., 2019,
pp. 1–8.
[106] U. Rosyidah, H. Haryanto, and A. Kardianawati, “Usability evalu-
ation using GOMS model for education game ‘Play and learn en-
glish’,” in Proc. Int. Seminar Appl. Technol. Inf. Commun., 2019,
pp. 1–5.
[107] L. R. Prada and D. A. Boehm-Davis, “GOMS on the flight deck: A case
study of the boeing 777 MCP,” in Proc. Hum. Factors Ergonom. Soc.
Annu. Meeting, 2004, pp. 104–107.
[108] A. Din, “Usable security using GOMS: A study to evaluate and compare
the usability of user accounts on E-Government websites,” Ph.D. disser-
tation, Graduate School Comput. Inf. Sci., Nova Southeastern Univ., Fort
Lauderdale, FL, USA, 2015.
[109] J. Park and M. Zahabi, “Comparison of cognitive workload assessment
techniques in EMG-based prosthetic device studies,” in Proc. IEEE Int.
Conf. Syst., Man, Cybern., 2020, pp. 1242–1248.
[110] R. L. Charles and J. Nixon, “Measuring mental workload using phys-
iological measures: A systematic review,” Appl. Ergonom., vol. 74,
pp. 221–232, 2019.
[111] Y.-Y. Yeh and C. D. Wickens, “Dissociation of performance and subjec-
tive measures of workload,Hum. Factors, vol. 30, no. 1, pp. 111–120,
1988.
[112] Y. Zhang and C. Wu, “Learn to integrate mathematical models in human
performance modeling,” in Proc. Hum. Factors Ergonom. Soc. Annu.
Meeting, 2017, pp. 776–780.
[113] M.Markovic et al., “The clinical relevance of advanced artificial feedback
in the control of a multi-functional myoelectric prosthesis,” J. Neuroeng.
Rehabil., vol. 15, no. 1, pp. 1–15, 2018.
[114] A. H. Mason and J. L. Bruyn, “Manual asymmetries in bimanual pre-
hension tasks: Manipulation of object size and object distance,” Hum.
Movement Sci., vol. 28, no. 1, pp. 48–73, 2009.
[115] K. Finstad, “The usability metric for user experience, Interact. Comput.,
vol. 22, no. 5, pp. 323–327, 2010.
[116] J. Nielsen, “Ten usability heuristics,” 2005. [Online]. Available: http:
//www. nngroup.com/articles/ten- usability-heuristics/
[117] M. Zahabi et al., “Driver performance and attention allocation in use of
logo signs on freeway exit ramps,Appl. Ergonom., vol. 65, pp. 70–80,
2017.
[118] R. S. Johansson, G. Westling, A. Bäckström, and J. R. Flanagan, “Eye–
hand coordination in object manipulation,” J. Neurosci., vol. 21, no. 17,
pp. 6917–6932, 2001.
[119] M.K. Holden, “Virtual environments for motor rehabilitation,”Cyberpsy-
chol. Behav., vol. 8, no. 3, pp. 187–211, 2005.
[120] M. D. Serruya and M. J. Kahana, “Techniques and devices to restore
cognition,” Behav. Brain Res., vol. 192, no. 2, pp. 149–165, 2008.
[121] Q. H. Mach, M. D. Hunter, and R. S. Grewal, “Neurophysiological cor-
relates in interface design: An HCI perspective,Comput. Hum. Behav.,
vol. 26, no. 3, pp. 371–376, 2010.
[122] N. Bevana, J. Kirakowskib, and J. Maissela, “What is usability,” in Proc.
4th Int. Conf. Hum. Comput. Interact., 1991.
[123] J. Butkus, C. Dennison, A. Orr, and M. S. Laurent, “Occupational ther-
apy with the military upper extremity amputee: Advances and research
implications,” Curr. Phys. Med. Rehabil. Rep., vol. 2, no. 4, pp. 255–262,
2014.
Junho Park (Student Member, IEEE) received the
B.S. degree in industrial engineering (minor: psychol-
ogy) from Yonsei University, Seoul, South Korea, in
2008, and the M.S. degree in industrial engineering
from Seoul National University, Seoul, South Korea
in 2010. He is currently working toward the Ph.D.
degree in human-systems engineering at the Industrial
and Systems Engineering Department, Texas A&M
University, College Station, TX, USA.
From 2010 to 2015, he was a Researcher with
Korea Institute for Defense Analysis in Center for
Military Planning Division. From 2015 to 2019, he was an Associate Research
Fellow, he conducted force requirement analysis and evaluation research and
cognitive decision-making modeling studies as a Principal Investigator. His
research interests include human factors in assistive technologies and cognitive
performance modeling.
Maryam Zahabi (Member, IEEE) received the B.S.
and M.S. degrees in industrial and systems engineer-
ing from Sharif University of Technology, Tehran,
Iran, in 2011 and 2013, respectively, and the Ph.D.
degree in industrial and systems engineering (mi-
nor: statistics) from North Carolina State University,
Raleigh, NC, USA, in 2017.
She is currently an Assistant Professor with the In-
dustrial and Systems Engineering Department, Texas
A&M University, College Station, TX, USA. Her
research interests include applying human systems
engineering theories and principles in design and analysis of complex human-
in-the-loop systems. In particular, she is interested in usability evaluation and
cognitive load assessment of assistive technologies and cognitive performance
modeling applications in different domains.
Authorized licensed use limited to: Texas A M University. Downloaded on February 11,2022 at 16:28:01 UTC from IEEE Xplore. Restrictions apply.
... Petticrew & Roberts [27] state the first step in the generation of an SLR is to identify if a revision of a particular topic is really needed. Several systematic literature reviews have analyzed and illustrated ATs, for instance, navigation and walking assistants focused on persons with visual disabilities or impairments [28], [29], ambient assisted living solutions for elderly people [30], or prosthetics devices [31]. Then, this SLR searches to complement the previous studies, in the sense to map and analyze the role and implications of the OSHW and OSS in the development of ATs that are accessible to disabled people and contribute in part with the findings of this study to achieve the statements posed in the UN CRPD convention. ...
Article
Full-text available
Disabled People deal with a series of barriers that limit their inclusion, empowerment, well-being, and role in society with a special emphasis in low and medium-income countries. One of these barriers is concerning the accessibility and affordability of assistive technologies (ATs) that help to enhance the quality of life of these persons. In this context, this systematic literature review (SLR) analyzes and describes how free and open-source hardware (OSHW) and open software (OSS) are employed in the design, development, and deployment of low-cost ATs. In the SLR process, different ATs were analyzed for disabilities such as visual, mobility, upper body, prostheses, hearing & speaking, daily living, and participation in society. The ATs were designed with diverse OSHW and OSS technologies such as Arduino, Raspberry Pi, NVidia Jetson, OpenCV, YOLO, MobileNet, EEG and EMG signal conditioning devices, actuators, and sensors such as ultrasonic, LiDar, or flex. 809 studies were collected and analyzed from the database Web of Science, GitHub, and the specialized journals in OSHW HardwareX and the Journal of Open Hardware during the years 2013-2022. In the first part of the SLR, the bibliometric trends and topic clusters regarding the selected studies are described. Secondly, the ATs identified with open source technologies, e.g., sensor-based or computer vision-based, are described along with a complete state-of-art about these based on each disability recognized. Finally, the issues and challenges to this approach are explored including technical factors, documentation, government policies, and the inclusion of disabled people in open source co-creation. The purpose of this study is to inform practitioners, designers, or stakeholders about low-cost (frugal) ATs with OSHW and OSS, and thus promote their development, accessibility, and affordability, contributing to benefit the community of disabled people.
Article
Full-text available
Background Advanced prostheses can restore function and improve quality of life for individuals with amputations. Unfortunately, most commercial control strategies do not fully utilize the rich control information from residual nerves and musculature. Continuous decoders can provide more intuitive prosthesis control using multi-channel neural or electromyographic recordings. Three components influence continuous decoder performance: the data used to train the algorithm, the algorithm, and smoothing filters on the algorithm’s output. Individual groups often focus on a single decoder, so very few studies compare different decoders using otherwise similar experimental conditions. Methods We completed a two-phase, head-to-head comparison of 12 continuous decoders using activities of daily living. In phase one, we compared two training types and a smoothing filter with three algorithms (modified Kalman filter, multi-layer perceptron, and convolutional neural network) in a clothespin relocation task. We compared training types that included only individual digit and wrist movements vs. combination movements (e.g., simultaneous grasp and wrist flexion). We also compared raw vs. nonlinearly smoothed algorithm outputs. In phase two, we compared the three algorithms in fragile egg, zipping, pouring, and folding tasks using the combination training and smoothing found beneficial in phase one. In both phases, we collected objective, performance-based (e.g., success rate), and subjective, user-focused (e.g., preference) measures. Results Phase one showed that combination training improved prosthesis control accuracy and speed, and that the nonlinear smoothing improved accuracy but generally reduced speed. Phase one importantly showed simultaneous movements were used in the task, and that the modified Kalman filter and multi-layer perceptron predicted more simultaneous movements than the convolutional neural network. In phase two, user-focused metrics favored the convolutional neural network and modified Kalman filter, whereas performance-based metrics were generally similar among all algorithms. Conclusions These results confirm that state-of-the-art algorithms, whether linear or nonlinear in nature, functionally benefit from training on more complex data and from output smoothing. These studies will be used to select a decoder for a long-term take-home trial with implanted neuromyoelectric devices. Overall, clinical considerations may favor the mKF as it is similar in performance, faster to train, and computationally less expensive than neural networks.
Conference Paper
Full-text available
Abstract— Previous studies have found that electromyography (EMG)-based prosthetic devices provide higher grasping force, increase functional performance, and have greater range of motion over conventional prostheses. However, cognitive workload (CW) is still one of the issues that can negatively affect device usability and satisfaction. In order to evaluate CW of prosthetic devices early in the design cycle, it is first necessary to select the most appropriate measures. Therefore, the objectives of this study were to: (1) review the CW measurement techniques used in prior EMG-based prosthetic device evaluations; and (2) provide guidelines to select the most appropriate measurement techniques. The findings suggested that cognitive performance models (CPM), subjective measures, task performance measures, and some physiological measures were sensitive in detecting CW differences among prosthetic device configurations and therefore could be useful tools in usability evaluation of these technologies. However, in order to reduce intrusiveness and cost, methods such as subjective workload measures, task performance, and CPM are more beneficial as compared to physiological measurements. Guidelines proposed in this study can be beneficial to select the most appropriate CW measurement techniques in order to improve sensitivity and accuracy and reduce intrusiveness and cost.
Conference Paper
Full-text available
Numerous assistive devices possess complex ways to operate and interact with the subjects, influencing patients to shed them from their activities of daily living. With the purpose of presenting a better solution to mitigate issues generated by complex or expensive alternatives, a test comparing different user-prosthesis interfaces was elaborated to determine the effects of diverse aspects in their user-friendliness, including that of a version created for this work. A simplistic, anthropomorphic and 3D-printed upper-limb prosthesis was adapted to evaluate all the renditions considered. The chosen design facilitates the modification of its operational mode, facilitating running the tests. Additionally, the selected prosthetic device can easily be adapted to the amputees’ lifestyle in a successful way, as shown by experimental results, providing validity to the study. For the interaction process, a wireless third party device was elected to gather the user intent and, in some renditions, to work in tandem with some sort of visual feedback or with a multimodal alternative to verify their impact on the user.
Article
Full-text available
Background: Lower limb amputation (LLA) alters the sensorimotor control systems. Despite the self-reports of increased attention during mobility, the interaction between mobility and cognitive control mechanisms is not fully understood. Objective: Concurrently evaluate walking performance and prefrontal cortical (PFC) activity in persons with and without LLA during different walking conditions. Methods: Thirty-nine persons with LLA and thirty-three able-bodied controls participated. Walking performance was evaluated using the Figure-of 8-walk-test during three conditions: 1) UW (Usual walking with self-selected walking speed); 2) WCT (walking and carrying a tray with two cups filled with water); and 3) WUT (walking on uneven terrain). PFC activity was assessed using functional near-infrared spectroscopy (fNIRS). Linear mixed models were used to detect changes between groups and between walking conditions within each group. Results: Between-group comparisons showed increased PFC activity in persons with LLA during UW and WUT, and a significant decrease in walking performance during WCT and WUT compared to controls. Within-group comparisons showed increased PFC activity during WUT compared with UW and WCT and an overall difference in walking performance between the conditions (WU > WUT > WCT) in both groups. However, the effect of walking condition on PFC activity and walking performance was not modified by group (P > .1). Conclusion: The results suggest that persons with LLA have increased attentional demands during walking but choose the same cognitive-mobility strategy during challenging walking conditions as able-bodied persons. However, the attentional demands seem to depend on the complexity of the task.
Article
Full-text available
The strict development processes of commercial upper-limb prostheses and the complexity of research projects required for their development makes them expensive for end users, both in terms of acquisition and maintenance. Moreover, many of them possess complex ways to operate and interact with the subjects, influencing patients to not favor these devices and shed them from their activities of daily living. The advent of 3D printers allows for distributed open-source research projects that follow new design principles; these consider simplicity without neglecting performance in terms of grasping capabilities, power consumption and controllability. In this work, a simple, yet functional design based on 3D printing is proposed, with the aim to reduce costs and manufacturing time. The operation process consists in interpreting the user intent with electromyography electrodes, while providing visual feedback through a μLCD screen. Its modular, parametric and self-contained design is intended to aid people with different transradial amputation levels, despite of the socket’s constitution. This approach allows for easy updates of the system and demands a low cognitive effort from the user, satisfying a trade-off between functionality and low cost. It also grants an easy customization of the amount and selection of available actions, as well as the sensors used for gathering the user intent, permitting alterations to fit the patients’ unique needs. Furthermore, experimental results showed an apt mechanical performance when interacting with everyday life objects, in addition to a highly accurate and responsive controller; this also applies for the user-prosthesis interface.
Article
Full-text available
Motor learning and compensatory movement are important aspects of prosthesis training yet relatively little quantitative evidence supports our current understanding of how motor control and compensation develop in the novel body-powered prosthesis user. The goal of this study is to assess these aspects of prosthesis training through functional, kinematic, and kinetic analyses using a within-subject paradigm compared across two training time points. The joints evaluated include the left and right shoulders, torso, and right elbow. Six abled-bodied subjects (age 27 ± 3) using a body-powered bypass prosthesis completed the Jebsen-Taylor Hand Function Test and the targeted Box and Blocks Test after five training sessions and again after ten sessions. Significant differences in movement parameters included reduced times to complete tasks, reduced normalized jerk for most joints and tasks, and more variable changes in efficiency and compensation parameters for individual tasks and joints measured as range of motion, maximum angle, and average moment. Normalized jerk, joint specific path length, range of motion, maximum angle, and average moment are presented for the first time in this unique training context and for this specific device type. These findings quantitatively describe numerous aspects of motor learning and control in able-bodied subjects that may be useful in guiding future rehabilitation and training of body-powered prosthesis users.
Article
Full-text available
Background: The loss of a hand is a traumatic experience that substantially compromises an individual's capability to interact with his environment. The myoelectric prostheses are state-of-the-art (SoA) functional replacements for the lost limbs. Their overall mechanical design and dexterity have improved over the last few decades, but the users have not been able to fully exploit these advances because of the lack of effective and intuitive control. Bimanual tasks are particularly challenging for an amputee since prosthesis control needs to be coordinated with the movement of the sound limb. So far, the bimanual activities have been often neglected by the prosthetic research community. Methods: We present a novel method to prosthesis control, which uses a semi-autonomous approach in order to simplify bimanual interactions. The approach supplements the commercial SoA two-channel myoelectric control with two additional sensors. Two inertial measurement units were attached to the prosthesis and the sound hand to detect the movement of both limbs. Once a bimanual interaction is detected, the system mimics the coordination strategies of able-bodied subjects to automatically adjust the prosthesis wrist rotation (pronation, supination) and grip type (lateral, palmar) to assist the sound hand during a bimanual task. The system has been evaluated in eight able-bodied subjects performing functional uni- and bi-manual tasks using the novel method and SoA two-channel myocontrol. The outcome measures were time to accomplish the task, semi-autonomous system misclassification rate, subjective rating of intuitiveness, and perceived workload (NASA TLX). Results: The results demonstrated that the novel control interface substantially outperformed the SoA myoelectric control. While using the semi-autonomous control the time to accomplish the task and the perceived workload decreased for 25 and 27%, respectively, while the subjects rated the system as more intuitive then SoA myocontrol. Conclusions: The novel system uses minimal additional hardware (two inertial sensors) and simple processing and it is therefore convenient for practical implementation. By using the proposed control scheme, the prosthesis assists the user's sound hand in performing bimanual interactions while decreasing cognitive burden.
Article
An important question in automated driving research is how quickly drivers take over control of the vehicle in response to a critical event or a take-over request. Although a large number of studies have been performed, results vary strongly. In this study, we investigated mean take-over times from 129 studies with SAE level 2 automation or higher. We used three complementary approaches: (1) a within-study analysis, in which differences in mean take-over time were assessed for pairs of experimental conditions, (2) a between-study analysis, in which correlations between experimental conditions and mean take-over times were assessed, and (3) a linear mixed-effects model combining between-study and within-study effects. The three methods showed that a shorter mean take-over time is associated with a higher urgency of the situation, not using a handheld device, not performing a visual non-driving task, having experienced another take-over scenario before in the experiment, and receiving an auditory or vibrotactile take-over request as compared to a visual-only or no take-over request. A consistent effect of age was not observed. We also found the mean and standard deviation of the take-over time were highly correlated, indicating that the mean is predictive of variability. Our findings point to directions for new research, in particular concerning the distinction between drivers’ ability and motivation to take over, and the roles of urgency and prior experience.
Article
Police in-vehicle technologies especially mobile computer terminals (MCTs) are the major cause of motor vehicle crashes for law enforcement officers. Previous studies have found several usability issues with the design of MCT interfaces. The objective of this study was to develop an algorithm for classification of MCT interface usability based on a combination of officer performance, visual attention allocation, and subjective measures. It was found that speed deviation, secondary task completion time, off-road fixation frequency and glance duration are informative features that associate with MCT usability. The developed algorithm has the potential to be used in MCT design and development process to ensure efficient interaction in multi-tasking situations of driving and using the MCT.