Article

Development of NASA-TLX results of empirical and theoretical research

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... For demand (including physical and cognitive) and overall performance we adopt the validated 6-item NASA-TLX scale [24]. For technological compatibility with participants' existing literature review workflows and the easiness of learning we adapted the Technology Acceptance Model survey from [66] (5 items). ...
... Overall WorkloadSum of the participants' responses to the five NASA-TLX's[24] 21-point scale questionnaire items below.FlowSum of the participants' responses to the 11 questionnaire items adopted from Webster et al.[65] measuring the flow aspect of participants' interaction with the system.TAMSum of the participants' responses to the 5 questionnaire items adopted from[66] measuring the technological compatibility with participants' existing literature review workflows and the easiness of learning.Confidence "Using the system increased my confidence in conducting literature review. (The response Likert scales for this question and below are 1: Strongly disagree, 7: Strongly agree)" ...
Preprint
Full-text available
Reviewing the literature to understand relevant threads of past work is a critical part of research and vehicle for learning. However, as the scientific literature grows the challenges for users to find and make sense of the many different threads of research grow as well. Previous work has helped scholars to find and group papers with citation information or textual similarity using standalone tools or overview visualizations. Instead, in this work we explore a tool integrated into users' reading process that helps them with leveraging authors' existing summarization of threads, typically in introduction or related work sections, in order to situate their own work's contributions. To explore this we developed a prototype that supports efficient extraction and organization of threads along with supporting evidence as scientists read research articles. The system then recommends further relevant articles based on user-created threads. We evaluate the system in a lab study and find that it helps scientists to follow and curate research threads without breaking out of their flow of reading, collect relevant papers and clips, and discover interesting new articles to further grow threads.
... There are too many techniques for cognitive load measurements. The first is the subjective rating scale with techniques such as the NASA Task Load Index [16] or 9 point symmetric category mental effort rating scale [17]. The second is an objective cognitive load measurement technique that allows online assessment of cognitive load at all levels. ...
... Participants were asked not to take alcohol and caffeine 24 hours before the experiment. After the experiment, participants were asked to fill in the NASA-TLX [16] to assess their workload subjectively. ...
Article
With the development of technology, augmented reality (AR) devices were introduced in many areas. One of these devices, AR glasses, is more convenient and easy to fulfil and still continuing to evolve. The AR glasses, which are frequently used in sectors such as education, entertainment, construction and automotive, bring many research areas. In this article, the cognitive load that AR glasses bring to the users who is working on an assembly line has been investigated. Cognitive load refers to the resources used by working memory in the brain. This study was carried out with healthy participants in the assembly line of an automotive manufacturing factory and 60 tests were performed for measurement. Effect of the AR glasses for participants under 35 and over 35 years of age was measured by electroencephalography (EEG). EEG is a common objective technique used in cognitive load measurements. EEG data collected were examined and no significant difference was observed between participants of under and over 35 years of age. When compared with the data obtained in the experiments without glasses, it was seen that the use of the AR glasses in assembly lines of automotive manufacturing factories did not create an additional cognitive load. It can be seen from the article that AR glasses did not have an age-related effect and can be use in the automotive manufacturing industry.
... The following information was recorded via entry into a recording diary each evening before bed, along with potential difficulties with data collection procedures that day. Subjective load was measured via the NASA Task Load Index (NASA-TLX) [38], consisting of one question for each of six subscales measuring levels of mental, physical, and temporal demands, performance, effort and frustration along a continuum, with an integer value of 0-100 in 5-point increments. The NASA-TLX is sensitive for the assessment of cognitive demands of a given workload [38,39]. ...
... Subjective load was measured via the NASA Task Load Index (NASA-TLX) [38], consisting of one question for each of six subscales measuring levels of mental, physical, and temporal demands, performance, effort and frustration along a continuum, with an integer value of 0-100 in 5-point increments. The NASA-TLX is sensitive for the assessment of cognitive demands of a given workload [38,39]. Participants' subjective ratings of fatigue (presleep and post-sleep) were assessed via the Samn-Perelli Fatigue Scale, scored on a sevenpoint Likert scale [40]. ...
Article
Full-text available
Ensuring a balance between training demands and recovery during basic military training (BMT) is necessary for avoiding maladaptive training responses (e.g., illness or injury). These can lead to delays in training completion and to training attrition. Previously identified predictors of injury and attrition during BMT include demographic and performance data, which are typically collected at a single time point. The aim of this study was to determine individual risk factors for injury and training delays from a suite of measures collected across BMT. A total of 46 male and female recruits undertaking the 12-week Australian Army BMT course consented to this study. Injury, illness, attrition, and demographic data were collected across BMT. Objective measures included salivary cortisol and testosterone, step counts, cardiorespiratory fitness, and muscular endurance. Perceptions of well-being, recovery, workload, fatigue, and sleep were assessed with questionnaires. Baseline and mean scores across BMT were evaluated as predictors of injury and attrition using generalized linear regressions, while repeated-measures ANOVA was used for the group comparisons. From the 46 recruits, 36 recruits completed BMT on time; 10 were delayed in completion or discharged. Multiple risk factors for injury during BMT included higher subjective ratings of training load, fatigue, and stress, lower sleep quality, and higher cortisol concentrations. Higher ratings of depression, anxiety, and stress, and more injuries were associated with a higher risk of delayed completion. Higher concentrations of testosterone and higher levels of fitness upon entry to BMT were associated with reduced risk of injury and delayed completion of BMT. Ongoing monitoring with a suite of easily administered measures may have utility in forewarning risk of training maladaptation in recruits and may complement strategies to address previously identified demographic and performance-based risk factors to mitigate injury, training delays, and attrition.
... It is hypothesised that the task is less demanding for remote participants in terms of perceived workload than undertaking this task locally. One of the well-established ways to measure the MWL is the NASA-Task Load Index (NASA-TLX) [5], which is based on six sub-scales associated with the MWL, namely mental, physical, and temporal demand (related to demands imposed on the human), as well as performance, effort and frustration (related to interaction of the human with the task). The rest of the paper is structured as follows: a brief literature review is presented in Section 2. Section 3 outlines the methodology that was applied for this study, including a description of the task, recruited subjects and experimental setup. ...
... In total, 282 change-cycles related to this experiment's ramp-up process were obtained from 16 participants, where 104 were carried out in Group A and 178 in Group B. To obtain the perceived workload estimations for the ramp-up experience during this experiment and help assess the proposed decision-support's usability and effectiveness from the participants' viewpoint, the widely accepted and used NASA-Task Load Index (NASA-TLX) [5] was employed in its raw or unweighted form in an online questionnaire. The index takes into consideration the six various subscale categories mental demand, physical demand, temporal demand, performance, effort, and frustration, which are scored by the individual directly after performing the task on a scale from 0 (good) to 20 (poor) for performance and 0 (low) to 20 (high) for the other categories. ...
Article
A new Coronavirus disease 2019 (COVID-19) has spread globally since 2019. Consequently, businesses from different sectors were forced to work remotely. At the same time, research in this area has seen a rise in studying and emerging technologies that allow and promote such a remote working style; not every sector is equipped for such a transition. The manufacturing sector especially, has faced challenges in this respect. This paper investigates the mental workload (MWL) of two groups of participants through a human-machine interaction task. Participants were required to bring a robotised cell to full production by tuning system and dispensing process parameters. Following the experiment, a self-assessment of the participants' perceived MWL using the raw NASA Task Load Index (RTLX) was collected. The results reveal that remote participants tend to have a lower perceived workload compared to the local participants, but mental demand was deemed higher while performance was rated lower.
... When participants arrived, the experimenter gave an overview of the experiment, and they practiced to be familiar with the rules and controls. Then, they were trained to use the computerized version of the NASA-Task Load Index (Hart and Staveland 1988) to report their perceived workload in the experiment. ...
... Only the overall workload score was of interest for this experiment (Hart and Staveland 1988). ...
Article
Interruption at work by social media (SM) is a pervasive phenomenon. This study investigated the impact of SM interruptions and task cognitive levels on mental workload (MWL) and physiological indexes. Each subject performed six simulated computer tasks differentiated by two factors: task cognitive level and performing condition. MWL was reflected through three categories of data: perceived mental workload, physiological indexes, and primary task performance. The results revealed significant effects of SM interruptions on heart rate, low-frequency/high-frequency (LF/HF) ratio, and skin conductance. ANOVA results showed there were main effects of task cognitive level on LF/HF and skin conductance. These effects during interrupted tasks were more profound. In addition, participants experienced higher MWL and recorded lower primary task performance in the knowledge-based task than the rule- and skill-based tasks. Our findings can guide managers and employees regarding appropriate use of SM in the workplace and better managing interruption and workload.
... fewer keystrokes to access the desired application controls compared to the status quo condition without AccessBolt. Furthermore, the participants also indicated that they experienced a much lighter interaction workload with AccessBolt -nearly 4 times lower NASA-TLX [20] score compared to status quo. In sum, our contributions in this paper are: ...
... After completing each task, the participants were asked to answer an SEQ (Single Ease Question) 4 , on a scale of 1(hard) to 7(easy). The participants were also administered the NASA-TLX [20] questionnaire to estimate their perceived workload after each study condition. Each study lasted for nearly 2 hours, and the participants were given monetary compensation for their time and contribution. ...
... A one-minute break between each condition was provided. Subjective workload due to the perturbation -based protocol was assessed by NASA TLX, a subjective multidimensional assessment tool that rates perceived workload to assess a task [33]. ...
... Acceptability was defined as how the individuals reacted to the protocol [34]. We evaluated this by examining: (1) adherence to the protocol, which was defined as if participants completed the study session or not; (2) number perturbations trials performed for each participant (to show that each participant received similar exposure to all conditions); and (3) how mentally and physically demanding was the protocol for the participants through NASA TLX [33]. Practicality is the extent to which an intervention can be delivered when resources, time, and/or participant commitment are constrained in some way [34]. ...
Article
This study investigated the feasibility of a perturbation-based balance protocol that incorporates a novel computer-controlled movable platform, the Surefooted Trainer, to induce losses of balance during overground walking under various environmental conditions. Twenty apparently healthy older adults (66.7 ± years old) participated in this study. The acceptability and safety of the perturbation-based balance protocol were assessed by tracking adherence, adverse events, and subjective physical and mental demands after the intervention. Additionally, biomechanical variables during perturbed and non-perturbed trials were analyzed and compared with behavioral outcomes. Overall, 95% of the participants completed the study. There were no serious or non-serious adverse events. The margin of stability and step length after perturbations were significantly lower during slip-perturbations in which the environmental conditions were more challenging. For trip-perturbation conditions, the maximum trunk angle was higher during the trials that resulted in losses of balance. We conclude that the Surefooted Trainer is an acceptable and valid device for an overground walking perturbation-based assessment and training protocol in older adults.
... Two different measures of workload will be used. The first measure of participants' perceived workload is the NASA task load index (TLX) (Hart and Staveland 1988). The NASA-TLX asks the participant to rate their level of subjective workload during the experiment. ...
... Participants' perceived workload was evaluated with the computerized version of the NASA-TLX questionnaire (Appendix D), which uses a pairwise comparison weighting procedure (Hart and Staveland 1988). The NASA-TLX is a self-reported questionnaire of perceived demands in six areas: mental demand, physical demand, temporal demand, effort (mental and physical), frustration, and performance. ...
Technical Report
This study examined the effects of communication style on human performance, trust, situation awareness, and perceptions of a robot in a human–robot team. In a 2 × 2 mixed-factor study, 32 participants conducted a simulated cordon-and-search-style task while teamed with a robot. Participants were assigned to a communication style (directive vs. nondirective; within) and both groups experienced periods of high versus low task load (among subjects). Results indicate task load was a greater influence on the participants’ task performance than communication style, although there were some differential effects on response time and workload due to communication style. This may be due to a difference in feedback inherent in the differing communication styles.
... For the post-task survey, we measured task-level cognitive load using NASA-TLX [20], and developed one question for task-level survey uncertainty and difficulty of the task. We also asked the participants to conduct pointing task, which asked participants to point to the starting point of the task to measure the completeness of the cognitive map [21]. ...
Preprint
Full-text available
Navigation uncertainty is crucial for understanding the wayfinding behaviors while no method has been developed to effectively measured uncertainty in real-world scenarios. We developed the Real-time Continous Uncertainty Annotation (RCUA) to continuously measure perceived uncertainty by asking users to push the joystick in during the wayfinding process. We tested its test-retest reliability and validated RCUA based on 40 participants using the known group and known treatment. We also compared it with a discrete self-report scale and continuous postexperiment video annotation (CUA). The result demonstrated that most participants were able to output four distinct levels of uncertainty, though high variability and errors were observed. Both known group and known treatment proved good validity of the measure and RCUA was moderately correlated with self-report uncertainty. Self-report surveys showed that participants can continuously push the joystick and conduct wayfinding tasks at the same time. A comparison between RCUA and CUA showed that RCUA had a higher granularity but participants tended to overreport uncertainty using RCUA.
... The collection of information about the external environment is the key to designing an unmanned intelligent vehicle, and the various data collection sensors installed on the vehicle are equivalent to the driver's eyes [19]. Only the timely and proper acquisition and processing of external environment data can ensure the safe and stable driving [20]. ...
Article
Full-text available
As an autonomous mobile robot, the unmanned intelligent vehicle is often installed with sensors to collect the road environment information, and then process the information and control the speed and steering. In this study, vehicle-mounted camera, laser scanning radar and other sensors were equipped to collect real-time environmental information to efficiently process and accurately detect the specific location and shape of the obstacle. This study then investigated the impact of two In-Vehicle Information Systems (IVISS) on both usability and driving safety. Besides, the laser perception sensing technology was applied to transmit the information of the surrounding around the real-time driving area to the vehicle system. Simulating vehicle checkerboard and hierarchical IVIS interface layouts, we also examined their usability based on task completion time, error rate, NASA-TLX, and System Usability Scale (SUS). It was suggested that the results offer a supporting evidence for further design of IVIS interface.
... Therefore, the tools for self-reported mental workload assessment developed around twenty years ago are still valid and used nowadays. One of the most often used scales is the NASA-Task Load Index (Hart, 2006;Hart & Staveland, 1988) and still very current in the measure of mental workload in tandem with other physiological indicators (Du et al., 2019;Foy & Chapman, 2018;Solís-Marcos & Kircher, 2018). The Driving Activity Load Index (DALI) (Pauzié, 2008) is an adaptation of the NASA-TLX to the driving context and is also used in current research (Melnicuk et al., 2021). ...
Thesis
Full-text available
Real-time monitoring of drivers’ functional states will soon become a required safety feature for commercially available vehicles with automated driving capability. Automated driving technology aims to mitigate human error from road transport with the progressive automatisation of specific driving tasks. However, while control of the driving task remains shared between humans and automated systems, the inclusion of this new technology is not exempt from other human factors-related challenges. Drivers’ functional states are essentially a combination of psychological, emotional, and cognitive states, and they generate a constant activity footprint available for measurement through neural and peripheral physiology, among other measures. These factors can determine drivers’ functional states and, thus, drivers’ availability to safely perform control transitions between human and vehicle. This doctoral project aims at investigating the potential of electrocardiogram (ECG), electrodermal activity (EDA) and functional near-infrared spectroscopy (fNIRS) as measures for a multimodal driver state monitoring (DSM) system for highly automated driving (i.e., SAE levels 3 and 4). While current DSM systems relying on gaze behaviour measures have proven valid and effective, several limitations and challenges could only be overcome using eye-tracking in tandem with physiological parameters. This thesis investigates whether ECG, EDA and fNIRS would be good candidates for such a purpose. Two driving simulator studies were performed to measure mental workload, trust in automation, stress and perceived risk, all identified as modulators of drivers’ functional states and that could eventually determine drivers’ availability to take-over manual control. The main findings demonstrate that DSM systems should adopt multiple physiological measures to capture changes in functional states relevant for driver readiness. Future DSM systems will benefit from the knowledge generated by this research by applying machine learning methods to these measures for determining drivers’ availability for optimal take-over performance.
... task load) yielded a non-significant, low correlation with neurofeedback success. Previous studies monitored task load, or aspects of task load such as effort 41 , and found that adapting the neurofeedback task accordingly increased the likelihood of neurofeedback success. This suggests that an adaptable neurofeedback software may be more important than the perceived difficulty. ...
Article
Full-text available
EEG-based neurofeedback uses mental behaviours (MB) to enable voluntary self-modulation of brain activity, and has potential to relieve central neuropathic pain (CNP) after a spinal cord injury (SCI). This study aimed to understand neurofeedback learning and the relationship between MB and neurofeedback success. Twenty-five non-CNP participants and ten CNP participants received neurofeedback training (reinforcing 9–12 Hz; suppressing 4–8 Hz and 20–30 Hz) on four visits. Participants were interviewed about the MB they used after each visit. Questionnaires examined the following factors: self-efficacy, locus of control, motivation, and workload of neurofeedback. MB were grouped into mental strategies (a goal-directed mental action) and affect (emotional experience during neurofeedback). Successful non-CNP participants significantly used more imagination-related MS and reported more negative affect compared to successful CNP participants. However, no mental strategy was clearly associated with neurofeedback success. There was some association between the lack of success and negative affect. Self-efficacy was moderately correlated with neurofeedback success (r = < 0.587, p = < 0.020), whereas locus of control, motivation, and workload had low, non-significant correlations (r < 0.300, p > 0.05). Affect may be more important than mental strategies for a successful neurofeedback performance. Self-efficacy was associated with neurofeedback success, suggesting that increasing confidence in one’s neurofeedback abilities may improve neurofeedback performance.
... With multiple-choice questions, comprehension and reproducibility are evaluated. Subsequently, the participants' stress levels can be measured using the Nasa Task Load Index (Hart, 1988). ...
Conference Paper
Full-text available
Supporting student mental health and wellbeing continues to be a foremost concern in Higher Education (HE), as rates of students presenting with mental health conditions, distress and poor wellbeing increase and as demand for counselling and support services exceeds supply. The age range of students in third level education often coincides with the challenging transitional period of emerging adulthood, where instabilities are further compounded by academic, financial, and social pressures. As HE institutes are distinct settings where academic work, hobbies, social life, as well as health and other services are often integrated, HE presents a unique opportunity to address this wider societal concern through a systems-based lens. Despite calls for holistic, whole of institution approaches, a transformation of student wellbeing has yet to be realised. During a national initiative for valuing teaching and learning in HE in Ireland, the authors hosted an engaged online event to mobilise learning and action in the student wellbeing community. The event included case study presentations from existing examples of wellbeing in the curriculum, a panel discussion on the national landscape, and an open discussion on the future of wellbeing in HE. Attendees included academic faculty, HE management, researchers, staff from health and counselling services and health promotion, student representatives, careers and support services, and others. Data were collected via the recorded oral contributions, Zoom chat, and an anonymous survey. A deductive thematic analysis was completed with the guiding concept of an institution as a system that supports wellbeing. Findings proposed shared values as the compass for organisational culture, leaders and decision makers as key enablers of change, academic structures as both a vehicle to promote positive wellbeing and mitigate negative impacts, academic staff as the embodiment of the institution and its values, and the student voice as a guide for informed decision making. Recognising an institution as part of a wider system of HE which is influenced by political and economic climates, there is a requirement for HE to set out its stall with respect to its purpose and responsibility to wellbeing. This affirmation could enable a shared understanding of and commitment to wellbeing across the sector, through which collaborative and system-based efforts to support wellbeing can be actioned.
... A low level of technology acceptance may introduce a risk for workers as it could compromise the success of interaction between humans and robots; • Stress: in general, it is defined as the human body's response to pressures from a situation or life event (Recipe for Stress, 2020). When this pressure exceeds certain limits, stress becomes a risk factor, as it can have a detrimental effect on performance and can lead to an increased error rate; • Cognitive workload: refers to the cognitive effort that an individual shows during a task or to achieve a particular level of performance (Hart and Staveland, 1988). Assuming that an individual's cognitive resources are limited, the more effort is requested by a task, and the higher is the cognitive workload. ...
Article
Industry 4.0 is the concept used to summarize the ongoing fourth industrial revolution, which is profoundly changing the manufacturing systems and business models all over the world. Collaborative robotics is one of the most promising technologies of Industry 4.0. Human-robot interaction and human-robot collaboration will be crucial for enhancing the operator's work conditions and production performance. In this regard, this enabling technology opens new possibilities but also new challenges. There is no doubt that safety is of primary importance when humans and robots interact in industrial settings. Nevertheless, human factors and cognitive ergonomics (i.e. cognitive workload, usability, trust, acceptance, stress, frustration, perceived enjoyment) are crucial, even if they are often underestimated or ignored. Therefore, this work refers to cognitive ergonomics in the design of human-robot collaborative assembly systems. A set of design guidelines has been developed according to the analysis of the scientific literature. Their effectiveness has been evaluated through multiple experiments based on a laboratory case study where different participants interacted with a low-payload collaborative robotic system for the joint assembly of a manufacturing product. The main assumption to be tested is that it is possible to improve the operator's experience and efficiency by manipulating the system features and interaction patterns according to the proposed design guidelines. Results confirmed that participants improved their cognitive response to human-robot interaction as well as the assembly performance with the enhancement of workstation features and interaction conditions by implementing an increasing number of guidelines.
... With multiple-choice questions, comprehension and reproducibility are evaluated. Subsequently, the participants' stress levels can be measured using the Nasa Task Load Index (Hart, 1988). ...
Conference Paper
Full-text available
A significant number of children in relation to educational institutions in the European Union, subjected to inappropriate and discriminatory position, represent the basis of the document Union of Equality: Strategy for the Rights of Persons with Disabilities 2021-2030 (COM, 2021). At the same time, the social model of inclusion in education (Slee et al., 2019; Sunko, 2021) and articles 2 and 23 of the Convention on the Rights of the Child (1989) advocating equal rights of all children to education, indicate the dichotomy of desired and achieved. Each individual, whether he/she has certain difficulties or not, differs in his/her abilities, and each of them has their “personal needs”. It is important to note that students with special educational needs are students with disabilities and gifted students. Meeting the diverse needs of students through inclusive practices is often difficult or even impossible for those teachers who have not acquired the necessary skills and knowledge, so it is imperative to empower and support teachers primarily through formal education so that teachers learn to use effective inclusive teaching methods at all levels (Loveys, 2022). The aim of this research was to determine whether personal experience and student’s attended academic year of the teacher study (N = 304, all academic years) from three Teachers’ Faculties in the Republic of Croatia, correlate with their sense of personal competence, motivation for further professional development, or the need to change the study program. The results of this research show that students’ personal experiences with children with developmental disabilities (DD) affect the sense of their personal competence for working with children with DD, and that students of all attended academic years are equally motivated to teach children with DD. They also point out the need for additional training, and 84.64% of them emphasize the importance of practice in learning that deals with teaching children with DD. Data suggest that the same percentage of students feel the need to change/adapt the content of the study program accordingly. The main feature and implication of this research is the insight into the development of future teachers’ needs for further higher education for teaching children with DD.
... NASA-TLX. NASA Task Load Index (TLX) [22] is a standard metric for perceived workload. We included this to understand how different conditions could affect the experience of creating ML models with different configurations of IMT systems. ...
Preprint
Interactive Machine Teaching (IMT) systems allow non-experts to easily create Machine Learning (ML) models. However, existing vision-based IMT systems either ignore annotations on the objects of interest or require users to annotate in a post-hoc manner. Without the annotations on objects, the model may misinterpret the objects using unrelated features. Post-hoc annotations cause additional workload, which diminishes the usability of the overall model building process. In this paper, we develop LookHere, which integrates in-situ object annotations into vision-based IMT. LookHere exploits users' deictic gestures to segment the objects of interest in real time. This segmentation information can be additionally used for training. To achieve the reliable performance of this object segmentation, we utilize our custom dataset called HuTics, including 2040 front-facing images of deictic gestures toward various objects by 170 people. The quantitative results of our user study showed that participants were 16.3 times faster in creating a model with our system compared to a standard IMT system with a post-hoc annotation process while demonstrating comparable accuracies. Additionally, models created by our system showed a significant accuracy improvement ($\Delta mIoU=0.466$) in segmenting the objects of interest compared to those without annotations.
... NASA-TLX was used in this study to evaluate the MWL levels of participants. This scale has been widely used in many human-machine interaction studies such as maritime ( NASA-TLX is a multidimensional task load assessment tool, developed by Hart and Staveland (1988). NASA-TLX has 6 sub-scales which are mental, physical and temporal loads (task related), performance and effort (behavioural and skill related) and frustration (individual related). ...
Article
Full-text available
The main aim of this study is to measure the mental workload of the operators according to the increasing workload during simulated ship navigation and it is aimed to contribute to the clarification of upper redline of task demands. Eye responses and performance results of twelve participants were recorded during the measurements carried out in bridge simulator. In addition, a specific tool (NASA-TLX) was used to assess twelve participants at the end of each step of the scenarios. The results showed that mental workload of the participants increased as the task load increased and their performance decreased. It was observed that the developed Artificial Neural Network model can predict operator mental workload based on eye response indices (accuracy: 79.2%). This study is considered to contribute to the literature by defining an upper redline of task demands for an operator and monitoring near real-time mental workload indicators based on the physiological data of operators in the presence of autonomous ships and in navigational conditions where the automation level of ships gradually increases.
... With multiple-choice questions, comprehension and reproducibility are evaluated. Subsequently, the participants' stress levels can be measured using the Nasa Task Load Index (Hart, 1988). ...
Chapter
Full-text available
After 2011 a new alternative educational form appeared on the palette of Hungarian public education: learning communities that provide alternative education for schoolchildren who take part in alternative or mainstream education as private pupils. The learning communities are not schools in the traditional way, but we can simplify it as home-schooling in a more organized way. The conditions of learning communities and regulations in connection with the fulfilment of compulsory education vary in different countries and there is a difference in private pupils’ legal relationship in regard to how permissive or restrictive the status of being a private pupil is. The learning community as an alternative way of education has appeared in more European countries and even beyond Europe, this research discusses three European countries – Austria, Hungary and Romania – the way they regulate the fulfilment of compulsory education and their regulations in how they permit being a private pupil, as well as the attitude of educational governance towards this new form ofalternative education.
... Dependent measures comprise of decision-making, measured through the trajectory decision the pilots make; safety, measured via the number of "unsafe" fuel predictions (i.e., overestimating own capabilities); workload, measured via the subjective NASA-TLX, given to the pilots after each condition [22]; situation awareness, measured via the subjective scale Situation Awareness Rating Technique (SART) [23], likewise given to the pilot after each experiment condition; and pilot preference, measured through a questionnaire given to the pilots at the end of the experiment. ...
Article
This paper investigates the effects of different automation design philosophies for a helicopter navigation task. A baseline navigation display is compared with two more advanced systems: an advisory display, which provides a discrete trajectory suggestion; and a constraint-based display, which provides information about the set of possible trajectory solutions. The results of a human-in-the-loop experiment with eight pilot participants show a significant negative impact of the advisory display on pilot trajectory decision-making: out of the 16 encountered off-nominal situations across the experiment, only 6 were solved optimally. The baseline and constraint-based display both lead to better decisions, with 14 out of 16 being optimal. However, pilots still preferred the advisory display, in particular in off-nominal situations. These results highlight that even when a support system is preferred by pilots, it can have strong inadvertent negative effects on their decision-making.
... A comprehensive list of performance measures is shown in Table 3. Mental workload was measured subjectively in all of the reviewed studies. The only subjective measure in the reviewed studies was the NASA Task Load Index (NASA-TLX) measure (Hart and Staveland, 1988). ...
Article
Full-text available
This study assessed the effects of different levels of automation and non-driving related tasks (NDRT) on driver performance and workload. A systematic literature review was conducted in March 2021 using Compendex, Google Scholar, Web of Science, and Scopus databases. Forty-five studies met the inclusion criteria. A meta-analysis was conducted and Cochrane risk of bias tool and Cochran's Q test were used to assess risk of bias and homogeneity of the effect sizes respectively. Results suggested that drivers exhibited safer performance when dealing with critical incidents in manual driving than partially automated driving (PAD) and highly automated driving (HAD) conditions. However, drivers reported higher workload in the manual driving mode as compared to the HAD and PAD conditions. Haptic, auditory, and visual-auditory takeover request modalities are preferred over the visual-only modality to improve takeover time. Use of handheld NDRTs significantly degraded driver performance as compared to NDRTs performed on mounted devices.
... As is clear from the discussion above, translation feasibility and difficulty have been a topic of discussion for a long time. And while the difficulties of translation can be investigated after the fact by means of questionnaires or self-assessment scales such as the NASA task load index (TLX) (Hart and Staveland, 1988), these approaches are highly subjective. The topic of translation difficulty has been adopted by fields such as psycholinguistics and empirical translation research, though, which provide a strong foundation of research to difficulty in experimental research. ...
Conference Paper
Full-text available
We present LeConTra, a learner corpus consisting of English-to-Dutch news translations enriched with translation process data. Three students of a Master's programme in Translation were asked to translate 50 different English journalistic texts of approximately 250 tokens each. Because we also collected translation process data in the form of keystroke logging, our dataset can be used as part of different research strands such as translation process research, learner corpus research, and corpus-based translation studies. Reference translations, without process data, are also included. The data has been manually segmented and tokenized, and manually aligned at both segment and word level, leading to a high-quality corpus with token-level process data. The data is freely accessible via the Translation Process Research DataBase and GitHub, which emphasises our commitment of distributing our dataset. The tool that was built for manual sentence segmentation and tokenization, Mantis, is also available as an open-source aid for data processing.
... Likewise, the study conducted by Szulewski et al. (2017) was able to validate the correlation between psychometric and physiological tools as empirical indicators to assess CL. From a psychometric standpoint, CL can be measured using self-report instruments like the NASA-TLX, which accounts for the amount of difficulty participants have while performing a task (Hart and Staveland 1988). The NASA-TLX is a questionnaire that by using a simple Likert scale accounts for six different aspects of task performance (mental demands, physical demands, temporal demands, own performance, effort, and frustration). ...
... Thanks to the theory development, the implementation of diverse subjective, objective, physiological measures of cognitive load has thrived. Cognitive load can be measured through such subjective measures as perceived mental effort and level of difficulty (Paas, 1992), time estimation (Baralt, 2013), NASA-task load index (Hart & Staveland, 1988), and Leppink's cognitive load scale (Andersen & Makransky, 2021). ...
Article
Full-text available
Recently, researchers have expressed their growing concern over the scrutiny of language test items in light of cognitive load theory (CLT). While cognitive load has been central to language learning research, it has not enjoyed due attention in high-stakes language tests. The current study set out to delve into the relationship between difficulty estimates and cognitive load of language test items. To measure cognitive load, examinees’ perceived level of difficulty and response time were considered. In this regard, empirical data were collected from 60 MA students and graduates through a quantitative correlational design. The current study further employed the Rasch model to estimate difficulties of the vocabulary and grammar items of the Iranian university entrance examination (IUEE) for MA in English majors held in 2018 and 2019. The study’s findings revealed statistically significant correlations between difficulty estimates and perceived level of difficulty for vocabulary items. As for grammar items, no statistically significant correlations were detected between the variables. Whereas the results indicated strong positive correlations between response time and difficulty estimates regarding vocabulary items, no statistically significant correlations were observed between the variables concerning grammar items. All in all, perceived level of difficulty, response time, and difficulty estimates appeared to be sound indicators of cognitive load with respect to vocabulary test items, but not with regard to grammar test items. The implications of the findings will be discussed.
... Generally speaking, these techniques come under three main categories: Subjective, Behavioural/Performance, and Physiological. Subjective measures such as the NASA-TLX workload questionnaire (Hart and Staveland 1988) or the Empirically Derived Trust Questionnaire (Jian et al. 2000) are a popular first choice in HSI evaluation due to their ease of use, non-invasiveness, and cost effectiveness (Rubio et al. 2004). However there are several drawbacks to subjective metrics such as task intrusiveness and the ability to only represent the entire task rather than changes within the task (Shaw et al. 2012). ...
Conference Paper
Full-text available
Understanding how the user will interact with the system is fundamental to ensuring success in achieving a given goal. Therefore adopting a human-centered design approach will assist in integrating the human as a key component of the system during the design process. With the increased use of autonomy across different domains, the role of the human will inevitably change; in that how the user interacts with the system is dependent on the level of delegated authority the system has been assigned. To understand these interactions and the impact this has on the user, it is important to assess how the human interacts with the system. However, as these systems become more complex we must ask whether the measures we currently use are sufficient in allowing us to better understand the underlying cognitive functions involved in human-autonomy interaction. Evaluating this partnership we can not only assess the effectiveness and efficiency of human-autonomy interaction, but also provide guidance for future designs. Novel techniques such as functional Near Infrared Spectroscopy (fNIRS) offer a direct measure of cortical blood flow changes related to brain activity. This paper discusses findings from an experiment that examined human-autonomy interaction in a simulated Autonomous Vehicle (AV) whilst exploring the neural correlates of trust and workload. Participants were asked to complete a series of primary driving scenarios with secondary distraction tasks using both manual and autonomous vehicles. fNIRS was used to assess driver cognition across both conditions. Participants were also confronted with different levels of system transparency to determine whether the level of information presented by the system effected driver trust. Findings suggest that when autonomy was presented then the cognitive activity in the right and left dorsolateral prefrontal cortex (dlPFC) and the left ventrolateral prefrontal cortex (vlPFC) was reduced, whilst secondary task performance improved. These regions are associated with effortful decision-making based on working memory (WM) and reasoning, suggesting that using autonomy helps to reduce cognitive effort by removing the user’s need to make these decisions. During the system transparency scenarios, areas of the right and left vlPFC and left dlPFC showed significantly increased activity when the system provided very little information. These regions have previously been associated with uncertainty of decision making and increased visual processing, suggesting that a lack of information provided by the system meant the driver attempted to process the decisions of the vehicle through monitoring the environment. These findings demonstrate how novel measures of cognitive function could inform the design of future systems and facilitate a more effective human-autonomy partnership.
... With multiple-choice questions, comprehension and reproducibility are evaluated. Subsequently, the participants' stress levels can be measured using the Nasa Task Load Index (Hart, 1988). ...
Chapter
Full-text available
Besides state-funded schools, private schools play a role in public education both abroad and in Hungary, however the financial aid they receive from the governmental budget is different from country to country. There are countries where they receive the same amount of support that state-funded institutions get. Whereas there are other private institutions that cannot gain any financial resources from the subsidy. Financial contribution by the government to educational costs, however, always goes together with a restriction of the autonomy of schools by said government. These restrictions may include forcing the exemption of tuition fees or mandating that private schools cannot control the admission of pupils. Moreover, it might convey the restriction of the pedagogical autonomy of alternative private schools according to the educational system’s degree of centralization. The liberal and decentralized Hungarian education system has become centralized again due to the current government’s aspiration of creating an integrated and unified educational policy. In this study, we seek to answer the question of how the financial contribution of the state to the operation of alternative private schools affects their pedagogical autonomy.
... At the end of the experiment, healthy participants were asked to fill a NASA-TLX auto-questionnaire for each type of feedback experienced (visual, visual + vibrotactile, and vibrotactile only) [39][40][41]. This evaluates factors influencing the workload. ...
Article
Full-text available
Background Current myoelectric prostheses lack proprioceptive information and rely on vision for their control. Sensory substitution is increasingly developed with non-invasive vibrotactile or electrotactile feedback, but most systems are designed for grasping or object discriminations, and few were tested for online control in amputees. The objective of this work was evaluate the effect of a novel vibrotactile feedback on the accuracy of myoelectric control of a virtual elbow by healthy subjects and participants with an upper-limb amputation at humeral level. Methods Sixteen, healthy participants and 7 transhumeral amputees performed myoelectric control of a virtual arm under different feedback conditions: vision alone (VIS), vibration alone (VIB), vision plus vibration (VIS + VIB), or no feedback at all (NO). Reach accuracy was evaluated by angular errors during discrete as well as back and forth movements. Healthy participants’ workloads were assessed with the NASA-TLX questionnaire, and feedback conditions were ranked according to preference at the end of the experiment. Results Reach errors were higher in NO than in VIB, indicating that our vibrotactile feedback improved performance as compared to no feedback. Conditions VIS and VIS+VIB display similar levels of performance and produced lower errors than in VIB. Vision remains therefore critical to maintain good performance, which is not ameliorated nor deteriorated by the addition of vibrotactile feedback. The workload associated with VIB was higher than for VIS and VIS+VIB, which did not differ from each other. 62.5% of healthy subjects preferred the VIS+VIB condition, and ranked VIS and VIB second and third, respectively. Conclusion Our novel vibrotactile feedback improved myoelectric control of a virtual elbow as compared to no feedback. Although vision remained critical, the addition of vibrotactile feedback did not improve nor deteriorate the control and was preferred by participants. Longer training should improve performances with VIB alone and reduce the need of vision for close-loop prosthesis control.
... The experiment was carried out by 40 testers, graduate and post-graduate students (with participants' age ranged from 18 to 35), not always acquainted with robotics applications. After each test, we asked testers to fill the questionnaire showed in Table 2, which is inspired by the NASA Task Load Index (NASA-TLX) (Hart & Staveland, 1988) and suitably adapted/extended with typical questions proposed in HRI literature (Steinfeld et al., 2006;Young et al., 2011;Broquère et al., 2014;Chen & Kemp, 2010;Maurtua et al., 2017) to assess the interaction and the performance. A 5-point Likert multi-item scale was employed for the survey. ...
Article
Full-text available
In this work, we consider a scenario in which a human operator physically interacts with a collaborative robot (CoBot) to perform shared and structured tasks. We assume that collaborative operations are formulated as hierarchical task networks to be interactively executed exploiting the human physical guidance. In this scenario, the human interventions are continuously interpreted by the robotic system in order to infer whether the human guidance is aligned or not with respect to the planned activities. The interpreted human interventions are also exploited by the robotic system to on-line adapt its cooperative behavior during the execution of the shared plan. Depending on the estimated operator intentions, the robotic system can adjust tasks or motions, while regulating the robot compliance with respect to the co-worker physical guidance. We describe the overall framework illustrating the architecture and its components. The proposed approach is demonstrated in a testing scenario consisting of a human operator that interacts with the Kuka LBR iiwa manipulator in order to perform a collaborative task. The collected results show the effectiveness of the proposed approach.
... Our evaluation for the user experience was through the NASA Task Load Index (NASA-TLX) questionnaire and the collaborative experience questionnaire. NASA-TLX is a multidimensional, validated, reliable, and standardized subjective test which can quantitatively evaluate the task workload [56]. The collaborative experience questionnaire (see Table 1) refers to networked minds measure of social presence questionnaire [57] and a few past MR remote collaboration research [5,9,43,58]. ...
Article
Full-text available
With the rapid development of mixed reality (MR) technology, many compact, lightweight, and powerful devices suitable for remote collaboration, such as MR headsets, hand trackers, and 3D cameras, become readily available, providing hardware and software support for remote collaboration. Consequently, exploring MR technologies for remote collaboration on physical industry tasks is becoming increasingly worthwhile. In many complex production scenarios, such as assembly tasks, significant gains can be achieved by having remote experts assist local workers to manipulate objects in local workspaces. However, it can be challenging for a remote expert to carry out effective spatial reference and action demonstration in a local scene. Sharing 3D stereoscopic scenes can provide depth perception and support remote experts to move and explore a local user’s environment freely. Previous studies have demonstrated that gesture-based interaction is natural and intuitive, and interaction based on virtual replicas can provide clear guidance, especially for industrial physical tasks. In this study, we develop an MR remote collaboration system that shares the stereoscopic scene of the local workspace by using real-time 3D video. This system combines gesture cues and virtual replicas in a complementary manner to support the remote expert to create augmented reality (AR) guidance for the local worker naturally and intuitively in the virtual reality immersive space. A formal user study was performed to explore the effects of two different modalities interface in industrial assembly tasks: our novel method of using the combination of virtual replicas and gesture cues in the 3D video (VG3DV), and a method similar to the popular method currently of using gesture cues in the 3D video (G3DV). We found that using the VG3DV can significantly improve the performance and user experience of MR remote collaboration in industrial assembly tasks. Finally, some conclusions and future research directions were given.
... Participants rated their perceived safety using four 7-point semantic differentials from -3 (anxious/agitated/unsafe/timid) to +3 (relaxed/calm/safe/confident) as proposed by Faas et al. [20]. We also employed the mental workload subscale of the raw NASA-TLX [25] on a 20-point scale ("How much mental and perceptual activity was required? Was the task easy or demanding, simple or complex?"; 1=Very Low to 20=Very High). ...
Conference Paper
Full-text available
Automated vehicles are expected to substitute or even improve driver-driver communication, for example, via LED strips or displays. Numerous situations exist where ambiguities have to be resolved via gestures or implicit communication (i.e., movement). An already demanding situation is the unsignalized four-way intersection. Additionally, Vehicle-To-Everything technology enables automated vehicles to perform maneuvers impossible before such as blocking an intersection to safely let an emergency vehicle pass. Therefore, we report the results of a within-subject Virtual Reality study (N=17) evaluating these two scenarios. Results show that external communication increased perceived safety and reduced mental workload, and also that the novel behavior confused participants. Our work helps better to integrate external communication in settings with manual drivers.
... We employed the mental workload subscale of the raw NASA-TLX [22] on a 20-point scale ("How much mental and perceptual activity was required? Was the task easy or demanding, simple or complex?"; 1=Very Low to 20=Very High). ...
Conference Paper
Full-text available
Automated vehicles should improve both traffic safety and user experience. While novel behavior patterns such as platooning become feasible to reduce fuel usage, such time-and fuel-reducing behavior at intersections can be perceived as unsafe and possibly disconcert users. Therefore, we designed and implemented nine feedback strategies for a simulated intersection and compared these in an online video-based between-subjects study (N=226). We found that visual feedback strategies limiting the view on the actual scene by providing calming views (a Landscape or the scene with hidden vehicles) were rated significantly higher in terms of perceived safety and trust. We discuss implications regarding future traffic and whether automated vehicles will necessitate altering reality for the user.
... After the experiments, the participants filled a questionnaire to rate different qualitative aspects of their experience for each controller. The questionnaire is composed of a standard part, namely the NASA-TLX [31], and a custom Likert scale-based part designed specifically for this study (see Table I). Besides, for the quantitative evaluation of the task-related performance, the following metrics were used. ...
Preprint
Full-text available
In this study, an adaptive object deformability-agnostic human-robot collaborative transportation framework is presented. The proposed framework enables to combine the haptic information transferred through the object with the human kinematic information obtained from a motion capture system to generate reactive whole-body motions on a mobile collaborative robot. Furthermore, it allows rotating the objects in an intuitive and accurate way during co-transportation based on an algorithm that detects the human rotation intention using the torso and hand movements. First, we validate the framework with the two extremities of the object deformability range (i.e, purely rigid aluminum rod and highly deformable rope) by utilizing a mobile manipulator which consists of an Omni-directional mobile base and a collaborative robotic arm. Next, its performance is compared with an admittance controller during a co-carry task of a partially deformable object in a 12-subjects user study. Quantitative and qualitative results of this experiment show that the proposed framework can effectively handle the transportation of objects regardless of their deformability and provides intuitive assistance to human partners. Finally, we have demonstrated the potential of our framework in a different scenario, where the human and the robot co-transport a manikin using a deformable sheet.
... Participants that struggled to answer the questions asked during phase two were given easier questions. After the experiment, participants removed the HMD and completed a 0-100 scale NASA Task Load Index (TLX) to measure their perceived workload [7]. ...
Preprint
Full-text available
Immersive Analytics (IA) and consumer adoption of augmented reality (AR) and virtual reality (VR) head-mounted displays (HMDs) are both rapidly growing. When used in conjunction, stereoscopic IA environments can offer improved user understanding and engagement; however, it is unclear how the choice of stereoscopic display impacts user interactions within an IA environment. This paper presents a pilot study that examines the impact of stereoscopic display type on object manipulation and environmental navigation using consumer-available AR and VR displays. This work finds that the display type can impact how users manipulate virtual content, how they navigate the environment, and how able they are to answer questions about the represented data.
... Each user completed a modified version of NASA-TLX questionnaire, a widely-used, subjective, multidimensional assessment tool that rates perceived mental workload and aspects of performance [225]. They rated both Gaze-PIN based entry system and our system (DyGazePass) across However, responses to Q2 and Q3 suggests that remembering a password as a set of colors and recollecting it during the authentication is mentally demanding when compared to remembering and recollecting a password as a set of numbers (P < 0.05). ...
Thesis
Full-text available
Every day we encounter a variety of scenarios that lead to situationally induced impairments and disabilities, i.e., our hands are assumed to be engaged in a task, and hence unavailable for interacting with a computing device. For example, a surgeon performing an operation, a worker in a factory with greasy hands or wearing thick gloves, a person driving a car, and so on all represent scenarios of situational impairments and disabilities. In such cases, performing point-and-click interactions, text entry, or authentication on a computer using conventional input methods like the mouse, keyboard, and touch is either inefficient or not possible. Unfortunately, individuals with physical impairments and disabilities, by birth or due to an injury, are forced to deal with these limitations every single day. Generally, these individuals experience difficulty or are completely unable to perform basic operations on a computer. Therefore, to address situational and physical impairments and disabilities it is crucial to develop hands-free, accessible interactions. In this research, we try to address the limitations, inabilities, and challenges arising from situational and physical impairments and disabilities by developing a gaze-assisted, multi-modal, hands-free, accessible interaction paradigm. Specifically, we focus on the three primary interactions: 1) point-and-click, 2) text entry, and 3) authentication. We present multiple ways in which the gaze input can be modeled and combined with other input modalities to enable efficient and accessible interactions. In this regard, we have developed a gaze and foot-based interaction framework to achieve accurate ``point-and-click" interactions and to perform dwell-free text entry on computers. In addition, we have developed a gaze gesture-based framework for user authentication and to interact with a wide range of computer applications using a common repository of gaze gestures. The interaction methods and devices we have developed are a) evaluated using the standard HCI procedures like the Fitts' Law, text entry metrics, authentication accuracy and video analysis attacks, b) compared against the speed, accuracy, and usability of other gaze-assisted interaction methods, and c) qualitatively analyzed by conducting user interviews. From the evaluations, we found that our solutions achieve higher efficiency than the existing systems and also address the usability issues. To discuss each of these solutions, first, the gaze and foot-based system we developed supports point-and-click interactions to address the "Midas Touch" issue. The system performs at least as good (time and precision) as the mouse, while enabling hands-free interactions. We have also investigated the feasibility, advantages, and challenges of using gaze and foot-based point-and-click interactions on standard (up to 24") and large displays (up to 84") through Fitts' Law evaluations. Additionally, we have compared the performance of the gaze input to the other standard inputs like the mouse and touch. Second, to support text entry, we developed a gaze and foot-based dwell-free typing system, and investigated foot-based activation methods like foot-press and foot gestures. we have demonstrated that our dwell-free typing methods are efficient and highly preferred over conventional dwell-based gaze typing methods. Using our gaze typing system the users type up to 14.98 Words Per Minute (WPM) as opposed to 11.65 WPM with dwell-based typing. Importantly, our system addresses the critical usability issues associated with gaze typing in general. Third, we addressed the lack of an accessible and shoulder-surfing resistant authentication method by developing a gaze gesture recognition framework, and presenting two authentication strategies that use gaze gestures. Our authentication methods use static and dynamic transitions of the objects on the screen, and they authenticate users with an accuracy of 99% (static) and 97.5% (dynamic). Furthermore, unlike other systems, our dynamic authentication method is not susceptible to single video iterative attacks, and has a lower success rate with dual video iterative attacks. Lastly, we demonstrated how our gaze gesture recognition framework can be extended to allow users to design gaze gestures of their choice and associate them to appropriate commands like minimize, maximize, scroll, etc., on the computer. We presented a template matching algorithm which achieved an accuracy of 93%, and a geometric feature-based decision tree algorithm which achieved an accuracy of 90.2% in recognizing the gaze gestures. In summary, our research demonstrates how situational and physical impairments and disabilities can be addressed with a gaze-assisted, multi-modal, accessible interaction paradigm.
... Questionnaires. NASA-TLX [28] was used to collect data from the participants at the end of each section. At the end of the experiment participants were asked which of the unmanned vehicles was more effective for them, if at all. . ...
... We measured cognitive load using the mental workload subscale of the raw NASA-TLX [32] on a 20-point scale ("How much mental and perceptual activity was required? Was the task easy or demanding, simple or complex?"; 1=Very Low to 20=Very High) and situation awareness using the situation awareness rating technique (SART) [68]. ...
Article
Full-text available
The successful introduction of automated vehicles (AVs) depends on the user's acceptance. To gain acceptance, the intended user must trust the technology, which itself relies on an appropriate understanding. Visualizing internal processes could aid in this. For example, the functional hierarchy of autonomous vehicles distinguishes between perception, prediction, and maneuver planning. In each of these stages, visualizations including possible uncertainties (or errors) are possible. Therefore, we report the results of an online study (N=216) comparing visualizations and their combinations on these three levels using a pre-recorded real-world video with visualizations shown on a simulated augmented reality windshield. Effects on trust, cognitive load, situation awareness, and perceived safety were measured. Situation Prediction-related visualizations were perceived as worse than the remaining levels. Based on a negative evaluation of the visualization, the abilities of the AV were also judged worse. In general, the results indicate the presence of overtrust in AVs.
... Five subjective measures were also recorded including cognitive workload, perceived parking self-confidence, perceived trust in the Tesla, and preferences for and future likelihood of using Autopark. To measure cognitive workload during parking conditions, we used the NASA Task Load Index (NASA-TLX) a subjective measure of workload commonly used for assessing human-machine interfaces [41], [42]. Workload scores were calculated as the average of its six subscales. ...
Preprint
Full-text available
In two studies, we evaluated the trust and usefulness of automated compared to manual parking using an experimental paradigm and by surveying owners of vehicles with automated parking features. In Study 1, we compared participants' ability to manually park a Tesla Model X and use the Autopark feature to complete perpendicular and parallel parking maneuvers. We investigated differences in parking success and duration, intervention behavior, self-reported levels of trust in and workload associated with the automation, as well as eye and head movements related to monitoring the automation. We found higher levels of trust in the automated parallel parking maneuvers compared to perpendicular parking. The Tesla’s automated perpendicular parking was found to be less efficient than manually executing this maneuver. Study 2 investigated the frequency with which owners of vehicles with automated parking features used those features and probed why they chose not to use them. Vehicle owners reported using their vehicle's autonomous parking features in ways consistent with the empirical findings from Study 1: higher usage rates of autonomous parallel parking. The results from both studies revealed that 1) automated parking is error-prone, 2) drivers nonetheless have calibrated trust in the automated parking system, and 3) the benefits of automated parallel parking surpass those of automated perpendicular parking with the current state of the technology.
... Response error was calculated based on the percentage of input or recognition errors made by the participant during the trial (e.g., executing the wrong gesture or choosing the wrong stimulus). Meanwhile, subjective workload ratings were measured using the National Aeronautics and Space Administration Task Load Index (NASA-TLX) developed by Hart and Staveland (1988) on a 10-point scale. Perceived modality appropriateness for each input and output modality condition was evaluated by the participants after every trial using a 10-point Likert-type rating scale ranging from "not at all" to "very much". ...
Article
Full-text available
Multimodal interaction (MMI) is being widely implemented, especially in new technologies such as augmented reality (AR) systems since it is presumed to support a more natural, efficient, and flexible form of interaction. However, limited research has been done to investigate the proper application of MMI in AR. More specifically, the effects of combining different input and output modalities during MMI in AR are still not fully understood. Therefore, this study aims to examine the independent and combined effects of different input and output modalities during a typical AR task. 20 young adults participated in a controlled experiment in which they were asked to perform a simple identification task using an AR device in different input (speech, gesture, multimodal) and output (VV-VA, VV-NA, NV-VA, NV-NA) conditions. Results showed that there were differences in the influence of input and output modalities on task performance, workload, perceived appropriateness, and user preference. Interaction effects between the input and output conditions on the performance metrics were also evident in this study, suggesting that although multimodal input is generally preferred by the users, it should be implemented with caution since its effectiveness is highly influenced by the processing code of the system output. This study, which is the first of its kind, has revealed several new implications regarding the application of MMI in AR systems.
... First, volunteers conducted experiments on a virtual experimental platform using the Noitom data glove [24] and the KINECT device; second, volunteers perform the same experiment on the NOBOOK platform [1], which is dominated by keyboard and mouse operations; third, volunteers use the traditional virtual-reality fusion experimental platform [2] to conduct experiments; at last, volunteers perform the same experiments using the smart glove. 30 volunteers were required to take turns in conducting experiments on the above four experimental platforms during a day and to perform NASA evaluation [25] of each experiment after completion. User evaluation metrics were categorized into mental demands (MD), physical demands (PD), time demands (TD), performance (P), effort (E), and frustration (F). ...
Article
Full-text available
At present, virtual-reality fusion smart experiments mostly employ visual perception devices to collect user behavior data, but this method faces the obstacles of distance, angle, occlusion, light, and a variety of other factors of indoor interactive input devices. Moreover, the essence of the traditional multimodal fusion algorithm (TMFA) is to analyze the user’s experimental intent serially using single-mode information, which cannot fully utilize the intent information of each mode. Therefore, this paper designs a multimodal fusion algorithm (hereinafter referred to as MFA, Algorithm 4) which focuses on the parallel fusion of user’s experimental intent. The essence of the MFA is the fusion of multimodal intent probability. At the same time, this paper designs a smart glove based on the virtual-reality fusion experiments, which can integrate multichannel data such as voice, visual, and sensor. This smart glove can not only capture user’s experimental intent but also navigate, guide, or warn user’s operation behaviors, and it has stronger perception capabilities compared to any other data glove or smart experimental device. The experimental results demonstrate that the smart glove presented in this paper can be widely employed in the chemical experiment teaching based on virtual-reality fusion.
... This experiment took about 1.5 h to complete, including setup time and a period for contact and familiarization with the stimuli and interface arrangement. At the end of each condition test, participants were asked to answer a simple questionnaire (Quest I) based on one part of 8/26 Gabriel Pires et al.: Preprint submitted to Journal of Neuroscience Methods the NASA Task Load Index (NASA-TLX) (Hart & Staveland, 1988). After completing the two conditions, participants were asked to answer a second questionnaire (Quest II) to compare the pleasantness of the two conditions. ...
Article
Full-text available
Background Brain-computer interfaces (BCIs) are a promising tool for communication with completely locked-in state (CLIS) patients. Despite the great efforts already made by the BCI research community, the cases of success are still very few, very exploratory, limited in time, and based on simple ‘yes/no’ paradigms. New method A P300-based BCI is proposed comparing two conditions, one corresponding to purely spatial auditory stimuli (AU-S) and the other corresponding to hybrid visual and spatial auditory stimuli (HVA-S). In the HVA-S condition, there is a semantic, temporal, and spatial congruence between visual and auditory stimuli. The stimuli comprise a lexicon of 7 written and spoken words. Spatial sounds are generated through the head-related transfer function. Given the good results obtained with 10 able-bodied participants, we investigated whether a patient entering CLIS could use the proposed BCI. Results The able-bodied group achieved 71.3% and 90.5% online classification accuracy for the auditory and hybrid BCIs respectively, while the patient achieved 30% and chance level accuracies, for the same conditions. Notwithstanding, the patient’s event-related potentials (ERPs) showed statistical discrimination between target and non-target events in different time windows. Comparison with existing methods The results of the control group compare favourably with the state-of-the-art, considering a 7-class BCI controlled visual-covertly and with auditory stimuli. The integration of Visual and auditory stimuli has not been tested before with CLIS patients. Conclusions The semantic, temporal, and spatial congruence of the stimuli increased the performance of the control group, but not of the CLIS patient, which can be due to impaired attention and cognitive function. The patient’s unique ERP patterns make interpretation difficult, requiring further tests/paradigms to decouple patients’ responses at different levels (reflexive, perceptual, cognitive). The ERPs discrimination found indicates that a simplification of the proposed approaches may be feasible.
... Yet, the vast majority of studies that assessed cognitive load in trauma teams used subjective tools. 33,34 Subjective instruments such as NASA-TLX 35 and SURG-TLX, 36 although used frequently in medical education research, are obtained post hoc, 37 not allowing for cognitive load monitoring in real-time. Moreover, as highlighted in a previous study, 38 small differences in phrasings of instrument questions included in self-report tools may lead to semantic interpretations that can compromise the reliability of such instruments. ...
Article
Introduction Literature has shown cognitive overload which can negatively impact learning and clinical performance in surgery. We investigated learners’ cognitive load during simulation-based trauma team training using an objective digital biomarker. Methods A cross-sectional study was carried out in a simulation center where a 3-h simulation-based interprofessional trauma team training program was conducted. A session included three scenarios each followed by a debriefing session. One scenario involved multiple patients. Learners wore a heart rate sensor that detects interbeat intervals in real-time. Low-frequency/high-frequency (LF/HF) ratio was used as a validated proxy for cognitive load. Learners’ LF/HF ratio was tracked through different phases of simulation. Results Ten subjects participated in 12 simulations. LF/HF ratios during scenario versus debriefing were compared for each simulation. These were 3.75 versus 2.40, P < 0.001 for scenario 1; 4.18 versus 2.77, P < 0.001 for scenario 2; and 4.79 versus 2.68, P < 0.001 for scenario 3. Compared to single-patient scenarios, multiple-patient scenarios posed a higher cognitive load, with LF/HF ratios of 3.88 and 4.79, P < 0.001, respectively. Conclusions LF/HF ratio, a proxy for cognitive load, was increased during all three scenarios compared to debriefings and reached the highest levels in a multiple-patient scenario. Using heart rate variability as an objective marker of cognitive load is feasible and this metric is able to detect cognitive load fluctuations during different simulation phases and varying scenario difficulties.
... The cognitive load was measured to assess whether the additional auditory feedback increased the user's attention and mental burden during the train stage. We used the NASA-TLX questionnaire [24] to measure cognitive load and compared the participants who used and did not use auditory feedback. c) Avatar manipulation times. ...
Article
Full-text available
Conventional motion tutorials rely mainly on a predefined motion and vision-based feedback that normally limits the application scenario and requires professional devices. In this paper, we propose VoLearn, a cross-modal system that provides operability for user-defined motion learning. The system supports the ability to import a desired motion from RGB video and animates the motion in a 3D virtual environment. We built an interface to operate on the input motion, such as controlling the speed, and the amplitude of limbs for the respective directions. With exporting of virtual rotation data, a user can employ a daily device (i.e., smartphone) as a wearable device to train and practice the desired motion according to comprehensive auditory feedback, which is able to provide both temporal and amplitude assessment. The user study demonstrated that the system helps reduce the amplitude and time errors of motion learning. The developed motion-learning system maintains the characteristics of high user accessibility, flexibility, and ubiquity in its application.
Article
Caregivers in eldercare can benefit from telepresence robots that allow them to perform a variety of tasks remotely. In order for such robots to be operated effectively and efficiently by non-technical users, it is important to examine if and how the robotic system’s level of automation (LOA) impacts their performance. The objective of this work was to develop suitable LOA modes for a mobile robotic telepresence (MRP) system for eldercare and assess their influence on users’ performance, workload, awareness of the environment, and usability at two different levels of task complexity. For this purpose, two LOA modes were implemented on the MRP platform: assisted teleoperation (low LOA mode) and autonomous navigation (high LOA mode). The system was evaluated in a user study with 20 participants, who, in the role of the caregiver, navigated the robot through a home-like environment to perform control and perception tasks. Results revealed that performance improved in the high LOA when task complexity was low. However, when task complexity increased, lower LOA improved performance. This opposite trend was also observed in the results for workload and situation awareness. We discuss the results in terms of the LOAs’ impact on users’ attitude towards automation and implications on usability.
Article
Full-text available
Despite the tremendous progress made for recognizing gestures acquired by various devices, such as the Leap Motion Controller, developing a gestural user interface based on such devices still induces a significant programming and software engineering effort before obtaining a running interactive application. To facilitate this development, we present QuantumLeap, a framework for engineering gestural user interfaces based on the Leap Motion Controller. Its pipeline software architecture can be parameterized to define a workflow among modules for acquiring gestures from the Leap Motion Controller, for segmenting them, recognizing them, and managing their mapping to functions of the application. To demonstrate its practical usage, we implement two gesture-based applications: an image viewer that allows healthcare workers to browse DICOM medical images of their patients without any hygiene issues commonly associated with touch user interfaces and a large-scale application for managing multimedia contents on wall screens. To evaluate the usability of QuantumLeap, seven participants took part in an experiment in which they used QuantumLeap to add a gestural interface to an existing application.
Chapter
Disabilities related to motor and/or cognitive impairment may impact driving abilities. The project Rip@rto aims at supporting INAIL personnel during the evaluation of the residual capabilities of people with disability who apply for a driving license. The newly designed simulator presents new features compared to the current state of art. It will provide a realistic 3D environment and a mechatronic platform to improve the realism of the driving experience. The simulator will also allow collecting objective (mental workload) and subjective (stress, risk perception, user-experience) data to support the operator’s decision.
Article
Full-text available
The objective of this study was to verify the effectiveness of eye-tacking metrics in indicating driver’s mental workload in semi-autonomous driving when the driver is engaged in different non-driving related tasks (NDRTs). A driving simulator was developed for three scenarios (high-, medium-, and low-mental workload presented by SAE (Society of Automotive Engineers) Levels 0, 1, and 2) and three uni-modality secondary tasks. Thirty-six individuals participated in the driving simulation experiment. NASA-TLX (Task Load Index), secondary task performance, and eye-tracking metrics were used as indicators of mental workload. The subjective rating using the NASA-TLX showed a main effect of autonomous level on mental workload in both visual and auditory tasks. Correlation-matrix calculation and principal-component extraction indicated that pupil diameter change, number of saccades, saccade duration, fixation duration, and 3D gaze entropy were effective indicators of a driver’s mental workload in the visual and auditory multi-tasking situations of semi-autonomous driving. The accuracy of predicting the mental-workload level using the K-Nearest Neighbor (KNN) classifier was 88.9% with bootstrapped data. These results can be used to develop an adaptive multi-modal interface that issues efficient and safe takeover requests.
Conference Paper
Full-text available
Automated vehicles will change the interaction with the user drastically. While freeing the user of the driving task for most of the journey, the "final 100 meters problem", directing the vehicle to the final parking spot, could require human intervention. Therefore, we present a classification of interaction concepts for automated vehicles based on modality and interaction mode. In a subsequent Virtual Reality study (N=16), we evaluated sixteen interaction concepts. We found that the medially abstracted interaction mode was consistently rated most usable over all modalities (joystick, speech, gaze, gesture, and tablet). While the steering wheel was still preferred, our findings indicate that other interaction concepts are usable if the steering wheel were unavailable.
Book
The main goal of the field of augmented cognition is to research and develop adaptive systems capable of extending the information management capacity of individuals through computing technologies. Augmented cognition research and development is therefore focused on accelerating the production of novel concepts in human-system integration and includes the study of methods for addressing cognitive bottlenecks (e.g., limitations in attention, memory, learning, comprehension, visualization abilities, and decision making) via technologies that assess the user’s cognitive status in real time. A computational interaction employing such novel system concepts monitors the state of the user, through behavioral, psychophysiological, and neurophysiological data acquired from the user in real time, and then adapts or augments the computational interface to significantly improve their performance on the task at hand. The International Conference on Augmented Cognition (AC), an affiliated conference of the HCI International (HCII) conference, arrived at its 16th edition and encouraged papers from academics, researchers, industry, and professionals, on a broad range of theoretical and applied issues related to augmented cognition and its applications. The field of augmented cognition has matured over the years to solve enduring issues such as portable, wearable neurosensing technologies and data fusion strategies in operational environments. These innovations coupled with better understanding of brain and behavior, improved measures of brain state change, and improved artificial intelligence algorithms have helped expand the augmented cognition focus areas to rehabilitation, brain-computer interfaces, and training and education. The burgeoning field of human-machine interfaces such as drones and autonomous agents are also benefitting from augmented cognition research. This volume of the HCII 2022 proceedings is dedicated to this year’s edition of the AC conference and focuses on topics related to understanding human cognition and behavior, brain activity measurement and electroencephalography, human and machine learning, and augmented cognition in extended reality. Papers of this one volume are included for publication after a minimum of two single-blind reviews from the members of the AC Program Board or, in some cases, from members of the Program Boards of other affiliated conferences. We would like to thank all of them for their invaluable contribution, support, and efforts.
Article
The notion of Ability-Based Design , put forth by Wobbrock et al. [80, 82] as a solution to the challenge of creating accessible technology, has been discussed in human-computer interaction research now for over a decade. However, despite being cited as influential on various projects, the concept still lacks a general characterization of how to implement its required focus on abilities. In particular, it lacks a formulation of how to perceive and model users within an articulated design process. To address this shortcoming, we rely on conceptual user modeling to examine Ability-Based Design and propose a characterization of it that is not dependent upon a specific project or research effort, but that enables the ability-based design of new technologies in a systematic manner. Our findings show that Ability-Based Design’s focus on abilities requires important changes in typical user modeling approaches that cannot be met with current techniques. Based on the challenges identified through our analysis, we propose a first modification not only of current user modeling, but of current requirements analysis approaches to address abilities and their intertwined dependencies with tasks and contexts as core elements of conceptual models in Ability-Based Design. We thereby demonstrate not only the complexity of modeling users’ abilities, but also draw out promising ideas and perspectives for future research, emphasizing the need for future evaluative work on our approach.
Chapter
Full-text available
Research in service robotics strives at having a positive impact on people’s quality of life by the introduction of robotic helpers for everyday activities. From this ambition arises the need of enabling natural communication between robots and ordinary people. For this reason, Human-Robot Interaction (HRI) is an extensively investigated topic, exceeding language-based exchange of information, to include all the relevant facets of communication. Each aspect of communication (e.g. hearing, sight, touch) comes with its own peculiar strengths and limits, thus they are often combined to improve robustness and naturalness. In this contribution, an HRI framework is presented, based on pointing gestures as the preferred interaction strategy. Pointing gestures are selected as they are an innate behavior to direct another attention, and thus could represent a natural way to require a service to a robot. To complement the visual information, the user could be prompted to give voice commands to resolve ambiguities and prevent the execution of unintended actions. The two layers (perceptive and semantic) architecture of the proposed HRI system is described. The perceptive layer is responsible for objects mapping, action detection, and assessment of the indicated direction. Moreover, it has to listen to uses’ voice commands. To avoid privacy issues and not burden the computational resources of the robot, the interaction would be triggered by a wake-word detection system. The semantic layer receives the information processed by the perceptive layer and determines which actions are available for the selected object. The decision is based on object’s characteristics, contextual information and user vocal feedbacks are exploited to resolve ambiguities. A pilot implementation of the semantic layer is detailed, and qualitative results are shown. The preliminary findings on the validity of the proposed system, as well as on the limitations of a purely vision-based approach, are discussed.
ResearchGate has not been able to resolve any references for this publication.