Edward P. Fitts’s research while affiliated with North Carolina State University and other places

What is this page?


This page lists works of an author who doesn't have a ResearchGate profile or hasn't added the works to their profile yet. It is automatically generated from public (personal) data to further our legitimate goal of comprehensive and accurate scientific recordkeeping. If you are this author and want this page removed, please let us know.

Publications (23)


A Conditional Variational Auto-encoder Model for Reducing Musculoskeletal Disorder Risk during a Human-Robot Collaboration Task
  • Article

October 2023

·

20 Reads

·

2 Citations

Proceedings of the Human Factors and Ergonomics Society Annual Meeting

Liwei Qing

·

Bingyi Su

·

Ziyang Xie

·

[...]

·

Edward P. Fitts

In recent years, there has been a trend to adopt human-robot collaboration (HRC) in the industry. In previous studies, computer vision-aided human pose reconstruction is applied to find the optimal position of point of operation in HRC that can reduce workers’ musculoskeletal disorder (MSD) risks due to awkward working postures. However, the reconstruction of human pose through computer-vision may fail due to the complexity of the workplace environment. In this study, we propose a data-driven method for optimizing the position of point of operation during HRC. A conditional variational auto-encoder (cVAE) model-based approach is adopted, which includes three steps. First, a cVAE model was trained using an open-access multimodal human posture dataset. After training, this model can output a simulated worker posture of which the hand position can reach a given position of point of operation. Next, an awkward posture score is calculated to evaluate MSD risks associated with the generated postures with a variety of positions of point of operation. The position of point of operation that is associated with a minimum awkward posture score is then selected for an HRC task. An experiment was conducted to validate the effectiveness of this method. According to the findings, the proposed method produced a point of operation position that was similar to the one chosen by participants through subjective selection, with an average difference of 4.5 cm.



Figure 2. The workflow of the proposed collision avoidance method. Directions of each axis in camera space are marked on the webcam.
Figure 3. The video captured that one was moving in x, y, z directions of robot workspace (top frame 1-6). The recorded motion in the camera space (a) and workspace (b) are shown. The colors of the rectangles in (a) and (b) indicate specific timeframe from 1 to 6. in video and plot. The workspace coordinate system is shown in (c).
A human-robot collision avoidance method using a single camera
  • Article
  • Full-text available

October 2022

·

28 Reads

·

3 Citations

Proceedings of the Human Factors and Ergonomics Society Annual Meeting

Human-robot collaboration is a flourishing work configuration in modern plants. Yet, the potentially hazardous collision between human workers and collaborative robots raises safety concerns. In this study, we proposed a collision avoidance method in which a single camera and a computer-vision algorithm were deployed to sense the location of human workers. Two collision avoidance schemes were further developed to determine the timing for robot to retract its arm. Specifically, the static scheme continuously monitors whether a worker is in a hazard zone, while the dynamic scheme predicts worker’s position after a short time, and monitors whether the predicted worker’s position is in a hazard zone. Preliminary validation showed that our proposed method can effectively enable a collaborative robot to retract its arms when a worker is approaching.

Download

A computer vision-based lifting task recognition method

October 2022

·

30 Reads

·

4 Citations

Proceedings of the Human Factors and Ergonomics Society Annual Meeting

Low-back musculoskeletal disorders (MSDs) are major cause of work-related injury among workers in manual material handling (MMH). Epidemiology studies show that excessive repetition is one of major risk factors of low-back MSDs. Thus, it is essential to monitor the frequency of lifting tasks for an ergonomics intervention. In the current field practice, safety practitioners need to manually observe workers to identify their lifting frequency, which is time consuming and labor intensive. In this study, we propose a method that can recognize lifting actions from videos using computer vision and deep neural networks. An open-source package OpenPose was first adopted to detect bony landmarks of human body in real time. Interpolation and scaling techniques were then applied to prevent missing points and offset different recording environments. Spatial and temporal kinematic features of human motion were then derived. These features were fed into long short-term memory networks for lifting action recognition. The results show that the F1-score of the lifting action recognition is 0.88. The proposed method has potential to monitor lifting frequency in an automated way and thus could lead to a more practical ergonomics intervention.


Working Memory Load Impact on Effective Connectivity: a Dynamic Causal Modeling Study

October 2022

·

20 Reads

Proceedings of the Human Factors and Ergonomics Society Annual Meeting

This study aims to examine the roles of the brain regions in working memory (WM) processing and the modulatory effects of WM load. Electroencephalogram (EEG) signals were recorded when participants perform a letter-version n-back task with three different levels of WM load. The directional causal connections between brain regions were estimated using Dynamic Causal Modeling (DCM). The directions and strengths of the connections were compared for different WM load conditions. The results showed a right-lateralized, backward-only connection pattern for the high WM load condition. The results also showed changes in the roles of the brain regions when the WM load increases. These findings of the modulatory effects of WM load may be utilized in measuring cognitive states, and designing adaptive automation in augmented cognition programs.


Measurements of Mental Stress and Safety Awareness during Human Robot Collaboration -Review

October 2022

·

34 Reads

·

1 Citation

Proceedings of the Human Factors and Ergonomics Society Annual Meeting

Human-robot collaboration is an emerging research area that has gained tremendous attention in both academia and industry. Yet, the feature that human and robot sharing the workplace has led to safety concerns. In particular, the psychological states of human teammates during human-robot collaboration remains unclear but is also of great importance to workplace safety. This manuscript briefly reviewed possible direct and indirect measures that can be used to evaluate workers’ mental stress and safety awareness during human robot collaboration. It was concluded that each measure reviewed in this paper has its validity and rationality, and a combination of different methods may provide a more comprehensive and accurate assessment.


A Single-Camera Computer Vision-Based Method for 3D L5/S1 MomentEstimation During Lifting Tasks

September 2021

·

21 Reads

·

3 Citations

Proceedings of the Human Factors and Ergonomics Society Annual Meeting

Excessive low back joint loading during material handling tasks is considered a critical risk factor of musculoskeletal disorders (MSD). Therefore, it is necessary to understand the low-back joint loading during manual material handling to prevent low-back injuries. Recently, computer vision-based pose reconstruction methods have shown the potential in human kinematics and kinetics analysis. This study performed L5/S1 joint moment estimation by combining VideoPose3D, an open-source pose reconstruction library, and a biomechanical model. Twelve participants lifting a 10 kg plastic crate from the floor to a knuckle-height shelf were captured by a camera and a laboratory-based motion tracking system. The L5/S1 joint moments obtained from the camera video were compared with those obtained from the motion tracking system. The comparison results indicate that estimated total peak L5/S1 moments during lifting tasks were positively correlated to the reference L5/S1 joint moment, and the percentage error is 7.7%.


Examining human perception of weight during loaded standing and walking

September 2021

·

35 Reads

Proceedings of the Human Factors and Ergonomics Society Annual Meeting

While the psychophysics of weight perception may help assess the effort needed in manual material handling tasks, the perception of weight is subjective and not necessarily accurate. The purpose of this study was to examine weight perception during standing and walking. Participants (n=10) performed a series of weight comparison trials against a reference load while holding loads (standing) or carrying loads (walking). Polynomial logistic regression models were built to examine the effects of walking, box weight ratio, and reference weight level on the probability of detecting a weight difference. The results showed that weight ratio and reference weight level had statistically significant effects on the detection probability while walking did not have a significant effect. Findings from this study can help inform the design of subjective evaluation of job demands involving motion, and it can be further extended to the gradual increase in load of strengthening tasks in therapeutic exercises.


Effect of body-gender transfer in virtual reality on the perception of sexual harassment

September 2021

·

36 Reads

·

6 Citations

Proceedings of the Human Factors and Ergonomics Society Annual Meeting

While sexual harassments are inappropriate behaviors in the society, the interpretation of and sensitivity toward sexual harassment can vary by individual. Differences across individuals, such as gender, may influence whether one interprets an action to be sexually harassing or not. Virtual reality technology enables human behavior assessment without interfacing with physical danger. The present work examined whether gender and body-gender transfer in VR influenced the perception of sexually harassing behaviors, and explored the utility of emerging technology in increasing one’s awareness of behaviors that may be considered sexually harassing. Participants (n=12) embodied in virtual characters of different genders and experienced seven sexually harassing scenarios in an immersive virtual environment in random order. In general, participants provided higher rating to the sensitivity toward sexual harassment in the VR harassment scenarios than those scenarios described on paper. There was an increase in participants’ sensitivity toward sexual harassment after experiencing sexual harassment scenarios from the perspective of the victim in VR. Participants perceived higher level of sexual harassment when they embodied in female avatars, which suggested there was an effect of VR with body-gender transfer on perception of sexual harassment. There were gender differences in awareness of harassing behaviors in VR environment, and VR may be a training method to narrow gender gap and increase awareness toward sexual harassment.


Neural Correlates of Trust During an Automated System Monitoring Task: Preliminary Results of an Effective Connectivity Study

November 2019

·

17 Reads

·

11 Citations

Proceedings of the Human Factors and Ergonomics Society Annual Meeting

As autonomous systems become more prevalent and their inner workings become more opaque, we increasingly rely on trust to guide our interactions with them especially in complex or rapidly evolving situations. When our expectations of what automation is capable of do not match reality, the consequences can be sub-optimal to say the least. The degree to which our trust reflects actual capability is known as trust calibration. One of the approaches to studying this is neuroergonomics. By understanding the neural mechanisms involved in human-machine trust, we can design systems which promote trust calibration and possibly measure trust in real time. Our study used the Multi Attribute Task Battery to investigate neural correlates of trust in automation. We used EEG to record brain activity of participants as they watched four algorithms of varying reliability perform the SYSMON subtask on the MATB. Subjects reported their subjective trust level after each round. We subsequently conducted an effective connectivity analysis and identified the cingulate cortex as a node, and its asymmetry ratio and incoming information flow as possible indices of trust calibration. We hope our study will inform future work involving decision-making and real-time cognitive state detection.


Citations (13)


... ML can model complex, non-linear interactions between numerous variables, making it well-suited for understanding the multifaceted etiology of WMSDs and aiding in their prevention. Despite a trade-off between interpretability and predictive performance, ML techniques are advancing primary WMSD prevention efforts [20] [21] [9] [2] [26]. ...

Reference:

Real-Time Posture Monitoring and Risk Assessment for Manual Lifting Tasks Using MediaPipe and LSTM
A computer vision-based lifting task recognition method
  • Citing Article
  • October 2022

Proceedings of the Human Factors and Ergonomics Society Annual Meeting

... Bullying, harassment, and, specifically, sexual harassment in VR environments has been reported (Chang et al., 2019;Parshall, 2022). As VR technology can be used to simulate experiences and perspectives in realistic ways, it is critical to ensure that it is not used to perpetuate harmful stereotypes, discrimination, or patterns of abuse (Bale et al., 2022;Wu et al., 2021). When these issues happen in the material world, they are often traumatizing. ...

Effect of body-gender transfer in virtual reality on the perception of sexual harassment
  • Citing Article
  • September 2021

Proceedings of the Human Factors and Ergonomics Society Annual Meeting

... For example, Hergeth et al. (2016) continuously measured human-machine trust by capturing the staring behavior of the operator through eye tracking. Choo et al. (2019) used the image features of EEG signals to detect the operator's human-machine trust state. Physiological information measurement is both continuous and real-time, making it suitable for the real-time monitoring of a driver's physiological state while performing the continuous task of driving. ...

Detecting Human Trust Calibration in Automation: A Deep Learning Approach
  • Citing Article
  • November 2019

Proceedings of the Human Factors and Ergonomics Society Annual Meeting

... Contemporary technology advances in automation, artificial intelligence (AI), and computational algorithms has created great benefits for human operators, along with increases in adverse or unexpected consequences [1], [2], [3]. To ensure safe and seamless human-machine interaction (HMI), trust between humans and machines (in the form of AI or automated tools) has become a widely discussed topic over the past three decades [4], [5], [6]. ...

Neural Correlates of Trust During an Automated System Monitoring Task: Preliminary Results of an Effective Connectivity Study
  • Citing Article
  • November 2019

Proceedings of the Human Factors and Ergonomics Society Annual Meeting

... Sci. 2021, 11, 9961 2 of 10 Horn [19] polyurethane foam is a good alternative for human cancellous bone as it displays similar mechanical properties and may be used as a medium for implant testing. Horak et al. [17] conducted experimental studies to evaluate its mechanical properties (temperature, strain and density) and reported that it is not only suitable for mechanical investigations but also for investigations involving surgical instruments that generate heat. ...

Design and manufacturing of bone analog models for the mechanical evaluation of custom medical implants

... , Helm and Van Oyen (2014), Berg et al. (2011), Price et al. (2011), Liu et al. (2010, Adan et al. (2009), Muthuraman andLawley (2008), Patrick et al. (2008), Hsu et al. (2003), Dexter and Traub (2002), Gerchak et al. (1996) Deglise Liu and Karimi (2008) (MILP), Thornton and Hunsucker (2004) (Heuristics) ...

Optimal Booking Strategies for Outpatient Procedure Centers

... Online trading adds further challenges [3], requiring the real-time use of one or multiple algorithms, often implemented as software agents. Currently, most trading systems are based on single or numerous algorithms without employing agents [4][5][6][7][8][9][10]. They are also based on the single agent architecture [11]. ...

Analysis of market returns using multifractal time series and agent-based simulation
  • Citing Conference Paper
  • December 2012

... Empathetic robots are more successful in establishing rapport with users, who interact more fluidly with service robots that exhibit empathy, leading to higher-quality service interactions (Leite et al. 2013). In healthcare services, incorporating anthropomorphic and interactive elements into service robotics may inspire older users to feel better (Zhang et al. 2010), which emphasizes the significance of anthropomorphic features in medical service robots on patient attitudes. Meanwhile, in order to further explore the patients' emotional experiences in their interactions with the medical service robot, we subdivided the original variable of anthropomorphism. ...

Service robot feature design effects on user perceptions and emotional responses

Intelligent Service Robotics

... It also allows for controlling continuous parameters, but requires careful design considerations on how the user's input can be mapped to control commands. Kim and Kaber[38] have found the users tend to prefer rate-based (1 st order) control approach than position-based (0 order) control in foot-based methods.• Visibility and feedback : Moreover, similar with the other gestural interaction, foot movements may be less accurate when there is no visibility of the limbs. ...

Design and evaluation of dynamic text-editing methods using foot pedals
  • Citing Article
  • March 2009

International Journal of Industrial Ergonomics

... A well-known case occurs when a random input variable follow a non-stationary (time-dependent) process in which its distribution changes in such a way that the basic functional form is stable, but it has parameter(s) that depend on time. Kuhl et al. (2009) refers to this problem as parameter uncertainty . For example, in many applications, customer arrival rates in rush hours are larger than the other periods. ...

ADVANCES IN MODELING AND SIMULATION OF NONSTATIONARY ARRIVAL PROCESSES