Available via license: CC BY 4.0
Content may be subject to copyright.
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3059710, IEEE Access
Date of publication xxxx 00, 0000, date of current version xxxx 00, 0000.
Digital Object Identifier 10.1109/ACCESS.2017.DOI
Effects of Machine Instability Feedback
on Safety During Digging Operation in
Teleoperated Excavators
MASARU ITO1, CHIAKI RAIMA2, SEIJI SAIKI3, YOICHIRO YAMAZAKI3,
AND YUICHI KURITA1, (Member, IEEE)
1Graduate School of Advanced Science and Engineering, Hiroshima University, Hiroshima 739-8527, Japan
2Office of Academic Research and Industry-Academia-Government and Community Collaboration, Hiroshima University, Hiroshima 739-8527, Japan
3Kobelco Construction Machinery Co., Ltd., Tokyo 141-8626, Japan
Corresponding author: Masaru Ito (e-mail: itoma@hiroshima-u.ac.jp).
ABSTRACT Teleoperated excavators have a risk of overturning or falling when digging because it is
more difficult to receive information regarding the machine posture and the work object condition than
in the case of actual machine boarding. In this study, the machine instability derived from the attachment
posture and the digging reaction force has been proposed. In addition, for feedback regarding the machine
instability to the operator, an intuitive visual presentation method using a meter is proposed. To verify the
effect of the proposed machine instability feedback, an excavation operation simulator was constructed, and
an experiment was conducted. As a result, by using the machine instability feedback, it was confirmed that
digging could be performed without generating the tilting and forward movements of the machine body,
and that the ratio of the time during which the machine instability exceeded 100% decreased, while the
productivity remained constant. Additionally, by verifying the effect of the subject experiment, even in
the actual teleoperated excavator, it was confirmed that the work could be conducted without generating a
greater tilting of the machine body, while maintaining productivity. Therefore, the possibility of working
safely in the teleoperated excavator using machine instability feedback was clarified.
INDEX TERMS Hydraulic excavator, man-machine systems, teleoperation, unmanned construction, user
interfaces
I. INTRODUCTION
HYDRAULIC excavators are used in various works
such as excavation and dismantling. In recent years,
teleoperated excavators have been applied in fields that are
challenging for humans to enter, such as disaster fields [1].
However, mainstream teleoperated excavators [2] have prob-
lems regarding work efficiency and safety.
The work efficiency of a teleoperated excavator is lower
than that of a boarding operation because of the delay, and
difficulty in understanding the worksite conditions. Addi-
tionally, with regard to safety, it is challenging to receive
information regarding the posture of the machine and the
condition of the work object compared to when on board
the actual machine [3], so there is a risk of overturning.
Especially in the case of excavation, if the reaction force
received from the soil increases, the machine body moves
depending on the posture of the attachment, and the risk
of overturning increases when operated on irregular grounds
and uplands. Eliminating such conditions is thus desirable.
According to Sakaida et al. [4], it is suggested that the
skilled operator operates the excavator while controlling the
load to the bucket according to the change in the posture of
the bucket. Thus, it is assumed that the operator estimates
the load applied to the machine from the slight change in
the tilt of the machine as perceived by the operator, and then
the difference between the maximum load that the machine
can sustain without the machine body moving and the current
load, that is, the load margin; the operator changes the oper-
ation when the load margin is small to dig without moving
the machine body. As information regarding the machine
tilt is received by the teleoperated excavator using monitors
without direct visibility, it is difficult for the operator to
estimate the load margin; thus, the risk of tipping over and
falling is high. Therefore, it is necessary to consider a method
that provides information to the teleoperated excavator oper-
ator regarding machine posture and the load applied to the
VOLUME 4, 2016 1
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3059710, IEEE Access
Ito et al.: Effects of Machine Instability Feedback During Digging Operation on Safety in Teleoperated Excavators
machine.
As a system for feeding back the movement of the ex-
cavator to the operator, Zhao et al. [5] and the authors [6]
researched the feedback of tilt information of the excavator
body using a motion base. It is possible to feel the movement
of the machine; therefore, the operator could estimate the
load acting on the bucket. However, there is a problem in that
expensive actuators and controllers are required to reproduce
slight tilting. Additionally, the introduction of force feedback
has also been studied to provide feedback on the digging
reaction force. For example, Parker et al. [7] and Lawrence et
al. [8] used a mounted force feedback operation lever for the
excavator. Ahn [9], Li [10], and Truong et al. [11] developed
a force feedback operation lever for a teleoperated excavator.
Gong et al. [12] and Hou and Zhao [13] developed a force
feedback system in the case of using a fork attachment.
Huang et al. [14] enabled the discrimination of the hardness
of an object grasped by a fork attachment using an operation
system that combines force and visual feedback. Zareinia et
al. [15] designed a control scheme for a haptic device that
provides haptic force based on the position error between the
displacements of the master and the slave. Lampinen et al.
[16] designed a full-dynamics-based bilateral force-reflected
teleoperation for hydraulic manipulators. Carvalho et al. [17]
proposed a force feedback strategy using a 3D haptic device
for teleoperated excavators. Haptic feedback has also been
studied [18]–[21]. However, it is not easy to estimate the
load margin from only the digging reaction force information
because this margin changes depending on the attachment
posture. In addition, the installation of the force and haptic
feedback has a problem in which the operation feeling may
be changed for the operator who usually works on board. To
overcome this problem, there is a study by Tanimoto et al.
[22], which provides assistance while maintaining the oper-
ation feeling; however, it is difficult to adapt to excavation
work without an ideal trajectory. Therefore, in this study, we
aimed to change the operation of the operator without chang-
ing the operation feeling by adding information presentation
to the operator.
On the other hand, regarding the safety of teleoperated
excavators, some studies have proposed tip-over prevention
control using a static compensation zero moment point al-
gorithm [23] and by predicting the center of gravity and the
zero moment point [24]. These studies focused on improving
safety when operating the attachment, but not on the excava-
tion. Research on improving safety during digging operations
has not been conducted thus far.
Herein, the operator is presented with the amount of the
load margin. Thus, it is expected that the operator is aware of
the load margin available, leading to safe digging operation.
In this study, machine instability derived from the attachment
posture and the digging reaction force is proposed. In addi-
tion, as a method for providing feedback about the machine
instability to the operator, a visual presentation method using
a meter is proposed, so that the amount of load margin
can be intuitively understood. This study verified that the
Bucket
Arm
Boom Attachment
Lower
Traveling
Body
Upper
Rotating
Body
FIGURE 1: Parts of hydraulic excavator.
machine instability feedback could improve the safety during
digging work using a teleoperated excavator and how work
performance would change.
The preliminary experimental results based on the simula-
tion experiments were explained in International Conference
on Human System Interaction (HSI) 2020 [25]. The new
contributions of this paper are the subject experiments using
an actual teleoperated excavator and investigation and an
evaluation of the proposed method based on the results of
both the simulator and the actual teleoperated excavator.
This paper is organized as follows. Section II introduces
the definition of the proposed machine instability. Section III
describes the digging operation simulator developed for the
verification of the effect of the machine instability feedback
and the results of the verification using the simulator. Section
IV describes the results of the verification of the effect of ma-
chine instability feedback on actual teleoperated excavators.
Section V describes the general discussion and Section VI
describes the conclusions drawn from this study.
II. DEFINITION OF MACHINE INSTABILITY
A hydraulic excavator carries out digging work by operating
an attachment, consisting of a boom, an arm, and a bucket
(Fig. 1). When the reaction force from the digging object
increases, the excavator body starts to move depending on
the posture of the attachment. For safe operation, it is de-
sirable to eliminate this condition while digging. Therefore,
it is proposed a parameter, the machine instability. Machine
instability is an index of the excavator condition with the
condition when the excavator body starts to move which is
assumed to be 100%, and it derived from the attachment
posture and the digging reaction force. Two instabilities were
considered: the behavior of tilting around the rear end of the
lower traveling body as the center of rotation, as shown in
Fig. 2a, and the behavior of the body being dragged forward,
as shown in Fig. 2b. The value that is greater between these
two instabilities is defined as machine instability. In this
study, it is assumed that the machine is installed on horizontal
ground. Machine motion in the lateral direction and the
weight of the attachment are not considered.
2VOLUME 4, 2016
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3059710, IEEE Access
Ito et al.: Effects of Machine Instability Feedback During Digging Operation on Safety in Teleoperated Excavators
(a) Upward direction. (b) Forward direction.
FIGURE 2: Example of machine behavior when digging
reaction force is large.
A. UPWARD INSTABILITY
As shown in Fig. 3, considering the moment when the rear
end of the lower traveling body is the center of rotation, O,
the moment in the counterclockwise direction caused by the
machine weight Mmis given by
Mm=lg·mg cos θg(1)
where mis the machine weight, lgis the distance from O
to the center of gravity of the machine weight, G, θgis the
angle between the ground and lg, and gis the gravitational
acceleration.
The clockwise moment Mfcaused by the digging reaction
force is given by
Mf=lb·Fsin (θb−θf)(2)
where Fis the digging reaction force, θfis the angle between
the digging reaction force direction and the ground, lbis the
distance from O to the digging reaction point, P, and θbis the
angle between the ground and lb.
From (1) and (2), the condition under which the machine
is lifted is Mf> Mm. Thus, the upward instability Iup (%)
is defined as
Iup =Mf
Mm
·100.(3)
If Iup >100, the machine body tilts.
B. FORWARD INSTABILITY
As shown in Fig. 4, considering the force in the horizontal
direction, the maximum static friction force Fmis given by
Fm=µmg (4)
where µis the coefficient of static friction between the
machine and ground.
The force Fbcaused by the digging reaction force is given
by
Fb=Fcos θf.(5)
b
fb
sin( b−f)
g
cosg
g
O
P
G
FIGURE 3: Moments of force around rear end of lower
traveling body ground surface.
f
cosf
FIGURE 4: Horizontal forces.
From (4) and (5), the condition under which the machine
is dragged forward is Fb> Fm. Based on this, the forward
instability Ifor (%) is defined as
Ifor =Fb
Fm
·100.(6)
If Ifor >100, the machine body will be dragged forward.
C. MACHINE INSTABILITY
From (3) and (6), the machine instability I(%) is defined as
I=Iup (Iup > Ifor)
Ifor (otherwise).(7)
This is an index of the excavator condition assuming that
when the machine body starts to lift or drag forward, machine
instability is 100%.
III. SIMULATOR EXPERIMENTS
A. SIMULATOR CONFIGURATION
First, an evaluation using a simulator was carried out to
verify the effect of presenting machine instability. In previous
research on the excavation simulator, Dopico et al. [26]
developed a full 3D physics-based excavator simulator made
up of 14 rigid bodies with 17 degrees of freedom and a
terrain mesh model of soil. Ni et al. [27] developed a multiple
display virtual reality system for the excavator simulator with
a real-time optimally adapting mesh algorithm for producing
terrain deformation. However, these simulators have a high
calculation cost for simulating soil behavior. Therefore, we
VOLUME 4, 2016 3
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3059710, IEEE Access
Ito et al.: Effects of Machine Instability Feedback During Digging Operation on Safety in Teleoperated Excavators
FIGURE 5: Excavator simulator appearance.
Operator
Joysticks
Monitors
Movement of attachments
Reaction force
Movement of machine
Excavated soil amount
Machine instability
Interface PC (Unity)
FIGURE 6: Schematic configuration of excavator simulator.
developed a new simulator specialized in digging operation
with a low computational cost compared with these studies,
according to the purpose of this study. The excavator sim-
ulator is shown in Fig. 5, and the schematic configuration
of the simulator is shown in Fig. 6. The simulator consisted
of two joysticks, three monitors, and a PC; the simulation
environment was created using a game engine, Unity. In the
simulation environment, a 3D model simulating a 13t-class
hydraulic excavator SK135SR-3 made by Kobelco Construc-
tion Machinery Co., Ltd. was arranged and rotated by 90◦.
The image displayed on the monitors was presented from
the viewpoint of the actual excavator operation seat, and the
joysticks were used in the same way as those of the actual
hydraulic excavator. As shown in Fig. 7, the operating pattern
conformed to the standards of the international organization
for standardization (ISO). The turning motion, which is not
used in the normal digging operation, is not simulated.
1) Movement of Attachments
The angular velocity of the attachment movement is changed
according to the input from the joysticks, as in the actual
Arm Roll-In
Arm Roll-Out
Swing
Left
Swing
Right
Boom Up
Boom Down
Bucket
Roll-In
Bucket
Roll-Out
Right Side
Left Side
FIGURE 7: Excavator control pattern (ISO standard).
TABLE 1: Values for Transfer Function Parameters
Bucket Edge Position Above the Ground Below the Ground
K T L K T L
Boom Up (Acceleration) 0.35 0.2 0.1
Boom Up (Deceleration) 0.35 0.1 0.1 0.18 0.2 0.1
Boom Down 0.4 0.2 0.1
Arm 0.56 0.2 0.1 KA0.2 0.1
Bucket 1.1 0.08 0.1 0.7 0.08 0.1
excavator. The motion includes a response delay because
the hydraulic excavator operates through a hydraulic circuit
and cylinder. In this study, as proposed by Koiwai et al.
[28], the dynamic characteristics G(s)of the attachment were
assumed to be a first-order plus dead time system as follows:
G(s) = K
1 + T s e−Ls (8)
where Kis the system gain, Tis the time constant, and L
is the dead time. To approximate the movement of a general
13t-class hydraulic excavator, these values were assumed as
shown in Table 1.
In the hydraulic excavator, the hydraulic pump is driven
by the engine output, and it sends hydraulic oil to the hy-
draulic cylinder, which moves the attachment. When the load
to the attachment increases, the amount of oil discharged
from the hydraulic pump is controlled to prevent the engine
output from exceeding the engine capacity and stall, and the
attachment operation speed decreases. Therefore, to improve
the reproducibility, the parameters were changed when the
bucket tip was above the ground (i.e., no digging reaction
force) and below the ground (i.e., digging reaction force
exists). Additionally, because the ratio of the arm motion
is larger than that of the other motions when digging, the
gain KAof the arm motion when the bucket tip is below the
ground is changed, according to the digging reaction force,
to improve the reproducibility of the arm motion. In this
study, the angular velocity is assumed to decrease linearly
as the digging reaction force increases, and KAis defined as
follows:
KA=
KAmax ·Fmax −F
Fmax
(F < Fmax )
0 (otherwise)
(9)
where KAmax is the maximum value of the gain, and Fmax
is the digging reaction force with zero gain. In this study,
4VOLUME 4, 2016
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3059710, IEEE Access
Ito et al.: Effects of Machine Instability Feedback During Digging Operation on Safety in Teleoperated Excavators
Ground Line
(, )
(−1,−1)
(2,2)
(1,1)
(0,0)∆
∆2
∆1
FIGURE 8: Excavated soil volume calculation.
KAmax was 0.56 and Fmax was 55 kN.
2) Calculation and Display Method of Excavated Soil Volume
To calculate the digging reaction force and set the goal in the
task of the simulator experiment, the excavated soil volume
was calculated from the trajectory of the bucket tip, and a
system for displaying the excavated soil in the bucket was
applied. The excavated soil volume was calculated using the
area surrounded by the trajectory of the bucket tip and the
ground. As shown in Fig. 8, assuming that the position of the
bucket tip at an arbitrary time tis (xt, yt), the bucket passing
area S(t)is as follows:
S(t) =
t
X
i=1
∆Si,(10)
∆Si=(1
2|xi−xi−1|(|yi|+|yi−1|) (xi> xi−1∧yi<0)
0 (otherwise).
(11)
In the actual digging work, the excavated soil volume is less
than the volume of the bucket passing area because not all of
the excavated soil enters the bucket. Therefore, the excavated
soil volume V(t)was calculated as
V(t) = S(t)Wbkv(12)
where kvis the ratio of the excavated soil volume to the
volume of the bucket passing area, and Wbis the bucket
width and assumed to be constant.
Here, we describe the manner in which the excavated
soil is displayed. The bucket section is approximated by a
quadratic function. In the x−ycoordinates shown in Fig. 9,
the bucket section is given by
y=4Db
Hb2(x−Hb
2)−Db(13)
where Hbis the bucket opening height, and Dbis the bucket
depth. Assuming that the excavated soil volume increases in
( )
b
( )
b
( )
FIGURE 9: Display method of excavated soil in bucket.
the bucket, as shown in Fig. 9, the bucket soil height H(t)for
the excavated soil volume until the bucket is full is
H(t) = 3
s3Hb2V(t)
2DbWb
.(14)
After the bucket is full, the soil assumes the shape of a square
pyramid, and its height increases, as shown in Fig. 9; the
display height D(t)is
D(t) = 3
WbHb
(V(t)−2
3DbHbWb).(15)
3) Digging Reaction Force
The digging reaction force is calculated using the attachment
posture and the bucket soil height calculated using (14).
Osumi et al. [29] and Meng et al. [30] proposed an estimation
method for the digging reaction force using passive earth
pressure. The passive earth pressure is the earth pressure
generated on the retaining wall that supports sand; the the-
oretical formula was proposed by Coulomb and Rankine,
wherein the digging reaction force was calculated using the
Rankine theory [31]. The theoretical formula of the passive
earth pressure Fpbased on the Rankine theory is
Fp=1
2γh2tan2(45◦+ϕ
2)(16)
where his the retaining wall height, γis the unit weight of
the soil, and ϕis the internal friction angle. γand ϕare
parameters determined by the soil properties. In this study,
the retaining wall height was determined as hs(t)of the
bucket soil height perpendicular to the ground, as shown in
Fig. 10. The angle between the bucket opening surface and
the bucket soil surface, θs(t), is
θs(t) =
tan−1
Db−4Db
Hb
(H(t)−Hb
2)2
H(t)(H(t)< Hb)
0 (otherwise).
(17)
VOLUME 4, 2016 5
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3059710, IEEE Access
Ito et al.: Effects of Machine Instability Feedback During Digging Operation on Safety in Teleoperated Excavators
( )
ℎs()Ground Line
BG
s
f
FIGURE 10: Digging reaction force calculation.
TABLE 2: Values for Soil Parameter
Soil Pattern γ[kN/m3]ϕ[◦]
1 23 45
2 22 44
3 21 43
Assuming that the angle between the bucket opening sur-
face and the ground surface is θBG,hs(t)is
hs(t) = H(t)
cos θs(t)sin(θs+θBG).(18)
In the actual work, the soil inside the bucket is assumed to be
loose. Assuming that the ratio of the height functioning as a
retaining wall to hs(t)is ks, the digging reaction force F(t)
is
F(t) = 1
2γh2
s(t)kstan2(45◦+ϕ
2).(19)
In this study, the digging reaction force was applied to the
bucket tip, and the angle was set to 90◦from the bucket
opening surface.
4) Movement of Machine
To reproduce the behavior of the machine during lifting
and dragging forward, when the machine instability exceeds
100%, the machine is rotated and moved in the reverse
direction of the change in the bucket tip position that is cal-
culated from the input. Thus, although the absolute position
of the bucket tip does not change, and the machine angle and
position are changed.
B. EXPERIMENTS
1) Condition and Protocol
To confirm the effect of presenting the machine instability, a
subject test using the simulator was conducted. Two condi-
tions were tested (with and without machine instability feed-
back), and in each case, three different soil parameter condi-
= 0 = 50 ≥100
FIGURE 11: Machine instability meter for simulator experi-
ments.
tions were used 10 times each for 30 digging trials. In actual
digging, it is difficult to determine the conditions below the
soil surface before digging. To simulate this situation, the
trial order of the soil parameter conditions was randomized.
The soil parameter conditions are given in Table 2. The soil
property is assumed to be constant, and the parameters of
the soil patterns are set so that a difference in the difficulty
of tasks occurs with reference to type (cohesionless gravels)
and soil description (very dense) based on [32]. According
to (16), the digging reaction force against the retaining wall
height decreases in the order of soil patterns 1, 2, and 3,
which is an easy trial. In this study, µwas 0.35, kvwas 0.625,
and kswas 0.667.
The test task was to dig a specified amount of soil as-
suming a full bucket. Assuming that the full bucket has a
20% increase in capacity, for a bucket of 0.5 m3capacity,
0.6 m3was set as the target excavated soil volume. When
the excavated soil volume reached the target excavated soil
volume, the color of the excavated soil was changed to
green, and the subject was informed that the target excavated
soil volume had been reached. The purpose of setting the
target excavated soil volume was to reduce the difference in
excavated soil volume between tasks and subjects, and the
data were not excluded for the trial when the target excavated
soil volume was not reached.
The machine angle, position, machine instability, exca-
vated soil volume, and task time were measured. The task
time started when either lever operation was input, and ended
when the bucket tip was lifted more than 1 m after digging.
The subjects were 10 healthy adult males aged 29 to
54 years with hydraulic excavator operating experience. To
eliminate the effect of trial order, five subjects performed the
task with the feedback of the machine instability first, and
the remaining five subjects performed the task without the
feedback first. All subjects performed the task after practice
under the condition of soil pattern 2 and without the feed-
back. Each subject was instructed to perform digging work
on the target excavated soil volume as rapidly as possible.
The subjects were informed beforehand that on the machine
instability feedback meter, a 100% level indicates the start of
machine motion. Informed consent based on the Declaration
of Helsinki was obtained from all participants prior to the
experiments.
6VOLUME 4, 2016
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3059710, IEEE Access
Ito et al.: Effects of Machine Instability Feedback During Digging Operation on Safety in Teleoperated Excavators
0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
1.6
Soil Pattern 1 Soil Pattern 2 Soil Pattern 3
Max. Machine Angle (°)
Without F/B With Machine Instability F/B
*
n
.s
.
*
FIGURE 12: Differences in mean values of maximum ma-
chine angle for simulation experiment. Error bars indicate
standard deviation. Asterisks indicate significant differences
(*p < .05). Hypothesis for student’s t-test is value of “With-
out F/B" greater than value of “With Machine Instability
F/B."
0
0.02
0.04
0.06
0.08
0.10
0.12
0.14
0.16
0.18
0.20
Soil Pattern 1 Soil Pattern 2 Soil Pattern 3
Machine H. Movement (m)
Without F/B With Machine Instability F/B
n.s.
**
*
FIGURE 13: Differences in mean values of horizontal ma-
chine movement for simulation experiment. Error bars indi-
cate standard deviation. Asterisks indicate significant differ-
ences (*p < .05, **p < .01). Hypothesis for student’s t-
test is value of “Without F/B" greater than value of “With
Machine Instability F/B."
2) Method for Providing Machine Instability Feedback
The machine instability meter is as shown in Fig. 11. The
meter position was to the left of the bucket, and when the
machine instability was 0%, it was displayed with a translu-
cent pattern and became nontransparent from the bottom
according to the magnitude of the machine instability. The
ratio displayed in the nontransparent pattern was the same as
that of the machine instability. The threshold value was set
at 100%, and the meter became completely nontransparent
when the machine instability exceeded the threshold value.
0.00
0.05
0.10
0.15
0.20
0.25
0.30
Soil Pattern 1 Soil Pattern 2 Soil Pattern 3
Unstable State Time Ratio
Without F/B With Machine Instability F/B
**
n.s.
**
FIGURE 14: Differences in mean values of unstable state
time ratio for simulation experiment. Error bars indicate
standard deviation. Asterisks indicate significant differences
(**p < .01). Hypothesis for student’s t-test is value of “With-
out F/B" greater than value of “With Machine Instability
F/B."
0.00
0.02
0.04
0.06
0.08
0.10
0.12
Soil Pattern 1 Soil Pattern 2 Soil Pattern 3
Work Efficiency (m
3
/s)
Without F/B With Machine Instability F/B
n.s.
n.s
.
n
.s
.
FIGURE 15: Differences in mean values of work efficiency
for simulation experiment. Error bars indicate standard devi-
ation.
3) Results
The experimental results were evaluated in terms of pro-
ductivity and safety. Each index refers to the mean value
per subject for each condition and soil pattern. “Without
F/B” in the figure means the case without machine instability
feedback, and “With Machine Instability F/B” means the case
with machine instability feedback. A p-value of less than 0.05
was regarded as statistically significant.
Fig. 12 demonstrates the mean of the maximum machine
angle in one task. Student’s t-test indicates that the maximum
machine angle with machine instability feedback is signifi-
cantly lower than that without machine instability feedback
in the case of soil pattern 1 (t(9) = 2.817, p =.010) and soil
pattern 2 (t(9) = 2.058, p =.035). No significant decrease
was observed in the case of soil pattern 3 (t(9) = 0.948, p =
.184) between the two conditions.
Fig. 13 shows the mean of the maximum horizontal move-
VOLUME 4, 2016 7
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3059710, IEEE Access
Ito et al.: Effects of Machine Instability Feedback During Digging Operation on Safety in Teleoperated Excavators
Actuator
Operator
Joysticks
Monitors
Cockpit Teleoperated Excavator
Controller
PC
Wireless
Network
Controller Excavator
Sensors
Camera
Control CommandsJoysticks
Boom Angle
Boom Cylinder Pressure
Digging Reaction Force
Point of Application
Arm Angle
Arm Cylinder Pressure
Bucket Angle
Bucket Cylinder Pressure
Machine Instability
Visual Image
Input Signal
Visual Image
FIGURE 16: Schematic configuration of teleoperated excavator.
ment in one task. Student’s t-test indicates that the maximum
horizontal movement with machine instability feedback is
significantly lower than that without machine instability feed-
back in the case of soil pattern 1 (t(9) = 2.001, p =.038)
and soil pattern 2 (t(9) = 2.856, p =.009). No significant
decrease was observed in the case of soil pattern 3 (t(9) =
0.874, p =.202) between the two conditions.
Fig. 14 shows the mean of the unstable state time ratio
obtained by dividing the time when the machine instability
exceeds 100% by the task time. A smaller vertical axis
value results in a shorter working time in the unstable state.
Student’s t-test indicates that the unstable state time ratio
with machine instability feedback is significantly lower than
that without machine instability feedback in the case of soil
pattern 1 (t(9) = 3.598, p =.003) and soil pattern 2 (t(9) =
2.905, p =.009). No significant decrease was observed in
the case of soil pattern 3 (t(9) = 1.322, p =.109) between
the two conditions.
In terms of productivity, Fig. 15 shows the mean work
efficiency obtained by dividing the excavated soil volume by
the task time. Student’s t-test was conducted for the condition
with or without machine instability feedback; there were no
significant differences in the case of soil pattern 1 (t(9) =
1.577, p =.149), soil pattern 2 (t(9) = 1.725, p =.119),
and soil pattern 3 (t(9) = 1.973, p =.080).
4) Discussion
The maximum machine angle, maximum horizontal move-
ment, and unstable state time ratio showed significant de-
creases except for soil pattern 3. Therefore, it can be said that
by presenting machine instability, the amount of movement
of the machine and the time when the machine body was
moving decreased. Soil pattern 3 was the easiest trial condi-
tion, and it was inferred that there was no difference because
the machine instability rarely exceeded 100%, as shown in
Fig. 14. Regarding productivity, there was no significant
difference in work efficiency under any soil pattern; it is con-
sidered that the machine instability feedback did not reduce
productivity. It was shown that by feeding back the machine
instability, digging work could be performed safely without
reducing productivity. This comment was given: “When the
machine instability becomes 100% or more, it is not possible
to check by how much the instability exceeds 100%, so the
extent to which the lever operation should be changed is not
clear." after the trial. Therefore, in the experiment using the
actual teleoperation excavator, the meter was changed.
IV. EXPERIMENTS WITH TELEOPERATED EXCAVATOR
A. TELEOPERATED EXCAVATOR CONFIGURATION
Next, to verify the effect of the machine instability feedback
in an actual teleoperated excavator, a test was conducted.
The schematic configuration of the teleoperated excavator
used for the experiment is shown in Fig. 16. This is the
modification of the one developed by the authors [6]; the list
of modifications are follows:
•The hydraulic excavator changed from 7t-class to 13t-
class (SK135SR-3).
•A digging reaction force measurement system was in-
stalled.
•The number of cameras was changed from two to one.
•Visual information was changed from the head mounted
display to monitors.
According to the input of the joysticks by the operator, the
joystick on the teleoperated excavator side is operated by the
actuator. The operation method is the same as that of the
simulator, as shown in Fig. 7. The digging reaction force and
its point of application are calculated by the controller in the
teleoperated excavator based on the values from the pressure
sensors and angle sensors of the attachment. The system
had a delay of about 160 ms from the input of joysticks
of the cockpit to the output of the actuator of teleoperated
excavator, about 60 ms from capturing of images by the
camera to displaying them on the monitors, and about 50
ms from getting the sensor value to displaying the machine
8VOLUME 4, 2016
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3059710, IEEE Access
Ito et al.: Effects of Machine Instability Feedback During Digging Operation on Safety in Teleoperated Excavators
1) Digging front area
3) Letting out soil
2) Swinging left
4) Swinging right and
returning to initial posture
FIGURE 17: Experimental task (one set).
FIGURE 18: Actual image of teleoperated hydraulic excava-
tor in experiment.
instability on the monitors. Because the purpose of this study
was to verify the effect of the machine instability feedback,
the motion simulator sheet was turned off. There are other
changes in the components, but they were omitted because
they deviate from the purpose of this study.
B. EXPERIMENTS
1) Condition and Protocol
To confirm the effect of presenting machine instability in
an actual teleoperated excavator, a test was conducted. Two
conditions (with and without machine instability feedback)
were tested. In each case, two sets of five repetitions from
task 1 to 4, as shown in Fig. 17, and a total of 10 digging
works were performed by the subjects. The excavator was
placed on horizontal ground and rotated by 90◦, as shown in
Fig. 18. In the experimental area, the behavior in which the
machine was dragged forward during the digging work could
not be reproduced. Therefore, the coefficient of static friction
used for the calculation of the forward instability was set to
1.0.
The machine angle, machine instability, excavated soil
FIGURE 19: Actual image of cockpit in experiment.
= 0 = 50 = 100 ≥100
FIGURE 20: Machine instability meter for teleoperated ex-
cavator experiments.
volume, and task time were measured. The machine angle
was measured using an inertial measurement unit installed
in the excavator body. To eliminate the effect of the initial
machine angle, a high-pass filter with a cutoff frequency
of 0.001 Hz was applied to the measured machine angle.
The excavated soil volume was estimated from the vertical
digging reaction force generated by the soil in the bucket
during the turning operation after the digging work. The unit
weight of soil used in the calculation was 22.0 kN/m3. The
task time started when the arm operation was input, and
ended when the rotation operation was input.
The subjects were six healthy adult males aged 29 to
54 years with hydraulic excavator operating experience. To
eliminate the effect of trial order, three subjects performed
the task with the feedback of the machine instability first,
and the remaining three subjects performed the task without
the feedback first. All subjects performed the task after
performing one set without the feedback. Each subject was
instructed to perform digging with as much soil volume
as possible and as rapidly as possible. The subjects were
informed beforehand that on the machine instability feedback
meter, a 100% level indicates the start of machine motion.
Informed consent based on the Declaration of Helsinki was
obtained from all participants before the experiments. A
photograph of the cockpit during the test is shown in Fig.
19.
VOLUME 4, 2016 9
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3059710, IEEE Access
Ito et al.: Effects of Machine Instability Feedback During Digging Operation on Safety in Teleoperated Excavators
0.0
0.2
0.4
0.6
0.8
1.0
1.2
Without F/B With Machine Instability F/B
Max. Machine Angle (°)
*
FIGURE 21: Differences in mean values of maximum ma-
chine angle for teleoperated excavator experiment. Error bars
indicate standard deviation. Asterisks indicate significant
differences (*p < .05). Hypothesis for student’s t-test is
value of “Without F/B" greater than value of “With Machine
Instability F/B."
0.00
0.05
0.10
0.15
0.20
0.25
Without F/B With Machine Instability F/B
Unstable State Time Ratio
n.s.
FIGURE 22: Differences in mean values of unstable state
time ratio for teleoperated excavator experiment. Error bars
indicate standard deviation. Hypothesis for student’s t-test is
value of “Without F/B" greater than value of “With Machine
Instability F/B."
2) Method for Providing Machine Instability Feedback
The machine instability was displayed as a meter as in the
simulator experiment. The meter used for the display is
shown in Fig. 20. The threshold value was set at 100%, and
the meter was changed to increase the nontransparent part
when the machine instability exceeded the threshold, based
on the comment after the simulator test.
3) Results
The experimental results were evaluated in terms of pro-
ductivity and safety. Each index refers to the mean value
per subject for each condition and soil pattern. “Without
F/B” in the figure means the case without machine instability
feedback, and “With Machine Instability F/B” means the case
with machine instability feedback. A p-value of less than 0.05
was regarded as statistically significant. The excavator was
0.00
0.02
0.04
0.06
0.08
0.10
0.12
Without F/B With Machine Instability F/B
Work Efficiency (m
3
/sec)
n.s.
FIGURE 23: Differences in mean values of work efficiency
for teleoperated excavator experiment. Error bars indicate
standard deviation.
not dragged forward in any of the tasks.
Fig. 21 shows the mean of the maximum machine angle
in one task. Student’s t-test indicates that the maximum
machine angle with machine instability feedback is signifi-
cantly lower than that without machine instability feedback
(t(5) = 2.094, p =.045).
Fig. 22 shows the mean of the unstable state time ratio.
Student’s t-test indicates that there was no significant de-
crease between the two conditions (t(5) = 0.704, p =.256).
In terms of productivity, Fig. 23 shows the mean work
efficiency. Student’s t-test was conducted for the condition
with or without machine instability feedback; there were no
significant differences (t(5) = 0.668, p =.534).
4) Discussion
Since a significant decrease in the maximum machine angle
was confirmed, it can be said that the amount of movement
of the machine was reduced by presenting machine instability
as in the experiment with the simulator. However, there was
no significant decrease in the unstable state time ratio, indi-
cating that the time at which the machine is moving has not
decreased. The presumable reason was that the soil property
was constant during one trial in the simulator, but the soil
property was not uniform at the actual site. For example,
when relatively large rocks or locally hard areas exist in
the soil, the machine instability suddenly increases at those
parts. Therefore, the adjustment of the operation was con-
sidered delayed, and the machine instability exceeded 100%
occasionally. As an example, Fig. 24 shows the changes in
machine angle, machine instability, and bucket tip height
in the test with a certain subject. It can be observed that
the machine instability suddenly changes after approximately
2 s, even though the bucket tip height does not change
substantially. It is possible that the same phenomenon can be
reproduced in the simulator by reproducing nonuniform soil
and by modeling ground deformation.
Besides, as shown in Fig. 24, the time when the machine
instability was exceeding 100% is almost the same as the time
when the machine angle was relatively large. Therefore, it
10 VOLUME 4, 2016
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3059710, IEEE Access
Ito et al.: Effects of Machine Instability Feedback During Digging Operation on Safety in Teleoperated Excavators
FIGURE 24: Time series graph of machine angle, machine
instability, and bucket tip height in one excavation for teleop-
erated excavator experiment.
is considered that the machine instability can be sufficiently
used as an index of the excavator condition.
Regarding productivity, there was no significant difference
in work efficiency, so it is considered that the presentation of
the machine instability did not degrade productivity, as in the
simulator experiment.
From the above discussion, it was clarified that the digging
work could be carried out without lowering the productivity
and without generating the tilt of the machine by feeding
back the machine instability.
V. GENERAL DISCUSSION
In this study, to improve the safety of the excavation by
a teleoperated excavator, machine instability, which is an
index of the excavator condition with the condition when the
excavator body starts to move is assumed to be 100%, was
proposed; as a method for feeding back the machine insta-
bility to the operator, a visual presentation method using a
meter that can intuitively represent the index was introduced.
To confirm the effect of this machine instability feedback,
experiments using a simulator and an actual teleoperated
excavator were conducted. The results of the simulator test
demonstrated the possibility of performing the excavation
safely without lowering the productivity using the machine
instability feedback. In addition, the results of the experiment
using an actual teleoperated excavator confirmed that, with
the machine instability feedback, it was possible to excavate
without lowering productivity and without increasing the tilt
of the excavator.
Regarding safety, when there is no feedback, it is necessary
to estimate the load margin only from the information re-
ceived from images. However, it is considered that this result
was obtained because it is possible to intuitively determine
the load margin due to the machine instability feedback. In
this study, only the horizontal ground test was conducted;
however, a large effect is expected on slopes where it is more
dangerous and difficult for the operator to obtain the tilt angle
of the teleoperated excavator because the load margin can be
intuitively obtained.
Regarding productivity, it was confirmed that machine
instability feedback had no adverse effect. In a previous study
by Lécuyer et al. [33], the effect of additional feedback in-
formation by haptic, visual, and auditory information on the
performance of the operator was investigated. For the task of
manipulating the ball and inserting it into the openings in the
five walls in order, as a result of feeding back the magnitude
of the reaction force acting when the ball collides with the
wall, it is reported that the task time increased because the op-
eration involving feedback was considered more prudent than
the case without the feedback in any condition. However, this
study does not demonstrate a decrease in work efficiency.
It is considered that this is the effect of providing feedback
regarding load margin, that is, providing feedback that the
condition is becoming unfavorable as compared to providing
feedback on whether the unfavorable condition occurs or not.
This machine instability is different from force feedback and
tactile feedback in previous studies and is considered to be
more effective for the safety of teleoperation because the
operator can directly obtain the load margin of the machine.
In addition, there was a delay in the actual teleoperated
excavator experiment, it is necessary to further clarify how
the delay affects the effect of the proposed method.
Further improvement is necessary in the method of ma-
chine instability feedback. In the experiment using the tele-
operated excavator, there was no significant difference in the
unstable state time ratio. This is because the operator cannot
make timely adjustments with respect to the rising rate of
change exhibited by the meter. To solve this problem, it is
necessary to consider the display method, such as setting the
threshold of the meter to less than 100%. In addition, after
both the simulation and the teleoperated excavator experi-
ments, a few subjects commented that it was difficult to dig
while looking at both the meter and the bucket. There is a
possibility that the visual and psychological loads increase
with the machine instability feedback. To solve this problem,
it is necessary to consider the meter position and the use of
non-visual feedback such as sound and vibration.
The limitation of this study is that the verification of
the forward instability has not been conducted in an actual
teleoperated excavator. Although the situation where the ex-
cavator was dragged forward could not be reproduced in the
experimental site, it is considered that additional verification
is necessary because the behavior may occur with the soil
condition and the tilt angle of the ground.
VOLUME 4, 2016 11
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3059710, IEEE Access
Ito et al.: Effects of Machine Instability Feedback During Digging Operation on Safety in Teleoperated Excavators
VI. CONCLUSIONS
In this study, to examine the safety improvement and the
effect on the work performance in excavation by the tele-
operated excavator, the machine instability derived from the
attachment posture and the digging reaction force was pro-
posed, considering the extent of load at which the excavator
body starts to move. In addition, a visual presentation method
using a meter capable of intuitively representing the index
as a method for feeding back the machine instability to the
operator was proposed. To verify the effect of the machine
instability feedback, an excavation operation simulator was
constructed, and an experiment was conducted. As a result,
it was confirmed that the work could be conducted without
generating the tilting and forward movement of the machine
body, and that the ratio of the time when the machine in-
stability exceeded 100% decreased, while the productivity
remained constant. In addition, as a result of verifying the ef-
fect of the subject experiment even in the actual teleoperated
excavator, it was confirmed that the work could be conducted
without causing the machine to tilt more, while the produc-
tivity was maintained. From the results, it was confirmed that
safer work could be conducted without using expensive and
large output actuators in the teleoperated excavator in which
the machine posture is difficult to obtain, using the machine
instability feedback. In the future, it would be of interest to
further re-examine the display method and the position of
the machine instability meter, as well as to consider methods
of providing additional non-visual feedback to the operator.
Besides, the proposed machine instability may be used to
assist the operation of hydraulic excavators.
REFERENCES
[1] H. Sulaiman, M. N. A. Saadun, and A. A. Yusof, “Modern manned,
unmanned and teleoperated excavator system,” J. Mech. Eng. Tech, vol.
7, no. 1, pp. 57–68, 2015.
[2] M. Moteki, K. Fujino, T. Ohtsuki, and T. Hashimoto, “Research on visual
point of operator in remote control of construction machinery,” in Proc.
Int. Symp. Autom. Robot. Const., Seoul, Korea, 2011, pp. 532–537.
[3] M. Moteki, K. Fujino, and A. Nishiyama, “Research on operator’s mastery
of unmanned construction,” in Proc. Int. Symp. Autom. Robot. Const. Min.,
Montréal, QC, Canada, 2013, pp. 540–547.
[4] Y. Sakaida, D. Chugo, H. Yamamoto, and H. Asama, “The analysis of
excavator operation by skillful operator - Extraction of common skills -,”
in SICE Annu. Conf., Tokyo, Japan, 2008, pp. 538–542.
[5] D. Zhao, Y. Xia, H. Yamada, and T. Muto, “Control method for realistic
motion in a construction tele-robotic system with a 3-DOF parallel mecha-
nism,” J. Robot. Mechatron., vol. 15, no. 4, pp. 361–368, Aug. 2003, DOI:
10.20965/jrm.2003.p0361.
[6] M. Ito, Y. Funahara, S. Saiki, Y. Yamazaki, and Y. Kurita, “Development
of a cross-platform cockpit for simulated and tele-operated excavators,”
J. Robot. Mechatron., vol. 31, no. 2, pp. 231–239, Apr. 2019, DOI:
10.20965/jrm.2019.p0231.
[7] N. R. Parker, S. E. Salcudean, and P. D. Lawrence, “Application of force
feedback to heavy duty hydraulic machines,” in Proc. IEEE Int. Conf.
Robot. Autom., Atlanta, GA, USA, 1993, pp. 375–381.
[8] P. D. Lawrence, S. E. Salcudean, N. Sepehri, D. Chan, S. Bachmann, N.
Parker, M. Zhu, and R. Frenette, “Coordinated and force-feedback control
of hydraulic excavators,” in Exp. Robot. IV., Stanford, CA, USA, 1995, pp.
181–194.
[9] K. Ahn, “Development of force reflecting joystick for hydraulic excava-
tor,” JSME Int. J. Ser. C Mech. Sys. Mach. Elem. Manuf., vol. 47, no. 3, pp.
858–863, Mar. 2005, DOI: 10.1299/jsmec.47.858.
[10] X. Li, “Study on master-slave control method using load force and
impedance identifiers for tele-operated hydraulic construction robot,”
Appl. Mech. Mater., vol. 29–32, pp. 2170–2175, Aug. 2010, DOI:
10.4028/www.scientific.net/AMM.29-32.2170.
[11] D. Q. Truong, B. N. M. Truong N. T. Trung, S. A. Nahian, and K. K. Ahn,
“Force reflecting joystick control for applications to bilateral teleoperation
in construction machinery,” Int. J. Precis. Eng. Manuf., vol. 18, no. 3, pp.
301–315, Mar. 2017, DOI: 10.1007/s12541-017-0038-z.
[12] M. Gong, B. Yang, and H. Yamada, “Performance experiment of a
hydraulic force feedback teleoperation robot based on position and rate
control,” in Int. Conf. Intel. Hum. Mach. Sys. Cybern., Nanjing, China,
2010, pp. 87–90.
[13] J. Hou and D. Zhao, “A new force feedback algorithm for hydraulic
teleoperation robot,” in Int. Conf. Comput. Appl. Sys. Model., Taiyuan,
China, 2010, pp. V15–15–V15–18.
[14] L. Huang, T. Kawamura, and H. Yamada, “Construction robot operation
system with object’s hardness recognition using force feedback and virtual
reality,” J. Robot. Mechatron., vol. 24, no. 6, pp. 958–966, Dec. 2012, DOI:
10.20965/jrm.2012.p0958.
[15] K. Zareinia and N. Sepehri, “A hybrid haptic sensation for teleoperation
of hydraulic manipulators,” J. Dyn. Sys. Meas. Control., vol. 137, no. 9,
091001, Sep. 2015, DOI: 10.1115/1.4030337.
[16] S. Lampinen, J. Koivumäki, and J. Mattila, “Full-dynamics-based bilateral
teleoperation of hydraulic robotic manipulators,” in IEEE Int. Conf. Autom.
Sci. Eng., Munich, Germany, 2018, pp. 1343–1350.
[17] L. Carvalho, P. Rezeck, M. V. Lima, L. Pinto, G. Freitas, E. R. Nascimento,
D. G. Macharet, L. Chaimowicz, G. Pessin, and M. F. Campos, “On the
evaluation of force feedback augmented teleoperation of excavator-like
mobile manipulators,” in IEEE Int. Conf. Autom. Sci. Eng., Hong Kong,
Hong Kong, 2020, pp. 1401–1407.
[18] K. Hayashi and T. Tamura, “Teleoperation performance using excava-
tor with tactile feedback,” in IEEE Int. Conf. Mechatron. and Autom.,
Changchun, China, 2009, pp. 2759–2764.
[19] D. Kim, K. W. Oh, C. S. Lee, and D. Hong, “Novel design of haptic devices
for bilateral teleoperated excavators using the wave-variable method,” Int.
J. Precis. Eng. Manuf., vol. 14, no. 2, pp. 223–230, Jan. 2013, DOI:
10.1007/s12541-013-0031-0.
[20] J. M. Jacinto-Villegas, M. Satler, A. Filippeschi, M. Ragaglia, A. Argiolas,
M. Niccolini, and C. A. Avizzano, “A Novel Wearable Haptic Controller
for Teleoperating Robotic Platforms,” IEEE Robot. Autom. Lett., vol. 2,
no. 4, pp. 2072–2079, Oct. 2017, DOI: 10.1109/LRA.2017.2720850.
[21] H. Nagano, H. Takenouchi, N. Cao, M. Konyo, and S. Tadokoro, “Tactile
feedback system of high-frequency vibration signals for supporting deli-
cate teleoperation of construction robots,” Adv. Robot., vol. 34, no. 11, pp.
730–743, Jun. 2020, DOI: 10.1080/01691864.2020.1769725.
[22] T. Tanimoto, K. Shinohara, and H. Yoshinada, “Research on effective
teleoperation of construction machinery fusing manual and automatic
operation,” Robomech J., vol. 4, no. 14, 2017, DOI: 10.1186/s40648-017-
0083-5.
[23] B. H. Yu, K. Y. Park, K. D. Lee, and C. S. Han, “Static compensation ZMP
algorithm preventing tips-over of a tele-operation excavator,” in Proc. Int.
Symp. Autom. Robot. Const., Eindhoven, Netherlands, 2012, pp. 652–659.
[24] K. Shigematsu, T. Tsubouchi, and S. Sarata, “Tip-over prevention control
of a teleoperated excavator based on ZMP prediction,” in IEEE Int. Conf.
Autom. Sci. Eng., Fort Worth, TX, USA, 2016, pp. 1380–1386.
[25] M. Ito, C. Raima, S. Saiki, Y. Yamazaki, and Y. Kurita, “A study on
machine instability feedback during digging operation in teleoperated
excavators,” in Int. Conf. Hum. Sys. Interact., Tokyo, Japan, 2020, pp. 14–
19.
[26] D. Dopico, S. Mendizabal, and M. González, “A soil model for a hydraulic
simulator excavator based on real-time multibody dynamics,” in Proc.
Asian Conf. Multibody Dyn., Kyoto, Japan, 2010, pp. 325–333.
[27] T. Ni, H. Zhang, C. Yu, D. Zhao, and S. Liu, “Design of
highly realistic virtual environment for excavator simulator,” Com-
put. Electr. Eng., vol. 39, no. 7, pp. 2112–2123, Oct. 2013, DOI:
10.1016/j.compeleceng.2013.06.010.
[28] K. Koiwai, R. Miyazaki, T. Yamamoto, K. Ueda, K. Yamashita, and Y.
Yamazaki, “Responsiveness evaluation index for an excavator operation
based on control engineering approach,” IEEJ Trans. Elec. Info. Sys., vol.
138, no. 5, pp. 506–511, May. 2018, DOI: 10.1541/ieejeiss.138.506.
[29] H. Osumi, T. Uehara, N. Okada, T. Fujiwara, and S. Sarata, “Effi-
cient scooping of rocks by autonomous controlled wheel loader,” J.
Robot. Mechatron., vol. 24, no. 6, pp. 924–932, Dec. 2012, DOI:
10.20965/jrm.2012.p0924.
12 VOLUME 4, 2016
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3059710, IEEE Access
Ito et al.: Effects of Machine Instability Feedback During Digging Operation on Safety in Teleoperated Excavators
[30] Y. Meng, H. Fang, G. Liang, Q. Gu, and L. Liu, “Bucket trajectory
optimization under the automatic scooping of LHD,” Energies, vol. 12,
no. 20, 3919, Oct. 2019, DOI: 10.3390/en12203919.
[31] W. J. Macquorn Rankine, “On the stability of loose earth,” Philos. Trans.
R. Soc. Lond., vol. 147, pp. 9–27, 1857.
[32] B. G. Look, “Soil properties and state of the soil,” in Handbook of
Geotechnical Investigation and Design Tables, 1st ed., London, UK: Taylor
& Francis, 2007, pp. 77–90.
[33] A. Lécuyer, C. Mégard, J. M. Burkhardt, T. Lim, S. Coquillart, P. Coiffet,
and L. Graux, “The effect of haptic, visual and auditory feedback on an
insertion task on a 2-screen workbench,” in Immers. Proj. Tech. Symp.,
Florida, USA, 2002.
MASARU ITO received the M.E. degree in me-
chanical engineering from Kumamoto University,
Kumamoto, Japan, in 2010.
In 2010, he joined Kobelco Construction Ma-
chinery Co., Ltd., Hiroshima, Japan. Since 2017,
he has been an Assistant Professor at the Next
Generation Human Interface Collaborative Re-
search Laboratory, Hiroshima University, Hi-
roshima. His research interests include human in-
terface, control engineering, and mechatronics.
Mr. Ito is a member of the Japan Society of Mechanical Engineers and the
Society of Instrument and Control Engineering.
CHIAKI RAIMA received the B.L.A., in 2014, the
M.A.S., in 2016, and the Ph.D. degrees, in 2019
from Hiroshima University, Hiroshima, Japan.
She was a Researcher from 2019 to 2020, and
has been a Specially Appointed Assistant Profes-
sor since 2020 at Kobelco Construction Machinery
Dream-Driven Co-Creation Research Center, Hi-
roshima University. Her research interests include
psychology about motor-skilled humans.
Ms. Raima is a member of the Japan Society of
Mechanical Engineers.
SEIJI SAIKI joined Kobelco Construction Ma-
chinery Co., Ltd., Hiroshima, Japan, in 2009. He
is currently a Group Manager of DX Consulting
Group, Business Development Department, Ko-
belco Construction Machinery Co., Ltd., Tokyo,
Japan.
YOICHIRO YAMAZAKI received the M.E. degree
in mechanical engineering from Ehime University,
Ehime, Japan, in 1992.
From 1992 to 1999, he was working at Kobe
Steel, Ltd. In 1999, he joined Kobelco Construc-
tion Machinery Co., Ltd., Hiroshima, Japan. Since
2019, he has been a Visiting Professor at Hi-
roshima University, Hiroshima, Japan. He is cur-
rently a General Manager of Business Develop-
ment Department, Kobelco Construction Machin-
ery Co., Ltd., Tokyo, Japan.
Mr. Yamazaki is a member of the Japan Society of Mechanical Engineers.
YUICHI KURITA (Member, IEEE) received the
B.E. degree from Osaka University, Osaka, Japan,
in 2000, and M.E. and Ph.D. degrees in informa-
tion science from Nara Institute of Science and
Technology (NAIST), Nara, Japan, in 2002 and
2004, respectively.
From 2005 to 2007, he was a Research As-
sociate with the Graduate School of Engineering
at Hiroshima University, Hiroshima, Japan. From
2007 to 2011, he was an Assistant Professor with
the Graduate School of Information Science at NAIST. From 2010 to 2011,
he was a Visiting Scholar with the School of Mechanical Engineering, Geor-
gia Institute of Technology, Georgia, USA. In 2011, he joined the Graduate
School of Advanced Science and Engineering, Hiroshima University, as
an Associate Professor, where he has been a Professor, since 2018. His
research interests include physical human-robot interaction (pHRI), human
augmentation, haptics, and medical engineering.
Mr. Kurita is a member of the Japan Society of Mechanical Engineers,
the Robotics Society of Japan, the Virtual Reality Society of Japan, and the
Society of Instrument and Control Engineering.
VOLUME 4, 2016 13
