Figure 4 - uploaded by Tobias Holstein
Content may be subject to copyright.
Source publication
High-end industrial vehicle simulators are generally expensive and aim at providing a high level of realism. The access to such simulators is often a limited resource to researchers and developers who find themselves using a PC-based simulator instead. We challenge this approach by introducing a low-cost mixed reality simulator for industrial vehic...
Context in source publication
Similar publications
Discrete-event simulation (DES) technology has been widely used in the construction context, particularly in the academic community for research purposes. However, it has been neglected by industry for various reasons, including the usual way in which its results are presented, which is abstract and confusing. In this regard, animation is viewed as...
The scientific articles have been recently enormously devoted to the issue of augmented, virtual,
and mixed reality and are already being fully established in our normal life. They infiltrated into
gaming, industry, medical, manufacturing, etc. However, the area of education (especially in
Slovakia) is still lagging behind and is influenced by the...
This paper presents a study aimed at creating an aerial mixed reality environment by using a first-person-view drone. We develop a drone with a stereoscopic camera based on human binocular vision. Then, we present a novel mixed reality environment with bidirectional interaction between the real and virtual worlds. Our approach is effective to perce...
The advancements in mixed reality (MR) technology in recent years have provided us with excellent prospects for creating novel approaches to supplement conventional physiotherapy to maintain a sufficient quantity and quality of rehabilitation. The use of MR systems to facilitate patients’ participation in intensive, repetitive, and task-oriented pr...
Serious games have a great potential to aid stroke patients during their physical rehabilitation process. By incorporating different technologies into a serious game and taking several serious game design guidelines into account, the game can take place into the mixed reality continuum and provide a Virtual Reality or Augmented Reality experience t...
Citations
... The dynamics of vehicles equipped with on-board sensors are simulated in a road environment. We can also mention the study of Kade & al [27] on a low cost mixed reality simulator for Industrial Vehicles. Human panel test results indicate that participants have a more realistic and natural view with the immersive experience of the Mixed Reality Simulator, unlike the Classic Simulator on PC. ...
One of the major challenges faced by Industry 4.0 is the use of Automated Guided Vehicles (AGVs) and, more
broadly, autonomous mobile robots. While autonomy in road transportation vehicles can already be well characterized, it is a
different story for autonomous vehicles used in industries, such as Autonomous Industrial Vehicles (AIVs). The implementation
and deployment of AIV fleets in industrial sectors encounter various issues, including vehicle localization, employee acceptance,
traffic flow, and the ability of vehicles to adapt to fluctuating and dynamic environments. The challenge that autonomous
vehicles represent for the future of the digital industry is so significant that it makes sense from our vision to go through a step of
joint physical and digital simulation of these vehicles and their environments. This step would assist industrials in their respective
development and fine-tuning activities. The objective of this article is to demonstrate that all the elements available to us at
present (research conducted in our laboratory, technological building blocks, operating scenarios already carried out, state of the
art) logically allow us to project ourselves towards a Co-Simulation platform based on a bijective model between physical and
virtual environments. Furthermore, this projection is enriched by the idea of a digital component of the platform, capable of
taking into account in an agnostic way, the different possible forms of the physical part of this platform. Thus, simulation enables
the consideration of constraints and requirements formulated by manufacturers and future users of autonomous vehicles. Our
approach is progressive, as presented in this article, and it is based on experiments with Co-Simulation platforms that combine
physical and virtual approaches. We provide a detailed description of our AIVs and their traffic environments. The static
architecture of the platform is described using class diagrams. The dynamic behavior of the platform is described, thanks to
sequence diagrams, state diagrams, and algorithmic flowcharts. We also propose an approach to estimate the position of AIVs
based on the combination of matrix-based tagging with a section change management technique. As a result of the presented
approach, all of these diagrams make it possible to document different operating scenarios of the Co-Simulation platform.
Beyond these results, and to complement them before concluding, we describe several application cases related to both
algorithmic position estimation metrology and electric battery characterization. This serves to illustrate the potential value of our
Co-Simulation platform model.
... Before full-scale testing of traffic scenarios involving autonomous vehicles in industrial or more complex traffic situations can begin , it is essential to consider the simulation involved. Perhaps the greatest advantage to be gained by running a simulation is that actionable results can be obtained without applying a scale factor [15,16,17,18,19]. As might be expected, there are numerous methods in use for such testing [20]. ...
The use of automated guided vehicles (AGVs) and other autonomous mobile robots is a challenge facing Industry 4.0. While the autonomy of autonomous vehicles has been well characterized in the field of road and road transport, this is not the case for the autonomous vehicles used in industry (autonomous industrial vehicles or AIVs). The establishment and deployment of AIV fleets in industrial companies remain problematic in several respects, including their acceptability by employees, the location of vehicles, the fluidity of traffic, and the perception by vehicles of changing and, therefore, dynamic environments. Thus, simulation serves to account for the constraints and requirements formulated by the manufacturers and future users of autonomous vehicles. In this paper, we present the development of a co-simulation platforms, and a method for estimating the positions of the vehicles simulated in this platform.
... Indeed, there are many urban and industrial challenges concerning autonomous vehicles, and simulation makes it possible to address them well: the autonomous vehicles in city traffic [88], autonomous vehicles storage and retrieval systems [89], collaborative autonomous vehicles [90], low-cost mixed reality for industrial vehicle environment [91], or end-to-end scalable autonomous vehicle testing when rare events appear [92]. As might be expected, there are numerous methods in use for such testing [93]. ...
Industry 4.0 leads to a strong digitalization of industrial processes, but also a significant increase in communication and cooperation between the machines that make it up. This is the case with autonomous industrial vehicles (AIVs) and other cooperative mobile robots which are multiplying in factories, often in the form of fleets of vehicles, and whose intelligence and autonomy are increasing. While the autonomy of autonomous vehicles has been well characterized in the field of road and road transport, this is not the case for the autonomous vehicles used in industry. The establishment and deployment of AIV fleets raises several challenges, all of which depend on the actual level of autonomy of the AIVs: acceptance by employees, vehicle location, traffic fluidity, collision detection, or vehicle perception of changing environments. Thus, simulation serves to account for the constraints and requirements formulated by the manufacturers and future users of AIVs. In this paper, after having proposed a broad state of the art on the problems to be solved in order to simulate AIVs before proceeding to experiments in real conditions, we present a method to estimate positions of AIVs moving in a closed industrial environment, the extension of a collision detection algorithm to deal with the obstacle avoidance issue, and the development of an agent-based simulation platform for simulating these two methods and algorithms. The resulting/final/subsequent simulation will allow us to experiment in real conditions.
... To facilitate the A/B test, we used a mixed reality environment that simulated an excavator operation. The mixed reality simulation was originally developed by Kade et al. [66], and then updated with three information placements: (1) on the physical head-down display, (2) on the physical windshield, (3) and virtually projected on the ground (see Figure 4.2). We selected these three information placements, as the variation of these setups is often used in research related to head-up displays and mixed reality interfaces in the automotive domain [110,111]. ...
Operating heavy machinery, such as mobile cranes and excavators, is a complex task. While driving the machine, operators are also performing industrial tasks, e.g. lifting or digging, monitoring the machine’s status, and observing the surroundings. Modern heavy machinery is increasingly equipped with information systems that present supportive information to operators, so that they could perform their work safely and productively. Supportive information in heavy machinery is generally presented visually using head-down displays, which are placed in lower positions inside the cabin in order to avoid obstructing operators’ view. However, this placement makes visual information presented using head-down displays tend to be overlooked by operators, as the information is presented outside their field of view.
This dissertation investigates the possible use of transparent mediums for presenting visual information on the windshield of mobile cranes and excavators. By presenting information on the windshield, operators are expected to acquire visual information without diverting their attention away from the operational area. The design process includes (1) observing heavy machinery operators in natural settings through available videos on the Internet, (2) conducting an empirical study on the impact of different information placements, (3) reviewing the state of the art of display technologies that could be used to visualize information around the windshield of heavy machinery, (4) reviewing relevant safety guidelines to determine what kinds of critical information that operators should know, (5) conducting design workshops to generate visualization designs that represent critical information in operations of mobile cranes and excavators, (6) involving professional operators to evaluate and improve the proposed visualization designs, and (7) developing a functioning transparent display prototype that visualizes one kind of critical information that professional operators considered as the most important one.
The main finding from the observation using online videos suggested that heavy machinery operators spent considerable amount of time looking through the front windshield, and thus the front windshield could be used as a potential space for presenting visual information. The main finding of the empirical study also indicated that presenting information closer to the line of sight produced higher information acquisition and lower workload, compared to when information was presented farther from the line of sight. Based on the evaluation with professional operators, there seemed to be a good match between the proposed visualization designs and the operators' way of thinking, since the operators were able to understand and use the proposed visualization designs with little explanations. On the basis of the three most important findings above, there is a strong indication that placing the developed transparent display on the front windscreen of heavy machinery would make it easier for operators to perceive and process the presented information.
... Realization of a mixed reality simulator with free head movement [24]. ...
As systems, for example vehicle systems, get increasingly autonomous and information intense, the information exchanged with the user, i.e., the operator, are increasingly becoming a designed interaction. This work investigates how interaction technologies, interaction design principles, and machine information systems can be used to provide user experiences and efficient interaction between the operator and industrial mobile machines; for example, agricultural machines and construction machines. In this pursuit the research aims to explore the use of mixed reality interaction and visual presentation using see-through interfaces and symbolic metaphors, to enhance the interaction for operators working with these types of machines.
Designing and testing new information and safety features for industrial vehicles do not need to involve the realization of high-fidelity and expensive simulators. We propose a low-cost mixed reality environment which allows for rapid development and rearrangement of a virtual and physical setup of a simulator for industrial vehicles.Our mixed reality simulator allows for safe testing of controls, information, and safety features to support drivers of industrial vehicles. In this paper, we test the implications of showing extra digital information to excavator drivers through a virtual environment, an external head-up display as well as a head-down display. Through user tests we have seen first indications that projected information through our mixed reality system and content on a head-up display is perceived as more helpful and intuitive than using head-down displays, when controlling our industrial vehicle simulator. Moreover, we have seen that the fear of overseeing an obstacle or other important information is lower when using a head-up display, in comparison to other tested visualization options.KeywordsRapid prototyping environmentHead-up displayMixed realityHead-worn projection displayDesign evaluation
