No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Physical and Virtual Robotic Cells in Industry 4.0 Towards Industry 5.0: An XR-Based Conceptual Framework
Abstract
The Digital Twin (DT) in the manufacturing domain is already the everyday tool for visualizing the various industrial systems, equipment, and produced products. When designing a new manufacturing unit or enlarging an existing factory, it is important to do so without affecting the manufacturing process flow itself. There are opportunities through simulation and digital manufacturing to plan and optimize this design process. Within usage of the actual physical machinery data gathered from the Industrial Internet of Things (IIoT) sensors and feeding to the DT, optimizing the layout can be done more precisely and effectively. However, there is no way to test the potential equipment simultaneously with the physical one in real-time. This paper aims to propose a Mixed Reality (MR) based system framework and toolkit, which will enable physical industrial robots to interact with virtual equipment and other virtual robots. This way, via Virtual Reality (VR), it will be possible to design a system layout. Furthermore, via the Augmented Reality (AR) view, it will be possible to simulate the interaction between multiple robots by enhancing the possibilities of the physical environment and using the new precise scale real-time design method.
... These applications include advanced training and simulation modules where XR facilitates immersive learning experiences by merging physical and virtual realms. The integration of Physical and virtual robotic cells with XR in the transition from Industry 4.0 to Industry 5.0, along with DT, heralds transformative industrial applications [89]. Real-time monitoring and control of robotic cells are enriched through XR interfaces, enabling operators to engage with DT for dynamic process visualization. ...
EXtended Reality (XR) alongside the Digital Twin (DT) in Industrial Internet of Things (IIoT) emerges as a promising next-generation technology. Its diverse applications hod the potential to revolutionize multiple facets of Industry 4.0 and serve as a cornerstone for the rise of Industry 5.0. However, current systems are still not effective in providing a high-quality experience for users due to various factors, one of which is their limited resources for processing and transmitting complex data and big data. To overcome these challenges, this paper presents an in-depth analysis of performance optimization techniques for resource-constrained Augmented Reality (AR) and/or Virtual Reality (VR) environments operating with DT, with a specific focus on Quality of Service (QoS), Quality of Experience (QoE), Edge-Cloud architectures and future research directions. Furthermore, this study delves into the intricate complex trade-off relationships involving optimization factors, including system quality, information quality, and QoE. In addition, it also explores potential solutions based on powerful emerging technological tools, including data compression, blockchain, cloud computing, quantum computing, Artificial Intelligence (AI) / Machine Learning (ML), and cybersecurity in the Cyber-Physical Systems (CPS). The insights provided in this comprehensive survey can inspire and guide researchers and industrial practitioners in optimizing performance for XR with DT applications in resource-constrained Smart Manufacturing System (SMS).
... Metaverse can be facilitated through virtual or augmented reality headsets, which enable users to participate in this virtual world. However, it is essential to note that the Metaverse is combination of Vertual Reality (VR), Augument Reality (AR) and Extended Reality (XR) [10], and recent advancements have made it accessible on desktop platforms [1]. The immersive nature of the Metaverse also creates opportunities for food brands to advertise their products, engage with consumers, and foster stronger connections with their target audience. ...
Metaverse is a new virtual reality environment rapidly gaining popularity, with various applications being developed. One key technology critical to the success of the Metaverse is semantic communication, which enables efficient and effective communication between humans and intelligent agents. In this paper, we discuss the challenges and opportunities associated with semantic communication (SemCom) in the Metaverse and potential future trends. We identify the critical challenges in semantic communication, such as context awareness, scalability, and interoperability, and discuss potential solutions to address these challenges. We also explore the various opportunities that SemComcan bring to the Metaverse, such as improved user experiences, interactions with virtual assistants, and enhanced data analysis capabilities. Finally, we examine the emerging trends in SemComfor the metaverse, such as the use of natural language processing and machine learning techniques, and we highlight some key research directions that will be important in the future. Overall, this paper provides a comprehensive overview of SemComin the Metaverse and identifies critical areas where further research and development are needed to realize this technology's full potential.
Extended reality (XR), which combines the real and virtual worlds, is greatly enhancing interaction possibilities between robots and humans, leading to a paradigm shift where the two entities can intuitively cooperate to perform shared-target tasks. Many XR devices are essentially performing the same spatial perception tasks as mobile robots (e.g., visual simultaneous localization and mapping), and thus XR provides an opportunity for robots and the humans using these devices to colocalize through a common understanding of their space, which also enables easier human–robot interactions.
Robotic manipulators have become part of manufacturing systems in recent decades. However, in the realm of Industry 4.0, a new type of manufacturing cell has been introduced—the so-called collaborative manufacturing cell. In such collaborative environments, communication between a human operator and robotic manipulators must be flawless, so that smooth collaboration, i.e., human safety, is ensured constantly. Therefore, engineers have focused on the development of suitable human–robot interfaces (HRI) in order to tackle this issue. This research work proposes a closed-loop framework for the human–robot interface based on the utilization of digital technologies, such as Mixed Reality (MR). Concretely, the framework can be realized as a methodology for the remote and safe manipulation of the robotic arm in near real-time, while, simultaneously, safety zones are displayed in the field of view of the shop-floor technician. The method is based on the creation of a Digital Twin of the robotic arm and the setup of a suitable communication framework for continuous and seamless communication between the user interface, the physical robot, and the Digital Twin. The development of the method is based on the utilization of a ROS (Robot Operating System) for the modelling of the Digital Twin, a Cloud database for data handling, and Mixed Reality (MR) for the Human–Machine Interface (HMI). The developed MR application is tested in a laboratory-based machine shop, incorporating collaborative cells.
Emerging technologies such as digital twins, blockchain, Internet of Things (IoT), and Artificial Intelligence (AI) play a vital role in driving the industrial revolution in all domains, including the healthcare sector. As a result of COVID-19 pandemic outbreak, there is a significant need for medical cyber-physical systems to adopt these emerging technologies to combat COVID-19 paramedic crisis. Also, acquiring secure real-time data exchange and analysis across multiple participants is essential to support the efforts against COVID-19. Therefore, we have introduced a blockchain-based collaborative digital twins framework for decentralized epidemic alerting to combat COVID-19 and any future pandemics. The framework has been proposed to bring together the existing advanced technologies (i.e., blockchain, digital twins, and AI) and then provide a solution to decentralize epidemic alerting to combat COVID-19 outbreaks. Also, we have described how the conceptual framework can be applied in the decentralized COVID-19 pandemic alerting use case.
Featured Application
A proof-of-concept version of a mixed reality application for controlling and monitoring a digital twin based overhead crane, featured by interactivity, flexibility, and immersiveness, together with a protocol defined to standardize quantitative evaluation on control accuracy of industrial mixed reality applications.
Abstract
Digital twin technology empowers the digital transformation of the industrial world with an increasing amount of data, which meanwhile creates a challenging context for designing a human–machine interface (HMI) for operating machines. This work aims at creating an HMI for digital twin based services. With an industrial crane platform as a case study, we presented a mixed reality (MR) application running on a Microsoft HoloLens 1 device. The application, consisting of visualization, interaction, communication, and registration modules, allowed crane operators to both monitor the crane status and control its movement through interactive holograms and bi-directional data communication, with enhanced mobility thanks to spatial registration and tracking of the MR environment. The prototype was quantitatively evaluated regarding the control accuracy in 20 measurements following a step-by-step protocol that we defined to standardize the measurement procedure. The results suggested that the differences between the target and actual positions were within the 10 cm range in three dimensions, which were considered sufficiently small regarding the typical crane operation use case of logistics purposes and could be improved with the adoption of robust registration and tracking techniques in our future work.
Digital twins (DTs) is a promising technology in the revolution of the industry and essential for Industry 4.0. DTs play a vital role in improving distributed manufacturing, providing up-to-date
operational data representation of physical assets, supporting decision-making, and avoiding the
potential risks in distributed manufacturing systems. Furthermore, DTs need to collaborate within
distributed manufacturing systems to predict the risks and reach consensus-based decision-making.
However, DTs collaboration suffers from single failure due to attack and connection in a centralized
manner, data interoperability, authentication, and scalability. To overcome the above challenges, we
have discussed the major high-level requirements for the DTs collaboration. Then, we have proposed
a conceptual framework to fulfill the DTs collaboration requirements by using the combination of
blockchain, predictive analysis techniques, and DTs technologies. The proposed framework aims to
empower more intelligence DTs based on blockchain technology. In particular, we propose a concrete
ledger-based collaborative DTs framework that focuses on real-time operational data analytics and
distributed consensus algorithms. Furthermore, we describe how the conceptual framework can
be applied using smart transportation system use cases, i.e., smart logistics and railway predictive
maintenance. Finally, we highlighted the future direction to guide interested researchers in this
interesting area.
In recent years, the use of digital twins (DT) to improve maintenance procedures has increased in various industrial sectors (e.g., manufacturing, energy industry, aerospace) but is more limited in the construction industry. However, the operation and maintenance (O&M) phase of a building’s life cycle is the most expensive. Smart buildings already use BIM (Building Information Modeling) for facility management, but they lack the predictive capabilities of DT. On the other hand, the use of extended reality (XR) technologies to improve maintenance operations has been a major topic of academic research in recent years, both through data display and remote collaboration. In this context, this paper focuses on reviewing projects using a combination of these technologies to improve maintenance operations in smart buildings. This review uses a combination of at least three of the terms “Digital Twin”, “Maintenance”, “BIM” and “Extended Reality”. Results show how a BIM can be used to create a DT and how this DT use combined with XR technologies can improve maintenance operations in a smart building. This paper also highlights the challenges for the correct implementation of a BIM-based DT combined with XR devices. An example of use is also proposed using a diagram of the possible interactions between the user, the DT and the application framework during maintenance operations.
Digital Twin (DT) refers to the virtual copy or model of any physical entity (physical twin) both of which are interconnected via exchange of data in real time. Conceptually, a DT mimics the state of its physical twin in real time and vice versa. Application of DT includes real-time monitoring, designing/planning, optimization, maintenance, remote access, etc. Its implementation is expected to grow exponentially in the coming decades. The advent of Industry 4.0 has brought complex industrial systems that are more autonomous, smart, and highly interconnected. These systems generate considerable amounts of data useful for several applications such as improving performance, predictive maintenance, training, etc. A sudden influx in the number of publications related to ‘Digital Twin’ has led to confusion between different terminologies related to the digitalization of industries. Another problem that has arisen due to the growing popularity of DT is a lack of consensus on the description of DT as well as so many different types of DT, which adds to the confusion. This paper intends to consolidate the different types of DT and different definitions of DT throughout the literature for easy identification of DT from the rest of the complimentary terms such as ‘product avatar’, ‘digital thread’, ‘digital model’, and ‘digital shadow’. The paper looks at the concept of DT since its inception to its predicted future to realize the value it can bring to certain sectors. Understanding the characteristics and types of DT while weighing its pros and cons is essential for any researcher, business, or sector before investing in the technology.
Purpose-The purpose of this research is to develop a generic framework of a digital twin (DT)-based automated construction progress monitoring through reality capture to extended reality (RC-to-XR).
Design/methodology/approach-IDEF0 data modeling method has been designed to establish an integration of reality capturing technologies by using BIM, DTs and XR for automated construction progress monitoring. Structural equation modeling (SEM) method has been used to test the proposed hypotheses and develop the skill model to examine the reliability, validity and contribution of the framework to understand the DRX model's effectiveness if implemented in real practice.
Findings-The research findings validate the positive impact and importance of utilizing technology integration in a logical framework such as DRX, which provides trustable, real-time, transparent and digital construction progress monitoring.
Practical implications-DRX system captures accurate, real-time and comprehensive data at construction stage, analyses data and information precisely and quickly, visualizes information and reports in a real scale environment, facilitates information flows and communication, learns from itself, historical data and accessible online data to predict future actions, provides semantic and digitalize construction information with analytical capabilities and optimizes decision-making process.
Originality/value-The research presents a framework of an automated construction progress monitoring system that integrates BIM, various reality capturing technologies, DT and XR technologies (VR, AR and MR), arraying the steps on how these technologies work collaboratively to create, capture, generate, analyze, manage and visualize construction progress data, information and reports.
Digital twin (DT) plays a pivotal role in the vision of Industry 4.0. The idea is that the real product and its virtual counterpart are twins that travel a parallel journey from design and development to production and service life. The intelligence that comes from DTs’ operational data supports the interactions between the DTs to pave the way for the cyber-physical integration of smart manufacturing. This paper presents a conceptual framework for digital twins collaboration to provide an auto-detection of erratic operational data by utilizing operational data intelligence in the manufacturing systems. The proposed framework provide an interaction mechanism to understand the DT status, interact with other DTs, learn from each other DTs, and share common semantic knowledge. In addition, it can detect the anomalies and understand the overall picture and conditions of the operational environments. Furthermore, the proposed framework is described in the workflow model, which breaks down into four phases: information extraction, change detection, synchronization, and notification. A use case of Energy 4.0 fault diagnosis for wind turbines is described to present the use of the proposed framework and DTs collaboration to identify and diagnose the potential failure, e.g., malfunctioning nodes within the energy industry.
During the last decades, there has been wide interest towards creating more agile and reconfigurable automation systems. This includes the research interest towards human–robot collaboration (HRC) solutions, where semi or fully automated process could be combined with human dexterity and flexibility without complexity and inflexibility. However, the communication between the human and the robot is not intuitive, fast or flexible yet. The emerging technologies such as Augmented Reality (AR), Virtual Reality (VR) and Mixed Reality (XR) are seen as good solution candidates for increasing the communication between human and the machine during the design, commission and operation phases. The design of the HRC system includes layout design evaluation, task scenario analysis, robot programming, cycle time calculations, and safety analysis. Virtual Reality and Augmented Reality technologies provide immersive experiences to visualize and analyze these procedures. This paper aims to review the current status of virtual and augmented reality solutions in HRC with meta-analysis and highlight missing elements. Finally, future research directions and requirements are presented.
Recently, the use of extended reality (XR) systems has been on the rise, to tackle various domains such as training, education, safety, etc. With the recent advances in augmented reality (AR), virtual reality (VR) and mixed reality (MR) technologies and ease of availability of high-end, commercially available hardware, the manufacturing industry has seen a rise in the use of advanced XR technologies to train its workforce. While several research publications exist on applications of XR in manufacturing training, a comprehensive review of recent works and applications is lacking to present a clear progress in using such advance technologies. To this end, we present a review of the current state-of-the-art of use of XR technologies in training personnel in the field of manufacturing. First, we put forth the need of XR in manufacturing. We then present several key application domains where XR is being currently applied, notably in maintenance training and in performing assembly task. We also reviewed the applications of XR in other vocational domains and how they can be leveraged in the manufacturing industry. We finally present some current barriers to XR adoption in manufacturing training and highlight the current limitations that should be considered when looking to develop and apply practical applications of XR.
In this article, we have investigated a VR simulator of a forestry crane used for loading logs onto a truck. We have mainly studied the Quality of Experience (QoE) aspects that may be relevant for task completion, and whether there are any discomfort related symptoms experienced during the task execution. QoE experiments were designed to capture the general subjective experience of using the simulator, and to study task performance. The focus was to study the effects of latency on the subjective experience, with regards to delays in the crane control interface. Subjective studies were performed with controlled delays added to the display update and hand controller (joystick) signals. The added delays ranged from 0 to 30 ms for the display update, and from 0 to 800 ms for the hand controller. We found a strong effect on latency in the display update and a significant negative effect for 800 ms added delay on latency in the hand controller (in total approx. 880 ms latency including the system delay). The Simulator Sickness Questionnaire (SSQ) gave significantly higher scores after the experiment compared to before the experiment, but a majority of the participants reported experiencing only minor symptoms. Some test subjects ceased the test before finishing due to their symptoms, particularly due to the added latency in the display update.
Various kinds of engineering software and digitalized equipment are widely applied through the lifecycle of industrial products. As a result, massive data of different types are being produced. However, these data are hysteretic and isolated from each other, leading to low efficiency and low utilization of these valuable data. Simulation based on theoretical and static model has been a conventional and powerful tool for the verification, validation, and optimization of a system in its early planning stage, but no attention is paid to the simulation application during system run-time. With the development of new-generation information and digitalization technologies, more data can be collected, and it is time to find a way for the deep application of all these data. As a result, the concept of digital twin has aroused much concern and is developing rapidly. Dispute and discussions around concepts, paradigms, frameworks, applications, and technologies of digital twin are on the rise both in academic and industrial communities. After a complete search of several databases and careful selection according to the proposed criteria, 240 academic publications about digital twin are identified and classified. This paper conducts a comprehensive and in-depth review of these literatures to analyze digital twin from the perspective of concepts, technologies, and industrial applications. Research status, evolution of the concept, key enabling technologies of three aspects, and fifteen kinds of industrial applications in respective lifecycle phase are demonstrated in detail. Based on this, observations and future work recommendations for digital twin research are presented in the form of different lifecycle phases.
Digital Twin (DT) concept nowadays is shown via the simulations of the manufacturing systems and included those production processes and parametric 3D models of the product. It is the primary method for planning, analysing and optimising the factory layout and processes. Moreover, work on management via the simulation in real-time is already done using Virtual Reality (VR) tools from a safe and remote environment. However, there is a list of limitation of such kind of digital systems, as connectivity speed and precision of the digital environment. The primary goal of this study is to access second listed limitation and on the example of the fully synchronised physical with its digital replica industrial robot, increase the level of precision of the developed DT environment. The proposed approach introduces transfer of the mathematical model to the virtual environment, thus creating a precise and scaled visual model of the Industrial Robot.
Augmented reality (AR) is used to enhance the perception of the real world by integrating virtual objects to an image sequence acquired from various camera technologies. Numerous AR applications in robotics have been developed in recent years. The aim of this paper is to provide an overview of AR research in robotics during the five year period from 2015 to 2019. We classified these works in terms of application areas into four categories: (1) Medical robotics: Robot-Assisted surgery (RAS), prosthetics, rehabilitation, and training systems; (2) Motion planning and control: trajectory generation, robot programming, simulation, and manipulation; (3) Human-robot interaction (HRI): teleoperation, collaborative interfaces, wearable robots, haptic interfaces, brain-computer interfaces (BCIs), and gaming; (4) Multi-agent systems: use of visual feedback to remotely control drones, robot swarms, and robots with shared workspace. Recent developments in AR technology are discussed followed by the challenges met in AR due to issues of camera localization, environment mapping, and registration. We explore AR applications in terms of how AR was integrated and which improvements it introduced to corresponding fields of robotics. In addition, we summarize the major limitations of the presented applications in each category. Finally, we conclude our review with future directions of AR research in robotics. The survey covers over 100 research works published over the last five years.
Currently, new technologies have enabled the design of smart applications that are used as decision-making tools in the problems of daily life. The key issue in designing such an application is the increasing level of user interaction. Mixed reality (MR) is an emerging technology that deals with maximum user interaction in the real world compared to other similar technologies. Developing an MR application is complicated, and depends on the different components that have been addressed in previous literature. In addition to the extraction of such components, a comprehensive study that presents a generic framework comprising all components required to develop MR applications needs to be performed. This review studies intensive research to obtain a comprehensive framework for MR applications. The suggested framework comprises five layers: the first layer considers system components; the second and third layers focus on architectural issues for component integration; the fourth layer is the application layer that executes the architecture; and the fifth layer is the user interface layer that enables user interaction. The merits of this study are as follows: this review can act as a proper resource for MR basic concepts, and it introduces MR development steps and analytical models, a simulation toolkit, system types, and architecture types, in addition to practical issues for stakeholders such as considering MR different domains.
Teleoperation remains a dominant control paradigm for human interaction with robotic systems. However, teleoperation can be quite challenging, especially for novice users. Even experienced users may face difficulties or inefficiencies when operating a robot with unfamiliar and/or complex dynamics, such as industrial manipulators or aerial robots, as teleoperation forces users to focus on low-level aspects of robot control, rather than higher level goals regarding task completion, data analysis, and problem solving. We explore how advances in augmented reality (AR) may enable the design of novel teleoperation interfaces that increase operation effectiveness, support the user in conducting concurrent work, and decrease stress. Our key insight is that AR may be used in conjunction with prior work on predictive graphical interfaces such that a teleoperator controls a virtual robot surrogate, rather than directly operating the robot itself, providing the user with foresight regarding where the physical robot will end up and how it will get there. We present the design of two AR interfaces using such a surrogate: one focused on real-time control and one inspired by waypoint delegation. We compare these designs against a baseline teleoperation system in a laboratory experiment in which novice and expert users piloted an aerial robot to inspect an environment and analyze data. Our results revealed that the augmented reality prototypes provided several objective and subjective improvements, demonstrating the promise of leveraging AR to improve human-robot interactions.
For several years, augmented and virtual reality technologies have attracted increasing interest in all areas. In the midst of this universe, the concept, already well known, of mixed reality has established itself as a distinct paradigm. However, and contrary to augmented and virtual realities, there is not a clear and one definition of what it is exactly and why it is different from the other concepts. In this article, we attempt to provide a new classification method to standardize the definition of virtual, augmented and mixed realities. First, a quick overview of existing taxonomies is made, then we present our classification which is based on three criteria we called 3iVClass (Immersion, Interaction, Information). Finally, in order to verify its reliability, we used this classification to propose a definition of mixed reality.
It isnot surprising that managers find it hard to distinguish similar-sounding, IT-based concepts such as augmented reality and virtual reality. To many, all of these constructs mean nearly the same and, as a result, the terms are often used interchangeably. This confusion holds back those eager to explore the different opportunities these new technologies present. This Executive Digest presents six different types of reality and virtual reality–—(1) reality, (2) augmented reality, (3) virtual reality, (4) mixed reality, (5) augmented virtuality, and (6) virtuality–—as part of our actual reality/virtual reality continuum. We then illustrate their differences using a common example and outline business applications for each type
With the advances in new generation information technologies (New IT), especially big data and digital twin, smart manufacturing is becoming the focus of global manufacturing transformation and upgrading. Intelligence comes from data. Integrated analysis for the manufacturing big data is beneficial to all aspects of manufacturing. Besides, the digital twin paves a way for the cyber-physical integration of manufacturing, which is an important bottleneck to achieve smart manufacturing. In this paper, the big data and digital twin in manufacturing are reviewed, including their concept as well as their applications in product design, production planning, manufacturing, and predictive maintenance, etc. On this basis, what similarities and differences there are between big data and digital twin are compared from the general and data perspectives. Since the big data and digital twinning can be complementary, so how they can be integrated to promote smart manufacturing are discussed.
Multi-robot missions can be compared to industrial processes or public services in terms of complexity, agents and interactions. Process mining is an emerging discipline that involves process modeling, analysis and improvement through the information collected by event logs. Currently, this discipline is successfully used to analyze several types of processes, but is hardly applied in the context of robotics. This work proposes a systematic protocol for the application of process mining to analyze and improve multi-robot missions. As an example, this protocol is applied to a scenario of fire surveillance and extinguishing with a fleet of UAVs. The results show the potential of process mining in the analysis of multi-robot missions and the detection of problems such as bottlenecks and inefficiencies. This work opens the way to an extensive use of these techniques in multi-robot missions, allowing the development of future systems for optimizing missions, allocating tasks to robots, detecting anomalies or supporting operator decisions.
Digitalization, connected products and services, and shortening innovation cycles are widely discussed topics in management practice and theory and demand for new concepts. We analysed how companies innovated their business models and how are the new the technology trends. We found out, that have a positive approach to digitalization but the technology strategy still runs its original business model. Digitalization forces to new solution orientation. For companies it is necessary to master the digital transformation, new innovations have to be developed. Furthermore, digitalization / Industry 4.0 linking the real-life factory with virtual reality, will play an increasingly important role in global manufacturing. Companies have to obtain new digital capabilities, in order to make their company sustainable for the future. A long term growth and welfare in Europe could be guaranteed only by new technology innovation.
We present a novel technology that allows real robots to perceive an augmented reality environment through virtual sensors. Virtual sensors are a useful and desirable technology for research activities because they allow researchers to quickly and efficiently perform experiments that would otherwise be more expensive, or even impossible. In particular, augmented reality is useful (i) for prototyping and assessing the impact of new sensors before they are physically produced; and (ii) for developing and studying the behaviour of robots that should deal with phenomena that cannot be easily reproduced in a laboratory environment because, for example, they are dangerous (e.g., fire, radiations). We realised an augmented reality system for robots in which a simulator retrieves real-time data on the real environment through a multi-camera tracking system and delivers post-processed information to the robot swarm according to each robot’s sensing range. We illustrate the proposed virtual sensing technology through an experiment involving 15 e-pucks.
This paper surveys the current state-of-the-art of technology, systems and applications in Augmented Reality. It describes work performed by many different research groups, the purpose behind each new Augmented Reality system, and the difficulties and problems encountered when building some Augmented Reality applications. It surveys mobile augmented reality systems challenges and requirements for successful mobile systems. This paper summarizes the current applications of Augmented Reality and speculates on future applications and where current research will lead Augmented Reality’s development. Challenges augmented reality is facing in each of these applications to go from the laboratories to the industry, as well as the future challenges we can forecast are also discussed in this paper. Section 1 gives an introduction to what Augmented Reality is and the motivations for developing this technology. Section 2 discusses Augmented Reality Technologies with computer vision methods, AR devices, interfaces and systems, and visualization tools. The mobile and wireless systems for Augmented Reality are discussed in Section 3. Four classes of current applications that have been explored are described in Section 4. These applications were chosen as they are the most famous type of applications encountered when researching AR apps. The future of augmented reality and the challenges they will be facing are discussed in Section 5.
Latest trends and developments in digital technologies have enabled a new manufacturing model. Digital systems can monitor, optimize and control processes by creating a virtual copy of the physical world and making decentralized decisions. This paradigm relies on the development of a digital counterpart, the Digital Twin, for each production resource taking part to the whole manufacturing process. Although real applications of Digital Twin may differ in technical and operational details, in the past years, a huge effort has been done in order to identify and define focal functionalities and properties, as well as main challenges for the practical implementation within real factories. This paper is intended to review and analyse principles, ideas and technological solutions of the Digital Twin vision for production processes focusing on the practical industrial implementation. The purpose of this document is therefore to summarize the current state-of-art on Digital Twin concepts, and to draw their up-to-date state for application and deployment in real industrial processes. Finally, future directions for further research are discussed.
The ability to combine digital information with the real world enables mixed reality (MR) technology to provide a better display of information, resulting in its increasing popularity in various fields. The architecture, engineering, construction, and operation (AECO) industry is no exception. However, existing reviews on the use of MR technology can hardly keep up with the rapid development of MR applications. Therefore, a state-of-the-art review focusing on MR technology applications in the AECO industry is needed to reflect the current status of MR implementation in the AECO industry. This review is based on articles retrieved from well-acknowledged academic journals within the domain of the AECO industry. In this paper, 87 journal papers on MR applications are identified and classified into four categories: (1) applications in architecture and engineering, (2) applications in construction, (3) applications in operation, and (4) applications in multiple stages. Five basic components of MR, including spatial registration, display, user interaction, data storage, and multiuser collaboration, in each of the aforementioned 87 journal papers are identified and discussed. After reviewing the selected applications and corresponding MR components, this paper summarizes the challenges of MR development and provides insights into future trends of the MR technology in four aspects, namely: (1) accuracy of spatial registration, (2) user interface (UI), (3) data storage and transfer, and (4) multiuser collaboration.
Modern Industrial Robot (IR) programming process is mainly performed by using three different methods — manual, offline, and online programming. Each of these methods has various advantages and disadvantages. Prominent automotive industries often use a combination of them, as there is no way to avoid one or another form of programming on one factory. However, the use of a combination of different programming methods is time-consuming and demands the operator’s presence on site for reconfiguration of the IR. The primary goal of this study is to introduce and test the concept of a hybrid IR programming method, which combines both: offline and online robotic cell design, programming, and re-configuration methods. Testing of this method is based on fully synchronized robotic cell’s Digital Twin (DT), developed in Industrial Virtual and Augmented Laboratory of Tallinn University of Technology. Usage of the virtual replica allows to plan and program robotic cell on the means of telepresence and interfere with the predefined path of the robot by online programming method. Moreover, this approach reduces the time for robotic cell design and re-programming, enables to minimize downtime of the robotic cell on the factory shop floor. Included Virtual Reality (VR) environment allows simulating a full-scale operator presence on site. Thus, the proposed approach supports an immersive and safe environment for the IR and similar equipment programming purposes.
The arrival of Virtual-Reality, Augmented-Reality, and Mixed-Reality technologies is shaping a new environment where physical and virtual objects are integrated at different levels. Due to the development of portable and embodied devices, together with highly interactive, physical-virtual connections, the customer experience landscape is evolving into new types of hybrid experiences. However, the boundaries between these new realities, technologies and experiences have not yet been clearly established by researchers and practitioners. This paper aims to offer a better understanding of these concepts and integrate technological (embodiment), psychological (presence), and behavioral (interactivity) perspectives to propose a new taxonomy of technologies, namely the “EPI Cube”. The cube allows academics and managers to classify all technologies, current and potential, which might support or empower customer experiences, but can also produce new experiences along the customer journey. The paper concludes with theoretical and managerial implications, as well as a future research agenda.
Originally initiated in Germany, Industry 4.0, the fourth industrial revolution, has attracted much attention in recent literatures. It is closely related with the Internet of Things (IoT), Cyber Physical System (CPS), information and communications technology (ICT), Enterprise Architecture (EA), and Enterprise Integration (EI). Despite of the dynamic nature of the research on Industry 4.0, however, a systematic and extensive review of recent research on it is has been unavailable. Accordingly, this paper conducts a comprehensive review on Industry 4.0 and presents an overview of the content, scope, and findings of Industry 4.0 by examining the existing literatures in all of the databases within the Web of Science. Altogether, 88 papers related to Industry 4.0 are grouped into five research categories and reviewed. In addition, this paper outlines the critical issue of the interoperability of Industry 4.0, and proposes a conceptual framework of interoperability regarding Industry 4.0. Challenges and trends for future research on Industry 4.0 are discussed.
Some industrial jobs are particularly difficult. Operators must work in noisy or risk-filled environment and achieve repetitive motion or carry heavy loads which may cause repetitive strain injury (RSI). One approach to fulfill these arduous tasks is to use cobots, a neologism for "cooperation" and "robotics". Cobotics refers to collaboration between a person and a robot in a same area. The main idea is to help the operator to realize hard work. For this kind of collaboration it is necessary to work in safe conditions and new robots have been built for this purpose. The main issue of these robots is that they are secure thanks to post-collisions algorithms. To improve the safety, pre-collision algorithms need to be developed and that's what this paper tends to propose. Thanks to virtual reality tools, it is now possible to secure an area and act on the robots before any injury occurs. The low-cost system we propose here allows to improve the safety of industrial robots thanks to new virtual reality device like a Kinect2. We also propose a solution of intelligent learning of new tasks for robots.
The vision of the Digital Twin itself refers to a comprehensive physical and functional description of a component, product or system, which includes more or less all information which could be useful in all—the current and subsequent—lifecycle phases. In this chapter we focus on the simulation aspects of the Digital Twin. Today, modelling and simulation is a standard process in system development, e.g. to support design tasks or to validate system properties. During operation and for service first simulation-based solutions are realized for optimized operations and failure prediction. In this sense, simulation merges the physical and virtual world in all life cycle phases. Current practice already enables the users (designer, SW/HW developers, test engineers, operators, maintenance personnel, etc) to master the complexity of mechatronic systems.
1 Virtual Reality.- 1.1 Introduction.- 1.2 What Is VR?.- 1.3 Who Should Read This Book?.- 1.4 The Aims and Objectives of This Book.- 1.5 Assumptions Made in This Book.- 1.6 How to Use This Book.- 1.7 Some VR Concepts and Terms.- 1.8 Navigation and Interaction.- 1.9 Immersion and Presence.- 1.10 What Is Not VR?.- 1.11 The Internet.- 1.12 Summary.- 2 The Benefits of VR.- 2.1 Introduction.- 2.2 3D Visualization.- 2.3 Navigation.- 2.4 Interaction.- 2.5 Physical Simulation.- 2.6 VEs.- 2.7 Applications.- 2.8 Summary.- 3 3D Computer Graphics.- 3.1 Introduction.- 3.2 From Computer Graphics to VR.- 3.3 Modelling Objects.- 3.4 Dynamic Objects.- 3.5 Constraints.- 3.6 Collision Detection.- 3.7 Perspective Views.- 3.8 3D Clipping.- 3.9 Stereoscopic Vision.- 3.10 Rendering the Image.- 3.11 Rendering Algorithms.- 3.12 Texture Mapping.- 3.13 Bump Mapping.- 3.14 Environment Mapping.- 3.15 Shadows.- 3.16 Radiosity.- 3.17 Other Computer Graphics Techniques.- 3.18 Summary.- 4 Human Factors.- 4.1 Introduction.- 4.2 Vision.- 4.3 Vision and Display Technology.- 4.4 Hearing.- 4.5 Tactile.- 4.6 Equilibrium.- 4.7 Summary.- 5 VR Hardware.- 5.1 Introduction.- 5.2 Computers.- 5.3 Tracking.- 5.4 Input Devices.- 5.5 Output Devices.- 5.6 Glasses.- 5.7 Displays.- 5.8 Audio.- 5.9 Summary.- 6 VR Software.- 6.1 Introduction.- 6.2 VR Software Features.- 6.3 Web-Based VR.- 6.4 Division's dVISE.- 6.5 Blueberry3D.- 6.6 Boston Dynamics.- 6.7 MultiGen.- 6.8 Summary.- 7 VR Applications.- 7.1 Introduction.- 7.2 Industrial.- 7.3 Training Simulators.- 7.4 Entertainment.- 7.5 VR Centres.- 7.6 Summary.- 8 Conclusion.- 8.1 The Past.- 8.2 Today.- 8.3 Conclusion.- Appendices.- Appendix A VRML Web Sites.- Appendix B HMDs.- Appendix C Trackers.- Appendix D VRML Program.- Appendix E Web Sites for VR Products.- Referebces.
The Important Difference Between Virtual Reality, Augmented Reality and Mixed Reality
- B Marr
B. Marr, "The Important Difference Between Virtual Reality,
Augmented Reality and Mixed Reality." [Online]. Available:
https://www.forbes.com/sites/bernardmarr/2019/07/19/theimportant-difference-between-virtual-reality-augmentedreality-and-mixed-reality/
Robot safe interaction system for intelligent industrial co-robots
- C Liu
- M Tomizuka
C. Liu and M. Tomizuka, "Robot safe interaction system for
intelligent
industrial
co-robots,"
arXiv
preprint
arXiv:1808.03983, 2018.
A systematic review of Augmented Reality interfaces for collaborative industrial robots
- F. De Pace
- F Manuri
- A Sanna
- C Fornaro
F. De Pace, F. Manuri, A. Sanna, and C. Fornaro, "A
systematic review of Augmented Reality interfaces for
collaborative industrial robots," Computers & Industrial
Engineering, vol. 149, p. 106806, 2020.
Pandemic Alerting Use Case
Pandemic Alerting Use Case," Computational Intelligence
and Neuroscience, vol. 2022, p. 7786441, 2022/01/13 2022,
doi: https://doi.org/10.1155/2022/7786441.