No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Comparison of Different Natural Language Processing Models to Achieve Semantic Interoperability of Heterogeneous Asset Administration Shells
... Very recently, language models have been also applied to Industry 4.0 applications, in particular for conversion of asset descriptions to OPC UA [25], semantics matching [10], and control logic generation [28]. To the best of our knowledge, there has been no propositions nor evaluations in the literature thus far how to use LLMs for generating AAS structures. ...
The digital transformation of manufacturing-Industry 4.0-has sparked interest in digital twins, which are virtual replicas of physical assets. One popular structure for these digital twins is the Asset Administration Shell (AAS), which has been widely adopted. However, the large-scale conversion of asset data into AAS structures is not trivial, particularly when the asset data is unstructured. In this study, we investigate the use of large language models such as CHATGPT for this task. Our approach involves clustering product descriptions into product categories, structuring manually parts of each cluster, and then using CHATGPT to generalize this structuring to unseen product descriptions. We evaluate our methodology and compare it to alternative approaches. Our findings indicate that large language models can effectively produce structured AAS v3.0 data. For some product categories full automation is possible-at worst, 17% of the structures need to be manually corrected. We provide a novel real-world industrial data set as well as tools for the comparison of AAS structures for future studies.
Zusammenfassung
Im Rahmen von Industrie 4.0 wurde die Verwaltungsschale als die Technologie für Interoperabilität entwickelt und durch Spezifikationen definiert. Während die syntaktische Interoperabilität durch standardisierte Schemata, z. B. JSON Schema, XSD oder RDFS, erreicht werden kann, bleibt die Sicherstellung der semantischen Interoperabilität eine Herausforderung, insbesondere wenn heterogene Definitionen von semantischen Beschreibungen verwendet werden. Aufgrund der Vielfalt und großen Anzahl an Teilnehmern innerhalb eines Industrie 4.0 Wertschöpfungsnetzwerks erscheint die Einigung auf einen einzigen Standard semantischer Beschreibungen unwahrscheinlich. Darüber hinaus existieren mehrere Methoden, um diese Beschreibungen aufeinander abzubilden (engl. matching). Diese reichen von der manuellen Definition von Äquivalenzbeziehungen über eine algorithmische Analyse von OWL-Ontologien bis hin zur Nutzung von speziell trainierten KI-Modellen. Zur Handhabung dieser Heterogenität wird ein Konzept vorgestellt, welches den Zugriff auf die verschiedenen Quellen semantischer Beschreibungen sowie verschiedene Methoden ihres Matchings durch ein einheitliches Interface abstrahiert, sodass ein Beitrag zur Verbesserung semantischer Interoperabilität geleistet wird.
This research introduces a novel approach for achieving semantic interoperability in digital twins and assisting the creation of Asset Administration Shell (AAS) as digital twin model within the context of Industry 4.0. The foundational idea of our research is that the communication based on semantics and the generation of meaningful textual data are directly linked, and we posit that these processes are equivalent if the exchanged information can be serialized in text form. Based on this, we construct a “semantic node” data structure in our research to capture the semantic essence of textual data. Then, a system powered by large language models is designed and implemented to process the “semantic node” and generate standardized digital twin models (AAS instance models in the context of Industry 4.0) from raw textual data collected from datasheets describing technical assets. Our evaluation demonstrates an effective generation rate of 62-79%, indicating a substantial proportion of the information from the source text can be translated error-free to the target digital twin instance model with the generative capability of large language models. This result has a direct application in the context of Industry 4.0, and the designed system is implemented as a data model generation tool for reducing the manual effort in creating AAS model by automatically translating unstructured textual data into a standardized AAS model. The generated AAS model can be integrated into AAS-compliant digital twin software for seamless information exchange and communication. In our evaluation, a comparative analysis of different LLMs and an in-depth ablation study of Retrieval-Augmented Generation (RAG) mechanisms provide insights into the effectiveness of LLM systems for interpreting technical concepts and translating data. Our findings emphasize LLMs’ capability to automate AAS instance creation and contribute to the broader field of semantic interoperability for digital twins in industrial applications. The prototype implementation and evaluation results are presented on our GitHub Repository: https://github.com/YuchenXia/AASbyLLM.
This research introduces a novel approach for achieving semantic interoperability in digital twins and assisting the creation of Asset Administration Shell (AAS) as digital twin model within the context of Industry 4.0. The foundational idea of our research is that the communication based on semantics and the generation of meaningful textual data are directly linked, and we posit that these processes are equivalent if the exchanged information can be serialized in text form. Based on this, we construct a “semantic node” data structure in our research to capture the semantic essence of textual data. Then, a system powered by large language models is designed and implemented to process the “semantic node” and generate standardized digital twin models (AAS instance models in the context of Industry 4.0) from raw textual data collected from datasheets describing technical assets. Our evaluation demonstrates an effective generation rate of 62-79%, indicating a substantial proportion of the information from the source text can be translated error-free to the target digital twin instance model with the generative capability of large language models. This result has a direct application in the context of Industry 4.0, and the designed system is implemented as a data model generation tool for reducing the manual effort in creating AAS model by automatically translating unstructured textual data into a standardized AAS model. The generated AAS model can be integrated into AAS-compliant digital twin software for seamless information exchange and communication. In our evaluation, a comparative analysis of different LLMs and an in-depth ablation study of Retrieval-Augmented Generation (RAG) mechanisms provide insights into the effectiveness of LLM systems for interpreting technical concepts and translating data. Our findings emphasize LLMs’ capability to automate AAS instance creation and contribute to the broader field of semantic interoperability for digital twins in industrial applications. The prototype implementation and evaluation results are presented on our GitHub Repository: https://github.com/YuchenXia/AASbyLLM.
Self-organizing systems represent the next stage in the development of automation technology. For being able to interact with each other in an interoperable manner, it requires a uniform digital representation of the system's components, in the form of digital twins. In addition, the digital twins must be semantically interoperable in order to realize interoperability without the need for costly engineering in advance. For this purpose, the current research approach focuses on a semantically homogeneous language space. Due to the multitude of actors within an automation network, the agreement on a single semantic standard seems unlikely. Different standards and vendor-specific descriptions of asset information will continue to exist. This paper presents a method extending the homogeneous semantics approach to heterogeneous semantics. For this purpose, a translation mechanism is designed. The mapping of unknown vocabularies to a target vocabulary enables the interactions of semantically heterogeneous digital twins. The mapping is based on methods from the artifcial intelligence domain, specifically machine learning and natural language processing. Semantic attributes (name, definition) as well as further classifying attributes (unit, data type, qualifier, category, submodel element subtype) of the digital twins' attributes are used therefore. For the mapping of the semantic attributes pre-trained language models on domain specific texts and sentence embeddings are combined. A decision tree classifies the other attributes. Different semantics for submodels of pumps and HVAC systems are used as the evaluation dataset. The combination of the classification of the attributes (decision tree) and the subsequent semantic matching (language model), leads to a significant increase in accuracy compared to previous studies.
Energy management systems are an important tool for increasing the energy efficiency of buildings. However, the widespread availability of such systems is offset by the high complexity and high costs of implementation, as well as a lack of data. By using standardized digital twins of technical components, these obstacles can be addressed. In combination with homogeneous semantics of the digital twins and standardized interfaces as uniform access points to the information, the implementation of an energy management system can be simplified. If all technical components of a building have the same information technology structure in the form of digital twins and make their standardized information uniformly available for query, simple query rules can be implemented. These enable the automated integration of the information into an energy management system. However, given the large number of different manufacturers of the technical components, agreement on a common semantic standard in particular seems unlikely. Studies show that methods from the field of Natural Language Processing can be used to process heterogeneous semantics. Agreement on a common vocabulary is no longer necessary. Instead, different semantics can be used and matched to a target vocabulary. In order to use semantic matching in Industrie 4.0 environments, it must be provided as an Industrie 4.0 service. The service provides a translation mechanism from a foreign vocabulary to one's own. For this purpose, a standardized Industrie 4.0 interface consisting of two operations is specified. This interface is implemented prototypically as an API to show how it can be used. The specified interface can be used within the digital twins to process heterogeneous semantics and map them to its own. Extending the Industrie 4.0 approach from homogeneous to heterogeneous semantics can help simplifying the implementation of energy management systems. Simpler implementation lowers the barriers to the use of such systems, which in turn can lead to their higher availability.
Zusammenfassung
Systeme im Bereich Industrie 4.0 sollen interoperabel miteinander agieren können. Damit dies automatisiert realisiert werden kann, müssen sie semantisch interoperabel sein. Hierfür fokussiert der aktuelle Industrie 4.0 Forschungsansatz einen semantisch homogenen Sprachraum. In diesem Paper wird eine Methode vorgestellt, die diesen Ansatz um heterogene Semantik erweitert. Die Abbildung unbekannter Vokabulare auf eine Zielontologie ermöglicht die Interaktionen heterogener Verwaltungsschalen. Basis der Abbildung sind Methoden aus dem Bereich Natural Language Processing. Hierzu werden auf ISO Standards vortrainierte language models und sentence embeddings kombiniert. Dies führt zu einer vielversprechenden Genauigkeit bei dem erstellten Evaluationsdatensatz, welcher unterschiedliche Semantiken für Identifikation- und Design-Teilmodelle des Projektes Pumpe 4.0 enthält.
The term "Linked Data" refers to a set of best practices for publishing and connecting structured data on the Web. These best practices have been adopted by an increasing number of data providers over the last three years, leading to the creation of a global data space containing billions of assertions-the Web of Data. In this article, the authors present the concept and technical principles of Linked Data, and situate these within the broader context of related technological developments. They describe progress to date in publishing Linked Data on the Web, review applications that have been developed to exploit the Web of Data, and map out a research agenda for the Linked Data community as it moves forward.
This paper surveys and organizes research works in a new paradigm in natural language processing, which we dub “prompt-based learning”. Unlike traditional supervised learning, which trains a model to take in an input x and predict an output y as P (y|x), prompt-based learning is based on language models that model the probability of text directly. To use these models to perform prediction tasks, the original input x is modified using a template into a textual string prompt x′ that has some unfilled slots, and then the language model is used to probabilistically fill the unfilled information to obtain a final string , from which the final output y can be derived. This framework is powerful and attractive for a number of reasons: it allows the language model to be pre-trained on massive amounts of raw text, and by defining a new prompting function the model is able to perform few-shot or even zero-shot learning, adapting to new scenarios with few or no labeled data. In this paper we introduce the basics of this promising paradigm, describe a unified set of mathematical notations that can cover a wide variety of existing work, and organize existing work along several dimensions, e.g. the choice of pre-trained language models, prompts, and tuning strategies. To make the field more accessible to interested beginners, we not only make a systematic review of existing works and a highly structured typology of prompt-based concepts, but also release other resources, e.g., a website NLPedia–Pretrain including constantly-updated survey, and paperlist.
Industrie 4.0 stands for intelligent networking of machines and processes. The I4.0 component is the core of the I4.0 developments. The I4.0 component consists of an administration shell and an asset. The administration shell represents the I4.0 component in the virtual digital world. The administration shell can be understood as a synonym for the concept of the digital twin. Details regarding the tasks and structure of administration shells are described in various papers. Among other things, they consist of submodels that describe different aspects of an asset. The development of the submodels and their description by standardised properties with unique identification represents a key task here. Current development work deals with the design of administration shells. A working group of the VDMA departments "Pumps + Systems" and "Compressors, Compressed Air and Vacuum Technology" is developing submodels for liquid and vacuum pumps.
As part of this group's work, a procedure model that can be used for the derivation and assignment of submodels has been developed. With the help of the procedure model, the development of submodels can be simplified. The goal is to create a uniform basis for the derivation of submodels and to prevent multiple definitions of characteristics that apply to multiple assets. On the one hand, basic asset functions were modeled on the basis of a life cycle model and they form the basis for the submodels. The functions are general in nature, so submodels can be developed for each asset based on these functions. However, the functions are only intended to define the framework for the development of the models and can be further refined in the actual definition of these. On the other hand, a classification is presented in which every asset can be located. The classification is based on the International Standard Industrial Classification, which was developed by the United Nations. However, the standard will be expanded to include more categories, allowing a finer asset split.
Based on this model, submodels for liquid and vacuum pumps were derived, which cover the areas of design, operation and maintenance. For the various submodels, lists of properties have been compiled which describe the use cases covered by the models. The submodels as a whole form the administration shell for pumps. For the development of an administration shell, an integration into a communication protocol is carried out in addition to the conceptual development of the submodels. For this purpose, the developed submodels are transferred into an OPC UA Companion Specification. In order to ensure independence from individual communication technologies, the submodels are developed independently from OPC UA. This enables later integration into other technologies. The communication connection should enable pumps to be integrated into an I4.0 environment.
Due to the importance of zero-shot learning, i.e. classifying images where there is a lack of labeled training data, the number of proposed approaches has recently increased steadily. We argue that it is time to take a step back and to analyze the status quo of the area. The purpose of this paper is three-fold. First, given the fact that there is no agreed upon zero-shot learning benchmark, we first define a new benchmark by unifying both the evaluation protocols and data splits of publicly available datasets used for this task. This is an important contribution as published results are often not comparable and sometimes even flawed due to, e.g. pre-training on zero-shot test classes. Moreover, we propose a new zero-shot learning dataset, the Animals with Attributes 2 (AWA2) dataset which we make publicly available both in terms of image features and the images themselves. Second, we compare and analyze a significant number of the state-of-the-art methods in depth, both in the classic zero-shot setting but also in the more realistic generalized zero-shot setting. Finally, we discuss in detail the limitations of the current status of the area which can be taken as a basis for advancing it.
The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train. Our model achieves 28.4 BLEU on the WMT 2014 English-to-German translation task, improving over the existing best results, including ensembles by over 2 BLEU. On the WMT 2014 English-to-French translation task, our model establishes a new single-model state-of-the-art BLEU score of 41.0 after training for 3.5 days on eight GPUs, a small fraction of the training costs of the best models from the literature. We show that the Transformer generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data.
Automation of the engineering of communication-capable plant components in plant-related asset management in the process industry
- J Müller
J. Müller, "Automation of the engineering of communication-capable
plant components in plant-related asset management in the process
industry", VDI-Verlag, Düsseldorf, 2007.
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
- V Sanh
- L Debut
- J Chaumond
- T Wolf
V. Sanh, L. Debut, J. Chaumond, T. Wolf, "DistilBERT, a distilled
version of BERT: smaller, faster, cheaper and lighter", 5th Workshop
on Energy Efficient Machine Learning and Cognitive Computing, Oct.
2, 2019.
ELECTRA: Pre-training Text encoders as discriminators rather than generators
- K Clark
- M Luong
- Q Manning
K. Clark, M. Luong, Q. Manning, "ELECTRA: Pre-training Text
encoders as discriminators rather than generators", ICLR Conference
2020, Mar. 23, 2020.
Improving Language Understanding by Generative Pre-Training
- A Radford
- K Narasimhan
- T Salimans
- I Sutskever
A. Radford, K. Narasimhan, T. Salimans, I. Sutskever, "Improving
Language Understanding by Generative Pre-Training", 2018.
Language Models are unsupervised multitask learners
- A Radford
- J Wu
- R Child
- D Luan
- D Amodei
- I Sutskever
A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever,
"Language Models are unsupervised multitask learners", 2019.
SentenceTransformers Documentation-Model Overview
- reimers
N. Reimers, "SentenceTransformers Documentation -Model
Overview",
SentenceTransformer
Ranking,
https://www.sbert.net/docs/pretrained_models.html (accessed: Jan. 5,
2022).
Linked Open Data Cloud
- J Mccrae
- A Abele
- P Buitelaar
- R Cyganiak
- A Jentzsch
- V Andryushechkin
J. McCrae, A. Abele, P. buitelaar, R. Cyganiak, A. Jentzsch, V.
Andryushechkin, "Linked Open Data Cloud", http://cas.lod-cloud.net
(accessed: Feb. 14, 2023).
Language Models are few-shot learners
- T Brown
- B Mann
- N Ryder
- M Subbiah
- J Kaplan
T. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal,
"Language Models are few-shot learners", May 28, 2020.
Scaling Language Models: Methods, Analysis & Insights from Training Gopher
- J Rae
- S Borgeaud
- T Cai
- K Millican
- J Hoffmann
- F Song
J. Rae, S. Borgeaud, T. Cai, K. Millican, J. Hoffmann, F. Song,
"Scaling Language Models: Methods, Analysis & Insights from
Training Gopher", Jan. 21, 2022.
ECLASS dataset -copus part
- J Beermann
- R Benfer
- M Both
- J Müller
J. Beermann, R. Benfer, M. Both, J. Müller, "ECLASS dataset -copus
part", Feb. 2023, doi: 10.57967/hf/0409.
Sentence transformer: all-mpnet-base-v2
- reimers
N. Reimers, "Sentence transformer: all-mpnet-base-v2", Huggingface,
https://huggingface.co/sentence-transformers/all-mpnet-base-v2
(accessed: Dez. 14, 2022).
Improving the mapping accuracy of heterogeneous Industry 4.0 Asset Administration Shells through the Integration of Knowledge Graphs
- R Benfer
- M Both
- J Müller
R. Benfer, M. Both, J. Müller, "Improving the mapping accuracy of
heterogeneous Industry 4.0 Asset Administration Shells through the
Integration of Knowledge Graphs", Automation 2023, VDI Verlag
2023, in press.
Multitask Prompted Training Enables Zero-Shot Task Generalization
- V Sanh
- A Webson
- C Raffel
- S Bach
- L Sutawika
- Z Alyafeai
V. Sanh, A. Webson, C. Raffel, S. Bach, L. Sutawika, Z. Alyafeai,
"Multitask Prompted Training Enables Zero-Shot Task
Generalization", Oct. 15, 2021.
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
- C Raffel
- N Shazeer
- A Roberts
- K Lee
- S Narang
- M Matena
C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena,
"Exploring the Limits of Transfer Learning with a Unified Text-to-Text
Transformer", Oct. 23, 2019.
ECLASS dataset -query part
- J Beermann
- R Benfer
- M Both
- J Müller
J. Beermann, R. Benfer, M. Both, J. Müller, "ECLASS dataset -query
part", Feb. 2023, doi: 10.57967/hf/0410.
gart-labor/engdistilBERT-se-eclass
- J Beermann
- R Benfer
- M Both
- J Müller
J. Beermann, R. Benfer, M. Both, J. Müller, "gart-labor/eng-distilBERT-se-eclass", Feb. 2023, doi: 10.57967/hf/0411.
gart-labor/all-mpnetbase-v2-eclass
- J Beermann
- R Benfer
- M Both
- J Müller
J. Beermann, R. Benfer, M. Both, J. Müller, "gart-labor/all-mpnetbase-v2-eclass", Feb. 2023, doi: 10.57967/hf/0412.
Sentence transformer: all-MiniLM-L6-v2
- reimers
N. Reimers, "Sentence transformer: all-MiniLM-L6-v2", Huggingface,
https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2
(accessed: Dez. 14, 2022).
RoBERTa: A Robustly Optimized BERT Pretraining Approach
- Y Liu
- M Ott
- N Goyal
- J Du
- M Joshi
- D Chen
Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, "RoBERTa: A
Robustly Optimized BERT Pretraining Approach", Jul. 26, 2019.
gart-labor/paraphraseMiniLM-L6-v2-eclass
- J Beermann
- R Benfer
- M Both
- J Müller
J. Beermann, R. Benfer, M. Both, J. Müller, "gart-labor/paraphrase-MiniLM-L6-v2-eclass", Feb. 2023, doi: 10.57967/hf/0413.
Electrotechnical Commission
- Iec Whitepaper
Electrotechnical
Commission,
"Semantic
Interoperability: Challenges in the digital Transformation Age", IEC
Whitepaper, 2019.
ECLASS -Standard for master data and semantics for digitalization
ECLASS, "ECLASS -Standard for master data and semantics for
digitalization", https://www.eclass.eu/index.html (accessed: Jan. 25,
2023).
Digital Nameplate for Industrial Equipment
Industrial Digital Twin Association, "Digital Nameplate for Industrial
Equipment", Oct. 20, 2022.
BERT: Pre-training of deep bidirectional transformers for language understanding
- J Delvin
- M Chang
- K Lee
- K Toutanova
J. Delvin, M. Chang, K. Lee, K. Toutanova, "BERT: Pre-training of
deep bidirectional transformers for language understanding", Oct. 11,
2018.
BRICK. A uniform metadata schema for buildings
- Brick Consortium
- Inc
Brick Consortium, Inc., "BRICK. A uniform metadata schema for
buildings", https://brickschema.org/ (accessed: Dez. 12, 2022).
Improving the mapping accuracy of heterogeneous Industry 4.0 Asset Administration Shells through the Integration of Knowledge Graphs
- benfer
Language Models are unsupervised multitask learners
- radford
ELECTRA: Pre-training Text encoders as discriminators rather than generators
- clark
Language Models are few-shot learners
- brown
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
- sanh
Semantic Interoperability: Challenges in the digital Transformation Age
- Iec Whitepaper
Automated mapping of Semantically Heterogeneous I4.0 Asset Administration Shells using Natural Language Processing methods
- both