No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
A Goal-Driven Natural Language Interface for Creating Application Integration Workflows
Abstract
Web applications and services are increasingly important in a distributed internet filled with diverse cloud services and applications, each of which enable the completion of narrowly defined tasks. Given the explosion in the scale and diversity of such services, their composition and integration for achieving complex user goals remains a challenging task for end-users and requires a lot of development effort when specified by hand. We present a demonstration of the Goal Oriented Flow Assistant (GOFA) system, which provides a natural language solution to generate workflows for application integration. Our tool is built on a three-step pipeline: it first uses Abstract Meaning Representation (AMR) to parse utterances; it then uses a knowledge graph to validate candidates; and finally uses an AI planner to compose the candidate flow. We provide a video demonstration of the deployed system as part of our submission.
... NL2LTL is built with a unique focus on extensibility: 1) The inputs and outputs are domain agnostic so it can be adopted The Thirty-Seventh AAAI Conference on Artificial Intelligence (AAAI-23) Figure 2: An application of NL2LTL to a real industrial application (Brachman et al. 2022 into any domain of choice; and 2) any and all componentsbe it the natural language understanding module or the scope of supported LTL formulas -are extendable or modifiable. Figure 3) illustrates the NL2LTL output -a list of translations, ranked by confidence. ...
... Industry Application For the real-world industry-scale application, we will build on a system demonstration at AAAI 2022 (Brachman et al. 2022) on a web service composition task using natural language. Such a system was dependent on code that is specific to the parser in order to look for specific patterns requested by the user -this means that there is a significant developer overhead in manually investigating the parser (Abstract Syntax Representations (Astudillo et al. 2020)) for a set of inputs, writing pattern extraction methods for it limited to that scope, while eventually that code is not reusable once the system upgrades to a different parser. ...
This is a demonstration of our newly released Python package NL2LTL which leverages the latest in natural language understanding (NLU) and large language models (LLMs) to translate natural language instructions to linear temporal logic (LTL) formulas. This allows direct translation to formal languages that a reasoning system can use, while at the same time, allowing the end-user to provide inputs in natural language without having to understand any details of an underlying formal language. The package comes with support for a set of default LTL patterns, corresponding to popular DECLARE templates, but is also fully extensible to new formulas and user inputs. The package is open-source and is free to use for the AI community under the MIT license. Open Source: https://github.com/IBM/nl2ltl. Video Link: https://bit.ly/3dHW5b1
... Recently, there have been several applications in which first multiple plans are generated and then the users are involved in the selection process. Some of these applications are in the area of patient monitoring , enterprise risk management , conversational systems (Chakraborti et al. 2022;Rizk et al. 2020;Sreedharan et al. 2020b), and web service composition (Brachman et al. 2022). However, the user interfaces for interacting with such systems has received little attention. ...
We present Lemming – a visualization tool for the interactive selection of plans for a given problem, allowing the user to efficiently whittle down the set of plans and select their plan(s) of choice. We demonstrate four different user experiences for this process, three of them based on the principle of using disjunctive action landmarks as guidance to cut down the set of choice points for the user, and one on the use of linear temporal logic (LTL) to impart additional constraints into the plan set using natural language (NL) instruction.
Selecting an effective utterance among countless possibilities that match a user's intention poses a challenge when using natural language interfaces. To address the challenge, we leveraged the principle of least collaborative effort in communication grounding theory and designed three grounded conversational interactions: 1) a grounding interface allows users to start with a provisional input and then invite a conversational agent to complete their input, 2) a multiple grounding interface presents multiple inputs for the user to select from, and 3) a structured grounding interface guides users to write inputs in a structure best understood by the system. We compared our three grounding interfaces to an ungrounded control interface in a crowdsourced study (N=80) using a natural language system that generates small programs. We found that the grounding interfaces reduced cognitive load and improved task performance. The structured grounding interface further reduced speaker change costs and improved technology acceptance, without sacrificing the perception of control. We discuss the implications of designing grounded conversational interactions in natural language systems.
Robotic process automation (RPA) is a technology that is presented as a universal tool that solves major problems of modern businesses. It aims to reduce costs, improve quality and create customer value. However, the business reality differs from this aspiration. After interviews with managers, we found that implementation of robots does not always lead to the assumed effect and some robots are subsequently withdrawn from companies. In consequence, people take over robotized tasks to perform them manually again, and in practice, replace back robots—what we call ‘re-manualization’. Unfortunately, companies do not seem to be aware of this possibility until they experience it on their own, to the best of our knowledge, no previous research described or analysed this phenomenon so far. This lack of awareness, however, may pose risks and even be harmful for organizations. In this paper, we present an exploratory study. We used individual interviews, group discussions with managers experienced in RPA, and secondary data analysis to elaborate on the re-manualization phenomenon. As a result, we found four types of ‘cause and effect’ narrations that reflect reasons for this to occur: (1) overenthusiasm for RPA, (2) low awareness and fear of robots, (3) legal or supply change and (4) code faults.KeywordsRobotic process automationRPASoftware robotInvestmentInformation systemsWork manualization
RPA technologies allow the automation of repeated processes through indirect or direct instruction from the end-user. While declarative authoring techniques provide a powerful tool to scale up process complexity with RPA elements, often such techniques are difficult to use without expertise. In this work, we will explore systems (in the context of web service composition and goal-oriented conversational agents) that both consumers and developers can interact with, in natural language, to compose RPA elements that demonstrate emergent intelligence as a composition of smaller units of automation. We will also discuss the overhead in authoring such systems, and potential learning opportunities in reducing said overhead. Finally, we will explore issues of explainability for the developer and transparency of dynamic compositions for the consumer in dealing with such systems with aggregated automation.KeywordsWeb service compositionAutomated planningNatural language processingProcess automation
The main aim of this paper is to present the results of a process-project maturity assessment of large organizations in Poland. The paper consists of two main parts: a theoretical part, which primarily outlines the rationale supporting the prospects and the need for an orientation towards process and project organizations, and an empirical part, presenting an attempt to integrate the MMPM and PMMM maturity models, in order to assess organizational level of process-project maturity. The empirical research carried out on a sample of 90 large organizations shows that vast majority of the organizations surveyed are characterized by low levels of process and project maturity, and 13 of the entities examined can be described, based on the assumptions adopted, as a process-project organization (level 4 of process-project maturity). Further, the research conducted has led to an outline of the factors supporting the recognition of process management as a method fundamental to the designing a process-project organization. Maturity model integration has demonstrated the levels of process and project maturity as well as a statistically positive correlation between the degree of process maturity and project maturity. The original character of this paper primarily concerns the need to fill the literature gap, consisting in the scarcity of publications describing integration of process and project management methods and the deficit of works presenting process-project maturity results.KeywordsProcess-project oriented organizationBPMProcess managementProject managementMaturity
Thanks to recent advances, AI Planning has become the un- derlying technique for several applications. Amongst these, a prominent one is automated Web Service Composition (WSC). One important issue in this context has been hardly addressed so far: WSC requires dealing with background on- tologies. The support for those is severely limited in current planning tools. We introduce a planning formalism that faith- fully represents WSC. We show that, unsurprisingly, planning in such a formalism is very hard. We then identify an inter- esting special case that covers many relevant WSC scenarios, and where the semantics are simpler and easier to deal with. This opens the way to the development of effective support tools for WSC. Furthermore, we show that if one additionally limits the amount and form of outputs that can be generated, then the set of possible states becomes static, and can be mod- elled in terms of a standard notion of initial state uncertainty. For this, effective tools exist; these can realize scalable WSC with powerful background ontologies. In an initial experi- ment, we show how scaling WSC instances are comfortably solved by a tool incorporating modern planning heuristics.
As programming skills increase in demand and utility, the learnability of end-user programming systems is of utmost importance. However, research on learning barriers in programming systems has primarily focused on languages, overlooking potential barriers in the environment and accompanying libraries. To address this, a study of beginning programmers learning Visual Basic.NET was performed. This identified six types of barriers: design, selection, coordination, use, understanding, and information. These barriers inspire a new metaphor of computation, which provides a more learner-centric view of programming system design
Diverse planning is an important problem in automated planning with many real world applications. Recently, diverse planning has seen renewed interest, with work that defines a taxonomy of computational problems with respect to both plan quality and solution diversity. However, despite the recent advances in diverse planning, the variety of approaches and the number of available planners are still quite limited, even nonexistent for several computational problems. In this work, we aim to extend the portfolio of planners for various computational problems in diverse planning. To that end, we introduce a novel approach to finding solutions for three computational problems within diverse planning and present planners for these three problems. For one of these problems, our approach is the first one that is able to provide solutions to the problem. For another, we show that top-k and top quality planners can provide, albeit naive, solutions to the problem and we extend these planners to improve the diversity of the solution. Finally, for the third problem, we show that some existing diverse planners already provide solutions to that problem. We suggest another approach and empirically show it to compare favorably with these existing planners.
Photo editing can be a challenging task, and it becomes even more difficult on the small, portable screens of mobile devices that are now frequently used to capture and edit images. To address this problem we present PixelTone, a multimodal photo editing interface that combines speech and direct manipulation. In this video, we demonstrate how our system uses natural language for expressing users' desired changes to an image. We also demonstrate how we combine natural language and touch gestures for creating named references and sketching to localize image operations to specific regions.
Component models have received much attention recently both from the software engineering research community and from industry. This is apparent through the sheer number of component models that have been developed in the last few years. The goal of each of these models is to increase reuse and to simplify the implementation and composition of new software. All these models focus on the specication and packaging of components but provide almost no support for the adaptation and easy composition of components. If components have not been written with each other in mind, their composition still has to be carried out programmatically. Since one of the advantages of software components is independent development, they will rarely be entirely compatible with each other.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
- J Devlin
- M.-W Chang
- K Lee
- K Toutanova
Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2019.
BERT: Pre-training of Deep Bidirectional Transformers for
Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association
for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 4171-4186. Minneapolis, Minnesota: Association for Computational Linguistics.
Cerberus: Red-black heuristic for planning tasks with conditional effects meets novelty heuristic and enchanced mutex detection
- M Katz
Katz, M. 2018. Cerberus: Red-black heuristic for planning
tasks with conditional effects meets novelty heuristic and
enchanced mutex detection. Ninth International Planning
Competition (IPC-9): planner abstracts, 47-51.
Reshaping diverse planning
- M Katz
- S Sohrabi
Katz, M.; and Sohrabi, S. 2020. Reshaping diverse planning. In Proceedings of the AAAI Conference on Artificial
Intelligence, volume 34.06, 9892-9899.
Delfi: Online planner selection for cost-optimal planning. IPC-9 planner abstracts
- M Katz
- S Sohrabi
- H Samulowitz
- S Sievers
Katz, M.; Sohrabi, S.; Samulowitz, H.; and Sievers, S.
2018a. Delfi: Online planner selection for cost-optimal planning. IPC-9 planner abstracts, 57-64.
A novel iterative approach to top-k planning
- M Katz
- S Sohrabi
- O Udrea
- D Winterer
Katz, M.; Sohrabi, S.; Udrea, O.; and Winterer, D. 2018b.
A novel iterative approach to top-k planning. In Twenty-Eighth International Conference on Automated Planning
and Scheduling.
Knowledge Graph Embedding using Graph Convolutional Networks with Relation-Aware Attention
- N Sheikh
- X Qin
- B Reinwald
- C Miksovic
- T Gschwind
- P Scotton
Sheikh, N.; Qin, X.; Reinwald, B.; Miksovic, C.; Gschwind,
T.; and Scotton, P. 2020. Knowledge Graph Embedding
using Graph Convolutional Networks with Relation-Aware
Attention. In The 2nd International Workshop on Deep
Learning on Graphs: Methods and Applications (KDD-DLG 2020).
Web service composition-current solutions and open problems
- B Srivastava
- J Koehler
Srivastava, B.; and Koehler, J. 2003.
Web service
composition-current solutions and open problems. In ICAPS
2003 workshop on Planning for Web Services, volume 35,
28-35.