Shiqi Zhang's research while affiliated with Binghamton University and other places
What is this page?
This page lists the scientific contributions of an author, who either does not have a ResearchGate profile, or has not yet added these contributions to their profile.
It was automatically created by ResearchGate to create a record of this author's body of work. We create such pages to advance our goal of creating and maintaining the most comprehensive scientific repository possible. In doing so, we process publicly available (personal) data relating to the author as a member of the scientific community.
If you're a ResearchGate member, you can follow this page to keep up with this author's work.
If you are this author, and you don't want us to display this page anymore, please let us know.
It was automatically created by ResearchGate to create a record of this author's body of work. We create such pages to advance our goal of creating and maintaining the most comprehensive scientific repository possible. In doing so, we process publicly available (personal) data relating to the author as a member of the scientific community.
If you're a ResearchGate member, you can follow this page to keep up with this author's work.
If you are this author, and you don't want us to display this page anymore, please let us know.
Publications (56)
Task planning systems have been developed to help robots use human knowledge (about actions) to complete long-horizon tasks. Most of them have been developed for "closed worlds" while assuming the robot is provided with complete world knowledge. However, the real world is generally open, and the robots frequently encounter unforeseen situations tha...
This paper presents intersection analysis using computer vision techniques with Simulation of Urban MObility (SUMO). At first, an efficient deep-visual tracking pipeline is proposed by using the off-the-shelf YOLO object detection architecture and cascading it with a discriminative correlation filter (CSRT) to produce reliable trajectories for traf...
Human-robot collaboration (HRC) has become increasingly relevant in industrial, household, and commercial settings. However, the effectiveness of such collaborations is highly dependent on the human and robots' situational awareness of the environment. Improving this awareness includes not only aligning perceptions in a shared workspace, but also b...
Large language models (LLMs) have demonstrated remarkable zero-shot generalization abilities: state-of-the-art chatbots can provide plausible answers to many common questions that arise in daily life. However, so far, LLMs cannot reliably solve long-horizon planning problems. By contrast, classical planners, once a problem is given in a formatted w...
Classical planning systems have shown great advances in utilizing rule-based human knowledge to compute accurate plans for service robots, but they face challenges due to the strong assumptions of perfect perception and action executions. To tackle these challenges, one solution is to connect the symbolic states and actions generated by classical p...
Robots frequently need to perceive object attributes, such as red, heavy, and empty, using multimodal exploratory behaviors, such as look, lift, and shake. One possible way for robots to do so is to learn a classifier for each perceivable attribute given an exploratory behavior. Once the attribute classifiers are learned, they can be used by robots...
Multi-object rearrangement is a crucial skill for service robots, and commonsense reasoning is frequently needed in this process. However, achieving commonsense arrangements requires knowledge about objects, which is hard to transfer to robots. Large language models (LLMs) are one potential source of this knowledge, but they do not naively capture...
In human-robot collaboration domains, augmented reality (AR) technologies have enabled people to visualize the state of robots. Current AR-based visualization policies are designed manually, which requires a lot of human efforts and domain knowledge. When too little information is visualized, human users find the AR interface not useful; when too m...
Given the current point-to-point navigation capabilities of autonomous vehicles, researchers are looking into complex service requests that require the vehicles to visit multiple points of interest. In this paper, we develop a layered planning framework, called GLAD, for complex service requests in autonomous urban driving. There are three layers f...
Automated task planning algorithms have been developed to help robots complete complex tasks that require multiple actions. Most of those algorithms have been developed for "closed worlds" assuming complete world knowledge is provided. However, the real world is generally open, and the robots frequently encounter unforeseen situations that can pote...
Goal-oriented dialog policy learning algorithms aim to learn a dialog policy for selecting language actions based on the current dialog state. Deep reinforcement learning methods have been used for dialog policy learning. This work is motivated by the observation that, although dialog is a domain with rich contextual knowledge, reinforcement learni...
Reasoning with declarative knowledge (RDK) and sequential decision‐making (SDM) are two key research areas in artificial intelligence. RDK methods reason with declarative domain knowledge, including commonsense knowledge, that is either provided a priori or acquired over time, while SDM methods (probabilistic planning [PP] and reinforcement learnin...
Task and motion planning (TAMP) algorithms have been developed to help robots plan behaviors in discrete and continuous spaces. Robots face complex real-world scenarios, where it is hardly possible to model all objects or their physical properties for robot planning (e.g., in kitchens or shopping centers). In this letter, we define a new object-cen...
Robot planning in partially observable domains is difficult, because a robot needs to estimate the current state and plan actions at the same time. When the domain includes many objects, reasoning about the objects and their relationships makes robot planning even more difficult. In this letter, we develop an algorithm called
scene analysis for ro...
Robot planning in partially observable domains is difficult, because a robot needs to estimate the current state and plan actions at the same time. When the domain includes many objects, reasoning about the objects and their relationships makes robot planning even more difficult. In this paper, we develop an algorithm called scene analysis for robo...
Task and motion planning (TAMP) algorithms aim to help robots achieve task-level goals, while maintaining motion-level feasibility. This paper focuses on TAMP domains that involve robot behaviors that take extended periods of time (e.g., long-distance navigation). In this paper, we develop a visual grounding approach to help robots probabilisticall...
Task and motion planning (TAMP) algorithms have been developed to help robots plan behaviors in discrete and continuous spaces. Robots face complex real-world scenarios, where it is hardly possible to model all objects or their physical properties for robot planning (e.g., in kitchens or shopping centers). In this paper, we define a new object-cent...
Mobile telepresence robots (MTRs) allow people to navigate and interact with a remote environment that is in a place other than the person's true location. Thanks to the recent advances in 360 degree vision, many MTRs are now equipped with an all-degree visual perception capability. However, people's visual field horizontally spans only about 120 d...
Human-robot collaboration frequently requires extensive communication, e.g., using natural language and gestures. Augmented reality (AR) has provided an alternative way of bridging the communication gap between robots and people. However, most current AR-based human-robot communication methods are unidirectional, focusing on how the human adapts to...
Key challenges to widespread deployment of mobile robots to interact with humans in real-world domains include the ability to: (a) robustly represent and revise domain knowledge; (b) autonomously adapt sensing and processing to the task at hand; and (c) learn from unreliable high-level human feedback. Partially observable Markov decision processes...
This paper introduces two new representation structures for tasks and situations, and a comprehensive approach for case-based planning (CBP). We focus on everyday tasks in open or semi-open domains, where exist a variety of situations that a planning (and execution) agent must deal with. This paper first introduces a new, generic structure for repr...
Everyday tasks are characterized by their varieties and variations, and frequently are not clearly specified to service agents. This paper presents a comprehensive approach to enable a service agent to deal with everyday tasks in open, uncontrolled environments. We introduce a generic structure for representing tasks, and another structure for repr...
Legged robots have been shown to be effective in navigating unstructured environments. Although there has been much success in learning locomotion policies for quadruped robots, there is little research on how to incorporate human knowledge to facilitate this learning process. In this paper, we demonstrate that human knowledge in the form of LTL fo...
Robots frequently need to perceive object attributes, such as "red," "heavy," and "empty," using multimodal exploratory actions, such as "look," "lift," and "shake." Robot attribute learning algorithms aim to learn an observation model for each perceivable attribute given an exploratory action. Once the attribute models are learned, they can be use...
Reinforcement learning (RL) enables an agent to learn from trial-and-error experiences toward achieving long-term goals; automated planning aims to compute plans for accomplishing tasks using action knowledge. Despite their shared goal of completing complex tasks, the development of RL and automated planning has been largely isolated due to their d...
Goal-oriented dialog systems aim to efficiently and accurately exchange information with people using natural language. A goal-oriented dialog policy is used for suggesting language actions for such dialog systems. Reinforcement learning have been used for computing dialog policies from the experience of language-based interaction. Learning efficie...
Reasoning with declarative knowledge (RDK) and sequential decision-making (SDM) are two key research areas in artificial intelligence. RDK methods reason with declarative domain knowledge, including commonsense knowledge, that is either provided a priori or acquired over time, while SDM methods (probabilistic planning and reinforcement learning) se...
Reinforcement learning and probabilistic reasoning algorithms aim at learning from interaction experiences and reasoning with probabilistic contextual knowledge respectively. In this research, we develop algorithms for robot task completions, while looking into the complementary strengths of reinforcement learning and probabilistic reasoning techni...
Reinforcement learning methods have been used to compute dialog policies from language-based interaction experiences. Efficiency is of particular importance in dialog policy learning, because of the considerable cost of interacting with people, and the very poor user experience from low-quality conversations. Aiming at improving the efficiency of d...
Model-based reinforcement learning (RL) enables an agent to learn world models from trial-and-error experiences toward achieving long-term goals. Automated planning, on the other hand, can be used for accomplishing tasks through reasoning with declarative action knowledge. Despite their shared goal of completing complex tasks, the development of RL...
Deep reinforcement learning (RL) algorithms frequently require prohibitive interaction experience to ensure the quality of learned policies. The limitation is partly because the agent cannot learn much from the many low-quality trials in early learning phase, which results in low learning rate. Focusing on addressing this limitation, this paper mak...
Robot sequential decision-making in the real world is a challenge because it requires the robots to simultaneously reason about the current world state and dynamics, while planning actions to accomplish complex tasks. On the one hand, declarative languages and reasoning algorithms well support representing and reasoning with commonsense knowledge....
Robots frequently face complex tasks that require more than one action, where sequential decision-making (sdm) capabilities become necessary. The key contribution of this work is a robot sdm framework, called lcorpp, that supports the simultaneous capabilities of supervised learning for passive state estimation, automated reasoning with declarative...
Autonomous vehicles need to plan at the task level to compute a sequence of symbolic actions, such as merging left and turning right, to fulfill people's service requests, where efficiency is the main concern. At the same time, the vehicles must compute continuous trajectories to perform actions at the motion level, where safety is the most importa...
Multi-robot planning (mrp) aims at computing plans, each in the form of a sequence of actions, for a team of robots to achieve their individual goals, while minimizing overall cost. Solving mrp problems requires modeling limited domain resources (e.g., corridors that allow at most one robot at a time), and the possibility of action synergy (e.g., m...
Effective human-robot collaboration (HRC) requires extensive communication among the human and robot teammates, because their actions can potentially produce conflicts, synergies, or both. In this paper, we develop an augmented reality-driven, negotiation-based (ARN) framework for HRC, where ARN supports planning-phase negotiations within human-rob...
Reinforcement learning methods have been used for learning dialogue policies. However, learning an effective dialogue policy frequently requires prohibitively many conversations. This is partly because of the sparse rewards in dialogues, and the very few successful dialogues in early learning phase. Hindsight experience replay (HER) enables learnin...
Some robots can interact with humans using natural language, and identify service requests through human-robot dialog. However, few robots are able to improve their language capabilities from this experience. In this paper, we develop a dialog agent for robots that is able to interpret user commands using a semantic parser, while asking clarificati...
Sequential decision-making (SDM) plays a key role in intelligent robotics, and can be realized in very different ways, such as supervised learning, automated reasoning, and probabilistic planning. The three families of methods follow different assumptions and have different (dis)advantages. In this work, we aim at a robot SDM framework that exploit...
The Association for the Advancement of Artificial Intelligence, in cooperation with Stanford University’s Department of Computer Science, presented the 2018 Spring Symposium Series, held Monday through Wednesday, March 26–28, 2018, on the campus of Stanford University. The seven symposia held were AI and Society: Ethics, Safety and Trustworthiness...
Task-motion planning (TMP) addresses the problem of efficiently generating executable and low-cost task plans in a discrete space such that the (initially unknown) action costs are determined by motion plans in a corresponding continuous space. However, a task-motion plan can be sensitive to unexpected domain uncertainty and changes, leading to sub...
Reinforcement learning (RL) agents aim at learning by interacting with an environment, and are not designed for representing or reasoning with declarative knowledge. Knowledge representation and reasoning (KRR) paradigms are strong in declarative KRR tasks, but are ill-equipped to learn from such experiences. In this work, we integrate logical-prob...
Reinforcement learning methods have been used for learning dialogue policies from the experience of conversations. However, learning an effective dialogue policy frequently requires prohibitively many conversations. This is partly because of the sparse rewards in dialogues, and the relatively small number of successful dialogues in early learning p...
Intelligent robots frequently need to explore the objects in their working environments. Modern sensors have enabled robots to learn object properties via perception of multiple modalities. However, object exploration in the real world poses a challenging trade-off between information gains and exploration action costs. Mixed observability Markov d...
General purpose planners enable AI systems to solve many different types of planning problems. However, many different planners exist, each with different strengths and weaknesses, and there are no general rules for which planner would be best to apply to a given problem. In this paper, we empirically compare the performance of state-of-the-art pla...
To operate in human-robot coexisting environments, intelligent robots need to simultaneously reason with commonsense knowledge and plan under uncertainty. Markov decision processes (MDPs) and partially observable MDPs (POMDPs), are good at planning under uncertainty toward maximizing long-term rewards; P-LOG, a declarative programming language unde...
Recent progress in both AI and robotics have enabled the development of general purpose robot platforms that are capable of executing a wide variety of complex, temporally extended service tasks in open environments. This article introduces a novel, custom-designed multi-robot platform for research on AI, robotics, and especially human–robot intera...
Planning in real-world environments can be challenging for intelligent robots due to incomplete domain knowledge that results from unpredictable domain dynamism, and due to lack of global observability. Action language \(\mathcal{BC}\) can be used for planning by formalizing the preconditions and (direct and indirect) effects of actions, and is esp...
In order to be fully robust and responsive to a dynamically changing real-world environment, intelligent robots will need to engage in a variety of simultaneous reasoning modalities. In particular, in this paper we consider their needs to i) reason with commonsense knowledge, ii) model their nondeterministic action outcomes and partial observabilit...
Citations
... In some cases, the reward specification is obtained from statistics and/or contextual knowledge provided by humans. For example, the iCORPP algorithm enables a robot to reason with contextual knowledge using P-log to automatically determine the rewards (and transition functions) of a POMDP used for planning (Zhang, Khandelwal, and Stone 2017). Another system, called LPPGI, enables robots to leverage human expertise for POMDPbased planning under uncertainty in the context of task specification and execution (Hoelscher et al. 2018). ...
... KRR is a subarea of AI focusing on how information about the world can be represented formally and manipulated in an automated way by some reasoning programs (Zhang and Sridharan, 2020;Brachman and Levesque, 2004). Research on KRR has a long history of more than half century along with the development of AI, covering various topics including knowledge representation languages, knowledge/rulebased systems, case-based/commonsense reasoning, spatial/temporal reasoning, action models, and uncertainties and beliefs, etc. Symbolic logic plays a vital role in KRR as logic provides a strict and formal representation of knowledge and their entailment relations. ...
![[object Object]](https://i1.rgstatic.net/ii/profile.image/272924036825090-1442081569456_Q64/Mohan-Sridharan.jpg)
... Similar to the evaluation measures, the benchmarks should instead challenge the robot to explore and use the interplay between the different components of the system being evaluated, for example, use reasoning to guide knowledge acquisition, and use the learned knowledge to inform reasoning. In this context, many different domains hold promise in terms of being suitable for evaluation of such RDK-for-SDM systems; these include games (Yang et al. 2018;, interactive dialog (Amiri et al. 2019;Zhang and Stone 2015), robot navigation and exploration (Hanheide et al. 2017;Leonetti, Iocchi, and Stone 2016), and scene understanding (Chitnis, Kaelbling, and Lozano-Pérez 2018;Jiang et al. 2019;Mota and Sridharan 2019;Mota, Sridharan, and Leonardis 2021). ...
... In the case where the object attributes refer to the object's function, they are then referred to as 0-order affordances [59]. Task and motion planning methods have been applied to object-centric perception while leveraging physics simulation [68]. Those methods focused on learning to improve the robots' perception capabilities. ...
![[object Object]](https://i1.rgstatic.net/ii/profile.image/801043420024832-1567995042916_Q64/Yan-Ding-12.jpg)
... Due to its efficient representation capacity, this task holds strong promises for other downstream tasks such as Image Captioning [38,33,40] or Visual Question Answering [7,16]. Recent contributions to the field highlight an opportunity for SGG to support the reasoning of an embodied agent by leveraging both the spatial and semantic latent context of a scene in a single representation [4,21,3]. However, despite a vast amount of work in the last few years, the performance of the best approaches is far from optimal, and the usage in downstream tasks is limited [50]. ...
![[object Object]](https://i1.rgstatic.net/ii/profile.image/791226127224834-1565654417749_Q64/Saeid-Amiri-4.jpg)
... In this context, virtual-, augmented-, and mixed-reality (VAM) frameworks can be useful to improve human-robot collaboration, providing a common interface for the human and robot to interact [5]. Such interfaces allow for more intuitive and natural communication, simplifying for the human to give instructions to the robot and for the robot to communicate its intentions [6]. Different frameworks have been used in robotics research for data collection and for automating and facilitating industrial processes [7], [8]. ...
... Initial versions of the morc and morc-itrs algorithms were introduced in two separate conference papers [12,13]. Both papers aimed to enable a robot manipulator to identify object attributes using multiple exploratory behaviors and the produced multimodal sensory data. ...
![[object Object]](https://i1.rgstatic.net/ii/profile.image/277239938732033-1443110560073_Q64/Jivko-Sinapov.jpg)
... Recent classical planning systems designed for robotics frequently use planning domain description language (PDDL) or answer set programming (ASP) as the underlying action language for the planners [18,19,20,21]. For example, researchers have used classical planning algorithms for sequencing actions for a mobile robot working on delivery tasks [22], reasoning about safe and efficient urban driving behaviors for autonomous vehicles [23], and planning actions for a team of mobile robots [24]. Task and motion planning (TAMP) is a hierarchical planning framework that combines classical planning in discrete spaces and robot motion planning in continuous space [25,26]. ...
... It is also important to highlight that research is pushing towards the extension of the answer set semantics with probability theory. The main example is P-log ( Baral et al. 2004) which is based on weighted specifications as Problog, used, e.g., in the LCORPP framework for sequential robot planning (Amiri et al. 2019). In this way, in the next future it may be possible to exploit recent advances in SAT solving strategies in the field of probabilistic logic. ...
![[object Object]](https://i1.rgstatic.net/ii/profile.image/881669447884801-1587217785562_Q64/Mohammad-Shokrolah-Shirazi.jpg)
... In the current literature, there are at least three ways of addressing the open-world planning problem. The first involves acquiring knowledge via human-robot interaction, e.g., dialog-based, to handle situations in an open-world context [13][14][15]. Those methods require human involvement, which might hinder their autonomy capability and limit their applicability in the real world. ...
