James Kuffner

Toyota Research Institute, Palo Alto, CA, United States

This chapter serves as an introduction to Part D by giving an overview of motion generation and control strategies in the context of robotic manipulation tasks. Automatic control ranging from the abstract, high-level task specification down to fine-grained feedback at the task interface are considered. Some of the important issues include modeling...

System and methods for rendering three-dimensional (3D) object data models based on a comparison of images. A 3D object data model of an object can be characterized by parameters defining rendering features of the 3D object data model. A comparison can be made of a first rendering of the 3D object data model to one or more reference images related...

Methods and computer program products for generating robot grasp patterns are disclosed. In one embodiment, a method for generating robot grasp patterns includes generating a plurality of approach rays associated with a target object. Each approach ray of the plurality of approach rays extends perpendicularly from a surface of the target object. Th...

Methods and systems are provided for determining and transmitting applicable lighting information, applicable viewing perspective, and a 3D model for an object in response to a search query. An example method includes receiving, at a server, a search query regarding an object. A 3D model for the object is determined. The 3D model includes three-dim...

Examples disclose a method and system to transform a colored point cloud to a 3D textured mesh. The method may be executable to identify a location on a 2D image of an object, identify a location on a 3D image of the object, and determine a color associated with the location on the 2D image. Determining a color may include receiving data associated...

Methods and systems for interacting with multiple three-dimensional (3D) object data models are provided. An example method may involve providing to a display device for display a first 3D object data model and a second 3D object data model. Information associated with a modification to the first 3D object data model may be received. Based on the r...

Methods and systems for superimposing virtual views of 3D objects with live images are provided. An example method may include determining 3D geometry information associated with an environment using one or more sensors of a mobile device, and recognizing 3D objects in the environment based on a database of 3D objects and the 3D geometry informatio...

Robots, computer program products, and methods for trajectory plan optimization are disclosed. In one embodiment, a method of controlling a robot having a first manipulator and a second manipulator includes receiving a trajectory plan including a plurality of sequential motion segments. The method further includes determining a moveable motion segm...

Methods and systems for writing, interpreting, and translating three-dimensional (3D) scenes are provided. An example method may involve accessing data associated with a three-dimensional (3D) scene that includes one or more objects of the 3D scene and one or more rendering effects for the one or more objects. Requests for assets and instructions a...

Methods and systems for determining three-dimensional (3D) object data models of an object based on movement of the object. A processor may receive first scan data and second scan data of an object within in a space. The first scan data may comprise a first position of the object in the space and a first mesh image defining a first view of a shape...

The present application discloses shared robot knowledge bases for use with cloud computing systems. In one embodiment, the cloud computing system collects data from a robot about an object the robot has encountered in its environment, and stores the received data in the shared robot knowledge base. In another embodiment, the cloud computing system...

Methods and systems for material refinement for portions of a three-dimensional (3D) object data model are provided. An example method may include rendering a portion of a 3D object data model, and determining a first appearance metric between an appearance of the portion in the rendered view and a two-dimensional (2D) image. For one or more iterat...

Methods and systems for interacting with multiple three-dimensional (3D) object data models are provided. An example method may involve receiving an annotated template of a first three-dimensional (3D) object data model. The annotated template may be associated with a given category of objects and may include one or more annotations to one or more...

Methods and systems for prioritization of display of portions of three-dimensional (3D) object data models are described. An example method may include receiving information associated with a plurality of renderings of a 3D model. The received information may identify portions of interest of the 3D model which are inspected while the 3D model is re...

Methods and systems for robotic command and operation are provided. In some examples, a robot may be configured to receive a short-form command input that is comprised of an action verb and an object/target, and to analyze contextual/situational data for event outcomes from which the robot can determine an action of a plurality of possible actions...

Rapidly expanding internet resources and wireless networking have potential to liberate robots and automation systems from limited onboard computation, memory, and software. “Cloud Robotics” describes an approach that recognizes the wide availability of networking and incorporates open-source elements to greatly extend earlier concepts of “Online R...

Methods and systems for distributing tasks among robotic devices are described. A server may receive from a first device, information associated with an assigned task of the first device. The server may determine whether the assigned task can be executed more efficiently with assistance from additional devices, and may further determine available d...

Methods and systems for robot cloud computing are described. Within examples, cloud-based computing generally refers to networked computer architectures in which application execution and storage may be divided, to some extent, between client and server devices. A robot may be any device that has a computing ability and interacts with its surroundi...

Methods and systems for merging scanned images of objects using materials and appearance information are described. An example method may include receiving material information and 3D geometry information for surfaces of an object based on a first and second viewpoint. The first viewpoint may differ from the second viewpoint by an amount of motion...

In this paper new grasp quality measures considering both object dynamics and pose uncertainty are proposed. Dynamics of the object is incorporated into our grasping simulation to capture the change of its pose and contact points during grasping. Pose uncertainty is considered by running multiple simulations starting from slightly different initial...

The this special issue of Autonomous Robots aims at exploring the motion safety issue from the decision-making point of view, assuming that the robotic system is working properly and has an accurate understanding of its current situation. The nine papers comprising this special issue address different collision avoidance instances of varying comple...

We present a manipulation planning framework that allows robots to plan in the presence of constraints on end-effector pose, as well as other common constraints. The framework has three main components: constraint representation, constraint-satisfaction strategies, and a general planning algorithm. These components come together to create an effici...

This paper introduces space-filling trees and analyzes them in the context of sampling-based motion planning. Space-filling trees are analogous to space-filling curves, but have a branching, tree-like structure, and are defined by an incremental process that results in a tree for which every point in the space has a finite-length path that converge...

Abstract One of the challenges in developing real-world autonomous,robots is the need for integrating and rigorously test- ing high-level scripting, motion planning, perception, and control algorithms. For this purpose, we introduce an open-source cross-platform software architecture called OpenRAVE, the Open Robotics and Animation Virtual Envi- ro...

This paper presents a probabilistic framework for reasoning about the safety of robot trajectories in dynamic and uncertain environments with imperfect information about the future motion of surrounding objects. For safety assessment, the overall collision probability is used to rank candidate trajectories by considering the probability of collidin...

We present a novel approach to legged locomotion over rough terrain that is thoroughly rooted in optimization. This approach relies on a hierarchy of fast, anytime algorithms to plan a set of footholds, along with the dynamic body motions required to execute them. Components within the planning framework coordinate to exchange plans, cost-to-go est...

The traditional A*-search method builds a search tree of potential solution paths during runtime. An alternative approach is to compute this search tree in advance, and then use it during runtime to efficiently find a solution. Recent work has shown the potential for this idea of precomputation. However, these previous methods do not scale to the m...

Simulation is essential for different robotic research fields such as mobile robotics, motion planning and grasp planning. For grasping in particular, there are no software simulation packages, which provide a holistic environment that can deal with the variety of aspects associated with this problem. These aspects include development and testing o...

We present a novel approach to legged locomotion over rough terrain that is thoroughly rooted in optimization. This approach relies on a hierarchy of fast, anytime algorithms to plan a set of footholds, along with the dynamic body motions required to execute them. Components within the planning framework coordinate to exchange plans, cost-to-go est...

We present a vision-centric manipulation framework for reliably performing reach-and-grasp tasks in everyday environments. By combining grasp planning and visual feedback algorithms, and constantly considering sensor visibility, the framework can recover from sensor calibration errors and unexpected changes in the environment. Although many current...

We present an efficient approach to generating paths for humanoids and other robotic manipulators that uses the Task Space Region (TSR) framework to specify manipulation tasks. TSRs can define acceptable goal poses of an end-effector or constraints on the end-effector's pose during the path, or both. First presented as a method for goal-specificati...

We describe the architecture, algorithms, and experiments with HERB, an autonomous mobile manipulator that performs useful manipulation tasks in the home. We present new algorithms for searching for objects, learning to navigate in cluttered dynamic indoor scenes, recognizing and registering objects accurately in high clutter using vision, manipula...

We present an approach to navigating a biped robot safely and efficiently through a complicated environment of previously unknown obstacles and terrain using only on-board sensing and odometry. Sensing of the environment is performed by a pivoting laser scanner, which continues to update the terrain representation as the robot walks. Safe stepping...

We present an efficient approach to generating paths for a robotic manipulator that are collision-free and guaranteed to meet task specifications despite pose uncertainty. We first describe how to use task space regions (TSRs) to specify grasping and object placement tasks for a manipulator. We then show how to modify a set of TSRs for a certain ta...

We present a method for interactively guiding the navigation of a humanoid robot through complex terrain via an intuitive path-drawing interface. In contrast to full autonomy or direct teleoperation of the robot, the user suggests an overall global navigation route by ¿drawing¿ a path onto the environment while the robot is walking. The path is u...

We present a new method to model and synthesize variation in human motion. Given a small amount of input motion data, we learn a generative model that is able to synthesize new output mo- tion variations that are statistically similar to the input data. The new variations retain the features of the original data examples, but are not exact copies....

In this paper, we present efficient solutions for planning motions of dual-arm manipulation and re-grasping tasks. Motion planning for such tasks on humanoid robots with a high number of degrees of freedom (DoF) requires computationally efficient approaches to determine the robot's full joint configuration at a given grasping position, i.e. solving...

In this paper we study the problem of path planning among movable obstacles, in which a robot is allowed to move the obstacles if they block the robot’s way from a start to a goal position. We make the observation that we can decouple the computations of the robot motions and the obstacle movements, and present a probabilistically complete algorith...

We present an approach to motion planning for humanoid robots that aims to ensure reliable execution by augmenting the planning process to reason about the robot's ability to successfully perceive its environment during operation. By efficiently simulating the robot's perception system during search, our planner utilizes a perceptive capability met...

Abstract— We,present,an,approach,to path,planning,for manipulators,that uses,Workspace,Goal Regions (WGRs) to specify goal end-effector poses. Instead of specifying a discrete set of goals in the manipulator’s configuration space, we specify goals more,intuitively as volumes,in the manipulator’s workspace. We show,that WGRs provide,a common,framewo...

We present the Constrained Bi-directional Rapidly- Exploring Random Tree (CBiRRT) algorithm for planning paths in configuration spaces with multiple constraints. This algo- rithm provides a general framework for handling a variety of constraints in manipulation planning including torque limits, constraints on the pose of an object held by a robot,...

We present an algorithm for planning goal-directed footstep navigation strategies for biped robots through obstacle-filled environ- ments and uneven ground. Planning footsteps is more general than most existing navigation methods designed for wheeled robots, since the op- tions of stepping over or upon obstacles are available. Given a height map of...

We present an approach to navigation planning for humanoid robots that aims to ensure reliable execution by augmenting the planning process to reason about the robotpsilas ability to successfully perceive its environment during operation. By efficiently simulating the robotpsilas perception system during search, our planner generates a metric, the...

Motion planning for robots with many degrees of freedom (DoF) is a generally unsolved problem in the robotics context. In this work an approach for trajectory planning is presented, which takes account of the different kinematic parts of a humanoid robot. Since not all joints of the robot are important for different planning phases, the RRT-based p...

We present a planning algorithm called BiSpace that produces fast plans to complex high-dimensional problems by simultaneously exploring multiple spaces. We specifically focus on finding robust solutions to manipulation and grasp planning problems by using BiSpace's special characteristics to explore the work and configuration spaces of the environ...

We have accelerated a robust model-based 3D tracking system by programmable graphics hardware to run online at frame-rate during operation of a humanoid robot and to efficiently auto-initialize. The tracker recovers the full 6 degree-of-freedom pose of viewable objects relative to the robot. Leveraging the computational resources of the GPU for per...

We present heuristic algorithms for pruning large sets of candidate paths or trajectories down to smaller subsets that maintain desirable characteristics in terms of overall reachability and path length. Consider the example of a set of candidate paths in an environment that is the result of a forward search tree built over a set of actions or beha...

The widespread success of sampling-based planning algorithms stems from their ability to rapidly discover the connectivity of a configuration space. Past research has found that non-uniform sampling in the configuration space can significantly outperform uniform sampling; one important strategy is to bias the sampling distribution based on features...

This paper combines grasp analysis and manipulation planning techniques to perform fast grasp planning in complex scenes. In much previous work on grasping, the object being grasped is assumed to be the only object in the environment. Hence the grasp quality metrics and grasping strategies developed do not perform well when the object is close to o...

Navigation planning for legged robots via foot placement planning has enabled several humanoids to traverse interesting environments autonomously. In this paper we explore methods of adapting foot placement actions to the terrain during the search process, allowing for fuller use of the robot's capabilities, and better resulting paths. We show the...

This chapter serves as an introduction to Part D by giving an overview of motion generation and control strategies in the context of robotic manipulation tasks. Automatic control ranging from the abstract, high-level task specification down to fine-grained feedback at the task interface are considered. Some of the important issues include modeling...

We present a novel motion planning algorithm for performing constrained tasks such as opening doors and drawers by robots such as humanoid robots or mobile manip-ulators. Previous work on constrained manipulation transfers rigid constraints imposed by the target object motion directly into the robot configuration space. This often unnecessarily res...

We present an approach to motion planning for highly articulated systems that aims to ensure robust execution by augmenting the planning process to reason about the robot's ability to successfully perceive its environment during operation. By simulating the robot's perception system during search, our planner generates a metric, the so-called perce...

We have equipped a Honda ASIMO humanoid with the ability to navigate autonomously in obstacle-filled environments. In addition to finding its way through known, fixed obstacle configurations, the planning system can reason about the future state of the world to locomote through challenging environments when the obstacle motions can be inferred from...

This paper describes a path smoothing method that results in paths with clothoidal curvature and which includes a safety margin from obstacles. The method modifies grid based A* results, and tries to push the given path away from obstacles. A technique to take into account the initial direction of robot is also shown.

For humanoid robots to fully realize their biped potential in a three-dimensional world and step over, around or onto obstacles such as stairs, appropriate and efficient approaches to execution, planning and perception are required. To this end, we have accelerated a robust model-based three-dimensional tracking system by programmable graphics hard...

This paper explores the use of statical learning methods on randomized path planning algorithms. A continuous, randomized version of A* is presented along with an empirical analysis showing planning time convergence rates in the robotic manipulation domain. The algorithm relies on several heuristics that capture a manipulator's kinematic feasibilit...

The rapidly-exploring random tree (RRT) algorithm has found widespread use in the field of robot motion planning because it provides a single-shot, probabilistically complete planning method which generalizes well to a variety of problem domains. We present the multipartite RRT (MP-RRT), an RRT variant which supports planning in unknown or dynamic...

We present a novel two-phase approach to motion planning for small fixed-wing Unmanned Aerial Vehicles(UAVs) navigating in complex 3D air slalom environments. A coarse global planner first computes a kinematically feasible obstacle- free path in a discretized 3D workspace which roughly satisfies the kinematic constraints of the UAV. Given a coarse...

This paper presents the ResolveSpatialConstraints (RSC) algorithm for manipulation planning in a domain with movable obstacles. Empirically we show that our algorithm quickly generates plans for simulated articulated robots in a highly nonlinear search space of exponential dimension. RSC is a reverse-time search that samples future robot actions an...

This paper gives an overview of the humanoid robot 'H7', which was developed over several years as an experimental platform for walking, autonomous behaviour and human interaction research at the University of Tokyo. H7 was designed to be a human-sized robot capable of operating autonomously in indoor environments designed for humans. The hardware...

We present an algorithm for computing compact sphere tree hierarchies over general 3D polygonal meshes for the purpose of efficient collision detection and minimum distance computation. The sphere tree construction method we use optimizes the location of the sphere centers and radii at each level of the hierarchy to compactly contain the underlying...

This paper explores autonomous locomotion, reaching, grasping and manipulation for the domain of navigation among movable obstacles (NAMO). The robot perceives and constructs a model of an environment filled with various fixed and movable obstacles, and automatically plans a navigation strategy to reach a desired goal location. The planned strategy...