Weiwei Wan

Osaka University | Handai · Graduate School of Engineering Science

This study explores a pick-and-toss (PT) as an alternative to pick-and-place (PP), allowing a robot to extend its range and improve task efficiency. Although PT boosts efficiency in object arrangement, the placement environment critically affects the success of tossing. To achieve accurate and efficient object arrangement, we suggest choosing betwe...

This study presents the design of a miniature Vision-based Tactile Sensor (VTS) for compact integration with robotic grippers. VTSs have become a significant research area in the past decade known for their high-resolution capabilities. However, a common challenge faced by the VTS is their large size, which significantly limits the application scen...

Inspired by traditional handmade crafts, where a person improvises assemblies based on the available objects, we formally introduce the Craft Assembly Task. It is a robotic assembly task that involves building an accurate representation of a given target object using the available objects, which do not directly correspond to its parts. In this work...

In human–robot collaboration (HRC), having a clear understanding between the human and robot is essential for maintaining safety and productivity throughout the task. A robot not only needs to be aware of the task sequences required to finish a task but also needs to be able to predict the future actions of the human. Predicting the human motions c...

Inspired by traditional handmade crafts, where a person improvises assemblies based on the available objects, we formally introduce the Craft Assembly Task. It is a robotic assembly task that involves building an accurate representation of a given target object using the available objects, which do not directly correspond to its parts. In this work...

In this paper, we present a novel approach for planning an object's Next Best Views (NBV) so that a depth camera can collect the object's surface point cloud and reconstruct its 3D model with a small number of consequent views. Our focus is especially on thin and curved metal plates, and we use a robot manipulator and an externally installed statio...

This paper presented a regrasp planning method to eliminate grasp uncertainty while considering the geometric constraints of a fixture. The method automatically finds the Stable Placement Poses (SPPs) of an object on a Triangular Corner Fixture (TCF), elevates the object from its SPPs to dropping poses and finds the Deterministic Dropping Poses (DD...

In this paper, we propose an autonomous, effective and general approach for picking up tangled-prone objects for industrial bin picking. First, we learn PickNet - a network that maps the visual observation to pixel-wise possibilities of picking isolated objects or separating tangled objects and infers the corresponding grasp. Then, we propose two e...

This chapter introduces the task of robotic bin picking in manufacturing, the development of the required techniques, the motivation for handling entangled objects in bin picking, a brief overview of the proposed methods and the organization of this book.

In this paper, we deploy a dual-arm robot that can grasp, extract and disentangle wire harnesses from dense clutter using dynamic manipulation. The robot can swing to dynamically discard the entangled objects and regrasp to adjust the undesirable grasp pose. To improve the robustness and accuracy of the system, we leverage a closed-loop framework t...

In this paper, we propose a novel bin-picking system that can grasp avoiding entanglement for both rigid and flexible objects. Our method can (1) infer the full state of the objects in the clutter by skeletonizing and restoring the occluded shapes and (2) estimate the degree of entanglement quantitatively for each restored object and determine the...

In this research, we tackle the challenge of picking only one object from a randomly stacked pile where the objects can potentially be entangled. No solution has been proposed to solve this challenge due to the complexity of picking one and only one object from the bin of entangled objects. Therefore, we propose a method for avoiding the situation...

Wire harnesses are essential connecting components in the manufacturing industry but are challenging to automate in industrial tasks such as bin picking. They are long, flexible and tend to get entangled when randomly placed in a bin. This makes it difficult for the robot to grasp a single one in dense clutter. Besides, training or collecting data...

This paper addresses the problem of picking up only one object at a time avoiding any entanglement in bin-picking. To cope with a difficult case where the complex-shaped objects are heavily entangled together, we propose a topology-based method that can generate non-tangle grasp positions on a single depth image. The core technique is the entanglem...

This paper proposed a task allocation method for human-robot collaborative assembly from a graphical instruction manual based on the assembly complexity while considering the stability of each sub-assembly task. Without using the CAD model of each part to construct the liaison graph, we generate the liaison graph of a product from the 2D graphical...

Synergy hands absorb the complexity of control and mechanisms inherent in multi-fingered hands, enabling human-like grasping with simple mechanisms and low-dimensional control. However, they struggle with functional object grasping and dexterous manipulation expected of multi-fingered hands. In this study, we propose a synergy hand using a fluid ne...

This special issue will encompass state-of-the-art research on emerging topics related to development and learning in natural and artificial systems. The primary focus of this special issue is to explore the facets of development and learning from a multidisciplinary perspective by convening researchers from the fields of computer science, robotics...

Home service robots operating indoors, such as inside houses and offices, require the real-time and accurate identification and location of target objects to perform service tasks efficiently. However, images captured by visual sensors while in motion states usually contain varying degrees of blurriness, presenting a significant challenge for objec...

Since fresh cherry tomatoes have a high likelihood of spoilage during post-harvest transport and storage, assessing the fruit grade based on its firmness or predicting its potential storage time becomes necessary. The study proposes a method for non-destructive fruit firmness detection using a photometric stereo vision-based tactile sensor. The tac...

Diversified manufacturing requires a robot system to be highly generalizable to different novel objects. Previous studies try to achieve this goal by using learning-based methods but are high-cost and lack of generalization due to insufficient knowledge in training. In this paper, we propose a new approach of novel object grasping using an object o...

Accurate robotic manipulation of test tubes in biology and medical industries is becoming increasingly important to address workforce shortages and improve worker safety. The detection and localization of test tubes are essential for the robots to successfully manipulate test tubes. In this paper, we present a framework to detect and estimate poses...

Industrial bin picking for tangled-prone objects requires the robot to either pick up untangled objects or perform separation manipulation when the bin contains no isolated objects. The robot must be able to flexibly perform appropriate actions based on the current observation. It is challenging due to high occlusion in the clutter, elusive entangl...

To extend the paralyzed fingers of a stroke victim with hemiplegia, we have proposed a low-dimensional control system for a robotic finger which can grasp an object between the robotic finger and the paralyzed human fingers. In this study, we newly propose an extra finger that can face multiple fingers with low-dimensional control. This can be real...

Purpose This paper aims to study using a mobile manipulator with a collaborative robotic arm component to manipulate objects beyond the robot’s maximum payload. Design/methodology/approach This paper proposes a single-short probabilistic roadmap-based method to plan and optimize manipulation motion with environment support. The method uses an expa...

When human beings see different objects, they can quickly make correct grasping strategies through brain decisions. However, grasp, as the first step of most manipulation tasks, is still an open issue in robotics. Although many detection methods have been proposed to take RGB-D images or point clouds as input and output grasp candidates, these meth...

p>To realize a dense object placement into a container, we propose a robotic packing motion planner by pushing objects to the side of other objects. Our method comprises three planning strategies, i.e., object placement planning, robotic packing-action planning, and action sequence planning. Object placement planning generates objects’ placement in...

p>To realize a dense object placement into a container, we propose a robotic packing motion planner by pushing objects to the side of other objects. Our method comprises three planning strategies, i.e., object placement planning, robotic packing-action planning, and action sequence planning. Object placement planning generates objects’ placement in...

This paper addresses the challenge of industrial bin picking using entangled wire harnesses. Wire harnesses are essential in manufacturing but poses challenges in automation due to their complex geometries and propensity for entanglement. Our previous work tackled this issue by proposing a quasi-static pulling motion to separate the entangled wire...

To safely and efficiently extract an object from the clutter, this paper presents a bimanual manipulation planner in which one hand of the robot is used to slide the target object out of the clutter while the other hand is used to support the surrounding objects to prevent the clutter from collapsing. Our method uses a neural network to predict the...

Aiming at solving the problems of low compliance and unsatisfactory therapeutic effect in most existing waist rehabilitation robots, this letter presents a compact waist rehabilitation robot (CWRR) for patients with low back pain (LBP). First, the CWRR with a rigid pose adjustment mechanism and a soft curvature adjustment mechanism is designed to a...

This paper presents a stiffness-changeable soft finger using chain mail jamming. This finger can achieve adaptive grasping and in-hand manipulation by reshaping and exerting changeable gripping force. The jamming phenomenon happens when particles in a chamber get interlocked where confining pressure is exerted at their boundaries, which is widely u...

This paper presents a remotely operated robotic system that includes two mobile manipulators to extend the functional capabilities of a human body. Compared with previous tele-operation or robotic body extension systems, using two mobile manipulators helps with enlarging the workspace and allowing manipulation of large or long objects. The system c...

This paper uses a mobile manipulator with a collaborative robotic arm to manipulate objects beyond the robot's maximum payload. It proposes a single-shot probabilistic roadmap-based method to plan and optimize manipulation motion with environment support. The method uses an expanded object mesh model to examine contact and randomly explores object...

Industrial bin picking for tangled-prone objects requires the robot to either pick up untangled objects or perform separation manipulation when the bin contains no isolated objects. The robot must be able to flexibly perform appropriate actions based on the current observation. It is challenging due to high occlusion in the clutter, elusive entangl...

Deep learning methods have recently exhibited impressive performance in object detection. However, such methods needed much training data to achieve high recognition accuracy, which was time-consuming and required considerable manual work like labeling images. In this paper, we automatically prepare training data using robots. Considering the low e...

This paper proposes a method to estimate the 6D pose of an object grasped by a robot hand using RGB cameras mounted on the palm and visuotactile sensors installed at the fingertips. It can deal with objects made from a wide range of materials thanks to combining the two types of sensors. The method allows a robot to robot to perform in-hand pose es...

To automate operations in a logistic warehouse, a robot needs to extract items from the clutter on a shelf without collapsing the clutter. To address this problem, this study proposes a multi-step motion planner to stably extract an item by using the support relations of each object included in the clutter. This study primarily focuses on safe extr...

Elasto-plastic metal wire curving task is commonly seen in manufacturing and medical fields. This paper presents a combined task and motion planner (TAMP) for a robot arm to work aside a bending machine and carry out 3D metal wire curving tasks. We assume a collaborative robot that is safe for humans but has a weak payload and develop the combined...

Deep learning methods have recently exhibited impressive performance in object detection. However, such methods needed much training data to achieve high recognition accuracy, which was time-consuming and required considerable manual work like labeling images. In this paper, we automatically prepare training data using robots. Considering the low e...

To enhance the accuracy of robotic assembly planning by understanding the graphical instruction manual, this paper proposes a novel two-step error correction method. While constructing the Assembly Task Sequence Graph (ATSG) from the instruction manual, we performed an error correction focusing on the component, symbol, speech bubble, and model num...

This paper proposes to use a high-compliance soft wrist to improve the performance of robotic peg-in-hole in uncertain environments. In contrast to past research in this field, in which force control with relatively low compliance has been used, we propose a method that searching and aligning motions can be easily realized by taking advantage of hi...

Soft multi-fingered robotic hands are safe in human environments and can perform human-like behaviors. However, structural complexities and nonlinearities in soft actuators complicate torque sensing, a critical function for dexterous object manipulation. This study introduces a torque-sensing finger joint mechanism using a soft actuator composed of...

This paper presents a system integration approach for a 6-DoF (Degree of Freedom) collaborative robot to operate a pipette for liquid dispensing. Its technical development is three-fold. First, we designed an end-effector for holding and triggering manual pipettes. Second, we took advantage of direct teaching to specify global labware poses and pla...

This study discusses a framework for the autonomous calibration of pipetting in a human-robot collaborative environment. Supposing that the collaborative robot works in a typical biology laboratory, in which there are unconnected devices and instruments, we developed adaptation approaches to fit the robot system for the environment. The end-effecto...

To automate operations in a logistic warehouse, a robot needs to extract items from the clutter on a shelf without collapsing the clutter. This study proposes a multi-step motion planner to address this problem by using the support relations of objects within the clutter. The focus of this study is on safe extraction, where the robot can select the...

We propose a method for fast estimation of reflection parameters of an object surface using only our optical reflection proximity sensor. The sensor values are the distance to a object surface, the tilt angle, and the reflected light intensity value, and the reflection parameters are searched by comparing them with a ray-tracing simulator values. S...

In this study, we propose a high-speed grasping strategy for picking up a single sheet object placed from multiple sheets. The robot hand is teleoperated by a data glove to generate the fingertip trajectory for the pickup motion. At this point, it is difficult to grasp a single sheet with only the fingertip trajectory, and it is also difficult to o...

We propose a method for direct teaching of a robotic gripper position with zero contact force using proximity sensor feedback. The proximity sensor mounted on the gripper detects a marker position without contact, and the robot track it to generate a continuous trajectory. This system enables us to teach the gripper position in three dimensional po...

To shorten the work time (cycle time) of the conventional pick-and-place operation by robots, we propose a method for a robot to achieve pick-and-toss operation through learning. In this paper, for simplicity, the optimal hand opening width and arm joint velocity are obtained by learning for a tossing motion with a reduced amount of rotation to the...

In the conventional, Simultaneous Localization and Mapping (SLAM), it is difficult to calculate the camera movement using the image features in the environment with less features. To cope with this problem, we address this research proposes a method coordinating SLAM and AR marker recognition in a featureless space. We assume a processing/inspectio...

A wide range of technological developments are inspired by biological individuals and advanced synthetic materials, cognitive sensors, control algorithms, artificial intelligence technology, and intelligent systems. One of the major challenges is to create a comprehensive study by integrating different techniques into robotic systems so that the pe...

High-mix, low-volume assembly has been a long-standing challenge for robot systems. We present a complete 2-armed robot system with general-purpose grippers and hand-held tools, which can perform assembly for a wide variety of objects with tight tolerances. The complete source code and 3D-printed-part designs are available for download and can be e...

A vision-based tactile sensor (VBTS) is an innovative optical sensor widely applied in robotic perception. The VBTS consists of a contact module, an illumination module, and a camera module. Its hardware technologies contain preparation, performance, functions, materials, and optimization design of components. However, the current literature lacks...

It is difficult to effectively operate a dual-arm robot using only the information written in a cooking recipe. To cope with this problem, this paper proposes a graph-based approach on bimanual cooking motion planning from a cooking recipe. In our approach, we first decompose the cooking recipe into graph elements. Then, we try to connect the graph...

p>To be presented at IROS 2022. </p

p>To be presented at IROS 2022. </p

It is difficult to effectively operate a dual-arm robot using only the information written in a cooking recipe. To cope with this problem, this paper proposes a graph based approach on bimanual cooking motion planning from a cooking recipe. In our approach, we first decompose the cooking recipe into graph elements. Then, we try to connect the graph...

The future robotics will be more and more closer to human in many applications such as in a future manufacturing, human rehabilitation, sport training and physical assistance, or medical and robotized surgery. Also, a domain of object co-manipulation jointly by human and robot is a very interesting and promising way of research. All these lead nece...

Picking a thin object placed on a flat surface is challenging as contact regions are limited. Scooping grasp is a commonly used method for handling thin objects. However, the sharp fingertips are required to achieve the successful scooping motion, which are dangerous. This paper proposes a novel grasping strategy, prying grasp, to pick thin objects...

Object manipulation automation in logistic warehouses has recently been actively researched. However, shelf replenishment is a challenge that requires the precise and careful handling of densely piled objects. The irregular arrangement of objects on a shelf makes this task particularly difficult. This paper presents an approach for generating a saf...

Synergy Hand is a synergy-based multi-fingered hand that can express various postures with limited control inputs. Conventional synergies compressing the dimension of hand's motion by using the external characteristics of various grasping postures ruin the hand's dexterity. This is because functional information that each finger should perform is l...

Disc cutters are representative cutting tools in tunnel boring machines (TBMs). They are installed at the cutter head of the TBM to disintegrate rocks during tunnel boring. Disc cutters require regular replacement to avoid wear and malfunctions. Currently, most replacement work is performed manually, with intensive labor requirements and hazardous...

This paper presents a system integration approach for a 6-DoF (Degree of Freedom) collaborative robot to operate a pipette for liquid dispensing. Its technical development is threefold. First, we designed an end-effector for holding and triggering manual pipettes. Second, we took advantage of a collaborative robot to recognize labware poses and pla...

Pivoting gait is an efficient way for robots to manipulate a heavy object. Although we can cope with the contact constraints of the pivoting gait by using the model predictive control (MPC), systems with complex dynamics, including the pivoting gait, usually require long horizons in the MPC and it leads to a heavy computational load. To overcome th...

Robotic pick-and-place has been researched for a long time to cope with uncertainty of novel objects and changeable environments. Past works mainly focus on learning-based methods to achieve high precision. However, they have difficulty being generalized for the limitation of specified training models. To break through this drawback of learning-bas...

A vacuum lifter is widely used to hold and pick up large, heavy, and flat objects. Conventionally, when using a vacuum lifter, a human worker watches the state of a running vacuum lifter and adjusts the object's pose to maintain balance. In this work, we propose using a dual-arm robot to replace the human workers and develop planning and control me...