A new open-source benchmark for robotic manipulation of deformable objects.
Nithya Satheesh
credits: Google AI
Google AI, a division of Google mainly dedicated to Artificial Intelligence, released a new open-source simulated benchmark named ‘DeformableRavens’ with the aim to manipulate deformable objects. The research on this was conducted during an internship of Daniel Seita at Google’s NYC office in summer 2020.
Robotics research community has always spent more time developing robots that can grasp rigid objects only. Less research is devoted to developing algorithms that can grasp deformable objects. In case of deformable object manipulation, one of the biggest challenges is the difficulty to specify the configuration of objects. For example, in case of rigid cube, if the configuration of fixed point relative to its centre is known, it is enough to know how the object is arranged in 3D, but if a single point on a piece of fabric is fixed while other points shifts, it is difficult to describe the state of fabric completely. Additional to that even if the state is descriptive, its dynamics are complex. Thus it is difficult to predict the future states, after some force or action is applied to the deformable object.
DeformableRavens is able to perform 12 tasks which include manipulating cables, bags, fabrics and also a set of model architectures for manipulating deformable objects towards a desired set of configurations. These architectures help a robot to rearrange the cables into the target shape, to move a fabric to the target zone and to insert an item inside a bag.
$ads={1}
Open-Source Benchmark - DeformableRavens
DeformableRavens rearranges the object and does a set of 12 simulated tasks for 1D, 2D and 3D deformable structures. Each of the tasks contains a simulated UR5 arm with a mock gripper for grasping as well as scripted demonstrators that gather data autonomously for imitation learning. To measure generality to various object configurations, tasks randomly change the starting state of the objects in a distribution.
In case of deformable objects, specifying goal configurations for manipulation is a challenging task. Even if their complex dynamics and configuration spaces are given, sometimes the goals cannot be easily specified. Thus in addition to the other tasks, DeformableRavens also has goal-conditioned tasks with specified goal images. In case of goal-conditioned tasks, we must combine a provided starting configuration of objects with a separate image that displays the optimal configuration of those same objects. If the robot is able to get the current configuration somewhat accurately close to the configuration in the goal image, then it is considered a success.
Goal-Conditioned Transporter Networks
The goal conditioned tasks are integrated into Transporter Network architecture to help the tasks in simulated benchmark. Transporter Network architecture is an action-centric model that works on manipulation of a rigid object by rearranging deep features to infer spatial displacements from visual input. It takes both the present environment image and target image with a desired set of configuration of objects, computes deep visual features and then combines the features. They combine the feature using the element wise multiplication to manipulate both rigid and deformable objects. An advantage of Transporter Network is that it retains the spatial structure of the visual images, resulting in inductive biases that reformulate image-based target conditioning into a simpler feature matching problem and increase learning performance with convolutional networks.
For example, if the robot wants to place a green block inside a yellow bag, then it first has to learn the spatial feature that allows it to perform multiple tasks of spreading the opening of the bag and then placing the block in it. After placing the block inside it, the demonstration is considered a success. It should act just as it is shown in the goal image. Like if the block is placed in a blue bag in the goal image, then the demonstrator must have to put the block in the blue bag.
Results
The outcomes of this research suggests that goal-conditioned Transporter Networks allows agents to manipulate deformable structures into any flexibly specified configurations in real time. Using Transporter Networks, deformable objects can be manipulated on tasks with 2D and 3D deformable. This approach appears to be more efficient than already available approaches which rely on using ground-truth pose and vertex position as input.
Future Works
This work has many scopes in future development. Some failures were observed during the testing like when the robot pulls the bag upwards, the object placed inside it falls out. Another failure is when the robot places the object on the exterior surface of the bag and the object falls out. So improvements must be made in the algorithm so that the robot can counteract such failures in real time. Other than this, another area is to train Transporter Network-based models to manipulate deformable objects without expert demonstration techniques.