Robot grips new objects in new poses from 10 examples using neural descriptor fields

Robot grips new objects in new poses from 10 examples using neural descriptor fields

Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulation
arXiv paper abstract https://arxiv.org/abs/2112.05124v1
arXiv PDF paper https://arxiv.org/pdf/2112.05124v1.pdf
GitHub https://github.com/anthonysimeonov/ndf_robot
Project page https://yilundu.github.io/ndf
Colab demo https://colab.research.google.com/drive/16bFIFq_E8mnAVwZ_V2qQiKp4x4D0n1sG?usp=sharing
Twitter post https://twitter.com/taiyasaki/status/1469356579203862530

… present Neural Descriptor Fields (NDFs), … encodes both points and relative poses between an object and a target (such as a robot gripper or a rack used for hanging) via category-level descriptors.

… employ this representation for object manipulation, where given a task demonstration, we want to repeat the same task on a new object instance from the same category.

… achieve … by searching (via optimization) for the pose whose descriptor matches that observed in the demonstration.

… trained in a self-supervised fashion via a 3D auto-encoding task that does not rely on expert-labeled keypoints.

… NDFs are SE(3)-equivariant, guaranteeing performance that generalizes across all possible 3D object translations and rotations.

… demonstrate learning of manipulation tasks from few (5–10) demonstrations … on a real robot. … and significantly outperforms a recent baseline that relies on 2D descriptors. …

Stay up to date. Subscribe to my posts https://morrislee1234.wixsite.com/website/contact
Web site with my other posts by category https://morrislee1234.wixsite.com/website

LinkedIn https://www.linkedin.com/in/morris-lee-47877b7b

Photo by Trollinho on Unsplash