Robot grips new objects in new poses from 10 examples using neural descriptor fields

Robot grips new objects in new poses from 10 examples using neural descriptor fields

Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulation
arXiv paper abstract https://arxiv.org/abs/2112.05124v1
arXiv PDF paper https://arxiv.org/pdf/2112.05124v1.pdf
GitHub https://github.com/anthonysimeonov/ndf_robot
Project page https://yilundu.github.io/ndf
Colab demo https://colab.research.google.com/drive/16bFIFq_E8mnAVwZ_V2qQiKp4x4D0n1sG?usp=sharing
Twitter post https://twitter.com/taiyasaki/status/1469356579203862530

… present Neural Descriptor Fields (NDFs), … encodes both points and relative poses between an object and a target (such as a robot gripper or a rack used for hanging) via category-level descriptors.

… employ this representation for object manipulation, where given a task demonstration, we want to repeat the same task on a new object instance from the same category.

… achieve … by searching (via optimization) for the pose whose descriptor matches that observed in the demonstration.

… trained in a self-supervised fashion via a 3D auto-encoding task that does not rely on expert-labeled keypoints.

… NDFs are SE(3)-equivariant, guaranteeing performance that generalizes across all possible 3D object translations and rotations.

… demonstrate learning of manipulation tasks from few (5–10) demonstrations … on a real robot. … and significantly outperforms a recent baseline that relies on 2D descriptors. …

Stay up to date. Subscribe to my posts https://morrislee1234.wixsite.com/website/contact
Web site with my other posts by category https://morrislee1234.wixsite.com/website

LinkedIn https://www.linkedin.com/in/morris-lee-47877b7b

Photo by Trollinho on Unsplash

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
AI News Clips by Morris Lee: News to help your R&D

I apply innovative technologies like machine learning, computer vision, and physics to further an organization's goals. Am recognized innovator with 66 patents.