Separate one image into moving and non-moving parts then do segmentation or editing with see3d
Separate one image into moving and non-moving parts then do segmentation or editing with see3d
Seeing 3D Objects in a Single Image via Self-Supervised Static-Dynamic Disentanglement
arXiv paper abstract https://arxiv.org/abs/2207.11232
arXiv PDF paper https://arxiv.org/pdf/2207.11232.pdf
Twitter video https://twitter.com/_akhaliq/status/1551414918724952065
Project page https://prafullsharma.net/see3d
Human perception reliably identifies movable and immovable parts of 3D scenes, and completes the 3D structure of objects and background from incomplete observations.
… propose an approach that observes unlabeled multi-view videos at training time and learns to map a single image observation of a complex scene, such as a street with cars, to a 3D neural scene representation that is disentangled into movable and immovable parts while plausibly completing its 3D structure.
… separately parameterize movable and immovable scene parts via 2D neural ground plans.
These ground plans are 2D grids of features aligned with the ground plane that can be locally decoded into 3D neural radiance fields.
… model is trained self-supervised via neural rendering.
… demonstrate … enables … extraction of object-centric 3D representations, novel view synthesis, instance segmentation, and 3D bounding box prediction, … enables scene editing via object manipulation such as deletion, insertion, and rigid-body motion.
Stay up to date. Subscribe to my posts https://morrislee1234.wixsite.com/website/contact
Web site with my other posts by category https://morrislee1234.wixsite.com/website
LinkedIn https://www.linkedin.com/in/morris-lee-47877b7b
