3D shape, segmentation, appearance, and object poses from images using neural fields with PNF

3D shape, segmentation, appearance, and object poses from images using neural fields with PNF

Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representation
arXiv paper abstract https://arxiv.org/abs/2205.04334v1
arXiv PDF paper https://arxiv.org/pdf/2205.04334v1.pdf

… present Panoptic Neural Fields (PNF), an object-aware neural scene representation that decomposes a scene into a set of objects (things) and background (stuff).

Each object is represented by an oriented 3D bounding box and a multi-layer perceptron (MLP) that takes position, direction, and time and outputs density and radiance.

The background stuff is represented by a similar MLP that additionally outputs semantic labels.

… model builds a panoptic radiance field representation of any scene from just color images.

… use off-the-shelf algorithms to predict camera poses, object tracks, and 2D image semantic segmentations.

… model can be used effectively for several tasks like novel view synthesis, 2D panoptic segmentation, 3D scene editing, and multiview depth prediction.

Stay up to date. Subscribe to my posts https://morrislee1234.wixsite.com/website/contact
Web site with my other posts by category https://morrislee1234.wixsite.com/website

LinkedIn https://www.linkedin.com/in/morris-lee-47877b7b

Photo by Jasper Gronewold on Unsplash

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
AI News Clips by Morris Lee: News to help your R&D

I apply innovative technologies like machine learning, computer vision, and physics to further an organization's goals. Am recognized innovator with 66 patents.