Segment untrained objects from text descriptions using CLIP-based image embeddings with OpenMask3D
Segment untrained objects from text descriptions using CLIP-based image embeddings with OpenMask3D
OpenMask3D: Open-Vocabulary 3D Instance Segmentation
arXiv paper abstract https://arxiv.org/abs/2306.13631
arXiv PDF paper https://arxiv.org/pdf/2306.13631.pdf
Project page https://openmask3d.github.io
… introduce the task of open-vocabulary 3D instance segmentation.
… propose OpenMask3D, which is a zero-shot approach for open-vocabulary 3D instance segmentation.
Guided by predicted class-agnostic 3D instance masks, … model aggregates per-mask features via multi-view fusion of CLIP-based image embeddings.
… conduct experiments … on the ScanNet200 dataset to evaluate the performance of OpenMask3D, and provide insights about the open-vocabulary 3D instance segmentation task.
… approach outperforms other open-vocabulary counterparts, particularly on the long-tail distribution.
… OpenMask3D goes beyond the limitations of close-vocabulary approaches, and enables the segmentation of object instances based on free-form queries describing object properties such as semantics, geometry, affordances, and material properties.
Stay up to date. Subscribe to my posts https://morrislee1234.wixsite.com/website/contact
Web site with my other posts by category https://morrislee1234.wixsite.com/website
LinkedIn https://www.linkedin.com/in/morris-lee-47877b7b
