Scene segmentation without labeling by combining images and LiDAR with Drive&Segment
Scene segmentation without labeling by combining images and LiDAR with Drive&Segment
Drive&Segment: Unsupervised Semantic Segmentation of Urban Scenes via Cross-modal Distillation
arXiv paper abstract https://arxiv.org/abs/2203.11160
arXiv PDF paper https://arxiv.org/pdf/2203.11160.pdf
GitHub https://github.com/vobecant/DriveAndSegment
Project page https://vobecant.github.io/DriveAndSegment/
Demo https://huggingface.co/spaces/vobecant/DaS
… investigates learning pixel-wise semantic image segmentation in urban scenes without any manual annotation, just from the raw non-curated data collected by cars which, equipped with cameras and LiDAR sensors, drive around a city.
… First, … propose … cross-modal unsupervised learning of semantic image segmentation by leveraging synchronized LiDAR and image data.
… method is the use of an object proposal module that analyzes the LiDAR point cloud to obtain proposals for spatially consistent objects.
Second, … show that these 3D object proposals can be aligned with the input images and reliably clustered into semantically meaningful pseudo-classes.
Finally, … develop a cross-modal distillation approach that leverages image data partially annotated with the resulting pseudo-classes to train a transformer-based model for image semantic segmentation.
… without any finetuning, and demonstrate significant improvements compared to the current state of the art …
Stay up to date. Subscribe to my posts https://morrislee1234.wixsite.com/website/contact
Web site with my other posts by category https://morrislee1234.wixsite.com/website
LinkedIn https://www.linkedin.com/in/morris-lee-47877b7b
