Detect unknown objects using DINO and SAM to generate object proposals with NIDS-Net
Detect unknown objects using DINO and SAM to generate object proposals with NIDS-Net
Adapting Pre-Trained Vision Models for Novel Instance Detection and Segmentation
arXiv paper abstract https://arxiv.org/abs/2405.17859
arXiv PDF paper https://arxiv.org/pdf/2405.17859
GitHub https://github.com/YoungSean/NIDS-Net
Novel Instance Detection and Segmentation (NIDS) aims at detecting and segmenting novel object instances given a few examples of each instance.
… propose a unified framework (NIDS-Net) comprising object proposal generation, embedding creation for both instance templates and proposal regions, and embedding matching for instance label assignment.
… utilize the Grounding DINO and Segment Anything Model (SAM) to obtain object proposals with accurate bounding boxes and masks.
… generation of … instance embeddings … utilize foreground feature averages of patch embeddings from the DINOv2 ViT backbone, followed by refinement through a weight adapter
… weight adapter can adjust the embeddings locally within their feature space and effectively limit overfitting … enables a straightforward matching strategy
… framework surpasses … state-of-the-art … across four detection datasets … instance segmentation … outperforms the top RGB methods … competitive with the best RGB-D method …
Stay up to date. Subscribe to my posts https://morrislee1234.wixsite.com/website/contact
Web site with my other posts by category https://morrislee1234.wixsite.com/website
LinkedIn https://www.linkedin.com/in/morris-lee-47877b7b
