Segment scene with moving objects and camera by get motion of camera then of object with MCDS-VSS
Segment scene with moving objects and camera by get motion of camera then of object with MCDS-VSS
MCDS-VSS: Moving Camera Dynamic Scene Video Semantic Segmentation by Filtering with Self-Supervised Geometry and Motion
arXiv paper abstract https://arxiv.org/abs/2405.19921
arXiv PDF paper https://arxiv.org/pdf/2405.19921
… in video semantic segmentation, existing approaches ignore important inductive biases and lack structured and interpretable internal representations.
… propose MCDS-VSS, a structured filter model that learns in a self-supervised manner to estimate scene geometry and ego-motion of the camera, while also estimating the motion of external objects.
… model leverages these representations to improve the temporal consistency of semantic segmentation without sacrificing segmentation accuracy.
MCDS-VSS follows a prediction-fusion approach in which scene geometry and camera motion are first used to compensate for ego-motion, then residual flow is used to compensate motion of dynamic objects, and finally the predicted scene features are fused with the current features to obtain a temporally consistent scene segmentation.
… model parses automotive scenes into multiple decoupled interpretable representations such as scene geometry, ego-motion, and object motion.
… MCDS-VSS achieves superior temporal consistency on video sequences while retaining competitive segmentation performance.
Stay up to date. Subscribe to my posts https://morrislee1234.wixsite.com/website/contact
Web site with my other posts by category https://morrislee1234.wixsite.com/website
LinkedIn https://www.linkedin.com/in/morris-lee-47877b7b
