Get better depth from monocular images by using segmentation masks with MaskDepth

Get better depth from monocular images by using segmentation masks with MaskDepth

Layered Depth Refinement with Mask Guidance
arXiv paper abstract https://arxiv.org/abs/2206.03048
arXiv PDF paper https://arxiv.org/pdf/2206.03048.pdf
Project page https://sooyekim.github.io/MaskDepth/

Depth maps … predicted by single image depth estimation (SIDE) models often fail to capture isolated holes in objects and/or have inaccurate boundary regions.

… high-quality masks are much easier to obtain, using commercial auto-masking tools or off-the-shelf methods of segmentation and matting or even by manual editing.

… formulate a novel problem of mask-guided depth refinement that utilizes a generic mask to refine the depth prediction of SIDE models.

… framework performs layered refinement and inpainting/outpainting, decomposing the depth map into two separate layers signified by the mask and the inverse mask.

As datasets with both depth and mask annotations are scarce, … propose a self-supervised learning scheme that uses arbitrary masks and RGB-D datasets.

… method is robust to different types of masks and initial depth predictions, accurately refining depth values in inner and outer mask boundary regions …

Stay up to date. Subscribe to my posts https://morrislee1234.wixsite.com/website/contact
Web site with my other posts by category https://morrislee1234.wixsite.com/website

LinkedIn https://www.linkedin.com/in/morris-lee-47877b7b

Photo by Wolfgang Hasselmann on Unsplash