Amodal video instance segmentation

Abstract

We explore approaches to improve over existing amodal prediction models for the task of semantic amodal instance level video object segmentation, i.e., the task to delineate objects and their occluded parts in video data. We propose Amodal-Net with three improvements: First, we leverage temporal information. Specifically, we employ 3D convolutions and a flow alignment module which permits to aggregate the objects’ features across frames. Second, we develop a cascaded box-head with soft-non-maximum-suppression to address the challenge that amodal segmentations overlap significantly. Third, we address the challenge that occlusions require observation information to be propagated over larger distances by developing an attention-based mask-head. Then we also study reprojection, another way of using temporal information which also uses 3D information. We evaluate our approach on amodal segmentation for video data, SAILVOS.