Abstract by Fanqing(Alex) Lin
Flow Adaptive Video Object Segmentation
We tackle the task of semi-supervised video object segmentation, i.e, pixel-level classification of the images in video sequences using only the ground truth mask for the first frame of its corresponding video. Recently introduced online adaptation of convolutional neural networks for video object segmentation (OnAVOS) has achieved excellent results by pretraining the network, fine-tuning on the first frame and training the network at test time using its approximate prediction as newly obtained ground truth. While achieving impressive performance, OnAVOS uses primitive approximation of its online prediction as ground truth for online updates, which leaves significant potential information wasted. We propose Flow Adaptive Video Object Segmentation(FAVOS) that refines the generated adaptive ground truth for online updates and utilizes temporal consistency between video frames with the help of optical flow. Our experiments show that FAVOS improves state of the art on 2016 DAVIS Challenge from a mean intersection-over-union score of 85.7 to 87.0.