Detecting and Removing Visual Distractors for Video Aesthetic Enhancement

Fang-Lue Zhang Victoria University of Wellington Xian Wu Tsinghua University Rui-Long Li Tsinghua University Jue Wang Facc++ Zhao-Heng Zheng University of Michigan Shi-Min Hu Tsinghua University

Machine Learning mathscidoc:1808.41001

IEEE Transactions on Multimedia, 20, (8), 1987 - 1999, 2018.8
Personal videos often contain visual distractors, which are objects that are accidentally captured that can distract viewers from focusing on the main subjects. We propose a method to automatically detect and localize these distractors through learning from a manually labeled dataset. To achieve spatially and temporally coherent detection, we propose extracting features at the Temporal-Superpixel (TSP) level using a traditional SVM-based learning framework. We also experiment with end-to-end learning using Convolutional Neural Networks (CNNs), which achieves slightly higher performance than other methods. The classification result is further refined in a post-processing step based on graph-cut optimization. Experimental results show that our method achieves an accuracy of 81% and a recall of 86%. We demonstrate several ways of removing the detected distractors to improve the video quality, including video hole filling; video frame replacement; and camera path re-planning. The user study results show that our method can significantly improve the aesthetic quality of videos.
video distractor, machine learning, visual quality
[ Download ] [ 2018-08-17 21:26:47 uploaded by shimin ] [ 120 downloads ] [ 0 comments ]
@inproceedings{fang-lue2018detecting,
  title={Detecting and Removing Visual Distractors for Video Aesthetic Enhancement },
  author={Fang-Lue Zhang, Xian Wu, Rui-Long Li, Jue Wang, Zhao-Heng Zheng, and Shi-Min Hu},
  url={http://archive.ymsc.tsinghua.edu.cn/pacm_paperurl/20180817212647221806145},
  booktitle={IEEE Transactions on Multimedia},
  volume={20},
  number={8},
  pages={1987 - 1999},
  year={2018},
}
Fang-Lue Zhang, Xian Wu, Rui-Long Li, Jue Wang, Zhao-Heng Zheng, and Shi-Min Hu. Detecting and Removing Visual Distractors for Video Aesthetic Enhancement . 2018. Vol. 20. In IEEE Transactions on Multimedia. pp.1987 - 1999. http://archive.ymsc.tsinghua.edu.cn/pacm_paperurl/20180817212647221806145.
Please log in for comment!
 
 
Contact us: office-iccm@tsinghua.edu.cn | Copyright Reserved