Visual-Attention-Based Background Modeling for Detecting Infrequently Moving Objects

Lin, Yuewei; Tong, Yan; Cao, Yu; Zhou, Youjie; Wang, Song

doi:10.1109/TCSVT.2016.2527258

Visual-Attention-Based Background Modeling for Detecting Infrequently Moving Objects

Journal Article · Mon Feb 08 04:00:00 EST 2016 · IEEE Transactions on Circuits and Systems for Video Technology

DOI:https://doi.org/10.1109/TCSVT.2016.2527258· OSTI ID:1491679

^[1]; Tong, Yan ^[1]; Cao, Yu ^[2]; Zhou, Youjie ^[1]; Wang, Song ^[1]

Univ. of South Carolina, Columbia, SC (United States). Dept. of Computer Science and Engineering
IBM Almaden Research Center, San Jose, CA (United States); Univ. of South Carolina, Columbia, SC (United States)

Motion is one of the most important cues to separate foreground objects from the background in a video. Using a stationary camera, it is usually assumed that the background is static, while the foreground objects are moving most of the time. However, in practice, the foreground objects may show infrequent motions, such as abandoned objects and sleeping persons. Meanwhile, the background may contain frequent local motions, such as waving trees and/or grass. Such complexities may prevent the existing background subtraction algorithms from correctly identifying the foreground objects. We propose a new approach that can detect the foreground objects with frequent and/or infrequent motions. Specifically, we use a visual-attention mechanism to infer a complete background from a subset of frames and then propagate it to the other frames for accurate background subtraction. Furthermore, we develop a feature-matching-based local motion stabilization algorithm to identify frequent local motions in the background for reducing false positives in the detected foreground. The proposed approach is fully unsupervised, without using any supervised learning for object detection and tracking. Extensive experiments on a large number of videos have demonstrated that the proposed approach outperforms the state-of-the-art motion detection and background subtraction methods in comparison.

View Accepted Manuscript (DOE)

Research Organization:: Univ. of South Carolina, Columbia, SC (United States)

Sponsoring Organization:: National Science Foundation (NSF) (United States); Army Research Laboratory (ARL)

OSTI ID:: 1491679

Report Number(s):: BNL--210902-2019-JAAM

Journal Information:: IEEE Transactions on Circuits and Systems for Video Technology, Journal Name: IEEE Transactions on Circuits and Systems for Video Technology Journal Issue: 6 Vol. 27; ISSN 1051-8215

Publisher:: IEEECopyright Statement

Country of Publication:: United States

Language:: English

References (42)

Adaptive Background Modeling for Paused Object Regions Shimad, Atsushi; Yoshinaga, Satoshi; Taniguchi, Rin-ichiro Computer Vision – ACCV 2010 Workshops https://doi.org/10.1007/978-3-642-22822-3_2	book	January 2011
Change detection by probabilistic segmentation from monocular view Hernandez-Lopez, Francisco J.; Rivera, Mariano Machine Vision and Applications, Vol. 25, Issue 5 https://doi.org/10.1007/s00138-013-0564-3	journal	October 2013
Time-Delayed Correlation Analysis for Multi-Camera Activity Understanding Loy, Chen Change; Xiang, Tao; Gong, Shaogang International Journal of Computer Vision, Vol. 90, Issue 1 https://doi.org/10.1007/s11263-010-0347-5	journal	May 2010
A saliency map in primary visual cortex Li, Zhaoping Trends in Cognitive Sciences, Vol. 6, Issue 1 https://doi.org/10.1016/S1364-6613(00)01817-9	journal	January 2002
A comprehensive review of background subtraction algorithms evaluated with synthetic and real videos Sobral, Andrews; Vacavant, Antoine Computer Vision and Image Understanding, Vol. 122 https://doi.org/10.1016/j.cviu.2013.12.005	journal	May 2014
Distinctive Image Features from Scale-Invariant Keypoints Lowe, David G. International Journal of Computer Vision, Vol. 60, Issue 2, p. 91-110 https://doi.org/10.1023/B:VISI.0000029664.99615.94	journal	November 2004
A model of saliency-based visual attention for rapid scene analysis Itti, L.; Koch, C.; Niebur, E. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 20, Issue 11 https://doi.org/10.1109/34.730558	journal	January 1998
Adaptive background mixture models for real-time tracking Stauffer, C.; Grimson, W. E. L. Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149) https://doi.org/10.1109/CVPR.1999.784637	conference	June 1999
Multi-Layer Background Subtraction Based on Color and Texture Yao, Jian; Odobez, Jean-Marc 2007 IEEE Conference on Computer Vision and Pattern Recognition https://doi.org/10.1109/CVPR.2007.383497	conference	June 2007
Background subtraction in highly dynamic scenes Mahadevan, Vijay; Vasconcelos, Nuno 2008 IEEE Conference on Computer Vision and Pattern Recognition https://doi.org/10.1109/CVPR.2008.4587576	conference	June 2008
Warping background subtraction Ko, Teresa; Soatto, Stefano; Estrin, Deborah 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition https://doi.org/10.1109/CVPR.2010.5539813	conference	June 2010
Modeling pixel process with scale invariant local patterns for background subtraction in complex scenes Liao, Shengcai; Zhao, Guoying; Kellokumpu, Vili 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition https://doi.org/10.1109/CVPR.2010.5539817	conference	June 2010
Evaluation of background subtraction techniques for video surveillance Brutzer, Sebastian; Hoferlin, Benjamin; Heidemann, Gunther CVPR 2011 https://doi.org/10.1109/CVPR.2011.5995508	conference	June 2011
Background modeling using adaptive pixelwise kernel variances in a hybrid feature space Narayana, M.; Hanson, A.; Learned-Miller, E. 2012 IEEE Conference on Computer Vision and Pattern Recognition https://doi.org/10.1109/CVPR.2012.6247916	conference	June 2012
Foreground detection using spatiotemporal projection kernels Moshe, Y.; Hel-Or, H.; Hel-Or, Y. 2012 IEEE Conference on Computer Vision and Pattern Recognition https://doi.org/10.1109/CVPR.2012.6248056	conference	June 2012
Background Modeling Based on Bidirectional Analysis Shimada, Atsushi; Nagahara, Hajime; Taniguchi, Rin-ichiro 2013 IEEE Conference on Computer Vision and Pattern Recognition https://doi.org/10.1109/CVPR.2013.258	conference	June 2013
Changedetection.net: A new change detection benchmark dataset Goyette, Nil; Jodoin, Pierre-Marc; Porikli, Fatih 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops https://doi.org/10.1109/CVPRW.2012.6238919	conference	June 2012
The SOBS algorithm: What are the limits? Maddalena, Lucia; Petrosino, Alfredo 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops https://doi.org/10.1109/CVPRW.2012.6238922	conference	June 2012
Background segmentation with feedback: The Pixel-Based Adaptive Segmenter Hofmann, Martin; Tiefenbacher, Philipp; Rigoll, Gerhard 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops https://doi.org/10.1109/CVPRW.2012.6238925	conference	June 2012
CDnet 2014: An Expanded Change Detection Benchmark Dataset Wang, Yi; Jodoin, Pierre-Marc; Porikli, Fatih 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) https://doi.org/10.1109/CVPRW.2014.126	conference	June 2014
A Fast Self-Tuning Background Subtraction Algorithm Wang, Bin; Dudek, Piotr 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops https://doi.org/10.1109/CVPRW.2014.64	conference	June 2014
Static and Moving Object Detection Using Flux Tensor with Split Gaussian Models Wang, Rui; Bunyak, Filiz; Seetharaman, Guna 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops https://doi.org/10.1109/CVPRW.2014.68	conference	June 2014
Wallflower: principles and practice of background maintenance Toyama, K.; Krumm, J.; Brumitt, B. Proceedings of the Seventh IEEE International Conference on Computer Vision https://doi.org/10.1109/ICCV.1999.791228	conference	January 1999
A Simple and efficient saliency detector for background subtraction Rahtu, Esa; Heikkila, Janne 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops https://doi.org/10.1109/ICCVW.2009.5457577	conference	September 2009
Left-Luggage Detection from Finite-State-Machine Analysis in Static-Camera Videos Lin, Kevin; Chen, Shen-Chi; Chen, Chu-Song 2014 22nd International Conference on Pattern Recognition https://doi.org/10.1109/ICPR.2014.787	conference	August 2014
Mixture of Gaussians-Based Background Subtraction for Bayer-Pattern Image Sequences Suhr, Jae Kyu; Jung, Ho Gi; Li, Gen IEEE Transactions on Circuits and Systems for Video Technology, Vol. 21, Issue 3 https://doi.org/10.1109/TCSVT.2010.2087810	journal	March 2011
Adaptive Object Tracking by Learning Hybrid Template Online No authors listed IEEE Transactions on Circuits and Systems for Video Technology, Vol. 21, Issue 11 https://doi.org/10.1109/TCSVT.2011.2129410	journal	November 2011
Moving Object Detection Revisited: Speed and Robustness Han, Hong; Zhu, Jianfei; Liao, Shengcai IEEE Transactions on Circuits and Systems for Video Technology, Vol. 25, Issue 6 https://doi.org/10.1109/TCSVT.2014.2367371	journal	June 2015
Hierarchical Ensemble of Background Models for PTZ-Based Video Surveillance Liu, Ning; Wu, Hefeng; Lin, Liang IEEE Transactions on Cybernetics, Vol. 45, Issue 1 https://doi.org/10.1109/TCYB.2014.2320493	journal	January 2015
Abandoned Object Detection via Temporal Consistency Modeling and Back-Tracing Verification for Visual Surveillance Lin, Kevin; Chen, Shen-Chi; Chen, Chu-Song IEEE Transactions on Information Forensics and Security, Vol. 10, Issue 7 https://doi.org/10.1109/TIFS.2015.2408263	journal	July 2015
Statistical Modeling of Complex Backgrounds for Foreground Object Detection Li, L.; Huang, W.; Gu, I. Y. -H. IEEE Transactions on Image Processing, Vol. 13, Issue 11 https://doi.org/10.1109/TIP.2004.836169	journal	November 2004
Real-Time Discriminative Background Subtraction No authors listed IEEE Transactions on Image Processing, Vol. 20, Issue 5 https://doi.org/10.1109/TIP.2010.2087764	journal	May 2011
ViBe: A Universal Background Subtraction Algorithm for Video Sequences Barnich, O.; Van Droogenbroeck, M. IEEE Transactions on Image Processing, Vol. 20, Issue 6 https://doi.org/10.1109/TIP.2010.2101613	journal	June 2011
Complex Background Subtraction by Pursuing Dynamic Spatio-Temporal Models No authors listed IEEE Transactions on Image Processing, Vol. 23, Issue 7 https://doi.org/10.1109/TIP.2014.2326776	journal	July 2014
PISA: Pixelwise Image Saliency by Aggregating Complementary Appearance Contrast Measures With Edge-Preserving Coherence Wang, Keze; Lin, Liang; Lu, Jiangbo IEEE Transactions on Image Processing, Vol. 24, Issue 10 https://doi.org/10.1109/TIP.2015.2432712	journal	October 2015
Stopped Object Detection by Learning Foreground Model in Videos Maddalena, L.; Petrosino, A. IEEE Transactions on Neural Networks and Learning Systems, Vol. 24, Issue 5 https://doi.org/10.1109/TNNLS.2013.2242092	journal	May 2013
A performance evaluation of local descriptors Mikolajczyk, K.; Schmid, C. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 27, Issue 10 https://doi.org/10.1109/TPAMI.2005.188	journal	October 2005
Bayesian modeling of dynamic scenes for object detection Sheikh, Y.; Shah, M. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 27, Issue 11 https://doi.org/10.1109/TPAMI.2005.213	journal	November 2005
A Visual-Attention Model Using Earth Mover's Distance-Based Saliency Measurement and Nonlinear Feature Combination Lin, Yuewei; Tang, Yuan Yan; Fang, Bin IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 35, Issue 2 https://doi.org/10.1109/TPAMI.2012.119	journal	February 2013
Background Subtraction with DirichletProcess Mixture Models Haines, Tom S. F.; Xiang, Tao IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 36, Issue 4 https://doi.org/10.1109/TPAMI.2013.239	journal	April 2014
Robust Detection of Abandoned and Removed Objects in Complex Surveillance Videos Tian, YingLi; Feris, Rogerio Schmidt; Liu, Haowei IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), Vol. 41, Issue 5 https://doi.org/10.1109/TSMCC.2010.2065803	journal	September 2011
Recent Advanced Statistical Background Modeling for Foreground Detection - A Systematic Survey Bouwmans, Thierry Recent Patents on Computer Sciencee, Vol. 4, Issue 3 https://doi.org/10.2174/2213275911104030147	journal	September 2011

Cited By (6)

Smart city framework based on intelligent sensor network and visual surveillance Cai, Zhiming; Li, Daming; Deng, Lianbing Concurrency and Computation: Practice and Experience https://doi.org/10.1002/cpe.5301	journal	June 2019
Anomaly detection with a moving camera using multiscale video analysis de Carvalho, Gustavo H. F.; Thomaz, Lucas A.; da Silva, Allan F. Multidimensional Systems and Signal Processing, Vol. 30, Issue 1 https://doi.org/10.1007/s11045-018-0558-4	journal	February 2018
Moving Object Detection for a Moving Camera Based on Global Motion Compensation and Adaptive Background Model Yu, Yang; Kurnianggoro, Laksono; Jo, Kang-Hyun International Journal of Control, Automation and Systems, Vol. 17, Issue 7 https://doi.org/10.1007/s12555-018-0234-3	journal	July 2019
Saliency Subtraction Inspired Automated Event Detection in Underwater Environments Kumar, Nitin; Sardana, H. K.; Shome, S. N. Cognitive Computation, Vol. 12, Issue 1 https://doi.org/10.1007/s12559-019-09671-x	journal	August 2019
Automatic underwater moving object detection using multi-feature integration framework in complex backgrounds Vasamsetti, Srikanth; Setia, Supriya; Mittal, Neerja IET Computer Vision, Vol. 12, Issue 6 https://doi.org/10.1049/iet-cvi.2017.0013	journal	September 2018
Real-Time Detection and Recognition of Multiple Moving Objects for Aerial Surveillance Rahmaniar, Wahyu; Wang, Wen-June; Chen, Hsiang-Chieh Electronics, Vol. 8, Issue 12 https://doi.org/10.3390/electronics8121373	journal	November 2019

Similar Records

Salient Points for Tracking Moving Objects in Video

Conference · Sun Dec 19 23:00:00 EST 2004 · OSTI ID:15011517

Robust Background Subtraction with Foreground Validation for Urban Traffic Video

Journal Article · Wed Jan 14 23:00:00 EST 2004 · Eurasip Journal on Applied Signal Processing · OSTI ID:859918

Detecting and Analyzing Multiple Moving Objects in Crowded Environments with Coherent Motion Regions

Software · Fri Jan 04 00:00:00 EST 2013 · OSTI ID:1231644

Related Subjects

42 ENGINEERING
infrequently moving objects
local motion stabilization
object detection
visual attention

Visual-Attention-Based Background Modeling for Detecting Infrequently Moving Objects

Citation Formats

References (42)

Cited By (6)

Similar Records

Related Subjects