Search Menu
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information
Search Scholarly Publications

Title: Visual-Attention-Based Background Modeling for Detecting Infrequently Moving Objects

Abstract

Motion is one of the most important cues to separate foreground objects from the background in a video. Using a stationary camera, it is usually assumed that the background is static, while the foreground objects are moving most of the time. However, in practice, the foreground objects may show infrequent motions, such as abandoned objects and sleeping persons. Meanwhile, the background may contain frequent local motions, such as waving trees and/or grass. Such complexities may prevent the existing background subtraction algorithms from correctly identifying the foreground objects. We propose a new approach that can detect the foreground objects with frequent and/or infrequent motions. Specifically, we use a visual-attention mechanism to infer a complete background from a subset of frames and then propagate it to the other frames for accurate background subtraction. Furthermore, we develop a feature-matching-based local motion stabilization algorithm to identify frequent local motions in the background for reducing false positives in the detected foreground. The proposed approach is fully unsupervised, without using any supervised learning for object detection and tracking. Extensive experiments on a large number of videos have demonstrated that the proposed approach outperforms the state-of-the-art motion detection and background subtraction methods in comparison.

Authors:
ORCiD logo [1];  [1];  [2];  [1];  [1]
+ Show Author Affiliations
  1. Univ. of South Carolina, Columbia, SC (United States). Dept. of Computer Science and Engineering
  2. IBM Almaden Research Center, San Jose, CA (United States); Univ. of South Carolina, Columbia, SC (United States)
Publication Date:
Research Org.:
Univ. of South Carolina, Columbia, SC (United States)
Sponsoring Org.:
National Science Foundation (NSF); Army Research Laboratory (ARL)
OSTI Identifier:
1491679
Report Number(s):
BNL-210902-2019-JAAM
Journal ID: ISSN 1051-8215
Grant/Contract Number:  
IIS-1017199; IIS-1149787; W911NF-10-2-0060
Resource Type:
Accepted Manuscript
Journal Name:
IEEE Transactions on Circuits and Systems for Video Technology
Additional Journal Information:
Journal Volume: 27; Journal Issue: 6; Journal ID: ISSN 1051-8215
Publisher:
IEEE
Country of Publication:
United States
Language:
English
Subject:
42 ENGINEERING; infrequently moving objects; local motion stabilization; object detection; visual attention

Citation Formats

Lin, Yuewei, Tong, Yan, Cao, Yu, Zhou, Youjie, and Wang, Song. Visual-Attention-Based Background Modeling for Detecting Infrequently Moving Objects. United States: N. p., 2016. Web. doi:10.1109/TCSVT.2016.2527258.
Copy to clipboard
Lin, Yuewei, Tong, Yan, Cao, Yu, Zhou, Youjie, & Wang, Song. Visual-Attention-Based Background Modeling for Detecting Infrequently Moving Objects. United States. https://doi.org/10.1109/TCSVT.2016.2527258
Copy to clipboard
Lin, Yuewei, Tong, Yan, Cao, Yu, Zhou, Youjie, and Wang, Song. Mon . "Visual-Attention-Based Background Modeling for Detecting Infrequently Moving Objects". United States. https://doi.org/10.1109/TCSVT.2016.2527258. https://www.osti.gov/servlets/purl/1491679.
Copy to clipboard
@article{osti_1491679,
title = {Visual-Attention-Based Background Modeling for Detecting Infrequently Moving Objects},
author = {Lin, Yuewei and Tong, Yan and Cao, Yu and Zhou, Youjie and Wang, Song},
abstractNote = {Motion is one of the most important cues to separate foreground objects from the background in a video. Using a stationary camera, it is usually assumed that the background is static, while the foreground objects are moving most of the time. However, in practice, the foreground objects may show infrequent motions, such as abandoned objects and sleeping persons. Meanwhile, the background may contain frequent local motions, such as waving trees and/or grass. Such complexities may prevent the existing background subtraction algorithms from correctly identifying the foreground objects. We propose a new approach that can detect the foreground objects with frequent and/or infrequent motions. Specifically, we use a visual-attention mechanism to infer a complete background from a subset of frames and then propagate it to the other frames for accurate background subtraction. Furthermore, we develop a feature-matching-based local motion stabilization algorithm to identify frequent local motions in the background for reducing false positives in the detected foreground. The proposed approach is fully unsupervised, without using any supervised learning for object detection and tracking. Extensive experiments on a large number of videos have demonstrated that the proposed approach outperforms the state-of-the-art motion detection and background subtraction methods in comparison.},
doi = {10.1109/TCSVT.2016.2527258},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
number = 6,
volume = 27,
place = {United States},
year = {Mon Feb 08 00:00:00 EST 2016},
month = {Mon Feb 08 00:00:00 EST 2016}
}
Copy to clipboard
Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
https://doi.org/10.1109/TCSVT.2016.2527258
Copyright Statement
Other availability
Search WorldCat Search WorldCat to find libraries that may hold this journal
Citation Metrics:
Cited by: 30 works
Citation information provided by
Web of Science
Save / Share:
Export Metadata
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.

Works referenced in this record:

Stopped Object Detection by Learning Foreground Model in Videos
journal, May 2013

  • Maddalena, L.; Petrosino, A.
  • IEEE Transactions on Neural Networks and Learning Systems, Vol. 24, Issue 5
  • DOI: 10.1109/TNNLS.2013.2242092

Abandoned Object Detection via Temporal Consistency Modeling and Back-Tracing Verification for Visual Surveillance
journal, July 2015

  • Lin, Kevin; Chen, Shen-Chi; Chen, Chu-Song
  • IEEE Transactions on Information Forensics and Security, Vol. 10, Issue 7
  • DOI: 10.1109/TIFS.2015.2408263

Moving Object Detection Revisited: Speed and Robustness
journal, June 2015

  • Han, Hong; Zhu, Jianfei; Liao, Shengcai
  • IEEE Transactions on Circuits and Systems for Video Technology, Vol. 25, Issue 6
  • DOI: 10.1109/TCSVT.2014.2367371

Modeling pixel process with scale invariant local patterns for background subtraction in complex scenes
conference, June 2010

  • Liao, Shengcai; Zhao, Guoying; Kellokumpu, Vili
  • 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition
  • DOI: 10.1109/CVPR.2010.5539817

Complex Background Subtraction by Pursuing Dynamic Spatio-Temporal Models
journal, July 2014

Left-Luggage Detection from Finite-State-Machine Analysis in Static-Camera Videos
conference, August 2014

  • Lin, Kevin; Chen, Shen-Chi; Chen, Chu-Song
  • 2014 22nd International Conference on Pattern Recognition
  • DOI: 10.1109/ICPR.2014.787

Multi-Layer Background Subtraction Based on Color and Texture
conference, June 2007

  • Yao, Jian; Odobez, Jean-Marc
  • 2007 IEEE Conference on Computer Vision and Pattern Recognition
  • DOI: 10.1109/CVPR.2007.383497

Hierarchical Ensemble of Background Models for PTZ-Based Video Surveillance
journal, January 2015

Static and Moving Object Detection Using Flux Tensor with Split Gaussian Models
conference, June 2014

  • Wang, Rui; Bunyak, Filiz; Seetharaman, Guna
  • 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops
  • DOI: 10.1109/CVPRW.2014.68

Background Modeling Based on Bidirectional Analysis
conference, June 2013

  • Shimada, Atsushi; Nagahara, Hajime; Taniguchi, Rin-ichiro
  • 2013 IEEE Conference on Computer Vision and Pattern Recognition
  • DOI: 10.1109/CVPR.2013.258

A Fast Self-Tuning Background Subtraction Algorithm
conference, June 2014

  • Wang, Bin; Dudek, Piotr
  • 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops
  • DOI: 10.1109/CVPRW.2014.64

Adaptive Object Tracking by Learning Hybrid Template Online
journal, November 2011

Background Subtraction with DirichletProcess Mixture Models
journal, April 2014

  • Haines, Tom S. F.; Xiang, Tao
  • IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 36, Issue 4
  • DOI: 10.1109/TPAMI.2013.239

Bayesian modeling of dynamic scenes for object detection
journal, November 2005

  • Sheikh, Y.; Shah, M.
  • IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 27, Issue 11
  • DOI: 10.1109/TPAMI.2005.213

Foreground detection using spatiotemporal projection kernels
conference, June 2012

  • Moshe, Y.; Hel-Or, H.; Hel-Or, Y.
  • 2012 IEEE Conference on Computer Vision and Pattern Recognition
  • DOI: 10.1109/CVPR.2012.6248056

Background modeling using adaptive pixelwise kernel variances in a hybrid feature space
conference, June 2012

  • Narayana, M.; Hanson, A.; Learned-Miller, E.
  • 2012 IEEE Conference on Computer Vision and Pattern Recognition
  • DOI: 10.1109/CVPR.2012.6247916

Background segmentation with feedback: The Pixel-Based Adaptive Segmenter
conference, June 2012

  • Hofmann, Martin; Tiefenbacher, Philipp; Rigoll, Gerhard
  • 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops
  • DOI: 10.1109/CVPRW.2012.6238925

Change detection by probabilistic segmentation from monocular view
journal, October 2013

  • Hernandez-Lopez, Francisco J.; Rivera, Mariano
  • Machine Vision and Applications, Vol. 25, Issue 5
  • DOI: 10.1007/s00138-013-0564-3

Time-Delayed Correlation Analysis for Multi-Camera Activity Understanding
journal, May 2010

  • Loy, Chen Change; Xiang, Tao; Gong, Shaogang
  • International Journal of Computer Vision, Vol. 90, Issue 1
  • DOI: 10.1007/s11263-010-0347-5

The SOBS algorithm: What are the limits?
conference, June 2012

  • Maddalena, Lucia; Petrosino, Alfredo
  • 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops
  • DOI: 10.1109/CVPRW.2012.6238922

Warping background subtraction
conference, June 2010

  • Ko, Teresa; Soatto, Stefano; Estrin, Deborah
  • 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition
  • DOI: 10.1109/CVPR.2010.5539813

Mixture of Gaussians-Based Background Subtraction for Bayer-Pattern Image Sequences
journal, March 2011

  • Suhr, Jae Kyu; Jung, Ho Gi; Li, Gen
  • IEEE Transactions on Circuits and Systems for Video Technology, Vol. 21, Issue 3
  • DOI: 10.1109/TCSVT.2010.2087810

ViBe: A Universal Background Subtraction Algorithm for Video Sequences
journal, June 2011

  • Barnich, O.; Van Droogenbroeck, M.
  • IEEE Transactions on Image Processing, Vol. 20, Issue 6
  • DOI: 10.1109/TIP.2010.2101613

Real-Time Discriminative Background Subtraction
journal, May 2011

Evaluation of background subtraction techniques for video surveillance
conference, June 2011

Recent Advanced Statistical Background Modeling for Foreground Detection - A Systematic Survey
journal, September 2011

A comprehensive review of background subtraction algorithms evaluated with synthetic and real videos
journal, May 2014

Robust Detection of Abandoned and Removed Objects in Complex Surveillance Videos
journal, September 2011

  • Tian, YingLi; Feris, Rogerio Schmidt; Liu, Haowei
  • IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), Vol. 41, Issue 5
  • DOI: 10.1109/TSMCC.2010.2065803

Adaptive Background Modeling for Paused Object Regions
book, January 2011

CDnet 2014: An Expanded Change Detection Benchmark Dataset
conference, June 2014

  • Wang, Yi; Jodoin, Pierre-Marc; Porikli, Fatih
  • 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
  • DOI: 10.1109/CVPRW.2014.126

Changedetection.net: A new change detection benchmark dataset
conference, June 2012

  • Goyette, Nil; Jodoin, Pierre-Marc; Porikli, Fatih
  • 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops
  • DOI: 10.1109/CVPRW.2012.6238919

Wallflower: principles and practice of background maintenance
conference, January 1999

  • Toyama, K.; Krumm, J.; Brumitt, B.
  • Proceedings of the Seventh IEEE International Conference on Computer Vision
  • DOI: 10.1109/ICCV.1999.791228

Adaptive background mixture models for real-time tracking
conference, June 1999

  • Stauffer, C.; Grimson, W. E. L.
  • Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149)
  • DOI: 10.1109/CVPR.1999.784637

A performance evaluation of local descriptors
journal, October 2005

  • Mikolajczyk, K.; Schmid, C.
  • IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 27, Issue 10
  • DOI: 10.1109/TPAMI.2005.188

Statistical Modeling of Complex Backgrounds for Foreground Object Detection
journal, November 2004

  • Li, L.; Huang, W.; Gu, I. Y. -H.
  • IEEE Transactions on Image Processing, Vol. 13, Issue 11
  • DOI: 10.1109/TIP.2004.836169

A saliency map in primary visual cortex
journal, January 2002

A Visual-Attention Model Using Earth Mover's Distance-Based Saliency Measurement and Nonlinear Feature Combination
journal, February 2013

  • Lin, Yuewei; Tang, Yuan Yan; Fang, Bin
  • IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 35, Issue 2
  • DOI: 10.1109/TPAMI.2012.119

A model of saliency-based visual attention for rapid scene analysis
journal, January 1998

  • Itti, L.; Koch, C.; Niebur, E.
  • IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 20, Issue 11
  • DOI: 10.1109/34.730558

PISA: Pixelwise Image Saliency by Aggregating Complementary Appearance Contrast Measures With Edge-Preserving Coherence
journal, October 2015

  • Wang, Keze; Lin, Liang; Lu, Jiangbo
  • IEEE Transactions on Image Processing, Vol. 24, Issue 10
  • DOI: 10.1109/TIP.2015.2432712

Background subtraction in highly dynamic scenes
conference, June 2008

  • Mahadevan, Vijay; Vasconcelos, Nuno
  • 2008 IEEE Conference on Computer Vision and Pattern Recognition
  • DOI: 10.1109/CVPR.2008.4587576

A Simple and efficient saliency detector for background subtraction
conference, September 2009

  • Rahtu, Esa; Heikkila, Janne
  • 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops
  • DOI: 10.1109/ICCVW.2009.5457577
    Sort by:
    [ × clear filter / sort ]

    Works referencing / citing this record:

    Smart city framework based on intelligent sensor network and visual surveillance
    journal, June 2019

    • Cai, Zhiming; Li, Daming; Deng, Lianbing
    • Concurrency and Computation: Practice and Experience
    • DOI: 10.1002/cpe.5301

    Real-Time Detection and Recognition of Multiple Moving Objects for Aerial Surveillance
    journal, November 2019

    Automatic underwater moving object detection using multi-feature integration framework in complex backgrounds
    journal, September 2018

    Anomaly detection with a moving camera using multiscale video analysis
    journal, February 2018

    • de Carvalho, Gustavo H. F.; Thomaz, Lucas A.; da Silva, Allan F.
    • Multidimensional Systems and Signal Processing, Vol. 30, Issue 1
    • DOI: 10.1007/s11045-018-0558-4

    Saliency Subtraction Inspired Automated Event Detection in Underwater Environments
    journal, August 2019

    Moving Object Detection for a Moving Camera Based on Global Motion Compensation and Adaptive Background Model
    journal, July 2019

    • Yu, Yang; Kurnianggoro, Laksono; Jo, Kang-Hyun
    • International Journal of Control, Automation and Systems, Vol. 17, Issue 7
    • DOI: 10.1007/s12555-018-0234-3
      Sort by:
      [ × clear filter / sort ]

      Similar Records in DOE PAGES and OSTI.GOV collections: