WO2011022279A3 - Background model for complex and dynamic scenes - Google Patents

Background model for complex and dynamic scenes Download PDF

Info

Publication number
WO2011022279A3
WO2011022279A3 PCT/US2010/045227 US2010045227W WO2011022279A3 WO 2011022279 A3 WO2011022279 A3 WO 2011022279A3 US 2010045227 W US2010045227 W US 2010045227W WO 2011022279 A3 WO2011022279 A3 WO 2011022279A3
Authority
WO
WIPO (PCT)
Prior art keywords
complex
scene
background model
dynamic scenes
art networks
Prior art date
Application number
PCT/US2010/045227
Other languages
French (fr)
Other versions
WO2011022279A2 (en
Inventor
Wesley Kenneth Cobb
Ming-Jung Seow
Tao Yang
Original Assignee
Behavioral Recognition Systems, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Behavioral Recognition Systems, Inc. filed Critical Behavioral Recognition Systems, Inc.
Priority to EP10810411.8A priority Critical patent/EP2468005A4/en
Priority to BR112012003718A priority patent/BR112012003718A2/en
Publication of WO2011022279A2 publication Critical patent/WO2011022279A2/en
Publication of WO2011022279A3 publication Critical patent/WO2011022279A3/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/254—Analysis of motion involving subtraction of images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G06F18/232—Non-hierarchical techniques
    • G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
    • G06F18/23211—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with adaptive number of clusters
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
    • G06V10/763—Non-hierarchical techniques, e.g. based on statistics of modelling distributions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30232—Surveillance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/44—Event detection

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Data Mining & Analysis (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Probability & Statistics with Applications (AREA)
  • General Health & Medical Sciences (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Biology (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Medical Informatics (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Human Computer Interaction (AREA)
  • Computational Linguistics (AREA)
  • Image Analysis (AREA)

Abstract

Techniques are disclosed for learning and modeling a background for a complex and/or dynamic scene over a period of observations without supervision. A background/foreground component of a computer vision engine may be configured to model a scene using an array of ART networks. The ART networks learn the regularity and periodicity of the scene by observing the scene over a period of time. Thus, the ART networks allow the computer vision engine to model complex and dynamic scene backgrounds in video.
PCT/US2010/045227 2009-08-18 2010-08-11 Background model for complex and dynamic scenes WO2011022279A2 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP10810411.8A EP2468005A4 (en) 2009-08-18 2010-08-11 Background model for complex and dynamic scenes
BR112012003718A BR112012003718A2 (en) 2009-08-18 2010-08-11 background template for complex and dynamic scenes

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US12/543,336 2009-08-18
US12/543,336 US8358834B2 (en) 2009-08-18 2009-08-18 Background model for complex and dynamic scenes

Publications (2)

Publication Number Publication Date
WO2011022279A2 WO2011022279A2 (en) 2011-02-24
WO2011022279A3 true WO2011022279A3 (en) 2011-06-03

Family

ID=43605425

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2010/045227 WO2011022279A2 (en) 2009-08-18 2010-08-11 Background model for complex and dynamic scenes

Country Status (4)

Country Link
US (3) US8358834B2 (en)
EP (1) EP2468005A4 (en)
BR (1) BR112012003718A2 (en)
WO (1) WO2011022279A2 (en)

Families Citing this family (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9633275B2 (en) 2008-09-11 2017-04-25 Wesley Kenneth Cobb Pixel-level based micro-feature extraction
US8358834B2 (en) 2009-08-18 2013-01-22 Behavioral Recognition Systems Background model for complex and dynamic scenes
AU2010241260B2 (en) * 2010-10-29 2013-12-19 Canon Kabushiki Kaisha Foreground background separation in a scene with unstable textures
CN102509119B (en) * 2011-09-30 2014-03-05 北京航空航天大学 Method for processing image scene hierarchy and object occlusion based on classifier
US11470285B2 (en) * 2012-02-07 2022-10-11 Johnson Controls Tyco IP Holdings LLP Method and system for monitoring portal to detect entry and exit
EP2698764A1 (en) * 2012-08-14 2014-02-19 Thomson Licensing Method of sampling colors of images of a video sequence, and application to color clustering
US9741098B2 (en) * 2012-10-12 2017-08-22 Nvidia Corporation System and method for optimizing image quality in a digital camera
US9911361B2 (en) 2013-03-10 2018-03-06 OrCam Technologies, Ltd. Apparatus and method for analyzing images
US9101459B2 (en) 2013-03-15 2015-08-11 OrCam Technologies, Ltd. Apparatus and method for hierarchical object identification using a camera on glasses
FR3005188A1 (en) 2013-04-26 2014-10-31 Univ Paris Curie VISUAL FOLLOW-UP
US10660533B2 (en) * 2014-09-30 2020-05-26 Rapsodo Pte. Ltd. Remote heart rate monitoring based on imaging for moving subjects
US9460522B2 (en) * 2014-10-29 2016-10-04 Behavioral Recognition Systems, Inc. Incremental update for background model thresholds
US9349054B1 (en) 2014-10-29 2016-05-24 Behavioral Recognition Systems, Inc. Foreground detector for video analytics system
US9471844B2 (en) * 2014-10-29 2016-10-18 Behavioral Recognition Systems, Inc. Dynamic absorption window for foreground background detector
AU2014271236A1 (en) * 2014-12-02 2016-06-16 Canon Kabushiki Kaisha Video segmentation method
KR102366521B1 (en) * 2015-01-19 2022-02-23 한화테크윈 주식회사 System and Method for Detecting Moving Objects
US10176365B1 (en) * 2015-04-21 2019-01-08 Educational Testing Service Systems and methods for multi-modal performance scoring using time-series features
US9824455B1 (en) * 2015-12-21 2017-11-21 Amazon Technologies, Inc. Detecting foreground regions in video frames
US9842402B1 (en) * 2015-12-21 2017-12-12 Amazon Technologies, Inc. Detecting foreground regions in panoramic video frames
CN105825195A (en) * 2016-03-25 2016-08-03 广州美易来智能电器有限公司 Intelligent cooking behavior identification device and method
US10679669B2 (en) * 2017-01-18 2020-06-09 Microsoft Technology Licensing, Llc Automatic narration of signal segment
US10438072B2 (en) * 2017-02-27 2019-10-08 Echelon Corporation Video data background tracking and subtraction with multiple layers of stationary foreground and background regions
US10373316B2 (en) * 2017-04-20 2019-08-06 Ford Global Technologies, Llc Images background subtraction for dynamic lighting scenarios
CN107274435B (en) * 2017-05-05 2018-09-04 西安交通大学 The correlation filter updating device of scene classification is considered in a kind of target following
CN107133972A (en) * 2017-05-11 2017-09-05 南宁市正祥科技有限公司 A kind of video moving object detection method
US10536700B1 (en) * 2017-05-12 2020-01-14 Gopro, Inc. Systems and methods for encoding videos based on visuals captured within the videos
CN110309240A (en) * 2018-03-14 2019-10-08 北京京东尚科信息技术有限公司 The method and apparatus for removing dynamic object
EP3543902B1 (en) * 2018-03-22 2021-06-02 Canon Kabushiki Kaisha Image processing apparatus and method and storage medium storing instructions
US11200536B2 (en) * 2018-09-07 2021-12-14 Texas Tech University System Systems and methods for predictively managing collections of items
CN111047624A (en) * 2019-12-27 2020-04-21 成都英飞睿技术有限公司 Image dim target detection method, device, equipment and storage medium
CN113973175A (en) * 2021-08-27 2022-01-25 天津大学 Rapid HDR video reconstruction method

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6678413B1 (en) * 2000-11-24 2004-01-13 Yiqing Liang System and method for object identification and behavior characterization using video analysis
US20070183663A1 (en) * 2006-02-07 2007-08-09 Haohong Wang Intra-mode region-of-interest video object segmentation
US20080095436A1 (en) * 2006-10-18 2008-04-24 Samsung Electronics Co., Ltd. Image analysis method, medium and apparatus and moving image segmentation system

Family Cites Families (55)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4679077A (en) * 1984-11-10 1987-07-07 Matsushita Electric Works, Ltd. Visual Image sensor system
US5113507A (en) * 1988-10-20 1992-05-12 Universities Space Research Association Method and apparatus for a sparse distributed memory system
JP3123587B2 (en) * 1994-03-09 2001-01-15 日本電信電話株式会社 Moving object region extraction method using background subtraction
WO1996029678A1 (en) * 1995-03-22 1996-09-26 Idt International Digital Technologies Deutschland Gmbh Method and apparatus for depth modelling and providing depth information of moving objects
US7076102B2 (en) * 2001-09-27 2006-07-11 Koninklijke Philips Electronics N.V. Video monitoring system employing hierarchical hidden markov model (HMM) event learning and classification
US5969755A (en) * 1996-02-05 1999-10-19 Texas Instruments Incorporated Motion based event detection system and method
US5751378A (en) * 1996-09-27 1998-05-12 General Instrument Corporation Scene change detector for digital video
US5974172A (en) * 1997-02-14 1999-10-26 At&T Corp Method and apparatus for coding segmented regions which may be transparent in video sequences for content-based scalability
US6263088B1 (en) * 1997-06-19 2001-07-17 Ncr Corporation System and method for tracking movement of objects in a scene
US6711278B1 (en) * 1998-09-10 2004-03-23 Microsoft Corporation Tracking semantic objects in vector image sequences
US6570608B1 (en) * 1998-09-30 2003-05-27 Texas Instruments Incorporated System and method for detecting interactions of people and vehicles
WO2000034919A1 (en) * 1998-12-04 2000-06-15 Interval Research Corporation Background estimation and segmentation based on range and color
US6480615B1 (en) * 1999-06-15 2002-11-12 University Of Washington Motion estimation within a sequence of data frames using optical flow with adaptive gradients
US7136525B1 (en) * 1999-09-20 2006-11-14 Microsoft Corporation System and method for background maintenance of an image sequence
US6674877B1 (en) * 2000-02-03 2004-01-06 Microsoft Corporation System and method for visually tracking occluded objects in real time
US6940998B2 (en) * 2000-02-04 2005-09-06 Cernium, Inc. System for automated screening of security cameras
US7868912B2 (en) * 2000-10-24 2011-01-11 Objectvideo, Inc. Video surveillance system employing video primitives
US8457401B2 (en) * 2001-03-23 2013-06-04 Objectvideo, Inc. Video segmentation using statistical pixel modeling
US20030107650A1 (en) * 2001-12-11 2003-06-12 Koninklijke Philips Electronics N.V. Surveillance system with suspicious behavior detection
US20060165386A1 (en) * 2002-01-08 2006-07-27 Cernium, Inc. Object selective video recording
US7436887B2 (en) * 2002-02-06 2008-10-14 Playtex Products, Inc. Method and apparatus for video frame sequence-based object tracking
US6856249B2 (en) * 2002-03-07 2005-02-15 Koninklijke Philips Electronics N.V. System and method of keeping track of normal behavior of the inhabitants of a house
US7260257B2 (en) * 2002-06-19 2007-08-21 Microsoft Corp. System and method for whiteboard and audio capture
US7227893B1 (en) * 2002-08-22 2007-06-05 Xlabs Holdings, Llc Application-specific object-based segmentation and recognition system
US7200266B2 (en) * 2002-08-27 2007-04-03 Princeton University Method and apparatus for automated video activity analysis
US6999600B2 (en) * 2003-01-30 2006-02-14 Objectvideo, Inc. Video scene background maintenance using change detection and classification
US7026979B2 (en) * 2003-07-03 2006-04-11 Hrl Labortories, Llc Method and apparatus for joint kinematic and feature tracking using probabilistic argumentation
US7127083B2 (en) * 2003-11-17 2006-10-24 Vidient Systems, Inc. Video surveillance system with object detection and probability scoring based on object class
WO2005109893A2 (en) * 2004-04-30 2005-11-17 Washington University System and method for detecting anomalies in a video image sequence
US20060018516A1 (en) * 2004-07-22 2006-01-26 Masoud Osama T Monitoring activity using video information
AU2005269310C1 (en) * 2004-07-30 2010-05-20 Euclid Discoveries, Llc Apparatus and method for processing video data
US20060067580A1 (en) 2004-09-01 2006-03-30 Lee C C Consumer electronic device supporting navigation of multimedia content across multiple camera views of a scene
US7620266B2 (en) * 2005-01-20 2009-11-17 International Business Machines Corporation Robust and efficient foreground analysis for real-time video surveillance
US20060190419A1 (en) * 2005-02-22 2006-08-24 Bunn Frank E Video surveillance data analysis algorithms, with local and network-shared communications for facial, physical condition, and intoxication recognition, fuzzy logic intelligent camera system
DE602006017977D1 (en) * 2005-03-17 2010-12-16 British Telecomm TRACKING OBJECTS IN A VIDEO SEQUENCE
EP1866836A2 (en) * 2005-03-30 2007-12-19 Cernium Corporation Intelligent video behavior recognition with multiple masks and configurable logic inference module
US7825954B2 (en) * 2005-05-31 2010-11-02 Objectvideo, Inc. Multi-state target tracking
US7623677B2 (en) * 2005-06-17 2009-11-24 Fuji Xerox Co., Ltd. Methods and interfaces for visualizing activity across video frames in an action keyframe
WO2007110555A1 (en) * 2006-03-28 2007-10-04 The University Court Of The University Of Edinburgh A method for automatically characterizing the behavior of one or more objects
US8467570B2 (en) * 2006-06-14 2013-06-18 Honeywell International Inc. Tracking system with fused motion and object detection
US7916944B2 (en) * 2007-01-31 2011-03-29 Fuji Xerox Co., Ltd. System and method for feature level foreground segmentation
WO2008098188A2 (en) * 2007-02-08 2008-08-14 Behavioral Recognition Systems, Inc. Behavioral recognition system
WO2008103929A2 (en) * 2007-02-23 2008-08-28 Johnson Controls Technology Company Video processing systems and methods
US8086036B2 (en) * 2007-03-26 2011-12-27 International Business Machines Corporation Approach for resolving occlusions, splits and merges in video images
US8411935B2 (en) * 2007-07-11 2013-04-02 Behavioral Recognition Systems, Inc. Semantic representation module of a machine-learning engine in a video analysis system
US8064639B2 (en) * 2007-07-19 2011-11-22 Honeywell International Inc. Multi-pose face tracking using multiple appearance models
US8175333B2 (en) * 2007-09-27 2012-05-08 Behavioral Recognition Systems, Inc. Estimator identifier component for behavioral recognition system
US8300924B2 (en) * 2007-09-27 2012-10-30 Behavioral Recognition Systems, Inc. Tracker component for behavioral recognition system
WO2009049314A2 (en) 2007-10-11 2009-04-16 Trustees Of Boston University Video processing system employing behavior subtraction between reference and observed video image sequences
US8452108B2 (en) * 2008-06-25 2013-05-28 Gannon Technologies Group Llc Systems and methods for image recognition using graph-based pattern matching
US8121968B2 (en) * 2008-09-11 2012-02-21 Behavioral Recognition Systems, Inc. Long-term memory in a video analysis system
US9373055B2 (en) * 2008-12-16 2016-06-21 Behavioral Recognition Systems, Inc. Hierarchical sudden illumination change detection using radiance consistency within a spatial neighborhood
WO2010080121A1 (en) * 2008-12-18 2010-07-15 The Regents Of The University Of California, Santa Cruz Training-free generic object detection in 2-d and 3-d using locally adaptive regression kernels
US8493409B2 (en) * 2009-08-18 2013-07-23 Behavioral Recognition Systems, Inc. Visualizing and updating sequences and segments in a video surveillance system
US8358834B2 (en) 2009-08-18 2013-01-22 Behavioral Recognition Systems Background model for complex and dynamic scenes

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6678413B1 (en) * 2000-11-24 2004-01-13 Yiqing Liang System and method for object identification and behavior characterization using video analysis
US20070183663A1 (en) * 2006-02-07 2007-08-09 Haohong Wang Intra-mode region-of-interest video object segmentation
US20080095436A1 (en) * 2006-10-18 2008-04-24 Samsung Electronics Co., Ltd. Image analysis method, medium and apparatus and moving image segmentation system

Also Published As

Publication number Publication date
US8358834B2 (en) 2013-01-22
BR112012003718A2 (en) 2016-04-05
EP2468005A4 (en) 2017-12-13
US10032282B2 (en) 2018-07-24
US20110044537A1 (en) 2011-02-24
US20130136353A1 (en) 2013-05-30
US9959630B2 (en) 2018-05-01
US20160163065A1 (en) 2016-06-09
WO2011022279A2 (en) 2011-02-24
EP2468005A2 (en) 2012-06-27

Similar Documents

Publication Publication Date Title
WO2011022279A3 (en) Background model for complex and dynamic scenes
WO2010138582A3 (en) Real time retargeting of skeletal data to game avatar
WO2008112806A3 (en) System and method for processing video images using point clouds
WO2016051366A3 (en) Switching between the real world and virtual reality
WO2010096279A3 (en) Method and system for gesture recognition
WO2012092451A3 (en) Searching recorded video
WO2012092429A3 (en) Searching recorded video
WO2012004682A3 (en) Scene background blurring including range measurement
WO2007120558A3 (en) Image classification based on a mixture of elliptical color models
WO2015031138A3 (en) Detecting trends from images uploaded to a social network
WO2014182638A3 (en) Mobile security technology
WO2011150126A3 (en) Deghosting seismic data
WO2012174174A3 (en) System and method for user preference augmentation through social network inner-circle knowledge discovery
WO2012031026A3 (en) High dynamic range image sensor
WO2011044257A3 (en) Human tracking system
WO2011045768A3 (en) Animation of photo-images via fitting of combined models
WO2011149648A3 (en) Determining key video snippets using selection criteria to form a video summary
WO2008097552A3 (en) System and method for cell analysis in microscopy
WO2015119838A3 (en) Controlling a robot in the presence of a moving object
WO2010053507A3 (en) Modifying color and panchromatic channel cfa image
WO2014004482A3 (en) Providing streams of filtered photographs for user consumption
WO2011028379A3 (en) Foreground object tracking
WO2013003534A3 (en) Measuring device user experience through display outputs
WO2008094499A3 (en) Wafer level camera module and method of manufacture
WO2012033658A3 (en) A 3-d camera

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 10810411

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 2010810411

Country of ref document: EP