WO2012074352A1 - Système et procédé pour détecter un événement d'arrêt anormal dans une région - Google Patents

Système et procédé pour détecter un événement d'arrêt anormal dans une région Download PDF

Info

Publication number
WO2012074352A1
WO2012074352A1 PCT/MY2011/000078 MY2011000078W WO2012074352A1 WO 2012074352 A1 WO2012074352 A1 WO 2012074352A1 MY 2011000078 W MY2011000078 W MY 2011000078W WO 2012074352 A1 WO2012074352 A1 WO 2012074352A1
Authority
WO
WIPO (PCT)
Prior art keywords
region
detect
subject
loitering
accordance
Prior art date
Application number
PCT/MY2011/000078
Other languages
English (en)
Inventor
Mei Kuan Lim
Kim Meng Liang
Sze Ling Tang
Weng Kin Lai
Kadim Zulaikha
Samudin Norshuhada
Kiran Maleeha
A. Bahaa Ahmed
Original Assignee
Mimos Bhd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mimos Bhd. filed Critical Mimos Bhd.
Publication of WO2012074352A1 publication Critical patent/WO2012074352A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/254—Analysis of motion involving subtraction of images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects

Definitions

  • the technical field of the invention relates to a system and method of surveillance to detect loitering event, more particularly to a system and method to detect loitering events during occlusion, non-occlusion or both in a region.
  • Video surveillance has since progressed to capture images, digitize and analyze those images, and predicting response to events in those images based on the analysis.
  • the common events in surveillance today include detection of pedestrian entering a shop, a vehicle turning against the traffic flow, or a person loitering about a secured area. Detection of loitering event is crucial, as the behavior of lingering around a specific area have been proven to be linked to harmful activities such as tailgating, drug dealing activity or vandalism.
  • drug dealers will loiter around the bus stops while their customers come in on buses, purchase their material, and then leave on other buses. Thus, suspected drug-dealing activities can be detected by monitoring bus stops for individual loitering.
  • US 7088846 B2 disclosed a video surveillance system using rule-based reasoning and multiple-hypothesis scoring to detect predefined behaviors based on movement through zone patterns. Trajectory hypothesis spawning allows for trajectory splitting and/or merging and includes local pruning to manage hypothesis growth. This method requires robust tracker as the accuracy of detecting a loitering event is highly dependant on the tracking results. Also, the rule-based reasoning and multiple hypotheses require thorough understanding on the deployment environment as multiple zones are considered for the analysis.
  • US 2009/002155 disclosed an introduction of electronic tracking device and a transmitter for a more accurate tracking information. Each of the transmitters can be configured accordingly to be associated with a particular subject or a group of subjects.
  • US 6985172 disclosed surveillance apparatus and methods effective for detecting incidents based upon improved image processing techniques applied to infrared and visible light spectrum images in a time sequence.
  • This invention applies advanced algorithms to include temporal processing and model-based analysis to achieve machine perception and recognition of normal scene motions. Subsequent discrepancies between the scene and the model are then detected and compared to classification criteria for disposition.
  • the model-based approach requires the scene to be monitored over a period of time to construct the model of the scene.
  • a system and a method of surveillance to detect loitering event in a region in accordance with the present invention are disclosed herein.
  • the object of the present invention is to provide a system and method to detect loitering event in a region during occlusion and non-occlusion, wherein the system comprises a subtracting means for receiving and processing an input, a tracking means, a timer means, a storage means, and a processor electrically coupled to the storage means in a mutual relationship for processing and determining a loitering event.
  • the system to detect loitering event in a region comprising a tracking means for identifying and labeling subjects with an identity, and a timer means for receiving individual identity information of a plurality of subjects to measure time span of each subject in the region.
  • the timer means for receiving individual identity information of a plurality of subjects to measure time span of each subject in a region includes a first timer for measuring the time span of each subject during non-occlusion, and a second timer for measuring the time span of each subject during an occlusion.
  • the system to detect loitering event in a region comprising a storage means for recording and retrieving individual identity information and timing information of each said subjects, and a processor electrically coupled to the storage means in a mutual relationship to provide time analysis, and loitering detection of the subjects for processing and determining a loitering event.
  • the method to detect loitering event in a region comprises detecting at least a subject in the region from a series of captured frames, labeling each subject with an identity to establish correspondence between the subjects with the series of captured frames, determining time span for each subject, determining total time span for each subject, recording and storing the identity information and time span information of each subject, detecting a loitering event, and triggering an alarm when the loitering event is detected.
  • the aim of the present invention is to solve common problems arising from the current systems and methods to detect loitering event, such as problems of unknown starting time or a first frame when a subject appears in a region due to occlusion or crowded scene, as well as the subject whom reappears in the region before the total time span of said subject exceeds the limit of the predefined loitering threshold, which will not be detected by a common system and method.
  • Figure 2 Describes a process flow of a method of surveillance in accordance with the present invention.
  • Figure 3 Describes a process flow of a timer analysis in accordance with the present invention.
  • Figure 4 Describes a loitering detection process flow in accordance with the present invention.
  • Figure 5 Describes an array of selected captured frames, of frame 3, frame 10, frame 1 1, frame 15, frame 21 and a result frame in accordance with the present invention.
  • a system of surveillance to detect loitering event in a region more particularly to detect a loitering event in a crowded scene
  • a subtracting means (101) for receiving an input (100) and extracting a plurality of motion pixels generated from the input (100)
  • a tracking means (102) for identifying a plurality of subjects from the motion pixels
  • a timer means (103) for receiving individual identity information of a plurality of subjects to measure time span of each subject in the region
  • a storage means (104) for recording and retrieving individual identity information and timing information of each said subjects
  • a processor (105) electrically coupled to the storage means in a mutual relationship to provide time analysis of the subjects for processing and determining a loitering event.
  • Input (100) is the main source for analysis of a detection, wherein said input (100) is preferably a series of frames from a video scene of video cameras, closed circuit television, or any image producing media sources that are inputted into the subtracting means (101) for a preliminary analysis.
  • the subtracting means (101) mainly extracts a plurality of motion pixel images from background scenes and processes outputs of binary map where binary zero pixels indicate the background, and binary non-zero pixels represent the motion pixels.
  • a connecting component criterion an application of widely developed algorithm, such as a heuristic approach to cluster identified data that represents structures of the motion pixels under uncertainty is then applied on the binary map to group motion pixels with similar properties as motion blobs.
  • the key idea of grouping motion pixels into motion blobs is to emphasize blobs corresponding to subjects and eliminating noise as much as possible as smaller motion blobs which are usually due to noise will not be connected via this connecting component criterion, and can be removed by a way of setting a threshold. For example, any blobs with the size of less than 100 pixels will be removed from the system.
  • the tracking means (102) receives the motion blobs from the subtracting means (101) for identifying a plurality of subjects from the motion pixels, and labeling each subjects with an identity to establish correspondence between the subjects.
  • Motion blobs that are fed into this tracking means (102) are processed of temporal image sequence to establish correspondence between the subjects across the series of frames and labeled with a consistent identity throughout their appearance in the image sequences.
  • an object classification procedure where any existing algorithms that can be applied beforehand to discriminate between humans and non-humans, such as an example of using the height-width ration of the motion blob to discriminate between human and non-human so that only preferred subject is analyzed further by subsequent processes. This step is dependent on the aim or functionality of the system, where if the system is only interested in analyzing human subject, then a human classification method may be applied.
  • the labeled motion blobs are then fed into the timer means (103) where the computation of time span of subjects during occlusion is carried out in the timer means (103).
  • the timer means (103) in the present invention herein preferably includes at least a first timer for measuring the time span of each subject during non- occlusion, and at least a second timer for measuring the time span of each subject during an occlusion. Firstly, the location of the subject is checked against a binary map representing a region of interest, where binary zero to indicate region of non- interest and binary one to indicate region of interest. Subsequent processes focuses only on subjects that are detected within the region of interest.
  • the first timer information, and the second timer information of a first frame when subject exits the region of interest are stored in the storage means (104). This step is crucial to allow detection of subject that appears in the region of interest again at any instance, if the time span of the subject does not exceed a predefined loitering threshold.
  • the predefined loitering threshold is set upon configuration of the system, and prior to the deployment of this system because said predefined loitering threshold depends on the criticality of the environment the system is deployed and may vary from one image producing media source to another.
  • the predefined loitering threshold at a lobby may have a longer timestamp (i.e. 5 minutes) as compared to a corridor (i.e. 1 minute), due to the fact that a subject has higher tendency to spend more time at the lobby and thus should not be detected as a suspicious behavior, in this case loitering.
  • a subject lingering about the corridor is not common and thus should be detected as loitering with a shorter timestamp.
  • identity of the subject will be first examined against identity information stored in the storage means (104) to identify subject that re-appears in the region.
  • the identity information may include unique identifier given based on the similarity between temporal features of the subject which includes but not limited to colour properties, texture information, size, and velocity. If the subject's identity information matches any of that in the storage means (104), then it is concluded that the particular subject had appeared in the region before. Thus, timer information of the subject in the storage means (104) is copied to the present subject, and the subject will now continue to carry its history of identity information, and the past timer information in the storage means (104) will be deleted.
  • the system proceeds computation based on concurrent timer information of the subject. Then, status of each subject is determined using a method of overlapping between motion blobs, where correspondence between frames is established via overlapping between motion blobs of the previous and current frame, and comparing similar temporal features to identify subjects to determine events of merging and splitting.
  • splitting event occurs.
  • the present invention assumes both blobs as new subjects and they will be stored into their respective region of interest database in the storage means (104) for further analysis. This is to cater for new detection of subjects, which enters the region of interest in a group, or merely an occurrence of occlusion. Since, one previous blob is associated to more than one current blob, then splitting event is preferentially detected in the present invention and a first timer of the subject is activated. In other words, the first timer is only activated in situations where subject appears as single blob; when there is no occlusion.
  • a second timer is activated and the first timer pauses as long as the subject reappears as a single blob.
  • the second timer is activated when there is more than one subject in a single blob; when the tracking means (102) does not have sufficient information on the subject due to high occlusion.
  • the first timer contributes to the total time span of each subject directly whereas the second timer contributes to the total time span according to a predefined sensitivity threshold.
  • the predefined sensitivity threshold indicates the responsiveness of the system and can be set to a value as preferred upon configuration of said system.
  • a low sensitivity of 5 seconds for example will result in an alert of 5 seconds after a loitering event is detected.
  • a sensitivity threshold of 0 seconds will result in an immediate alert upon detection of an event.
  • the processor (105) that is electrically coupled to the storage means (104) in a mutual relationship provides time analysis of the subjects for processing and determining a loitering event through obtaining the predefined sensitivity threshold from a user interface, depending on the deployment of the system framework.
  • the threshold may be represented by a predetermined value to indicate the least sensitive, which detects reduced events, and by another predetermined value to indicate the most sensitive, which detects more events.
  • the predefined loitering threshold which indicates detection for loitering, that differentiates between a normal and an abnormal time span of subjects in the region, is programmed where subsequently the total time span of each subject is computed.
  • the process flow initializes with primarily inputting a system of the present invention with frame sequences of a region of interest: the area of surveillance, and defining a sensitivity threshold and defining a loitering threshold that depends on the deployment of the system environment as mentioned in the earlier figure. Both thresholds are of time variable, therefore not affecting any detection for differences in the frame rate.
  • Process flow continues with the preferred system to operate on background subtraction (51) of the available frame sequences to extract only motion pixels of interest.
  • motion tracking (52) will be communicated on the extracted motion pixels to identify and label subjects based on temporal image processing as explained previously such as colour properties, texture information, size, and velocity, with the intention that of forming motion blob for each subject.
  • the system will loop through each subject (53) to determine whether or not if any of the subjects are appearing in the region of interest for the very first instant, or if the subjects are reappearing in said region.
  • the well-identified motion blobs of subjects are now prepared for a time analysis (54), and ensuing that, the motion blobs are inspected for a loitering detection (55).
  • next stage is decision making stage, where if a loitering is detected, the system triggers an alert and stores related information of the loitering subject into the storage means (104) for further investigations, or to remain unperturbed if no loitering events are detected.
  • the alert is preferably a highlight projected on the locality of the detected loitering event, and preferably accompanied by a sound alert.
  • Figure 3 there is illustrated a process flow of time analysis procedures in accordance with the method to detect loitering event in a region in the present invention.
  • the process flow initiates a decision of whether or not a detected subject is in the region of interest. If the subject is detected to be in the region of interest, the system determines whether or not if the subject has been stored in a storage means (104) before, as to detect if the subject is of reappearance, so that the system retrieves the last timer information of the subject and activates time analysis for that particular subject, as well as deleting history of that subject from the storage means (104). For a fresh subject, the system proceeds to compute timer information of the subject as elaborated as follows.
  • the process flow continues with determining status of subjects using a method of overlapping between motion blobs in the previous and current frame in the technique of merging and splitting as explained in figure 1.
  • the first timer will be activated upon detection of subject being a single motion blob
  • the second timer will be activated upon detection of subjects in a group of motion blobs.
  • the system determines exit status of the subject, where the first timer information, and the second timer information of a first frame when said subject exits the region of interest are stored in the storage means (104).
  • loitering detection is dependent on the constant values of a predefined sensitivity threshold, and a predefined loitering threshold, which are then computed for the total time span of a subject in the region of interest.
  • the total time span equals sum of value of the first timer with the product value of the second timer and a predefined sensitivity threshold.
  • the total time span of each subject is compared against the predefined loitering threshold, and subjects that appear in the region for a total time span that exceeding allowed predefined loitering threshold are deemed as loitering. Otherwise, the subject is deemed not loitering.
  • FIG. 5 the figure illustrates an array of selected captured frames, of frame 3 (71), frame 10 (72), frame 11 (73), frame 15 (74), frame 21 (75) and a result frame (76) to further explain a case study example on the operations of the preferred embodiment.
  • the frame numbers are one-step incremental series starting from frame 1 , and the first timer and the second timer are represented as definite timer and ambiguous timer respectively in the shown frames. It is assumed that one frame equals to one second in this case study for ease of explanation. From frame 1 to frame 10 (72), a first timer will be activated, as a subject of interest is not occluded.
  • the second timer will be in activation mode until the subject appears as a single blob.
  • the sensitivity threshold is set at 80% and loitering threshold is set to be 15 seconds.
  • the total time span of the particular subject is the sum of its first timer and the product of second timer and sensitivity threshold, which is calculated to be 19 seconds - ⁇ - in this case. Then, the total time span of the subject is compared against the loitering threshold, which is 15s. Since the total time span of the subject is greater than allowed, the subject is deemed as loitering, which concludes that loitering event has been detected.
  • the final stage is decision making, where a subject deemed as loitering, the system stores all information with regards to the subject in the storage means (104). This is important to allow post mortem analysis on the image sequences by an authorized personnel. Otherwise, if a subject is deemed normal, the system continues to process other subjects in the current frame and proceeds to analyze succeeding frames.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Signal Processing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)
  • Alarm Systems (AREA)

Abstract

La présente invention porte sur un système et sur un procédé de surveillance pour détecter un événement d'arrêt anormal, et, plus particulièrement, sur un système et sur un procédé pour détecter des événements d'arrêt anormal durant une occlusion, une non-occlusion, ou les deux, dans une région. Le système comprend des moyens de soustraction (100), des moyens de suivi (101), des moyens formant minuteur (102), des moyens de stockage (103) et un processeur (104) électriquement couplé aux moyens de stockage (105) pour traiter les procédés de détermination d'un événement d'arrêt anormal. Comme le système est apte à détecter des événements d'arrêt anormal durant une occlusion et une non-occlusion, un procédé de mise en œuvre d'une approche à multi-temporisateur est adapté dans le système ; de plus, ledit procédé adapté est apte à détecter une réapparition de sujets.
PCT/MY2011/000078 2010-11-29 2011-06-06 Système et procédé pour détecter un événement d'arrêt anormal dans une région WO2012074352A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
MYPI2010005660A MY159285A (en) 2010-11-29 2010-11-29 System and method to detect loitering event in a region
MYPI2010005660 2010-11-29

Publications (1)

Publication Number Publication Date
WO2012074352A1 true WO2012074352A1 (fr) 2012-06-07

Family

ID=46172113

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/MY2011/000078 WO2012074352A1 (fr) 2010-11-29 2011-06-06 Système et procédé pour détecter un événement d'arrêt anormal dans une région

Country Status (2)

Country Link
MY (1) MY159285A (fr)
WO (1) WO2012074352A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10748011B2 (en) 2017-12-19 2020-08-18 Axis Ab Method, device and system for detecting a loitering event
EP3537379A4 (fr) * 2016-12-13 2020-09-02 Nippon Telegraph and Telephone Corporation Dispositif et procédé de détermination de différence entre images, dispositif et procédé d'estimation de période de changement, et programme
US20220292281A1 (en) * 2021-03-15 2022-09-15 Toniya Anthonipillai Loitering and Vagrancy Computer Vision Ai

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080232688A1 (en) * 2007-03-20 2008-09-25 Senior Andrew W Event detection in visual surveillance systems
WO2009135253A1 (fr) * 2008-05-05 2009-11-12 Iomniscient Pty Ltd Système et procédé de surveillance électronique

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080232688A1 (en) * 2007-03-20 2008-09-25 Senior Andrew W Event detection in visual surveillance systems
WO2009135253A1 (fr) * 2008-05-05 2009-11-12 Iomniscient Pty Ltd Système et procédé de surveillance électronique

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3537379A4 (fr) * 2016-12-13 2020-09-02 Nippon Telegraph and Telephone Corporation Dispositif et procédé de détermination de différence entre images, dispositif et procédé d'estimation de période de changement, et programme
US11398056B2 (en) 2016-12-13 2022-07-26 Nippon Telegraph And Telephone Corporation Image difference deterination device and method that determines whether or not there is a temporary difference between images and non-transitory computer-readable medium
US10748011B2 (en) 2017-12-19 2020-08-18 Axis Ab Method, device and system for detecting a loitering event
US20220292281A1 (en) * 2021-03-15 2022-09-15 Toniya Anthonipillai Loitering and Vagrancy Computer Vision Ai

Also Published As

Publication number Publication date
MY159285A (en) 2016-12-30

Similar Documents

Publication Publication Date Title
US11157778B2 (en) Image analysis system, image analysis method, and storage medium
CN109887281B (zh) 一种监控交通事件的方法及系统
CN102521578B (zh) 一种入侵检测和识别方法
Vishwakarma et al. Automatic detection of human fall in video
WO2019153193A1 (fr) Procédé et système de surveillance d'exploitation d'un taxi, dispositif et support de stockage
KR101980551B1 (ko) 기계학습 객체검출을 이용한 실시간 지능형 cctv 영상 분석 행위 탐지 시스템 및 기계학습 객체검출을 이용한 실시간 지능형 cctv 영상 분석 행위 탐지 방법
US9224278B2 (en) Automated method and system for detecting the presence of a lit cigarette
KR101923555B1 (ko) 비디오 인에이블 전자 물품 감시 검출 시스템 및 방법
KR20180072561A (ko) 비디오 감시 시스템에서 경보들을 발생시키는 방법
CN106981163A (zh) 一种人员入侵异常事件报警方法
JP2008217602A (ja) 不審行動検知システム及び方法
CN107133564A (zh) 一种工装工帽检测方法
EP1428189A1 (fr) Procede et appareil fondes sur la vision, permettant de deceler des faits frauduleux dans un lieu de vente au detail
WO2018179202A1 (fr) Dispositif de traitement d'informations, procédé de commande, et programme
CN111814510B (zh) 一种遗留物主体检测方法及装置
CN113343913A (zh) 目标确定方法、装置、存储介质及计算机设备
JP2012128877A (ja) 不審行動検知システム及び方法
WO2012074352A1 (fr) Système et procédé pour détecter un événement d'arrêt anormal dans une région
KR20200062065A (ko) 이동 경로 예측 방법 및 장치
KR101407394B1 (ko) 방치물 및 도난물 탐지 시스템
WO2012074366A2 (fr) Système et procédé pour détecter un événement d'attente
EP1405279A1 (fr) Procede et dispositif assiste par la vision permettant de detecter un evenement necessitant une assistance ou une documentation
US11341773B2 (en) Detection device and control method of the same
JPH1132322A (ja) 凶器所持者自動追尾装置
JP2003143593A (ja) 画像処理による監視方法およびシステム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11845493

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 11845493

Country of ref document: EP

Kind code of ref document: A1