WO2010123342A2

WO2010123342A2 - Method to generate an analytical path deviation model

Info

Publication number: WO2010123342A2
Application number: PCT/MY2010/000060
Authority: WO
Inventors: Hock Woon Hon
Original assignee: Mimos Bhd
Current assignee: Mimos Bhd
Priority date: 2009-04-22
Filing date: 2010-04-16
Publication date: 2010-10-28
Anticipated expiration: 2011-10-22
Also published as: MY158097A; WO2010123342A3; MY149998A

Abstract

A model to calculate the movement risk probability of an object to deviate from a known standard path and use of the same in a surveillance system.

Description

Method To Generate An Analytical Path Deviation Model

Field of Invention

The present invention relates generally to the field of surveillance, and more particularly to a model for analytically determining the probability of an object deviating from a standard path and use of the same.

Background of Invention

Surveillance systems are being implemented in various areas such as car parks of supermarkets, international borders and airports. In most of this areas, surveillance is used as a means to detect and deter a criminal or suspicious act from being committed. Employing humans to monitor the images from surveillance video is human-intensive and often tedious. Moreover, each human perceive the images seen in a different way from others. To some certain acts may seen suspicious however to another the same act may seem normal.

Therefore, there is a need for an automated surveillance system which is able to automatically generate conclusions . However, most of the automated surveillance system is trained to classify an act as suspicious or otherwise based on the comparison of instantaneous event and that which has been stored in its database. The system is unable to interpret the movement trend of an object whether if there exist a possible risk of breach of security, because historical information is not taken into consideration when drawing a conclusion.

Therefore there exist a need for a surveillance system which is automatically able to interpret the risk of an object committing a suspicious act based on the historical information of the objects movement.

hi view of the above, there also exist a need for a model which is able to generate a continuous movement pattern risk probability of an object in real time, hi order for a surveillance system to be able to draw a conclusion if the object is about to commit a suspicious act or not.

Summary of Invention

It is an object of the present invention to provide a method to generate a model to analytically determine the probability of a mobile object, which is on the move in the area under surveillance, to deviate from the learned standard path based on the object's temporal moving pattern.

It is also an object of the present invention to provide a method for an automated surveillance system which is able to alert the security personnel when an object's deviation probability pattern indicates a suspicious behavior.

Brief Description of Drawings

Figure 1 Overview of Analytical Path Deviation Model generating method

Figure 2 Example of standard and suspicious path in an area under surveillance

Figure 3 Standard path computed by machine learning or neural network

Figure 4 Method to calculate shortest distance

Figure 5 One or more standard path

Figure 6 Motion trend of an object

Figure 7 Sliding Window

Figure 8 Assignment of weight for sliding window pair

Figure 9 Normalised value of model

Figure 10 Calculation of ratio

Figure 11 Over view of Deviation Risk Monitoring System

Figure 12 Example of threshold value for semantic reasoning

Figure 13 Standard path in an customs declaration area Detailed Description

The present invention will now be detailed with reference to the accompanying drawings, in which exemplary embodiments of the invention are shown. It should be understood that the description and specific examples are intended for purposes of illustration only and are not intended to limit the scope of the present disclosure.

A general overview of the exemplary method to generate the Analytical Path Deviation Model can been seen from Figure 1. It is an aspect of the present invention to generate a model that is able to calculate the probability of a mobile object to deviate from a standard path. The definition of standard path here is a common path which is generally taken by moving objects in the area under surveillance. A example of a standard path and suspicious path can been seen from Figure 2 (a) and (b).

The probability value generated by the model is also an indication of the degree of the intention of the mobile object to deviate from a standard path. The model generates a continuous probability value at real time. The intention of the mobile object i.e. whether the object is about commit a suspicious act or not can be analytically interpreted by analyzing the deviation probability valued obtained over a plurality of frames, hi other words, a conclusion on the objects intention is drawn based on its moving pattern history.

The present Analytical Path Deviation Model takes into consideration a few criteria when computing the probability value. The criteria are: a. The instantaneous distance between the object and the outmost pixel of the standard path b. The relative distance between the two consecutive instant point c. The deviation rate d. The ratio between the width of the standard path and shortest distance.

Hereinafter, the process to generate Analytical Path Deviation Model will be discussed in detail. a. Standard path detection

In order to initiate the Analytical Path Deviation Model generation process, the standard path used in the area under surveillance must first be determined. A known standard path generation technique either from machine learning or neural network is used here. Generally an area under surveillance is observed over a period of time and the trend of the path taken by objects when traversing the area is collected. A standard path Le. the common path taken based on these observation is then computed by means of machine learning or neural network. An example of a standard path can be observed from Figure 3.

b. Border Extraction

Once the standard path is established, then the pixel co-ordinate of standard path border is extracted . This pixel co-ordinate is then organized in an orderly manner, in order to enable efficient comparison between the border co-ordinate and the instantaneous pixel co-ordinate of the mobile object.

c. Obtaining Instantaneous co-ordinate of the object of interest

The mobile object is tracked using known motion tracking methods. The centroid of the tracked mobile object is used to obtain its instantaneous co-ordinate.

d. Determining the shortest distance, d_sh_ortest

To calculate the shortest distance, each co-ordinate of the mobile object's centroid at time t_n is compared against the closest border co-ordinates of the standard path.. The sampling interval t_n is predetermined, for example 30 seconds. Therefore at every 30 second a sampling is done and the shortest distance, d_shom_st is determined. It is to be noted that the length of the sampling interval has a direct effect on the efficiency of the detection. Wherein if the sampling interval is short it allows the system to capture the information correctly, especially in fast moving scenes. However, if the sampling is done too frequently it will take up too much of the resources of the computer. Therefore, ideally the sampling interval is set within the range of 100ms to 1 second.

The instantaneous sample co-ordinate in the image is compared with a set of pixel co-ordinates that denotes the outermost layer of the standard path i.e. standard path boundary. Figure 4 illustrates the calculation of the shortest distance.

When there are a plurality of standard path in an area as illustrated by Figure 5, the object's centroid is compared against the closest border co-ordinate of both the standard paths. The instantaneous centroid co-ordinate is compared with the pixel co-ordinate of plurality of standard path and the shortest distance is then determined.

Determining the motion trend

The motion trend of the^'mobile object is determined by analyzing the shortest distance, d_sh_on_est samples collected over time, t . If the trend of the shortest distance, dshortest is increasing over time, t as illustrated in Figure 6 (a), then the motion trend of the mobile object is classified as deviating from the standard path. However, if the pattern follows Figure 6 (b) then the motion trend of the mobile object is conforming to the standard path.

Trend values are assigned according to the motion trend of the mobile object. When the motion trend is deviating from the standard path, the trend value is set as a negative value. On the other hand, when the motion trend is conforming to the standard path the trend value is set as a positive value. For the purpose of example^" the trend value will be set as -0.15 when the motion trend is deviating and as 0.1 when the motion trend is conforming hereinafter. / Determining sliding window and sliding window weight

Sliding windows are defined as a user-deteπnined number of consecutive video frames of the surveillance video that is used for calculating the probability of the mobile objects deviation from the standard path. The number sliding window can be pre-defined i.e. a default value or be defined at any point of time. An example of sliding window is illustrated in Figure 7. In this example the number of sliding window has been set at 5.

Weight is assigned to each consecutive pair of sliding window as can be seen in Figure 8. The sliding window pair that is closest to the current time, t is assigned the highest weight, and the further the sliding window pair is the lesser is the value assigned to them. An example of the weight assignment can be seen in Figure 8.

g. Normalised model

The probability value obtained from the model which ranges from negative to positive is then normalised to create a single polarity of possibility. The normalised probability value range is set from 0.1 to 1 in order to ease the interpretation of the risk in percentage.

An example of the model value and its corresponding normalized probability value is illustrated in Figure 9.

h. Determining the ratio, r

Ratio, r is defined as instantaneous shortest distance/longest standard path. Figure 10 illustrates the calculation of the ratio. Calculation of deviation risk probability

The instantaneous deviation risk probability of a mobile object is calculated by using the below equation.

Risk Probability = Shortest Distance, d_shortest x trend x sliding window weight x ratio, r

The Analytical Path Deviation Model generates a continuous instantaneous deviation risk probability of a mobile object according to real time in the above steps. The model is integrated into a video surveillance system to achieve an automated Deviation Risk Monitoring System.

The Deviation Risk Monitoring System is used to continuously monitor the risk probability that an object within the area under surveillance is about to commit a suspicious or criminal act. The premise for this lies in the fact that when an object is deviating from what is known as a standard path, it is an indication that the object is intentionally not conforming to the standard transition pattern observed in that particular area, which leads to the conclusion that the object is possibly having an criminal or suspicious intend. Examples of areas which will benefit from this system are areas where the objects are expected to follow a standard path due to the existence of fixed exit or entry. Examples of such areas are customs declaration counter at an airport, international borders and areas with multiple entries and exits such as a stadium.

A general overview of the surveillance system can be seen in Figure 11. The system can generally be divided into 3 sections: a. Motion Tracking b. Analytical Path Deviation Model c. Semantic component a. Motion Tracking

Image input for the system is obtained from a plurality of image acquiring devices such as a surveillance camera. The image acquired is then manipulated using a known motion tracking method. Objects that are being tracked are then classified. Since the purpose of the system is to track mobile objections, the objects are classified as being human or non- human. The tracked information such as centroid co-ordinate are then stored in a circular buffer so that it is available from the system's memory to be used in the following steps.

b. Analytical Path Deviation Model

In this section, N number of instantaneous tracked information is obtained from an image buffer. However, in the preferred embodiment a circular buffer is used for the same purpose, wherein N denotes the number of sliding window. This windowed tracked information is then sent to the reference model to be assigned the weight for the respective sliding window pair.

Instantaneous deviation probability of an object is calculated using the preceding methods of Analytical Path Deviation Model. This probability value is generated continuously and will be used as the input for the semantic component of the system, which will be described in detail in the following section.

C; Semantic component.

This component of the surveillance system is responsible for analytically reasoning whether there arise a need to trigger the security alarm based on the probability values generated by the Analytical Path Deviation Model.

The security alarm is triggered when the reasoning value of the semantic interpretation stage reaches or exceeds the predetermined threshold value. The threshold value is a border value indicating that a possible criminal act is about to be committed. The value is predetermined by the user. Threshold value can be set high or low depending on the sensitivity level of an area. An example of threshold value can be seen in Figure 12 The Deviation Risk Monitoring System will now be further discussed with reference to the preferred embodiments.

hi a preferred embodiment the Deviation Risk Monitoring System is used in the customs declaration counters, for example in airports. An example Of standard paths in an custom's declaration area can be seen in figure 13 (a). Figure 13 (b) illustrates a graph indicating the probability of deviation of an object in the area based over time and the threshold value.

In this example when an object is moving in neither the standard path leading to the - declaration counter nor the nothing to declare counter, it triggers the Deviation Risk Monitoring System to track the object. Also, when an object which was moving in a standard path but at some point exits the boundaries of the standard paths, it will also trigger the system to track its movement. As a rule, the Deviation Risk Monitoring System will initiate tracking of an object which is not moving within the boundaries of the standard path.

Based on the probability values generated by the Analytical Path Deviation Model, the semantic component reasons if the movement pattern of the object is suspicious or not. The security alarm will be triggered if the semantic component reasons that the movement pattern of the object is suspicious.

Referring to Figure 13 (b), the system is also able to pick an anomaly in the movement. For example, although the end result in Figure 13 (b) indicates that the object is in the standard path, there is a pattern indicating a very likely possibility that the object is not going to return to the standard path. This is indicated by the probability pattern within the circle. This would be considered as an anomaly, because there is an abrupt change of movement pattern. Therefore, this could also indicate that the object had an suspicious intend.

While a limited number of embodiments of the invention have been described, these embodiments are not intended to limit the scope of the invention as otherwise described and. claimed herein. Those of ordinary skill hi the art will recognize that variations and modifications from the described embodiments exist. Moreover, unless otherwise specified, the steps of the methods described herein are not limited to any particular order or sequence.

Claims

1. A method for predicting a suspicious event in a surveillance or monitoring system comprising the steps of :

predetermining a reference path in a video scene for an object to follow generated from spatial-temporal coherence trajectories of a test object obtainable from a sequence of video frames;

calibrating instantaneous deviations of the object from the reference path to a predetermined value to indicate the possibility of the occurrence of the suspicious event.

2. A method for predicting a suspicious event in a surveillance or monitoring system as claimed in Claim 1 wherein the calibration is based on the distance of the deviated object from the reference path.

3. A method for predicting a suspicious event in a surveillance or monitoring system as claimed in Claim 1 wherein the calibration is based on rate of the object deviated from the reference path.

4. A method for predicting a suspicious event in a surveillance or monitoring system as claimed in Claim 1 wherein the calibration is based on the ratio between width and the shortest distance of the path.

5. A method for predicting a suspicious event in a surveillance or monitoring system as claimed in Claim 1 wherein the calibration is based on the pixel coordinate between points on the deviated object and the reference path.