CN106780547A - Monitor video velocity anomaly mesh object detection method is directed to based on kinergety model - Google Patents

Abstract

The invention discloses a method for detecting an abnormal target in a surveillance video based on a motion energy model, which includes: considering the motion vector information of the abnormal target and its surrounding 8 neighborhoods, establishing a motion energy model, and using the sum of data items and difference items , to represent the motion energy value of each spatial block; boundary value extraction is performed on each spatial block of the video frame, and each boundary value represents the boundary value of its corresponding spatial block, which distinguishes normal motion from abnormal motion; based on the sigmoid function The binary classifier classifies the motion energy value in the test set, takes the difference between the motion energy value of each block of the test set and the extracted boundary value as input, and outputs a probability value of [0,1]; the corresponding space block The larger the probability value, the higher the possibility of abnormal motion, that is, abnormal speed. The invention effectively expresses the motion intensity pattern, has low complexity, balances detection accuracy and detection efficiency, and can detect objects with abnormal speed in monitoring video in real time.

Description

Detection method for abnormal speed target in surveillance video based on motion energy model

technical field

The invention relates to the technical field of video surveillance abnormality detection, in particular to a method for detecting objects with abnormal velocity in surveillance video based on a motion energy model.

Background technique

In recent years, with the frequent occurrence of some terrorist attacks and mass violence incidents, people's requirements for urban life quality and life safety precautions are increasing day by day. Although various video surveillance devices have been widely installed in various crowded public places, such as railway stations, subway stations, hospitals, squares, and communities, etc., the current video surveillance system can only perform simple monitoring of a certain monitoring scene. Video recording, video transmission to the monitoring room, manual scene monitoring by monitoring personnel; or storage of monitoring video for later search and evidence collection. This traditional monitoring system lacks real-time and intelligence, and relies heavily on the experience and subjective judgment of monitoring personnel. Therefore, in order to enable the monitoring system to automatically analyze video content in real time, judge abnormal targets and abnormal events in the monitoring video, and assist the monitoring personnel to make judgments, it is necessary to greatly improve the accuracy of monitoring video abnormality judgment and the quality of public safety precautions.

Abnormal events in public scenes are diverse, such as: robbery, fall, intrusion, crowd gathering, retrograde flow of people, objects speeding up and so on. Among them, the application range of detecting abnormal speed targets is very wide, and the practicability is very high. In research in recent years, many researchers have proposed different object speed anomaly detection methods based on different technologies.

Helbing et al. proposed a social force model based on motion information to calculate the interaction force of the crowd, so as to judge the abnormality of direction and speed, but this method is not suitable for scenes with sparse crowds that lack sufficient motion information ^[1] ; Adam et al. Multiple monitoring units with fixed positions make statistics on the low-level features of the scene and detect speeding objects in different positions, but this method is limited by the number and density of monitoring units ^[2] ; Weixin et al. extract the dynamic textual feature description of objects in the scene Objects in different states can be used to classify abnormal and normal, but the feature extraction process of this method has high time complexity and the real-time effect is not good ^[3] ; The detector judges the abnormality, but this method is limited by the selection of the template and the setting of the classification detection mechanism ^[4] .

Contents of the invention

The present invention provides a detection method based on a motion energy model for monitoring video speed abnormal targets. The present invention considers the motion information of the target object and its adjacent objects at the same time, and has good robustness to the detection of motion intensity, which can effectively To accurately detect abnormal speed targets in different scenarios, see the following description for details:

A detection method based on a motion energy model for an abnormal target in a surveillance video, the detection method comprising the following steps:

Considering the motion vector information of the abnormal target and its surrounding 8 neighborhoods, a motion energy model is established, which includes: data items and difference items, and the sum of the data items and difference items is used to represent the motion energy value of each space block ;

Boundary value extraction is performed on each spatial block of the video frame, and each boundary value represents the boundary value of its corresponding spatial block to distinguish normal motion from abnormal motion;

The binary classifier based on the sigmoid function classifies the motion energy value in the test set, takes the difference between the motion energy value of each block of the test set and the extracted boundary value as input, and outputs a probability value of [0,1];

The larger the probability value corresponding to the space block, the higher the possibility of abnormal motion, that is, the higher the possibility of abnormal speed.

The detection method also includes:

Combined with the image pyramid, use the Lucas-Kanade optical flow method to extract the optical flow vector for each frame in the surveillance video at different scales, establish the optical flow field, and divide the optical flow field of each frame into M×N non-overlapping Spatial block, get the motion vector of each spatial block.

The motion vector is specifically:

B _ij =(f _x,y (h,v)|x=1,2,...,M,y=1,2,...,N)

i=1,2,...,P,j=1,2,...,Q

Among them, B _{i, j} represents the spatial block with position (i, j); f _{x, y} (h, v) represents the motion vector contained in block B _{i, j} with position (x, y), including a Horizontal component h and vertical component v; M is the horizontal length of each spatial block; N is the vertical length of each spatial block; P is the number of spatial blocks in the horizontal direction of each frame; Q is the vertical direction of each frame the number of space blocks.

The method also includes quantizing the motion vectors.

The extraction of the boundary value is specifically:

If the data growth rate is greater than a preset certain threshold, the motion energy value of the previous data point is regarded as a boundary value.

The beneficial effects of the technical solution provided by the present invention are: the motion energy model proposed by the present invention can effectively express the motion intensity pattern, the algorithm complexity is low, the detection accuracy and detection efficiency are balanced, and the abnormal speed in the monitoring video can be detected in real time. Target.

Description of drawings

Fig. 1 has provided the flow chart of the detection method for monitoring video speed abnormal target based on motion energy model;

Figure 2 shows an example of a monitoring scene with local speed anomalies;

(a) is a schematic diagram of a fast-moving motor vehicle on the sidewalk; (b) is a schematic diagram of a bicycle passing through the sidewalk.

Figure 3 shows an example diagram of a local velocity anomaly target and its motion energy value;

(a), (c) are schematic diagrams of any two frames containing abnormal motion in the test set; (b), (d) are schematic diagrams of motion energy values of corresponding frames in (a) and (c), respectively.

Fig. 4 has provided the example figure that utilizes the method for data growth rate to estimate motion energy boundary value;

(a) is all the frames in the training set, and this example counts the motion energy value of the square marked area; (b) is the distribution of the block motion energy value from small to large, it can be seen that most of the values tend to zero, and there are very few very large noise value; (c) is the data growth rate of the motion energy value, when the data growth rate is too large, it is considered that the value has exceeded the range of the normal motion energy value; (d) is the final selected motion energy Boundary value (intersection point of two dotted lines), the bold dotted line indicates the range of normal motion energy value.

Figure 5 shows the final motion energy boundary value diagram;

Fig. 6 has provided the comparison result figure of the performance of the present invention and Social Force, MDT and MPPCA method on UCSD dataset;

(a) Schematic diagram of frame-level anomaly detection performance of UCSD Ped1 test set; (b) Schematic diagram of frame-level anomaly detection performance of UCSD Ped2 test set; (c) Schematic diagram of pixel-level anomaly detection performance of UCSD Ped2 test set .

Fig. 7 shows the visualization result map (marking abnormal targets) of the present invention on the UCSD data set.

detailed description

In order to make the purpose, technical solution and advantages of the present invention clearer, the implementation manners of the present invention will be further described in detail below.

Example 1

The embodiment of the present invention aims at the detection of abnormal speed targets in the monitoring scene. Based on the scene motion information, a motion energy model is proposed to describe the motion mode of the object, and by classifying the motion energy values in different regions, the real-time detection of abnormal speed targets is realized. Detect, referring to Fig. 1, this method comprises the following steps:

101: Extract the motion information of the target area, and obtain the motion intensity value;

102: Using the SSD metric to calculate the difference between the target area and its 8 neighborhoods, and obtain a motion difference value;

103: Use the sum of two items to represent the motion energy value of the object in the central area, and then detect the target with abnormal speed by classifying the motion energy value.

To sum up, the motion energy model proposed by the embodiment of the present invention takes into account the motion information of the target object and its adjacent objects at the same time, has better robustness to the detection of motion intensity, and can effectively detect speed anomalies in different scenarios Target.

Example 2

The scheme in embodiment 1 is further introduced below in combination with specific calculation formulas, see the following description for details:

201: Extract motion information;

The motion vector is an important feature that can express the motion information of an object, and the optical flow method is one of the most effective methods to calculate the motion vector. In the embodiment of the present invention, in combination with the image pyramid, the Lucas-Kanade optical flow method is used to extract the optical flow vector for each frame in the surveillance video at different scales, and establish the optical flow field. Afterwards, the optical flow field of each frame is divided into non-overlapping spatial blocks of size M×N, totaling P×Q blocks. For each block, the motion vector is expressed as follows:

B _ij =(f _x,y (h,v)|x=1,2,...,M,y=1,2,...,N)

i=1,2,...,P,j=1,2,...,Q

Among them, B _{i, j} represents the spatial block whose position is (i, j); f _{x, y} (h, v) represents the motion vector contained in the block B _{i, j} whose position is (x, y), which contains A horizontal component h and a vertical component v. The strength of the motion vector at (x,y) is expressed as:

mag _x,y ＝||f _x,y (h,v)|| ₂

For the convenience of statistics, the embodiment of the present invention properly quantizes the motion vector of each pixel point. Experiments show that the detection effect is the best when the quantization level is 16. The specific quantization method is as follows: first find the maximum value of the motion vector strength value of each spatial block The value max mag _{x, y} divides the interval [0, max mag _{x, y} ] into 16 equal parts, and the motion vector strength value falling on each small equal interval is quantized as the midpoint value of the interval, and finally mag _x is obtained _,y , that is, the intensity of the quantized motion vector at the (x,y) position.

202: motion energy model;

For local anomalies, abnormal objects must be surrounded by normal objects. Therefore, in the embodiment of the present invention, a motion energy model is established by considering the motion vector information of the abnormal target and its surrounding 8 neighborhoods. The model consists of two items: a data term and a difference term, as follows:

Among them, DataTerm(i,j) represents the data item; DifferenceTerm(i,j) represents the difference term; the main motion intensity in the spatial block B _i,j is expressed as Dom mag ^{i _, j} _x,y , which is the quantity in the block The strength value of the most motion vectors. In the difference item, S _i,j represents the 8-neighborhood area of the center block B _i,j , and B _m,n is a spatial block belonging to the 8-neighborhood. The difference in motion intensity between the central block and its adjacent areas is expressed by the SSD metric as follows:

Among them, SSD(B _i,j ,B _m,n ) is the sum of the squares of the errors of the corresponding pixels of the two image blocks, and measures the difference in motion intensity between the two spatial blocks; is the motion vector intensity at the position (x, y) in the spatial block with the serial number (i, j); is the strength of the motion vector at the position (x, y) in the spatial block with sequence number (m, n); N _f is the number of motion vectors in B _{m, n} . Since the closer the area is to the central block, the association with the central block is greater. Therefore, assign different weight values ω _m,n to spatial blocks located at different distances to adjust the weights of different regions.

Use the sum of the data item and the difference item to represent the motion energy value of each space block, and adjust the proportion of the two items through α and β, as shown below:

E(i,j)=α·DataTerm(i,j)+β·DifferenceTerm(i,j)

203: Boundary value extraction;

The motion energy value is calculated for each spatial block of all video frames in the training set, and the distribution of the motion energy value of each spatial block is counted. In order to reduce the influence of the noise of the optical flow field, the excessive value of the motion energy value should be eliminated, and the boundary value between normal motion and abnormal motion should be found. The embodiment of the present invention extracts the boundary value by calculating the data growth rate of the motion energy value of each spatial block. If the data growth rate is greater than a preset certain threshold, the motion energy value of the previous data point is regarded as a boundary value. According to experiments, setting the threshold to 0.5 works best.

204: Spatial block classification.

Due to the position of the surveillance lens in the surveillance scene, moving objects will be detected with different motion intensity values and motion energy values in different areas. In general, objects closer to the camera have greater motion intensity. Therefore, boundary value extraction is performed on each spatial block of the video frame, and each boundary value represents a boundary value of its corresponding spatial block to distinguish normal motion from abnormal motion.

In the embodiment of the present invention, a binary classifier based on the sigmoid function is used to classify the motion energy value in the test set. The binary classifier takes the difference between the motion energy value of each block of the test set and the extracted boundary value as input, and outputs a [ 0,1] probability value. The larger the probability value corresponding to the space block, the higher the possibility of abnormal motion, that is, the higher the possibility of abnormal speed. In other words, the motion energy value of the space block is too large beyond the normal range. A binary classifier looks like this:

Among them, E'(i,j) and E(i,j) represent the boundary energy value and the motion energy value to be tested, respectively.

To sum up, the motion energy model proposed by the embodiment of the present invention takes into account the motion information of the target object and its adjacent objects at the same time, has better robustness to the detection of motion intensity, and can effectively detect speed anomalies in different scenarios Target.

Example 3

Below in conjunction with concrete accompanying drawing, experimental data, the scheme in embodiment 1 and 2 is carried out feasibility verification, see the following description for details:

The experiment is simulated and tested on the UCSD data set on the MATLAB platform. The UCSD dataset is widely used in local anomaly detection, and it contains two subsets of different scenes - Ped1 and Ped2. The training sets of the two subsets only contain normal motion (walking pedestrians), and the test set contains some moving objects with abnormal speeds, such as passing bicycles, fast-moving motor vehicles, skateboards, and so on. For frame-level detection, as long as a pixel is detected as abnormal during the test, the frame is marked as abnormal; for pixel-level abnormal detection, as long as the pixels marked as abnormal account for the real abnormal target pixels during the test If more than 40% of the points are detected, it can be regarded as correct detection; otherwise, the detection is wrong.

The embodiment of the present invention compares with Social Force, MPPCA, and MDT methods that also belong to low-level feature detection methods, and the comparison results are as follows:

In the frame-level detection task, it can be seen from the ROC curve results in Figure 6(a) and (b) that this method performs best: it is comparable to the MDT method in Ped1, and the gap is small; in Ped2, it outperforms the MDT method . As shown in Table 1, the average error rate of the present invention is similar to that of MDT-temporal. In the pixel-level detection task, it can be seen from the ROC curve results in Figure 6(c) that the detection rate of the present invention is slightly different from that of the MDT method. However, as shown in Table 2, the algorithm efficiency of this method is much higher than other methods, and it can basically realize real-time abnormal target detection after optimization.

Table 1 Frame-level average error rate (PED1 and PED2)

Table 2 Detection rate and algorithm efficiency (PED2)

Figure 7(a) and (b) show the visualization results: the white area in the figure is the area where the abnormal speed occurs, and it can be clearly seen that the present invention effectively detects abnormal speed targets such as motor vehicles, bicycles, and skateboards.

To sum up, the motion energy model proposed by the embodiment of the present invention takes into account the motion information of the target object and its adjacent objects at the same time, has better robustness to the detection of motion intensity, and can effectively detect speed anomalies in different scenarios Target.

references

[1] Helbing, Dirk, and Peter Molnar."Social force model for pedestrian dynamics."Physical review E 51.5(1995):4282.

[2] Adam, Amit, et al."Robust real-time unusual event detection using multiple fixed-location monitors."IEEE Transactions on Pattern Analysis and Machine Intelligence 30.3(2008):555-560.

[3] Li, Weixin, Vijay Mahadevan, and Nuno Vasconcelos. "Anomaly detection and localization in crowded scenes." IEEE transactions on pattern analysis and machine intelligence 36.1(2014):18-32.

[4]Reddy, Vikas, Conrad Sanderson, and Brian C.Lovell."Improved anomaly detection in crowded scenes via cell-based analysis of foreground speed, size and texture."CVPR 2011WORKSHOPS.IEEE,2011.

Those skilled in the art can understand that the accompanying drawing is only a schematic diagram of a preferred embodiment, and the serial numbers of the above-mentioned embodiments of the present invention are for description only, and do not represent the advantages and disadvantages of the embodiments.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection of the present invention. within range.

Claims (5)

1. a kind of detection method based on motion energy model for monitoring video speed abnormal target, it is characterized in that, described detection method comprises the following steps: Considering the motion vector information of the abnormal target and its surrounding 8 neighborhoods, a motion energy model is established, which includes: data items and difference items, and the sum of the data items and difference items is used to represent the motion energy value of each space block ; Boundary value extraction is performed on each spatial block of the video frame, and each boundary value represents the boundary value of its corresponding spatial block to distinguish normal motion from abnormal motion; The binary classifier based on the sigmoid function classifies the motion energy value in the test set, takes the difference between the motion energy value of each block of the test set and the extracted boundary value as input, and outputs a probability value of [0,1]; The larger the probability value corresponding to the space block, the higher the possibility of abnormal motion, that is, the higher the possibility of abnormal speed. 2. a kind of detection method based on motion energy model according to claim 1 for monitoring video speed abnormal target, it is characterized in that, described detection method also comprises: Combined with the image pyramid, use the Lucas-Kanade optical flow method to extract the optical flow vector for each frame in the surveillance video at different scales, establish the optical flow field, and divide the optical flow field of each frame into M×N non-overlapping Spatial block, get the motion vector of each spatial block. 3. A kind of detection method based on motion energy model according to claim 1 and 2 for monitoring video speed abnormal target, it is characterized in that, described motion vector is specifically: B _ij =(f _x,y (h,v)|x=1,2,...,M,y=1,2,...,N) i=1,2,...,P,j=1,2,...,Q Among them, B _{i, j} represents the spatial block with position (i, j); f _{x, y} (h, v) represents the motion vector contained in block B _{i, j} with position (x, y), including a Horizontal component h and vertical component v; M is the horizontal length of each spatial block; N is the vertical length of each spatial block; P is the number of spatial blocks in the horizontal direction of each frame; Q is the vertical direction of each frame the number of space blocks. 4. A method for detecting objects with abnormal speed in surveillance video based on a motion energy model according to claim 1 or 2, characterized in that the method further comprises: quantifying motion vectors. 5. a kind of detection method based on motion energy model according to claim 1 for monitoring video speed abnormal target, is characterized in that, described boundary value extraction is specifically: If the data growth rate is greater than a preset certain threshold, the motion energy value of the previous data point is regarded as a boundary value.