CN106503647A - The accident detection method that structural sparse is represented is approached based on low-rank - Google Patents

Abstract

The invention discloses an abnormal event detection method based on low-rank approximation structured sparse representation, which includes three processes of feature extraction, training and testing. 1) Extract the multi-scale 3D gradient features of the video sequence; 2) Reduce the dimensionality of the multi-scale 3D gradient features to form a training feature set and a test feature set; 3) Initialize the remaining training features and related parameters; 4) Perform training on the remaining training features Iteratively learn the group sparse dictionary to obtain the normal mode dictionary set; 5) use the group sparse dictionary set obtained by the training process to perform sparse reconstruction on the test features; 6) judge whether the test features are abnormal features according to the reconstruction error. The invention solves the shortcomings of not fully exploiting the low-rank characteristics of video data and slow detection rate in the abnormality detection technology.

Description

Anomaly Event Detection Method Based on Low Rank Approximation Structured Sparse Representation

technical field

The invention relates to the fields of pattern recognition and video analysis, and more specifically, relates to a method for detecting abnormal events based on low-rank approximation structured sparse representation.

Background technique

Anomaly event detection in video sequences is an active research topic in computer vision and has been widely used in many applications, such as crowd monitoring, public place detection, traffic safety, and abnormal personal behavior. In the face of massive video data, the traditional manual marking of abnormal events is time-consuming and inefficient. Therefore, automated and fast anomaly detection methods for video sequences are urgently needed.

Although research on anomalous event detection has made great progress in feature extraction, behavior modeling, and anomaly measurement, anomalous event detection in video sequences is still a very challenging task. First, there is no precise definition of an anomalous event in a video. A common abnormal behavior identification method is abnormal behavior pattern clustering, and the other is to treat those detection samples with low occurrence rate as anomalies. The difficulty of the first method is that there is not enough prior knowledge to describe the abnormal behavior pattern; the second method needs to build a probabilistic model, and anomaly detection relies on the definition of normal patterns and the multi-scale variation of features. Second, anomaly detection in dense scenes requires that the behavior model can handle high-density moving targets, which needs to consider the effects of occlusion and interaction between multiple targets.

From the perspective of feature extraction, abnormal event detection methods can be divided into object trajectory-based methods and low-level feature-based methods. The method based on object trajectory first tracks the moving object, and then utilizes the object trajectory to detect anomalous events. The method based on the target trajectory can clearly represent the spatial state of the target at each moment, but this type of method is sensitive to noise, occlusion and tracking errors, and cannot detect anomalies in dense scenes. Low-level feature-based methods can overcome the shortcomings of object trajectory methods by extracting pixel-level motion features and morphological features in video sequences.

Currently, mainstream methods for anomalous event detection include Dynamic Bayesian Networks (DBNs), Probabilistic Topic Models (PTMs) and Sparse Representation Models. In DBNs, Hidden Markov Models (HMMs) and Markov Random Fields (MRFs) increase the modeling cost geometrically as the number of detected objects increases, making these models insufficient to handle dense scenes. Compared with DBNs, PTMs, such as PLSA and LDA, only focus on spatially co-occurring visual words, but ignore the temporal information of features, making probabilistic topic models unable to localize anomalous events in space and time. In recent years, sparse representation models for anomaly detection have attracted attention. Most sparse representation models are trained with an over-complete dictionary, but do not fully exploit the low-rank characteristics and inherent structural redundancy of video data.

Contents of the invention

The object of the present invention is to propose an abnormal event detection method based on low-rank approximation structured sparse representation in view of the shortcomings of the low-rank characteristics of video data and the slow detection rate in the above-mentioned anomaly detection technology.

The technical solution to realize the purpose of the present invention is: a method for abnormal event detection based on low-rank approximation structured sparse representation, including three processes of feature extraction, training and testing:

The feature extraction process includes the following steps:

1) Extract multi-scale three-dimensional gradient features of video sequences;

2) Dimensionality reduction is performed on multi-scale 3D gradient features to form training feature sets and test feature sets.

The training process includes the following steps:

3) Initialize the remaining training features and related parameters;

4) Iteratively learn the group sparse dictionary on the remaining training features to obtain the normal mode dictionary set.

The testing process includes the following steps:

5) Sparsely reconstruct the test features using the group sparse dictionary set obtained by the training process;

6) According to the reconstruction error, it is judged whether the test feature is an abnormal feature.

In the above method, said step 1) includes the following specific steps:

1.1) Each frame of the video sequence is scaled to different scales to form a three-layer image pyramid.

1.2) Sampling the spatio-temporal cube of each layer image to extract the 3D gradient features of spatially non-overlapping regions.

1.3) For each layer of video sequence, the 3D gradient features of 5 consecutive frames on the same spatial region are superimposed together to form a spatio-temporal feature.

In the above method, said step 2) includes the following specific steps:

2.1) Use Principal Component Analysis (PCA) to reduce the dimensionality of each of the above extracted spatio-temporal features.

2.2) Using the above method, convert the training video sequence and the testing video sequence into a training feature set and a testing feature set.

In the above method, said step 3) includes the following specific steps:

3.1) Initialize the training feature set obtained in step 2.2) as the remaining training feature set;

3.2) Initialize regularization parameters, error threshold, number of iterations and normal mode dictionary set;

In the above method, said step 4) includes the following specific steps:

4.1) If the remaining feature set is empty, the training process ends; if the remaining feature set is not empty, determine the number of clusters, and perform K-means clustering on the remaining feature set.

4.2) Carry out dictionary learning for each feature cluster respectively to obtain group sparse dictionary.

4.3) Choose a suitable dictionary to represent the remaining features. If the dictionary can represent the remaining features, the dictionary is added to the normal mode dictionary set, and the features that can be represented by the dictionary are removed from the remaining training feature set; if the dictionary cannot represent any of the remaining features, the dictionary is discarded.

4.4) Add 1 to the number of iterations, skip to step 4.1);

In the above method, said step 5) includes the following specific steps:

5.1) For each test feature, traverse the normal pattern dictionary set obtained by the training process, and estimate the sparse coefficient corresponding to the dictionary;

5.2) According to specific dictionary and its corresponding sparse coefficient, calculate the reconstruction error corresponding to this group of sparse dictionary of this test feature;

In the above method, said step 6) includes the following specific steps:

6.1) For a certain test feature, if a dictionary whose corresponding reconstruction error is less than the reconstruction threshold is found, the test feature is judged to be a normal feature;

6.2) For a certain test feature, if the reconstruction error of any dictionary set corresponding to it is greater than the reconstruction threshold, then it is judged that the test feature is an abnormal feature;

Compared with the prior art, the present invention has significant advantages: First, because of the low-rank characteristic of each video feature cluster, by utilizing an efficient low-rank solution algorithm, such as the SVD threshold algorithm, the method can learn the normal behavior pattern group sparse dictionary; second, this method adaptively determines the number of dictionary bases for each normal behavior pattern, which can more accurately understand the semantics of dynamic scenes; third, different from traditional sparse representation methods, this method uses a dictionary Concentrate on selecting an appropriate dictionary to represent the test samples, making the sparse reconstruction of video events more accurate, significantly improving the detection speed and ensuring real-time performance.

Description of drawings

Figure 1 is an overview of adaptive anomaly event detection methods.

Fig. 2 is a flow chart of three-dimensional spatio-temporal gradient feature extraction.

Figure 3 shows multi-scale video frames.

Figure 4 is a three-dimensional gradient feature.

Figure 5 is a space-time cube overlaid on a spatial region.

Fig. 6 is a flow chart of group sparse dictionary learning.

Fig. 7 is a flow chart of abnormal event detection.

detailed description

The present invention will be described in further detail below in conjunction with the accompanying drawings.

The abnormal event detection method of the present invention includes three main processes: a feature extraction process, a training process and a testing process, as shown in FIG. 1 .

The feature extraction process is shown in Figure 2, including the following specific steps:

Video sequence image frames are converted into 3D image pyramid process 21 . Convert each frame of the video sequence to a grayscale image, and scale each frame of the grayscale image to three different scales: 20×20, 30×40 and 120×160, forming a three-level image pyramid. For each scale of the image pyramid, each frame is divided into non-overlapping regions of the same spatial size (10×10), as shown in Figure 3.

The process of extracting the 3D gradient feature of the video sequence 22. In order to take into account the morphological features and motion features of objects in dense scenes, the three-dimensional space-time gradient is used as the target of feature extraction, as shown in Figure 4. Assuming that x, y and t represent the horizontal, vertical and time directions in the video sequence respectively, G is a three-dimensional gradient of a pixel, and its corresponding three-dimensional projections are G _x , G _y and G _t respectively.

A spatiotemporal feature process 23 is formed using spatiotemporal cube sampling. The same spatial region on 5 consecutive frames forms a space-time cube, each space-time cube is composed of all pixels in a cube with a size of 10×10×5, l _x ×l _y =10×10 and l _t =5 represent the space-time The spatial size and timing length of the cube. The 3D gradients of all pixels in a spatiotemporal cube constitute a 3D spatiotemporal gradient feature. To preserve the timing information of scene events, temporally overlapping space-time cubes are sampled on each spatial region, as shown in Figure 5. Assuming that the body center of a space-time cube is (S _x ,S _y ,S _t ), then its temporally adjacent space-time cubes are located at (S _x ,S _y ,S _t -l _t /2) and (S _x ,S _y , S _t + l _t /2).

Form training feature set and test feature set process 24 by using spatio-temporal feature dimensionality reduction. Since the dimension of the above-mentioned three-dimensional gradient feature (10×10×5×3=1500) is too high, it will lead to too much calculation in the training process and testing process, which will affect the real-time performance of detection, so PCA is used to reduce the spatio-temporal feature to 100 dimensions . Finally, the training feature set and test feature set are obtained.

It should be noted that only normal events are included in the training video sequence; the test video sequence contains both normal events and abnormal events.

The training process refers to the process of learning normal behavior patterns through known normal behavior sample sets. Include the following specific steps:

The present invention adopts an iterative method to train all the features on each spatial position respectively, and obtain a group of sparse dictionary sets on each spatial position, which is used to represent all the training features on the spatial position. For all features at each spatial location, similar feature clusters are obtained using K-means, and then a low-rank approximation algorithm is used for dictionary learning for each similar feature cluster. Each iteration removes the features that can be represented by the dictionary, and then continues the dictionary learning of the next iteration for the remaining features until all the features can be represented by the trained dictionary.

Initialize training parameters process 61 . The remaining training feature set is initialized as the training feature set obtained by the feature extraction process, that is, X=[x ₁ ,x ₂ ,...,x _n ], where n is the number of features, m is the dimension of each feature, the regularization parameter τ=0.015, the error threshold T=0.01, the number of iterations j=1 and the normal mode dictionary set

The remaining training feature set clustering process 62 is performed using K-means. According to the number of remaining training features, first adaptively select the number of clusters, and then perform K-means clustering on the remaining feature sets. Assume that the feature representation of the ^cth cluster in the ^jth iteration is where j=1,2,...,N, Indicates the number of features of the c ^th cluster, and the function f(·) maps the serial number of the feature in the cluster to the serial number of the feature in the initial feature set.

As shown in FIG. 6 , the process 63 of learning a group sparse dictionary using a low-rank approximation algorithm. Different from the traditional sparse representation model, the present invention utilizes the low-rank structure of video feature clustering and adopts a low-rank approximation algorithm to learn group sparse dictionaries containing highly correlated dictionary bases, thereby discarding redundant information in video data. The objective function for group sparse dictionary learning is as follows:

in, Using singular value decomposition, τ is the regularization parameter, and ω _i is the weight of the ith singular value λ _i . The above weighted low-rank optimization problem adopts the singular value thresholding algorithm, and the closed solution of formula (1) is as follows:

in, r is operated by soft thresholding Estimated rank. The soft threshold operation acts on each singular value, the singular value smaller than τ·ω _i is set to 0, and the singular value greater than τ·ω _i remains the original value.

Select group sparse dictionary process 64 . Once determined for any feature The objective function is as follows:

in, express The sparse coefficient vector of , Express features Is it possible to be a dictionary Indicates that the constraint items ∑γ _i,j =1 and γ _i,j ={0,1} are used to ensure that only one dictionary can be selected to represent represent a dictionary Represent feature clustering A vector of coefficients, T represents the error threshold. The closed solution of γ is as follows:

if can express then the dictionary Add the dictionary set D, retain the singular value information corresponding to the dictionary, so that the number of iterations j=j+1, and the sparse coefficient Characteristics Removed from the remaining feature set X ^j ; if Cannot represent any of the remaining features then discard the dictionary

Processes 63 and 64 are iteratively performed until the remaining feature set is an empty set.

Finally, a set of sparse dictionaries of normal features is obtained, each dictionary is used to represent a normal behavior pattern. Note that since the video sequences corresponding to the training feature set only contain normal events, the learned dictionaries all represent normal behavior patterns.

As shown in Figure 7, the test process refers to the process of detecting whether the test sample is an abnormal sample from the normal pattern dictionary set learned through the training process. Include the following specific steps:

Initialize the test parameters process 71 . Initialize the dictionary set of normal behavior patterns as the dictionary set obtained by the training process, that is, D=[D ₁ , D ₂ ,...D _N ], and the reconstruction error threshold is .

Compute the reconstruction error process 72 for the test features. For any test feature x, calculate the reconstruction error for each dictionary D _k , the calculation method is as follows:

The optimal solution of β _k in formula (5) is:

At this point the reconstruction error is:

Judging whether the test feature is an abnormal feature process 73 . A suitable dictionary is searched to represent the test sample, and then the reconstruction error is used to judge whether the test sample is an abnormal event. If the reconstruction error is less than the reconstruction error threshold, it indicates that the test feature x can be represented by the dictionary _Dk , that is, the test feature x belongs to a normal event; otherwise, the test feature x cannot be represented by the dictionary _Dk . If none of the dictionaries in the normal pattern dictionary set can represent the test feature x, the test feature x is an abnormal event.

It should be pointed out here that, compared with the most advanced algorithms at present, the present invention adopts an iterative low-rank approximation method, which improves the detection accuracy by at least 2%. Through the method of retrieving the normal mode dictionary set, the detection speed of the present invention can be increased by more than 20 times compared with the traditional anomaly detection method. In addition, compared with the traditional sparse representation method, the present invention can reduce the training time by at least 10 times by adopting the SVD threshold algorithm.

Claims (6)

1. An abnormal event detection method based on low-rank approximation structured sparse representation is characterized by comprising the following steps of: the method comprises three processes of feature extraction, training and testing:

the feature extraction process comprises the following steps:

1) extracting multi-scale three-dimensional gradient features of a video sequence;

2) reducing the dimension of the multi-scale three-dimensional gradient feature to form a training feature set and a testing feature set;

the training process comprises the following steps:

3) initializing residual training characteristics and related parameters;

4) performing iterative learning on the residual training features to form a sparse dictionary, and obtaining a normal mode dictionary set;

the test process comprises the following steps:

5) carrying out sparse reconstruction on the test features by utilizing a group sparse dictionary set obtained in the training process;

6) and judging whether the test feature is an abnormal feature or not according to the reconstruction error.

2. The method for detecting the abnormal events based on the low-rank approximation structured sparse representation as claimed in claim 1, wherein: the step 1) comprises the following specific steps:

1.1) zooming each frame image of the video sequence in different scales to form a three-layer image pyramid;

1.2) performing space-time cube sampling on each layer of image, and extracting three-dimensional gradient features of non-overlapped regions in space;

1.3) for each layer of video sequence, superposing three-dimensional gradient features of 5 continuous frames in the same spatial region together to form a space-time feature.

3. The abnormal event detection method based on the low-rank approximation structured sparse representation as claimed in claims 1 and 2, wherein: the step 2) comprises the following specific steps:

2.1) carrying out dimensionality reduction on each extracted space-time feature by using Principal Component Analysis (PCA);

2.2) converting the training video sequence and the test video sequence into a training characteristic set and a test characteristic set by using the method.

4. The method for detecting abnormal events based on the low-rank approximation structured sparse representation according to claim 1 or 3, wherein: the step 3) comprises the following specific steps:

3.1) initializing the training feature set obtained in the step 2.2) into a residual training feature set;

3.2) initializing a regularization parameter, an error threshold, iteration times and a normal pattern dictionary set.

5. The method for detecting abnormal events based on the low-rank approximation structured sparse representation as claimed in claim 1, wherein: the step 4) comprises the following specific steps:

assuming that the number of iterations j is 1, the remaining training features are denoted as X^j＝[x₁,x₂,...,x_n]Whereinn is the number of features, m is the dimension of each feature, the regularization parameter is τ, the normal pattern dictionary set is

4.1) determining the number of clusters C^jFor the remaining feature set X^jPerforming K-means clustering, at this time j^thC in the sub-iteration^thThe characteristics of the individual clusters are represented asWherein j is 1, 2., N,denotes the c^thThe feature number of each cluster, and a function f (-) maps the feature serial number in each cluster to the serial number of the feature in the initial feature set;

4.2) using formula (1) and formula (2):

$(D_c^j,{\mathrm{\Λ}}_c^j,V_c^j)=\arg \underset{D,\mathrm{\Λ},V}{min}\left|\right|X_c^j-D_c^j{\mathrm{\Λ}}_c^j{\left(V_c^j\right)}^T|{|}_2^2+\mathrm{\τ}\underset{i=1}{\overset{n_c^j}{\Σ}}{\mathrm{\ω}}_i{\mathrm{\λ}}_i---\left(1\right)$

$\left\{\begin{array}{c}(D_c^j,{\mathrm{\Λ}}_c^j,V_c^j)=svd\left(X_c^j\right)\\ {\widehat{\mathrm{\λ}}}_i=S_{\mathrm{\τ}\·{\mathrm{\ω}}_i}\left({\mathrm{\λ}}_i\right)\end{array}\right.---\left(2\right)$

performing dictionary learning on all the clusters of the residual features to obtain the jth^thIn the next iteration C^jDictionary set for individual feature clusteringWherein,using singular value decomposition, τ being a regularization parameter, ω_iIs the ith singular value λ_iThe weight of (a) is determined,r is operated by soft thresholdAn estimated rank; soft threshold operation on each singular value, less than τ · ω_iIs set to 0 and is greater than tau omega_iThe singular value of (A) still keeps the original value;

4.3) Once determinedFor any remaining featureSelecting a dictionary by using formula (3) and formula (4)To represent the remaining features;

$\begin{array}{c}\underset{\mathrm{\γ}}{\min }\underset{i=1}{\overset{n_c^j}{\Σ}}{\mathrm{\γ}}_{f(i,c)}^j\left(\right||x_{f(i,c)}^j-D_c^j{\mathrm{\β}}_{f(i,c)}^j|{|}_2^2-T)\\ s.t.\underset{j=1}{\overset{N}{\Σ}}{\mathrm{\γ}}_{f(i,c)}^j=1,{\mathrm{\γ}}_{f(i,c)}^j=\{0,1\}\end{array}---\left(3\right)$

wherein,to representThe sparse coefficient vector of (a) is,representation featureWhether or not to be able to be usedRepresents, constraint term ∑ gamma_i,j1 and γ_i,jWith {0,1} being used to ensure that only one dictionary can be picked for representation Dictionary for representationRepresenting feature clustersT denotes an error threshold;

if it is notCan representThen the dictionaryAdding dictionary set D and thinning out coefficientIs characterized byFrom the remaining feature set X^jRemoving; if it is notMay not represent any of the remaining featuresThen the dictionary is discarded

4.4) the number of iterations plus 1, j ═ j + 1;

4.5) when the residual feature set is an empty set, the group sparse dictionary learning process is ended.

6. The method for detecting abnormal events based on the low-rank approximation structured sparse representation as claimed in claim 1, wherein: the step 5) comprises the following specific steps:

suppose the normal pattern dictionary set is D ═ D₁,D₂,...D_N]；

5.1) given a test feature x, for a dictionary D_k(k ═ 1, 2.. N) the reconstruction error was calculated:

$\underset{{\mathrm{\β}}_k}{min}\left|\right|x-D_k{\mathrm{\β}}_k|{|}_2^2---\left(5\right)$

5.2) solving β in equation (5)_kThe optimal solution of (2):

${\widehat{\mathrm{\β}}}_k={\left({D_k}^T{D}_k\right)}^{-1}{D_k}^{T}x---\left(6\right)$

5.3) calculating the reconstruction error of the test feature x by using the formula (7):

$E_k=\left|\right|x-D_k{\widehat{\mathrm{\β}}}_k|{|}_2^2=||x-D_k{\left({D_k}^T{D}_k\right)}^{-1}{D_k}^{T}x|{|}_2^2---\left(7\right)$

if the reconstruction error is less than the reconstruction error threshold, it indicates that the test feature x can be stored in the dictionary D_kIndicating that the test feature x belongs to a normal event; otherwise, the test feature x cannot be used by dictionary D_kRepresenting that k is k +1, return to step 5.1);

5.4) if all dictionaries in the normal pattern dictionary set can not represent the test feature x, the test feature x belongs to the abnormal event.