CN104820824A - Local abnormal behavior detection method based on optical flow and space-time gradient - Google Patents

Abstract

The invention discloses a local abnormal behavior detection method based on an optical flow and a space-time gradient. The method comprises following steps: dividing video images into space-time blocks, detecting a zone, where an abnormal behavior is most likely to happen, by utilizing a statistical method based on a semi-parametric model, and determining whether the abnormal behavior really exists in a suspicious zone by utilizing a cell maximum optical flow energy method and a local nearest neighbor descriptor. An experiment in a general UCSD data set shows that the method provided by the invention can effectively improve accuracy and rapidity of detection of the local abnormal behavior.

Description

Based on the local anomaly behavioral value method of light stream and spatio-temporal gradient

Technical field

The present invention relates to a kind of local anomaly behavioral value method, particularly relate to a kind of local anomaly behavioral value method based on light stream and spatio-temporal gradient, belong to technical field of image processing.

Background technology

In field of intelligent video surveillance, crowd's unusual checking technology is one of key subject, and its main task from monitor video, attempts automatic screening go out various types of anomalous event, and security protection personnel can be reminded in time to tackle process.

At present, crowd's anomaly detection method is mainly divided into following two classes: the first kind is the method that based target is followed the tracks of, and Equations of The Second Kind is the method based on population characteristic.First first kind method follows the tracks of each target, then extracts the movable information comprising movement locus, and the last movable information according to extracting realizes the detection of abnormal behaviour.Owing to easily blocking between crowd, therefore, the target following in complex scene is still a difficult problem.Equations of The Second Kind method is by as a whole for whole scene visual, and extracts useful movable information, thus carries out the detection of abnormal behaviour.Due to the Optic flow information of whole scene need be calculated, thus, calculated amount is very large.And the present invention can solve problem above well.

Summary of the invention

The object of the invention is to propose a kind of new local anomaly behavioral value method, the method increases accuracy and the rapidity of unusual checking.First, utilize word bag model to characterize light stream characteristic, and realized the detection in local anomaly region by the statistical method of semi-parameter model; Then, on the basis that abnormal area is divided equally, utilize the cell that maximum light stream energy finds abnormal behaviour to occur; Finally, utilize arest neighbors descriptor and mix model-naive Bayesian, realizing the final judgement of abnormal behaviour.

The present invention solves the technical scheme that its technical matters takes: a kind of local anomaly behavioral value method based on light stream and spatio-temporal gradient, the method comprises the steps:

Step 1: according to space-time characterisation, video image is divided into space-time block of the same size;

Step 2: utilize the statistical method based on semi-parameter model, detects the most possible region that abnormal behaviour occurs;

Step 3: the maximum light stream energy method of range site lattice and local arest neighbors descriptor, realize the detection of suspicious region abnormal behaviour.

Unusual checking of the present invention, based on the unusual checking of population characteristic, is by as a whole for whole scene visual, and extracts useful movable information, thus carry out the detection of abnormal behaviour.

The abnormal area testing process of semi-parameter model of the present invention, comprising:

First, frame of video is divided into equal-sized region S, and obtains region S mono-stack features and represent X; Then according to following formula, whether judging area S is abnormal area:

$\mathrm{\λ}\left(S\right)=\frac{\mathrm{Pr}\left(x\right|H_1\left(S\right))}{\mathrm{Pr}\left(x\right|H_0\left(S\right))}---\left(1\right)$

Pr (X|H in above formula ₀(S)) represent that region S is the likelihood ratio of character representation X under normal region, Pr (X|H ₁(S)) represent that region S is the likelihood ratio of character representation X under abnormal area.

For accurately calculating the likelihood ratio in each region, a suitable probability model Pr (X|H need be built _i).Different from the specific non-parametric density probability model of hypothesis one, the present invention adopts the density probability model based on half parameter to calculate likelihood ratio.

The modeling process of half non-parametric density probability of the present invention, comprising:

First, by the characteristic present X inside and outside the S of region ₁with X ₂and probability density function f (x) of correspondence is expressed as with g (x):

$\left\{\begin{array}{c}{x}_1={(x_{11},x_{12},......,x_{{1n}_1})}^T~f\left(x\right)\\ x_2={(x_{21},x_{22},......,x_{{2n}_2})}^T~g\left(x\right)\\ \frac{f\left(x\right)}{g\left(x\right)}=\exp (\mathrm{\α}+{\mathrm{\β}}^{T}h\left(x\right))\end{array}\right.---\left(2\right)$

N in above formula ₁and n ₂represent the size inside and outside the S of region respectively, h (x) is pre-defined function.

Then, half parameter likelihood of sample can be expressed as:

$L(\mathrm{\α},\mathrm{\β},G)=\underset{i=1}{\overset{n}{\mathrm{\Π}}}{p}_i\underset{j=1}{\overset{n_1}{\mathrm{\Π}}}\exp (\mathrm{\α}+{\mathrm{\β}}^{T}h\left(x_{1j}\right))---\left(3\right)$

N=n in above formula ₁+ n ₂, p _i=dG (t _i)=Pr (X=t _i), t=(t ₁, t ₂..., t _n) ^t=(x ₁₁, x ₁₂..., x _1n1, x ₂₁, ..., x _2n2) ^t, G (x) is the cumulative distribution function of g (x).

Finally, calculate the likelihood value under f (x)=g (x) and f (x) ≠ g (x) condition respectively, thus draw the likelihood value in formula (1).

This have following two major advantages based on half non-parametric density model method: the first, utilizes mark sheet to collect t and estimate α, β and distribution thereof, this is because when window size is very little, estimates that α, β distribution is infeasible respectively; The second, this method does not need to suppose specific parameter probability model, although this is because need select the type of h (x), can select dissimilar for different distributions.

The characteristic extraction procedure that abnormal area of the present invention detects, comprising:

First, on the basis to the average grid division of every two field picture, extract the Optical-flow Feature of each grid;

Then, cluster is carried out to the Optical-flow Feature (that is: 8 direction value and 1 velocity amplitude) of all grids, and each classification is considered as a visual word;

Finally, visual word histogram is utilized to represent video information.View-based access control model word histogram represents video method, not only compressible image information, and can retain the local feature of interested picture frame.

The likelihood ratio defined from formula (1), the λ (S) in the λ (S) in abnormal behaviour region always normal behaviour region is much larger.Therefore, selected threshold T based on experience value: if λ (S) >T, then this regional determination is abnormal behaviour region.

But due to the complicacy of human body behavior campaign, the λ (S) in some normal behaviour regions also can be very large.Therefore, further local anomaly behavior need be carried out to local abnormal area to determine, to reduce false drop rate.

Generally, relative normal behaviour, abnormal behaviour has the obvious characteristics such as movement velocity is fast, direction of motion is chaotic usually, and thus its light stream energy is also generally high than the light stream energy of normal behaviour.Based on this, local anomaly behavioral value process of the present invention comprises: utilize the arest neighbors descriptor of mixing model-naive Bayesian to normal cell lattice to train, then carry out analysis abnormal behaviour according to arest neighbors descriptor to test cell lattice to judge, its concrete steps are shown in algorithm:

Abnormal area is on average divided into several W × H cells, and calculates the light stream energy in each cell region;

According to maximum light stream energy model method, abnormality detection is carried out to the cell of maximum light stream energy: if there are abnormal conditions, then continue to perform next step; Otherwise then think that this abnormal area does not exist abnormal behaviour, this detects end;

Mixing model-naive Bayesian is utilized to carry out abnormality detection to four neighborhood cells of anomaly unit lattice respectively: if exist abnormal, then to perform next step; Otherwise then think that this abnormal area does not exist abnormal behaviour, this detects end;

To anomaly unit lattice, red-label is utilized to position.

The acquisition process of arest neighbors descriptor of the present invention, comprising:

First, every width image is divided several h × w cells;

Then, the spatio-temporal gradient amplitude v of each pixel in unit lattice is calculated respectively _ij, and the variance M of each pixel spatio-temporal gradient amplitude ₂, degree of bias M ₃with kurtosis M ₄:

$\left\{\begin{array}{c}{M}_r=\left[m_{i,j}\right],i=\mathrm{1,2},...,h,j=\mathrm{1,2},...,w\\ m_{i,j}=\frac{1}{h*w}{\mathrm{\Σ}}_{i,j}{\left(v_{\mathrm{ij}}\right)}^r,r=\mathrm{2,3,4}\end{array}\right.---\left(4\right)$

V in above formula _ijrepresent the spatio-temporal gradient amplitude of (i, j) pixel.

Next, by matrix M _rbe converted to vector and merge into vector further

M＝[m ₂m ₃m ₄] (5)

Finally, according to the distance between cell S and S ';

$d(V_s,V_{s^{\′}})=\underset{\mathrm{\υ}}{\mathrm{\Σ}}{2}^{-2\mathrm{\υ}}{\mathrm{WAV}}_{\mathrm{\υ}}\left(\right|M_s|-|M_{s^{\′}}\left|\right)---\left(6\right)$

K arest neighbors on space-time is searched in sequence of video images:

X ^sd＝[d ₁,d ₂,...,d _k,...,d _K] ^T(7)

D in above formula _krepresent the distance between cell and its kth neighborhood.

The present invention is only containing in the training video of normal behaviour, to the arest neighbors descriptor training mixing model-naive Bayesian of each cell, comprising:

Choose mixing member vectors π ~ Dirichlet (ζ);

For each non-disappearance feature x of X _j:

Choose a composition z _j=c ~ dicrete (π);

Choose an eigenwert x _j~ p _{Ψ j}(x _j| θ _jc), wherein Ψ _jand θ _jcan ED~* class of common decision feature j and composition c;

By the training of normal behaviour video, obtain the mixing model-naive Bayesian of each normal behaviour cell, comprising:

In setting models parameter with Ω=(μ _jc, σ _jc, [j] ₁ ^d, [c] ₁ ^k) condition under, mixing model-naive Bayesian probability density be:

μ in above formula _jc, σ ² _jcbe respectively average and the variance of c the composition of a jth Gauss;

Given one group of training set X=[X ₁, X ₂..., X _l], the optimized parameter of mixing model-naive Bayesian and Ω ^*by maximizing whole data set likelihood try to achieve:

At study optimized parameter and Ω ^*process in, adopt a kind of EM algorithm of quick variation, to obtain mixing model-naive Bayesian expression-form fast;

At test phase, utilize the mixing model-naive Bayesian learning to obtain, calculate the log-likelihood l=log p (X| α, Ω) of local arest neighbors descriptor, and the X meeting following formula is considered as exception:

l＜T (10)

T in above formula is the threshold value preset.

Accompanying drawing explanation

Fig. 1 is method flow diagram of the present invention.

Fig. 2 (a) is the normal behaviour from ed1 data set, and (b)-(c) is the local anomaly behavior from Ped1 data set.

Fig. 3 (a) is the normal behaviour from Ped2 data set, and (b)-(c) is the local anomaly behavior from Ped2 data set.

Fig. 4 is the part unusual checking result of Ped1.

Fig. 5 is the part unusual checking result schematic diagram of Ped2.

Fig. 6 is the ROC curve synoptic diagram of the Pixel-level of Ped1 data set unusual checking.

Fig. 7 is the ROC curve synoptic diagram of the Pixel-level of Ped2 data set unusual checking.

Embodiment

Below in conjunction with Figure of description, the invention is described in further detail.

As shown in Figure 1, the invention provides a kind of based on new local anomaly behavioral value method, video image is divided into space-time block by the method; The statistical method based on semi-parameter model is utilized to detect the most possible region that abnormal behaviour occurs; The maximum light stream energy method of range site lattice and local arest neighbors descriptor, confirm suspicious region whether necessary being abnormal behaviour.

In the present invention, the size h of split window and w is respectively 60 and 40, and the space-time Neighborhood Number K of arest neighbors descriptor is taken as 8, and wavelet transform dimension value υ is taken as 8, and in mixing model-naive Bayesian, threshold value T is taken as-1.25.

The present invention adopts the UCSD common data sets comprising Ped1 and Ped2 two subsets as the detected set of local anomaly event, wherein the resolution of Ped1 data subset is 238 × 158, the resolution being re-set as 240 × 160, Ped2 data set in experiment is 360 × 240.In addition, Ped1 data set comprises 34 training video sequences and 36 test video sequence, and each video sequence comprises 200 frames; Ped2 data set comprises 16 training video sequences and 12 test video sequence, and the frame number of each video sequence is 120,150 or 180.

The normal event of UCSD common data sets is defined as pedestrian and normally walks, and local anomaly event definition is for occurring the improper walking events such as slide plate, bicycle, automobile.Fig. 2 and Fig. 3 provides the example of Ped1 and Ped2 two data centralization normal behaviours and local anomaly behavior respectively.

For the effect of accurate evaluation local anomaly behavioral value, the present invention adopts the evaluation criterion based on Pixel-level.

Pixel scale: for accurately representing unusual checking result, often the actual conditions of testing result and Pixel-level are compared.If unusual checking area pixel in certain frame, be no less than 40% of actual abnormal behaviour region pixel, then judge that this frame contains abnormal behaviour.

Performance Evaluating Indexes is recipient's operating characteristic curve (the receiver operating characteristic curve based on Pixel-level, or be ROC curve), it is True Positive Rate (True Positive Rate, and the overall target of false positive rate (False Positive Rate, FPR) TPR).Area below ROC curve is larger, illustrates that the accuracy detected is higher.On ROC curve, the critical value the closer to upper left expression TPR and FPR is higher.

Interpretation:

In the present invention, local anomaly behavioral value result red rectangle frame is marked.Fig. 4 and Fig. 5 provides the part unusual checking result of Ped1 and Ped2 data set respectively.As can be seen from result shown in Fig. 4 and Fig. 5, institute of the present invention extracting method can detect dissimilar local anomaly well, and the middle abnormal behaviour such as automobile, the bicycle of the middle appearance of Fig. 5 (b)-(d), the slide plate of the middle appearance of Fig. 5 (d) occurred of the bicycle of appearance in the wheelchair of appearance as middle in Fig. 4 (a), the slide plate of the middle appearance of Fig. 4 (b), Fig. 4 (c), the automobile of the middle appearance of Fig. 4 (d), Fig. 5 (a) can accurately detect.Issued by experimental result, also there is detection leakage phenomenon for abnormal behaviour slowly of moving, such as pedestrian walks one's bicycle and slowly walks.

Fig. 6 and Fig. 7 provides respectively and comprises based on the method (Social Force) of societal forces model, based on the method (MDT) of mixing dynamic texture feature, the ROC curve based on four kinds of method Pixel-level of dynamic optical stream energy method (AOF), institute of the present invention extracting method.As can be seen from result shown in Fig. 6 and Fig. 7, the inventive method is better than other three kinds of methods.

Under table 1 provides four kinds of methods, ROC area under a curve (area under curve, AUC) result.Result can be found out as shown in Table 1, and the inventive method is better than other three kinds of methods.

Table 1: the AUC between distinct methods

Method	Ped1	Ped2	Average
				Social Force	17.9％	26％	21.95％
MDT	44.1％	50％	47.05％
				AOF
Our Method	51.3％	56％	53.65％

Claims (10)

1., based on a local anomaly behavioral value method for light stream and spatio-temporal gradient, it is characterized in that, described method comprises the steps:

Step 1: according to space-time characterisation, video image is divided into space-time block of the same size;

Step 2: utilize the statistical method based on semi-parameter model, detects the most possible region that abnormal behaviour occurs;

Step 3: the maximum light stream energy method of range site lattice and local arest neighbors descriptor, realize the detection of suspicious region abnormal behaviour.

2. a kind of local anomaly behavioral value method based on light stream and spatio-temporal gradient according to claim 1, it is characterized in that, described unusual checking is the unusual checking based on population characteristic, by as a whole for whole scene visual, and extracts useful movable information.

3., according to claim 1 based on the local anomaly behavioral value method of light stream and spatio-temporal gradient, it is characterized in that, the abnormal area testing process of the semi-parameter model of described method comprises:

First, frame of video is divided into equal-sized region S, and obtains region S mono-stack features and represent X; Then according to following formula, whether judging area S is abnormal area:

$\mathrm{\λ}\left(S\right)=\frac{\mathrm{Pr}\left(x\right|H_1\left(S\right))}{\mathrm{Pr}\left(x\right|H_0\left(S\right))}---\left(1\right)$

Pr (X|H in above formula ₀(S)) represent that region S is the likelihood ratio of character representation X under normal region, Pr (X|H ₁(S)) represent that region S is the likelihood ratio of character representation X under abnormal area;

Build probability model Pr (X|H _i); The density probability model based on half parameter is adopted to calculate likelihood ratio.

4. a kind of local anomaly behavioral value method based on light stream and spatio-temporal gradient according to claim 1, it is characterized in that, the modeling process of half non-parametric density probability of described method comprises:

First, by the characteristic present X inside and outside the S of region ₁with X ₂and probability density function f (x) of correspondence is expressed as with g (x):

$\left\{\begin{array}{c}{x}_1={(x_{11},x_{12},\·\·\·\·\·\·,x_{{1n}_1})}^T~f\left(x\right)\\ x_2={(x_{21},x_{22},\·\·\·\·\·\·,x_{{2n}_2})}^T~g\left(x\right)\\ \frac{f\left(x\right)}{g\left(x\right)}=\exp (\mathrm{\α}+{\mathrm{\β}}^{T}h\left(x\right))\end{array}\right.---\left(2\right)$

N in above formula ₁and n ₂represent the size inside and outside the S of region respectively, h (x) is pre-defined function;

Then, half parameter likelihood of sample can be expressed as:

$L(\mathrm{\α},\mathrm{\β},G)=\underset{i=1}{\overset{n}{\mathrm{\Π}}}{p}_i\underset{j=1}{\overset{n_1}{\mathrm{\Π}}}\exp (\mathrm{\α}+{\mathrm{\β}}^{T}h\left(x_{1j}\right))---\left(3\right)$

N=n in above formula ₁+ n ₂, p _i=dG (t _i)=Pr (X=t _i), t=(t ₁, t ₂..., t _n) ^t=(x ₁₁, x ₁₂..., x _1n1, x ₂₁, ..., x _2n2) ^t, G (x) is the cumulative distribution function of g (x);

Finally, calculate the likelihood value under f (x)=g (x) and f (x) ≠ g (x) condition respectively, thus draw the likelihood value in formula (1).

5. a kind of local anomaly behavioral value method based on light stream and spatio-temporal gradient according to claim 1, is characterized in that, the characteristic extraction procedure that the abnormal area of described method detects comprises:

First, on the basis to the average grid division of every two field picture, extract the Optical-flow Feature of each grid;

Then, to the Optical-flow Feature of all grids, that is: 8 direction value and 1 velocity amplitude carry out cluster, and each classification is considered as a visual word;

Finally, visual word histogram is utilized to represent video information; View-based access control model word histogram represents video method, not only compressible image information, and can retain the local feature of interested picture frame.

6. a kind of local anomaly behavioral value method based on light stream and spatio-temporal gradient according to claim 1, is characterized in that, the λ (S) in the λ (S) in the abnormal behaviour region of described method always normal behaviour region is much larger; Selected threshold T based on experience value: if λ (S) >T, then this regional determination is abnormal behaviour region; Local anomaly behavior need be carried out to local abnormal area to determine.

7. a kind of local anomaly behavioral value method based on light stream and spatio-temporal gradient according to claim 1, it is characterized in that, described method local anomaly behavioral value be utilize the mixing arest neighbors descriptor of model-naive Bayesian to normal cell lattice to train, then carry out analysis abnormal behaviour according to arest neighbors descriptor to test cell lattice to judge, comprising:

Abnormal area is on average divided into several W × H cells, and calculates the light stream energy in each cell region;

According to maximum light stream energy model method, abnormality detection is carried out to the cell of maximum light stream energy: if there are abnormal conditions, then continue to perform next step; Otherwise then think that this abnormal area does not exist abnormal behaviour, this detects end;

Mixing model-naive Bayesian is utilized to carry out abnormality detection to four neighborhood cells of anomaly unit lattice respectively: if exist abnormal, then to perform next step; Otherwise then think that this abnormal area does not exist abnormal behaviour, this detects end;

To anomaly unit lattice, red-label is utilized to position.

8. a kind of local anomaly behavioral value method based on light stream and spatio-temporal gradient according to claim 1, it is characterized in that, the acquisition process of the arest neighbors descriptor of described method comprises:

First, every width image is divided several h × w cells;

Then, the spatio-temporal gradient amplitude v of each pixel in unit lattice is calculated respectively _ij, and the variance M of each pixel spatio-temporal gradient amplitude ₂, degree of bias M ₃with kurtosis M ₄:

$\left\{\begin{array}{c}{M}_r=\left[m_{i,j}\right],i=\mathrm{1,2},...,h,j=\mathrm{1,2},...,w\\ m_{i,j}=\frac{1}{h*w}{\mathrm{\Σ}}_{i,j}{\left(v_{\mathrm{ij}}\right)}^r,r=\mathrm{2,3,4}\end{array}\right.---\left(4\right)$ V in above formula _ijrepresent the spatio-temporal gradient amplitude of (i, j) pixel;

Next, by matrix M _rbe converted to vector and merge into vector further

M＝[m ₂m ₃m ₄] (5)

Finally, according to the distance between cell S and S ';

$d(V_s,V_{s^{\′}})=\underset{\mathrm{\υ}}{\mathrm{\Σ}}{2}^{-2\mathrm{\υ}}\mathrm{WA}{V}_{\mathrm{\υ}}\left(\right|M_s|-|M_{s^{\′}}\left|\right)---\left(6\right)$

K arest neighbors on space-time is searched in sequence of video images:

X ^sd＝[d ₁,d ₂,…,d _k,…,d _K] ^T(7)

D in above formula _krepresent the distance between cell and its kth neighborhood.

9. according to claim 1 based on the local anomaly behavioral value method of light stream and spatio-temporal gradient, it is characterized in that, described method is only containing in the training video of normal behaviour, to the arest neighbors descriptor training mixing model-naive Bayesian of each cell, comprising:

Choose mixing member vectors π ~ Dirichlet (ζ);

For each non-disappearance feature x of X _j:

Choose a composition z _j=c ~ dicrete (π);

Choose an eigenwert x _j~ p _{Ψ j}(x _j| θ _jc), wherein Ψ _jand θ _jcan ED~* class of common decision feature j and composition c;

By the training of normal behaviour video, obtain the mixing model-naive Bayesian of each normal behaviour cell, comprising:

In setting models parameter with Ω=(μ _jc, σ _jc, [j] ₁ ^d, [c] ₁ ^k) condition under, mixing model-naive Bayesian probability density be:

μ in above formula _jc, be respectively average and the variance of c the composition of a jth Gauss;

Given one group of training set X=[X ₁, X ₂..., X _l], the optimized parameter of mixing model-naive Bayesian and Ω ^*by maximizing whole data set likelihood try to achieve:

At study optimized parameter and Ω ^*process in, adopt a kind of EM algorithm of quick variation, to obtain mixing model-naive Bayesian expression-form fast;

At test phase, utilize the mixing model-naive Bayesian learning to obtain, calculate the log-likelihood l=log p (X| α, Ω) of local arest neighbors descriptor, and the X meeting following formula is considered as exception:

l＜T (10)

T in above formula is the threshold value preset.

10. according to claim 1 based on the local anomaly behavioral value method of light stream and spatio-temporal gradient, it is characterized in that, described method comprises: first, utilizes word bag model to characterize light stream characteristic, and is realized the detection in local anomaly region by the statistical method of semi-parameter model; Then, on the basis that abnormal area is divided equally, utilize the cell that maximum light stream energy finds abnormal behaviour to occur; Finally, utilize arest neighbors descriptor and mix model-naive Bayesian, realizing the final judgement of abnormal behaviour.