CN103679215A - Video monitoring method based on group behavior analysis driven by big visual big data - Google Patents

Abstract

A video monitoring method achieved by a computer comprises the steps of receiving video data captured by a vidicon; establishing a group behavior model according to the received video data; estimating parameters of the group behavior model to obtain multiple crowd behaviors existing in a scene; using the obtained group behavior model to obtain behavior feature sets of different crowds; converting the obtained behavior feature sets and using the converted behavior feature sets to obtain statistical people number values according to all of crowd behaviors. The vidicon has general applicability in angle. The video monitoring method can be used for people counting at open entrances and exits, is small in calculation quantity and can meet the real-time video processing requirement.

Description

The video frequency monitoring method of the group behavioural analysis based on the large data-driven of vision

Technical field

The present invention relates to a kind of video frequency monitoring method, particularly a kind of video frequency monitoring method of the group behavioral analysis technology based on the large data-driven of vision.

Background technology

Most of traditional supervisory systems need special monitor staff to carry out artificial judgment for monitor video.This need to expend a large amount of manpowers, and people is absorbed in something for a long time, may neglect some abnormal conditions, thereby bring negative consequence.Intelligent video monitoring system can be identified different objects, during abnormal conditions in finding monitored picture, can in fast and the most best mode, give the alarm and useful information is provided, thereby can more effectively assist monitor staff obtain accurate information and process accident, and reduce to greatest extent wrong report and fail to report phenomenon.

In correlation technique, according to the difference of crowd behaviour detection method, video frequency monitoring method can be divided into two classes.The many people behavior recognition methods of one class methods based on motion tracking, the method is subject to the challenge of number in crowd.When number is more, serious shielding, cannot carry out single tracking, therefore can only be applied to the situation that scene is simple and number is few.The crowd behaviour recognition methods of Equations of The Second Kind method based on feature learning or structure behavior model, is used in crowd abnormal behaviour and detects, as crowd massing, crowd scatter and crowd is run and the abnormal behaviour such as crowd fighting etc.The method is more suitable for many scenes in people, by extracting feature, sets up model, and uses machine learning method to obtain model parameter, is conducive to improve verification and measurement ratio.But model can not be described all behaviors, so the model different to specific behavior needs.The sample that lacks in training in addition still brings challenges to obtaining optimum model parameter.

Summary of the invention

The object of this invention is to provide a kind of video frequency monitoring method, can detect and identify crowd behaviour, and add up the number that different rows is crowd.

To achieve these goals, a kind of video frequency monitoring method can comprise step:

1) receive the video data of being caught by video camera;

2) according to the video data receiving, set up group behavior model;

3) estimate the parameter of described group behavior model, obtain the multiple crowd behaviour existing in scene;

4) use the group behavior model obtaining to obtain the behavioural characteristic collection of different crowd;

5) the behavioural characteristic collection obtaining is changed, and come for often with the behavioural characteristic collection of conversion

Plant crowd behaviour and obtain people's numerical value of adding up.

According to technical scheme of the present invention, its advantage is: 1) mathematical model is simple, and parameter is few, and training is convenient; 2) can be used for crowded environment, calculate the semi-invariant of specific behavior number; 3) setting of camera angle has general applicability, can be for being opened to entrance demographics; 4) calculated amount is little, can meet the requirement that real-time video is processed.

Accompanying drawing explanation

Fig. 1 shows according to the process flow diagram of the video frequency monitoring method of the embodiment of the present invention;

Fig. 2 shows the word-document model structure according to the embodiment of the present invention;

Fig. 3 shows the field scene example according to the embodiment of the present invention;

Fig. 4 shows according to different crowd behavioural characteristic collection in the field scene of the embodiment of the present invention;

Fig. 5 shows the geometry correction schematic diagram according to the embodiment of the present invention;

Fig. 6 shows the on-the-spot garden number obtaining according to the embodiment of the present invention and changes example.

Embodiment

For making the object, technical solutions and advantages of the present invention clearer, below in conjunction with specific embodiment, and with reference to accompanying drawing, the present invention is described in more detail.

It should be noted that, in accompanying drawing or instructions description, similar or identical part is all used identical figure number.The implementation that does not illustrate in accompanying drawing or describe is form known to a person of ordinary skill in the art in affiliated technical field.In addition, although the demonstration of the parameter that comprises particular value can be provided herein, should be appreciated that, parameter is without definitely equaling corresponding value, but can in acceptable error margin or design constraint, be similar to corresponding value.In addition, the direction term of mentioning in following examples, such as " on ", D score, 'fornt', 'back', " left side ", " right side " etc., be only the direction with reference to accompanying drawing.Therefore, the direction term of use is to be not used for limiting the present invention for illustrating.

According to technical scheme of the present invention, first, for scene crowd's complicacy, with group behavior model, excavate the multiple behavior in scene; Then, the K class crowd behaviour according to detecting, obtains behavioural characteristic collection for every class crowd; Then, the behavioural characteristic collection obtaining is converted to for example 5 dimensional feature vectors, to reduce intrinsic dimensionality, and by parameter correlation time, obtains a 5*G dimensional feature vector; Then, utilize the 5*G dimensional feature vector training of human artificial neural networks obtaining, thereby add up the semi-invariant of every class crowd's behavior number.The whole technical scheme process flow diagram of the embodiment of the present invention as shown in Figure 1.Below embodiments of the invention are elaborated.

Step 1: receive the video data being obtained by video camera, and can carry out such as processing such as denoisings the video data receiving.

Step 2: the video data based on receiving is set up group behavior model.

Due to the complicacy of crowd behaviour, conventionally in a scene, there is different crowd behaviours, be difficult to all behaviors of single model description.Therefore can obtain by group behavior model the feature set of every kind of behavior, and usage behavior feature set is done population analysis.This group behavior model can be a kind of word-document model, that is: low-level image feature is as word, and video segment is as document, thereby the crowd behaviour in excavation video implies theme, and obtains the feature set of every kind of crowd behaviour, i.e. low-level image feature set.

The model low-level image feature that the embodiment of the present invention adopts is local motion information.For example, can obtain motion pixel by frame difference method, then use optical flow method (Horn B K P, Schunck B G.Determining optical flow[J] .Artificial intelligence, 1981,17 (1): 185-203) calculate the velocity vector of motion pixel, and then obtain the feature of motion pixel, be i.e. position and movement velocity.Here, using each motion pixel as a word w _i, one section of video can comprise M two field picture, i.e. M document, and each document can represent by a word set, that is, and document W={w _i, i=1 ..., N}, wherein w _i={ x _i, y _i, u _i, v _i, N is the number of pixels in this frame of video, and x represents the horizontal level of pixel, and y represents the upright position of pixel, and u represents the speed of pixel along continuous straight runs, v represents pixel speed vertically.Certainly, those skilled in the art can adopt other known technologies in field of motion estimation to represent document W.

Fig. 2 shows word-document model structure that the embodiment of the present invention is used.Wherein, α represents the power relatively between implicit theme in collection of document, and β represents the probability distribution of all implicit themes self, stochastic variable π _jcharacterize document level j, stochastic variable π _jsize represent the proportion of each implicit theme in destination document.At word layer, z _jirepresent that destination document j distributes to the implicit theme share of each word i, x _jiit is the term vector representation of destination document.Suppose to have K behavior theme, each theme is the multinomial distribution of word, and α can be that the Dirichlet of corpus distributes.For each document j, Dirichlet distribution Dir (π _j| be α) with π _jfor parameter.For each the word i in document j, theme z _jiprobability distribution be π _jk, word x _jiabout parameter multinomial distribution.π wherein _jand z _jifor dependent variable, α and β are the parameters that needs optimization.When given α and β, stochastic variable π _j, theme z _j={ z _ji, word x _j={ x _jijoint probability distribution as shown in formula (1):

$\begin{array}{c}p(x_j,z_j,{\mathrm{\π}}_j|\mathrm{\α},\mathrm{\β})=p\left({\mathrm{\π}}_j\right|\mathrm{\α})\underset{i=1}{\overset{N}{\mathrm{\Π}}}p\left(z_{\mathrm{ji}}\right|{\mathrm{\π}}_j)p\left(x_{\mathrm{ji}}\right|z_{\mathrm{ji},}\mathrm{\β})\\ =\frac{\mathrm{\Γ}\left({\mathrm{\Σ}}_{k=1}^K{\mathrm{\α}}_k\right)}{{\mathrm{\Π}}_{k=1}^K\mathrm{\Γ}\left({\mathrm{\α}}_k\right)}{\mathrm{\π}}_{j1}^{{\mathrm{\α}}_1-1}\·\·\·{\mathrm{\π}}_{\mathrm{jk}}^{{\mathrm{\α}}_k-1}\underset{i=1}{\overset{N}{\mathrm{\Π}}}{\mathrm{\π}}_{{\mathrm{jz}}_{\mathrm{ji}}}{\mathrm{\β}}_{z_{\mathrm{ji}}{x}_{\mathrm{ji}}}\end{array}---\left(1\right)$

Therefore, the key problem that builds word-document model is the deduction that implicit variable distributes, and obtains the configuration information (π, z) of the inner implicit theme of destination document.Yet, due to posteriority distribution p (z _j, π _j| α, β) be not easy to calculate, can utilize the variation distribution of formula as follows (2) to be similar to this distribution:

$q(z_j,{\mathrm{\π}}_j|{\mathrm{\γ}}_j,{\mathrm{\φ}}_j)=q\left({\mathrm{\π}}_j\right|{\mathrm{\γ}}_j)\underset{i=1}{\overset{N}{\mathrm{\Π}}}q\left(z_{\mathrm{ji}}\right|{\mathrm{\φ}}_{\mathrm{ji}})---\left(2\right)$

Wherein, γ _jfor Dirichlet distribution q (π _j| γ _j) parameter, { φ _jibe multinomial distribution q (z _j| φ _j) parameter.(γ _j, φ _j) can be by calculating logp (x _j| α, β) maximal value obtain.

Step 3: estimate the parameter of group behavior model, obtain the various crowd behaviours that exist in scene.

Optimized parameter (α, β) can be by calculating logp (x _j| α, β) maximal value obtain, as shown in formula (3).

$({\mathrm{\α}}^{*},{\mathrm{\β}}^{*})=\underset{(\mathrm{\α},\mathrm{\β})}{\arg \max }\underset{j=1}{\overset{M}{\mathrm{\Σ}}}\log p\left(x_j\right|\mathrm{\α},\mathrm{\β})---\left(3\right)$

Equally due to p (x _j| α, β) be not easy direct calculating, can carry out estimated parameter (α, β) by a kind of maximal possibility estimation EM method of variation: in E-step, for each document j, find optimum variational parameter , use the variation distribution of the optimum variational parameter being obtained by E-step to be similar to above-mentioned formula (2), by two step cycle calculations, obtain optimized parameter (α ^*, β ^*).

As example, Fig. 3 shows certain two field picture of the video data of reception, wherein use the group behavior model of the embodiment of the present invention to excavate and under this scene, comprise for example four implicit themes (crowd behaviour), that is: move upward, move downward, to left movement, move right.

Step 4.: use the group behavior model obtaining to obtain different crowd behavioural characteristic collection.

Every two field picture in video all comprises different crowd behaviours, can use the parameter of the group behavior model obtaining in step 3, obtains the feature set of every kind of crowd behaviour, by word-document model shown in (4).

$\left\{\begin{array}{c}{f}_{k^{*}}=\left\{x_{k^{*}i}\right|i=1,...,F\}\\ k^{*}=\underset{k\∈\{1,...,K\}}{\arg \max }p(x_i,z_{k,i}|\mathrm{\α},\mathrm{\β})\end{array}\right.---\left(4\right)$

Wherein,

be the feature set of k* behavior, the number of feature in the feature set that F is k* behavior, x _kifor word is the feature of i pixel of k kind behavior.

Fig. 4 shows the crowd behaviour in scene, wherein utilize Optical-flow Feature point (only having shown Partial Feature point in image) to represent different behaviors, in figure, have Three Groups of Population behavior: the Based on Feature Points in rectangular area 1 moves upward, the Based on Feature Points in rectangular area 2 moves downward to the Based on Feature Points in left movement, rectangular area 3.

Step 5: the behavioural characteristic collection obtaining is changed, and obtained people's numerical value of statistics for every kind of behavior with the behavioural characteristic collection of conversion.

By group behavior model, obtained above the feature set of different crowd behaviour and every kind of behavior.Although behavioural characteristic collection also can be described behavior crowd's number, intrinsic dimensionality is higher, and the parameter training time is longer, and can not directly obtain accumulation number.Therefore, the method according to this invention, can be converted to 5 dimensional feature vectors by the behavioural characteristic collection of every two field picture, reduces thus intrinsic dimensionality.Meanwhile, time parameter can be added behavioural characteristic to concentrate, for each the behavioural characteristic collection that utilizes above-mentioned formula (4) to obtain, can obtain the proper vector NF={AS of a 5*G dimension _g, SV _g, DV _g, DD _g, NP _g, wherein G is time parameter, represents G frame, for adding up the semi-invariant of specific behavior number.Particularly, can utilize following methods to obtain above-mentioned 5*G dimensional feature vector:

(1) average velocity vector AS _g:

AS _g={ AS _g, g=1 ..., G}, wherein AS _gbe the average velocity of g two field picture, can shown in (5), obtain AS _g.

${\mathrm{AS}}_g=\frac{1}{F}\underset{i=1}{\overset{F}{\mathrm{\&Sigma;}}}\sqrt{v_{\mathrm{<figure-callout\; id="2"\; label="gi"\; filenames="HDA0000450148960000031.png"\; state="\{\{state\}\}">gi</figure-callout>}}^2+u_{\mathrm{gi}}^2}---\left(5\right)$

Wherein, u _jiand v _githe x and the y direction speed component that represent respectively i feature in g two field picture.

(2) speed variance vectors AV _g:

SV _g={ SV _g, g=1 ..., G}, wherein SV _gbe the speed variance of g two field picture, for weighing the complexity of every two field picture light stream speed, can shown in (6), obtain SV _g.

${\mathrm{SV}}_g=\frac{1}{F}\underset{i=1}{\overset{F}{\mathrm{\&Sigma;}}}{(\sqrt{v_{\mathrm{<figure-callout\; id="2"\; label="gi"\; filenames="HDA0000450148960000031.png"\; state="\{\{state\}\}">gi</figure-callout>}}^2+u_{\mathrm{gi}}^2}-{\mathrm{AS}}_g)}^2---\left(6\right)$

(3) direction variance vector DV _g:

DV _g={ DV _g, g=1 ..., G}, wherein DV _gbe the direction variance of g two field picture, for weighing the complexity of light stream direction, can shown in (7), obtain DV _g.

${\mathrm{DV}}_g=\frac{1}{8}\underset{i=1}{\overset{8}{\mathrm{\&Sigma;}}}{({\mathrm{ND}}_{\mathrm{gi}}-{\stackrel{\&OverBar;}{\mathrm{ND}}}_g)}^2---\left(7\right)$

Be divided into 8 intervals by 0～360 °, concentrate the direction character of light stream to vote according to angular interval every kind of behavioural characteristic, obtain the direction histogram of every kind of behavior.ND _gifor i of direction histogram interval statistical value,

for { ND _gi, i=1 ..., the mean value of 8}.

(4) divergence vector DD _g:

DD _g={ DD _g, g=1 ..., G}, wherein DD _gbe the divergence of g two field picture, can shown in (5), obtain DD _g.

$\left\{\begin{array}{c}{\mathrm{DD}}_g=\underset{i=1}{\overset{8}{\mathrm{\&Sigma;}}}{\mathrm{ND}}_{\mathrm{gi}}\&times;\left|{\mathrm{RD}}_g\left(i\right)\right|\\ {\mathrm{RD}}_g\left(i\right)=\mathrm{mod}(i-{\mathrm{MD}}_g,8)-8\&times;(\mathrm{mod}(i-{\mathrm{MD}}_g,8)\&GreaterEqual;4)\end{array}\right.---\left(8\right)$

MD wherein _g=max (ND _gi), i=1 ..., 8.

(5) behavior sum of all pixels vector

Because the monitoring scene depth of field is generally larger, there is more serious perspective phenomenon (same object in the projection of scene on the plane of delineation, close to video camera, seem large, away from video camera, seem little), therefore need to do weighting to the contribution of different pixels on the plane of delineation and process.Suppose that ground is plane, people is perpendicular to ground.As shown in Figure 5, establish the some P that disappears _vcoordinate be (x _v, y _v), reference line is y _r=H/2, can be suc as formula the contribution factor that obtains any one pixel I (x, y) on the plane of delineation shown in (9).

$S_C(x,y)={\left(\frac{y_r-y_v}{y-y_v}\right)}^2---\left(9\right)$

The sum of all pixels for the behavior is: the sum of all pixels vector of the behavior is NP _g={ NP _g, g=1 ..., G}

Obtain after 5*G dimensional feature vector, by craft, demarcate the number of two kinds of different behaviors of turnover, for training of human artificial neural networks model, the neural network model training is added up for number of people entering.Can obtain demographics with known neural net method.Experiment enters the total number of persons in garden by adding up the number of entering exporting under this scene with the method acquisition of the difference of the number of going out.Fig. 6 (a) shows on-the-spot certain two field picture turnover behavior crowd's number.In the image upper right corner, by red font, express from starting to count down to the number of entering so far and going out number: enter (In): 157, go out (Out): 39, in image, only shown part Optical-flow Feature point, Based on Feature Points in elliptic region 1 " goes out ", Based on Feature Points in elliptic region 2 " enters ", arrow representation feature spot moving direction, black box is number statistical regions.Fig. 6 (b) shows the variation (every 2 minutes being unit) of number in garden, and wherein in garden, the average accuracy of demographics is 92.35%.

Above-described specific embodiment; object of the present invention, technical scheme and beneficial effect are further described; institute is understood that; the foregoing is only specific embodiments of the invention and oneself; be not limited to the present invention; within the spirit and principles in the present invention all, any modification of making, be equal to replacement, improvement etc., within all should being included in protection scope of the present invention.

Claims (10)

1. a computer implemented video frequency monitoring method, comprises step:

A) receive the video data of being caught by video camera;

B) according to the video data receiving, set up group behavior model;

C) estimate the parameter of described group behavior model, obtain the multiple crowd behaviour existing in scene;

D) use the group behavior model obtaining to obtain the behavioural characteristic collection of different crowd;

E) the behavioural characteristic collection obtaining is changed, and for every kind of crowd behaviour, obtained people's numerical value of statistics with the behavioural characteristic collection of conversion.

2. method according to claim 1, wherein, described step b) comprising: set up word-document model, wherein using each motion pixel as a word w _i, the M two field picture of one section of video, corresponding to M document, is used word set W={w _i, i=1 ..., N} represents document, wherein w _i={ x _i, y _i, u _i, v _i, N is the number of pixels in described frame of video, and x represents the horizontal level of pixel, and y represents the upright position of pixel, and u represents the speed of pixel along continuous straight runs, v represents pixel speed vertically.

3. method according to claim 1, wherein, described step c) comprising: the parameter of estimating described group behavior model by maximal possibility estimation EM method.

4. method according to claim 2, wherein, described step c) comprising: according to following formula, with group behavior model, detect the behavior existing in scene, and obtain the feature set of every kind of behavior:

$\left\{\begin{array}{c}{f}_{k^{*}}=\left\{x_{k^{*}i}\right|i=1,...,F\}\\ k^{*}=\underset{k\&Element;\{1,...,K\}}{\arg \max }p(x_i,z_{k,i}|\mathrm{\&alpha;},\mathrm{\&beta;})\end{array}\right.$

Wherein, α represents the power relatively between implicit theme in collection of document, and β represents the probability distribution of all implicit themes self, supposes total K behavior,

be the feature set of k* specific behavior, the number of feature in the feature set that F is k* behavior.

5. method according to claim 4, wherein, described steps d) comprising: the proper vector NF={AS that the feature set of the behavior obtaining is converted to 5*G dimension _g, SV _g, DV _g, DD _g, NP _g, and training of human artificial neural networks is with statistical number of person, and wherein, AS _grepresent average velocity vector, SV _gexpression speed variance vectors, DV _grepresent direction variance vector, DD _grepresent divergence vector, and NP _gexpression behavior sum of all pixels vector.

6. method according to claim 5, wherein, average velocity vector AS _gbe calculated as:

AS _G＝{AS _g，g＝1，...，G}

AS wherein _gbe the average velocity of g two field picture, u _giand v _githe x and the y direction speed component that represent respectively i feature in g two field picture.

7. method according to claim 5, wherein, speed variance vectors SV _gbe calculated as:

SV _G＝{SV _g，g＝1，...，G}

SV wherein _gbe the speed variance of g two field picture, ${\mathrm{SV}}_g=\frac{1}{F}\underset{i=1}{\overset{F}{\mathrm{\&Sigma;}}}{(\sqrt{v_{\mathrm{gi}}^2+u_{\mathrm{gi}}^2}-{\mathrm{AS}}_g)}^2,$ U _giand v _githe x and the y direction speed component that represent respectively i feature in g two field picture.

8. method according to claim 5, wherein, direction variance vector DV _gbe calculated as:

DV _G＝{DV _g，g＝1，...，G}

DV wherein _gbe the direction variance of g two field picture,

nD _gifor i of direction histogram interval statistical value,

mean value, u _giand v _githe x and the y direction speed component that represent respectively i feature in g two field picture.

9. method according to claim 5, wherein, divergence vector DD _gbe calculated as:

DD _G＝{DD _g，g＝1，...，G}

DD wherein _git is the divergence of g two field picture.

Wherein $\left\{\begin{array}{c}{\mathrm{DD}}_g=\underset{i=1}{\overset{8}{\mathrm{\&Sigma;}}}{\mathrm{ND}}_{\mathrm{gi}}\&times;\left|{\mathrm{RD}}_g\left(i\right)\right|\\ {\mathrm{RD}}_g\left(i\right)=\mathrm{mod}(i-{\mathrm{MD}}_g,8)-8\&times;(\mathrm{mod}(i-{\mathrm{MD}}_g,8)\&GreaterEqual;4)\end{array}\right.$

MD wherein _g=max (ND _gi), i=1 ..., 8, ND _gifor i of direction histogram interval statistical value.

10. method according to claim 5, wherein, behavior sum of all pixels vector NP _gbe calculated as:

NP _G＝{NP _g，g＝1，...，G}

NP wherein _gthe behavior sum of all pixels of g two field picture, ${\mathrm{NP}}_g=\frac{1}{F}\underset{i=1}{\overset{F}{\mathrm{\&Sigma;}}}{S}_C(x_{\mathrm{gi}},y_{\mathrm{gi}}).$

Images