CN103942793A - Video consistent motion area detection method based on thermal diffusion - Google Patents

Abstract

The invention discloses a video consistent motion area detection method based on thermal diffusion. The thermal diffusion process in the physics is introduced, an original optical flow field serves as a hot source and external force, and the original optical flow field becomes a thermograph better reflecting the scene motion mode according to the anisotropic diffusion principle. By building Delaunay triangulation, the motion relation between points can be better mastered, and the boundaries of different areas of the thermograph are found. The method can effectively master the motion information of global scenes, the points which are spaced by a long distance can be mutually affected through the correlation of motion information, and noise generated by calculation of optical flow has better robustness.

Description

The consistent motion region detection method of video based on thermal diffusion

Technical field

The present invention relates to a kind of image cuts apart and clustering technique, relate in particular to the consistent moving region of a kind of video based on thermal diffusion detection technique, that is: thermal diffusion process is introduced in Video processing, by the analysis to video motion pattern, detected the region of consistent motion.

Background technology

Consistent moving region detection technique is a very important technology in video and image processing field.It utilizes light stream extraction, image the technology such as to cut apart, by the analysis to key point or entire image space time motor pattern, by Region Segmentation consistent motor behavior out.The consistent region of described motor behavior can be player of a starting stride etc. on the vehicle toward a direction running, marathon course on road.This technology has very important effect in application such as dense population behavioural analysis, traffic monitorings.

Through the literature search of prior art is found, paper " A lagrang ian particle dynamics approach for crowd flow segmentation and stability analysis " that the people such as S.Ali deliver for 2007 on " Conference on Computer Vision and Pattern Recognition " proposes to utilize Lagrangian particle dynamic method (Lagrangian particle dynamics) to cut apart the high density consistent moving region of trooping, and the border that obtains consistent moving region by building FTLE field is used for cutting apart.The subject matter of the method is, thereby the error of optical flow computation can cause the discontinuous segmentation effect that affects in border obtaining.In addition, the method can only be confined to high density and troop, and in the time that crowd or Che Qun etc. in video are sparse, the accuracy of the method has significantly and declines.

Dividing the consistent region of motion by the calculating to key point time-space relationship is current comparatively popular method.Such as, the limitation of above two kinds of methods has been only to consider the relation between short distance key point, and has ignored kinematic relation between key point relatively far apart, in the time of the scenes such as appearance covers, blocks, consistent moving region is difficult to divide.And in the time that prospect background is close, the accuracy of cutting apart can be had a greatly reduced quality.

Therefore,, for above variety of problems, be necessary to find and a kind ofly can react the method that global motion relation can accurately be held again motion details and can overcome optical flow computation equal error.

Summary of the invention

The present invention is directed to the deficiencies in the prior art, proposed a kind of consistent motion region detection method based on thermal diffusion.Thermal diffusion process in physics is introduced to this method, by processing the optical flow field of original calculation, can overcome the error of optical flow computation, also can distant point be connected by thermal diffusion, hold motion details simultaneously, improve the accuracy that consistent moving region is detected.

Principle of the present invention is: this method subject matter to be processed is exactly how to find the correlativity of different pixels motion and how will closely couple together with remote related pixel.Consider the light stream of image in processing and the similarity of hot-fluid in physics, all relevant with the motion of particle, the present invention is dexterously by thermal diffusion process, allows closely and remote particle interaction, automatically connection.Diffusion in this method is anisotropic, like this, is more easily linked together in a direction and the relative position example consistent with direction of motion, and the thermal map building by thermal diffusion is also more close to real sports ground.

According to the consistent motion region detection method of a kind of video based on thermal diffusion provided by the invention, comprise following steps:

Step 1: obtain input video optical flow field F, with F=(F _x, F _y) represent F _xrepresent x direction optical flow field, F _yrepresent y direction optical flow field, F _x(i, j) represents to be positioned at the x direction light stream of (i, j) position, F _y(i, j) represents to be positioned at the y direction light stream of (i, j) position; As Fig. 2;

Step 2: using the optical flow field of T0 frame in step 1 as thermal source and acting force, carry out thermal diffusion, obtain the thermal map E after thermal diffusion, wherein, E=(E _x, E _y), E _xrepresent x direction thermal map, E _yrepresent y direction thermal map; As Fig. 3;

Step 3: choose at random on thermal map it is individual,, the length that M is video image, what N was video image is wide, and obtains the De Luonei triangulated graph (Delaunay Triangulation) of these points, as Fig. 4, calculates the weights on all limits in De Luonei triangulated graph;

Step 4: find weights in step 3 to be greater than the limit of threshold value Th, and using the institute on these limits a little as frontier point, build bianry image, obtain ultimate bound by the method expanding; And according to the thermal map in the ultimate bound obtaining and step 2, cut apart with dividing ridge method; Segmentation result is shown in Fig. 5.

Preferably: the E of thermal map described in step 2 obtains by following steps:

Step a) builds the two-dimentional null matrix E that two sizes are M × N _xand E _y, E _x(i, j) represents to be positioned at the x direction thermal map of (i, j) position, E _y(i, j) represents to be positioned at the y direction thermal map of (i, j) position; Build U=(U _x, U _y) make U=F, wherein, E _xrepresent x direction thermal map, E _yrepresent y direction thermal map;

Step b), for the every bit (x, y) in M × N point, is upgraded the thermal map E (x, y) that is positioned at (x, y) position, E (x, y)=[E by formula (1) _x(x, y), E _y(x, y)], wherein, E _x(x, y) represents to be positioned at the x direction thermal map of (x, y) position, E _y(x, y) represents to be positioned at the y direction thermal map of (x, y) position;

$E(x,y)=\frac{{\mathrm{\Σ}}_{i=1}^M{\mathrm{\Σ}}_{j=1}^N{e}_{(i,j)}(x,y)}{M\×N}---\left(1\right)$

Wherein, e _{(i, j)}the energy of the point that (x, y) represents to be positioned at (i, j) to (x, y) diffusion;

In formula (1), be positioned at (x, y) position e _{(i, j)}(x, y)=[e _{x (i, j)}(x, y), e _{y (i, j)}(x, y)] obtained by formula (2) (3), wherein, e _{x (i, j)}(x, y) represents point that x direction is positioned at (i, the j) energy to (x, y) diffusion, e _{y (i, j)}(x, y) represents point that y direction is positioned at (i, the j) energy to (x, y) diffusion;

$e_{x,(i,j)}(x,y)=\left\{\begin{array}{c}{U}_x(i,j)\&CenterDot;e^{-k_{p}D((x,y),(i,j))}\&CenterDot;e^{-k_f|F_x(i,j)\&CenterDot;(x-i)+F_y(i,j)\&CenterDot;(y-j)|}\\ \mathrm{if}C(F(x,y),F(i,j))\&GreaterEqual;{\mathrm{TH}}_c\\ 0,\mathrm{if}C(F(x,y),F(i,j))<{\mathrm{TH}}_c\end{array}\right.---\left(2\right)$

$e_{y,(i,j)}(x,y)=\left\{\begin{array}{c}{U}_y(i,j)\&CenterDot;e^{-k_{p}D((x,y),(i,j))}\&CenterDot;e^{-k_f|F_x(i,j)\&CenterDot;(x-i)+F_y(i,j)\&CenterDot;(y-j)|}\\ \mathrm{if}C(F(x,y),F(i,j))\&GreaterEqual;{\mathrm{TH}}_c\\ 0,\mathrm{if}C(F(x,y),F(i,j))<{\mathrm{TH}}_c\end{array}\right.---\left(3\right)$

Wherein, U _x(i, j) represents to be positioned at the some x direction external force field of (i, j), U _y(i, j) represents to be positioned at the some y direction external force field of (i, j), and e represents natural logarithm, k _p, k _frepresent constant, x represents the horizontal ordinate of propagation energy receiving station, and y represents the ordinate of propagation energy receiving station, and i represents the horizontal ordinate that propagation energy is put, and j represents the ordinate of propagation energy receiving station, TH _crepresent a threshold value;

In formula (2), (3), for formula $D((x,y),(i,j))=\sqrt{{(x-i)}^2+{(y-j)}^2},$

For formula $C(F(x,y),F(i,j))=\frac{F_x(x,y)\&CenterDot;F_x(i,j)+F_y(x,y)\&CenterDot;F_y(i,j)}{\sqrt{F_x{(x,y)}^2+F_y{(x,y)}^2}\&CenterDot;\sqrt{F_x{(i,j)}^2+F_y{(i,j)}^2}};$

Make U=E;

Step c) repeats b) several times; Now, thermal diffusion process completes, E=(E _x, E _y) be gained thermal map.

Preferably: the weights described in step 3 are obtained by following methods:

The limit of 2 that is positioned at (x, y) and (i, j) for connection, its weights are

Preferably: the bianry image construction method described in step 4 is as follows:

Building size is the null matrix B of M × N, is greater than the limit of Th for weights in each step 3, carries out following steps:

Be greater than Th if connect (x, y) with the limit weights of (i, j) 2, be made as 1 by connecting points all on the limit of (x, y) and (i, j) 2 in matrix B; B is required bianry image.

Compared with prior art, the present invention has following beneficial effect:

1, local neighbor point can be connected by thermal diffusion process, also can hold the relation between overall situation reference point relatively far apart, automatically the point of join dependency connection.Therefore, the present invention can process better block, the situation such as covering.In addition, in diffusion process, introducing particle light stream correlativity (is C (F (x, y), F (i in formula (2) (3), j))) can make just received energy mutually of the particle that is really associated, avoid the destruction to primary light stream computational data.

2, the present invention can play the effect of amplifying tiny light stream by diffusion process.In previous method, being all difficult to detect or be mistaken as for tiny light stream is at a distance error, and in the present invention, thermal diffusion process interacts the tiny light stream point close with ambient light flow path direction, makes the region that is originally difficult to detect become more obvious in the situation that not makeing mistakes.

3, the present invention can automatically overcome the error of optical flow computation.In the time there is error in optical flow computation, because particle light stream correlativity (is C (F (x in formula (2) (3), y), F (i, j))) existence, around the point of optical flow computation mistake cannot be diffused into, and point around can be by energy to error point through spreading several times, like this, by diffusion process repeatedly, the error of optical flow computation can reduce even to disappear.

Brief description of the drawings

By reading the detailed description of non-limiting example being done with reference to the following drawings, it is more obvious that other features, objects and advantages of the present invention will become:

Fig. 1 is the process flow diagram of the inventive method.

Fig. 2 is the optical flow field of actual traffic video example frame.

Fig. 3 is the thermal map being obtained by Fig. 2 optical flow field.

Fig. 4 is Fig. 2 frame triangulation schematic diagram.

Fig. 5 is the last segmentation result of Fig. 2 frame.

Embodiment

Below in conjunction with specific embodiment, the present invention is described in detail.Following examples will contribute to those skilled in the art further to understand the present invention, but not limit in any form the present invention.It should be pointed out that to those skilled in the art, without departing from the inventive concept of the premise, can also make some distortion and improvement.These all belong to protection scope of the present invention.

The present embodiment comprises following steps:

Step 1: obtain the optical flow field (each is two two-dimensional matrixs that size is M × N, and is x direction, and is y direction) of input video, with F=(F _x, F _y) represent F _x(i, j) represents to be positioned at the x direction light stream of (i, j) position, F _y(i, j) represents to be positioned at the y direction light stream of (i, j) position.

The method that the paper " High accuracy optical flow estimation based on a theory for warping " that wherein optical flow field described in step 1 is delivered by T.Brox etc. in the present embodiment for 2004 on " European Conference on Computer Vision " proposes obtains.

Step 2: using the optical flow field of T0 frame in step 1 as thermal source and acting force, carry out thermal diffusion, obtain the thermal map E after thermal diffusion.

The wherein thermal map E=(E described in step 2 _x, E _y) can be obtained by following methods:

A) build two two-dimentional null matrix E that size is M × N _xand E _y, E _x(i, j) represents to be positioned at the x direction thermal map of (i, j) position, E _y(i, j) represents to be positioned at the y direction thermal map of (i, j) position.Build U=(U _x, U _y) make U=F.

B), for the every bit (x, y) in M × N point, upgrade E (x, the y)=[E that is positioned at (x, y) position by formula (1) _x(x, y), E _y(x, y)].

$E(x,y)=\frac{{\mathrm{\&Sigma;}}_{i=1}^M{\mathrm{\&Sigma;}}_{j=1}^N{e}_{(i,j)}(x,y)}{M\&times;N}---\left(1\right)$

In formula, in (1), be positioned at (x, y) position e _{(i, j)}(x, y)=[e _{x (i, j)}(x, y), _{ey (i, j)}(x, y)] obtained by formula (2) (3).

$e_{x,(i,j)}(x,y)=\left\{\begin{array}{c}{U}_x(i,j)\&CenterDot;e^{-k_{p}D((x,y),(i,j))}\&CenterDot;e^{-k_f|F_x(i,j)\&CenterDot;(x-i)+F_y(i,j)\&CenterDot;(y-j)|}\\ \mathrm{if}C(F(x,y),F(i,j))\&GreaterEqual;{\mathrm{TH}}_c\\ 0,\mathrm{ifC}(F(x,y),F(i,j))<{\mathrm{TH}}_c\end{array}\right.---\left(2\right)$

$e_{y,(i,j)}(x,y)=\left\{\begin{array}{c}{U}_y(i,j)\&CenterDot;e^{-k_{p}D((x,y),(i,j))}\&CenterDot;e^{-k_f|F_x(i,j)\&CenterDot;(x-i)+F_y(i,j)\&CenterDot;(y-j)|}\\ \mathrm{if}C(F(x,y),F(i,j))\&GreaterEqual;{\mathrm{TH}}_c\\ 0,\mathrm{ifC}(F(x,y),F(i,j))<{\mathrm{TH}}_c\end{array}\right.---\left(3\right)$

In formula (2) (3), $D((x,y),(i,j))=\sqrt{{(x-i)}^2+{(y-j)}^2},$

$C(F(x,y),F(i,j))=\frac{F_x(x,y)\&CenterDot;F_x(i,j)+F_y(x,y)\&CenterDot;F_y(i,j)}{\sqrt{F_x{(x,y)}^2+F_y{(x,y)}^2}\&CenterDot;\sqrt{F_x{(i,j)}^2+F_y{(i,j)}^2}}.$

K in the present embodiment _pget 0.2, k _fget 0.8, TH _cget 0.7.

Make U=E.

C) repeat b) n time (n=3 in the present embodiment).Now, thermal diffusion process completes, E=(E _x, E _y) be gained thermal map.

Step 3: choose at random on thermal map individual, and obtain the De Luonei triangulated graph (Delaunay Triangulation) of these points, calculate the weights on all limits in De Luonei triangulated graph.

The method proposing in the paper " Two algorithms for constructing a Delaunay triangulation " that wherein the De Luonei triangulation described in step 3 can be delivered by D.Lee etc. for 1980 on " International Journal of Computer & Information Sciences " obtains.

Wherein the weights described in step 3 are obtained by following methods.The limit of 2 that is positioned at (x, y) and (i, j) for connection, its weights are $\frac{\sqrt{(E_x(x,y)-E_x(i,j){)}^2+(E_y(x,y)-E_y(i,j){)}^2}}{\sqrt{{(x-i)}^2+{(y-j)}^2}}.$

Step 4: find weights in step 3 to be greater than the limit of threshold value Th, and using the institute on these limits a little as frontier point, build bianry image, obtain ultimate bound by the method expanding.And according to the thermal map in the ultimate bound obtaining and step 2, cut apart by the method for watershed divide.

Wherein the Th described in step 4 gets 0.7 in the present embodiment.

Wherein the bianry image construction method described in step 4 is as follows, and building size is the null matrix B of M × N, is greater than the limit of Th for weights in each step 3, carries out following steps.Be greater than Th if connect (x, y) with the limit weights of (i, j) 2, be made as 1 by connecting points all on the limit of (x, y) and (i, j) 2 in matrix B.B is required bianry image.

Wherein the ultimate bound described in step 4 be in image promising 1 point.

The method that " the Watersheds in digital spaces:an efficient algorithm based on immersion simulations " that wherein dividing ridge method described in step 4 uses L.Vincent etc. to be published in " IEEE Transactions on Pattern Analysis and Machine Intelligence " for 1991 in the present embodiment proposes realizes.

Implementation result

According to above-mentioned steps, the different videos that comprises object, group movement is carried out to the detection of consistent moving region, experiment completes on computers.

First, utilize the present invention to compare with previously mentioned Lagrangian dynamic particles method and consistent region unchangeability method, testing result of the present invention is 7.8% with actual consistent domain error, and Lagrangian dynamic particles method is respectively 32.5% and 25.6% with consistent region unchangeability method error, can find out, accuracy rate of the present invention is far away higher than additive method.

Secondly, on the different videos such as traffic video, religion distinguished gathering, square, test, the present invention can adapt to various videos, region consistent with the reality number error of calculation only has 14%, and other two kinds of methods all have the example of failure, it is respectively 124% and 105% with the actual consistent region number error of calculation.Can find out thus, the robustness of this method is better than additive method.

Finally, for different parameters, on same video database, test.Different parameters (as Th etc.) testing result fluctuating error of the present invention is no more than 2.5%.

Experiment shows, than existing consistent motion region detection method, the present invention has higher robustness to different video, has better effect detecting consistent moving region.

Above specific embodiments of the invention are described.It will be appreciated that, the present invention is not limited to above-mentioned specific implementations, and those skilled in the art can make various distortion or amendment within the scope of the claims, and this does not affect flesh and blood of the present invention.

Claims (4)

1. the consistent motion region detection method of the video based on thermal diffusion, is characterized in that, comprises following steps:

Step 1: obtain input video optical flow field F, with F=(F _x, F _y) represent F _xrepresent the light stream of x direction, F _yrepresent the light stream of y direction, F _x(i, j) represents to be positioned at the x direction light stream of (i, j) position, F _y(i, j) represents to be positioned at the y direction light stream of (i, j) position;

Step 2: using the optical flow field of T0 frame in step 1 as thermal source and acting force, carry out thermal diffusion, obtain the thermal map E after thermal diffusion, wherein, E=(E _x, E _y), E _xrepresent x direction thermal map, E _yrepresent y direction thermal map;

Step 3: choose at random on thermal map the length individual, M is video image, what N was video image is wide, and obtains the De Luonei triangulated graph of these points, calculates the weights on all limits in De Luonei triangulated graph;

Step 4: find weights in step 3 to be greater than the limit of threshold value Th, and using the institute on these limits a little as frontier point, build bianry image, obtain ultimate bound by the method expanding; And according to the thermal map in the ultimate bound obtaining and step 2, cut apart with dividing ridge method.

2. the consistent motion region detection method of the video based on thermal diffusion according to claim 1, is characterized in that: the E of thermal map described in step 2 obtains by following steps:

Step a) builds the two-dimentional null matrix E that two sizes are M × N _xand E _y, E _x(i, j) represents to be positioned at the x direction thermal map of (i, j) position, E _y(i, j) represents to be positioned at the y direction thermal map of (i, j) position; Build U=(U _x, U _y) make U=F, wherein, U represents to spread suffered external force field, U _xrepresent x direction external force field, U _yrepresent y direction external force field;

Step b), for the every bit (x, y) in M × N point, is upgraded the thermal map E (x, y) that is positioned at (x, y) position, E (x, y)=[E by formula (1) _x(x, y), E _y(x, y)], wherein, E _x(x, y) represents to be positioned at the x direction thermal map of (x, y) position, E _y(x, y) represents to be positioned at the y direction thermal map of (x, y) position;

$E(x,y)=\frac{{\mathrm{\&Sigma;}}_{i=1}^M{\mathrm{\&Sigma;}}_{j=1}^N{e}_{(i,j)}(x,y)}{M\&times;N}---\left(1\right)$

Wherein, e _{(i, j)}the energy of the point that (x, y) represents to be positioned at (i, j) to (x, y) diffusion;

In formula (1), be positioned at (x, y) position e _{(i, j)}(x, y)=[e _{x (i, j)}(x, y), e _{y (i, j)}(x, y)] obtained by formula (2) (3), wherein, e _{x (i, j)}(x, y) represents point that x direction is positioned at (i, the j) energy to (x, y) diffusion, e _{y (i, j)}(x, y) represents point that y direction is positioned at (i, the j) energy to (x, y) diffusion;

$e_{x,(i,j)}(x,y)=\left\{\begin{array}{c}{U}_x(i,j)\&CenterDot;e^{-k_{p}D((x,y),(i,j))}\&CenterDot;e^{-k_f|F_x(i,j)\&CenterDot;(x-i)+F_y(i,j)\&CenterDot;(y-j)|}\\ \mathrm{if}C(F(x,y),F(i,j))\&GreaterEqual;{\mathrm{TH}}_c\\ 0,\mathrm{ifC}(F(x,y),F(i,j))<{\mathrm{TH}}_c\end{array}\right.---\left(2\right)$

$e_{y,(i,j)}(x,y)=\left\{\begin{array}{c}{U}_y(i,j)\&CenterDot;e^{-k_{p}D((x,y),(i,j))}\&CenterDot;e^{-k_f|F_x(i,j)\&CenterDot;(x-i)+F_y(i,j)\&CenterDot;(y-j)|}\\ \mathrm{if}C(F(x,y),F(i,j))\&GreaterEqual;{\mathrm{TH}}_c\\ 0,\mathrm{ifC}(F(x,y),F(i,j))<{\mathrm{TH}}_c\end{array}\right.---\left(3\right)$

Wherein, U _x(i, j) represents to be positioned at the some x direction external force field of (i, j), U _y(i, j) represents to be positioned at the some y direction external force field of (i, j), and e represents natural logarithm, k _p, k _frepresent constant, x represents the horizontal ordinate of propagation energy receiving station, and y represents the ordinate of propagation energy receiving station, and i represents the horizontal ordinate that propagation energy is put, and j represents the ordinate of propagation energy receiving station, TH _crepresent a threshold value;

In formula (2), (3), for formula $D((x,y),(i,j))=\sqrt{{(x-i)}^2+{(y-j)}^2},$

For formula $C(F(x,y),F(i,j))=\frac{F_x(x,y)\&CenterDot;F_x(i,j)+F_y(x,y)\&CenterDot;F_y(i,j)}{\sqrt{F_x{(x,y)}^2+F_y{(x,y)}^2}\&CenterDot;\sqrt{F_x{(i,j)}^2+F_y{(i,j)}^2}};$

Make U=E;

Step c) repeats b) several times; Now, thermal diffusion process completes, E=(E _x, E _y) be gained thermal map.

3. the consistent motion region detection method of the video based on thermal diffusion according to claim 2, is characterized in that: the weights described in step 3 are obtained by following methods:

The limit of 2 that is positioned at (x, y) and (i, j) for connection, its weights are

4. the consistent motion region detection method of the video based on thermal diffusion according to claim 3, is characterized in that: the bianry image construction method described in step 4 is as follows:

Building size is the null matrix B of M × N, is greater than the limit of Th for weights in each step 3, carries out following steps:

Be greater than Th if connect (x, y) with the limit weights of (i, j) 2, be made as 1 by connecting points all on the limit of (x, y) and (i, j) 2 in matrix B; B is required bianry image.