CN104933726A - Dense crowd segmentation method based on space-time information constraint - Google Patents

Abstract

The invention provides a dense crowd segmentation method based on space-time information constraints. A foreground detection method is utilized to obtain the space relation of crowd motion, and a characteristic point tracking method is utilized to obtain the motion relation of the crowd motion in terms of time. Through a method of the two motion relations assisting each other, the sub-crowd division precision is improved, and a division result accords with a group, i.e., a group of three to five people, formed spontaneously by people in real life. The division result provides an important analysis material for research of crowd interaction behaviors. The method is widely applied to the fields of crowd management, emergency monitoring or the like in public places.

Description

A kind of pod dividing method based on space time information constraint

Technical field

The invention belongs to the crowd behaviour analysis directions in video monitoring, be specifically related to a kind of method crowd being divided under intensive scene sub-group, to the research of crowd's interbehavior, there is important Research Significance.

Background technology

Along with the quickening of increase and the urbanization process day by day of world population, crowd's safety problem of public place is also outstanding day by day.Large-scale crowd activity has caused the Mass disturbances such as various traffic, public security again and again.In order to safeguard public safety, prevention public contingent even, in market, gymnasium, and the place that the crowd is dense such as subway station, carry out intelligent monitoring to crowd and seem particularly important.And the basis of intelligent monitoring analyzes the crowd behaviour in video.Although the research of people to individual movement concrete in colony is long-standing, colony is not the superposition of simple individual movement.Research finds, in the scene of crowd movement's comparatively dense, people are not often freely independent action, but more tend to associated movement in threes and fours, intensive crowd, and this phenomenon is obvious.In these sub-groups, the motion state that motion unit can adjust oneself makes it to be consistent with other people, forms the cohort with Movement consistency.Therefore, we can go to treat pod from the angle of these little cohorts, but not crowd is regarded as individuality independent one by one.But current research also concentrates on the holistic approach of behavioral study to motion unit and global motion.Carry out group behavior analysis from the angle of individuality to have some limitations, such as during video scene comparatively dense, blocking the identification impact of pedestrian between human body is comparatively large, and behavior complicated between pedestrian is alternately also for analysis adds difficulty.And colony's division of macroscopic view can ignore interpersonal local feature, the behavioural analysis of local is made to become more difficult.Although go the research analyzing group behavior also fewer from the angle of sub-group, all actively exploring both at home and abroad, and achieving certain achievement.Such as external Harvard University, Cornell University, Japanese sophisticated technologies research institute and domestic Beijing Institute of Technology, the Chinese Academy of Sciences, Southeast China University etc., their research has promoted the research of group behavior analysis.But because this field is scarcely out of swaddling-clothes at present, therefore there are important theory significance and researching value to the research of colony's cutting techniques.

The result granularity of now a lot of colony's partitioning algorithms is comparatively large, and each sub-group is generally greater than five to ten people, and this does not just meet the actual conditions of the colony of the spontaneous composition of people, is thus unfavorable for the research of further crowd behaviour.

Summary of the invention

The object of the invention is to solve the problem of the sub-group how crowd of intensive scene being divided into three to five people.

For ease of the group interaction behavior in research monitor video scene, the present invention proposes a kind of in monitor video, by space time information constraint, dense population is divided into the method for sub-group.The method at least comprises the following steps:

Step 1), the extraction of motion feature dot information is carried out to the every frame video image in monitor video;

Step 2), the motion of the foreground point of every frame video image is carried out binaryzation patch extract;

Step 3), utilize the boundary information of described patch to carry out constraint spatially to the unique point of foreground point to divide;

Step 4), utilize the temporal continuous position information of unique point, obtain the movement tendency information of unique point, thus sub-group further divided.

Preferably, the method comprises step 5 further), the abnormity point in division result is revised and rejected, finally completes the division of sub-group.

Preferably, described step 1) specifically comprise the steps:

(1) motion feature point is followed the tracks of and trajectory extraction: utilize KLT algorithm to carry out matched jamming to motion feature point, based on the KLT matching algorithm of optimal estimation, utilize the gray scale difference quadratic sum SSD of interframe as tolerance, in window W to be tracked, find matching characteristic point, therefore obtain the track of unique point in successive frame namely p is put from the track in time t to t → d, wherein, p _tthe position of representative point p when moment t;

(2) information extraction of motion feature point: according to the track of unique point obtain the side-play amount d=(Δ x, Δ y) of unique point in interframe, be seen as the translational speed of unique point, be designated as

Preferably, described step 2) specifically comprise the steps:

(1) obtain the motion patch of foreground point: the method adopting mixed Gaussian background modeling GMM, obtain the foreground mask of binaryzation; Wherein, mixed Gauss model uses K Gauss model to carry out the feature of each pixel in token image, mixed Gauss model is upgraded after a new two field picture obtains, mate with mixed Gauss model with each pixel in present image, if success, judges that this point is as background dot, otherwise is foreground point;

(2) region being communicated with patch is obtained: the connection patch in foreground mask is labeled as different ID values, obtains some connection patches, the set B of note patch _i={ B ₁, B ₂, B ₃..., B _k, represent in picture frame I and comprise k the patch be communicated with.

Preferably, described step 3) specifically comprise the steps:

(1) rectangular area of patch is calculated: after finding the position of the boundary pixel of patch, this patch region is marked off a rectangular area according to the position of boundary pixel, and obtain frontier properties BR and TL of this patch, be respectively lower right corner point coordinate and the upper right corner point coordinate of rectangle;

(2) utilize the spatial information of patch to carry out constraint to unique point to divide: if the coordinate of unique point p is (p _x, p _y), to all p ∈ I, if there is TL _x≤ p _x≤ BR _x, and BR _y≤ p _y≤ TL _y, so then have p ∈ B _k.If p ∈ is B _k, then p is added patch B _kin; Otherwise, then next patch coupling is carried out; If p does not belong to any one patch, then rejected; After Planar Mechanisms, the patch set B in picture frame I _i={ B ₁, B ₂, B ₃..., B _kin patch, all comprise some unique points; Suppose that in picture frame I, all unique points form S set _i, segmentation result so now can be expressed as S _i={ B _i, F _i, wherein B _ifor comprising the patch of the unique point through space constraint in picture frame I, F _ifor the point set in picture frame I not on foreground moving patch; Next will to B _ifurther divide.

Preferably, described step 4) specifically comprise the steps:

(1) its velocity vector of unique point p is calculated the direction cosine cos θ formed with x-axis, and then obtain its angular separation θ (0≤θ≤2 π); Computing method are as follows:

$\mathrm{\θ}=\arccos \frac{\mathrm{\Δx}}{\sqrt{({\mathrm{\Δx}}^2+{\mathrm{\Δy}}^2)}}---\left(1\right)$

(2) angular separation is divided into 12 equal portions from 0 to 2 π, each interval is labeled as D respectively _i, wherein i=1,2,3 ..., 12; For each unique point adds property value Pdirc, represent the movement tendency of each unique point; For Pdirc assignment, specific practice is as follows:

$Pdirc=\left\{\begin{array}{cc}D1,& 0\&le;\mathrm{\&theta;}<\frac{\mathrm{\&pi;}}{6}\\ D2,& \frac{\mathrm{\&pi;}}{6}\&le;\mathrm{\&theta;}<\frac{\mathrm{\&pi;}}{3}\\ D3,& \frac{\mathrm{\&pi;}}{3}\&le;\mathrm{\&theta;}<\frac{\mathrm{\&pi;}}{2}\\ D4,& \frac{\mathrm{\&pi;}}{2}\&le;\mathrm{\&theta;}<\frac{2\mathrm{\&pi;}}{3}\\ & ...\\ D12,& \frac{11\mathrm{\&pi;}}{6}\&le;\mathrm{\&theta;}<2\mathrm{\&pi;}\end{array}\right.---\left(2\right)$

Unique point identical for property value Pdirc in same patch point is polymerized to a point set C _k, after the constraint of angular separation, patch set B originally _iset Further Division is B _i={ C}, wherein C={C ₁, C ₂, C ₃..., C _k.

Preferably, described step 5) specifically comprise the steps:

(1) revise abnormity point: the division mistake that the trickle differential seat angle for head and foot direction causes, the division interval of the attribute Pdirc of the representative movement tendency of unique point p is amplified, reduce because velocity reversal divides the meticulous error caused; Specific practice is as follows:

Calculate in K the point of proximity of unique point p, the mark value that the property value Pdirc frequency of occurrences of each point of proximity is maximum, be assumed to be D _i, the Pdirc value of unique point p is assumed to be D _j, so for all unique point p, if i+1=j or i-1=j, so just the Pdirc value of p is modified to D _i.

Preferably, described step 5) also comprise the steps:

(2) rejecting abnormalities point: the number N calculating point identical with its movement tendency in L the point of proximity of unique point p; Set a critical value M (M≤L), if N<M, then p point is thought abnormity point, from sub-group C _kmiddle rejecting.

Utilization prospects detection method of the present invention obtains the spatial relationship of crowd movement, the method of feature point tracking is utilized to obtain crowd movement's kinematic relation in time, mutually auxiliary method is carried out by two kinds of kinematic relations, improve the precision that sub-group divides, and division result meets the group of the spontaneous composition of crowd in actual life, the group be namely made up of 3 to 5 people.The result divided is that the research of people group's interbehavior provides important analysis material, and the crowd's management in public places, has a wide range of applications in the fields such as emergency case monitoring.

Accompanying drawing explanation

Fig. 1 is shown as the method concrete steps process flow diagram that dense population is divided into sub-group by space time information of the present invention constraint.

Fig. 2 is shown as the method step schematic diagram that dense population is divided into sub-group by space time information of the present invention constraint.

Embodiment

Below by way of specific instantiation, embodiments of the present invention are described, those skilled in the art the content disclosed by this instructions can understand other advantages of the present invention and effect easily.The present invention can also be implemented or be applied by embodiments different in addition, and the every details in this instructions also can based on different viewpoints and application, carries out various modification or change not deviating under spirit of the present invention.It should be noted that, when not conflicting, the feature in following examples and embodiment can combine mutually.

It should be noted that, the diagram provided in following examples only illustrates basic conception of the present invention in a schematic way, then only the assembly relevant with the present invention is shown in graphic but not component count, shape and size when implementing according to reality is drawn, it is actual when implementing, and the kenel of each assembly, quantity and ratio can be a kind of change arbitrarily, and its assembly layout kenel also may be more complicated.

Refer to shown in Fig. 1 to Fig. 2, sub-group dividing method of the present invention comprises following steps:

1, unique point extraction of motion information is carried out to the every two field picture in monitor video:

(1) motion feature point is followed the tracks of and trajectory extraction.KLT (Kanade, Lucas, and Tomasi) algorithm is utilized to carry out matched jamming to unique point.KLT matching algorithm based on optimal estimation utilizes the gray scale difference quadratic sum (SSD) of interframe as tolerance, finds matching characteristic point, therefore can obtain the track of unique point in successive frame in window W to be tracked namely p is put from the track in time t to t → d, p _tthe position of representative point p when moment t.

(2) extraction of motion information of unique point.According to the track of unique point the coordinate position of unique point in adjacent two two field pictures is subtracted each other, the side-play amount d=(Δ x, Δ y) of unique point in interframe can be obtained, can be regarded as the translational speed of unique point, be designated as as shown in (A) in Fig. 2, the velocity vector of arrow representative feature point.

2, the extraction of binaryzation patch is carried out to the sport foreground in video frame image:

(1) foreground moving patch is obtained.Adopt the method for mixed Gaussian background modeling, obtain the foreground mask of binaryzation.Mixed Gauss model uses K (being generally 3 to 5) individual Gauss model to carry out the feature of each pixel in token image, mixed Gauss model is upgraded after a new two field picture obtains, mate with mixed Gauss model with each pixel in present image, if success, judges that this point is as background dot, otherwise is foreground point.

(2) region being communicated with patch is obtained.Connection patch in foreground mask is labeled as different ID values, obtains some connection patches, the set B of note patch _i={ B ₁, B ₂, B ₃..., B _k, represent in picture frame I and comprise k the patch be communicated with.The patch be communicated with illustrates that the pedestrian's distance in this space is more close, and as shown in (B) in Fig. 2, different rectangular area represents different patches.

3, the boundary information being communicated with patch is utilized to divide the constraint that foreground features point carries out spatially:

(1) rectangular area being communicated with patch is calculated.After finding the position of the boundary pixel being communicated with patch, this connection patch region is marked off a rectangular area according to the position of boundary pixel, and obtain frontier properties BR (Bottom-Right) and the TL (Top-Left) of this patch, be respectively lower right corner point coordinate and the upper right corner point coordinate of rectangle.

(2) utilize the spatial information being communicated with patch to carry out constraint to unique point to divide.

Can judge whether a unique point belongs to this patch region fast by rectangular area, and avoid the computation complexity that broken edge brings.If the coordinate of unique point p is (p _x, p _y), to all p ∈ I, if there is TL _x≤ p _x≤ BR _x, and BR _y≤ p _y≤ TL _y, so then have p ∈ B _k.If p ∈ is B _k, then p is added patch B _kin.Otherwise, then next patch coupling is carried out.If p does not belong to any one patch, then rejected.After Planar Mechanisms, the patch set B in picture frame I _i={ B ₁, B ₂, B ₃..., B _kin patch, all comprise some unique points.As shown in (C) in Fig. 2, in the patch of different rectangular area, contain some unique points.Suppose that in picture frame I, all unique points form S set _i, segmentation result so now can be expressed as S _i={ B _i, F _i, wherein B _ifor comprising the patch of the unique point through space constraint in picture frame I, F _ifor the point set in picture frame I not on foreground moving patch.Next will to B _ifurther divide.

4, utilize the temporal continuous position information of unique point, obtain the movement tendency information of unique point, thus sub-group is further divided;

(1) its velocity vector of unique point p is calculated the direction cosine cos θ formed with x-axis (horizontal direction of image).And then obtain its angular separation θ (0≤θ≤2 π).Computing method are as follows:

$\mathrm{\&theta;}=\arccos \frac{\mathrm{\&Delta;x}}{\sqrt{({\mathrm{\&Delta;x}}^2+{\mathrm{\&Delta;y}}^2)}}---\left(1\right)$

(2) angular separation is divided into 12 equal portions from 0 to 2 π, each interval is labeled as D respectively _i, wherein i=1,2,3 ..., 12.For each unique point adds property value Pdirc, represent the movement tendency of each unique point.For Pdirc assignment, specific practice is as follows:

$Pdirc=\left\{\begin{array}{cc}D1,& 0\&le;\mathrm{\&theta;}<\frac{\mathrm{\&pi;}}{6}\\ D2,& \frac{\mathrm{\&pi;}}{6}\&le;\mathrm{\&theta;}<\frac{\mathrm{\&pi;}}{3}\\ D3,& \frac{\mathrm{\&pi;}}{3}\&le;\mathrm{\&theta;}<\frac{\mathrm{\&pi;}}{2}\\ D4,& \frac{\mathrm{\&pi;}}{2}\&le;\mathrm{\&theta;}<\frac{2\mathrm{\&pi;}}{3}\\ & ...\\ D12,& \frac{11\mathrm{\&pi;}}{6}\&le;\mathrm{\&theta;}<2\mathrm{\&pi;}\end{array}\right.---\left(2\right)$

Unique point identical for property value Pdirc in same patch point is polymerized to a point set C _k, after the constraint of angular separation, patch set B originally _iset Further Division is B _i={ C}, wherein C={C ₁, C ₂, C ₃..., C _k.

As shown in (D) in Fig. 2, this step obtains final colony's division result, and spatially apart from close, the point that movement tendency is identical is divided among same colony.If the granularity of division of angular separation amplified, such as, be divided into 16 direction of motion, then the crowd's granularity divided becomes large; Otherwise due to the difference of the movement tendency at each position on the person, the noise of division will increase, and not easily eliminate.But, in different application scenarioss, this parameter can be changed, divide effect preferably to obtain.

5, the abnormity point in division result is revised and is rejected, complete the division of sub-group:

There are some abnormity point in division result, by analyzing the position of the appearance of these exceptions, we find that the head of pedestrian and foot are usually mistaken as is two agglomerates, and the swing of arm and head also can cause the error of division.We have proposed some improving one's methods based on general knowledge, the method can improve the degree of accuracy of segmentation greatly for this reason.

(1) abnormity point is revised.For the division mistake that the trickle differential seat angle in head and foot direction causes, the division interval of the attribute Pdirc of the representative movement tendency of unique point p is amplified by we, reduces as far as possible because velocity reversal divides the meticulous error caused.Specific practice is as follows:

Calculate in K the point of proximity of unique point p, the mark value that the property value Pdirc frequency of occurrences of each point of proximity is maximum, be assumed to be D _i.The Pdirc value of unique point p is assumed to be D _j, so for all unique point p, if i+1=j or i-1=j, so just the Pdirc value of p is modified to D _i.

(2), rejecting abnormalities point.Found by the observation of mass data, the surrounding of abnormal point value lacks the existence of similar point usually, especially the abnormity point that causes of the swing of arm place and head.Therefore we calculate the number N of point identical with its movement tendency in L the point of proximity of unique point p.Set a critical value M (M≤L), if N<M, then p point is thought abnormity point, from sub-group C _kmiddle rejecting.

Utilization prospects detection method of the present invention obtains the spatial relationship of crowd movement, the method of feature point tracking is utilized to obtain crowd movement's kinematic relation in time, mutually auxiliary method is carried out by two kinds of kinematic relations, improve the precision that sub-group divides, and division result meets the group of the spontaneous composition of crowd in actual life, the group be namely made up of 3 to 5 people.The result divided is that the research of people group's interbehavior provides important analysis material, and the crowd's management in public places, has a wide range of applications in the fields such as emergency case monitoring.

In sum, the present invention effectively overcomes various shortcoming of the prior art and tool high industrial utilization.

Above-described embodiment is illustrative principle of the present invention and effect thereof only, but not for limiting the present invention.Any person skilled in the art scholar all without prejudice under spirit of the present invention and category, can modify above-described embodiment or changes.Therefore, such as have in art usually know the knowledgeable do not depart from complete under disclosed spirit and technological thought all equivalence modify or change, must be contained by claim of the present invention.

Claims (8)

1., based on a pod dividing method for space time information constraint, it is characterized in that: the method at least comprises the following steps:

Step 1), the extraction of motion feature dot information is carried out to the every frame video image in monitor video;

Step 2), the motion of the foreground point of every frame video image is carried out binaryzation patch extract;

Step 3), utilize the boundary information of described patch to carry out constraint spatially to the unique point of foreground point to divide;

Step 4), utilize the temporal continuous position information of unique point, obtain the movement tendency information of unique point, thus sub-group further divided.

2. the pod dividing method based on space time information constraint according to claim 1, is characterized in that: the method comprises step 5 further), the abnormity point in division result is revised and rejected, finally completes the division of sub-group.

3. the pod dividing method based on space time information constraint according to claim 2, is characterized in that: described step 1) specifically comprise the steps:

(1) motion feature point is followed the tracks of and trajectory extraction: utilize KLT algorithm to carry out matched jamming to motion feature point, based on the KLT matching algorithm of optimal estimation, utilize the gray scale difference quadratic sum SSD of interframe as tolerance, in window W to be tracked, find matching characteristic point, therefore obtain the track of unique point in successive frame namely p is put from the track in time t to t → d, wherein, p _tthe position of representative point p when moment t;

(2) information extraction of motion feature point: according to the track of unique point obtain the side-play amount d=(Δ x, Δ y) of unique point in interframe, be seen as the translational speed of unique point, be designated as

4. the pod dividing method based on space time information constraint according to claim 3, is characterized in that: described step 2) specifically comprise the steps:

(1) obtain the motion patch of foreground point: the method adopting mixed Gaussian background modeling GMM, obtain the foreground mask of binaryzation; Wherein, mixed Gauss model uses K Gauss model to carry out the feature of each pixel in token image, mixed Gauss model is upgraded after a new two field picture obtains, mate with mixed Gauss model with each pixel in present image, if success, judges that this point is as background dot, otherwise is foreground point;

(2) region being communicated with patch is obtained: the connection patch in foreground mask is labeled as different ID values, obtains some connection patches, the set B of note patch _i={ B ₁, B ₂, B ₃..., B _k, represent in picture frame I and comprise k the patch be communicated with.

5. the pod dividing method based on space time information constraint according to claim 4, is characterized in that: described step 3) specifically comprise the steps:

(1) rectangular area being communicated with patch is calculated: after finding the position of the boundary pixel being communicated with patch, this connection patch region is marked off a rectangular area according to the position of boundary pixel, and obtain frontier properties BR and TL of this connection patch, be respectively lower right corner point coordinate and the upper right corner point coordinate of rectangle;

(2) utilize the spatial information of patch to carry out constraint to unique point to divide: if the coordinate of unique point p is (p _x, p _y), to all p ∈ I, if there is TL _x≤ p _x≤ BR _x, and BR _y≤ p _y≤ TL _y, so then have p ∈ B _k.If p ∈ is B _k, then p is added patch B _kin; Otherwise, then next patch coupling is carried out; If p does not belong to any one patch, then rejected; After Planar Mechanisms, the patch set B in picture frame I _i={ B ₁, B ₂, B ₃..., B _kin patch, all comprise some unique points; Suppose that in picture frame I, all unique points form S set _i, segmentation result so now can be expressed as S _i={ B _i, F _i, wherein B _ifor comprising the patch of the unique point through space constraint in picture frame I, F _ifor the point set in picture frame I not on foreground moving patch; Next will to B _ifurther divide.

6. the pod dividing method based on space time information constraint according to claim 5, is characterized in that: described step 4) specifically comprise the steps:

(1) its velocity vector of unique point p is calculated the direction cosine cos θ formed with x-axis, and then obtain its angular separation θ (0≤θ≤2 π); Computing method are as follows:

$\mathrm{\&theta;}=arccos\frac{\mathrm{\&Delta;}x}{\sqrt{{\mathrm{\&Delta;x}}^2+{\mathrm{\&Delta;y}}^2}}---\left(1\right)$

(2) angular separation is divided into 12 equal portions from 0 to 2 π, each interval is labeled as D respectively _i, wherein i=1,2,3 ..., 12; For each unique point adds property value Pdirc, represent the movement tendency of each unique point; For Pdirc assignment, specific practice is as follows:

$Pdirc=\left\{\begin{array}{c}{D}_1,0\&le;\mathrm{\&theta;}<\frac{\mathrm{\&pi;}}{6}\\ D_2,\frac{\mathrm{\&pi;}}{6}\&le;\mathrm{\&theta;}<\frac{\mathrm{\&pi;}}{3}\\ D_3,\frac{\mathrm{\&pi;}}{3}\&le;\mathrm{\&theta;}<\frac{\mathrm{\&pi;}}{2}\\ D_4,\frac{\mathrm{\&pi;}}{2}\&le;\mathrm{\&theta;}<\frac{2\mathrm{\&pi;}}{3}\\ \mathrm{...}\\ D_{12},\frac{11\mathrm{\&pi;}}{6}\&le;\mathrm{\&theta;}<2\mathrm{\&pi;}\end{array}\right.---\left(2\right)$

Unique point identical for property value Pdirc in same patch point is polymerized to a point set C _k, after the constraint of angular separation, patch set B originally _iset Further Division is B _i={ C}, wherein C={C ₁, C ₂, C ₃..., C _k.

7. the pod dividing method based on space time information constraint according to claim 2 to 6 any one, is characterized in that: described step 5) specifically comprise the steps:

(1) revise abnormity point: the division mistake that the trickle differential seat angle for head and foot direction causes, the division interval of the attribute Pdirc of the representative movement tendency of unique point p is amplified, reduce because velocity reversal divides the meticulous error caused; Specific practice is as follows:

Calculate in K the point of proximity of unique point p, the mark value that the property value Pdirc frequency of occurrences of each point of proximity is maximum, be assumed to be D _i, the Pdirc value of unique point p is assumed to be D _j, so for all unique point p, if i+1=j or i-1=j, so just the Pdirc value of p is modified to D _i.

8. the pod dividing method based on space time information constraint according to claim 7 any one, is characterized in that: described step 5) also comprise the steps:

(2) rejecting abnormalities point: the number N calculating point identical with its movement tendency in L the point of proximity of unique point p; Set a critical value M (M≤L), if N<M, then p point is thought abnormity point, from sub-group C _kmiddle rejecting.