CN106599776B - A kind of demographic method based on trajectory analysis - Google Patents

Abstract

The present invention is a kind of demographic method based on trajectory analysis, the depth map of scene is obtained by binocular camera or RGBD camera, camera is demarcated, three-dimensional point cloud is converted by depth map using calibrated camera parameter, three-dimensional point cloud is projected to X-Y plane, the top view of scene is obtained, number of people target in top view is locked using number of people locking means proposed by the present invention, track is judged by detection faces, the judgement that track is entered and gone out.The present invention is used to count in scene the pedestrians of (bus, staircase, passway etc.), the number for accurately being entered and being gone out.

Description

A kind of demographic method based on trajectory analysis

Technical field

The present invention relates to a kind of demographic methods based on trajectory analysis.

Background technique

Demographics are always the hot topic in monitoring system, and there is also many methods, for example, using laser beam Method, if ray is blocked, someone passes through, in addition there are the method for using ultrasonic wave, by ultrasound measure away from From, to detect pedestrian, still, these methods can not pick out the direction of pedestrian, so, number cannot be widely used in Statistics.Carrying out statistics using number of the monocular camera to the pedestrian in scene has many technological difficulties, can under simple scenario It, can not very accurate geo-statistic due to blocking the influence with pseudo- target to count number very well, and under sufficiently complex scene The number of scene out, meanwhile, it can use under monocular camera there is no highly stable pedestrian's feature, variation of light etc. is all More reasons, the accuracy for also resulting in demographics are relatively low；Binocular or RGB-D camera pair can be used in defect based on monocular Number is counted, and under crowded environment, pedestrian has obviously feature, i.e. the head of pedestrian is had to than shoulder Height is based on this feature, it is easy to lock the number of people, by matching stage, form track, there are certain mode in these tracks, lead to The method for crossing machine learning extracts the mode of track, so that the behavior to pedestrian judges, finally to the number in channel into Row statistics.

Summary of the invention

For above-mentioned problems of the prior art, the object of the present invention is to provide a kind of based on trajectory analysis Demographic method realizes and is judged by the track 3D the behavior of pedestrian under binocular camera or RGB-D camera, Neng Goujing True statistical number of person.

To achieve the goals above, the present invention adopts the following technical scheme:

A kind of demographic method based on trajectory analysis, comprising the following steps:

Step 1: setting up RGB-D camera in the scene of channel, demarcate to camera, calculate the parameter matrix of camera, leads to Road includes the direction A and the direction B, and the two is contrary；

Step 2: being continuously shot the channel comprising human body target using camera, obtains N width depth map；It seeks every The top view of width depth map；Background I is sought using all top views sought_b；

Step 3: the channel comprising human body target is shot using camera, obtains the depth map of a certain moment m；Needle Its corresponding top view is obtained to the width depth map；Background operation is carried out for top view and obtains foreground picture, for prospect Picture carries out piecemeal and operates to obtain the picture after piecemeal, carries out searching local maxima region for the picture after piecemeal and operates to obtain Local maxima regional ensemble is extended the part after the operation of local maxima region is expanded for local maxima regional ensemble Maximum region set is filtered rectangle frame processing for the local maxima regional ensemble after extension, obtains including multiple members One rectangle frame set S of element_Fm；

Step 4: if step 3 obtain be initial time rectangle frame set S_Fm, then the rectangle frame of initial time is utilized Set S_FmA plurality of track is generated, a plurality of track forms a track set T_m；If step 3 obtain be the non-initial moment square Shape frame set S_Fm, then the rectangle frame set S at the non-initial moment is utilized_FmThe track set that previous moment has been formed is carried out It updates, obtains updated track set T1_m；

Step 5: if track set T_mOr updated track set T1_mIn certain track continuous several times not by more Newly, then the track is marked, and by the track by track set T_mOr updated track set T1_mInterior deletion, obtains New track set T2_m；

Step 6: in the track set T2 that step 5 obtains_mIt is middle to select track as sample, form set T_smIf sample Number reaches setting value, thens follow the steps seven, otherwise, executes step 3；

Step 7: it is directed to set T_smIn every track, extract the attribute of track, and record the corresponding label in every track Value L multiple attributes corresponding with the track, the corresponding mark value L in every track multiple attributes corresponding with the track form one Set D_l；

Step 8: upper detection faces δ of the channel on the direction A is arranged in the set obtained according to step 7_uWith lower detection faces δ_d, The upper detection faces δ of channel in directionb_u' and lower detection faces δ_d′；

Step 9: repeating the process of step 3 to step 5, in the process, for due to continuous δ_disSecondary do not update and From track set T_mEvery track of middle removal extracts the attribute of this track, this rail using method described in step 7 The attribute of mark includes the Y value F of the starting point of track₉With the Y value F of the terminal of track₁₀；

If F in the attribute of the track₉-F₁₀> 0, then it is the direction A, if meeting F₉≥δ_dAnd F₁₀≤δ_u, then the direction A number adds 1；If F₉-F₁₀< 0 is the direction B, if meeting F₉≤δ_u' and F₁₀≥δ_d', then the direction B number adds 1.

Step 10: repeating step 9, until camera stops shooting, obtain demographics result of the channel on the direction A and The demographics result of channel in directionb.

Further, the top view for seeking every width depth map in step 2 and step 3, the formula of use are as follows:

Len=m*r

Wherein,For the outer ginseng matrix of camera, p₁₁,p₁₂,p₁₃,p₁₄,p₂₁,p₂₂,p₂₃,p₂₄, p₃₁,p₃₂,p₃₃,p₃₄Element in corresponding Camera extrinsic matrix, θ are to pass through P (x on depth map_p,y_p,z_p) point correspondence ray with The angle of ground level；G(x_G,y_G, 0) and it was the oblique line of P point and the intersection point of ground level；H_CFor camera heights；M is P point in depth map In depth value and 0 < m < D, wherein D be max pixel value set by user；R is in world space corresponding to unit depth value Distance；

Top view I is obtained using following formula:

Wherein, (u, v) indicates that the pixel in the corresponding top view I of point P on depth map, I (u, v) indicate pixel Pixel value at (u, v), (r_x,r_y) it is to point (x_p,y_p) zoom factor, (d_x,d_y) it is to point (x_p,y_p) translation coefficient；

For each of depth map point, the picture at the pixel and the pixel in the corresponding top view of point is obtained Element value, all pixel values form top view I.

Further, it carries out background operation for top view in the step 3 and obtains foreground picture, the public affairs of use Formula is as follows:

Wherein, δ_FTo be set by user for extracting the threshold value of prospect, I_F(u, v) indicates foreground picture I_FMiddle pixel (u, V) pixel value at place, I_b(u, v) is Background I_bPixel value at the position pixel (u, v), I (u, v) indicate top view I picture Pixel value at vegetarian refreshments (u, v).

Further, it operates to obtain the picture after piecemeal for foreground picture progress piecemeal in the step 3, use Formula it is as follows:

Wherein, I_BTo carry out the picture after piecemeal, I for foreground picture_F(u, v) is foreground picture I_FCoordinate is (u's, v) Pixel value, I_B(x, y) is picture I_BPixel value at the position pixel (x, y), the size of the block of delimitation are w_b×w_b。

Further, carrying out finding local maxima region for the picture after piecemeal and operate to obtain office in the step 3 Portion's maximum region set, specifically includes the following steps:

For picture I_BOn pixel (x, y), eight pixels around the pixel are searched, if the pixel The corresponding pixel value of than eight pixels of corresponding pixel value will be big, which is put into local maxima regional ensemble S_L In, utilize S_L ⁽ⁱ⁾Indicate S_LMember, and S_L ⁽ⁱ⁾=(u_i,v_i,d_i), (u_i,v_i) indicate the pixel, d_iFor pixel (u_i,v_i) In picture I_BIn pixel value.

Further, being extended local maxima region for local maxima regional ensemble and operate in the step 3 Local maxima regional ensemble after to extension, specifically includes the following steps:

For local maxima regional ensemble S_LEach element S_L ⁽ⁱ⁾, find S_L ⁽ⁱ⁾In foreground picture I_FIn corresponding pixel Position, the formula of use are as follows:

Wherein, (x_i,y_i) it is S_L ⁽ⁱ⁾Corresponding to foreground picture I_FIn position；Enable S_S ⁽ⁱ⁾=(x_i,y_i,z_i), gathered S_S, S_S ⁽ⁱ⁾For set S_SElement；

For S_SIn each member S_S ⁽ⁱ⁾=(x_i,y_i,z_i), with S_S ⁽ⁱ⁾For seed, using se ed filling algorithm, to extending out Exhibition, the condition of extension are as follows: if | I_F(x_i,y_i)-z_i|≤δ_E, then a rectangle frame S is used_E ⁽ⁱ⁾=(u_i,v_i,H_i,W_i,z_i) frame chooses All pixels for meeting condition, wherein (u_i,v_i) it is rectangle frame upper left angle point, (H_i,W_i) be rectangle frame height and width, z_iFor original Beginning pixel value, δ_EFor defined threshold value, the set S of an extension rear region is formed_E, S_E ⁽ⁱ⁾For set S_EElement.

Further, being filtered at rectangle frame for the local maxima regional ensemble after extension in the step 3 Reason, obtain include multiple elements rectangle frame set, comprising the following steps:

Using two filter conditions to set S_EIn element be filtered:

(1) if element S_E ⁽ⁱ⁾Meet the following conditions:Then the element is deleted, wherein δ_HFor minimum length threshold Value, δ_WFor minimum widith threshold value；

(2) if two rectangle frame S_E ⁽ⁱ⁾=(u_i,v_i,H_i,W_i,z_i) and S_E ^(j)=(u_j,v_j,H_j,W_j,z_j), meetThen determine S_E ⁽ⁱ⁾And S_E ^(j)It is overlapped, if be overlapped, retains z_iAnd z_jIn biggish rectangle frame；

The rectangle frame retained is formed into rectangle frame set S_Fm, rectangle frame set S_FmIn element be S_Fm ⁽ⁱ⁾, wherein m table Show the moment.

Further, the step three in the step 4 obtain be initial time rectangle frame set, then using initial The rectangle frame set at moment generates a plurality of track, and a plurality of track forms a track set；If step 3 obtains being non-initial The rectangle frame set at moment, then using the non-initial moment rectangle frame set to the track that previous moment has been formed gather into Row updates, and obtains updated track set；Specifically includes the following steps:

If the rectangle frame set S that step 3 obtains_FmIn m be equal to 1, then with rectangle frame set S_FmIn each rectangle frame S_Fm ⁽ⁱ⁾For starting point, new track T is respectively created_m ⁽ⁱ⁾, i.e. T_m ⁽ⁱ⁾={ S_Fm ⁽ⁱ⁾, track T_m ⁽ⁱ⁾As track set T_mIn a member Element, i.e. T_m={ T_m ⁽ⁱ⁾| i=1 ..., N_Tm, wherein N_TmFor the rectangle frame set S at m moment_FmThe number of the track of formation；

If the rectangle frame set S that step 3 obtains_FmM be not equal to 1, then by rectangle frame set S_FmEach of element S_Fm ⁽ⁱ⁾With the rectangle frame set S at m-1 moment_F(m-1)The track set T formed_m-1={ T_m-1 ⁽ⁱ⁾Each of track point It is not matched, specific matching process is as follows:

Remember element S_Fm ⁽ⁱ⁾Central point isWhereinThe respectively cross of central point Coordinate and ordinate coordinate,Centered on the coordinate of short transverse put,For coordinate The pixel value at place；Track T_m-1 ⁽ⁱ⁾The last one rectangle frame central point be (x_m-1,y_m-1,I_F(x_m-1,y_m-1))；

IfWherein, δ_match The max-thresholds of rectangle frame are matched for two, then rectangle frame S_Fm ⁽ⁱ⁾With track T_m-1 ⁽ⁱ⁾Matching, if track T_m-1 ⁽ⁱ⁾Not with rectangle frame Set S_FmIn other rectangle frames matching, then by rectangle frame S_Fm ⁽ⁱ⁾It is added to track T_m-1 ⁽ⁱ⁾In, if track T_m-1 ⁽ⁱ⁾With square Shape frame set S_FmIn another rectangle frame S_Fm ^(j)Match, S_Fm ^(j)Central point be Then make the following judgment:

If

Then by rectangle frame S_Fm ^(j)By track T_m-1 ⁽ⁱ⁾It removes, by rectangle frame S_Fm ⁽ⁱ⁾It is added to track T_m-1 ⁽ⁱ⁾In；If being unsatisfactory for Above-mentioned condition, then rectangle frame S_Fm ^(j)It is retained in track T_m-1 ⁽ⁱ⁾In.

In the above process, rectangle frame set S_FmIn all elements S_Fm ⁽ⁱ⁾After above-mentioned matching, if it exists not with appoint It anticipates the rectangle frame of a path matching, then generates a new track, using the rectangle frame as first point of new track, and will The track of generation is added to track set T1_mIn.

Further, the attribute of the track in the step 7 includes that tracing point number scale is characterized variable F₁, track is in the side Y Upward span is denoted as characteristic variable F₂, the span of track in z-direction is denoted as characteristic variable F₃, the average number of people size of track It is denoted as characteristic variable F₄, the Euclidean distance between track and fit standard track is denoted as feature vector F₅, the slope of track is denoted as spy Levy variable F₆, the mean breadth of ' locked ' zone is denoted as feature vector F on track₇, the average length note of ' locked ' zone on track For feature vector F₈, the starting point Y value of track is denoted as feature vector F₉, the terminal Y value of track is denoted as feature vector F₁₀。

Further, the upper detection faces on the direction A in the step 8: δ_u=min ({ F₁₀ ⁽ⁱ⁾), { F₁₀ ⁽ⁱ⁾Indicate A The attribute F of all tracks in direction₁₀；

Lower detection faces on the direction A: δ_d=max ({ F₉ ⁽ⁱ⁾), { F₉ ⁽ⁱ⁾Indicate the direction A all tracks attribute F₉；

Upper detection faces on the direction B: δ_u'=min ({ F₉ ⁽ⁱ⁾), { F₉ ⁽ⁱ⁾Indicate the direction B all tracks attribute F₉；

Lower detection faces on the direction B: δ_d'=max ({ F₁₀ ⁽ⁱ⁾), { F₁₀ ⁽ⁱ⁾Indicate the direction B all tracks attribute F₁₀。

Compared with prior art, the present invention has following technical effect that this obtains field by binocular camera or RGBD camera The depth map of scape, demarcates camera, three-dimensional point cloud is converted by depth map using calibrated camera parameter, by three-dimensional point Cloud is projected to X-Y plane, obtains the top view of scene, using number of people locking means proposed by the present invention to the number of people in top view Target is locked, and is judged by detection faces track, or by trained classifier (Adaboost, SVM, Bayies etc.) judgement that track is entered and gone out.The present invention is for counting in scene (bus, staircase, passway Deng) pedestrians, the number for accurately being entered and being gone out.

Detailed description of the invention

Fig. 1 is camera scheme of installation；

Fig. 2 is that world coordinate system establishes schematic diagram；

Explanation and illustration in further detail is done to the solution of the present invention with reference to the accompanying drawings and detailed description.

Specific embodiment

Demographic method based on trajectory analysis of the invention, comprising the following steps:

Step 1: setting up camera in the scene of channel, demarcate to camera, calculates the parameter matrix P of camera；Specific packet Include following steps:

Step 1.1: choose scene of a certain channel as demographics, referring to Fig. 1, by camera be mounted on channel just on Side, multiple human body targets are walked along the direction A or the direction B on a passage, and the direction A and B are contrary；

Step 1.2: establishing world coordinate system.Referring to fig. 2, camera is located on the Z axis of world coordinate system, along the direction in channel It is the Y direction of world coordinate system, the direction perpendicular to channel is the X-direction of world coordinate system, and camera is in world coordinate system Position coordinates be (0,0, H), wherein H is the distance of camera distance world coordinate system origin.

Step 1.3: camera is demarcated.During camera calibration, N (N >=6) group image coordinate and right therewith is selected The world coordinates answered:

The parameter matrix P of camera is calculated using following formula:

Wherein,

Step 2: being continuously shot the channel comprising human body target using camera, obtains N (N >=50) width depth map； Seek the top view of every width depth map；Background I is sought using top view_b。

Wherein, the top view of every width depth map is sought, comprising the following steps:

What the depth value in depth map represented is the point in world coordinate space, such as the distance of point such as point P to video camera Len schemes the length of medium and small hypotenuse, we can obtain public as follows according to the geometrical relationship of object under world coordinate system Formula:

Len=m*r (4)

Wherein, θ is the angle on depth map by the correspondence ray of P point and ground level；G(x_G,y_G, 0) and it was the oblique of P point The intersection point of line and ground level；H_CFor camera heights；M (0 < m < D) is depth value of the P point in depth map, and wherein D is set by user Max pixel value；R is the distance in world space corresponding to unit depth value.

After obtaining the coordinate of P point, zooming and panning are carried out to P point, are located at the center of top view I, then:

Wherein, (u, v) indicates that the pixel in the corresponding top view I of point P, I (u, v) indicate the picture at pixel (u, v) Element value, wherein (r_x,r_y) it is to (the x of point P_p,y_p) zoom factor, (d_x,d_y) it is to (the x of point P_p,y_p) translation coefficient.

For each of depth map point, the pixel of the pixel and the pixel in the corresponding top view of point is obtained Value, all pixel values form top view I.N width top view I can get using the above method for N width depth map_i(i= 1,...N)。

Wherein, Background I is sought using top view_b, the formula of use is as follows；

Wherein, H is the length of top view, and W is the width of top view, I_b(x, y) is Background I_bIn the position pixel (x, y) The pixel value at place is set, Background I can be acquired_b。

Step 3: the channel comprising human body target is shot using camera, obtains the depth map at a certain moment；For The width depth map obtains its corresponding top view；Background, block are carried out for top view, find local maxima region, extension Local maxima region and filtering rectangle frame processing, obtain a rectangle frame set S_Fm；Specifically includes the following steps:

Step 3.1: channel being shot using camera, according to RGB-D camera, camera shooting can directly obtain a width Depth map, i.e. a certain moment m (m=1,2 ...) depth map；It shoots, can be shot according to camera according to binocular camera Left figure and right figure, by picture correction, Stereo matching, available frame depth map, i.e. a certain moment m (m=1,2 ...) Depth map.

Step 3.2, its corresponding top view I is obtained for the width depth map taken_m, used method and step The method that top view is obtained in two is identical.

Step 3.3, the top view I obtained for step 3.2_m, background, block are carried out, local maxima region is found, expands Local maxima region and filtering rectangle frame processing are opened up, a rectangle frame set S is obtained_Fm, concrete processing procedure is as follows:

It goes background: for top view I, foreground picture I being obtained using formula (8)_F:

Wherein, δ_FTo be set by user for extracting the threshold value of prospect, I_F(u, v) indicates foreground picture I_FMiddle pixel (u, V) pixel value at place.

Piecemeal operation: use size for w_b×w_bBlock to foreground picture I_FBlock is carried out, picture I is obtained_B, the public affairs of use Formula are as follows:

Wherein, I_F(u, v) is foreground picture I_FCoordinate is the pixel value of (u, v), I_B(x, y) is picture I_BPixel (x, Y) pixel value at position.

It finds local maxima region: being directed to picture I_BOn pixel (x, y), search eight pictures around the pixel The pixel is put into office if the corresponding pixel value of than eight pixels of the corresponding pixel value of the pixel will be big by vegetarian refreshments Portion maximum region set S_LIn, using S_L ⁽ⁱ⁾Indicate S_LElement, and S_L ⁽ⁱ⁾=(u_i,v_i,d_i), (u_i,v_i) indicate the pixel, d_iFor pixel (u_i,v_i) in picture I_BIn pixel value.

It extends local maxima region: being directed to local maxima regional ensemble S_LEach element S_L ⁽ⁱ⁾, find S_L ⁽ⁱ⁾In foreground picture Piece I_FIn corresponding location of pixels, the formula of use are as follows:

Wherein, (x_i,y_i) it is S_L ⁽ⁱ⁾Corresponding to foreground picture I_FIn position.Enable S_S ⁽ⁱ⁾=(x_i,y_i,z_i), (x_i,y_i) table Show S_L ⁽ⁱ⁾Corresponding to foreground picture I_FPixel, set S can be obtained_S, S_S ⁽ⁱ⁾For set S_SElement.

For S_SIn each member S_S ⁽ⁱ⁾=(x_i,y_i,z_i), with S_S ⁽ⁱ⁾For seed, using se ed filling algorithm, to extending out Exhibition, the condition of extension are as follows: if | I_F(x_i,y_i)-z_i|≤δ_E, δ_EFor defined threshold value=10, then a rectangle frame S is used_E ⁽ⁱ⁾= (u_i,v_i,H_i,W_i,z_i) frame chooses all pixels for meeting condition, wherein (u_i,v_i) it is rectangle frame upper left angle point, (H_i,W_i) be The height and width of rectangle frame, z_iFor original pixel value (i.e. the spatial altitude of rectangle frame), the collection of an extension rear region is eventually formed Close S_E, S_E ⁽ⁱ⁾For set S_EElement.

Filtering rectangle frame processing: it is expanded behind region, needs to filter overlapping region and improper region, use two mistakes Filter condition, if 1. rectangle frame S_E ⁽ⁱ⁾Meet the following conditions:Then not retain；2. if two rectangle frame S_E ⁽ⁱ⁾= (u_i,v_i,H_i,W_i,z_i) and S_E ^(j)=(u_j,v_j,H_j,W_j,z_j), meetThen determine S_E ⁽ⁱ⁾And S_E ^(j) It is overlapped, if be overlapped, retains z_iAnd z_jBiggish rectangle frame.

The rectangle frame retained forms rectangle frame set S_Fm, rectangle frame set S_FmIn element be S_Fm ⁽ⁱ⁾, so far complete mesh Mark locking task.

Step 4: if step 3 obtain be initial time rectangle frame set, utilize initial time rectangle frame collection At a plurality of track, a plurality of track forms a track set for symphysis；If step 3 obtain be the non-initial moment rectangle frame collection It closes, then the track set that previous moment has been formed is updated using the rectangle frame set at the non-initial moment, is obtained more Track set after new；Specifically includes the following steps:

If the rectangle frame set S that step 3 obtains_FmIn m be equal to 1, then be directed to rectangle frame set S_FmIn each element S_Fm ⁽ⁱ⁾, create a track T_m ⁽ⁱ⁾, and by S_Fm ⁽ⁱ⁾As track T_m ⁽ⁱ⁾First point, i.e. T_m ⁽ⁱ⁾={ S_Fm ⁽ⁱ⁾, track T_m ⁽ⁱ⁾ As track set T_mIn an element, i.e. T_m={ T_m ⁽ⁱ⁾| i=1 ..., N_Tm, wherein N_TmFor the rectangle frame set at m moment S_FmThe number of the track of formation；

If the rectangle frame set S that step 3 obtains_FmM be not equal to 1, then by rectangle frame set S_FmEach of element S_Fm ⁽ⁱ⁾With the rectangle frame set S at m-1 moment_F(m-1)The track set T formed_m-1={ T_m-1 ⁽ⁱ⁾Each of track point It is not matched, specific matching process is as follows:

Remember element S_Fm ⁽ⁱ⁾Central point isWhereinThe respectively cross of central point Coordinate and ordinate coordinate,Centered on the coordinate of short transverse put；Track T_m-1 ⁽ⁱ⁾The last one rectangle frame Central point be (x_m-1,y_m-1,I_F(x_m-1,y_m-1))；

If

Then rectangle frame S_Fm ⁽ⁱ⁾With track T_m-1 ⁽ⁱ⁾Matching, if track T_m-1 ⁽ⁱ⁾Not with rectangle frame set S_FmIn other rectangles Frame matching, then by rectangle frame S_Fm ⁽ⁱ⁾It is added to track T_m-1 ⁽ⁱ⁾In, if track T_m-1 ⁽ⁱ⁾With rectangle frame set S_FmIn in addition One rectangle frame S_Fm ^(j)Match, S_Fm ^(j)Central point be Then make the following judgment:

If

Then by rectangle frame S_Fm ^(j)By track T_m-1 ⁽ⁱ⁾It removes, by rectangle frame S_Fm ⁽ⁱ⁾It is added to track T_m-1 ⁽ⁱ⁾In；If being unsatisfactory for Above-mentioned condition, then rectangle frame S_Fm ^(j)It is retained in track T_m-1 ⁽ⁱ⁾In.

In the above process, rectangle frame set S_FmIn all elements S_Fm ⁽ⁱ⁾After above-mentioned matching, if it exists not with appoint It anticipates the rectangle frame of a path matching, then generates a new track, using the rectangle frame as first point of new track, and will The track of generation is added to track set T1_mIn.

Step 5: if track set T_mOr updated track set T1_mIn certain track continuous several times not by more Newly, then the track is marked, and by the track by track set T_mOr updated track set T1_mInterior deletion, obtains New track set T2_m；

Step 6: the artificial observation step 5 track set T2 that treated_mIn each track, if observing certain rail Mark determination is number of people track, then this track is labeled as positive sample, if observing, certain track determination is not number of people track (ratio Such as: the track of manpower, people's shoulder or knapsack), then this track is labeled as negative sample, if certain track can not determine whether for people Head rail mark, then do not mark.Set T will be put into labeled as the track of positive sample and negative sample_sm={ < L, T2_m ⁽ⁱ⁾>|T2_m ⁽ⁱ⁾For The track of label } in, L ∈ { -1,1 }, wherein -1 is negative sample, 1 is positive sample.If marker samples are enough, i.e. sample number M > 1000, step 7 is executed, step 3 is otherwise executed.

Step 7, for set T_smIn every track, extract the attribute of track, and record the corresponding label in every track Value L attribute corresponding with the track.

Wherein the attribute of track includes:

Track points: the number for the rectangle frame that track includes is denoted as characteristic variable F₁；

The span of track in the Y direction: the difference of the beginning and end Y value of track is denoted as characteristic variable F₂；

The span of track in z-direction: the difference of the Z value of the Z-direction highest point and Z-direction minimum point of track is denoted as feature Variable F₃；

The average number of people size of track: the average value of region area is locked on track, i.e., on track in all rectangle frames Area is denoted as characteristic variable F₄；

Euclidean distance between track and fit standard track, calculation method are as follows:

The central point of each rectangle frame in fit standard track is calculated with averaging method first

Wherein,For the central point of j-th of rectangle frame on fit standard track, S_F ^(i)(j)For set T_smIn j-th of rectangle frame on i-th track central point, N_T ^(j)For institute's rail at j there are rectangle frame The item number of mark.Fit standard track is represented by

By standard trajectoryIt projects on Y-Z plane, during realization, the track after projection is linearly inserted Value, the standard trajectory after obtaining linear interpolationLinear interpolation is referred to for some in Y-axis without correspondingPoint, benefit With the two sidesAverage value to the point carry out assignment.It will set T_smIn certain rail T⁽ⁱ⁾={ S_F ⁽¹⁾,S_F ⁽²⁾,…S_F ^(j)…,S_F ^(N(i))Project on Y-Z plane, wherein S_F ^(j)=(u_j,v_j,H_j,W_j,d_j) indicate track T⁽ⁱ⁾On j-th of rectangle The central point of frame, N (i) are track T⁽ⁱ⁾On rectangle frame number.Track T⁽ⁱ⁾For set T_smIn i-th track.

Calculate track T⁽ⁱ⁾With the Euclidean distance between fit standard track:

Euclidean distance is as feature vector F₅。

Current track: being fitted in alignment by the slope of track by the method for least square fitting, and it is straight to calculate this The slope of line is denoted as characteristic variable F₆。

The mean breadth of ' locked ' zone on track: the mean breadth for calculating the ' locked ' zone on track is denoted as feature vector F₇, i.e., the mean breadth of all rectangle frames.

The average length of ' locked ' zone on track: the average length for calculating the ' locked ' zone on track is denoted as feature vector F₈, i.e., the average length of all rectangle frames.

The starting point Y value of track is denoted as feature vector F₉, the terminal Y value of track is denoted as feature vector F₁₀。

The corresponding mark value L of recording track attribute corresponding with the track: < L, F₁,F₂,F₃,F₄,F₅,F₆,F₇,F₈,F₉,F₁₀ >。

The corresponding mark value L in every track attribute corresponding with the track, forms a set D_l={ D_l ⁽ⁱ⁾, i=1 ... N_l}{<L⁽ⁱ⁾,F₁ ⁽ⁱ⁾,F₂ ⁽ⁱ⁾,F₃ ⁽ⁱ⁾,F₄ ⁽ⁱ⁾,F₅ ⁽ⁱ⁾,F₆ ⁽ⁱ⁾,F₇ ⁽ⁱ⁾,F₈ ⁽ⁱ⁾,F₉ ⁽ⁱ⁾,F₁₀ ⁽ⁱ⁾> | i=1 ... N_l, wherein D_l ⁽ⁱ⁾Table Show set D_lIn i-th of element, N_lFor set D_lElement number.

Step 8: the set obtained according to step 7 is arranged upper detection faces and lower detection faces of the channel on the direction A, leads to The upper detection faces and lower detection faces of road in directionb.Specific setting method is as follows:

For set T_smIn certain track, if its corresponding attribute F₉-F₁₀> 0, then this track is the direction A；If its is right The attribute F answered₉-F₁₀< 0, then this track is the direction B；

Upper detection faces on the direction A: δ_u=min ({ F₁₀ ⁽ⁱ⁾), { F₁₀ ⁽ⁱ⁾Indicate the direction A all tracks attribute F₁₀；

Lower detection faces on the direction A: δ_d=max ({ F₉ ⁽ⁱ⁾), { F₉ ⁽ⁱ⁾Indicate the direction A all tracks category

Property F₉；

Upper detection faces on the direction B: δ_u'=min ({ F₉ ⁽ⁱ⁾), { F₉ ⁽ⁱ⁾Indicate the direction B all tracks attribute F₉；

Lower detection faces on the direction B: δ_d'=max ({ F₁₀ ⁽ⁱ⁾), { F₁₀ ⁽ⁱ⁾Indicate the direction B all tracks attribute F₁₀；

Step 9: repeating the process of step 3 to step 5, in the process, for due to continuous δ_disSecondary do not update and From track set T1_mEvery track of middle removal, the attribute of this track is extracted using method described in step 7, if the rail F in the attribute of mark₉-F₁₀> 0, then it is the direction A, if meeting F₉≥δ_dAnd F₁₀≤δ_u, then the direction A number adds 1；If F₉-F₁₀< 0 For the direction B, if meeting F₉≤δ_u' and F₁₀≥δ_d', then the direction B number adds 1.

Step 10: repeating step 9, until camera stops shooting, obtain demographics result of the channel on the direction A and The demographics result of channel in directionb.

Claims (10)

1. a kind of demographic method based on trajectory analysis, which comprises the following steps:

Step 1: setting up RGB-D camera in the scene of channel, demarcate to camera, calculates the parameter matrix of camera, channel packet The direction A and the direction B are included, the two is contrary；

Step 2: being continuously shot the channel comprising human body target using camera, obtains N width depth map；It is deep to seek every width Spend the top view of figure；Background I is sought using all top views sought_b；

Step 3: the channel comprising human body target is shot using camera, obtains the depth map of a certain moment m；For this Width depth map obtains its corresponding top view；Background operation is carried out for top view and obtains foreground picture, for foreground picture It carries out piecemeal to operate to obtain the picture after piecemeal, carries out searching local maxima region for the picture after piecemeal and operate to obtain part Maximum region set is extended the local maxima after the operation of local maxima region is expanded for local maxima regional ensemble Regional ensemble is filtered rectangle frame processing for the local maxima regional ensemble after extension, obtains including multiple elements One rectangle frame set S_Fm；

Step 4: if step 3 obtain be initial time rectangle frame set S_Fm, then the rectangle frame set of initial time is utilized S_FmA plurality of track is generated, a plurality of track forms a track set T_m；If step 3 obtain be the non-initial moment rectangle frame Set S_Fm, then the rectangle frame set S at the non-initial moment is utilized_FmThe track set that previous moment has been formed is updated, Obtain updated track set T1_m；

Step 5: if track set T_mOr updated track set T1_mIn certain track continuous several times be not updated, then The track is marked, and by the track by track set T_mOr updated track set T1_mInterior deletion obtains new Track set T2_m；

Step 6: in the track set T2 that step 5 obtains_mIt is middle to select track as sample, form set T_smIf sample number reaches To setting value, seven are thened follow the steps, otherwise, executes step 3；

Step 7: it is directed to set T_smIn every track, extract the attribute of track, and record the corresponding mark value L in every track Multiple attributes corresponding with the track, the corresponding mark value L in every track multiple attributes corresponding with the track form a collection Close D_l；

Step 8: upper detection faces δ of the channel on the direction A is arranged in the set obtained according to step 7_uWith lower detection faces δ_d, channel Upper detection faces δ in directionb_u' and lower detection faces δ_d′；

Step 9: repeating the process of step 3 to step 5, in the process, for due to continuous δ_disSecondary do not update and from rail Trace set T_mEvery track of middle removal extracts the attribute of this track using method described in step 7, this track Attribute includes the Y value F of the starting point of track₉With the Y value F of the terminal of track₁₀；

If F in the attribute of the track₉-F₁₀> 0, then it is the direction A, if meeting F₉≥δ_dAnd F₁₀≤δ_u, then the direction A number adds 1；If F₉-F₁₀< 0 is then the direction B, if meeting F₉≤δ_u' and F₁₀≥δ_d', then the direction B number adds 1；

Step 10: repeating step 9, until camera stops shooting, obtains demographics result and channel of the channel on the direction A Demographics result in directionb.

2. as described in claim 1 based on the demographic method of trajectory analysis, which is characterized in that in step 2 and step 3 The top view for seeking every width depth map, the formula of use is as follows:

Len=m*r

Wherein,For the outer ginseng matrix of camera, p₁₁,p₁₂,p₁₃,p₁₄,p₂₁,p₂₂,p₂₃,p₂₄,p₃₁, p₃₂,p₃₃,p₃₄Element in corresponding Camera extrinsic matrix, θ are to pass through P (x on depth map_p,y_p,z_p) point correspondence ray and Horizon The angle in face；G(x_G,y_G, 0) and it was the oblique line of P point and the intersection point of ground level；H_CFor camera heights；M is P point in depth map Depth value and 0 < m < D, wherein D is max pixel value set by user；R is in world space corresponding to unit depth value Distance；

Top view I is obtained using following formula:

Wherein, (u, v) indicates that the pixel in the corresponding top view I of point P on depth map, I (u, v) indicate top view I pixel Pixel value at point (u, v), (r_x,r_y) it is to point (x_p,y_p) zoom factor, (d_x,d_y) it is to point (x_p,y_p) translation coefficient；

For each of depth map point, the pixel at the pixel and the pixel in the corresponding top view of point is obtained Value, all pixels form top view I.

3. as claimed in claim 2 based on the demographic method of trajectory analysis, which is characterized in that the needle in the step 3 Background operation is carried out to top view and obtains foreground picture, the formula of use is as follows:

Wherein, δ_FTo be set by user for extracting the threshold value of prospect, I_F(u, v) indicates foreground picture I_FAt middle pixel (u, v) Pixel value, I_b(u, v) is Background I_bPixel value at the position pixel (u, v), I (u, v) indicate top view I pixel Pixel value at (u, v).

4. as claimed in claim 3 based on the demographic method of trajectory analysis, which is characterized in that the needle in the step 3 Picture after piecemeal operates to obtain piecemeal is carried out to foreground picture, the formula of use is as follows:

Wherein, I_BTo carry out the picture after piecemeal, I for foreground picture_F(u, v) is foreground picture I_FCoordinate is the pixel of (u, v) Value, I_B(x, y) is picture I_BPixel value at the position pixel (x, y), the size of the block of delimitation are w_b×w_b。

5. as claimed in claim 4 based on the demographic method of trajectory analysis, which is characterized in that the needle in the step 3 Searching local maxima region is carried out to the picture after piecemeal to operate to obtain local maxima regional ensemble, specifically includes the following steps:

For picture I_BOn pixel (x, y), eight pixels around the pixel are searched, if the pixel is corresponding The corresponding pixel value of than eight pixels of pixel value will be big, which is put into local maxima regional ensemble S_LIn, benefit Use S_L ⁽ⁱ⁾Indicate S_LMember, and S_L ⁽ⁱ⁾=(u_i,v_i,d_i), (u_i,v_i) indicate the pixel, d_iFor pixel (u_i,v_i) in picture I_BIn pixel value.

6. as claimed in claim 5 based on the demographic method of trajectory analysis, which is characterized in that the needle in the step 3 Local maxima regional ensemble after the operation of local maxima region is expanded is extended to local maximum region set, it is specific to wrap Include following steps:

For local maxima regional ensemble S_LEach element S_L ⁽ⁱ⁾, find S_L ⁽ⁱ⁾In foreground picture I_FIn corresponding location of pixels, The formula of use are as follows:

Wherein, (x_i,y_i) it is S_L ⁽ⁱ⁾Corresponding to foreground picture I_FIn position；Enable S_S ⁽ⁱ⁾=(x_i,y_i,z_i), obtain set S_S, S_S ⁽ⁱ⁾ For set S_SElement；

For S_SIn each member S_S ⁽ⁱ⁾=(x_i,y_i,z_i), with S_S ⁽ⁱ⁾It is extended to the outside, is expanded using se ed filling algorithm for seed The condition of exhibition are as follows: if | I_F(x_i,y_i)-z_i|≤δ_E, wherein I_F(x_i,y_i) indicate foreground picture I_FMiddle coordinate is (x_i,y_i) pixel Value then uses a rectangle frame S_E ⁽ⁱ⁾=(u_i,v_i,H_i,W_i,z_i) frame chooses all pixels for meeting condition, wherein (u_i,v_i) For rectangle frame upper left angle point, (H_i,W_i) be rectangle frame height and width, z_iFor the spatial altitude of rectangle frame, δ_EFor defined threshold value, Form the set S of an extension rear region_E, S_E ⁽ⁱ⁾For set S_EElement.

7. as claimed in claim 6 based on the demographic method of trajectory analysis, which is characterized in that the needle in the step 3 To the local maxima regional ensemble after extension be filtered rectangle frame processing, obtain include multiple elements rectangle frame set, The following steps are included:

Using two filter conditions to set S_EIn element be filtered:

(1) if element S_E ⁽ⁱ⁾Meet the following conditions:Then the element is deleted, wherein δ_HFor minimum constructive height threshold value, δ_W For minimum widith threshold value；

(2) if two rectangle frame S_E ⁽ⁱ⁾=(u_i,v_i,H_i,W_i,z_i) and S_E ^(j)=(u_j,v_j,H_j,W_j,z_j), meetThen determine S_E ⁽ⁱ⁾And S_E ^(j)It is overlapped, if be overlapped, retains z_iAnd z_jIn biggish rectangle frame；

The rectangle frame retained is formed into rectangle frame set S_Fm, rectangle frame set S_FmIn element be S_Fm ⁽ⁱ⁾, wherein when m is indicated It carves.

8. as described in claim 1 based on the demographic method of trajectory analysis, which is characterized in that the step in the step 4 Rapid three obtain be initial time rectangle frame set, then generate a plurality of track using the rectangle frame set of initial time, it is a plurality of Track forms a track set；If step 3 obtain be the non-initial moment rectangle frame set, using this it is non-initial when The rectangle frame set at quarter is updated the track set that previous moment has been formed, and obtains updated track set；Specifically The following steps are included:

If the rectangle frame set S that step 3 obtains_FmIn m be equal to 1, then with rectangle frame set S_FmIn each rectangle frame S_Fm ⁽ⁱ⁾ For starting point, new track T is respectively created_m ⁽ⁱ⁾, i.e. T_m ⁽ⁱ⁾={ S_Fm ⁽ⁱ⁾, track T_m ⁽ⁱ⁾As track set T_mIn an element, i.e., T_m={ T_m ⁽ⁱ⁾| i=1 ..., N_Tm, wherein N_TmFor the rectangle frame set S at m moment_FmThe number of the track of formation；

If the rectangle frame set S that step 3 obtains_FmM be not equal to 1, then by rectangle frame set S_FmEach of element S_Fm ⁽ⁱ⁾ With the rectangle frame set S at m-1 moment_F(m-1)The track set T formed_m-1={ T_m-1 ⁽ⁱ⁾Each of track respectively into Row matching, specific matching process are as follows:

Remember element S_Fm ⁽ⁱ⁾Central point isWhereinThe respectively abscissa of central point And ordinate,For coordinateThe pixel value at place；Track T_m-1 ⁽ⁱ⁾The last one rectangle frame center Point is (x_m-1,y_m-1,I_F(x_m-1,y_m-1))；

IfWherein, δ_matchIt is two The max-thresholds of a matching rectangle frame, then rectangle frame S_Fm ⁽ⁱ⁾With track T_m-1 ⁽ⁱ⁾Matching, if track T_m-1 ⁽ⁱ⁾Not with rectangle frame set S_FmIn other rectangle frames matching, then by rectangle frame S_Fm ⁽ⁱ⁾It is added to track T_m-1 ⁽ⁱ⁾In, if track T_m-1 ⁽ⁱ⁾With rectangle frame Set S_FmIn another rectangle frame S_Fm ^(j)Match, S_Fm ^(j)Central point beThen into The following judgement of row:

If

Then by rectangle frame S_Fm ^(j)By track T_m-1 ⁽ⁱ⁾It removes, by rectangle frame S_Fm ⁽ⁱ⁾It is added to track T_m-1 ⁽ⁱ⁾In；If being unsatisfactory for above-mentioned Condition, then rectangle frame S_Fm ^(j)It is retained in track T_m-1 ⁽ⁱ⁾In；

In the above process, rectangle frame set S_FmIn all elements S_Fm ⁽ⁱ⁾After above-mentioned matching, if it exists not with it is any one The rectangle frame of path matching then generates a new track, using the rectangle frame as first point of new track, and will generate Track be added to track set T1_mIn.

9. as described in claim 1 based on the demographic method of trajectory analysis, which is characterized in that the rail in the step 7 The attribute of mark includes that tracing point number scale is characterized variable F₁, the span of track in the Y direction is denoted as characteristic variable F₂, track is in the side Z Upward span is denoted as characteristic variable F₃, the average number of people size of track is denoted as characteristic variable F₄, track and fit standard track it Between Euclidean distance be denoted as feature vector F₅, the slope of track is denoted as characteristic variable F₆, the mean breadth note of ' locked ' zone on track To be denoted as feature vector F₇, the average length of ' locked ' zone is denoted as feature vector F on track₈, the starting point Y value of track is denoted as feature Vector F₉, the terminal Y value of track is denoted as feature vector F₁₀。

10. as claimed in claim 9 based on the demographic method of trajectory analysis, which is characterized in that the A in the step 8 Upper detection faces on direction: δ_u=min ({ F₁₀ ⁽ⁱ⁾), { F₁₀ ⁽ⁱ⁾Indicate the direction A all tracks attribute F₁₀；

Lower detection faces on the direction A: δ_d=max ({ F₉ ⁽ⁱ⁾), { F₉ ⁽ⁱ⁾Indicate the direction A all tracks category

Property F₉；

Upper detection faces on the direction B: δ_u'=min ({ F₉ ⁽ⁱ⁾), { F₉ ⁽ⁱ⁾Indicate the direction B all tracks attribute F₉；

Lower detection faces on the direction B: δ_d'=max ({ F₁₀ ⁽ⁱ⁾), { F₁₀ ⁽ⁱ⁾Indicate the direction B all tracks attribute F₁₀。