CN105894505A - Quick pedestrian positioning method based on multi-camera geometrical constraint - Google Patents

Abstract

The invention provides a quick pedestrian positioning method based on multi-camera geometrical constraint. In view of multiple points and high algorithm complexity of a multi-camera pedestrian positioning method, the quick pedestrian positioning method comprises: for a captured scene, firstly measuring the coordinate of the ground projective point of each camera center and setting an identification point on a scene plane, computing a homographic matrix by using a normalization method according to a corresponding relation between the spatial point and the image point of the identification point; framing a pedestrian area by using a rectangular frame, projecting the two upper vertexes of the rectangular frame onto the scene plane, connecting the projective points with the projective points of the respective camera centers, and using the intersections of the connecting lines as candidate target foot points; projecting the foot points of the rectangular frame onto the scene plane, computing the centers of gravity of the foot points, and assigning appropriate weights to the candidate target points; and determining the space coordinates of pedestrians. The quick pedestrian positioning method may quickly position the foot point coordinate of the pedestrians on the scene plane, and may guarantee a high-precision position result both on shielded and non-shielded conditions.

Description

A kind of quick pedestrian's localization method based on multiple-camera geometrical constraint

Technical field

The invention belongs to image real time transfer or the image analysis technology in the field of generation, be specifically related to a kind of based on many shootings Quick pedestrian's localization method of machine geometrical constraint.

Background technology

In recent years, along with smart city and the fast development of intelligent transportation, Intelligent Video Surveillance Technology was also in fast development In.People is freed from complicated monitoring work by the development of intelligent video monitoring, utilize image procossing, computer vision with And the technology such as pattern recognition, automatically identify the different objects in monitored picture, the useful information in extraction video source, quickly position The scene of the accident, it is judged that abnormal conditions.

Current Video Supervision Technique focuses primarily upon the detect and track of target position in two dimensional image plane, right The fewest in the sterically defined research of target, and the locus of target has the feature of uniqueness.At prison focusing on people In Ore-controlling Role, utilizing video camera to position the pedestrian in scene is an important technology.Divide from visual angle, video camera Location technology can be divided into two kinds, single camera location and multiple-camera colocated.Single camera positions mainly by foundation Single camera model carries out space orientation to target, but single camera location has a weak point to be hidden in target exactly In the case of gear, limited owing to obtaining target prospect information, therefore cannot be recovered space mesh by two-dimensional image information accurately Mark.The conventional method setting up camera model is to try to achieve camera interior and exterior parameter by camera calibration, and then tries to achieve shooting The projection matrix of machine, obtains space coordinates by the match point of on plane of delineation point or different visual angles is carried out projection Point.Wherein the demarcation of inside and outside parameter has multiple method, and the most classical is the plane template standardizition of Zhang Zhengyou.In recent years, along with The scope of video monitoring progressively expands, and the human body in video monitoring positions the research with tracking the most progressively from single camera Being transformed in multiple-camera, compare and single camera, multiple-camera can show better performance.First multiple cameras environment Lower some geometrical constraints of existence, can be the certain help of positioning belt, secondly, if there is the overlapping ken in multiple-camera, So when target is blocked in the overlapping ken, it is possible to use multiple foreground information obtains more accurate object space and sits Mark.

The method utilizing multiple-camera geometrical constraint character to carry out pedestrian location is usually foreground pixel points whole to pedestrian Carry out list should project, in conjunction with multiple-camera geometrical constraint, subpoint is carried out optimization process, obtain target foot point position.This The method of kind is numerous due to subpoint data to be processed, and the real-time of location algorithm is very poor, it is generally required to use hardware-accelerated next Realize real-time location, be difficult to apply in the video monitoring system of reality.

Summary of the invention

The present invention is directed to technical problem is that multiple-camera pedestrians based on all foreground pixel points location in prior art Method problem numerous, that algorithm complex is high of counting to be processed.

For solving the problems referred to above, the present invention proposes a kind of quick pedestrian side of location based on multiple-camera geometrical constraint Method, it is possible to realize the quick space orientation of pedestrian in monitoring scene.

The basic thought of the present invention is: for captured scene, first, utilizes gps receiver to measure camera center Floor projection point coordinates also arranges identification point in scene plane, closes by the spatial point of identification point is corresponding with the coordinate of picture point System, utilizes normalization method to calculate homography matrix；Then, with rectangle frame, the pedestrian area frame on foreground image is lived, by rectangle frame Both the above summit projects in scene plane, connects these subpoints and the camera center subpoint corresponding with it, will be even The intersection point of line is as candidate target foot point；Then, by the foot spot projection of rectangle frame to scene plane, and its center of gravity is calculated, right Apart from this center, candidate's foot point farther out gives less weight, and the nearer candidate's foot point of this center of gravity of adjusting the distance gives bigger power Weight；Finally, using candidate's foot point coordinates of center of gravity and different weights and be added and be averaging the space coordinates as pedestrian.

The present invention specifically includes following steps:

(1) normalization method is utilized to calculate the homography matrix between scene plane and camera image plane；

(2) center of gravity of vertex of surface subpoint under the rectangle frame of expression pedestrian's foreground area is calculated；

(3) pedestrian's candidate's foot point coordinates in scene plane is calculated；

(4) being that each candidate's foot point above-mentioned calculates weight, the meansigma methods that the some weighted sum of candidate's foot is added with center of gravity is made For pedestrian's space coordinates.

Further, in above-mentioned steps 1, calculate the homography matrix between scene plane and camera image plane and use normalizing Change method, specifically includes following steps:

(1) normalized image identification point x_i: calculate a similitude transformation matrix N only comprising displacement and scaling, will some x_iBecome Change to an x_i ², its midpoint x_i ²Each coordinate components and be positioned at initial point (0,0)^T, and they to the average distance of initial point are

(2) normalization scene plane mark point X_i: for scene plane, design a similitude transformation matrix N', will some X_iBecome Change to X_i ²；

(3) singular value decomposition: utilize the Method of Direct Liner Transformation of singular value decomposition to solve scene plane and camera review Corresponding point x in plane_i ²With an X_i ²Between homography matrix H'；

(4) normalization is solved: make H=N'^-1H'N。

In above-mentioned steps 2, when calculating the center of gravity of vertex of surface subpoint under the rectangle frame representing pedestrian's foreground area, first will In each video camera prospect, represent the rectangle frame following two summit of pedestrian's foreground area, shown up by homography matrix projection In scape plane, obtainThen utilize method of least square, try to achieve the central point of optimum as above-mentioned projection Center of gravity P of point₀。

In above-mentioned steps 3, calculating pedestrian's candidate's foot point coordinates in scene plane, foundation is under multiple cameras environment Geometrical constraint, specifically includes following steps:

(1) gps receiver is utilized to record the camera center gps coordinate at scene plane upright projection point_gC₁、_gC₂, letter Claim camera center subpoint；

(2) by each video camera prospect, represent the rectangle frame both the above summit of pedestrian's foreground area, answered by single Matrix projection, in scene plane, obtains

(3) connect above-mentioned subpoint and corresponding camera center subpoint, obtain 4 straight lines being positioned in scene plane

(4) the intersection point P of above-mentioned straight line is calculated₁、P₂、P₃、P₄, and using intersecting point coordinate as candidate's foot point coordinates.

In above-mentioned steps 4, first it is that each candidate's foot point calculates weight, then utilizes the candidate's foot after center of gravity and weighting Point calculates pedestrian position in scene plane, and wherein, the size of weight depends on the distance of candidate point distance center of gravity, specifically wraps Containing following steps:

(1) distance of candidate's foot point distance center of gravity is calculated | P₀P_i|, wherein i=1,2,3,4；

(2) to above-mentioned apart from inverted, r is obtained_i；

(3) by r corresponding for each candidate point_iWith all r_iThe ratio of sum is as the weight of corresponding candidate point；

(4) using the meansigma methods after the results added of center of gravity and the some weighted sum of candidate's foot as pedestrian's space coordinates O.

Beneficial effect: the present invention utilizes a rectangle frame to represent the foreground area of pedestrian, only enters the summit of rectangle frame Row projection, decreases prospect to be processed and counts, therefore, it is possible to quickly position the foot point coordinates in pedestrian's scape on the scene plane, improves The real-time of location；Utilize head point and the relation of foot spot projection simultaneously, use the method that weighted sum is averaging so that even if In the case of target foot o'clock is sightless in a camera coverage, still the method for above-mentioned weighting can be utilized to try to achieve target accurate Positioning result, it is possible under unobstructed and circumstance of occlusion, all ensure high-precision positioning result.

Accompanying drawing explanation

Fig. 1 pinhole camera modeling.

Fig. 2 candidate's foot point produces schematic diagram.

Candidate's foot point of Fig. 3 Weight and center of gravity schematic diagram.

The schematic flow sheet of Fig. 4 localization method of the present invention specific embodiment.

Detailed description of the invention

Detailed description of the invention to the present invention does the most detailed explanation below in conjunction with the accompanying drawings.

The invention provides a kind of quick pedestrian's localization method based on multiple-camera geometrical constraint, the method is embodied as Mode is divided into following 4 steps:

1, normalization method is utilized to calculate the homography matrix between scene plane and camera image plane.

The camera model that the present invention uses is pinhole camera modeling, and Fig. 1 is the coordinate relation of pinhole camera modeling. As an embodiment, use 2 video cameras.Fig. 4 is the flow process that localization method of the present invention is applied in this specific embodiment Schematic diagram.By 2 camera arrangements at height 4.5m, the position of shooting angle about 90 ° is fixing puts, and is directed at scene center simultaneously Region shoots.Photographed scene plane is chosen 6 identification point X_i(i=1,2,3,4,5,6), and survey with gps receiver Obtain the gps coordinate of these 6 points, record these 6 some coordinate x in 2 camera image plane simultaneously_i、x_i' (i=1,2,3, 4,5,6), x_i=(m_i,n_i)^T。

As a example by one of them video camera, identification point coordinate is normalized conversion, then utilizes direct linear transformation (DLT) method seeks homography matrix, and concrete grammar is as follows.

First identification point image coordinate is normalized, orderCan be constructed as follows is similar Transformation matrix:

$N_1=\left[\begin{array}{cc}1& -r_1/r_2\\ 0& 1\end{array}\right]---\left(1\right)$

WarpAfter similarity transformationAnd meetI.e. image after conversion Coordinate center falls at zero.

Then another one similitude transformation matrix can be constructed:

OrderSimilitude transformation matrix is:

$N_2=\left[\begin{array}{cc}1& 0\\ 0& s\end{array}\right]---\left(2\right)$

WarpAfter similarity transformation,And meet

I.e. after conversion, the distribution of each coordinate components is more uniform,

Then, identification point gps coordinate is normalized conversion,

Concrete steps are converted by above-mentioned normalization, can be bySolve the corresponding point after normalization's Homography matrix H₁', then do normalized inverse transformation, by H₁=N'^-1H₁' N obtains corresponding pointHomography matrix.With The available another one camera image plane of reason and the homography matrix H of scene plane₂。

2, the center of gravity of vertex of surface subpoint under the rectangle frame of expression pedestrian's foreground area is calculated.

The scene size about 120m of video camera shooting², 2 camera field are overlapping, clap the pedestrian in scene domain Take the photograph, obtain video sequence PARK.Being lived by the pedestrian area frame photographed with rectangle frame, now the both the above summit of rectangle frame is i.e. Can represent pedestrian head, following two summit can represent pedestrian foot.Use T₀、T₁Represent both the above summit, be designated as a point, B₀、 B₁Represent following two summit, be designated as foot point.Foot point in each visual angle carries out list should project, and little square law meter is made in utilization Calculate center of gravity P of foot spot projection point₀, such as formula (3).

$P_0=\arg \underset{P}{min}\underset{i=1}{\overset{2}{\Σ}}\underset{j=0}{\overset{1}{\Σ}}D(B_j^i,P)---\left(3\right)$

Wherein i=1,2 represent i-th video camera, j=0, and 1 represents jth point, and P represents the stochastic variable of focus point,Representing the distance of foot spot projection point and center of gravity variable, the physical significance of whole formula represents that foot spot projection point is heavy with this The value of center of gravity variable corresponding during heart variable distance sum minimum.

3, pedestrian's candidate's foot point coordinates in scene plane is calculated.

Gps receiver is utilized to record the camera center gps coordinate at scene plane upright projection point_gC₁、_gC₂, abbreviation is taken the photograph Camera central projection point.Correct point carries out list and should project, and connects this subpoint and corresponding camera center subpoint, according to taking the photograph more Geometrical constraint under camera environment, the intersection point obtained is candidate's foot point, such as Fig. 2.Wherein geometrical constraint is as follows:

Constraint 1: in the single camera visual field, the two projection line extended lines of singly answering being perpendicular to the vertical line on ground meet at a bit, should Point is video camera center of gravity subpoint.

In constraint 2: two camera coverages, the projection line of singly answering of a vertical line being perpendicular to ground meets at a bit, and this point is This root vertical line intersection point on the ground.

Note l1, l2 are respectively two straight lines, and the end points of straight line is respectively (x₀,y₀),(x₁,y₁),(x₂,y₂),(x₃,y₃)。 L1:y=k₁(x-x₀)+y₀, l2:y=k₂(x-x₂)+y₂, k₁、k₂Being the slope of two straight lines, two straight lines meet at a bit that (x y), is Candidate's foot point.

Simultaneous

$\left\{\begin{array}{c}y=k_1(x-x_0)+y_0\\ y=k_2(x-x_2)+y_2\end{array}\right.---\left(4\right)$

Can obtain

$\left\{\begin{array}{c}x=\frac{y_{22}-y_1+k_1{x}_1-k_2{x}_2}{(k_1-k_2)}\\ y=\frac{k_1{k}_2(x_2-x_0)+k_1{y}_2-k_2{y}_1}{k_2-k_1}\end{array}\right.---\left(5\right)$

4, weight, the meansigma methods conduct that the some weighted sum of candidate's foot is added are calculated with center of gravity for each candidate's foot point above-mentioned Pedestrian's space coordinates.

Note candidate's foot point is P_i, i=1,2,3,4, P_iDistance P₀Distance be | P₀P_i|.OrderCandidate point distance The biggest r of distance of center of gravity_iThe least.Weight w is defined for i-th candidate point_iAs follows:

$w_1=\frac{r_1}{\underset{i=1}{\overset{4}{\Σ}}{r}_i},w_2=\frac{r_2}{\underset{i=1}{\overset{4}{\Σ}}{r}_i},w_3=\frac{r_3}{\underset{i=1}{\overset{4}{\Σ}}{r}_i},w_4=\frac{r_4}{\underset{i=1}{\overset{4}{\Σ}}{r}_i}---\left(6\right)$

Fig. 3 is the candidate's foot point comprising weight and the schematic diagram of foot point center of gravity, then space coordinates O of pedestrian can be by formula (7) obtain:

$O=\frac{P_0+\underset{i=1}{\overset{4}{\Σ}}{w}_i{P}_i}{2}---\left(7\right)$

When a visual angle is blocked target foot point wherein, corresponding weighted term can be removed, utilize and be not blocked Foot point weighted term position, thus effectively process occlusion issue.

Experimental result as shown in Table 1 and Table 2, table 1 be unobstructed in the case of two target positioning results, table 2 is two targets The mutually target positioning result under circumstance of occlusion.In the case of unobstructed, the actual range between two targets is 2.62m, blocks In the case of distance between two targets be 1.10m, the operation time refers to run 200 frame video images, and the most every 4 frames process one The secondary required time.

Table 1

Table 2

It should be noted that the foregoing is only a specific embodiment of the present invention, not in order to limit the present invention, this In embodiment, data set used and attack mode are only limitted to the present embodiment, all within the spirit and principles in the present invention, made Any modification, equivalent substitution and improvement etc., should be included within the scope of the present invention.

Claims (5)

1. quick pedestrian's localization method based on multiple-camera geometrical constraint, it is characterised in that the method includes following step Rapid:

(1) normalization method is utilized to calculate the homography matrix between scene plane and camera image plane；

(2) center of gravity of vertex of surface subpoint under the rectangle frame of expression pedestrian's foreground area is calculated；

(3) pedestrian's candidate's foot point coordinates in scene plane is calculated；

(4) being that each candidate's foot point above-mentioned calculates weight, the meansigma methods that the some weighted sum of candidate's foot is added with center of gravity is as row People's space coordinates.

Quick pedestrian's localization method based on multiple-camera geometrical constraint the most according to claim 1, it is characterised in that institute State the homography matrix employing normalization method that step 1 calculates between scene plane and camera image plane, specifically include following step Rapid:

(1) normalized image identification point x_i: calculate a similitude transformation matrix N only comprising displacement and scaling, will some x_iTransform to Point x_i ², its midpoint x_i ²Each coordinate components and be positioned at initial point (0,0)^T, and they to the average distance of initial point are

(2) normalization scene plane mark point X_i: for scene plane, design a similitude transformation matrix N', will some X_iTransform to X_i ²；

(3) singular value decomposition: utilize the Method of Direct Liner Transformation of singular value decomposition to solve scene plane and camera image plane Upper corresponding point x_i ²With an X_i ²Between homography matrix H'；

(4) normalization is solved: make H=N'^-1H'N。

Quick pedestrian's localization method based on multiple-camera geometrical constraint the most according to claim 1, it is characterised in that institute State step 2 and calculate the center of gravity of vertex of surface subpoint under the rectangle frame representing pedestrian's foreground area, first by before each video camera Jing Zhong, represents the rectangle frame following two summit of pedestrian's foreground area, projects to, in scene plane, obtain by homography matrix Then utilize method of least square, try to achieve central point center of gravity P as above-mentioned subpoint of optimum₀。

Quick pedestrian's localization method based on multiple-camera geometrical constraint the most according to claim 1, it is characterised in that described Step 3 calculates pedestrian's candidate's foot point coordinates in scene plane, according to being the geometrical constraint under multiple cameras environment, specifically wraps Include following steps:

(1) gps receiver is utilized to record the camera center gps coordinate at scene plane upright projection point_gC₁、_gC₂, it is called for short shooting Machine central projection point；

(2) by each video camera prospect, represent the rectangle frame both the above summit of pedestrian's foreground area, pass through homography matrix Project to, in scene plane, obtain

(3) connect above-mentioned subpoint and corresponding camera center subpoint, obtain 4 straight lines being positioned in scene plane

(4) the intersection point P of above-mentioned straight line is calculated₁、P₂、P₃、P₄, and using intersecting point coordinate as candidate's foot point coordinates.

Quick pedestrian's localization method based on multiple-camera geometrical constraint the most according to claim 1, it is characterised in that institute State step 4 and first calculate weight for each candidate's foot point, then utilize the candidate's foot point after center of gravity and weighting to calculate pedestrian and exist Position in scene plane, wherein, the size of weight depends on the distance of candidate point distance center of gravity, specifically comprises the steps of

(1) distance of candidate's foot point distance center of gravity is calculated | P₀P_i|, wherein i=1,2,3,4；

(2) to above-mentioned apart from inverted, r is obtained_i；

(3) by r corresponding for each candidate point_iWith all r_iThe ratio of sum is as the weight of corresponding candidate point；

(4) using the meansigma methods after the results added of center of gravity and the some weighted sum of candidate's foot as pedestrian's space coordinates O.