CN108447090A

CN108447090A - The method, apparatus and electronic equipment of object gesture estimation

Info

Publication number: CN108447090A
Application number: CN201611130138.9A
Authority: CN
Inventors: 熊怀欣
Original assignee: Ricoh Co Ltd
Current assignee: Ricoh Co Ltd
Priority date: 2016-12-09
Filing date: 2016-12-09
Publication date: 2018-08-24
Anticipated expiration: 2036-12-09
Also published as: CN108447090B

Abstract

The present invention provides the method, apparatus and electronic equipment of a kind of estimation of object gesture, belong to image and field of video processing.The method of object gesture estimation is estimated posture object progress Attitude estimation using multiple cameras to same, the method includes：It determines the multiple camera and to be estimated in posture object composition system camera posture between camera two-by-two poor, the view shot using the multiple camera carries out Attitude estimation respectively, multiple initial attitude estimated results are obtained, at least two initial attitude estimated results that can be merged are selected from the multiple initial attitude estimated result according to the camera posture difference；Optimization object function and its constraints are established using at least two initial attitudes estimated result, is solved to obtain the amendment increment of each initial attitude estimated result；Object's pose estimation result is obtained using the amendment incremental computations.Technical scheme of the present invention can improve the precision and confidence level of object gesture estimation.

Description

The method, apparatus and electronic equipment of object gesture estimation

Technical field

The present invention relates to images and field of video processing, particularly relate to a kind of method, apparatus and electricity of object gesture estimation Sub- equipment.

Background technology

The behavioural analysis of driver is the most important thing of safety driving system, and the head pose estimation of wherein driver is The basis of fatigue driving and dispersion attention detection, has attracted numerous researchers to estimate to optimize the head pose of driver Meter, to reduce the possibility of traffic accident generation.Wherein the Attitude estimation of view-based access control model works because of its non-interfering type and easily disposes Characteristic and be widely used.The posture on usual head has 3 degree of freedom, can use Pitch (pitch angle), Roll (roll angle), Indicate that these three angles correspond to a unique spin matrix with the angle of three orthogonal directions of Yaw (yaw angle).Based on regarding The 3D Attitude estimations of feel are exactly to estimate head from the 2D images of shooting relative to the suitable rotation and translation of camera coordinates to become It changes.

Current most head pose estimation method can be divided into 2 classes：Method based on model and based on the apparent side of image Method.The former is normally based on the correspondence of 2D faces feature and 3D head models or the geometric attribute of feature based point.Then Person is returned or posture is estimated in the methods of manifold insertion by classifying, and in most cases can only be obtained and relatively coarse to be estimated Meter.

Since the head of people can be seen as cylinder or spherical, after one larger angle of head rotation, no matter using Which kind of method, the Attitude estimation result based on single camera all can be less smart because of the limitation of single camera Limited eyeshot Really, in order to improve the precision and confidence level of Attitude estimation, there is the method for the multiple view Attitude estimation based on polyphaser.

A kind of method of existing multiple view Attitude estimation is that selection has the minimum angles Yaw from the view that multiple cameras provide Degree that, with expansion Attitude Calculation.This method is dependent on one it is assumed that i.e. the view with minimum Yaw angles is more other View has the most reliable and highest Attitude estimation result of precision.It is apparent that being influenced by various factors, the hypothesis is in multiple view Be not in system must satisfaction.Another multiple view Attitude estimation uses from a camera and is switched to another camera Switchover policy, but similarly, this Attitude estimation method depends on the correctness of that selected view estimation.

It is the fusion operator summed by weighted average to melt that the prior art, which also has a kind of method of multiple view Attitude estimation, Close the individual Attitude estimation result that each view provides.In fact, different view estimated results all has different confidences Degree and precision, regardless of preset weighted sum is arranged, the confidence level of final carriage result of calculation can not obtain effectively It improves.The prior art, which also has a kind of method of multiple view Attitude estimation, to be merged under different views by the way of average probability Attitude estimation as a result, but this mode be suitable only for the rough estimate to posture.

In conclusion the precision especially confidence level for the Attitude estimation result that existing multiple view Attitude estimation obtains all does not have Obtain practical raising.

Invention content

The technical problem to be solved in the present invention is to provide the method, apparatus and electronic equipment of a kind of estimation of object gesture, bases Merge object gesture estimated result different under multiple view in the method for optimization, can improve Attitude estimation result precision and Confidence level.

In order to solve the above technical problems, the embodiment of the present invention offer technical solution is as follows：

On the one hand, a kind of method of object gesture estimation is provided, is estimated the progress of posture object to same using multiple cameras Attitude estimation, the multiple camera can shoot the view for being estimated posture object, and the multiple phase with different view Posture subject area of being estimated in machine two-by-two taken by associated camera has the part of overlapping, the method includes：

It determines the multiple camera and to be estimated in posture object composition system camera posture between camera two-by-two poor, it is described Camera posture difference by different cameral visual angle to same fixed difference between being estimated the posture result that posture object obtains, It is equal under world coordinate system the rotation angle along three orthogonal directions to another camera, two two-phase from a camera The posture subject area of being estimated that machine respectively takes has the part of overlapping；

The view shot using the multiple camera carries out Attitude estimation respectively, obtains multiple initial attitude estimated results；

Can merge at least two are selected from the multiple initial attitude estimated result according to the camera posture difference Initial attitude estimated result, the posture difference between described two initial attitude estimated results and the camera posture difference of corresponding camera Between difference distance be less than preset first threshold value；

Optimization object function and its constraints are established using at least two initial attitudes estimated result, is solved Obtain the amendment increment of each initial attitude estimated result, wherein the optimization object function is to minimize described at least two Initial attitude estimated result correct after in corresponding view all characteristic points average back projection error, the constraints is The average back projection error of characteristic point is small in corresponding each view after at least two initial attitudes estimated result is corrected It is after each initial attitude estimated result amendment and another in default second threshold, and in at least two initial attitudes estimated result The camera posture that the one revised difference of initial attitude estimated result is equal to corresponding camera two-by-two is poor, and revised initial attitude is estimated It is that initial attitude estimated result adds amendment increment to count result；

Object's pose estimation is calculated as a result, the targeted attitude result is at least two initial attitude estimation knot Any initial attitude estimated result corrects the sum of increment with it in fruit, and the targeted attitude result is transformed into and is estimated posture pair As under the world coordinate system of place.

Preferably, any two cameras are first camera and second camera in the multiple camera, and the determination is the multiple Camera and by the camera posture difference estimated in posture object composition system between camera two-by-two include：

By it is described estimated posture object centered on establish world coordinate system, to the first camera and the second camera point Not carry out location position obtain respective external parameter, obtain the corresponding spin matrix R of first camera_LRotation corresponding with second camera Torque battle array R_R, the spin matrix is rotation transformation of the camera coordinates system to world coordinate system；

Utilize spin matrix R_LWith spin matrix R_RThe spin matrix R from first camera to second camera is calculated_LoR, Middle R_LoR=R_L ^-*R_R；

By the calculating of Eulerian angles by R_LoRIt is decomposed into pitch angle, the rotation angle in three directions of roll angle and yaw angle obtains Camera posture between first camera and second camera is poor.

Preferably, the method that the view using the shooting of the multiple camera carries out Attitude estimation respectively is based on 3D moulds Type method corresponding with 2D characteristic points to the view that camera is shot estimated the Attitude estimation of object；

It is A and B in at least two initial attitudes estimated result, A is to carry out posture to the view that first camera is shot It is obtained after estimation, B is to be obtained after carrying out Attitude estimation to the view that second camera is shot, and A and the respective amendment increments of B are △ A It is described to establish optimization object function and its constraints using at least two initial attitudes estimated result when with △ B, it carries out It solves and obtains the increment of each initial attitude estimated result and include：

Establish optimization object function

Constraints is：

A+ Δ A+CONST ≈ B+ Δ B constraintss 1；

F_L(A+ Δs A, i)=| | f_L(A+ΔA,MP_i)-Pi (L) | |,

F_R(B+ Δs B, i)=| | f_R(B+ΔB,MP_i)-Pi(R)||；

Wherein, camera postures of the CONST between first camera and second camera is poor, MP_iIt is 3D models in object coordinates I-th point under system, Pi (L) corresponds to 3D model points MP_iFirst camera shooting view in correspondence 2D characteristic points, Pi (R) 3D model points MP is corresponded to_iSecond camera shooting view in correspondence 2D characteristic points, f_L() is the 3D of first camera Model points are to the projective transformation formula of 2D characteristic points, f_R() be second camera 3D model points to 2D characteristic points projective transformation Formula, V are preset second threshold；

If there is n fusion object, then the sum term that adds up of target formula 1 just has n phase plus item to each i, constrains simultaneously Condition formula 2 also has corresponding n constraints formula, if fusion object m corresponding camera postures of association are poor, about Beam condition 1 also has m corresponding constraint formulations；

The optimization object function is solved using enumerative technique or iterative method, obtains △ A and △ B.

Preferably, it is described using iterative method to the optimization object function carry out solve include：

It calculates and corrects total amount △ D=B-A-CONST；

Enable n=0, S=△ D, A₀=A,

Into iterative process：

while(S>=preset third threshold value)

n++；

Scheme { the A candidate from three_n-1,A_n-1+S,A_n-1- S } in selected most based on optimization object function and constraints Good scheme, it is A to enable it_n, wherein each scheme Ci meets condition C i in 3 candidate schemes>=min { A, A+ △ D } and Ci< max{A,A+△D}；

S=S/2；

}

S is the step-length of each round iteration；

If S<Default third threshold value, stops iteration；

At this point, △ A=An-A are obtained, △ B=△ A- △ D.

The embodiment of the present invention additionally provides a kind of device of object gesture estimation, is estimated posture to same using multiple cameras Object carries out Attitude estimation, and the multiple camera can shoot the view for being estimated posture object, and institute with different view Stating in multiple cameras the posture subject area of being estimated taken by associated camera two-by-two has the part of overlapping, described device packet It includes：

Camera posture difference determining module, for determining the multiple camera and being estimated two two-phases in posture object composition system Camera posture between machine is poor, and the camera posture difference is estimated what posture object obtained by different cameral visual angle to same Fixed difference between posture result is equal under world coordinate system from a camera along three orthogonal directions to another phase The rotation angle of machine, the posture subject area of being estimated that the camera two-by-two respectively takes have the part of overlapping；

Initial estimation module, the view for being shot using the multiple camera are carried out Attitude estimation, obtained multiple respectively Initial attitude estimated result；

Optimization Solution module, for selecting energy from the multiple initial attitude estimated result according to the camera posture difference At least two initial attitude estimated results enough merged, the posture difference between described two initial attitude estimated results and corresponding phase Difference distance between the camera posture difference of machine is less than preset first threshold value；Utilize at least two initial attitudes estimated result Optimization object function and its constraints are established, is solved to obtain the amendment increment of each initial attitude estimated result, wherein All spies in corresponding view after the optimization object function is corrected for the minimum at least two initial attitudes estimated result The average back projection error of point is levied, the constraints is corresponding after at least two initial attitudes estimated result is corrected The average back projection error of characteristic point is less than default second threshold in each view, and at least two initial attitude is estimated As a result it is equal to corresponding two with another revised difference of initial attitude estimated result after each initial attitude estimated result is corrected in The camera posture of two cameras is poor, and revised initial attitude estimated result is initial attitude estimated result plus amendment increment；

Computing module, for object's pose estimation to be calculated as a result, the targeted attitude result is described at least two Any initial attitude estimated result corrects the sum of increment with it in initial attitude estimated result；

Conversion module, for after object's pose estimation result is calculated, the targeted attitude result to be transformed into Under world coordinate system where being estimated posture object.

Preferably, the camera posture difference determining module includes：

Location position unit, for by it is described estimated posture object centered on establish world coordinate system, to first phase Machine and the second camera carry out location position and obtain respective external parameter respectively, obtain the corresponding spin matrix R of first camera_L Spin matrix R corresponding with second camera_R, the spin matrix is rotation transformation of the camera coordinates system to world coordinate system；

Computing unit, for utilizing spin matrix R_LWith spin matrix R_RIt is calculated from first camera to second camera Spin matrix R_LoR, wherein R_LoR=R_L ^-*R_R；

Resolving cell, for by the calculating of Eulerian angles by R_LoRIt is decomposed into pitch angle, three directions of roll angle and yaw angle Rotation angle, the camera posture obtained between first camera and second camera is poor.

It is A and B in at least two initial attitudes estimated result, A is to carry out posture to the view that first camera is shot It is obtained after estimation, B is to be obtained after carrying out Attitude estimation to the view that second camera is shot, and A and the respective amendment increments of B are △ A When with △ B, the Optimization Solution module includes：

Selecting unit, for selecting to melt from the multiple initial attitude estimated result according to the camera posture difference At least two initial attitude the estimated result A and B closed；

Optimization object function establishes unit, for establishing optimization object function

Constraints is：

A+ Δ A+CONST ≈ B+ Δ B constraintss 1；

F_L(A+ Δs A, i)=| | f_L(A+ΔA,MP_i)-Pi (L) | |,

F_R(B+ Δs B, i)=| | f_R(B+ΔB,MP_i)-Pi(R)||；

Wherein, camera postures of the CONST between first camera and second camera is poor, MP_iIt is 3D models in object coordinates I-th point under system, Pi (L) corresponds to 3D model points MP_iFirst camera shooting view in correspondence 2D characteristic points, Pi (R) 3D model points MP is corresponded to_iSecond camera shooting view in correspondence 2D characteristic points, f_L() is the 3D of first camera Model points are to the projective transformation formula of 2D characteristic points, f_R() be second camera 3D model points to 2D characteristic points projective transformation Formula, V are preset second threshold；If there is n fusion object, then the sum term that adds up of target formula 1 just has n to each i Phase plus item, while constraints formula 2 also has corresponding n constraints formula, if fusion object association m is corresponding Camera posture is poor, then constraints 1 also has m corresponding constraint formulations；

Solve unit, for being solved to the optimization object function using enumerative technique or iterative method, obtain △ A and △B。

Preferably, the solution unit is specifically used for：

It calculates and corrects total amount △ D=B-A-CONST；

Enable n=0, S=△ D, A₀=A,

Into iterative process：

while(S>=preset third threshold value)

n++；

S=S/2；

}

S is the step-length of each round iteration；

If S<Default third threshold value, stops iteration；

At this point, △ A=An-A are obtained, △ B=△ A- △ D.

The embodiment of the present invention additionally provides a kind of electronic equipment of object gesture estimation, is estimated to same using multiple cameras Posture object carries out Attitude estimation, and the multiple camera can shoot the view for being estimated posture object with different view, and And the posture subject area of being estimated in the multiple camera two-by-two taken by associated camera has the part of overlapping, the electricity Sub- equipment includes：

Processor；With

Memory is stored with computer program instructions in the memory,

Wherein, when the computer program instructions are run by the processor so that the processor executes following step Suddenly：

Optimization object function and its constraints are established using at least two initial attitudes estimated result, is solved Obtain the increment of each initial attitude estimated result, wherein the optimization object function is to minimize described at least two initially After Attitude estimation modified result in corresponding view all characteristic points average back projection error, the constraints is described The average back projection error of characteristic point is less than in advance in corresponding each view after at least two initial attitude estimated results are corrected If second threshold, and after each initial attitude estimated result is corrected in at least two initial attitudes estimated result with it is another just The camera posture that difference after beginning Attitude estimation modified result is equal to corresponding camera two-by-two is poor, revised initial attitude estimation knot Fruit is initial attitude estimated result plus amendment increment；

The embodiment of the present invention has the advantages that：

In said program, the view for being estimated posture object is shot with different view using multiple cameras, two in multiple cameras Posture subject area of being estimated taken by two associated cameras has the part of overlapping, so that multiple camera shootings regard There is relevance between figure.Since the Attitude estimation result of each view has that confidence level and precision are to be improved, Therefore, the present invention improves the confidence level and precision of Attitude estimation result using the relevance between multiple views.First, according to Camera posture difference between camera screens the view Attitude estimation result of corresponding multiple camera shootings, therefrom selects energy The view result enough merged establishes optimization according to the initial attitude estimated result that can merge view and corresponding camera posture difference Scheme, balance the error of the initial attitude estimated result of each view, multiple view mean projection errors and with camera posture Error between difference is corrected to participating in the initial attitude estimated result of each view of fusion, well thus will be more A associated not completely independent event is modified to concurrent event, and which raises the essences of Attitude estimation result known to probability theory theory Degree and confidence level.

Description of the drawings

Fig. 1 is the flow diagram of the method for an object Attitude estimation of the embodiment of the present invention；

Fig. 2 is that the embodiment of the present invention one determines in multiple cameras that the flow of the camera posture difference between camera two-by-two is illustrated Figure；

Fig. 3 is that the embodiment of the present invention one establishes optimization object function and its about using at least two initial attitude estimated results Beam condition is solved to obtain the flow diagram of the increment of each initial attitude estimated result；

Fig. 4 is the structure diagram of the device of two object gesture of embodiment of the present invention estimation；

Fig. 5 is the structure diagram of two camera posture difference determining module of the embodiment of the present invention；

Fig. 6 is the structure diagram of two Optimization Solution module of the embodiment of the present invention；

Fig. 7 is the structure diagram of the electronic equipment of three object gesture of embodiment of the present invention estimation；

Fig. 8 is the flow diagram of the method for four object gesture of embodiment of the present invention estimation；

Fig. 9 is Pitch (pitch angle), the schematic diagram of Roll (roll angle) and Yaw (yaw angle)；

Figure 10 is the embodiment of the present invention four using being symmetrically distributed in two cameras of user's head forward direction both sides to account The schematic diagram that the view in portion is shot；

The characteristic point distribution schematic diagram that Figure 11 behaves on the face；

Figure 12 is one and waits perspective view of the 3D spheres in 2D planes for rotating spacing mesh.

Specific implementation mode

To keep the embodiment of the present invention technical problems to be solved, technical solution and advantage clearer, below in conjunction with Drawings and the specific embodiments are described in detail.

The embodiment of the present invention for all relatively low problem of the precision of Attitude estimation result in the prior art and confidence level, A kind of method, apparatus and electronic equipment of object gesture estimation are provided, merged based on the method for optimization different under multiple view Object gesture estimated result can improve the precision and confidence level of Attitude estimation result.

Embodiment one

The method for present embodiments providing a kind of estimation of object gesture, using multiple cameras to it is same estimated posture object into Row Attitude estimation, the multiple camera can shoot the view for being estimated posture object with different view, and the multiple Posture subject area of being estimated in camera two-by-two taken by associated camera has the part of overlapping, as shown in Figure 1, the side Method includes：

Step 101：It determines the multiple camera and is estimated in posture object composition system camera appearance between camera two-by-two State is poor, the camera posture difference by different cameral visual angle to it is same estimated it is solid between the posture result that posture object obtains Fixed difference is equal under world coordinate system the rotation angle along three orthogonal directions to another camera from a camera, The posture subject area of being estimated that the camera two-by-two takes has the part of overlapping；

Step 102：The view shot using the multiple camera carries out Attitude estimation respectively, obtains multiple initial attitudes and estimates Count result；

Step 103：It is selected from the multiple initial attitude estimated result can to merge according to the camera posture difference At least two initial attitude estimated results, the camera of posture difference and corresponding camera between described two initial attitude estimated results Difference distance between posture difference is less than preset first threshold value；It is established and is optimized using at least two initial attitudes estimated result Object function and its constraints are solved to obtain the amendment increment of each initial attitude estimated result, wherein the optimization Object function is minimize after at least two initial attitudes estimated result is corrected all characteristic points in corresponding view flat Equal back projection error, the constraints are corresponding each view after at least two initial attitudes estimated result is corrected The back projection error of middle characteristic point is less than default second threshold, and each first in at least two initial attitudes estimated result It is equal to the camera of corresponding camera two-by-two after beginning Attitude estimation modified result with another revised difference of initial attitude estimated result Posture is poor, refers to that initial attitude estimated result adds amendment increment after the amendment of initial attitude estimated result herein；

Step 104：Object's pose estimation is calculated as a result, the targeted attitude result is described at least two initial appearances Any initial attitude estimated result corrects the sum of increment with it in state estimated result, and by the targeted attitude result be transformed by Estimate under posture world of object coordinate system.

In the present embodiment, the view for being estimated posture object is shot with different view using multiple cameras, two in multiple cameras Posture subject area of being estimated taken by two associated cameras has the part of overlapping, so that multiple camera shootings regard There is relevance between figure.Since the Attitude estimation result of each view has that confidence level and precision are to be improved, Therefore, the present invention improves the confidence level and precision of Attitude estimation result using the relevance between multiple views.First, according to Camera posture difference between camera screens the view Attitude estimation result of corresponding multiple camera shootings, therefrom selects energy The view result enough merged establishes optimization according to the initial attitude estimated result that can merge view and corresponding camera posture difference Scheme, balance the error of the initial attitude estimated result of each view, multiple view mean projection errors and with camera posture Error between difference is corrected to participating in the initial attitude estimated result of each view of fusion, well thus will be more A associated not completely independent event is modified to concurrent event, and which raises the essences of Attitude estimation result known to probability theory theory Degree and confidence level.

Due to object's pose estimation under different views the result is that being estimated camera coordinates of the posture object corresponding to the view Therefore posture under system is calculated object's pose estimation result and also needs to being converted to object's pose estimation result later Output after being estimated under posture world of object coordinate system.

Further, any two cameras are first camera and second camera in the multiple camera, as shown in Fig. 2, described Determine that the camera posture difference in the multiple camera between any two cameras includes：

Step 201：By it is described estimated posture object centered on establish world coordinate system, to the first camera and described Two cameras carry out location position and obtain external parameter respectively, obtain the corresponding spin matrix R of first camera_LIt is corresponding with second camera Spin matrix R_R, the spin matrix is rotation transformation of the camera coordinates system to world coordinate system；

Step 202：Utilize spin matrix R_LWith spin matrix R_RThe spin moment from first camera to second camera is calculated Battle array R_LoR, wherein R_LoR=R_L-*R_R；

Step 203：By the calculating of Eulerian angles by R_LoRIt is decomposed into pitch angle, the rotation in three directions of roll angle and yaw angle Corner, the camera posture obtained between first camera and second camera are poor.

Further, the method that the view using the shooting of the multiple camera carries out Attitude estimation respectively is based on 3D Model method corresponding with 2D characteristic points to the view that camera is shot estimated the Attitude estimation of object；

It is A and B in at least two initial attitudes estimated result, A is to carry out posture to the view that first camera is shot It is obtained after estimation, B is to be obtained after carrying out Attitude estimation to the view that second camera is shot, and A and the respective amendment increments of B are △ A When with △ B, as shown in figure 3, the step 103 includes：

Step 301：It is selected from the multiple initial attitude estimated result can to merge according to the camera posture difference At least two initial attitude estimated results；

Step 302：Optimization object function and its constraints are established using at least two initial attitudes estimated result；

Optimization object function is：

Constraints is：

A+ Δ A+CONST ≈ B+ Δ B constraintss 1；

F_L(A+ Δs A, i)=| | f_L(A+ΔA,MP_i)-Pi (L) | |,

F_R(B+ Δs B, i)=| | f_R(B+ΔB,MP_i)-Pi(R)||；

Wherein, camera postures of the CONST between first camera and second camera is poor, MP_iIt is 3D models in object coordinates I-th point under system, Pi (L) corresponds to 3D model points MP_iFirst camera shooting view in correspondence 2D characteristic points, Pi (R) 3D model points MP is corresponded to_iSecond camera shooting view in correspondence 2D characteristic points f_L() is the 3D of first camera Model points are to the projective transformation formula of 2D characteristic points, f_R() be second camera 3D model points to 2D characteristic points projective transformation Formula, V are preset second threshold；

Above-mentioned formula is with there are two fusion objects to illustrate, further, when there are n fusion object, then The sum term that adds up of target formula 1 just has a n phase plus item to each i, while constraints formula 2 also has corresponding n and constrains Condition formula, if while merging that object association m corresponding cameras are poor, and constraints 1 also has m corresponding constraint formulations.

Step 303：The optimization object function is solved using enumerative technique or iterative method, obtains △ A and △ B.

Further, it is described using iterative method to the optimization object function carry out solve include：

It calculates and corrects total amount △ D=B-A-CONST；

Enable n=0, S=△ D, A₀=A,

Into iterative process：

while(S>=preset third threshold value)

n++；

S=S/2；

}

S is the step-length of each round iteration；

If S<Default third threshold value, stops iteration；

At this point, △ A=An-A are obtained, △ B=△ A- △ D.

Embodiment two

The device for present embodiments providing a kind of estimation of object gesture, using multiple cameras to it is same estimated posture object into Row Attitude estimation, the multiple camera can shoot the view for being estimated posture object with different view, and the multiple Posture subject area of being estimated in camera two-by-two taken by associated camera has the part of overlapping, as shown in figure 4, the dress Set including：

Camera posture difference determining module 41, for determining the multiple camera and being estimated in posture object composition system two-by-two Camera posture between camera is poor, and the camera posture difference is estimated posture object and obtained by different cameral visual angle to same Posture result between fixed difference, be equal under world coordinate system from a camera along three orthogonal directions to another The rotation angle of camera, the posture subject area of being estimated that the camera two-by-two respectively takes have the part of overlapping；

Initial estimation module 42, the view for being shot using the multiple camera are carried out Attitude estimation, obtained more respectively A initial attitude estimated result；

Optimization Solution module 43, for being selected from the multiple initial attitude estimated result according to the camera posture difference At least two initial attitude estimated results that can be merged, posture difference between described two initial attitude estimated results with it is corresponding Difference distance between the camera posture difference of camera is less than preset first threshold value；Estimate to tie using at least two initial attitude Fruit establishes optimization object function and its constraints, is solved to obtain the amendment increment of each initial attitude estimated result, In, the optimization object function is to minimize after at least two initial attitudes estimated result is corrected to own in corresponding view The average back projection error of characteristic point, the constraints are corresponding after at least two initial attitudes estimated result is corrected Each view in the average back projection error of characteristic point be less than default second threshold, and at least two initial attitude is estimated With another revised difference of initial attitude estimated result equal to corresponding after each initial attitude estimated result is corrected in meter result The camera posture of camera is poor two-by-two, and revised initial attitude estimated result is initial attitude estimated result plus amendment increment；

Computing module 44, for object's pose estimation to be calculated as a result, the targeted attitude result is described at least two Any initial attitude estimated result corrects the sum of increment with it in a initial attitude estimated result；

Conversion module 45, for after object's pose estimation result is calculated, the targeted attitude result to be converted Under the world coordinate system to where being estimated posture object.

Further, as shown in figure 5, the camera posture difference determining module 41 includes：

Location position unit 411, for by it is described estimated posture object centered on establish world coordinate system, to described first Camera and the second camera carry out location position and obtain respective external parameter respectively, obtain the corresponding spin matrix of first camera R_LSpin matrix R corresponding with second camera_R, the spin matrix is rotation transformation of the camera coordinates system to world coordinate system；

Computing unit 412, for utilizing spin matrix R_LWith spin matrix R_RIt is calculated from first camera to second camera Spin matrix R_LoR, wherein R_LoR=R_L ^-*R_R；

Resolving cell 413, for by the calculating of Eulerian angles by R_LoRIt is decomposed into pitch angle, roll angle and yaw angle three The rotation angle in direction, the camera posture obtained between first camera and second camera are poor.

It is A and B in at least two initial attitudes estimated result, A is to carry out posture to the view that first camera is shot It is obtained after estimation, B is to be obtained after carrying out Attitude estimation to the view that second camera is shot, and A and the respective amendment increments of B are △ A When with △ B, as shown in fig. 6, the Optimization Solution module 43 includes：

Selecting unit 431, for selecting energy from the multiple initial attitude estimated result according to the camera posture difference At least two initial attitude the estimated result A and B enough merged；

Optimization object function establishes unit 432, for establishing optimization object function

Constraints is：

A+ Δ A+CONST ≈ B+ Δ B constraintss 1；

F_L(A+ Δs A, i)=| | f_L(A+ΔA,MP_i)-Pi (L) | |,

F_R(B+ Δs B, i)=| | f_R(B+ΔB,MP_i)-Pi(R)||；

It solves unit 433 and obtains △ for being solved to the optimization object function using enumerative technique or iterative method A and △ B.

Further, the solution unit 433 is specifically used for：

It calculates and corrects total amount △ D=B-A-CONST；

Enable n=0, S=△ D, A₀=A,

Into iterative process：

while(S>=preset third threshold value)

n++；

S=S/2；

}

S is the step-length of each round iteration；

If S<Default third threshold value, stops iteration；

At this point, △ A=An-A are obtained, △ B=△ A- △ D.

Embodiment three

The present embodiment additionally provides a kind of electronic equipment of object gesture estimation, is estimated posture to same using multiple cameras Object carries out Attitude estimation, and the multiple camera can shoot the view for being estimated posture object, and institute with different view Stating in multiple cameras the posture subject area of being estimated taken by associated camera two-by-two has the part of overlapping, as shown in fig. 7, The electronic equipment 60 includes：

Processor 62；With

Memory 64 is stored with computer program instructions in the memory 64,

Wherein, when the computer program instructions are run by the processor so that the processor 62 executes following Step：

Further, as shown in fig. 7, object gesture estimation electronic equipment further include network interface 61, input equipment 63, Hard disk 65 and display equipment 66.

It can be interconnected by bus architecture between above-mentioned each interface and equipment.Bus architecture can be may include arbitrary The bus and bridge of the interconnection of quantity.One or more central processing unit (CPU) specifically represented by processor 62, and by depositing The various of one or more memory that reservoir 64 represents are electrically connected to together.Bus architecture can also such as will be set periphery The various other of standby, voltage-stablizer and management circuit or the like are electrically connected to together.It is appreciated that bus architecture is for real Connection communication between these existing components.Bus architecture in addition to including data/address bus, further include power bus, controlling bus and Status signal bus in addition, these are all it is known in the art, therefore is no longer described in greater detail herein.

The network interface 61 can be connected to network (such as internet, LAN), dependency number is obtained from network According to, and can be stored in hard disk 65.

The input equipment 63, can receive the various instructions of operating personnel's input, and be sent to processor 62 for holding Row.The input equipment 63 may include keyboard or pointing device (for example, mouse, trace ball (trackball), touch-sensitive plate Or touch screen etc..

The display equipment 66, the result that processor 62 can be executed instruction to acquisition are shown.

The memory 64 is calculated for program and data and processor 62 necessary to storage program area operation The data such as intermediate result in the process.

It is appreciated that the memory 64 in the embodiment of the present invention can be volatile memory or nonvolatile memory, Both or may include volatile and non-volatile memory.Wherein, nonvolatile memory can be read-only memory (ROM), Programmable read only memory (PROM), Erasable Programmable Read Only Memory EPROM (EPROM), electrically erasable programmable read-only memory (EEPROM) or flash memory.Volatile memory can be random access memory (RAM), be used as External Cache.Herein The memory 64 of the device and method of description is intended to the memory of including but not limited to these and any other suitable type.

In some embodiments, memory 64 stores following element, executable modules or data structures, or Their subset or their superset：Operating system 641 and application program 642.

Wherein, operating system 641, including various system programs, such as ccf layer, core library layer, driving layer etc., for real Existing various basic businesses and the hardware based task of processing.Application program 642, including various application programs, such as browser (Browser) etc., for realizing various applied business.Realize that the program of present invention method may be embodied in application program In 642.

Above-mentioned processor 62, when calling and execute the application program and data that are stored in the memory 64, specifically, When can be the program stored in application program 642 or instruction, it may be determined that the multiple camera is formed with posture object is estimated Camera posture in system two-by-two between camera is poor, and the camera posture difference is that different cameral visual angle is estimated posture to same Fixed difference between the posture result that object is obtained is equal under world coordinate system from a camera along three orthogonal sides To the rotation angle to another camera, the posture subject area of being estimated that the camera two-by-two respectively takes has the portion of overlapping Point；The view shot using the multiple camera carries out Attitude estimation respectively, obtains multiple initial attitude estimated results；According to institute It states camera posture difference and selects at least two initial attitudes that can be merged estimation knot from the multiple initial attitude estimated result Fruit, the posture difference between described two initial attitude estimated results and the difference between the camera posture difference of corresponding camera are apart from small In preset first threshold value；Optimization object function and its constraints are established using at least two initial attitudes estimated result, Solved to obtain the increment of each initial attitude estimated result, wherein the optimization object function be minimize described at least After two initial attitude estimated results are corrected in corresponding view all characteristic points average back projection error, the constraint item Part is that the average back projection of characteristic point in corresponding each view after at least two initial attitudes estimated result is corrected misses After difference is less than default second threshold, and each initial attitude estimated result is corrected in at least two initial attitudes estimated result The camera posture for being equal to corresponding camera two-by-two with another revised difference of initial attitude estimated result is poor, revised initial appearance State estimated result is initial attitude estimated result plus amendment increment；Object's pose estimation is calculated as a result, the target appearance State result corrects the sum of increment for any initial attitude estimated result in at least two initial attitudes estimated result with it, and The targeted attitude result is transformed under world coordinate system where being estimated posture object.

The method that the above embodiment of the present invention discloses can be applied in processor 62, or be realized by processor 62.Place It may be a kind of IC chip to manage device 62, the processing capacity with signal.During realization, each step of the above method It can be completed by the integrated logic circuit of the hardware in processor 62 or the instruction of software form.Above-mentioned processor 62 can To be general processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), ready-made programmable gate array (FPGA) Either either transistor logic, discrete hardware components may be implemented or execute for other programmable logic device, discrete gate Disclosed each method, step and logic diagram in the embodiment of the present invention.General processor can be microprocessor or this at It can also be any conventional processor etc. to manage device.The step of method in conjunction with disclosed in the embodiment of the present invention, can directly embody Execute completion for hardware decoding processor, or in decoding processor hardware and software module combination execute completion.Software Module can be located at random access memory, flash memory, read-only memory, programmable read only memory or electrically erasable programmable storage In the storage medium of this fields such as device, register maturation.The storage medium is located at memory 64, and processor 62 reads memory 64 In information, in conjunction with its hardware complete the above method the step of.

It is understood that embodiments described herein can use hardware, software, firmware, middleware, microcode or its It combines to realize.For hardware realization, processing unit may be implemented in one or more application-specific integrated circuits (ASIC), number letter Number processor DSP), digital signal processing appts (DSPD), programmable logic device (PLD), field programmable gate array (FPGA), general processor, controller, microcontroller, microprocessor, other electronics lists for executing herein described function In member or combinations thereof.

For software implementations, it can be realized herein by executing the module (such as process, function etc.) of function described herein The technology.Software code is storable in memory and is executed by processor.Memory can in the processor or It is realized outside processor.

Specifically, processor 62 by it is described estimated posture object centered on establish world coordinate system, to the first camera It carries out location position respectively with the second camera and obtains respective external parameter, obtain the corresponding spin matrix R of first camera_LWith The corresponding spin matrix R of second camera_R, the spin matrix is rotation transformation of the camera coordinates system to world coordinate system；It utilizes Spin matrix R_LWith spin matrix R_RThe spin matrix R from first camera to second camera is calculated_LoR, wherein R_LoR=R_L ^-* R_R；By the calculating of Eulerian angles by R_LoRIt is decomposed into pitch angle, the rotation angle in three directions of roll angle and yaw angle obtains first Camera posture between camera and second camera is poor.

Specifically, processor 62 carries out posture based on 3D models method corresponding with 2D characteristic points to the view that camera is shot Estimation；

Establish optimization object function

Constraints is：

A+ Δ A+CONST ≈ B+ Δ B constraintss 1；

F_L(A+ Δs A, i)=| | f_L(A+ΔA,MP_i)-Pi (L) | |,

F_R(B+ Δs B, i)=| | f_R(B+ΔB,MP_i)-Pi(R)||；

Wherein, camera postures of the CONST between first camera and second camera is poor, MP_iIt is 3D models in object coordinates I-th point under system, Pi (L) corresponds to 3D model points MP_iFirst camera shooting view in correspondence 2D characteristic points, Pi (R) 3D model points MP is corresponded to_iSecond camera shooting view in correspondence 2D characteristic points, f_L() is the 3D of first camera Model points are to the projective transformation formula of 2D characteristic points, f_R() be second camera 3D model points to 2D characteristic points projective transformation Formula, V are preset second threshold；Formula is stated so that there are two fusion objects to illustrate, further, there are n When merging object, then the sum term that adds up of target formula 1 just has n phase plus item to each i, while constraints formula 2 also has Corresponding n constraints formula, if while merging that object association m corresponding cameras are poor, and constraints 1 also has m Corresponding constraint formulations.

Example IV

Below by estimated posture object be user's head for, to the present invention object gesture estimation method into advance one Step is introduced.As shown in figure 8, the method for the object gesture estimation of the present embodiment includes the following steps：

Step 801：Determine that the camera posture between two cameras is poor；

Normally, as shown in figure 9, the head pose of user include three degree of freedom, can use Pitch, Roll and Raw this three A orthogonal direction angle indicates that these three angles correspond to a 3x3 spin matrix.Object gesture estimation based on 3D models is just It is that the suitable spin matrix of object and translation transformation are found from the 2D views that camera is shot.

Since the head of people can approximatively regard cylinder or sphere as, when head rotation is to a larger inclination angle, If using the view in single camera account portion for shooting, because the limited visual field of one camera can lead to Attitude estimation result precision Decline.For the visual field for obtaining larger, changed especially for the continuous head pose of driver is tracked in cockpit, this implementation Example can dispose the direction that multiple cameras go to observation head from different angles, preferably capture the view of user's head.Multiphase A basic requirement is the facial parts that the user's head view obtained with different view needs to have overlapping two-by-two in machine deployment. Figure 10 shows schematically the schematic diagram that head pose estimation is carried out using 2 cameras.2 cameras are symmetrically distributed in Figure 10 In the positive both sides of user's head face, the common face for facing user constitutes the posture estimation system regarded one more.

It is regarding in environment, each camera works independently and its Attitude estimation result has certain randomness more.When all Camera all to the same object carry out Attitude estimation when, they are just interrelated.The position relationship that these cameras are stablized It will cause the head pose estimation result obtained under different visual angles that there is certain relevance.Relevance most directly indicates After camera and object's position determine, there is stable attitude angle between the object gesture observed in any two views Difference, this same target from different views to attitude angle difference in the present invention be known as camera posture it is poor.

Illustrate this poor parameter of camera posture by taking system shown in Fig. 10 as an example below.2 cameras are pair in Figure 10 What is claimed is distributed horizontally to both sides immediately ahead of user's head, is formed on 3 position vertical views that 2 cameras and user's head are constituted 90 degree of angle.When therefore immediately ahead of face face, user's head appearance this moment that the camera on the left side and the camera on the right provide Angle of the state on the directions Yaw is -45 ° and 45 ° respectively.If this moment face left direction deflect 10 °, at the time of new under, Angle of the user's head posture that the camera on the left side and the camera on the right provide on the directions Yaw will be -35 ° and 55 ° respectively, can To find out, user's head posture that the camera on the left side and the camera on the right provide remains between the angle on the directions Yaw 90 ° of differential seat angle, 90 ° of differential seat angle herein is exactly value of the camera posture difference parameter on the directions Yaw in the present invention.

Exist when describing same target posture it should be noted that a camera posture difference only represents associated 2 views Posture it is poor, that is, the rotation angle under three degree of freedom.Under more viewing systems that one is made of n camera, two two-phases If taken posture object to be estimated is there are overlapping region between machine, there are camera posture is poor, in so much viewing system Camera posture difference at most numbers should be between n* (n-1)/2, such as 2 cameras there are one camera posture is poor, between 3 cameras It is poor to share 3 camera postures.

After camera and object's position determine, it is poor camera posture can be calculated in advance by camera calibration technology.Specifically Ground says that user's head posture i.e. object gesture is equal under world of object coordinate system direction outside camera, that is, determines that camera is sat The rotation and translation of mark system to world coordinate system converts.The camera external parameter obtained in camera calibration be exactly camera relative to The rotation and translation matrix of world coordinate system.Therefore world coordinate system can be established centered on user's head, complete left camera and The location position of right camera obtains left camera spin matrix R_LWith right camera spin matrix R_R, it can be based on R later_LAnd R_RIt calculates From left camera transformation to the spin matrix R of right camera_LoR(R_LoR=R_L ^-*R_R), finally R_LoREuler angle (Eulers can be passed through Angle) calculating be decomposed into Pitch, the rotation angle in the directions Roll and Yaw, formed two camera attitude parameter CONST (Pitch, Roll,Yaw)。

Step 802：View based on the shooting of each camera independently carries out head pose estimation, obtains initial head pose and estimates Count result；

Under more viewing systems, each camera works independently, and is based respectively on captured view and carries out head pose estimation, obtains To initial head pose estimation result.The method based on model can be used to estimate in single view in each camera in the present embodiment Count out head pose.Attitude estimation method based on model is mainly using corresponding geometry between 3D model points and 2D picture points Correspondence estimates the posture of object.In a concrete implementation mode, ASM/AAM (active shape may be used Model) method of face alignment obtains a series of human face characteristic point, and relatively effective human face characteristic point is typically distributed on face Profile, near eyebrow eyes nose and mouth.Distribution of 77 human face characteristic points on face is given in Figure 11, it then can be from It is middle to select part or all of characteristic point, with Attitude estimation method POSIT (the pose from orthography based on model And scaling with iterations) calculate these characteristic points it is corresponding human face posture.POSIT can be from single width The posture of object is estimated in view, it needs 1 non-coplanar points to complete mapping of the 3D models to 2D views, herein 3D models can obtain head model scanning by 3D scanners.

The ability and facial modeling that the precision of POSIT Attitude estimations describes current face dependent on 3D models Precision, often influenced by human face posture angle.With the increase of end rotation angle, the essence of corresponding Attitude estimation result Degree and confidence level can gradually weaken.It in this sense, can the Attitude estimation result under single view is as one and right As the sample value for the random distribution that true posture is associated with, the true posture of object is different, and the distribution corresponded to is also different, distribution Difference also means that confidence level is also different.

Step 803：The initial head pose estimation that can be merged is judged whether as a result, if it does, turning to step 804, if it does not, turning to step 806, the view computation directly shot using existing camera is obtained final head pose and estimated Count result；

Enhance the precision and confidence level of Attitude estimation result using multiple view, core is to merge the independence of multiple views Attitude estimation result.But the positive promotion for contributing to precision can be provided by being not each view.By some enchancement factors Influence, the Attitude estimation result of partial view is perhaps wrong, it is therefore highly preferred that select suitably to merge object carry out it is more Depending on the fusion of Attitude estimation.

The standard of fusion Object Selection is the camera of posture difference and corresponding camera between two initial attitude estimated results Difference between posture difference is less than preset first threshold value.Camera posture difference represents the posture stablized between 2 views in the present invention Difference constraint.If it is poor that the difference of the Attitude estimation result of current 2 views deviates from the camera posture that this is obtained ahead of time significantly, Illustrate that necessarily there are one be wrong in current associated 2 Attitude estimation results.On the contrary, this 2 view being associated with Attitude estimation result is optional to do fusion object.

Since posture is the three-dimensional vector of a three degree of freedom, the posture of pair 2 views when calculating posture difference Estimated result can directly subtract each other on each component, and when calculating is compared with camera posture difference parameter, then it uses European Distance calculates.Citing, the Attitude estimation result obtained by view 1 are A, are B by the Attitude estimation result that view 2 obtains, this 2 Difference Pd=B-the A of current pose between view, the camera posture difference being corresponding to it is Pc, then working as Euclidean distance | Pd-PC |< When threshold value T (i.e. preset first threshold value), Attitude estimation result is A and B optional as fusion object.

Step 804：Using the initial head pose estimation result structure fusion prioritization scheme that can be merged, prioritization scheme is merged Including optimization object function and its constraints；

In more viewing systems, if there is multiple fusion objects, can the fusion based on prioritization scheme come lifting system appearance The precision and confidence level of state estimation.

The core concept of optimization object function is to find a kind of self adjustment of optimization for each fusion object to make fusion Object approaches that the camera posture that it is associated with is poor, while should meet the constraint of camera posture difference, do not allow also to destroy currently from The basis of posture is estimated in view.This is because the Attitude estimation result obtained by each view can be seen as from specific distribution In a sample value, if fine tuning sample value makes 2 independent events to become synchronous after meeting camera posture difference Event, and do not change each sample from distribution (abide by view estimate posture principle constraint), then the confidence of system Degree will be promoted therewith.

In the Attitude estimation method based on model, object gesture is estimated in 2D views, and usually projected by 3D Change brings verification Attitude estimation result correctness, and projective transformation herein may be defined as follows：

A series of to surround Pitch, the rotation in the directions Roll and Yaw is represented by spin matrix R, the 3D object of a 3x 3 Posture can be retouched strictly with it relative to world coordinates present position T (3 dimension translation vector) by the spin matrix R of direction jointly State composition, therefore posture P=[R | T] is 4 matrix of 3x, give on 3D object models under object coordinates system a bit (X, Y, Z), its position of corresponding subpoint (x, y) in 2D views can be defined as follows：

(x,y)^T=(x0+fx*Xc/Zc, y0+fy*Yc/Zc)^T (1)

(Xc, Yc, Zc) herein^T=[R | T] (X, Y, Z)^T

Wherein, (x0, y0) is the coordinate of 2D view central points, and (fx, fy) is focal length of the camera on the direction " x " and " y " Parameter.

It can reflect in spin matrix after being adjusted to the rotation angle of posture, and then pass through 3D projective transformations and view In characteristic point associate.By 3D model points are projected into 2D points that 2D views are obtained and correspond in 2D views The distance between characteristic point detected is referred to as 3D projection errors.The present embodiment verifies Attitude estimation by 3D projection errors As a result correctness assigns projection error as the constraints of Attitude estimation.Figure 11 gives 5 3D model points in the side Yaw Obtained subpoint after to 5 ° of adjustment of progress (the larger circle of subpoint white marks).

It should be between camera posture difference and the constraint of view Attitude estimation to the amendment of the Attitude estimation result of each view It carries out, while optimization aim is to minimize multiple view mean projection error, while it is pre- to keep the projection error of each view to be less than If second threshold.By taking two cameras for being symmetrically distributed in user's head forward direction both sides in more viewing systems as an example, prioritization scheme is available Following mathematical formulae is described：

Constraints is：

A+ΔA+CONST≈B+ΔB

Wherein, F_L(.)/F_R() is the projection error calculating function for left/right camera；

F_L(A+ Δs A, i)=| | f_L(A+ΔA,MP_i)-Pi(L)||；

F_R(B+ Δs B, i)=| | f_R(B+ΔB,MP_i)-Pi(R)||；

Wherein, camera postures of the CONST between first camera and second camera is poor, and MPi is 3D models in object coordinates I-th point under system, Pi (L) corresponds to 3D model points MP_iFirst camera shooting view in characteristic point, Pi (R) is Corresponding to 3D model points MP_iSecond camera shooting view in characteristic point, f_L() is the 3D model points of first camera to 2D The projective transformation formula of characteristic point, f_R() be second camera 3D model points arrive 2D characteristic points projective transformation formula, definition join Examine formula (1)；V is preset second threshold.

After carrying out independent Attitude estimation by single view, obtained from the view that the right camera of left camera is shot respectively Attitude estimation result A (Raw, Pitch, Roll) and B (Raw, Pitch, Roll) of the object under respective camera coordinates this moment, this Carve the camera posture difference CONST (Raw, Pitch, Roll) between this 2 cameras be after camera and object's position determine just It calculates in advance.Optimization problem is to calculate respective amendment increment △ A and the △ B of Attitude estimation result A and B, meets phase The constraint requirements A+ Δ A+CONST=B+ Δ B of machine posture difference, the target of optimization is to all in left camera and right camera view Characteristic point minimizes mean projection errorKeep each view simultaneously Projection error is less than default second threshold V.

Above-mentioned optimization object function is to find a kind of correction amount of optimization to balance various errors, includes the appearance of each view State estimates structural failure, mean projection error and the error between camera posture difference.It anticipated that the correction amount of each view It is different, Figure 12 is one and waits the 3D spheres of rotation spacing mesh in the perspective view of 2D planes, each crosspoint in figure It can all regard 3D subpoints, self the adjustment plan for illustrating each fusion view posture under optimization aim of the diagrammaticalIy as Slightly.For the 3D spheres of equity rotation spacing mesh, even if they have rotated same angle, but their postrotational projections miss Difference but with the different posture of sphere and difference (is located at the projection error maximum at front center, then to the left and right with upper and lower two Side is gradually reduced, and reaches minimum to edge), this and head model are much like.It can further be obtained in conjunction with Figure 12 and optimization aim Know, merged in object at this 2, preferentially to giving bigger correction amount (corresponding 3D balls side compared with the object of large deflection angle At edge), and to having compared with the object of small deflection angle with smaller correction amount (center immediately ahead of corresponding 3D balls), to reach The constant lower minimum ensemble average projection error of total correction amount.

Step 805：Optimization object function is solved；

Simplest method for solving is that all to meet constraints possible to being enumerated after total correction amount application discretization △ A and △ B combinations enumerate all satisfactions about since optimization processing relates only to matrix operation rather than view processing Beam condition possible △ A and △ B combinations do not need to expend more calculating.

Other than being solved using enumeration methodology, the method that the search correction amount of iteration can also be utilized, with 2 cameras Respectively for adjustment amount calculating, the method for this iterative search is represented by following process in the directions Yaw.Due to the throwing in the directions Yaw Shadow value affects only X-coordinate value, therefore the difference that can simplify the definition of projection error as x direction characters point and subpoint is used as measurement To assess optimization aim

It calculates first and corrects total amount △ D=B-A-CONST；

Enable n=0, S=△ D, A₀=A,

Into iterative process：

while(S>=preset third threshold value)

n++；

S=S/2；

}

S is the step-length of each round iteration；

If S<Default third threshold value, stops iteration；

At this point, △ A=An-A are obtained, △ B=△ A- △ D.

In above process, A and B is the object appearance estimated respectively using the view captured by left camera and right camera State, each candidate scheme represents in iterative process is modified rear new Attitude estimation as a result, it is right to Attitude estimation result A New Attitude estimation result can be calculated from the camera posture difference being associated after the amendment of the Attitude estimation result B answered.Therefore The mean projection error that two cameras correspond to this moment can be calculated for each candidate scheme, corresponded in these three candidate schemes That of minimum average B configuration projection error can be selected as in current iteration than preferably scheme, and next round iterative processing is just with current It is carried out based on the preferred embodiment chosen.Since iterative process is to preset third threshold value until the step-length of iteration is less than, therefore be somebody's turn to do Third threshold value also represents the precision of Attitude estimation.Similar side can also be used in the correction amount calculating of other both direction rotation angles Case carries out.

Step 806：Final head pose estimation result is calculated.

After obtaining △ A and △ B, you can final head pose estimation result is calculated.Final head pose Estimated result is the sum of A and △ A or the sum of B and △ B.

New Attitude estimation result meets the constraint requirements of camera posture difference after being corrected by △ A and △ B, therefore they The same posture immediately ahead of relative to object can independently be exported as a result, 2 independent events become concurrent event as a result, According to probability theory theory, the confidence level and precision of system are also improved therewith.

Since the coordinate system of the calculated head pose estimation result in image is camera coordinates system, therefore, it is also desirable to It is exported after the coordinate system of final Attitude estimation result is converted to world coordinate system from camera coordinates system, this is well known skill Art.Usual world coordinate system is centered on object (user's head), towards front direction.

In the present embodiment, the view of user's head is shot with different view using two cameras, two camera shootings User's head region has the part of overlapping, so that has relevance between the view of two cameras shooting.Due to each The Attitude estimation result of view all has that confidence level and precision are to be improved, therefore, the present invention using two views it Between relevance improve the confidence level and precision of Attitude estimation result.First, according to the camera posture difference pair two between camera The view Attitude estimation result of a camera shooting is screened, the view Attitude estimation that therefrom selection can merge as a result, according to The initial attitude estimated result and corresponding camera posture difference that view can be merged establish prioritization scheme, balance the first of each view The error of beginning Attitude estimation result, all fusion view mean projection errors and the error between camera posture difference, to every The initial attitude estimated result of a view is all corrected well, to be modified to multiple associated not completely independent events Concurrent event, which raises the precision of head pose estimation and confidence levels known to probability theory theory.

The above is the preferred embodiment of the present invention, it is noted that for those skilled in the art For, without departing from the principles of the present invention, it can also make several improvements and retouch, these improvements and modifications It should be regarded as protection scope of the present invention.

Claims

1. a kind of method of object gesture estimation, which is characterized in that estimated posture object progress appearance to same using multiple cameras State estimates that the multiple camera can shoot the view for being estimated posture object, and the multiple camera with different view In posture subject area of being estimated two-by-two taken by associated camera there is the part of overlapping, the method includes：

Determine the multiple camera and poor, the camera of being estimated in posture object composition system camera posture between camera two-by-two Posture difference, to same fixed difference between being estimated the posture result that posture object obtains, is equal by different cameral visual angle From a camera along the rotation angle of three orthogonal directions to another camera under world coordinate system, the camera two-by-two is each Estimated part of the posture subject area with overlapping from taking；

It is selected from the multiple initial attitude estimated result according to the camera posture difference can merge at least two initial Attitude estimation is as a result, the posture between described two initial attitude estimated results is poor between the camera posture difference of corresponding camera Difference distance is less than preset first threshold value；

Optimization object function and its constraints are established using at least two initial attitudes estimated result, is solved to obtain The amendment increment of each initial attitude estimated result, wherein the optimization object function is to minimize described at least two initially After Attitude estimation modified result in corresponding view all characteristic points average back projection error, the constraints is described The average back projection error of characteristic point is less than in advance in corresponding each view after at least two initial attitude estimated results are corrected If second threshold, and after each initial attitude estimated result is corrected in at least two initial attitudes estimated result with it is another just The camera posture that difference after beginning Attitude estimation modified result is equal to corresponding camera two-by-two is poor, revised initial attitude estimation knot Fruit is initial attitude estimated result plus amendment increment；

Object's pose estimation is calculated as a result, the targeted attitude result is in at least two initial attitudes estimated result Any initial attitude estimated result corrects the sum of increment with it, and the targeted attitude result is transformed into and is estimated posture object institute Under world coordinate system.

2. the method for object gesture estimation according to claim 1, which is characterized in that any two phases in the multiple camera Machine is first camera and second camera, the multiple camera of determination and estimated in posture object composition system two-by-two camera it Between camera posture difference include：

By it is described estimated posture object centered on establish world coordinate system, to the first camera and the second camera respectively into Row location position obtains respective external parameter, obtains the corresponding spin matrix R of first camera_LSpin moment corresponding with second camera Battle array R_R, the spin matrix is rotation transformation of the camera coordinates system to world coordinate system；

Utilize spin matrix R_LWith spin matrix R_RThe spin matrix R from first camera to second camera is calculated_LoR, wherein R_LoR=R_L ^-*R_R；

By the calculating of Eulerian angles by R_LoRIt is decomposed into pitch angle, the rotation angle in three directions of roll angle and yaw angle obtains first Camera posture between camera and second camera is poor.

3. the method for object gesture estimation according to claim 1, which is characterized in that described to be clapped using the multiple camera The method that the view taken the photograph carries out Attitude estimation respectively is to be regarded to camera shooting based on 3D models method corresponding with 2D characteristic points Figure estimated the Attitude estimation of object；

It is A and B in at least two initial attitudes estimated result, A is to carry out Attitude estimation to the view that first camera is shot After obtain, B is to be obtained after view shoot to second camera carries out Attitude estimation, and A and the respective amendment increments of B are △ A and △ B When, it is described to establish optimization object function and its constraints using at least two initial attitudes estimated result, it is solved The increment for obtaining each initial attitude estimated result includes：

Establish optimization object function

Constraints is：

A+ΔA+CONST≈B+ΔB；

F_L(A+ Δs A, i)=| | f_L(A+ΔA,MP_i)-Pi (L) | |,

F_R(B+ Δs B, i)=| | f_R(B+ΔB,MP_i)-Pi(R)||；

Wherein, camera postures of the CONST between first camera and second camera is poor, MP_iIt is 3D models under object coordinates system I-th point, Pi (L) corresponds to 3D model points MP_iFirst camera shooting view in correspondence 2D characteristic points, Pi (R) is Corresponding to 3D model points MP_iSecond camera shooting view in correspondence 2D characteristic points, f_L() is the 3D models of first camera Point arrives the projective transformation formula of 2D characteristic points, f_R() be second camera 3D model points to 2D characteristic points projective transformation formula, V is preset second threshold；

4. the method for object gesture estimation according to claim 3, which is characterized in that described to utilize iterative method to described excellent Change object function solve：

It calculates and corrects total amount △ D=B-A-CONST；

Enable n=0, S=△ D, A₀=A,

Into iterative process：

while(S>=preset third threshold value)

n++；

Scheme { the A candidate from three_n-1,A_n-1+S,A_n-1- S } in selected based on optimization object function and constraints it is best Scheme, it is A to enable it_n, wherein each scheme Ci meets condition C i in 3 candidate schemes>=min { A, A+ △ D } and Ci<max {A,A+△D}；

S=S/2；

}

S is the step-length of each round iteration；

If S<Default third threshold value, stops iteration；

At this point, △ A=An-A are obtained, △ B=△ A- △ D.

5. a kind of device of object gesture estimation, which is characterized in that estimated posture object progress appearance to same using multiple cameras State estimates that the multiple camera can shoot the view for being estimated posture object, and the multiple camera with different view In posture subject area of being estimated two-by-two taken by associated camera there is the part of overlapping, described device to include：

Camera posture difference determining module, for determine the multiple camera and estimated in posture object composition system two-by-two camera it Between camera posture it is poor, the camera posture difference is by different cameral visual angle to the same posture estimated posture object and obtained As a result fixed difference between is equal under world coordinate system from a camera along three orthogonal directions to another camera Rotation angle, the posture subject area of being estimated that the camera two-by-two respectively takes have the part of overlapping；

Initial estimation module, the view for being shot using the multiple camera carry out Attitude estimation respectively, obtain multiple initial Attitude estimation result；

Optimization Solution module, for selecting to melt from the multiple initial attitude estimated result according to the camera posture difference At least two initial attitude estimated results closed, the posture difference between described two initial attitude estimated results and corresponding camera Difference distance between camera posture difference is less than preset first threshold value；It is established using at least two initial attitudes estimated result Optimization object function and its constraints are solved to obtain the amendment increment of each initial attitude estimated result, wherein described All characteristic points in corresponding view after optimization object function is corrected for the minimum at least two initial attitudes estimated result Average back projection error, the constraints be at least two initial attitudes estimated result correct after it is corresponding each The average back projection error of characteristic point is less than default second threshold, and at least two initial attitudes estimated result in view In each initial attitude estimated result correct after with another revised difference of initial attitude estimated result be equal to corresponding two two-phase The camera posture of machine is poor, and revised initial attitude estimated result is initial attitude estimated result plus amendment increment；

Computing module, for object's pose estimation to be calculated as a result, the targeted attitude result is described at least two initial Any initial attitude estimated result corrects the sum of increment with it in Attitude estimation result；

Conversion module, for after object's pose estimation result being calculated, the targeted attitude result being transformed into and is estimated Under world coordinate system where posture object.

6. the device of object gesture estimation according to claim 5, which is characterized in that the camera posture difference determining module Including：

Location position unit, for by it is described estimated posture object centered on establish world coordinate system, to the first camera and The second camera carries out location position and obtains respective external parameter respectively, obtains the corresponding spin matrix R of first camera_LWith The corresponding spin matrix R of two cameras_R, the spin matrix is rotation transformation of the camera coordinates system to world coordinate system；

Computing unit, for utilizing spin matrix R_LWith spin matrix R_RThe rotation from first camera to second camera is calculated Matrix R_LoR, wherein R_LoR=R_L ^-*R_R；

Resolving cell, for by the calculating of Eulerian angles by R_LoRIt is decomposed into pitch angle, the rotation in three directions of roll angle and yaw angle Corner, the camera posture obtained between first camera and second camera are poor.

7. the device of object gesture estimation according to claim 5, which is characterized in that described to be clapped using the multiple camera The method that the view taken the photograph carries out Attitude estimation respectively is to be regarded to camera shooting based on 3D models method corresponding with 2D characteristic points Figure estimated the Attitude estimation of object；

It is A and B in at least two initial attitudes estimated result, A is to carry out Attitude estimation to the view that first camera is shot After obtain, B is to be obtained after view shoot to second camera carries out Attitude estimation, and A and the respective amendment increments of B are △ A and △ B When, the Optimization Solution module includes：

Selecting unit, for selected can to merge from the multiple initial attitude estimated result according to the camera posture difference At least two initial attitude estimated result A and B；

Constraints is：

A+ΔA+CONST≈B+ΔB；

F_L(A+ Δs A, i)=| | f_L(A+ΔA,MP_i)-Pi (L) | |,

F_R(B+ Δs B, i)=| | f_R(B+ΔB,MP_i)-Pi(R)||；

It solves unit and obtains △ A and △ B for being solved to the optimization object function using enumerative technique or iterative method.

8. the device of object gesture estimation according to claim 7, which is characterized in that the solution unit is specifically used for：

It calculates and corrects total amount △ D=B-A-CONST；

Enable n=0, S=△ D, A₀=A,

Into iterative process：

while(S>=preset third threshold value)

n++；

S=S/2；

}

S is the step-length of each round iteration；

If S<Default third threshold value, stops iteration；

At this point, △ A=An-A are obtained, △ B=△ A- △ D.

9. a kind of electronic equipment of object gesture estimation, which is characterized in that using multiple cameras to it is same estimated posture object into Row Attitude estimation, the multiple camera can shoot the view for being estimated posture object with different view, and the multiple Posture subject area of being estimated in camera two-by-two taken by associated camera has the part of overlapping, the electronic equipment packet It includes：

Processor；With

Memory is stored with computer program instructions in the memory,

Wherein, when the computer program instructions are run by the processor so that the processor executes following steps：