CN103578133B - A kind of method and apparatus that two-dimensional image information is carried out to three-dimensional reconstruction - Google Patents

Abstract

The embodiment of the present invention relates to perspective geometry technical field, be particularly related to a kind of method and apparatus that two-dimensional image information is carried out to three-dimensional reconstruction, in order to solve the existing process more complicated that two-dimensional image information is reverted to corresponding three-dimensional body information, be unfavorable for the extensive problem using. The method of two-dimensional image information being carried out to three-dimensional reconstruction that the embodiment of the present invention provides comprises: determine the intrinsic dot information on location parameter and hardware parameter and two dimensional image; Determine mapping matrix according to definite location parameter, hardware parameter and intrinsic dot information; According to definite mapping matrix, the pixel on two dimensional image is converted to the coordinate points in three-dimensional coordinate; And according to each coordinate points in the three-dimensional coordinate being converted to, determine the three-dimensional body information that two dimensional image is corresponding. The embodiment of the present invention has realized has simplified the process that two-dimensional image information is reverted to corresponding three-dimensional body information, and then is conducive to further expand range of application.

Description

A kind of method and apparatus that two-dimensional image information is carried out to three-dimensional reconstruction

Technical field

The present invention relates to perspective geometry technical field, particularly a kind of side that two-dimensional image information is carried out to three-dimensional reconstructionMethod and equipment.

Background technology

The video camera that is fixed on commanding elevation place can obtain a series of image in the time taking. By these images, energyObtain the two-dimensional image information of object, but but cannot obtain the three-dimensional information of object, such as velocity information, the size letter of objectBreath and shape information etc. In order to obtain these information, the two-dimensional image information of object must be reverted to corresponding three-dimensional letterBreath, i.e. three-dimensional reconstruction. Carrying out three-dimensional reconstruction need to be converted into two dimensional image by three-dimensional scenic by camera or video camera, analyzesThe corresponding relation of three-dimensional body point and X-Y scheme picture point, and utilize this corresponding relation, calculate object by two-dimensional image informationThree-dimensional information. Wherein, mapping matrix is the matrix that characterizes the corresponding relation between two-dimensional image information and three-dimensional body.

Existing way is that calibration device is placed in monitoring scene, by analyzing and processing the calibration device that contains photographingImage determine the corresponding relation between two dimensional image and three-dimensional body, and according to this corresponding relation, two dimensional image is believedBreath reverts to corresponding three-dimensional body information. Owing to requiring must to place calibration device in each monitoring scene, and require to demarcateThe putting position of device is suitable, therefore operates cumbersome, causes two-dimensional image information to revert to corresponding three-dimensional letterBreath process meeting more complicated, is unfavorable for large-scale use.

In sum, current three-dimensional rebuilding method must be placed calibration device in monitoring scene, and requires to demarcateThe putting position of device is suitable, due to higher to the requirement of calibration device and monitoring scene, therefore operates cumbersome, leadsCause the process meeting more complicated that two-dimensional image information is reverted to corresponding three-dimensional information, be unfavorable for large-scale use.

Summary of the invention

The embodiment of the present invention provides a kind of method and apparatus that two-dimensional image information is carried out to three-dimensional reconstruction, existing in order to solveThe process more complicated that two-dimensional image information is reverted to corresponding three-dimensional body information having, is unfavorable for extensive asking of usingTopic.

A kind of method of two-dimensional image information being carried out to three-dimensional reconstruction that the embodiment of the present invention provides, comprising:

Determine that the equipment of finding a view is taking location parameter in two dimensional image process and hardware parameter and with the equipment bat of finding a viewIntrinsic dot information on the two dimensional image of taking the photograph;

Be identified for characterizing according to definite location parameter, hardware parameter and intrinsic dot information the two dimension that the equipment of finding a view is takenThe mapping matrix of corresponding relation between image and three-dimensional body;

According to definite mapping matrix, the pixel relevant to object on the two dimensional image that the described use equipment of finding a view is takenPoint is converted to the coordinate points in three-dimensional coordinate;

According to each coordinate points in the three-dimensional coordinate being converted to, determine the two dimensional image correspondence of taking with the equipment of finding a viewThree-dimensional body information.

A kind of equipment that two-dimensional image information is carried out to three-dimensional reconstruction provided by the invention, comprising:

The first determination module, for determining location parameter and the hardware ginseng of the equipment of finding a view in shooting two dimensional image processIntrinsic dot information on the two dimensional image of counting and take with the equipment of finding a view;

The second determination module, characterizes for being identified for according to definite location parameter, hardware parameter and intrinsic dot informationThe mapping matrix of corresponding relation between the two dimensional image that the equipment of finding a view is taken and three-dimensional body;

Modular converter, for according to definite mapping matrix, on the two dimensional image that the described use equipment of finding a view is taken withThe pixel that object is relevant is converted to the coordinate points in three-dimensional coordinate;

Processing module, for according to each coordinate points of the three-dimensional coordinate being converted to, determines with the equipment shooting of finding a viewThree-dimensional body information corresponding to two dimensional image.

In embodiments of the present invention, determine that the equipment of finding a view is in location parameter and the hardware ginseng of taking in two dimensional image processIntrinsic dot information on the two dimensional image of counting and take with the equipment of finding a view; According to definite location parameter, hardware parameter and basisLevy the mapping matrix that dot information is identified for characterizing corresponding relation between two dimensional image and three-dimensional body; According to definite mapping squareBattle array, is converted to the coordinate points in three-dimensional coordinate by the pixel relevant to object on two dimensional image; And according to being converted toEach coordinate points in three-dimensional coordinate, determines the three-dimensional body information that two dimensional image is corresponding. Due to whether placing in sceneCalibration device does not have specific requirement, simplifies the process of demarcating thereby realized, and correspondingly, has simplified two-dimensional image information is reverted toThe process of corresponding three-dimensional body information, and then be conducive to further expand range of application.

Brief description of the drawings

Fig. 1 is the method flow schematic diagram that the embodiment of the present invention is carried out two-dimensional image information three-dimensional reconstruction;

Fig. 2 is that the corresponding relation of pixel in calibration point in embodiment of the present invention three-dimensional system of coordinate and two-dimensional coordinate system showsIntention;

Fig. 3 is the embodiment of the present invention line schematic diagram that goes out;

Fig. 4 is the device structure schematic diagram that the embodiment of the present invention is carried out two-dimensional image information three-dimensional reconstruction.

Detailed description of the invention

The embodiment of the present invention, determine the equipment of finding a view take location parameter in two dimensional image process and hardware parameter, withAnd with find a view equipment take two dimensional image on intrinsic dot information; According to definite location parameter, hardware parameter and intrinsic pointInformation is identified for characterizing the mapping matrix of corresponding relation between two dimensional image and three-dimensional body; According to definite mapping matrix,The pixel relevant to object on two dimensional image is converted to the coordinate points in three-dimensional coordinate; And according to the three-dimensional being converted toEach coordinate points in coordinate, determines the three-dimensional body information that two dimensional image is corresponding. Due to whether place demarcation in sceneDevice does not have specific requirement, simplifies the process of demarcating thereby realized, and correspondingly, has simplified two-dimensional image information is reverted to correspondenceThe process of three-dimensional body information, and then be conducive to further expand range of application.

Preferably, the equipment of finding a view can be video camera, be camera, the mobile phone with camera function, can also be with taking the photographAny equipment of picture head.

Below in conjunction with Figure of description, the embodiment of the present invention is described in further detail.

As shown in Figure 1, the method that two-dimensional image information is carried out three-dimensional reconstruction by the embodiment of the present invention comprises the following steps:

Step 101, determine that the equipment of finding a view is taking location parameter in two dimensional image process and hardware parameter and with gettingIntrinsic dot information on the two dimensional image that scape equipment is taken;

Step 102, location parameter, hardware parameter and intrinsic dot information according to definite are identified for characterizing the equipment of finding a view and clapThe mapping matrix of corresponding relation between the two dimensional image of taking the photograph and three-dimensional body;

Step 103, according to definite mapping matrix, by described use find a view equipment take two dimensional image on object phaseThe pixel closing is converted to the coordinate points in three-dimensional coordinate;

Each coordinate points in step 104, the basis three-dimensional coordinate being converted to, determines the two dimension of taking with the equipment of finding a viewThe three-dimensional body information that image is corresponding.

Preferably, step 101, definite location parameter and the hardware parameter of equipment in shooting two dimensional image process of finding a view,Specifically comprise:

Determine that the equipment of finding a view is at many groups location parameter and the hardware parameter taken in two dimensional image process;

From definite many groups location parameter and hardware parameter, select one group of location parameter and hardware parameter;

Using one group of location parameter selecting and hardware parameter as the equipment of finding a view in the position of taking in two dimensional image processParameter and hardware parameter.

Wherein, the equipment of finding a view described in comprises at the location parameter of taking in two dimensional image process:

The described equipment of finding a view is around the X-axis anglec of rotation, described in find a view equipment around the Y-axis anglec of rotation, described in find a view equipment around Z axisThe anglec of rotation and described in find a view equipment apart from the height on ground, wherein, X-axis, Y-axis and Z axis form left-handed coordinate system;

The described equipment of finding a view comprises at the hardware parameter of taking in two dimensional image process: the focal length of the equipment of finding a view and distortion systemNumber.

Preferably, hardware parameter find a view the focal length of equipment be find a view equipment around the X-axis anglec of rotation according to location parameter andThe location parameter equipment of finding a view is determined around the Y-axis anglec of rotation.

In concrete enforcement, determine that the equipment of finding a view comprises at many groups location parameter of taking in two dimensional image process:

Determine that the equipment of finding a view is around the span of the X-axis anglec of rotation, the equipment of finding a view is around the span of the Y-axis anglec of rotation,And the span of equipment apart from the height on ground of finding a view;

Span according to the equipment of finding a view of determining around the X-axis anglec of rotation, the value of equipment around the Y-axis anglec of rotation of finding a viewScope, and the equipment of finding a view determines that apart from the span of the height on ground the equipment of finding a view is taking many in two dimensional image processGroup location parameter.

Taking the equipment of finding a view apart from the height on ground as definite value 1m(rice), the equipment of finding a view is around the X-axis anglec of rotation and find a view and establishStandby is that the value changing is within the specific limits example around the Y-axis anglec of rotation, introduces according to the equipment of finding a view of determining around the X-axis anglec of rotationSpan, the equipment of finding a view is around the span of the Y-axis anglec of rotation, and the value model of equipment apart from the height on ground of finding a viewEnclose and determine that the equipment of finding a view is in the detailed process of taking the many groups location parameter in two dimensional image process.

In concrete enforcement, the equipment of finding a view around X-axis anglec of rotation α span is: 0 ° ~ 360 °, the equipment of finding a view rotates around Y-axisAngle beta span is: 0 ° ~ 360 °.

The example that is spaced apart with 1 ° is introduced, and the probable value of α is 0 °, and 1 °, 2 ° ... 360 °, the probable value of β is 0 °,1 °, 2 ° ... 360 °, likely the combining and comprise of α and β: (0 °, 0 °), (0 °, 1 °) ..., (0 °, 360 °),(1°,0°),(1°,1°),......,(1°,360°),......,(360°,0°),(360°,1°),......,(360°,360 °) combination in totally 130321, can determine that the equipment of finding a view is at 130321 groups of location parameters taking in two dimensional image process.

Preferably, can select as required 130321 groups of part group location parameters in location parameter as getting of determiningScape equipment is at many groups location parameter of taking in two dimensional image process.

Preferably, can dwindle as required the interval of span or the change value of α and β, correspondingly, that determines getsScape equipment also can change in the location parameter combination of taking in two dimensional image process.

Preferably, the equipment of finding a view can, for the value changing within the specific limits, be supposed the equipment of finding a view apart from the height on groundBe 1m or 1.5m apart from the height value on ground, can determine that the equipment of finding a view is taking 260642 in two dimensional image process(130321 and 2 products) group location parameter.

Preferably, the equipment of finding a view can be definite value apart from the height on ground, and α and β change within the specific limitsValue; α can be definite value, and the equipment of finding a view can be the value changing within the specific limits, the equipment distance of finding a view apart from height and the β on groundHeight, α and β overhead can be within the specific limits the value changing, or other find a view equipment apart from the height on ground,The random combine of α and β is all suitable for the present invention, can determine with application scenarios as required particularly.

In force, for select one group of location parameter from many groups location parameter of determining, can select arbitrarily, also canTo select as requested, the element in below using the α in location parameter and β as the location parameter combination that will selectBe specifically introduced, in enforcement, the element in can only combining using α as the location parameter that will select, can only do with βFor the element in the location parameter combination that will select, can be using the equipment of finding a view apart from the height on ground as selectingLocation parameter combination in element, also can be using α, β and the equipment of finding a view apart from the random combine of the height on ground as enteringElement in the location parameter combination that row is selected, the method for the element in any one chosen position parameters combination is all suitable for thisBright, due to the performance of other kind of parameters combination with using α and β as the location parameter combination that will select in elementPerformance is similar, does not just do concrete introduction at this.

Mode one, from many groups location parameter, select one group of location parameter arbitrarily;

From all groups of α and β, select arbitrarily one group of location parameter;

The example that is spaced apart with 1 ° is introduced, and the probable value of α is 0 °, and 1 °, 2 ° ... 360 °, the probable value of β is 0 °,1 °, 2 ° ... 360 °, likely the combining and comprise of α and β: (0 °, 0 °), (0 °, 1 °) ..., (0 °, 360 °),(1°,0°),(1°,1°),......,(1°,360°),......,(360°,0°),(360°,1°),......,(360°,360 °) combination in totally 130321.

Preferably, can from 130321 combinations, select arbitrarily one group of location parameter.

Preferably, can select at random, can select from minimum of a value (first selecting (0 °, 0 °) combination), can be from maximumSelect (first selecting (360 °, 360 °) combination), also can select from centre.

In enforcement, select different α and β combination, the error difference of the three-dimensional coordinate that definite pixel is corresponding.

Preferably, the value of α and β is taking 1 ° as interval, preferably, can be taking 2 ° as interval, also can taking 0.5 ° as betweenEvery, can set as required particularly.

Mode two, select as requested;

The highest in order to meet the precision of the three-dimensional coordinate that definite pixel is corresponding, according to following method from all groups of α and βOne group of location parameter of middle selection. In concrete enforcement, also can select according to other requirements, such as wanting of concrete application scenariosAsk.

From many groups location parameter of determining, select one group of location parameter according to the following step:

S1, for each group location parameter, determine that respectively any two calibration points are respectively corresponding on described two dimensional imageThe coordinate points of pixel in three-dimensional system of coordinate;

S2, determine distance and two demarcation of two points corresponding to coordinate points in three-dimensional system of coordinate in three-dimensional system of coordinateThe error amount of the actual physics distance between point;

S3, using one group of location parameter of corresponding minimum error values as select one group of location parameter.

Preferably, step S1, for each group location parameter, determines and appoints according to mapping matrix corresponding to this group location parameterTwo calibration points of anticipating are distinguished the coordinate points of corresponding pixel in three-dimensional system of coordinate on described two dimensional image.

In force, the embodiment of the present invention two-dimensional image information is carried out to the method for three-dimensional reconstruction can be at the Horizon of sceneOn face, determine calibration point, also can on the object that is placed in scene, determine calibration point, as long as the calibration point of choosing is in same ginsengExamine in plane.

The embodiment of the present invention need to not placed calibration device in scene, there is no strict wanting for definite calibration point yetAsk, thereby improved the flexibility using, expanded range of application.

Actual physics distance between any two calibration points of supposing to determine is d, and two calibration points are corresponding on imageTwo pixel coordinates be respectively (u₁,v₁) and (u₂,v₂)。

Wherein, in the calibration point in three-dimensional system of coordinate and two-dimensional coordinate system, the corresponding relation of pixel as shown in Figure 2, is being schemedIn 2, some M is a calibration point in three-dimensional system of coordinate, and some m is that calibration point M is clapping with the equipment of finding a view after the equipment imaging of finding a viewPixel on the image of taking the photograph, the coordinate at some M place is the three-dimensional system of coordinate that meets left-handed coordinate system principle, some m placeCoordinate is two-dimensional coordinate system.

For one group by α_iAnd β_iThe location parameter forming, determines the mapping matrix P that this group location parameter is corresponding_i, according to P_iDetermine (u₁,v₁) and (u₂,v₂) two point (X corresponding to coordinate points in three-dimensional system of coordinate₁,Y₁,Z₁) and (X₂,Y₂,Z₂), andCalculate two point (X₁,Y₁,Z₁) and (X₂,Y₂,Z₂) between distance d_i。

Preferably, determine two point (X according to following formula₁,Y₁,Z₁) and (X₂,Y₂,Z₂) between Euclidean distance d_i：

$d_i=\sqrt{{(X_1-X_2)}^2+{(Y_1-Y_2)}^2+{(Z_1-Z_2)}^2}$ Formula (1).

Preferably, other can calculate two point (X₁,Y₁,Z₁) and (X₂,Y₂,Z₂) between distance d_iMethod be all suitable forThe present invention.

Preferably, determine d and d by following formula_iBetween error amount:

Δ(α_i,β_i)＝|d-d_i| formula (2).

Wherein, Δ (α_i,β_i) represent for one group by α_iAnd β_i(the u that the location parameter forming is determined₁,v₁) and (u₂,v₂)Two point (X corresponding to coordinate points in three-dimensional system of coordinate₁,Y₁,Z₁) and (X₂,Y₂,Z₂) between distance d_iWith two calibration pointsBetween actual range d between error.

Preferably, can pass through formula | d²-d_i ²| determine d and d_iBetween error amount, can pass through formulaDetermine d andd_iBetween error amount, also can pass through formula | d_i ²-d²| determine d and d_iBetween error amount, any side that can determine errorFormula is all suitable for the present invention.

Preferably, S3, one group of location parameter of corresponding minimum error amount is comprised as one group of location parameter selecting:

Suppose α_nAnd β_nThe one group of location parameter forming makes d and d_nError amount minimum, α_nAnd β_nThe one group of position formingPut parameter as selected one group of location parameter,

${\mathrm{\α}}_n,{\mathrm{\β}}_n=\underset{({\mathrm{\α}}_i,{\mathrm{\β}}_i)}{\arg \min }\mathrm{\Δ}({\mathrm{\α}}_i,{\mathrm{\β}}_i)$ Formula (3).

Wherein, α_nAnd β_nFor making d and d_nOne group of location parameter of error amount minimum, calculate and obtain by argmin ΔCorresponding α when error minimum of a value_nAnd β_nNumerical value, the value of i is 0,1,2 ... 360, α accordingly_iValue be 0 °, 1 °,2°......360°，β_iValue be 0 °, 1 °, 2 ° ... 360 °.

Preferably, according to the one group of α determining_iAnd β_i, can determine one group of α_iAnd β_iThe focal distance f of the corresponding equipment of finding a view_i。

In above-mentioned example, α_iAnd β_iValue be taking 1 ° as interval, preferably, can be taking 2 ° as interval, also can be with 0.5 °For interval, can set as required particularly.

Preferably, step 102, be identified for characterizing corresponding pass between two dimensional image and the three-dimensional body that the equipment of finding a view takesThe mapping matrix of system, specifically comprises:

Q1, according to the focal length of intrinsic dot information, the distortion factor of described two dimensional image and the equipment of finding a view, determine inner ginsengMatrix number;

Q2, basis are found a view equipment around the X-axis anglec of rotation, and the equipment of finding a view revolves around Z axis around the Y-axis anglec of rotation and the equipment of finding a viewGyration, determines spin matrix;

Q3, according to definite inner parameter matrix, spin matrix and the equipment of finding a view apart from the height on ground, be identified forThe mapping matrix of corresponding relation between the two dimensional image that the sign equipment of finding a view is taken and three-dimensional body.

Preferably, in step Q1, determine inner parameter matrix according to following formula:

$K=\left[\begin{array}{ccc}f& \mathrm{\λ}& u_0\\ 0& f& v_0\\ 0& 0& 1\end{array}\right]$ Formula (4).

Wherein, K is the inner parameter matrix of 3 × 3, (u₀,v₀) the intrinsic point of presentation graphs picture, f represents the equipment of finding a viewFocal length, λ represents the distortion factor of the equipment of finding a view.

In concrete enforcement, because the distortion of camera or video camera is all less, thereby the value of λ can be 0, i.e. λ=0.

Preferably, the central point that the intrinsic point of image is image.

Preferably, the focal distance f in hardware parameter is according to the α in corresponding location parameter, β and vanishingline(line goes out) is definite, and specifically shifting details onto is prior art, referring to bibliography " FengjunLvTaoZhaoRamNevatia，SelfCalibrationofacamerafromvideoofawalkinghuman，ICPR，2002”。

Preferably, in step Q2, determine spin matrix according to following formula:

$R=\left[\begin{array}{ccc}\cos \mathrm{\γ}& -\sin \mathrm{\γ}& 0\\ \sin \mathrm{\γ}& \cos \mathrm{\γ}& 0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{ccc}1& 0& 0\\ 0& \cos {\mathrm{\β}}_i& -{\sin \mathrm{\β}}_i\\ 0& \sin {\mathrm{\β}}_i& \cos {\mathrm{\β}}_i\end{array}\right]\left[\begin{array}{ccc}\cos {\mathrm{\α}}_i& 0& -\sin {\mathrm{\α}}_i\\ 0& 1& 0\\ \sin {\mathrm{\α}}_i& 0& \cos {\mathrm{\α}}_i\end{array}\right]$ Formula (5).

Wherein, matrix R is the spin matrix of 3 × 3, and γ represents that the equipment of finding a view is around the Z axis anglec of rotation, α_iExpression is found a viewEquipment is around the X-axis anglec of rotation, β_iRepresent to find a view equipment around the Y-axis anglec of rotation, α_iAnd β_iFor determining corresponding mapping matrixOne group of location parameter, the value of i is 0,1,2 ... 360, α accordingly_iValue be 0 °, 1 °, 2 ° ... 360 °, β_iValueBe 0 °, 1 °, 2 ° ... 360 °.

Preferably, the equipment of finding a view equals the inclination angle of vanishingline with respect to horizontal direction around Z axis anglec of rotation γ.

Wherein, vanishingline is a concept the most basic in projective geometry, refers in three-dimensional system of coordinateAll parallel lines in the reference planes at the calibration point place of choosing form through the intersection point on two dimensional surface after perspectiveStraight line, as shown in Figure 3, through the straight line dotting of a and the b line that goes out exactly, wherein, the parallel lines warp in three-dimensional system of coordinateAfter crossing perspective, on two dimensional surface, can intersect, such as, real world Zhong street is two parallel lines in three-dimensional system of coordinate,Through intersecting on two dimensional surface after perspective.

Preferably, in prior art, any one method of obtaining the line that goes out is all suitable for the present invention, wherein, and the calculating of the line that goes outMethod can bibliography " Single-ViewMetrology:AlgorithmsandApplications ", AntonioCriminisi，proceedingof24DAGMsymposiumonPatternRecognition。

Preferably, in step Q3, determine mapping matrix according to following formula:

P=K[R|t] formula (6).

Wherein, matrix P is the mapping matrix of 3 × 4, and K is the inner parameter matrix of 3 × 3, and R is one 3 × 3Spin matrix, t is the matrix of 3 × 1, can be expressed as: t=R[0, Hc, 0] T, R is the spin matrix of 3 × 3, HcRepresent the height of the equipment of finding a view apart from ground, T represents [0, Hc, 0] to carry out transposition computing.

3 × 3 spin matrix R and 3 × 1 matrix t splice become after computing 3 × 4 matrix again with 3 × 3 inPortion's parameter matrix multiplies each other and obtains 3 × 4 mapping matrix P.

Preferably, for determine the situation of calibration point on the ground level of scene, can be to the processing of degenerating of P matrix,Make P matrix deteriorate to 3 × 3 matrix from 3 × 4 matrix.

In concrete enforcement, relevant to object on the two dimensional image of the described use equipment of finding a view being taken based on following formulaPixel is converted to the coordinate points in three-dimensional coordinate:

$\mathrm{\α}\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=P\left[\begin{array}{c}X\\ Y\\ Z\\ 1\end{array}\right]=K\left[R\right|t]\left[\begin{array}{c}X\\ Y\\ Z\\ 1\end{array}\right]=K[R_1\left|R_2\right|R_3\left|t\right]\left[\begin{array}{c}X\\ Y\\ Z\\ 1\end{array}\right]$ Formula (7)

Wherein, (u, v) represent after the equipment imaging of finding a view on image the coordinate of pixel; (X, Y, Z) represents three-dimensionalThe coordinate of correspondence in coordinate system; α is an arbitrary constant; P represents mapping matrix, and K represents inner parameter matrix, and R represents rotationMatrix, R₁Represent the first row of R matrix, R₂Represent the secondary series of R matrix, R₃The 3rd row that represent R matrix, t is one 3 × 1Matrix.

In concrete enforcement, α is a proportionality coefficient.

Owing to determining calibration point on the ground level of scene, Z=0,3 × 3 spin matrix R and 3 × 1 matrix t carry out3 × 4 the matrix that splicing obtains after computing and 4 × 1 matrix [X, Y, 0,1]^TWhile multiplying each other, the 3rd row of the matrix obtaining allBe 0, the tertial value of spin matrix R is nonsensical, can be by matrix [X, Y, 0,1]^TIn the third line and spin momentThe 3rd row of battle array R are removed, and P matrix is degenerated and deteriorated to 3 × 3 matrix from 3 × 4 matrix. Correspondingly, formula (7) can be shownBe shown:

$\mathrm{\α}\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=P\left[\begin{array}{c}X\\ Y\\ 1\end{array}\right]=K\left[R_1\right|R_2\left|t\right]\left[\begin{array}{c}X\\ Y\\ 1\end{array}\right]$ Formula (8)

Preferably, mapping matrix P is the matrix for characterizing corresponding relation between two dimensional image and three-dimensional body.

Preferably, for one group of location parameter selecting according to mode two, except determining and select according to aforesaid wayMapping matrix corresponding to one group of location parameter outside, can also be by selecting in one group of location parameter process according to mode twoEstablished location parameter combination is determined one group of mapping matrix that location parameter is corresponding with the corresponding relation of mapping matrix.

Preferably, in selecting one group of location parameter process according to mode two, establish and preserve location parameter combination and mappingThe corresponding relation of matrix.

Preferably, can determine any pixel correspondence on the two dimensional image of taking with the equipment of finding a view according to formula (7)Three-dimensional coordinate.

Preferably, for determine the situation of calibration point on the ground level of scene, can be according to formula (7), formula (8) orThe following formula of person is determined three-dimensional coordinate corresponding to any pixel on the image of taking with the equipment of finding a view:

$\left[\begin{array}{c}X\\ Y\\ 1\end{array}\right]=\mathrm{\α}{P}^{-1}\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]$ Formula (9)

Wherein, P^-1Represent the inverse matrix of the mapping matrix of degeneration after treatment 3 × 3.

Preferably, determine any pixel on the two dimensional image of taking with the equipment of finding a view by formula (8) or formula (9)Three-dimensional coordinate corresponding to point can simplified operation, raises the efficiency and the degree of accuracy.

Preferably, can each coordinate points in the three-dimensional coordinate obtaining in step 103 be analyzed and be processed, therebyObtain the three-dimensional body information corresponding to two dimensional image of taking with the equipment of finding a view.

Preferably, adopt the method for the embodiment of the present invention two-dimensional image information can be changed into three-dimensional body information, enterAnd can obtain shape, size and the velocity information of object.

Preferably, the method for the embodiment of the present invention can be widely used in target classification and the vehicle speed measuring in monitoring field.

Preferably, because the method for the embodiment of the present invention does not need to use calibration device, thereby can save a large amount of expenses,Bring great convenience to user's practical operation, be conducive to promote the use of on a large scale.

Based on same inventive concept, in the embodiment of the present invention, also provide a kind of two-dimensional image information has been carried out to three-dimensional reconstructionEquipment, because the principle that equipment is dealt with problems is similar to the method for the embodiment of the present invention, therefore the enforcement of equipment can be referring toThe enforcement of method, repeats part and repeats no more.

Fig. 4 is the device structure schematic diagram that the embodiment of the present invention is carried out two-dimensional image information three-dimensional reconstruction, as shown in the figure,The embodiment of the present invention really equipment of position fixing comprises:

The first determination module 401, for determining that the equipment of finding a view is at location parameter and the hardware of taking two dimensional image processParameter and with find a view equipment take two dimensional image on intrinsic dot information;

The second determination module 402, for being identified for table according to definite location parameter, hardware parameter and intrinsic dot informationThe mapping matrix of corresponding relation between the two dimensional image that the scape equipment of collecting is taken and three-dimensional body;

Modular converter 403, for according to definite mapping matrix, on the two dimensional image that the described use equipment of finding a view is takenThe pixel relevant to object is converted to the coordinate points in three-dimensional coordinate;

Processing module 404, for according to each coordinate points of the three-dimensional coordinate being converted to, determines with the equipment bat of finding a viewThree-dimensional body information corresponding to two dimensional image of taking the photograph.

Preferably, described the first determination module 401 specifically for:

Determine that the equipment of finding a view is at many groups location parameter and the hardware parameter taken in two dimensional image process; From many groups that determineIn location parameter and hardware parameter, select one group of location parameter and hardware parameter; And by one group of location parameter selecting and hardware ginsengNumber is as location parameter and the hardware parameter of equipment in shooting two dimensional image process of finding a view.

Preferably, described the first determination module 401 specifically for:

For each group location parameter and hardware parameter, determine that respectively any two calibration points divide on described two dimensional imageThe not coordinate points of corresponding pixel in three-dimensional system of coordinate; Determine that corresponding two o'clock of coordinate points in three-dimensional system of coordinate is threeThe error amount of the distance in dimension coordinate system and the actual physics distance between two calibration points; And by one of corresponding minimum error valuesGroup location parameter and hardware parameter are as one group of location parameter selecting and hardware parameter.

Preferably, the equipment of finding a view described in comprises at the location parameter of taking in two dimensional image process:

The described equipment of finding a view is around the X-axis anglec of rotation, described in find a view equipment around the Y-axis anglec of rotation, described in find a view equipment around Z axisThe anglec of rotation and described in find a view equipment apart from the height on ground, wherein, X-axis, Y-axis and Z axis form left-handed coordinate system;

The described equipment of finding a view comprises at the hardware parameter of taking in two dimensional image process: the focal length of the equipment of finding a view and distortion systemNumber.

Preferably, described the second determination module 402 specifically for:

According to the focal length of intrinsic dot information, the distortion factor of described two dimensional image and the equipment of finding a view, determine inner parameterMatrix; According to the equipment of finding a view around the X-axis anglec of rotation, the equipment of finding a view around the Y-axis anglec of rotation and the equipment of finding a view around the Z axis anglec of rotationDegree, determines spin matrix; And according to definite inner parameter matrix, spin matrix and the height of equipment apart from ground of finding a viewDegree, is identified for characterizing the mapping matrix of corresponding relation between two dimensional image that the equipment of finding a view takes and three-dimensional body.

Preferably, described modular converter 403 specifically for:

The pixel relevant to object on the two dimensional image of the described use equipment of finding a view being taken based on formula (7) is converted toCoordinate points in three-dimensional coordinate.

Preferably, the equipment of finding a view described in is around the Z axis anglec of rotation inclination angle of line with respect to horizontal direction that equal to go out;

Wherein, the line that goes out refers to all parallel lines processes in the reference planes at the calibration point place of choosing in three-dimensional system of coordinateThe straight line that intersection point after perspective on two dimensional surface forms.

Those skilled in the art should understand, embodiments of the invention can be provided as method, equipment (system) or calculateMachine program product. Therefore, the present invention can adopt complete hardware implementation example, complete implement software example or in conjunction with software and hardware sideThe form of the embodiment of face. And the present invention can adopt in one or more computer usable program code that wherein includeThe upper computer of implementing of computer-usable storage medium (including but not limited to magnetic disc store, CD-ROM, optical memory etc.)The form of program product.

The present invention is that reference is according to the flow process of the method for the embodiment of the present invention, equipment (system) and computer programFigure and/or block diagram are described. Should understand can be by computer program instructions realization flow figure and/or block diagram often first-classFlow process in journey and/or square frame and flow chart and/or block diagram and/or the combination of square frame. These computer programs can be providedInstruction is arrived the processor of all-purpose computer, special-purpose computer, Embedded Processor or other programmable data processing device to produceA raw machine, produces for reality the instruction of carrying out by the processor of computer or other programmable data processing deviceThe device of the function of specifying in flow process of flow chart or multiple flow process and/or square frame of block diagram or multiple square frame now.

These computer program instructions also can be stored in can vectoring computer or other programmable data processing device with spyDetermine in the computer-readable memory of mode work, the instruction generation that makes to be stored in this computer-readable memory comprises fingerMake the manufacture of device, this command device realize at flow process of flow chart or multiple flow process and/or square frame of block diagram orThe function of specifying in multiple square frames.

These computer program instructions also can be loaded in computer or other programmable data processing device, make at meterOn calculation machine or other programmable devices, carry out sequence of operations step to produce computer implemented processing, thus at computer orThe instruction of carrying out on other programmable devices is provided for realizing at flow process of flow chart or multiple flow process and/or block diagram oneThe step of the function of specifying in individual square frame or multiple square frame.

Although described the preferred embodiments of the present invention, once obtaining cicada, those skilled in the art substantially createProperty concept, can make other change and amendment to these embodiment. So it is excellent that claims are intended to be interpreted as comprisingSelect embodiment and fall into all changes and the amendment of the scope of the invention.

Obviously, those skilled in the art can carry out various changes and modification and not depart from essence of the present invention the present inventionGod and scope. Like this, if these amendments of the present invention and modification belong to the scope of the claims in the present invention and equivalent technologies thereofWithin, the present invention be also intended to comprise these change and modification interior.

Claims (10)

1. a method of two-dimensional image information being carried out to three-dimensional reconstruction, is characterized in that, the method comprises:

Determine that the equipment of finding a view is taking location parameter in two dimensional image process and hardware parameter and with the equipment shooting of finding a viewIntrinsic dot information on two dimensional image;

Be identified for characterizing according to definite location parameter, hardware parameter and intrinsic dot information the two dimensional image that the equipment of finding a view is takenAnd the mapping matrix of corresponding relation between three-dimensional body;

According to definite mapping matrix, the pixel relevant to object on the two dimensional image that the described use equipment of finding a view is taken turnsBe changed to the coordinate points in three-dimensional coordinate;

According to each coordinate points in the three-dimensional coordinate being converted to, determine with find a view equipment take two dimensional image corresponding threeDimension object information;

Described definite equipment of finding a view, at location parameter and the hardware parameter taken in two dimensional image process, specifically comprises:

Determine that the equipment of finding a view is at many groups location parameter and the hardware parameter taken in two dimensional image process; From many groups position of determiningIn parameter and hardware parameter, select one group of location parameter and hardware parameter; Using one group of location parameter selecting and hardware parameter asThe equipment of finding a view is at location parameter and the hardware parameter taken in two dimensional image process;

Described from determine many groups location parameter and hardware parameter select one group of location parameter and hardware parameter, comprising:

For each group location parameter and hardware parameter, determine that respectively any two calibration points are right respectively on described two dimensional imageThe coordinate points of the pixel of answering in three-dimensional system of coordinate; And two points corresponding to coordinate points in definite three-dimensional system of coordinate are in three-dimensionalThe error amount of the actual physics distance between the distance in coordinate system and two calibration points; By one group of position of corresponding minimum error valuesPut parameter and hardware parameter as one group of location parameter selecting and hardware parameter.

2. the method for claim 1, is characterized in that, described in find a view equipment in the position of taking in two dimensional image processParameter comprises:

The described equipment of finding a view is around the X-axis anglec of rotation, described in find a view equipment around the Y-axis anglec of rotation, described in the equipment of finding a view rotate around Z axisAngle and described in find a view equipment apart from the height on ground, wherein, X-axis, Y-axis and Z axis form left-handed coordinate system;

The described equipment of finding a view comprises at the hardware parameter of taking in two dimensional image process: the focal length of the equipment of finding a view and distortion factor.

3. method as claimed in claim 2, is characterized in that, is identified for characterizing the two dimensional image and three that the equipment of finding a view is takenThe mapping matrix of corresponding relation between dimension object, specifically comprises:

According to the focal length of intrinsic dot information, the distortion factor of described two dimensional image and the equipment of finding a view, determine inner parameter matrix;And

According to the equipment of finding a view around the X-axis anglec of rotation, the equipment of finding a view around the Y-axis anglec of rotation and the equipment of finding a view around the Z axis anglec of rotation,Determine spin matrix;

, be identified for characterizing and get apart from the height on ground according to definite inner parameter matrix, spin matrix and the equipment of finding a viewThe mapping matrix of corresponding relation between the two dimensional image that scape equipment is taken and three-dimensional body.

4. the method for claim 1, is characterized in that, according to definite mapping matrix, described in inciting somebody to action based on following formulaThe pixel relevant to object on the two dimensional image of taking with the equipment of finding a view is converted to the coordinate points in three-dimensional coordinate:

Wherein, (u, v) represents the coordinate of pixel on two dimensional image; (X, Y, Z) represents coordinate corresponding in three-dimensional system of coordinate; αIt is an arbitrary constant; P represents mapping matrix.

5. method as claimed in claim 2, is characterized in that, described in the equipment of finding a view equal around the Z axis anglec of rotation line that goes outVanishingline is with respect to the inclination angle of horizontal direction;

Wherein, the line that goes out refers to that all parallel lines in the reference planes at the calibration point place of choosing in three-dimensional system of coordinate are through perspectiveThe straight line that intersection point on two dimensional surface forms afterwards.

6. an equipment that two-dimensional image information is carried out to three-dimensional reconstruction, is characterized in that, this equipment comprises:

The first determination module, for determine the equipment of finding a view take the location parameter of two dimensional image process and hardware parameter, withAnd with find a view equipment take two dimensional image on intrinsic dot information;

The second determination module, finds a view for being identified for according to definite location parameter, hardware parameter and intrinsic dot information characterizingThe mapping matrix of corresponding relation between the two dimensional image that equipment is taken and three-dimensional body;

Modular converter, for according to definite mapping matrix, on the two dimensional image that the described use equipment of finding a view is taken and objectRelevant pixel is converted to the coordinate points in three-dimensional coordinate;

Processing module, for according to each coordinate points of the three-dimensional coordinate being converted to, determines with two of the equipment shooting of finding a viewThree-dimensional body information corresponding to dimension image;

Described the first determination module specifically for:

Determine that the equipment of finding a view is at many groups location parameter and the hardware parameter taken in two dimensional image process; From many groups position of determiningIn parameter and hardware parameter, select one group of location parameter and hardware parameter; And one group of location parameter selecting and hardware parameter are doneFor the equipment of finding a view is at location parameter and the hardware parameter taken in two dimensional image process;

Described the first determination module is selected one group of location parameter and hardware ginseng from definite many groups location parameter and hardware parameterWhen number, described the first determination module specifically for:

For each group location parameter and hardware parameter, determine that respectively any two calibration points are right respectively on described two dimensional imageThe coordinate points of the pixel of answering in three-dimensional system of coordinate; Determine that two points corresponding to coordinate points in three-dimensional system of coordinate sit in three-dimensionalThe error amount of the distance in mark system and the actual physics distance between two calibration points; And by one group of position of corresponding minimum error valuesPut parameter and hardware parameter as one group of location parameter selecting and hardware parameter.

7. equipment as claimed in claim 6, is characterized in that, described in find a view equipment in the position of taking in two dimensional image processParameter comprises:

The described equipment of finding a view is around the X-axis anglec of rotation, described in find a view equipment around the Y-axis anglec of rotation, described in the equipment of finding a view rotate around Z axisAngle and described in find a view equipment apart from the height on ground, wherein, X-axis, Y-axis and Z axis form left-handed coordinate system;

The described equipment of finding a view comprises at the hardware parameter of taking in two dimensional image process: the focal length of the equipment of finding a view and distortion factor.

8. equipment as claimed in claim 7, is characterized in that, described the second determination module specifically for:

According to the focal length of intrinsic dot information, the distortion factor of described two dimensional image and the equipment of finding a view, determine inner parameter matrix;According to the equipment of finding a view, around the X-axis anglec of rotation, the equipment of finding a view, is determined around the Z axis anglec of rotation around the Y-axis anglec of rotation and the equipment of finding a viewSpin matrix; And according to definite inner parameter matrix, spin matrix and the equipment of finding a view apart from the height on ground, determine and useThe mapping matrix of corresponding relation between the two dimensional image of taking in the sign equipment of finding a view and three-dimensional body.

9. equipment as claimed in claim 6, is characterized in that, described modular converter specifically for:

The pixel relevant to object on the two dimensional image of the described use equipment of finding a view being taken based on following formula is converted to threeCoordinate points in dimension coordinate:

Wherein, (u, v) represents the coordinate of pixel on two dimensional image; (X, Y, Z) represents coordinate corresponding in three-dimensional system of coordinate; αIt is an arbitrary constant; P represents mapping matrix.

10. equipment as claimed in claim 7, is characterized in that, described in the equipment of finding a view equal around the Z axis anglec of rotation line that goes outVanishingline is with respect to the inclination angle of horizontal direction;

Wherein, the line that goes out refers to that all parallel lines in the reference planes at the calibration point place of choosing in three-dimensional system of coordinate are through perspectiveThe straight line that intersection point on two dimensional surface forms afterwards.