CN104376558B - Cuboid-based intrinsic parameter calibration method for Kinect depth camera - Google Patents

Abstract

The invention discloses a cuboid-based intrinsic parameter calibration method for a Kinect depth camera. The method comprises the steps that a high-precision cuboid is placed on the ground, and the Kinect depth camera is used for photographing the cuboid; according to multi-frame image depth data, normal maps with the same frame number are generated; the plane of a calibration object is divided to obtain plane depth data; a three-dimensional point set is obtained through the inverse process of perspective projection; the least squares fitting method is performed to obtain corresponding planes; the included angle and the distance between the marked planes are calculated; the angle and the distance of the high-precision cuboid are actually measured and compared, an optimized objective function with the difference value being the smallest as the purpose is constructed, intrinsic parameters of the Kinect depth camera are optimized, in this way, the objective function is minimized, and calibration on the Kinect depth camera is completed. According to the method, only depth information is used, and the method can be widely applied to a series of depth cameras and application scenarios; the intrinsic parameters of the calibrated camera can be used for three-dimensional reconstruction, and the method improves accuracy compared with an existing calibration method.

Description

A kind of internal reference scaling method of the Kinect depth camera based on cuboid

Technical field

The present invention relates to a kind of camera calibration method, especially relate to computer vision field a kind of based on cuboid The internal reference scaling method of Kinect depth camera.

Background technology

The attached body-sensing interactive device Kinect for Xbox 360 game machine that Microsoft sells at the end of 2010 is subject to Arrive the concern of computer research worker.Kinect comprises an ordinary optical camera and by an infrared camera and one The depth camera of infrared projection composition, can return photographed scene according to its depth detection method with the speed of 30 frames per second in real time Depth information.This equipment compares traditional depth camera as a kind of depth camera of consumer level, and to have larger price excellent Gesture, thus receive the welcome of researcher, in human body attitude identification, robot application, object identification, the field such as 3D measurement all goes out Show corresponding research work.Recently as the emergence of 3D printing technique, occur in that using Kinect depth camera to scene Or object carries out the work of real-time three-dimensional reconstruction, wherein more famous work is the KinectFusion work of Microsoft in 2012 Make, this technology by the use of Kinect as depth scan equipment, in conjunction with the parallel processing capability of GPU, can be with real-time reconstruction scanning field The threedimensional model of scape.

Demarcating has very important meaning for camera applications.It would be desirable to understand object in three-dimensional space in 3D vision Between in coordinate and imaging plane coordinate relation.Carry out three-dimensional reconstruction process using depth information mainly to include from depth information To three-dimensional projection, and the internal reference of depth camera directly take part in projection and calculates, thus internal reference directly affects backprojection reconstruction Precision.The calibration process of camera is as asked for being used for the process of the parameter of modeling projection in camera model.Different application scenarios Requirement to camera calibration has differences, and Kinect device when appearing on the scene, once can be demarcated to equipment, and will mark by manufacturer Among the fixed firmware of result write device, this group camera parameter is referred to as manufacturer and demarcates internal reference.Manufacturer demarcates internal reference and disclosure satisfy that The daily entertainment applications of Kinect, but for three-dimensional reconstruction such measurement property application, the calibrating parameters of manufacturer then can not be fine Meet precise requirements, so that demarcating to Kinect depth camera again, in the hope of relatively more accurately deep Degree camera internal reference, improves the accuracy of three-dimensional reconstruction.

Content of the invention

The purpose of the present invention is for the deficiencies in the prior art, provides a kind of Kinect depth camera based on cuboid Internal reference scaling method, carries out depth camera parameter calibration using three-dimensional body, for Kinect depth camera, carries out accurate internal reference Demarcate.

The purpose of the present invention is technical scheme by the following method to be to realize：

1) at least three high accuracy cuboid composition cuboid groups of known dimensions are put on the ground, and deep with Kinect Degree camera is shot in multiple different angles；The flatness on described each surface of high accuracy cuboid reaches 0.1mm, high-precision The infrared light that Kinect depth camera is launched can be reflected in the surface of degree cuboid；

2) according to step 1) the middle multiframe depth data shooting in the depth image obtaining, given birth to by perspective projection inverse process Become the normal direction figure generating same number of frames in the three dimensions to camera local coordinate system for the backing up；

3) and divide, on normal direction figure, the plane demarcating object, obtain plane depth data；

4) by step 3) in the plane depth data that obtains, by perspective projection inverse process backing up to camera local coordinate system Under three dimensions in, obtain corresponding three-dimensional point set Q of each plane；

5) to step 4) each three-dimensional point set Q of obtaining carries out least-square fitting approach, obtains three-dimensional point set Q corresponding Plane；

6) according to step 5) angle between the plane that is labeled of the plane computations that obtain and plane and distance；

7) angle and distance between the labeled plane of actual measurement high accuracy cuboid and plane, and with step 6) The angle obtaining is compared with distance, optimization object function for the purpose of difference minimum for the construction, is made by this optimization object function Use nonlinear iteration optimization method, repeat step 4) arrive step 6), the internal reference of Kinect depth camera is optimized, makes Obtain the minimization of object function, complete the internal reference to Kinect depth camera and demarcate.

Described step 1) at least three high accuracy cuboids pendulum arrangements on the ground enable Kinect depth camera Photograph the public vertex face group being made up of three adjacent surfaces with public vertex and be made up of two faces being parallel to each other Parallel surface group, at least one group of public vertex face group, at least two groups of parallel surface group.

Described step 1) in the different angle shot multiple with respect to the central rotation of cuboid group of Kinect depth camera, At least 9 degree of the angle of rotation every time.

Described step 1) in, in each angle shot, set multiple different Kinect depth cameras to cuboid group The distance between center scope, and randomly choose a shooting distance from each distance range and shot.

Described step 2) in the detailed process of the depth image that obtained by the shooting normal direction figure that generates same number of frames be：2.1) To each pixel in each depth image I, take using centered on it window as method of counting to window W, window width Spend for r；

2.2) each depth point q ' in window is passed through by perspective projection according to the internal reference empirical value being set by production firm In the three dimensions to camera local coordinate system for the inverse process backing up, obtain comprising r²Three-dimensional point set Q={ the q of individual point_i| i=1, 2 ..., r², and q=(x, y, z), wherein each depth point are expressed as q '=(u, v, d (u, v)), (u, v) represents a pixel Coordinate in depth image for the point, d (u, v) represents depth value in depth map for the pixel (u, v)；

2.3) according to three-dimensional point set Q, obtain a plane with least square fitting method, its normal vector is three-dimensional vector n= (a, b, c), translational movement is m；

2.4) by three components a, b, c of three-dimensional vector n be respectively adopted below equation be mapped in rgb space red, green, In blue three values, obtain the RGB color value of window W center pixel：

$R=125.5(a/\sqrt{a^2+b^2+c^2}+1)$

$G=125.5(b/\sqrt{a^2+b^2+c^2}+1)$

$B=125.5(c/\sqrt{a^2+b^2+c^2}+1)$

2.5) to each pixel repeat step 2.1 in depth image I)～2.4), you can for each pixel life in I Become one group of RGB color value, and then obtain a coloured image as the normal direction figure N of this depth image I.

Described step 3) on normal direction figure divide demarcate object plane be specially：

3.1) in cuboid combination, least one set public vertex face group and at least two groups parallel surface groups are selected, to wherein All faces are marked with Arabic numerals；

3.2) in step 2) in generate each normal direction figure on, on each marked face draw one be used for representing The polygon of effective depth pixel region profile on this face；

3.3) polygon is mapped back each self-corresponding depth map of normal direction figure, obtain the corresponding plane depth of each polygon Data.

Described step 2) in by perspective projection inverse process and step 4) in identical by perspective projection inverse process, tool Body process is：According to by the internal reference empirical value initially setting, each depth point q ' in window is passed through below equation backing up to phase In three dimensions under machine local coordinate system, obtain comprising r²Three-dimensional point set Q={ the q of individual point_i| i=1,2 ..., r²}：

$x=\frac{z(u-c_x)}{f_x}......\left(1\right)$

$y=\frac{z(v-c_y)}{f_y}......\left(2\right)$

$z=\frac{1}{{\mathrm{\γ}}_1(d(u,v)+D(u,v)e^{{\mathrm{\α}}_0-{\mathrm{\α}}_{1}d(u,v)})+{\mathrm{\γ}}_0}......\left(3\right)$

Wherein, D is two-dimensional matrix, the line number in two-dimensional matrix D and columns respectively with the vertical and horizontal of depth image I Pixel quantity is identical, and D (u, v) is one of D element value, f_x, f_yRepresent Kinect depth camera in x-axis and y-axis direction respectively On two focal lengths, c_x, c_vRepresent horizontal stroke on imaging surface for the photocentre, the ordinate of Kinect depth camera, γ respectively₀, γ₁Point It is not first, second depth conversion parameter of Kinect depth camera, α₀, α₁, D is the first of Kinect depth camera, respectively Two and the 3rd distortion correction parameter.

Described step 7) in angle and distance between the labeled plane of actual measurement high accuracy cuboid and plane Measure in the following manner：Go out the angle between each plane in above-mentioned labeled public vertex face group with angulation tolerance, use Slide measure measures the distance between above-mentioned labeled parallel surface group midplane.

Described step 6) in angle between labeled plane and plane be specially with distance：Labeled public vertex The distance of the angle of each plane and two planes in labeled parallel surface group in the group of face, angle and distance are respectively by following two Formula obtains：

${\mathrm{\θ}}_{i,j,k}=\arg \cos \left(\frac{n_{i,k}\·n_{j,k}}{\left|n_{i,k}\right|\left|n_{j,k}\right|}\right)......\left(4\right)$

${\mathrm{\δ}}_{i,j,k}=\frac{1}{2}(\frac{1}{\left|Q_{i,k}\right|}\underset{q\∈Q_{i,k}}{\mathrm{\Σ}}\frac{|P_{j,k}\·\hat{q}|}{\left|n_{j,k}\right|}+\frac{1}{\left|Q_{j,k}\right|}\underset{q\∈Q_{j,k}}{\mathrm{\Σ}}\frac{|P_{i,k}\·\hat{q}|}{\left|n_{i,k}\right|})......\left(5\right)$

Wherein, θ_{I, j, k}Refer to plane P_{I, k}And P_{J, k}Between angle, δ_{I, j, k}Refer to plane P_{I, k}And P_{J, k}The distance between, k It is the index of depth map, Q_{I, k}And Q_{J, k}It is any two three-dimensional point set of kth frame depth map, q is a three-dimensional point,It is three-dimensional The homogeneous coordinates of point q, n_{I, k}=(a_{I, k,}b_{I, k}, c_{I, k}) and n_{J, k}=(a_{J, k}, b_{J, k}, c_{J, k}) be respectively and Q_{I, k}And Q_{J, k}Corresponding flat Face normal vector, P_{I, k}=(a_{I, k,}b_{I, k}, c_{I, k},-m_{I, k}) and P_{J, k}=(a_{I, k}, b_{I, k}, c_{I, k},-m_{I, k}) be respectively and Q_{I, k}And Q_{J, k}Corresponding Plane parameter four dimensional vector, P_{I, k}And P_{J, k}Each self-corresponding plane general equation is a respectively_{I, k}x+b_{I, k}y+c_{I, k}Z=m_{I, k}With a_{J, k}x+b_{J, k}y+c_{J, k}Z=m_{J, k}, a_{I, k}、b_{I, k}、c_{I, k}With-m_{I, k}It is respectively and Q_{I, k}The first of corresponding plane general equation, 2nd, the third and fourth parameter, a_{J, k}、b_{J, k}、c_{J, k}With-m_{J, k}It is respectively and Q_{J, k}Corresponding plane general equation first, second, Third and fourth parameter.

Described step 7) in object function be：

$\begin{array}{c}E=S\underset{k\∈{\mathrm{\σ}}_{\mathrm{view}}}{\mathrm{\Σ}}(\frac{\mathrm{\λ}}{{\mathrm{\Σ}}_{(i,j)\∈{\mathrm{\σ}}_{\mathrm{angle}}}1}\underset{(i,j)\∈{\mathrm{\σ}}_{\mathrm{angle}}}{\mathrm{\Σ}}{\left(\frac{{\mathrm{\θ}}_{i,j,k}-{\stackrel{\‾}{\mathrm{\θ}}}_{i,j}}{{\stackrel{\‾}{\mathrm{\θ}}}_{i,j}}\right)}^2\\ +\frac{1-\mathrm{\λ}}{{\mathrm{\Σ}}_{(i,j)\∈{\mathrm{\σ}}_{\mathrm{dis}\tan \mathrm{ce}}}1}\underset{(i,j)\∈{\mathrm{\σ}}_{\mathrm{dis}\tan \mathrm{ce}}}{\mathrm{\Σ}}{\left(\frac{{\mathrm{\δ}}_{i,j,k}-{\stackrel{\‾}{\mathrm{\δ}}}_{i,j}}{{\stackrel{\‾}{\mathrm{\δ}}}_{i,j}}\right)}^2)\end{array}......\left(6\right)$

In above formula, S is quantity normalized parameter,σ_viewIt is the set of the depth index of the picture k shooting, σ_angleIt is the Arabic numerals mark combination calculating surface used by angle, σ_distanceIt is the Arab calculating apart from surface used Numeral mark combines, and λ represents angle difference weight, λ ∈ [0,1],WithRepresent the measured value of angle and distance respectively.

The invention has the beneficial effects as follows：

The inventive method can lift the three-dimensional modeling precision based on Kinect depth camera, realizes the high accuracy of low cost Three-dimensional modeling.At present, high accuracy three-dimensional scanning device is relatively costly, and the usual precision of cheap three-dimensional scanning device is relatively Low.The present invention realizes high accuracy three-dimensional modeling using the relatively more cheap Kinect depth camera of price, greatly reduces The cost of high-precision three-dimensional scanning, is conducive to high-precision three-dimensional scanning technique in 3 D-printing, contactless dimensional measurement etc. The popularization of more applications.

Brief description

Fig. 1 is disposing way and the surface markers of high accuracy cuboid.

Fig. 2 is the RGB image of four kinds of high-precision calibrating squares and corresponding depth image.

Fig. 3 is to shoot for the depth image demarcated.

Fig. 4 is the result that normal direction figure and user divide the plane demarcating object.

Fig. 5 is the disposing way of a high accuracy cuboid and the surface markers having neither part nor lot in demarcation.

Specific embodiment

The invention will be further described with reference to the accompanying drawings and examples.

The calibration process of the present invention is based on by the angle projecting between calculated space plane and distance.Method is first Combining camera internal reference will form space point set under depth data backing up to camera local coordinate system, then by the space of point set matching Plane carrys out constitution optimization target equation, and obtains camera internal reference optimal value using nonlinear iteration optimization method.

Embodiments of the invention are as follows：

1) four high accuracy cuboid composition cuboid groups of known dimensions are put on the ground, as shown in figure 1, arrangement side Formula referring to accompanying drawing 1, this respective outward appearance of four high accuracy cuboids referring to the RGB image in accompanying drawing 2 and corresponding depth image, And shot in 10 different angles with Kinect depth camera；

The flatness of four high accuracy cuboids reaches 0.1mm, and Kinect can be reflected in the surface of high accuracy cuboid The infrared light that depth camera is launched.

Above-mentioned steps 1) in the arrangement on the ground of four high accuracy cuboids pendulum enable the bat of Kinect depth camera Take the photograph the public vertex face group being made up of three adjacent surfaces with public vertex and by putting down that two faces being parallel to each other form Row face group, public vertex face group has 3 groups, and parallel surface group has 12 groups.

Above-mentioned steps 1) in Kinect depth camera with respect to cuboid group 10 different angle shots of central rotation, The angle of rotation is about 15 degree in the horizontal direction every time, at least 9 degree.

Above-mentioned steps 1) in, in each angle shot, set 3 different Kinect depth cameras in cuboid group The distance between heart scope, 0.75m～1.15m, 0.9m～1.35m and 1.15m～1.55m, and from each distance range with Machine ground is chosen shooting distance and is shot, and shoots all depth images obtaining referring to Fig. 3.

2) according to step 1) the middle multiframe depth data shooting in the image obtaining, generated back by perspective projection inverse process Throw in the three dimensions to camera local coordinate system, generate the normal direction figure of same number of frames；

2.1) to each pixel in each depth image I, take using centered on it window as method of counting to Window W, window width is 7 pixels；

2.2) according to by production firm's initial internal reference empirical value setting, each depth point q ' in window is passed through formula 1st, in the three dimensions to camera local coordinate system for the perspective projection inverse process backing up of 2 and 3 expressions, obtain comprising 49 points Three-dimensional point set Q={ q_i| i=1,2 ..., r², and q=(x, y, z), wherein each depth point q ' be expressed as q '=(u, v, d (u, V)), (u, v) represents coordinate in depth image for the pixel, and d (u, v) represents depth in depth map for the pixel (u, v) Value：

$x=\frac{z(u-c_x)}{f_x}......\left(1\right)$

$y=\frac{z(v-c_y)}{f_y}......\left(2\right)$

$z=\frac{1}{{\mathrm{\γ}}_1(d(u,v)+D(u,v)e^{{\mathrm{\α}}_0-{\mathrm{\α}}_{1}d(u,v)})+{\mathrm{\γ}}_0}......\left(3\right)$

Wherein, (f_x, f_y, c_x, c_y, γ₀, γ₁, α₀, α₁, D) be Kinect depth camera internal reference, its value is according to opinion Civilian Smisek J, Jancosek M, Pajdla T.3D with Kinect [C] .IEEE, the Kinect depth that 2011. are proposed Camera internal reference scaling method obtains.

2.3) according to three-dimensional point set Q, obtain a plane with least square fitting method, its normal vector is three-dimensional vector n= (a, b, c), translational movement is m；

2.4) by three components a, b, c of three-dimensional vector n respectively according to formula $G=125.5(b/\sqrt{a^2+b^2+c^2}+1)$ With $B=125.5(c/\sqrt{a^2+b^2+c^2}+1)$ It is mapped to the red, green, blue in rgb space In three values, obtain the RGB color value of window W center pixel, simultaneously as the depth value of some pixels in depth image I For invalid 2047, corresponding for these inactive pixels rgb value is set to (0,0,0)；

2.5) to each pixel repeat step 2.1 in depth image I)～2.4), you can for each pixel life in I Become one group of RGB color value, and then obtain a coloured image as the normal direction figure N of this depth image I, normal direction figure is referring to Fig. 4.

3) and divide, on normal direction figure, the plane demarcating object, obtain plane depth data, normal direction figure divides and demarcates The result of the plane of object is referring to Fig. 4；

3.1) in cuboid combination, select 3 groups of public vertex face groups and 5 groups of parallel surface groups, wherein all faces are used Ah Arabic numbers is marked, and ready-made mark can be constituted referring to Fig. 1, then any two face in every group of public vertex face group One group of face group being used for angle calculation, the set that these are used for the mark combination of the face group of angle calculation can be expressed as σ_angle= (1,2), (1,3), (2,3), (4,5), (4,6), (5,6), (7,8), (7,9), (8,9) }, each parallel surface group constitutes one Organize the face group calculating for distance, the set that these are used for the mark combination of the face group that distance calculates can be expressed as σ_distance= { (1,10), (4,10), (5,2), (7,10), (9,3) }；

3.2) in step 2) in generate each normal direction figure on, on each marked face draw one be used for representing The quadrangle of effective depth pixel region profile on this face, this quadrangle, in the inside of cuboid, does not cover black picture element, in method The reason draw quadrangle on figure, rather than draw on photo or depth map is can be easier on normal direction figure to distinguish Effectively pixel；

3.3) polygon is mapped back each self-corresponding depth map of normal direction figure, obtain the corresponding plane depth of each polygon Data.

4) by step 3) in the plane depth data that obtains, by perspective projection inverse process backing up to camera local coordinate system Under three dimensions in, obtain corresponding three-dimensional point set O of each plane；Step 4) perspective projection inverse process and step 2) saturating Identical depending on projection inverse process.

5) to step 4) three-dimensional point set O that obtains passes through vcglib (the Italian National Research Council Institute, The Visualization and Computer Graphics Library) carry out a young waiter in a wineshop or an inn Take advantage of approximating method, obtain the corresponding plane of three-dimensional point set O；

6) according to step 5) angle between the plane that is labeled of the plane computations that obtain and plane and distance；

Above-mentioned steps 6) in angle between labeled plane and plane be specially with distance：Labeled public vertex The distance of the angle of each plane and two planes in labeled parallel surface group in the group of face.

Described step 6) in angle between labeled plane and plane be specially with distance：Labeled public vertex The distance of the angle of each plane and two planes in labeled parallel surface group in the group of face, angle and distance are respectively by following public affairs Formula 4 and 5 obtains：

${\mathrm{\θ}}_{i,j,k}=\arg \cos \left(\frac{n_{i,k}\·n_{j,k}}{\left|n_{i,k}\right|\left|n_{j,k}\right|}\right)......\left(4\right)$

${\mathrm{\δ}}_{i,j,k}=\frac{1}{2}(\frac{1}{\left|Q_{i,k}\right|}\underset{q\∈Q_{i,k}}{\mathrm{\Σ}}\frac{|P_{j,k}\·\hat{q}|}{\left|n_{j,k}\right|}+\frac{1}{\left|Q_{j,k}\right|}\underset{q\∈Q_{j,k}}{\mathrm{\Σ}}\frac{|P_{i,k}\·\hat{q}|}{\left|n_{i,k}\right|})......\left(5\right)$

Wherein, θ_{I, j, k}Refer to plane P_{I, k}And P_{J, k}Between angle, δ_{I, j, k}Refer to plane P_{I, k}And P_{J, k}The distance between, k It is the index of depth map, Q_{I, k}And Q_{J, k}It is any two of kth frame depth map by step 4) three-dimensional point set that obtains, q is one Three-dimensional point,It is the homogeneous coordinates of three-dimensional point q, n_{I, k}=(a_{I, k}, b_{I, k,}c_{I, k}) and n_{J, k}=(a_{J, k}, b_{J, k}, c_{J, k}) be respectively with Q_{I, k}And Q_{J, k}Corresponding plane normal vector, P_{I, k}=(a_{I, k}, b_{I, k}, c_{I, k},-m_{I, k}) and P_{J, k}=(a_{I, k}, b_{I, k}, c_{I, k},-m_{I, k}) respectively It is and Q_{I, k}And Q_{J, k}Corresponding plane parameter four dimensional vector, P_{I, k}And P_{J, k}Each self-corresponding plane general equation is a respectively_{I, k}x+ b_{I, k}y+c_{I, k}Z=m_{I, k}And a_{J, k}x+b_{J, k}y+c_{J, k}Z=m_{J, k}, and all pass through step 5) and Q_{I, k}And Q_{J, k}Matching obtains, a_{I, k}、b_{I, k}、 c_{I, k}With-m_{I, k}It is respectively and Q_{I, k}The first, second, third and fourth parameter of corresponding plane general equation, a_{J, k}、b_{J, k}、c_{J, k} With-m_{J, k}It is respectively and Q_{J, k}The first, second, third and fourth parameter of corresponding plane general equation.

7) angle and distance between the labeled plane of actual measurement high accuracy cuboid and plane, uses angulation tolerance Go out the angle between each two plane in above-mentioned labeled public vertex face group, then measured with slide measure above-mentioned labeled The distance between parallel surface group midplane, all of measured value is as shown in table 1：

Table 1 cuboid group and in for demarcate plan range and angle measured value

Again with step 6) angle that obtains compared with distance, optimization object function for the purpose of difference minimum for the construction, its Optimization object function is：

$\begin{array}{c}E=S\underset{k\∈{\mathrm{\σ}}_{\mathrm{view}}}{\mathrm{\Σ}}(\frac{\mathrm{\λ}}{{\mathrm{\Σ}}_{(i,j)\∈{\mathrm{\σ}}_{\mathrm{angle}}}1}\underset{(i,j)\∈{\mathrm{\σ}}_{\mathrm{angle}}}{\mathrm{\Σ}}{\left(\frac{{\mathrm{\θ}}_{i,j,k}-{\stackrel{\‾}{\mathrm{\θ}}}_{i,j}}{{\stackrel{\‾}{\mathrm{\θ}}}_{i,j}}\right)}^2\\ +\frac{1-\mathrm{\λ}}{{\mathrm{\Σ}}_{(i,j)\∈{\mathrm{\σ}}_{\mathrm{dis}\tan \mathrm{ce}}}1}\underset{(i,j)\∈{\mathrm{\σ}}_{\mathrm{dis}\tan \mathrm{ce}}}{\mathrm{\Σ}}{\left(\frac{{\mathrm{\δ}}_{i,j,k}-{\stackrel{\‾}{\mathrm{\δ}}}_{i,j}}{{\stackrel{\‾}{\mathrm{\δ}}}_{i,j}}\right)}^2)\end{array}......\left(6\right)$

In above formula, quantity normalized parameterThe set σ of the depth index of the picture k shooting_view=[1, 30], σ_angleIt is the Arabic numerals mark combination calculating surface used by angle, σ_distanceBe calculate apart from surface used Ah Arabic numbers mark combination, λ represents angle difference weight, λ ∈ [0,1],WithRepresent the measurement of angle and distance respectively Value.

This function is with depth camera internal reference for nonlinearity in parameters function, is used non-linear by this optimization object function Nonlinear iteration optimization method in iterative optimization method (above-mentioned steps 7) is levermar method, i.e. Lourakis M.levmar：Levenberg-Marquardt nonlinear least squares algorithms in C/C++[J] .web page]http：//www.ics.forth.gr/～lourakis/levmar.2004.), repeat step 4) to step Rapid 6), are optimized to the internal reference of Kinect depth camera so that the minimization of object function, complete to Kinect depth camera Internal reference is demarcated.

8) finally evaluate embodiment using the new high accuracy cuboid having neither part nor lot in calibration process of to finally give Kinect depth camera internal reference stated accuracy.

8.1) by the new high accuracy cuboid having neither part nor lot in calibration process of 1 as another high accuracy cuboid relatively On big face, the flatness on each surface of this two high accuracy cuboids all reaches 0.1mm, as shown in figure 5, claiming this two Cuboid is to evaluate cuboid group.

8.2) with being labeled as the distance between 11 and 14 face in vernier caliper measurement Fig. 5, the distance measuring is 349.25mm, measures the angle between three groups of faces in Fig. 5 with protractor, and the mark in this three groups of faces is respectively (11,12), (11, 13) and (12,13), measured angle is respectively 89.99 °, 90.02 ° and 90.02 °.

8.3) evaluation cuboid group is shot in 10 different angles with Kinect depth camera.

8.4) to step 8.3) in shoot the depth data that obtains, using with step 2) and 3) identical method, obtain Fig. 5 In be labeled as 11,12,13 and 14 face plane depth data.

8.5) by step 4) in the internal reference that used of perspective projection inverse process replace with step 7) internal reference that obtains, and will This perspective projection inverse process is applied to step 8.4) the plane depth data that obtains, obtain being labeled as 11,12,13 and 14 in Fig. 5 Corresponding to face 4 three-dimensional point sets.

8.6) respectively by step 5) used in least-square fitting approach be applied to step 8.5) obtained by 4 three Dimension point set, obtains this 4 plane equation corresponding to 4 three-dimensional point sets.

8.7) utilize step 6) in plane and plane between angle computational methods, calculation procedure 8.6) obtain 4 In individual plane, mark combination is respectively (11,12), and angle between 3 set of planes of (11,13) and (12,13) is calculated Angle and step 8.2) in the average of the absolute value of difference between the angle that obtains of measurement be 0.307 °, using step 6) in made The computational methods of the distance between plane and plane, calculation procedure 8.6) in 4 planes obtaining, mark is respectively 11 Hes The distance between 14 two planes, calculated distance and step 8.3) in difference between the distance that obtains of measurement absolute The average of value is 1.331mm it can be seen that the non-cpntact measurement precision using the Kinect depth camera of this method calibration is very High, there is significant technique effect.

Above-mentioned specific embodiment is used for illustrating the present invention, rather than limits the invention, the present invention's In spirit and scope of the claims, any modifications and changes that the present invention is made, both fall within the protection model of the present invention Enclose.

Claims (9)

1. a kind of internal reference scaling method of the Kinect depth camera based on cuboid is it is characterised in that comprise the steps of：

1) at least three high accuracy cuboids composition cuboid groups of known dimensions are put on the ground, and with Kinect depth phase Machine is shot in multiple different angles；The flatness on described each surface of high accuracy cuboid reaches 0.1mm, and high accuracy is long The infrared light that Kinect depth camera is launched can be reflected in the surface of cube；

2) according to step 1) the middle multiframe depth data shooting in the depth image obtaining, generated back by perspective projection inverse process Throw in the three dimensions to camera local coordinate system, generate the normal direction figure of same number of frames；

3) and divide, on normal direction figure, the plane demarcating object, obtain plane depth data；

4) by step 3) in the plane depth data that obtains, by under perspective projection inverse process backing up to camera local coordinate system In three dimensions, obtain corresponding three-dimensional point set Q of each plane；

5) to step 4) each three-dimensional point set Q of obtaining carries out least-square fitting approach, obtains three-dimensional point set Q corresponding flat Face；

6) according to step 5) angle between the plane that is labeled of the plane computations that obtain and plane and distance；

7) angle and distance between the labeled plane of actual measurement high accuracy cuboid and plane, and with step 6) obtain Angle compare with distance, construction by difference minimum for the purpose of optimization object function, by this optimization object function use non- Linear iteraction optimization method, repeats step 4) arrive step 6), the internal reference of Kinect depth camera is optimized so that mesh Scalar functions minimize, and complete the internal reference to Kinect depth camera and demarcate；

Described step 1) at least three high accuracy cuboids pendulum arrangements on the ground enable the shooting of Kinect depth camera Parallel be made up of two faces being parallel to each other to the public vertex face group being made up of three adjacent surfaces with public vertex Face group, at least one group of public vertex face group, at least two groups of parallel surface group.

2. the internal reference scaling method of a kind of Kinect depth camera based on cuboid according to claim 1, its feature It is：Described step 1) in the different angle shot multiple with respect to the central rotation of cuboid group of Kinect depth camera, often At least 9 degree of the angle of secondary rotation.

3. the internal reference scaling method of a kind of Kinect depth camera based on cuboid according to claim 1, its feature It is：Described step 1) in, in each angle shot, set multiple different Kinect depth cameras to cuboid group center The distance between scope, and randomly choose a shooting distance from each distance range and shot.

4. the internal reference scaling method of a kind of Kinect depth camera based on cuboid according to claim 1, its feature It is：Described step 2) in the detailed process of the depth image that obtained by the shooting normal direction figure that generates same number of frames be：

2.1) to each pixel in each depth image I, take using centered on it window as method of counting to window W, window width is r；

2.2) each depth point q ' in window is passed through by perspective according to the internal reference empirical value being set by production firm

In the projection three dimensions to camera local coordinate system for the inverse process backing up, obtain comprising r²Three-dimensional point set Q=of individual point {q_i| i=1,2 ..., r², and q=(x, y, z), wherein each depth point are expressed as q '=(u, v, d (u, v)), (u, v) represents One pixel coordinate in depth image, d (u, v) represents depth value in depth map for the pixel (u, v)；

2.3) according to three-dimensional point set Q, obtain a plane with least square fitting method, its normal vector be three-dimensional vector n=(a, b, C), translational movement is m；

2.4) three components a, b, c of three-dimensional vector n are respectively adopted below equation and are mapped to the red, green, blue three in rgb space In individual value, obtain the RGB color value of window W center pixel：

$R=125.5(a/\sqrt{a^2+b^2+c^2}+1)$

$G=125.5(b/\sqrt{a^2+b^2+c^2}+1)$

$B=125.5(c/\sqrt{a^2+b^2+c^2}+1)$

2.5) to each pixel repeat step 2.1 in depth image I)～2.4), you can generate one for each pixel in I Group RGB color value, and then obtain a coloured image as the normal direction figure N of this depth image I.

5. the internal reference scaling method of a kind of Kinect depth camera based on cuboid according to claim 1, its feature It is：Described step 3) on normal direction figure divide demarcate object plane be specially：

3.1) in cuboid combination, least one set public vertex face group and at least two groups parallel surface groups are selected, to wherein all Face is marked with Arabic numerals；

3.2) in step 2) in generate each normal direction figure on, on each marked face draw one be used for representing this face The polygon of upper effective depth pixel region profile；

3.3) polygon is mapped back each self-corresponding depth map of normal direction figure, obtain each polygon corresponding plane depth number According to.

6. the internal reference scaling method of a kind of Kinect depth camera based on cuboid according to claim 1, its feature It is：Described step 2) in by perspective projection inverse process and step 4) in identical by perspective projection inverse process, specifically Process is：According to by the internal reference empirical value initially setting, each depth point q ' in window is passed through below equation backing up to camera In three dimensions under local coordinate system, obtain comprising r²Three-dimensional point set Q={ the q of individual point_i| i=1,2 ..., r²}：

$x=\frac{z\left(u-c_x\right)}{f_x}......\left(1\right)$

$y=\frac{z(v-c_y)}{f_y}......\left(2\right)$

$z=\frac{1}{{\mathrm{\γ}}_1\left(d\left(u,v\right)+D\left(u,v\right)e^{{\mathrm{\α}}_0-{\mathrm{\α}}_{1}d\left(u,v\right)}\right)+{\mathrm{\γ}}_0}......\left(3\right)$

Wherein, D is two-dimensional matrix, the line number in two-dimensional matrix D and the columns pixel with the vertical and horizontal of depth image I respectively Quantity is identical, and D (u, v) is one of D element value, f_x,f_yRepresent Kinect depth camera respectively in x-axis and y-axis direction Two focal lengths, c_x,c_yRepresent horizontal stroke on imaging surface for the photocentre, the ordinate of Kinect depth camera, γ respectively₀,γ₁It is respectively First, second depth conversion parameter of Kinect depth camera, α₀,α₁, D is first, second He of Kinect depth camera respectively 3rd distortion correction parameter.

7. the internal reference scaling method of a kind of Kinect depth camera based on cuboid according to claim 1, its feature It is：Described step 6) in angle between labeled plane and plane be specially with distance：Labeled public vertex face The distance of the angle of each plane and two planes in labeled parallel surface group in group, angle and distance are respectively by following two formulas Obtain：

${\mathrm{\θ}}_{i,j,k}=\arg \cos \left(\frac{n_{i,k}\·n_{j,k}}{\left|n_{i,k}\right|\left|n_{j,k}\right|}\right)......\left(4\right)$

${\mathrm{\δ}}_{i,j,k}=\frac{1}{2}(\frac{1}{\left|Q_{i,k}\right|}\underset{q\∈Q_{i,k}}{\Σ}\frac{|P_{j,k}\·\hat{q}|}{\left|n_{j,k}\right|}+\frac{1}{\left|Q_{j,k}\right|}\underset{q\∈Q_{j,k}}{\Σ}\frac{|P_{i,k}\·\hat{q}|}{\left|n_{i,k}\right|})......\left(5\right)$

Wherein, θ_{I, j, k}Refer to plane P_{I, k}And P_{J, k}Between angle, δ_{I, j, k}Refer to plane P_{I, k}And P_{J, k}The distance between, k is deep The index of degree figure, Q_{I, k}And Q_{J, k}It is any two three-dimensional point set of kth frame depth map, q is a three-dimensional point,It is three-dimensional point q Homogeneous coordinates, n_{I, k}=(a_{I, k}, b_{I, k}, c_{I, k}) and n_{J, k}=(a_{J, k}, b_{J, k}, c_{J, k}) be respectively and Q_{I, k}And Q_{J, k}Corresponding planar process Vector, P_{I, k}=(a_{I, k}, b_{I, k}, c_{I, k},-m_{I, k}) and P_{J, k}=(a_{I, k}, b_{I, k}, c_{I, k},-m_{I, k}) be respectively and Q_{I, k}And Q_{J, k}Corresponding flat Face parameter four dimensional vector, P_{I, k}And P_{J, k}Each self-corresponding plane general equation is a respectively_{I, k}x+b_{I, k}y+c_{I, k}Z=m_{I, k}And a_{J, k}x+ b_{J, k}y+c_{J, k}Z=m_{J, k}, a_{I, k}、b_{I, k}、c_{I, k}With-m_{I, k}It is respectively and Q_{I, k}The first, second, third of corresponding plane general equation With the 4th parameter, a_{I, k}、b_{J, k}、c_{J, k}With-m_{J, k}It is respectively and Q_{J, k}The first, second, third of corresponding plane general equation and Four parameters.

8. the internal reference scaling method of a kind of Kinect depth camera based on cuboid according to claim 7, its feature It is：Described step 7) in angle and distance between the labeled plane of actual measurement high accuracy cuboid and plane pass through In the following manner measures：Go out the angle between each plane in above-mentioned labeled public vertex face group with angulation tolerance, use vernier Slide calliper rule measure the distance between above-mentioned labeled parallel surface group midplane.

9. the internal reference scaling method of a kind of Kinect depth camera based on cuboid according to claim 1, its feature It is：Described step 7) in object function be：

$\begin{array}{c}E=S\underset{k\∈{\mathrm{\σ}}_{view}}{\mathrm{\Σ}}(\frac{\mathrm{\λ}}{{\mathrm{\Σ}}_{\left(i,j\right)\∈{\mathrm{\σ}}_{angle}}1}\underset{\left(i,j\right)\∈{\mathrm{\σ}}_{angle}}{\mathrm{\Σ}}{\left(\frac{{\mathrm{\θ}}_{i,j,k}-{\stackrel{\‾}{\mathrm{\θ}}}_{i,j}}{{\stackrel{\‾}{\mathrm{\θ}}}_{i,j}}\right)}^2\\ +\frac{1-\mathrm{\λ}}{{\mathrm{\Σ}}_{\left(i,j\right)\∈{\mathrm{\σ}}_{dis\tan ce}}1}\underset{\left(i,j\right)\∈{\mathrm{\σ}}_{dis\tan ce}}{\mathrm{\Σ}}{\left(\frac{{\mathrm{\δ}}_{i,j,k}-{\stackrel{\‾}{\mathrm{\δ}}}_{i,j}}{{\stackrel{\‾}{\mathrm{\δ}}}_{i,j}}\right)}^2)\end{array}......\left(6\right)$

In above formula, S is quantity normalized parameter,σ_viewIt is the set of the depth index of the picture k shooting, σ_angle It is the Arabic numerals mark combination calculating surface used by angle, σ_distanceIt is the Arabic numerals mark calculating apart from surface used Note combination, λ represents angle difference weight, λ ∈ [0,1],WithRepresent the measured value of angle and distance respectively.