CN103278138A - Method for measuring three-dimensional position and posture of thin component with complex structure - Google Patents

Abstract

The invention discloses a method for measuring three-dimensional position and posture of a thin component with complex structure. The method comprises the following steps of: establishing a mapping relation between a three-dimensional world coordinate system and a two-dimensional image coordinate in a binocular vision measurement system; extracting the features on a reference posture thin component binocular image, establishing a prior feature library, extracting the features on a to-be-measured posture thin component binocular image, matching the extracted features with the prior feature library so as to obtain the same amount of feature matched pair sets corresponding to the prior feature library; obtaining a three-dimensional point cloud of feature points; calculating a main plane of the reference posture thin component three-dimensional point cloud, calculating the main plane of three-dimensional point cloud of the to-be-measured thin component and calculating the spatial position relation with the main plane of the reference posture thin component so as to obtain the three-dimensional position and posture of the to-be-measured thin component. The measurement method provided by the invention is beneficial for accurate positioning for the thin component in the technological processes of automatic manufacturing, assembly, welding, and the like of an industrial mechanical arm. According to the method, the implementation is simple and easy, the positioning accuracy is high, the cost is low, and the operation is convenient.

Description

The measuring method of the thin parts three-dimensional position of a kind of labyrinth and attitude

Technical field

The invention belongs to the vision measurement field in the infotech, relate in particular to the vision measuring method of the thin parts three-dimensional position of a kind of complexity and attitude.

Background technology

Can realize three-dimensional dimension measurement, location and the line Quality Control of parts on the industrial production line based on the measuring method of binocular vision, be an important research field in the current information technology, and be applied at aspects such as the measurement of common small size parts and location.And for large scale and have three-dimensional position and the attitude measurement of the thin parts (as the both sides gusset of automobile body-in-white and car door etc.) of labyrinth, yet there are no it and clearly use.

Summary of the invention

For addressing the above problem, the invention provides the measuring method of a kind of thin parts three-dimensional position to be measured and attitude, it comprises:

Step 1: demarcate the spatial relation of inner parameter, external parameter and the binocular camera of binocular camera, set up the mapping relations of three-dimensional world coordinate system and two dimensional image coordinate in the binocular vision measuring system;

Step 2: use the SIFT unique point in the thin parts binocular of the described benchmark of the SIFT operator extraction image right, generate SIFT priori features storehouse, and use described SIFT unique point to the three-dimensional planar at the place reference plane as the thin parts of benchmark place;

Step 3: the binocular solid correcting image for thin parts to be measured is right, it is right to extract a plurality of SIFT unique points respectively, and with the SIFT unique point in the SIFT unique point extracted pair and the described SIFT priori features storehouse to mating, obtain with described SIFT feature priori storehouse in the SIFT unique point to equal number and corresponding SIFT unique point pair set;

Step 4: the image coordinate according to unique point in the resulting SIFT unique point pair set obtains the three-dimensional point cloud of described thin parts to be measured, and then obtains the space plane of these thin parts to be measured;

Step 5: by finding the solution the transformation matrix between described space plane and the reference plane, determine three-dimensional position and the attitude information of described thin parts to be measured.

The present invention adopts the binocular vision measuring technique: (1) is according to application scenarios and selected suitable camera and the camera lens of field range, build binocular vision system, and respectively left and right two cameras are demarcated respectively, obtain inner parameter and the lens distortion coefficient of left and right camera, according to the world coordinate system on the reference plane, obtain the external parameter of left and right camera, carry out binocular solid again and demarcate, further obtain the spatial relation between the left and right camera; (2) the binocular image to gathering, successively carrying out lens distortion calibration and three-dimensional correction handles, obtain binocular distortion correction image respectively to right with the binocular solid correcting image, binocular solid correcting image at the thin parts of benchmark pose is right, extract its SIFT feature, choose thin parts specific location (as pilot hole by hand, positions such as pilot pin) SIFT feature, obtain its binocular characteristic matching pair set, with it as SIFT priori features storehouse, binocular solid correcting image at thin parts to be measured is right, extract its SIFT feature, and carry out characteristic matching with the priori features storehouse, obtain equal number and corresponding characteristic matching pair set with it; (3) obtain the three-dimensional coordinate of characteristic matching pair set by the binocular mensuration, obtain the three-dimensional point cloud of unique point; (4) according to least square method, calculate the principal plane of the thin parts three-dimensional point cloud of benchmark pose and thin parts three-dimensional point cloud to be measured respectively, by calculating the spatial relation between two principal planes, obtain three-dimensional position and the attitude of thin parts to be measured.Complicated thin parts three-dimensional position and attitude measurement method that the present invention proposes help in the technological processs such as the industrial machine hand is made automatically, assembled, welding the accurate location to thin parts, method bearing accuracy height of the present invention, with low cost, be convenient to operation.

Description of drawings

Fig. 1 is the measuring method general diagram of complicated thin parts three-dimensional position and attitude among the present invention;

Fig. 2 is binocular camera imaging model and physics imaging and normalization imaging coordinate system among the present invention;

Fig. 3 is the three-dimensional synoptic diagram of demarcating of binocular camera among the present invention;

Fig. 4 is characteristic matching process flow diagram among the present invention;

Fig. 5 is the synoptic diagram that obtains that lacks the SIFT feature among the present invention;

Fig. 6 is unique point three-dimensional point cloud calculation flow chart among the present invention.

Embodiment

For making the purpose, technical solutions and advantages of the present invention clearer, below in conjunction with specific embodiment, and with reference to accompanying drawing, the present invention is described in more detail.

The invention discloses the vision measuring method of the thin parts three-dimensional position of a kind of complexity and attitude.

Fig. 1 shows the vision measuring method flow process general diagram of complicated thin parts three-dimensional position and attitude among the present invention.As shown in Figure 1, this method is divided into benchmark pose measurement and pose to be measured two stages of estimation.Wherein, the benchmark pose measurement stage comprises: camera calibration, binocular picture synchronization collection, binocular image to distortion correction and three-dimensional correction, binocular image to SIFT feature extraction, priori features collection obtain, the thin parts three-dimensional point cloud of benchmark pose obtains, the steps such as three-dimensional pose estimation of the thin parts of benchmark pose; Pose estimation stages to be measured comprises: binocular picture synchronization collection, binocular image to distortion correction and three-dimensional correction, binocular image to the feature set of SIFT feature extraction, the thin parts of pose to be measured obtain, the thin parts three-dimensional point cloud of pose to be measured obtains, the pose estimation of the thin parts of pose to be measured etc.

Because the said reference pose measurement is identical with the implementation of corresponding steps in the pose estimation to be measured, therefore below its summary being summarized as four big steps launches to describe, be that camera calibration, feature extraction and thin parts characterize, binocular vision is measured and three-dimensional point cloud obtains and thin parts three-dimensional position and attitude estimation, and in conjunction with relevant drawings each step of the present invention be described in further detail.

The vision measuring method of the thin parts three-dimensional position of complexity disclosed by the invention and attitude specifically comprises:

Step 1: camera calibration.Namely demarcate the spatial relation between inner parameter, lens distortion coefficient and the binocular camera of camera, to set up the mapping relations of reference plane three-dimensional world coordinate system and two dimensional image coordinate in the binocular vision measuring system.

Setting whole working range is 5 meters * 5 meters, at this visual field, selects the moderate camera lens in visual field for use.Take all factors into consideration these factors, preferably choose the camera lens that focal length is 6mm (its visual angle is about 53 degree).The binocular camera horizontal positioned, spacing is set at 60cm, is the benchmark camera with left side camera wherein.

Camera calibration relates generally to four coordinate systems, i.e. three-dimensional world coordinate system X _wY _wZ _w(initial point is on reference plane, and Z _wAxle is perpendicular to reference plane), three-dimensional camera coordinates is X _cY _cZ _c(initial point is positioned at the camera lens photocentre, and Z _cAxle and optical axis coincidence), the two dimensional image physical coordinates is xO ₁Y (initial point is positioned at picture centre, and coordinate is physical coordinates), two dimensional image pixel coordinate are uOv (initial point is positioned at the image upper left corner, and coordinate is pixel coordinate).

Accompanying drawing 2 shows each coordinate system and normalization coordinate system synoptic diagram thereof among the present invention.As shown in Figure 2, adopt linear video camera pin-hole model, the coordinate of definition space point P under the three-dimensional world coordinate system is [X _wY _wZ _w] ^T, its corresponding homogeneous coordinates are P=[X _wY _wZ _w1] ^TThe coordinate of defining point P under three-dimensional camera coordinates is is [X _cY _cZ _c] ^T, its homogeneous coordinates are P _c=[X _cY _cZ _c1] ^TThe subpoint of defining point P on two dimensional image plane is p ', and the coordinate under its two dimensional image physical coordinates system is [x y] ^T(unit: millimeter), the coordinate under its two dimensional image pixel coordinate system is [u v] ^T, its homogeneous coordinates are p '=[u v1] ^TThe intersection point O of the optical axis of camera and the plane of delineation ₁Pixel coordinate be [u ₀v ₀] ^TThe physical size of image as unit pixel on x axle and y direction of principal axis is respectively dx, dy.Then putting coordinate and its mapping relations the coordinate two dimensional image coordinate of subpoint p ' two dimensional image plane under between of P under three-dimensional system of coordinate is:

$Z_c\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=\left[\begin{array}{cccc}\frac{f}{\mathrm{dx}}& 0& u_0& 0\\ 0& \frac{f}{\mathrm{dy}}& v_0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{cc}R& t\\ O^t& 1\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]---\left(1\right)$

$=\left[\begin{array}{cccc}{a}_x& 0& u_0& 0\\ 0& a_y& v_0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{cc}R& t\\ O^t& 1\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]=M_1{M}_{2}P=\mathrm{MP}$

Wherein, P=[X _wY _wZ _w1] ^TBe the homogeneous coordinates under the three-dimensional world coordinate system; F is lens focus, a _x=f/dx, a _y=f/dy is the coke ratio of camera, and R, t are respectively rotation matrix and the translation vector between three-dimensional camera coordinates system and the three-dimensional world coordinate system; O ^t=[0 0 0], M ₁, M ₂Be respectively inner parameter matrix and the external parameter matrix (being homography matrix) of camera, M is the overall projection matrix of camera.

In the whole calibrating procedure, need to demarcate inner parameter [the f dx dy u of camera ₀v ₀] ^T, camera lens distortion factor [k ₁k ₂p ₁p ₂k ₃] ^TAnd external parameter, described inner parameter comprises coke ratio, the center position of camera, external parameter comprises rotation matrix R and the translation vector t between three-dimensional camera coordinates system and the three-dimensional world coordinate system.Use chequered with black and white square mesh scaling board to demarcate in calibration process, wherein, using little scaling board, its sizing grid when demarcating inner parameter is 50mm * 50mm, and meshes number is 8 (wide) * 12 (length); Using big scaling board, its sizing grid when demarcating external parameter is 100mm * 100mm, and meshes number is 4 (wide) * 6 (length).The Harris Corner Detection Algorithm is all adopted in the detection of each angle point on the scaling board, and obtains the sub-pixel corner location.

Step 1 specifically may further comprise the steps:

Step 11: left and right camera is demarcated respectively, obtained inner parameter and the external parameter matrix of each camera respectively.

(a) at first obtain the initial internal parameter of left and right camera and camera with respect to rotation matrix and the translation vector of little scaling board.

Demarcation with the benchmark camera is example, for obtaining the inner parameter matrix of camera, at first do not consider lens distortion, employing is based on the scaling method of plane homography matrix, hand-held little scaling board, by the attitude (at least 3 attitudes) of the little scaling board of continuous variation, utilize plane coupling (direct linear transformation's method) between different points of view to calculate the initial internal parameter of camera and camera with respect to rotation matrix and the translation vector of little scaling board; Camera inner parameter when namely trying to achieve zero lens distortion and camera are with respect to rotation matrix R and the translation vector t of little scaling board.Make little scaling board place plane Z _w=0, then:

$Z_c\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=\left[\begin{array}{cccc}{a}_x& 0& u_0& 0\\ 0& a_y& v_0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{cc}R& t\\ O^t& 1\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ 0\\ 1\end{array}\right]=\left[\begin{array}{ccc}{a}_x& 0& u_0\\ 0& a_y& v_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{cc}R& t\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ 0\\ 1\end{array}\right]\left[\mathrm{0,1}\right]$

$=\left[\begin{array}{ccc}{a}_x& 0& u_0\\ 0& a_y& v_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}{r}_1& r_2& r_3& t\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ 0\\ 1\end{array}\right]=\left[\begin{array}{ccc}{a}_x& 0& u_0\\ 0& a_y& v_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{ccc}{r}_1& r_2& t\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ 1\end{array}\right]---\left(2\right)$

$=H\·\left[\begin{array}{c}{X}_w\\ Y_w\\ 1\end{array}\right]$

Wherein, R=[r ₁r ₂r ₃].

Therefore, can finish the demarcation of benchmark camera by finding the solution the H matrix, the inner parameter of acquisition camera itself and camera are with respect to rotation matrix R and the translation vector t of little scaling board.

(b) based on above camera initial internal parameter of trying to achieve and camera rotation matrix R and the translation vector t with respect to little scaling board, consider lens distortion, further ask for the distortion factor of camera lens, the inner parameter of the one-step optimization camera of going forward side by side.

Making the image pixel coordinate of detected a certain angle point in the original lens distortion image is [u _Rawv _Raw] ^T, under the desirable pin-hole imaging model in the no lens distortion image image pixel coordinate of this angle point be [u _UnD v _Und] ^T, wherein, original lens distortion image is the acquired original image that has lens distortion, then:

(1) with the three-dimensional world coordinate transform of each angle point on the little scaling board in three-dimensional camera coordinates system, wherein, be initial point with an angle point on the little scaling board, set up the three-dimensional world coordinate system, and then obtain the three-dimensional world coordinate of each angle point on the scaling board according to the length computation of the chequered with black and white grid on the little scaling board.Namely

$\left[\begin{array}{c}{X}_c\\ Y_c\\ Z_c\end{array}\right]=R\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]+t---\left(3\right)$

Wherein, R and t are that camera is with respect to rotation matrix and the translation vector of little scaling board.

(2) further project to the plane of delineation, obtain angle point at the orthoscopic image physical coordinates [x of plane of delineation coordinate system _Undy _Und] ^TWith image pixel coordinate [u _Undv _Und] ^T, namely

$\left[\begin{array}{c}{x}_{\mathrm{und}}\\ y_{\mathrm{und}}\end{array}\right]=\left[\begin{array}{c}{\mathrm{fX}}_c/Z_c\\ f{Y}_c/Z_c\end{array}\right]---\left(4\right)$

$\left[\begin{array}{c}{u}_{\mathrm{und}}\\ v_{\mathrm{und}}\end{array}\right]\left[\begin{array}{ccc}{a}_x& 0& u_0\\ 0& a_y& v_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{c}{x}_{\mathrm{und}}\\ y_{\mathrm{und}}\end{array}\right]---\left(5\right)$

(3) will be in original lens distortion image the image pixel coordinate [u of detected angle point _Rawv _Raw] ^TBe transformed to image physical coordinates [x _Rawy _Raw] ^T, and introduce the lens distortion coefficient [k of a certain initial setting ₁k ₂p ₁p ₂k ₃] ^T, obtain behind the distortion correction angle point image pixel coordinate [u ' _UndV ' _Und] ^TWith the image physical coordinates [x ' _UndY ' _Und] ^T, namely

$\left[\begin{array}{c}{x}_{\mathrm{raw}}\\ y_{\mathrm{raw}}\\ 1\end{array}\right]=\left[\begin{array}{ccc}\mathrm{dx}& 0& -u_0\mathrm{dx}\\ 0& \mathrm{dy}& -v_0\mathrm{dy}\\ 0& 0& 1\end{array}\right]\left[\begin{array}{c}{u}_{\mathrm{raw}}\\ v_{\mathrm{raw}}\\ 1\end{array}\right]---\left(6\right)$

$\left[\begin{array}{c}{x^{\′}}_{\mathrm{und}}\\ {y^{\′}}_{\mathrm{und}}\end{array}\right]=(1+k_1{r}^2+k_2{r}^4+k_3{r}^6)\left[\begin{array}{c}{x}_{\mathrm{raw}}\\ y_{\mathrm{raw}}\end{array}\right]+\left[\begin{array}{c}2p_1{x}_d{y}_d+p_2(r^2+2x_d^2)\\ p_1(r^2+2y_d^2)+2p_2{x}_d{y}_d\end{array}\right]---\left(7\right)$

$\left[\begin{array}{c}{u^{\′}}_{\mathrm{und}}\\ {v^{\′}}_{\mathrm{und}}\end{array}\right]=\left[\begin{array}{ccc}{a}_x& 0& u_0\\ 0& a_y& v_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{c}{x^{\′}}_{\mathrm{und}}\\ {y^{\′}}_{\mathrm{und}}\end{array}\right]---\left(8\right)$

Wherein,

As seen in the said process, set up the image pixel coordinate [u of detected angle point in the image pixel coordinate of angle point behind the distortion correction and the original lens distortion image according to camera inner parameter and distortion parameter _Rawv _Raw] ^TBetween linear equation.

(4) at N the angle point that uses in demarcating, the objective definition function:

$\min F=\underset{i=1}{\overset{N}{\mathrm{\Σ}}}[{(u_{\mathrm{und}}^i-u_{\mathrm{und}}^{\′i})}^2+{(v_{\mathrm{und}}^i-v_{\mathrm{und}}^{\′i})}^2]---\left(9\right)$

With this non-linear least square problem, by nonlinear optimization algorithm iteration repeatedly, try to achieve the parameter value that makes the objective function minimum, obtain camera inner parameter [the f dx dy u of global optimization ₀v ₀] ^TAnd distortion parameter [k ₁k ₂p ₁p ₂k ₃] ^T

(c) external parameter of demarcation camera, namely the spatial relation between the plane of delineation and the reference plane obtains camera plane with respect to rotation matrix and the translation vector of reference plane.

The present invention need measure the three-dimensional coordinate of some specific location unique point on the complicated thin parts, the external parameter matrix of camera when therefore needing acquisition to be world coordinate system with the reference plane.Big scaling board is placed on the reference plane, and namely reference plane is world coordinate system X in the set scene _WO _WY _WThe plane, place, its concrete steps are as follows:

(1) determines three-dimensional world coordinate system X _wY _wZ _w, big scaling board is placed on the reference plane, as coordinate origin, and make Z with an angle point of grid on the big scaling board _wAxle is perpendicular to big scaling board reference plane;

(2) gather the big scaling board image of this moment, and utilize the camera inner parameter and the lens distortion coefficient that obtain before to carry out distortion correction, the image after obtaining to proofread and correct;

(3) use the Harris Corner Detection Algorithm to detect the two dimensional image pixel coordinate of each angle point in the image behind distortion correction;

(4) according to the three-dimensional camera coordinates of detected each angle point on the following distortion correction image of three-dimensional world coordinate and three-dimensional camera coordinates system of each angle point on the big scaling board, ask for camera with respect to rotation matrix and the translation vector of big scaling board, i.e. the external parameter of camera.

Step 12: binocular solid is demarcated, and sets up the spatial relation between the binocular camera.Namely obtain right camera with respect to rotation matrix and the translation matrix of benchmark camera, i.e. homography matrix, concrete steps are as follows:

Fig. 3 shows the three-dimensional synoptic diagram of demarcating of binocular camera among the present invention.According to rotation matrix and the translation vector of left and right camera with respect to reference plane, find the solution big scaling board spatial relation between right camera and the benchmark camera when reference plane, namely obtain target object when the level ground right side camera with respect to rotation matrix and the translation vector of left camera.As shown in Figure 3, the spatial point P under the three-dimensional world coordinate system, (photocentre or initial point are respectively O in the three-dimensional camera coordinates system of left camera and right camera _ClAnd O _CR) under spatial point be respectively P _ClAnd P _CR, make the external parameter matrix of left camera and right camera be respectively [R _lt _l] and [R _rt _r], basis then

P _Cl＝R _lP+t _l

P _Cr＝R _rP+t _r

P _Cl＝R ^T(P _Cr-t)

Right camera with respect to the spatial relation [R t] of left camera is

$R=R_r{R}_l^T---\left(10\right)$

t＝t _r-Rt _l (11)

Step 13: the binocular image is carried out three-dimensional correction, realize coplanar, the row alignment of binocular image.Even two camera optical axises are parallel, to left and right image of camera plane re-projection, it is accurately fallen at grade, aim at the row of left and right image simultaneously.Based on the rotation matrix between the camera of the left and right sides and translation vector etc., adopt the overlapping maximization criterion of view and Bouguet calibration algorithm to realize three-dimensional correction, try to achieve the three-dimensional correction internal matrix M of left camera _{Rect_l}With row alignment matrix R _{Rect_l}The three-dimensional correction internal matrix M of right camera _{Red_r}With row alignment matrix R _{Rect_r}Thereby it is right to obtain the binocular solid correcting image.The three-dimensional correction specific operation process is:

Order point P is at the image coordinate P of binocular original image centering _{Raw_l}And P _{Raw_r}Image coordinate P in binocular distortion correction image pair _{Und_l}And P _{Und_l}Image coordinate P binocular solid correcting image centering _{Rect_l}And P _{Rect_r}, then the three-dimensional correction formula is:

P _{rect_l}＝M _{rect_l}R _l(M _1l) ^-1P _{und_l} (12)

P _{rect_r}＝M _{rect_r}R _r(M _1r) ^-1P _{und_r} (13)

Wherein, R _lAnd R _rBe respectively the rotation matrix of left and right camera external parameter.

P _{und_l}＝M _1l·UnDistort(M _1lP _{raw_l})

P _{und_r}＝M _1r·UnDistort(M _1rP _{raw_r})

In the formula,

UnDistort () is the distortion correction function of pixel.

Step 2: feature extraction and thin parts characterize.Thin parts binocular solid correcting image centering, use SIFT feature and neighborhood thereof to describe respectively and characterize thin image of component.

Fig. 4 shows the method flow diagram that feature extraction and thin parts characterize among the present invention.Shown in accompanying drawing 4, this method specifically comprises the steps:

Step 21: gather the binocular image of thin parts, and carry out lens distortion calibration and three-dimensional correction respectively, and then it is right to, binocular solid correcting image to obtain binocular distortion correction image;

Step 22: use the SIFT feature of SIFT operator extraction binocular solid correcting image centering, described SIFT feature comprises SIFT unique point and neighborhood description thereof;

Step 23: the priori features collection obtains and thin parts characterize.

In the benchmark pose measurement stage, at the binocular solid correcting image of thin parts to detected SIFT feature in the left image, choose near the SIFT feature of a plurality of (at least 3) ad-hoc location (as pilot hole, pilot pin etc.) by hand, and the SIFT feature in itself and the right image carried out characteristic matching, it is right that manual rejecting mistake is mated, obtain SIFT characteristic matching pair set, and then be used for forming the priori SIFT feature database of the thin parts of benchmark pose.

In pose estimation stages to be measured, to carry out characteristic matching with priori SIFT feature database respectively from the SIFT feature that the binocular solid correcting image centering of thin parts to be measured is extracted, and use the RANSAC algorithm to reject the mistake matching result, it is right with priori SIFT feature database equal number and corresponding SIFT characteristic matching to form.The purpose of its coupling is, if the SIFT feature logarithm of the binocular solid correcting image centering of thin parts to be measured is less than the characteristic number of priori SIFT feature database, and the SIFT feature corresponding with the priori features storehouse according to homography matrix then, the SIFT feature of disappearance is tried to achieve in conversion.

Fig. 5 shows the obtain manner synoptic diagram of disappearance SIFT feature among the present invention.As shown in Figure 5, it is right to 7 pairs of SIFT characteristic matching of last extraction to have provided priori SIFT feature database (by 8 pairs of SIFT characteristic matching to forming) and thin parts binocular solid correcting image to be measured among the figure, for the SIFT characteristic matching of completion disappearance is right, according to the homography matrix H between two left images ₁With a P ₆, calculate the SIFT unique point P of thin parts binocular solid correcting image to be measured to lacking in the left image ₆', finally obtain the SIFT characteristic matching pair set with priori SIFT feature database equal number and corresponding thin parts to be measured.

Step 3: binocular vision is measured and the thin parts three-dimensional point cloud of benchmark obtains.

Fig. 6 shows among the present invention that binocular vision is measured and thin parts three-dimensional point cloud obtains synoptic diagram.Among the present invention, calculate the three-dimensional world coordinate of specific location feature point set on the thin parts by the binocular measuring method, thereby obtain the three-dimensional point cloud of thin parts.

Three-dimensional coordinate with certain specific location point P on the thin parts of three dimensions is calculated as example, describes computation process, and wherein, P is right corresponding to certain SIFT characteristic matching in the imaging of binocular solid correcting image centering for point.The coordinate of some P under the three-dimensional camera coordinates of the left and right camera of binocular is is respectively [X _ClY _ClZ _Cl] ^T[X _CrY _CrZ _Cr] ^TThe right image coordinate of this SIFT characteristic matching of binocular solid correcting image centering is respectively [u _{Rect_l}v _{Rect_l}] ^T[u _{Rect_r}v _{Rect_r}] ^T, its homogeneous coordinates are respectively P _{Rect_l}=[u _{Rect_l}v _Rect_ _l1] ^TAnd P _{Rect_r}=[u _{Rect_r}v _{Rect_r}1] ^TAs shown in Figure 6, described binocular vision measurement and thin parts three-dimensional point cloud obtain specifically and comprise the steps:

Step 31: calculate the SIFT unique point in the image coordinate of binocular distortion correction image pair.Wherein, in the benchmark pose measurement stage, calculate the image coordinate of the binocular distortion correction image pair that the SIFT unique point on the thin parts of benchmark obtains at the thin parts of benchmark; In the pose measurement to be measured stage, calculate the image coordinate of the binocular distortion correction image pair that the SIFT unique point on the thin parts to be measured obtains in UUT.

Order point P binocular left and right sides distortion correction image on image coordinate be respectively [u _{Und_l}v _{Und_l}] ^T[u _{Und_r}v _{Und_r}] ^T, its corresponding homogeneous coordinates are respectively P _{Und_l}=[u _{Und_l}v _{Und_l}1] ^TAnd P _{Und_r}=[u _{Und_r}v _{Und_r}1] ^T, get according to formula (1):

$Z_{\mathrm{cl}}\left[\begin{array}{c}{u}_{\mathrm{und}\_l}\\ v_{\mathrm{und}\_l}\\ 1\end{array}\right]=M_{1l}{M}_{2l}\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]=M_l\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]=\left[\begin{array}{cccc}{m}_{l11}& m_{l12}& m_{l13}& m_{l14}\\ m_{l21}& m_{l22}& m_{l23}& m_{l24}\\ m_{l31}& m_{l32}& m_{l33}& m_{l34}\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]---\left(14\right)$

$Z_{\mathrm{cr}}\left[\begin{array}{c}{u}_{\mathrm{und}\_r}\\ v_{\mathrm{und}\_r}\\ 1\end{array}\right]=M_{1r}{M}_{2r}\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]=M_r\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]=\left[\begin{array}{cccc}{m}_{r11}& m_{r12}& m_{r13}& m_{r14}\\ m_{r21}& m_{r22}& m_{r23}& m_{r24}\\ m_{r31}& m_{r32}& m_{r33}& m_{r34}\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]---\left(15\right)$

In the formula, M _1l, M _2lAnd M _1r, M _2rBe respectively inner parameter matrix and the external parameter matrix of the left and right camera of binocular, M _lAnd M _rBe respectively the overall projection matrix of the left and right camera of binocular, and

$M_l=\left[\begin{array}{cccc}{m}_{l11}& m_{l12}& m_{l13}& m_{l14}\\ m_{l21}& m_{l22}& m_{l23}& m_{l24}\\ m_{l31}& m_{l32}& m_{l33}& m_{l34}\end{array}\right]$

$M_r=\left[\begin{array}{cccc}{m}_{r11}& m_{r12}& m_{r13}& m_{r14}\\ m_{r21}& m_{r22}& m_{r23}& m_{r24}\\ m_{r31}& m_{r32}& m_{r33}& m_{r34}\end{array}\right]$

Known this image coordinate P binocular solid correcting image centering _{Rect_l}And P _{Rect_r}, then this is at the image coordinate P of binocular distortion correction image pair _{Und_l}And P _{Und_l}Be respectively:

P _{und_l}＝M _1l(M _{rect_l}R _l) ^-1P _{rect_l}

P _{Und_r}=M _1r(M _{Rect_r}R _r) ^-1P _{Rect_r}Step 32: calculate the three-dimensional coordinate of SIFT unique point under world coordinate system.Wherein, in the benchmark pose measurement stage, calculate three-dimensional coordinate under world coordinate system according to the image coordinate of the described SIFT unique point of the thin parts of benchmark; In the pose measurement to be measured stage, calculate three-dimensional coordinate under world coordinate system according to the image coordinate of the described SIFT unique point of thin parts to be measured;

With this image coordinate P on binocular distortion correction image _{Und_l}And P _{Und_r}Difference substitution pinhole camera modeling formula (14) and formula (15) are in conjunction with the population parameter matrix M of left and right sides camera _lAnd M _r, cancellation Z _ClAnd Z _Cr, obtain system of equations:

$\left\{\begin{array}{c}(m_{l11}-m_{l31}{u}_{\mathrm{und}\_l})X_w+(m_{l12}-m_{l32}{u}_{\mathrm{und}\_l})Y_w+(m_{l13}-m_{l33}{u}_{\mathrm{und}\_l})Z_w=m_{134}{u}_{\mathrm{und}\_l}-m_{l14}\\ (m_{l21}-m_{l31}{v}_{\mathrm{und}\_l})X_w+(m_{l22}-m_{l32}{v}_{\mathrm{und}\_l})Y_w+(m_{l23}-m_{l33}{v}_{\mathrm{und}\_l})Z_w=m_{134}{v}_{\mathrm{und}\_l}-m_{l24}\\ (m_{r11}-m_{r31}{u}_{\mathrm{und}\_r})X_x+(m_{r12}-m_{r32}{u}_{\mathrm{und}\_r})Y_w+(m_{r13}-m_{r33}{u}_{\mathrm{und}\_r})Z_w=m_{r34}{u}_{\mathrm{und}\_r}-m_{r14}\\ (m_{r21}-m_{r31}{v}_{\mathrm{und}-r})X_w+(m_{r22}-m_{r32}{v}_{\mathrm{und}\_r})Y_w+(m_{r23}-m_{r33}{v}_{\mathrm{und}\_r})Z_w=m_{r34}{v}_{\mathrm{und}\_r}-m_{r24}\end{array}\right.$

Write system of equations as matrix form:

$A=\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\end{array}\right]=B$

In the formula,

$A=\left[\begin{array}{ccc}{m}_{l11}-m_{l31}{u}_{\mathrm{und}\_l}& m_{l12}-m_{l32}{u}_{\mathrm{und}\_l}& m_{l13}-m_{l33}{u}_{\mathrm{und}\_l}\\ m_{l21}-m_{l31}{v}_{\mathrm{und}\_l}& m_{l22}-m_{l32}{v}_{\mathrm{und}\_l}& m_{l23}-m_{l33}{v}_{\mathrm{und}\_l}\\ m_{r11}-m_{r31}{u}_{\mathrm{und}\_r}& m_{r12}-m_{r32}{u}_{\mathrm{und}\_r}& m_{r13}-m_{r33}{u}_{\mathrm{und}\_r}\\ m_{r21}-m_{r31}{v}_{\mathrm{und}\_r}& m_{r22}-m_{r32}{v}_{\mathrm{und}\_r}& m_{r23}-m_{r33}{v}_{\mathrm{und}\_r}\end{array}\right]$

$B=\left[\begin{array}{c}{m}_{l34}{u}_{\mathrm{und}\_l}-m_{l14}\\ m_{l34}{v}_{\mathrm{und}\_l}-m_{l24}\\ m_{r34}{u}_{\mathrm{und}\_r}-m_{r14}\\ m_{r34}{v}_{\mathrm{und}\_r}-m_{r24}\end{array}\right]$

Utilize least square method to try to achieve this three-dimensional coordinate under world coordinate system.

[X _w Y _w Z _w] ^T＝((A ^TA) ^-1A ^TB) (16)

Step 4: thin parts three-dimensional position and attitude are estimated.

According to least square method, calculate the principal plane of the thin parts three-dimensional point cloud of benchmark pose and thin parts three-dimensional point cloud to be measured respectively, by calculating the spatial relation between two principal planes, obtain three-dimensional position and the attitude of thin parts to be measured.

Step 41: the space principal plane that calculates three-dimensional point cloud; Namely in the benchmark pose measurement stage, with the three-dimensional planar at the three-dimensional point cloud place of the described SIFT unique point of the thin parts of benchmark as reference space principal plane π ₀In the pose measurement to be measured stage, with the plane at the three-dimensional power supply place of the described SIFT unique point of thin parts to be measured as space principal plane π to be measured;

Adopt least square method, according to the three-dimensional point cloud of the thin parts of benchmark pose, calculate its space reference principal plane π ₀According to the three-dimensional point cloud of thin parts to be measured, calculate its space principal plane π to be measured.With the example that is solved to of space principal plane π to be measured, make the equation of space plane π be: ax+by+cz+1=0 makes plane π normal vector n=that (c), the three-dimensional coordinate of specific location unique point is P for a, b _i=(x _i, y _i, z _i) ^T, then put P _iTo fit Plane π apart from d _i=nP _i+ 1, according to the principle of least square, require S=∑ (d _i) ², then by finding the solution following system of equations, solve normal vector n.

$\left\{\begin{array}{c}\frac{\∂S}{a}=0\\ \frac{\∂S}{b}=0\\ \frac{\∂S}{c}=0\end{array}\right.---\left(17\right)$

Step 42: find the solution space principal plane π to be measured and reference space principal plane π ₀Between transformation matrix, the unique point of the thin parts of benchmark and thin parts to be measured is projected to respectively on its principal plane, utilize the subpoint on these principal planes, calculate rotation matrix and translation vector between two planes, thereby accurately determine three-dimensional position and the attitude information of the thin parts of thin parts relative datum to be measured.

In the description of said method step, for simplicity, benchmark pose measurement and pose measurement to be measured stage have been carried out also line description, but in practical application, at first carry out camera calibration, carry out the benchmark pose measurement then and be the calculating of the obtaining of priori features storehouse, reference plane etc., carry out the pose measurement stage to be measured at last, namely find the solution three-dimensional position and the attitude information of actual thin parts to be measured based on reference plane.

Above-described specific embodiment; purpose of the present invention, technical scheme and beneficial effect are further described; institute is understood that; the above only is specific embodiments of the invention; be not limited to the present invention; within the spirit and principles in the present invention all, any modification of making, be equal to replacement, improvement etc., all should be included within protection scope of the present invention.

Claims (10)

1. the measuring method of a thin parts three-dimensional position to be measured and attitude, it comprises:

Step 1: demarcate the spatial relation of inner parameter, external parameter and the binocular camera of binocular camera, set up the mapping relations of three-dimensional world coordinate system and two dimensional image coordinate in the binocular vision measuring system;

Step 2: use the SIFT unique point in the thin parts binocular of the described benchmark of the SIFT operator extraction image right, generate SIFT priori features storehouse, and use described SIFT unique point to the three-dimensional planar at the place reference plane as the thin parts of benchmark place;

Step 3: the binocular solid correcting image for thin parts to be measured is right, it is right to extract a plurality of SIFT unique points respectively, and with the SIFT unique point in the SIFT unique point extracted pair and the described SIFT priori features storehouse to mating, obtain with described SIFT feature priori storehouse in the SIFT unique point to equal number and corresponding SIFT unique point pair set;

Step 4: the image coordinate according to unique point in the resulting SIFT unique point pair set obtains the three-dimensional point cloud of described thin parts to be measured, and then obtains the space plane of these thin parts to be measured;

Step 5: by finding the solution the transformation matrix between described space plane and the reference plane, determine three-dimensional position and the attitude information of described thin parts to be measured.

2. measuring method as claimed in claim 1 is characterized in that, described step 1 specifically comprises:

Step 11: set up the mapping relations between one camera two dimensional image coordinate system and the three-dimensional world coordinate system respectively;

Step 12: respectively single camera is demarcated, obtained camera inner parameter and external parameter;

Step 13: according to the camera inner parameter and the external parameter that obtain, carry out solid and demarcate, set up the spatial relation between the binocular camera.

3. measuring method as claimed in claim 2 is characterized in that, the inner parameter of described camera comprises the center position of coke ratio and the camera of camera, and external parameter comprises rotation matrix and the translation vector between three-dimensional camera coordinates system and the three-dimensional world coordinate system.

4. measuring method as claimed in claim 2 is characterized in that, the spatial relation between the described binocular camera is represented rotation matrix and the translation vector between the camera.

5. measuring method as claimed in claim 1 is characterized in that, the concrete steps that SIFT priori features storehouse obtains in the step 2 comprise:

Step 21: the binocular original image of gathering the thin parts of benchmark is right, utilize the lens distortion coefficient of binocular camera and three-dimensional correction parameter to described binocular original image to proofreading and correct respectively, the binocular solid correcting image that obtains the thin parts of benchmark is right;

Step 22: extract the SIFT feature from the binocular solid correcting image centering of the thin parts of described benchmark, form the SIFT priori features storehouse of the thin parts of benchmark.

6. measuring method as claimed in claim 1 is characterized in that, step 3 specifically comprises the steps:

Step 31, the binocular original image of gathering thin parts to be measured are right, utilize the lens distortion coefficient of binocular camera and three-dimensional correction parameter to described binocular original image to proofreading and correct respectively, the binocular solid correcting image that obtains thin parts to be measured is right;

Step 32: it is right to extract the SIFT unique point from the binocular solid correcting image centering of described thin parts to be measured, and with the SIFT unique point in the SIFT unique point extracted pair and the described SIFT priori features storehouse to mating, and then obtain with described SIFT priori features storehouse in equal number and corresponding SIFT unique point right.

7. as each described measuring method of claim 5-6, it is characterized in that described three-dimensional correction parameter comprises based on the rotation matrix between the described binocular camera and translation vector, the three-dimensional correction internal matrix of the binocular camera of acquisition and row alignment matrix.

8. measuring method as claimed in claim 1 is characterized in that, step 4 specifically comprises:

Step 41: according to the right image coordinate of SIFT unique point on the described thin parts to be measured, the mapping relations of three-dimensional world coordinate system and two dimensional image coordinate obtain the right three-dimensional world coordinate of described SIFT unique point in the binocular vision measuring system that utilization is set up;

Step 42: obtain described SIFT unique point to the three-dimensional planar at place according to the right three-dimensional world coordinate of described SIFT unique point, and with its space plane as thin parts to be measured.

9. measuring method as claimed in claim 8, it is characterized in that, the image coordinate that described SIFT unique point is right is binocular solid correcting image coordinate on the thin parts to be measured, the three-dimensional world coordinate of described SIFT unique point according to its binocular distortion correction image on image coordinate obtain, wherein said binocular distortion correction image obtains last image coordinate is following:

P _{und_l}＝M _1l(M _{rect_l}R _l) ^-1P _{rect_l}

P _{und_r}＝M _1r(M _{rect_r}R _r) ^-1P _{rect_r}

Wherein, P _{Und_l}=[u _{Und_l}v _{Und_l}1] ^TAnd P _{Und_r}=[u _{Und_r}v _{Und_r}1] ^TBe the image coordinate of binocular distortion correction image pair, P _{Rect_l}And P _{Rect_r}Be the image coordinate of binocular solid correcting image centering, M _1lAnd M _1rBe respectively the inner parameter matrix of binocular camera, M _{Rect_l}And M _{Rect_r}Be respectively the three-dimensional correction parameter matrix of binocular camera.

10. measuring method as claimed in claim 9 is characterized in that, the following acquisition of the three-dimensional coordinate of described SIFT unique point:

[X _w Y _w Z _w] ^T＝((A ^TA) ^-1A ^TB)

$A=\left[\begin{array}{ccc}{m}_{l11}-m_{l31}{u}_{\mathrm{und}\_l}& m_{l12}-m_{l32}{u}_{\mathrm{und}\_l}& m_{l13}-m_{l33}{u}_{\mathrm{und}\_l}\\ m_{l21}-m_{l31}{v}_{\mathrm{und}\_l}& m_{l22}-m_{l32}{v}_{\mathrm{und}\_l}& m_{l23}-m_{l33}{v}_{\mathrm{und}\_l}\\ m_{r11}-m_{r31}{u}_{\mathrm{und}\_r}& m_{r12}-m_{r32}{u}_{\mathrm{und}\_r}& m_{r13}-m_{r33}{u}_{\mathrm{und}\_r}\\ m_{r21}-m_{r31}{v}_{\mathrm{und}\_r}& m_{r22}-m_{r32}{v}_{\mathrm{und}\_r}& m_{r23}-m_{r33}{v}_{\mathrm{und}\_r}\end{array}\right]$

$B=\left[\begin{array}{c}{m}_{l34}{u}_{\mathrm{und}\_l}-m_{l14}\\ m_{l34}{v}_{\mathrm{und}\_l}-m_{l24}\\ m_{r34}{u}_{\mathrm{und}\_r}-m_{r14}\\ m_{r34}{v}_{\mathrm{und}\_r}-m_{r24}\end{array}\right]$

Wherein, (X _w, Y _w, Z _w) be the three-dimensional world coordinate of described SIFT unique point; m _lAnd m _rBe respectively the inner parameter matrix M of binocular camera _1lAnd M _1rElement.

Images