CN103247053B - Based on the part accurate positioning method of binocular microscopy stereo vision - Google Patents

Abstract

The part accurate positioning method that the present invention is based on binocular microscopy stereo vision belongs to computer vision measurement technical field, relates to a kind of accurate positioning method of the precision component based on binocular microscopy stereo vision.Adopt binocular microscopy stereo vision system, utilize two ccd video cameras to gather the image of tested part, the image information in the region to be measured on tested part is amplified by stereo microscope; Gridiron pattern scaling board is adopted to demarcate two ccd video cameras; Harris Corner Detection Algorithm and sub-pixel detection algorithm is adopted to carry out the extraction of unique point.Unique point after extraction is carried out the correction of just coupling and matching double points, unique point image coordinate is input to the space actual coordinate obtaining unique point in the system of having demarcated.The invention solves the measurement difficult problem because the problems such as target area to be measured is little, positioning accuracy request is high, noncontact produce.Adopt the contactless measurement based on binocular microscopy stereo vision, well complete the accurate location of precision component.

Description

Based on the part accurate positioning method of binocular microscopy stereo vision

Technical field

The invention belongs to computer vision measurement technical field, particularly a kind of accurate positioning method of the precision component based on binocular microscopy stereo vision.

Background technology

Technique of binocular stereoscopic vision is by the processing mode of simulating human eyes, there is the ability obtaining object under test depth information, and then the spatial positional information of testee can be obtained, the advantage having again nondestructive measurement simultaneously and measure in real time, is widely used in all trades and professions.Binocular microscopy stereo vision technology is based upon on technique of binocular stereoscopic vision and Stereo microscope technology, the image of stereo microscope two light paths is caught respectively by two, left and right ccd video camera, existed with differential seat angle similar during Binocular vision photogrammetry by the image caught, the three-dimensional measurement to target can be realized.The spatial positional information of impact point accurately can be obtained while the advantages such as binocular microscopy stereo vision technology is untouchable in guarantee, real-time.It can be applicable to the aspects such as three-dimensional high-precision servocontrol, high-precision spatial location and high precision three-dimensional measurement.The setting accuracy on work of part directly affects the machining precision of part, and processing parts is likely directly affected for precision component positioning precision whether can use, locating and machining simultaneously for accurate ultra part often requires non-contacting feature, and the accurate positioning method based on the precision component of binocular microscopy stereo vision is arisen at the historic moment.

The camera calibration process that the patent of invention CN102567989A " space-location method based on binocular stereo vision " of the soup precious equality people application of University Of Chongqing exists for space orientation is loaded down with trivial details, the not high problem of positioning precision proposes a kind of space-location method based on binocular stereo vision newly, the method is by demarcating the homography matrix and distortion factor of trying to achieve video camera to video camera, obtained the theoretical coordinate of impact point again by calibration result, the actual coordinate of last combining target point calculates the volume coordinate of impact point.But, because binocular microscopy stereo vision system has more complicated light path, depth of focus is little, distortion many factors, visual field is narrow, and the camera marking method extensively adopted at present realizes under being all based upon Large visual angle, therefore said method can not solve the accurate location of the precision component based on binocular microscopy stereo vision.

Summary of the invention

The technical barrier that the present invention will solve is the defect overcoming prior art, invent a kind of accurate positioning method of the precision component based on binocular microscopy stereo vision, solve the measurement difficult problem because the problems such as target area to be measured is little, positioning accuracy request is high, noncontact produce.Adopt the accurate location realizing precision component based on binocular microscopy stereo vision, widen the range of application of traditional space-location method based on binocular stereo vision, and improve the precision of measurement, solve part zone to be measured little, positioning accuracy request is high, is difficult to the problem measured.

The technical solution adopted in the present invention is the accurate positioning method of the precision component based on binocular microscopy stereo vision, it is characterized in that, adopt binocular microscopy stereo vision system, utilize two, left and right ccd video camera 2,2 ' to gather the image location information of tested part 4, the image location information in the region to be measured 6 on tested part 4 amplifies by stereo microscope 3; Image after amplification passes in industrial control computer 9 through image pick-up card, adopt high precision compared with gridiron pattern scaling board two, left and right ccd video camera is demarcated; Harris Corner Detection Algorithm and sub-pixel detection algorithm is adopted to carry out the extraction of unique point to be measured; Unique point to be measured after extraction is carried out the correction of just coupling and matching double points; The unique point image coordinate matched is input to the space actual coordinate obtaining unique point in the system of having demarcated; The concrete steps of measuring method are as follows:

(1) demarcation of two ccd video cameras in left and right

The demarcation of two ccd video cameras in left and right comprises camera intrinsic parameter and external parameters of cameras; Camera intrinsic parameter comprises scale factor α, β, principal point coordinate u ₀, v ₀, and vertical factor γ; In calibration process, need five intrinsic parameters first obtaining video camera, on the basis of trying to achieve Intrinsic Matrix, solve external parameters of cameras matrix; Scale factor be spatial point after translation rotational transform, the scaling relation of its coordinate in camera coordinate system and its coordinate in image coordinate system, the present invention adopts Zhang Shi camera calibration method to the demarcation of scale factor; The relation formula can setting up image coordinate system and world coordinate system according to the pin-hole model of video camera is:

$Z_c\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=\left[\begin{array}{ccc}\mathrm{\α}& \mathrm{\γ}& u_0\\ 0& \mathrm{\β}& v_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{cc}R& t\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]---\left(1\right)$

Wherein: α and β is exactly the scale factor needing to demarcate, X _w, Y _w, Z _wbe the three dimensional space coordinate of 1 P in space, u, v are the image coordinate of P point on image, and R, t represent camera coordinate system relative to the rotation of world coordinate system and translation matrix; If make X ' _w=(X _w, Y _w, Z _w) ^t, z _c=s, then

$s{\stackrel{\‾}{x}}_w={\mathrm{HX}}_w^{\′}---\left(2\right)$

Wherein, H also referred to as homography matrix, and has

H＝K[r ₁,r ₂,t] (3)

The independent corresponding homography matrix of scaling board image of each position, makes H=[h ₁, h ₂, h ₃] then can be pushed away by above formula:

[h ₁h ₂h ₃]＝λK[r ₁r ₂t] (4)

Wherein, λ is arbitrary scale factor, and K camera intrinsic parameter matrix, because of r ₁and r ₂unit orthogonal vector, i.e. r ₁ ^tr ₁=r ₂ ^tr ₂=1 and r ₁ ^tr ₂=0:

$h_1^T{K}^{-T}{K}^{-1}{h}_1=h_2^T{K}^{-T}{K}^{-1}{h}_2=1---\left(5\right)$

h ₁K ^-TK ^-1h ₂＝0 (6)

According to formula 5 formula 6, calculate the video camera scale factor of stereo microscope left and right light path; For principal point coordinate, the principal point of zoom rate method to video camera is adopted to demarcate; In hypothesis space, the coordinate of any point P in camera coordinate system is x ₁, y ₁, z ₁, when not considering nonlinear distortion and image coordinate verticality, the coordinate of the plane of delineation projection of this point is:

u ₁＝rx ₁+u ₀

(7)

v ₁＝ry ₁+v ₀

Wherein u ₁, v ₁be the image coordinate of this point, r is arbitrary enlargement ratio, u ₀, v ₀it is principal point coordinate; By the r cancellation in formula, can straight-line equation be obtained:

$\frac{u_1-u_0}{x_1}=\frac{v_1-v_0}{y_1}---\left(8\right)$

Namely under any enlargement ratio, some x ₁, y ₁, z ₁image coordinate all on the same line, and this straight line is necessarily through video camera principal point u ₀, v ₀; In order to ask for video camera principal point u ₀, v ₀selected 12 points do not overlapped in image projection plane, these 12 points are made 0.7,1,2,2.4,3,5,5.8,8 to project under different enlargement ratio respectively, utilize least square method to carry out matching to 12 straight lines obtained, and ask for intersecting point coordinate and the video camera principal point u of 12 straight lines by least square method ₀, v ₀; Because vertical factor γ is little on imaging precision impact, therefore at initial parameter timing signal, γ is set to 0, obtains γ occurrence by optimization means; For external parameters of cameras rotation matrix R and translation matrix t; Known by formula 2, can in the hope of video camera rotation matrix R=[r after having demarcated Intrinsic Matrix K and homography matrix H ₁, r ₂, r ₃] and translation vector t as follows:

r ₁＝λK ^-1h ₁

r ₂＝λK ^-1h ₂(9)

r ₃＝r ₁×r ₂

t＝λK ^-1h ₃

After tentatively having demarcated the principal parameter of two ccd video cameras in left and right, next step has been the optimization to principal parameter and distortion parameter; In order to simplify Optimal Parameters, the present invention adopts Quaternion Method by rotation matrix in nine unknown numbers be reduced to four, formula is as follows:

$\begin{array}{c}{q}_1=\frac{\sqrt{1+r_{11}+r_{22}+r_{33}}}{2},q_2=\frac{r_{23}-r_{32}}{{4q}_1},q_3=\frac{r_{31}-r_{13}}{{4q}_1},q_4=\frac{r_{12}-r_{21}}{{4q}_1}\\ q_2=\frac{\sqrt{1+r_{11}-r_{22}-r_{33}}}{2},q_1=\frac{r_{23}-r_{32}}{{4q}_2},q_3=\frac{r_{12}+r_{21}}{{4q}_2},q_4=\frac{r_{31}-r_{13}}{{4q}_2}\\ q_3=\frac{\sqrt{1-r_{11}+r_{22}-r_{33}}}{2},q_1=\frac{r_{31}-r_{13}}{{4q}_3},q_2=\frac{r_{12}-r_{21}}{{4q}_3},q_4=\frac{r_{23}-r_{32}}{{4q}_3}\\ q_4=\frac{\sqrt{1-r_{11}-r_{22}+r_{33}}}{2},q_1=\frac{r_{12}-r_{21}}{{4q}_4},q_2=\frac{r_{31}+r_{13}}{{4q}_4},q_3=\frac{r_{23}-r_{32}}{{4q}_4}\end{array}---\left(10\right)$

New rotation matrix is calculated from formula (10):

$R=\left[\begin{array}{ccc}-1+{2q}_2^2+{2q}_1^2& {2q}_2{q}_3-{2q}_1{q}_4& {2q}_1{q}_3+{2q}_2{q}_4\\ {2q}_2{q}_3+{2q}_4{q}_1& -1+{2q}_1^2+{2q}_3^2& -{2q}_1{q}_2+{2q}_3{q}_4\\ {-2q}_1{q}_3+{2q}_4{q}_2& {2q}_1{q}_2+{2q}_3{q}_4& -1+{2q}_1^2+{2q}_4^2\end{array}\right]---\left(11\right)$

Due to hypercomplex number q ₁, q ₂, q ₃, q ₄quadratic sum equal one, therefore the parameter predigesting to be optimized of rotation matrix is three; Using two, the left and right ccd video camera calibrating parameters value of having tried to achieve as optimized initial value, optimized algorithm is utilized to carry out global optimization to video camera principal parameter and distortion parameter; Based on maximal possibility estimation criterion to the optimization of two, left and right ccd video camera principal parameter and distortion parameter, for given n m × n that scaling board picture provides calibration point coordinate, the optimization problem of two the ccd video camera parameters in left and right is expressed by the minimization problem of following formula:

$S\left(\mathrm{\θ}\right)=\underset{i=1}{\overset{n}{\mathrm{\Σ}}}\underset{i=1}{\overset{m}{\mathrm{\Σ}}}{\mathrm{\ϵ}}_i^2=\underset{i=1}{\overset{n}{\mathrm{\Σ}}}\underset{i=1}{\overset{m}{\mathrm{\Σ}}}{[y_i-m_i(C_j,p_j,X_j)]}^2---\left(12\right)$

Wherein j refers to the jth video camera participating in calculating, and i refers to i-th point that a jth video camera obtains, X _ithe spatial point coordinate of input, y _ithe image coordinate in i-th, space, C _jbe changeless camera parameters vector, its length is n ₀, p _jbe the camera parameters vector needing adjustment, its length is n ₁, n ₀+ n ₁be all parameter vector length of video camera, m _i(C _j, p _j, X _i) be the imaging equation of video camera; The present invention adopt solve above formula based on the light-stream adjustment of LM algorithm minimize optimization problem; Utilize light-stream adjustment write optimizer time some also will be provided except the calibrating parameters initial value of two ccd video cameras in left and right will be provided with the three dimensional space coordinate point of form and the respective image coordinate of these three-dimensional coordinate points, therefore in order to accurately obtain the angular coordinate of high-accuracy scaling board in whole view field space, the method that numerically-controlled machine z-axis drive microscope vertically moves is adopted; And adopt laser interferometer to obtain the actual range of z-axis movement to z-axis measurement, ensure that the z of institute's acquisition angle point coordinate point is to realistic accuracy; In light-stream adjustment, the Jacobian matrix that the iteration factor of optimized algorithm is made up of the first order derivative of iteration parameter is formed, and therefore needs in computation process, introduce the Jacobian matrix adjusting parameter in equation to be imaged;

$J=[\frac{\mathrm{df}}{\mathrm{dp}\left(1\right)},\frac{\mathrm{df}}{\mathrm{dp}\left(2\right)},....,\frac{\mathrm{df}}{\mathrm{dp}\left(n_1\right)}]---\left(13\right)$

Light-stream adjustment, with the Jacobian matrix of its parameter to be adjusted composition iteration factor, adjusts model parameter, obtains the model parameter optimum solution making image error S (θ) reach minimum.

(2) extraction of provincial characteristics point to be measured

What the extraction that the present invention is directed to provincial characteristics to be measured point adopted is Harris Corner Detection Algorithm; Harris proposes the algorithm replacing square region compute gradient with Gaussian function, if the pixel coordinate of impact point is x, y, it is respectively u and v in the displacement in x direction and y direction, then the grey scale change of this impact point is represented by following formula:

$\begin{array}{c}E(x,y)=\underset{u,v}{\mathrm{\Σ}}\mathrm{\ω}(u,v){[I(x+u,y+v)-I(x,y)]}^2\\ \≈\underset{u,v}{\mathrm{\Σ}}\mathrm{\ω}(u,v)(u,v)\left[\begin{array}{cc}{I}_x^2& I_x{I}_y\\ I_x{I}_y& I_y^2\end{array}\right]\left(\begin{array}{c}u\\ v\end{array}\right)\end{array}---\left(14\right)$

Order $M(x,y)=\underset{u,v}{\mathrm{\Σ}}\mathrm{\ω}(u,v)\left[\begin{array}{cc}{I}_x^2& I_x{I}_y\\ I_x{I}_y& I_y^2\end{array}\right]$ Then:

$E(x,y)\≈(u,v)M(x,y)\left(\begin{array}{c}u\\ v\end{array}\right)---\left(15\right)$

The eigenvalue λ of compute matrix M ₁, λ ₂if two eigenwerts are all larger, illustrate that the value on two orthogonal directionss of the gradation of image autocorrelation function of impact point is all comparatively large, then this point is unique point; After just extraction being carried out to unique point by Harris algorithm; Subpixel corner detecting algorithm is adopted to extract high-precision angular coordinate; For desirable angle point, the shade of gray direction of the pixel near it is all perpendicular to the line of this point and desirable angle point; This feature can be with equation expression:

$\▿\stackrel{\→}{H}(\stackrel{\→}{\mathrm{\α}}-\stackrel{\→}{\mathrm{\β}})=0---\left(16\right)$

Wherein the shade of gray direction of desirable angle point, vector points to the coordinate of desirable angle point for image origin, vector points to the coordinate of arbitrary marginal point near desirable angle point for image origin; In fact, owing to being subject to the impact of picture noise, formula (16) is usually all non-vanishing, can be regarded as error e, namely

$e=\▿\stackrel{\→}{H}(\stackrel{\→}{\mathrm{\α}}-\stackrel{\→}{\mathrm{\β}})---\left(17\right)$

In the neighborhood centered by angle point, all above formulas of pressing are calculated, error and be E, then

$E=\underset{i}{\mathrm{\Σ}}e=\underset{i}{\mathrm{\Σ}}\▿\stackrel{\→}{H}(\stackrel{\→}{\mathrm{\α}}-\stackrel{\→}{\mathrm{\β}})---\left(18\right)$

Wherein, i is i-th point in neighborhood; Ask like this and make error and the minimum point of E namely be other angular coordinate of sub-pixel.

(3) Stereo matching of unique point

Adopt the first coupling of normalization crossover algorithm (NCC) realization character point, the picture clapped using ccd video camera 2 as reference base picture, and with unique point p _lcentered by, topography's block of a structure N*N as masterplate figure T, and makes masterplate figure T travel through in the certain limit including unique point of image S to be matched, and the subgraph that search template covers is denoted as S ^i,j, i, j are the pixel coordinate of subgraph central point in matching image S, calculate masterplate figure T and search subgraph S by using normalization crossover algorithm (NCC) ^i,jbetween normalized-cross-correlation function, algorithm is as follows:

$R(i,j)=\frac{\underset{m,n}{\mathrm{\Σ}}|S^{i,j}(m,n)-E\left(S^{i,j}\right)||T(m,n)-E\left(T\right)|}{\sqrt{\underset{m,n}{\mathrm{\Σ}}{(S^{i,j}(m,n)-E\left(S^{i,j}\right))}^2\underset{m,n}{\mathrm{\Σ}}{(T(m,n)-E\left(T\right))}^2}}---\left(19\right)$

Wherein m, n represent the coordinate of each pixel, E (S ^i,j) and E (T) be search subgraph S respectively ^i,jwith the average gray of masterplate figure T; The value of cross-correlation coefficient R (i, j) is larger, then the matching degree of searching for subgraph and Prototype drawing is higher; After the threshold value of setting cross-correlation coefficient R (i, j), select unique point p _lcandidate matches point; Also need after using NCC algorithm to carry out first coupling to utilize outer limit restraint and the feature point pairs of distance restraint to coupling to correct; Normalization 8 algorithms proposed by using Longguet-Higgins calculate the fundamental matrix F of two ccd video cameras in left and right when camera calibration, theoretical according to EP point, to benchmark image unique point p _l, EP point l corresponding in image S to be matched _rcan be expressed as:

l _r＝F·p _l(20)

If the candidate feature point of first coupling is at EP point l _rnear then can determine that this candidate feature point is match point further; Finally to add up in each candidate feature point to the Distance geometry benchmark image of other unique points unique point to the equidistant number num of other unique point points; Candidate feature point corresponding during num maximal value is then the correct match point of unique point in benchmark image.

(4) the asking for of three-dimensional coordinate

First calculate the image coordinate value that in binocular image, unique point extracts through sub-pixel recognition, and roughly estimate the estimated value of unique point in world coordinate system; In optimizer, corresponding for two, left and right ccd video camera parameter Jacobian matrix is set to zero, calculates in iteration factor part the Jacobian matrix equation embedded about volume coordinate x, y, z simultaneously at optimizer; By two, the left and right ccd video camera parameter optimized and unique point image coordinate value and world coordinates estimated value input beam method of adjustment optimizer, just can be optimized the three-dimensional coordinate of spatial point, obtain the three-dimensional space measurement value of unique point, thus complete asking for and the location of final realization character point of three-dimensional coordinate point.

The invention has the beneficial effects as follows and solve that binocular micro-vision timing signal lens are many, visual field is narrow, the depth of field is little, there is certain enlargement ratio and include the technical difficulty of more distortion factor, achieve the high-precision calibrating of binocular microscopy stereo vision, thus complete the accurate location of precision component.

Accompanying drawing explanation

Figure 1 shows that the mounted cast figure that the precision component based on binocular microscopy stereo vision is located.Wherein, 1-provide the LED light source of illumination, 2-left ccd video camera, 2 '-right ccd video camera, 3-stereo microscope for binocular microscopy stereo vision system, 4-workpiece to be measured; 5-can incline numerical control rotating platform; 6-region to be measured; 7-accurate digital control displacement platform; 8-lathe bed; 9-industrial control computer.

Figure 2 shows that the process flow diagram of the precision component locating measurement method based on binocular microscopy stereo vision.

Figure 3 shows that the process flow diagram asked for of demarcation based on the binocular microscopy stereo vision of bundle adjustment algorithm and three-dimensional coordinate point.

Embodiment

The specific embodiment of the present invention is described in detail below in conjunction with technical scheme and accompanying drawing.Accompanying drawing 1 is the mounted cast figure located based on the precision component of binocular microscopy stereo vision.This device gathers the positional information in region to be measured in workpiece for measurement by two, left and right ccd video camera 2,2 ', by the relation between the image coordinate system that established and world coordinate system, finds the three dimensional space coordinate of point to be measured namely to realize precision positioning.The mounting means of its device is as follows: bed piece 8 is placed in ground; Accurate digital control displacement platform 7 is installed together by guide rail and bed piece, and it can meet in calibration experiment the high precision translational of gridiron pattern scaling board and angle rotation demand; The numerical control rotating platform 5 that can incline is bolted on the T-slot of cast iron platform; Workpiece for measurement 4 is arranged on by unit clamp and can inclines on numerical control rotating platform, and its region 6 to be measured is positioned at the visual field of binocular microscopy stereo vision system; Stereo microscope 3 is arranged on bed piece by special microscope fixture; LED light source 1 is fixed on fixture, and fixture is connected on lathe bed by bolt; Two ccd video cameras in left and right 2 and 2 ' are threaded connection and are arranged on stereo microscope; The image that two, left and right ccd video camera collects is delivered in industrial control computer 9 respectively by 1394 image pick-up cards and carries out image real time transfer.

Accompanying drawing 1 is one embodiment of the present of invention, two, left and right ccd video camera 2,2 ' is adopted to take the image location information of object under test, what two, left and right ccd video camera adopted is Olympus DP26 video camera, image resolution ratio: 2448*1920, picking rate: 7fps, chip size: 2/3 inch.What stereo microscope 3 adopted is the SZX-16 research grade stereo microscope that Olympus is produced.Zoom ratio: 0.7-11.5, operating distance: 60mm, gathers image maximum field of view: 12.5mm × 16.6mm, gathers the minimum visual field of image: 0.76mm × 1.02mm.For realizing the demarcation of binocular microscopy stereo vision system, what the scaling board needed for demarcation adopted is the CG-050-T-0.5 type substrate of glass gridiron pattern scaling board produced in Shenzhen Kechuang epoch.This type scaling board grid size is 0.5mm × 0.5mm, and gridiron pattern pattern overall width is 51mm × 51mm, and manufacturing accuracy is 1 μm, can meet the needs that stereo microscope is demarcated.

Figure 2 illustrates the process flow diagram of the precision component locating measurement method based on binocular microscopy stereo vision, the main flow of precision positioning comprises the demarcation of two ccd video cameras 2,2 ' in left and right, the extraction of binocular image unique point, the coupling of binocular image unique point and correction, the three-dimensional coordinate of binocular image unique point is asked for.Wherein, the demarcation of two ccd video cameras in left and right utilizes high-accuracy gridiron pattern scaling board to realize solving the principal parameter of left and right two ccd video camera 2,2 ' and distortion parameter, mainly comprises adopting Zhang Shi standardization to the demarcation of scale factor, adopting zoom rate method realize the demarcation of principal point coordinate and adopt light-stream adjustment to the optimization of principal parameter, distortion parameter; The unique point of binocular image utilizes Harris Corner Detection Algorithm and sub-pixel detection algorithm to extract; The reference points matching of binocular image is that the correction by carrying out just coupling and feature point pairs to the gauge point of binocular image realizes, and realizes asking for of three-dimensional coordinate point eventually through light-stream adjustment.

Attachedly Figure 3 shows that the process flow diagram that demarcation and three-dimensional coordinate point based on the binocular microscopy stereo vision of bundle adjustment algorithm are asked for, image under the different multiplying obtained by shooting uses zoom rate method to obtain principal point coordinate, Zhang Shi standardization is used to obtain scale factor by the image of 30 secondary various angles, and the outer parameter of two ccd video cameras in left and right is asked for by acquired intrinsic parameter, using the initial value that intrinsic parameter and outer parameter are optimized as light-stream adjustment, and use laser interferometer to obtain high-precision three-dimensional lattice as the constraint condition optimized, with the Jacobian matrix of parameter to be adjusted composition iteration factor, the parameter after optimizing is obtained by light-stream adjustment, parameter after optimization is brought in imaging model and obtains final imaging model, finally imaging model, image coordinate and the three-dimensional coordinate estimated value of tested point are input to the 3 d space coordinate asking for tested point in the program of light-stream adjustment.

The concrete steps of measuring method are as follows:

(1) demarcation of two ccd video cameras in left and right

The present invention adopts the relatively-stationary mode of video camera, the method for demarcating video camera with precision machined gridiron pattern scaling board.Zhang Shi camera calibration method requires that different scaling board picture has larger angular relationship, and usual best angle is 45 °, and we are limited to the depth of field factor of stereo microscope, when scaling board and surface level angle are more than 20 °, there will be many fuzzy angle points in visual field, in order to the scale factor obtaining video camera that can be stable, and the experimentation that Zhang Shi demarcates is summarized, find that scaling board picture number is more, the scaling board picture number be parallel to each other is fewer, the scale factor of trying to achieve video camera that more can be stable, therefore to left and right two ccd video camera 2, the concrete steps of the demarcation of the scale factor of 2 ' are as follows: rotated to be divided into by scaling board and rotate two kinds of modes along horizontal rotational shaft and vertical axes, vertical axes rotates each rotation 40 °, just there are nine horizontal angle positions in such scaling board.Carry out horizontal rotational shaft in each horizontal angle position, scaling board horizontal rotational shaft position is divided into+10 °, 0 °, 10 ° of three positions, at each position photographs scaling board image.The whole calibration process to scale factor takes 54 scaling board pictures altogether.For principal point coordinate, consider that principal point is actually the intersection point of optical axis and CCD imaging surface, and when microscope magnifications changes, the principal point coordinate that the position of optical axis is actually constant namely video camera remains constant, under any enlargement ratio, point (x ₁, y ₁, z ₁) image coordinate all on the same line, and this straight line is necessarily through video camera principal point (u ₀, v ₀); Like this, the spatial point that n projection does not on the image plane overlap, by projecting under different enlargement ratio, just can form the straight line that n bar intersects at same point; The coordinate of this intersection point is exactly video camera principal point coordinate.Zoom rate method is adopted to realize the demarcation concrete operation step of principal point coordinate: to determine 12 points do not overlapped in image projection plane, these 12 points are made 0.7,1,2,2.4,3,5,5.8,8 to project under different enlargement ratio respectively, utilize least square method to 12 straight lines intersected at a point obtained, ask for intersection coordinate value.Outer parameter matrix is demarcating intrinsic parameter and homography matrix utilizes formula 7 to solve.Adopt the process of the optimization of the internal outer calibrating parameters of light-stream adjustment: in optimizer, input imaging equation, spatial point image coordinate observed value, initial parameter values to be optimized, the output of program is the calibrating parameters value after S (θ) can be made to export minimum optimization.

In order to verify the precision of calibration result, utilizing calibrated model parameter, three-dimensional reconstruction being carried out to the three dimensions point that scaling board provides, obtains the measured value of three dimensions point.By the actual value of measured value and three dimensions point coordinate is compared analysis, carry out the stated accuracy of evaluating.Calibration result shows, and under 1 multiplying power, the average stated accuracy in x direction, y direction, z direction is respectively: 2.3um, 1.4um, 7.9um; Under 2 multiplying powers, the average stated accuracy in x direction, y direction, z direction is respectively: 1.5um, 1.2um, 5.4um; Under 5 multiplying powers, the average stated accuracy in x direction, y direction, z direction is respectively: 0.7um, 0.5um, 3.2um.Calibration result shows that calibrated model parameter has higher horizontal reconstruction precision and good longitudinal reconstruction precision.

(2) extraction of unique point

Harris Corner Detection Algorithm is adopted to carry out unique point X to two, left and right ccd video camera 2 and the 2 ' picture collected by using ₁, X ₂x _nfirst extraction, Harris Corner Detection Algorithm is by calculating eigenvalue λ ₁, λ ₂if two eigenwerts are all larger, illustrate that the value on two orthogonal directionss of the gradation of image autocorrelation function of impact point is all comparatively large, then this point is unique point.With the unique point X after extraction ₁, X ₂x _ncentered by, calculate error and the E of each unique point respectively, ask and make error and the minimum point of E namely be other corner location of sub-pixel.In this way, just can obtain sub-pix angular coordinate on the basis of Harris angle point grid, improve the extraction accuracy of unique point.

(3) coupling of unique point

Adopt the first coupling of normalization crossover algorithm (NCC) realization character point, according to algorithmic formula (19), the picture clapped using ccd video camera 2 as reference base picture, and with unique point p _lcentered by, topography's block of a structure N*N as masterplate figure T, and makes masterplate figure T travel through in the certain limit including unique point of image S to be matched, and the subgraph that search template covers is denoted as S ^i,j, i, j are the pixel coordinate of subgraph central point in matching image S, calculate masterplate figure T and search subgraph S by using normalization crossover algorithm (NCC) ^i,jbetween normalized-cross-correlation function.The value of cross-correlation coefficient R (i, j) is larger, then the matching degree of searching for subgraph and Prototype drawing is higher.After the threshold value of setting cross-correlation coefficient R (i, j), select unique point p _lcandidate matches point.Also need after using NCC algorithm to carry out first coupling to utilize epipolar line restriction and the feature point pairs of distance restraint to coupling to correct; Normalization 8 algorithms proposed by using Longguet-Higgins calculate the fundamental matrix F of two ccd video cameras in left and right when camera calibration, theoretical according to EP point, to benchmark image unique point p _l, use formula (20) calculates the unique point p in benchmark image _leP point position in image to be matched, if just the candidate feature point of coupling, near EP point, can determine that this candidate feature point is match point further.Finally correct matching double points further by distance restraint.Adopt normalization crossover algorithm (NCC) to carry out the first coupling of unique point, epipolar line restriction and distance restraint correct the feature point pairs of coupling, and the feature point pairs of two images match in left and right is respectively: X _l, X _r', X _2l, X _2r' ... X _3l, X _3r'.

(4) the asking for of three-dimensional coordinate

Light-stream adjustment, except can being optimized camera parameters, can also carry out the optimization of three-dimensional coordinate point simultaneously; Namely the optimization of three-dimensional coordinate point be herein asking for of three-dimensional coordinate; Ask for process as follows: corresponding for camera parameters in optimizer Jacobian matrix is set to zero, calculates in iteration factor part the Jacobian matrix equation embedded about volume coordinate x, y, z simultaneously at optimizer; By the image coordinate of two, the left and right ccd video camera parameter optimized and spatial point and world coordinates estimated value input beam method of adjustment optimizer, just can be optimized the three-dimensional coordinate of spatial point, obtain the measured value of three dimensions point, thus complete asking for of three-dimensional coordinate point.

The present invention preferably resolves the measurement difficult problem because the problems such as target area to be measured is little, positioning accuracy request is high, noncontact produce.Adopt the contactless measurement based on binocular microscopy stereo vision, well complete the accurate location of precision component.

Claims (1)

1. the part accurate positioning method based on binocular microscopy stereo vision, it is characterized in that, adopt binocular microscopy stereo vision system, utilize two, left and right ccd video camera (2,2 ') to gather the image location information of tested part (4), the image location information in the region to be measured (6) on tested part (4) amplifies by stereo microscope (3); Image after amplification passes in industrial control computer (9) through image pick-up card, adopts the higher gridiron pattern scaling board of precision to demarcate two, left and right ccd video camera; Harris Corner Detection Algorithm and sub-pixel detection algorithm is adopted to carry out the extraction of unique point to be measured; Unique point to be measured after extraction is carried out the correction of just coupling and matching double points; The unique point image coordinate matched is input to the space actual coordinate obtaining unique point in the system of having demarcated; The concrete steps of measuring method are as follows:

(1) demarcation of two ccd video cameras in left and right

The demarcation of two ccd video cameras in left and right comprises camera intrinsic parameter and external parameters of cameras; Camera intrinsic parameter comprises scale factor α, β, principal point coordinate u ₀, v ₀, and vertical factor γ; In calibration process, need five intrinsic parameters first obtaining video camera, on the basis of trying to achieve Intrinsic Matrix, solve external parameters of cameras matrix; Scale factor be spatial point after translation rotational transform, the scaling relation of its coordinate in camera coordinate system and its coordinate in image coordinate system, the present invention adopts Zhang Shi camera calibration method to the demarcation of scale factor; The relation formula can setting up image coordinate system and world coordinate system according to the pin-hole model of video camera is:

$Z_c\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=\left[\begin{array}{ccc}\mathrm{\α}& \mathrm{\γ}& u_0\\ 0& \mathrm{\β}& v_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{cc}R& t\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]---\left(1\right)$

Wherein: α and β is exactly the scale factor needing to demarcate, X _w, Y _w, Z _wbe the three dimensional space coordinate of 1 P in space, u, v are the image coordinate of P point on image, and R, t represent camera coordinate system relative to the rotation of world coordinate system and translation matrix; If make X ' _w=(X _w, Y _w, Z _w) ^t, z _c=s, then

$s{\stackrel{\‾}{x}}_w=H{X}_w^{\′}---\left(2\right)$

Wherein, H also referred to as homography matrix, and has

H＝K[r ₁,r ₂,t] (3)

The independent corresponding homography matrix of scaling board image of each position, makes H=[h ₁, h ₂, h ₃] then can be pushed away by above formula:

[h ₁h ₂h ₃]＝λK[r ₁r ₂t] (4)

Wherein, λ is arbitrary scale factor, and K camera intrinsic parameter matrix, because of r ₁and r ₂unit orthogonal vector, namely ${r_1}^T{r}_1={r_2}^T{r}_2=1$ And ${r_1}^T{r}_2=0$ Then:

$h_1^T{K}^{-T}{K}^{-1}{h}_1=h_2^T{K}^{-T}{K}^{-1}{h}_2=1---\left(5\right)$

h ₁K ^-TK ^-1h ₂＝0 (6)

According to formula (5) formula (6), calculate the video camera scale factor of stereo microscope left and right light path; For principal point coordinate, the principal point of zoom rate method to video camera is adopted to demarcate; In hypothesis space, the coordinate of any point P in camera coordinate system is x ₁, y ₁, z ₁, when not considering nonlinear distortion and image coordinate verticality, the coordinate of the plane of delineation projection of this point is:

u ₁＝rx ₁+u ₀

(7)

v ₁＝ry ₁+v ₀

Wherein u ₁, v ₁be the image coordinate of this point, r is arbitrary enlargement ratio, u ₀, v ₀it is principal point coordinate; By the r cancellation in formula, can straight-line equation be obtained:

$\frac{u_1-u_0}{x_1}=\frac{v_1-v_0}{y_1}---\left(8\right)$

Namely under any enlargement ratio, some x ₁, y ₁, z ₁image coordinate all on the same line, and this straight line is necessarily through video camera principal point u ₀, v ₀; In order to ask for video camera principal point u ₀, v ₀selected 12 points do not overlapped in image projection plane, these 12 points are made 0.7,1,2,2.4,3,5,5.8,8 to project under different enlargement ratio respectively, utilize least square method to carry out matching to 12 straight lines obtained, and ask for intersecting point coordinate and the video camera principal point u of 12 straight lines by least square method ₀, v ₀; Because vertical factor γ is little on imaging precision impact, therefore at initial parameter timing signal, γ is set to 0, obtains γ occurrence by optimization means; For external parameters of cameras rotation matrix R and translation matrix t; Known by formula (2), can in the hope of video camera rotation matrix R=[r after having demarcated Intrinsic Matrix K and homography matrix H ₁, r ₂, r ₃] and translation vector t as follows:

r ₁＝λK ^-1h ₁

r ₂＝λK ^-1h ₂(9)

r ₃＝r ₁×r ₂

t＝λK ^-1h ₃

After tentatively having demarcated the principal parameter of two ccd video cameras in left and right, next step has been the optimization to principal parameter and distortion parameter; In order to simplify Optimal Parameters, the present invention adopts Quaternion Method by rotation matrix $R=[r_1,r_2,r_3]=\left[\begin{array}{ccc}{r}_{11}& r_{21}& r_{31}\\ r_{12}& r_{22}& r_{32}\\ r_{13}& r_{23}& r_{33}\end{array}\right]$ In nine unknown numbers be reduced to four, formula is as follows:

$q_1=\frac{\sqrt{1+r_{11}+r_{22}+r_{33}}}{2},q_2=\frac{r_{23}-r_{32}}{4q_1},q_3=\frac{r_{31}-r_{13}}{4q_1},q_4=\frac{r_{12}-r_{21}}{4q_1}$

$q_2=\frac{\sqrt{1+r_{11}-r_{22}-r_{33}}}{2},q_1=\frac{r_{23}-r_{32}}{4q_2},q_3=\frac{r_{12}+r_{21}}{4q_2},q_4=\frac{r_{31}-r_{13}}{4q_2}$

(10)

$q_3=\frac{\sqrt{1-r_{11}+r_{22}-r_{33}}}{2},q_1=\frac{r_{31}-r_{13}}{4q_3},q_2=\frac{r_{12}-r_{21}}{4q_3},q_4=\frac{r_{23}-r_{32}}{4q_3}$

$q_4=\frac{\sqrt{1-r_{11}-r_{22}+r_{33}}}{2},q_1=\frac{r_{12}-r_{21}}{4q_4},q_2=\frac{r_{31}+r_{13}}{4q_4},q_3=\frac{r_{23}-r_{32}}{4q_4}$

New rotation matrix is calculated from formula (10):

$R=\left[\begin{array}{ccc}-1+2q_2^2+2q_1^2& 2q_2{q}_3-2q_1{q}_4& 2q_1{q}_3+2q_2{q}_4\\ 2q_2{q}_3+2q_4{q}_1& -1+2q_1^2+2q_3^2& -2q_1{q}_2+2q_3{q}_4\\ -2q_1{q}_3+2q_4{q}_2& 2q_1{q}_2+2q_3{q}_4& -1+2q_1^2+2q_4^2\end{array}\right]---\left(11\right)$

Due to hypercomplex number q ₁, q ₂, q ₃, q ₄quadratic sum equal one, therefore the parameter predigesting to be optimized of rotation matrix is three; Using two, the left and right ccd video camera calibrating parameters value of having tried to achieve as optimized initial value, optimized algorithm is utilized to carry out global optimization to video camera principal parameter and distortion parameter; Based on maximal possibility estimation criterion to the optimization of two, left and right ccd video camera principal parameter and distortion parameter, for given n m × n that scaling board picture provides calibration point coordinate, the optimization problem of two the ccd video camera parameters in left and right is expressed by the minimization problem of following formula:

$S\left(\mathrm{\θ}\right)=\underset{i=1}{\overset{n}{\mathrm{\Σ}}}\underset{i=1}{\overset{m}{\mathrm{\Σ}}}{\mathrm{\ϵ}}_i^2=\underset{i=1}{\overset{n}{\mathrm{\Σ}}}\underset{i=1}{\overset{m}{\mathrm{\Σ}}}{[y_i-m_i(C_j,p_j,X_i)]}^2---\left(12\right)$

Wherein j refers to the jth video camera participating in calculating, and i refers to i-th point that a jth video camera obtains, X _ithe spatial point coordinate of input, y _ithe image coordinate in i-th, space, C _jbe changeless camera parameters vector, its length is n ₀, p _jbe the camera parameters vector needing adjustment, its length is n ₁, n ₀+ n ₁be all parameter vector length of video camera, m _i(C _j, p _j, X _i) be the imaging equation of video camera; The present invention adopt solve above formula based on the light-stream adjustment of LM algorithm minimize optimization problem; Utilize light-stream adjustment write optimizer time some also will be provided except the calibrating parameters initial value of two ccd video cameras in left and right will be provided with the three dimensional space coordinate point of form and the respective image coordinate of these three-dimensional coordinate points, therefore in order to accurately obtain the angular coordinate of high-accuracy scaling board in whole view field space, the method that the feeding of numerically-controlled machine z-axis drives microscope to vertically move is adopted; And adopt laser interferometer (measuring accuracy can reach 0.1um) z-axis measurement to be obtained to the actual range of z-axis movement, ensure that the z of institute's acquisition angle point coordinate point is to realistic accuracy; In light-stream adjustment, the Jacobian matrix that the iteration factor of optimized algorithm is made up of the first order derivative of iteration parameter is formed, and therefore needs in computation process, introduce the Jacobian matrix adjusting parameter in equation to be imaged;

$J=[\frac{\mathrm{df}}{\mathrm{dp}\left(1\right)},\frac{\mathrm{df}}{\mathrm{dp}\left(2\right)},\·\·\·,\frac{\mathrm{df}}{\mathrm{dp}\left(n_1\right)}]---\left(13\right)$

Wherein, f is imaging equation, p (1) ... p (n ₁) be the camera parameters vector element needing adjustment;

Light-stream adjustment, with the Jacobian matrix of its parameter to be adjusted composition iteration factor, adjusts model parameter, obtains the model parameter optimum solution making image error S (θ) reach minimum;

(2) extraction of provincial characteristics point to be measured

What the extraction that the present invention is directed to provincial characteristics to be measured point adopted is Harris Corner Detection Algorithm; Harris proposes the algorithm replacing square region compute gradient with Gaussian function, if the pixel coordinate of impact point is x, y, it is respectively u and v in the displacement in x direction and y direction, ω (u, v) is window function, then the grey scale change of this impact point is represented by following formula:

$\begin{array}{c}E(x,y)=\underset{u,v}{\mathrm{\Σ}}\mathrm{\ω}(u,v){[I(x+u,y+v)-I(x,y)]}^2\\ \≈\underset{u,v}{\mathrm{\Σ}}\mathrm{\ω}(u,v)(u,v)\left[\begin{array}{cc}{I}_x^2& I_x{I}_y\\ I_x{I}_y& I_y^2\end{array}\right]\left(\begin{array}{c}u\\ v\end{array}\right)\end{array}---\left(14\right)$

Order $M(x,y)=\underset{u,v}{\mathrm{\Σ}}\mathrm{\ω}(u,v)\left[\begin{array}{cc}{I}_x^2& I_x{I}_y\\ I_x{I}_y& I_y^2\end{array}\right]$ Then:

$E(x,y)\≈(u,v)M(x,y)\left(\begin{array}{c}u\\ v\end{array}\right)---\left(15\right)$

The eigenvalue λ of compute matrix M (x, y) ₁, λ ₂if two eigenwerts are all larger, illustrate that the value on two orthogonal directionss of the gradation of image autocorrelation function of impact point is all comparatively large, then this point is unique point; After just extraction being carried out to unique point by Harris algorithm; Subpixel corner detecting algorithm is adopted to extract the higher angular coordinate of precision; For desirable angle point, the shade of gray direction of the pixel near it is all perpendicular to the line of this point and desirable angle point; This feature can be with equation expression:

$\▿\stackrel{\→}{H}(\stackrel{\→}{\mathrm{\α}}-\stackrel{\→}{\mathrm{\β}})=0---\left(16\right)$

Wherein the shade of gray direction of desirable angle point, vector points to the coordinate of desirable angle point for image origin, vector points to the coordinate of arbitrary marginal point near desirable angle point for image origin; In fact, owing to being subject to the impact of picture noise, formula (10) is usually all non-vanishing, can be regarded as error e, namely

$e=\▿\stackrel{\→}{H}(\stackrel{\→}{\mathrm{\α}}-\stackrel{\→}{\mathrm{\β}})---\left(17\right)$

In the neighborhood centered by angle point, all above formulas of pressing are calculated, error and be E, then

$E=\underset{i}{\mathrm{\Σ}}e=\underset{i}{\mathrm{\Σ}}\▿\stackrel{\→}{H}(\stackrel{\→}{\mathrm{\α}}-\stackrel{\→}{\mathrm{\β}})---\left(18\right)$

Wherein, i is i-th point in neighborhood; Ask like this and make error and the minimum point of E namely be other angular coordinate of sub-pixel;

(3) Stereo matching of unique point

Adopt the first coupling of normalization crossover algorithm (NCC) realization character point, the picture clapped using ccd video camera 2 as reference base picture, and with unique point p _lcentered by, construct an image pixel N*N, topography's block as masterplate figure T, and make masterplate figure T travel through in the certain limit including unique point of image S to be matched, the subgraph that search template covers is denoted as S ^i,j, i, j are the pixel coordinate of subgraph central point in matching image S, calculate masterplate figure T and search subgraph S by using normalization crossover algorithm (NCC) ^i,jbetween normalized-cross-correlation function, algorithm is as follows:

$R(i,j)=\frac{\underset{m,n}{\mathrm{\Σ}}|S^{i,j}(m,n)-E\left(S^{i,j}\right)||T(m,n)-E\left(T\right)|}{\sqrt{\underset{m,n}{\mathrm{\Σ}}{(S^{i,j}(m,n)-E\left(S^{i,j}\right))}^2\underset{m,n}{\mathrm{\Σ}}{(T(m,n)-E\left(T\right))}^2}}---\left(19\right)$

Wherein: m, n represent the coordinate of each pixel, T (m, n) is the gray-scale value at some m, n place,

E (S ^i,j) and E (T) be search subgraph S respectively ^i,jwith the average gray of masterplate figure T; The value of cross-correlation coefficient R (i, j) is larger, then the matching degree of searching for subgraph and Prototype drawing is higher; After the threshold value of setting cross-correlation coefficient R (i, j), select unique point p _lcandidate matches point; Also need after using NCC algorithm to carry out first coupling to utilize outer limit restraint and the feature point pairs of distance restraint to coupling to correct; Normalization 8 algorithms proposed by using Longguet-Higgins calculate the fundamental matrix F of two ccd video cameras in left and right when camera calibration, theoretical according to EP point, to benchmark image unique point p _l, EP point l corresponding in image S to be matched _rcan be expressed as:

l _r＝F·p _l(20)

If the candidate feature point of first coupling is at EP point l _rnear then can determine that this candidate feature point is match point further; Finally to add up in each candidate feature point to the Distance geometry benchmark image of other unique points unique point to the equidistant number num of other unique point points; Candidate feature point corresponding during num maximal value is then the correct match point of unique point in benchmark image;

(4) the asking for of three-dimensional coordinate

First calculate the image coordinate value that in binocular image, unique point extracts through sub-pixel recognition, and roughly estimate the estimated value of unique point in world coordinate system; In optimizer, corresponding for two, left and right ccd video camera parameter Jacobian matrix is set to zero, calculates in iteration factor part the Jacobian matrix equation embedded about volume coordinate x, y, z simultaneously at optimizer; By two, the left and right ccd video camera parameter optimized and unique point dot image coordinate figure and world coordinates estimated value input beam method of adjustment optimizer, just can be optimized the three-dimensional coordinate of spatial point, obtain the three-dimensional space measurement value of unique point, thus complete asking for and the location of final realization character point of three-dimensional coordinate point.