CN1508511A - Method for calibrating structure optical vision sensor - Google Patents

Abstract

The invention refers to the improvement of sensor scaling method in structured light three dimension vision measurement. The steps are: sets target mark, fixes sensor, collects image, gets target mark coordinate, irradiates structured light, gets the coordinate of light, gets the coordinate at any spot of the structured light strap. Calculates the three dimension object coordinate, calculate R and t, and calculates a, b, c, d.

Description

A kind of scaling method of structured light vision sensor

Technical field

The invention belongs to field of measuring technique, relate to improvement transducer calibration method in the structured light three-dimensional vision detection.

Background technology

In many visible detection methods, structured light three-dimensional vision detects and is widely used in the integrality of workpiece, the measurement of surface smoothness; The automatic detection of microelectronic component (IC chip, PC plate, BGA) etc.; The detection of soft, frangible parts; The detection of various mould 3D shapes; Robotic vision guiding etc.The structured light vision detection technology with its wide range, big visual field, measuring speed is fast, optical strip image is easy to extract and characteristics such as degree of precision have obtained using more and more widely in industrial environment.Effective scaling method of structured light vision sensor model parameter is an important research contents always.Mainly contain following method at present: the one, sawtooth target method.Duan Fajie etc. are at article " a kind of novel line structure optical sensor structural parameters calibration method " (Chinese journal of scientific instrument, Vol.21No.1,2000) and (measurement technology of " utilizing the new method of profile of tooth target calibration line structured light sensor structural parameters " at article such as Liu Fengmei, No7.pp.3～6,1999) stated this method in.The principle of this method is seen Fig. 1.This method mainly is to allow optical plane project on the jagged target, thereby forms some bright spots as calibration point on tip surface.There are the following problems for this kind method: the one, and easily reflective because of tip surface, cause the extraction precision of picture point lower.The 2nd, because of a kind of light distribution of bright spot performance itself, very difficult strict corresponding with the bright spot in the image with the bright spot that electronic theodolite aims in the space.The 3rd, limited because of tip surface, the calibration point number that obtains is few.Another kind method is a not political reform of double ratio.Nineteen ninety-five, the Xu Guang Yu of Tsing-Hua University etc. are at article " a kind of new three dimensional vision system scaling method based on structured light " (Chinese journal of computers, Vol.18 No.6,1995) proposed to utilize a double ratio principle of invariance to obtain the method for calibration point, but realize too trouble.1999, D.Q.HUYNH is at article " demarcation of line-structured light system: a kind of method of novelty " (Calibration a Structured Light Stripe System:A Novel Approach), the international periodical of computer vision, the 33rd volume, the 1st phase, 73～86 pages, (InternationalJournal of Computer Vision in 1999, Vol.33 No.1, pp.73～86,1999) a kind of calibration point acquisition methods that utilizes a double ratio principle of invariance has also been proposed, this method has been used 4 groups of non-coplanar points, every group 3 and conllinear, the three-dimensional coordinate that utilizes double ratio unchangeability to obtain 4 points on the optical plane by these 12 points is used for the demarcation of line structured light vision sensor.But it is also less that the two common problem is a calibration point quantity, so stated accuracy is still waiting to improve

Summary of the invention

Technical matters to be solved by this invention is: a kind of precision height is provided, is fit to the structured light vision sensor scaling method of on-site proving, further improve its stated accuracy, improve the convenience that its through engineering approaches is used.

Technical solution of the present invention is: a kind of scaling method of structured light vision sensor, it is characterized in that,

(1) set target, target is made up of two planes, and its angle theta is 45 °≤θ≤135 °, on two target faces the unique point that sets in advance is arranged, and the target face is one of following structure:

A, target first have the rectangular block of the black of two row triplex row projectioies on the target face, the quantity of rectangular block is 2～100 in every row, and the spacing of rectangular block is 10～200mm, and four summits of each rectangular block are unique point;

B, target second have into the circular hole of arranged on the target face, the line number of circular hole and columns are 3～100, and the center of adopting circular hole is as unique point;

C, target the third have into the crosshair of arranged on the target face, crosshair line number and columns are 3～100, with the point of crossing of crosshair as unique point;

(2) sensor and target are fixed at a distance of certain distance, should guarantee that the optical plane of laser instrument emission in the sensor and two target faces intersect;

(3) open the power supply of ccd video camera in the sensor, gather the piece image of demarcating target, store in the computing machine;

(4), extract the two-dimensional image coordinate (X on all summits of black square on the target for the target first _i, Y _i) and corresponding three-dimensional article coordinate (x _Wi, y _Wi, z _Wi), store in the computing machine;

(5) open the power supply of laser instrument, make the structured light optical plane of projection and two Plane intersects of target form two-strip structure light striation;

(6) the two-dimensional image coordinate (X of the intersection point at black square edge on extraction structural light strip 5 and the target body _Li, Y _Li), utilize double ratio formula r (A one time _i, B _i, C _i, D _i)=r ' (A _i', B _i', C _i', D _i') calculate as coordinate (X _Li, Y _Li) pairing three-dimensional article coordinate (x _WLi, y _WLi, z _WLi) and storage, r (A in the formula _i, B _i, C _i, D _i) expression A _i, B _i, C _i, D _i4 double ratio, r ' (A _i', B _i', C _i', D _i') expression A _i', B _i', C _i', D _i4 double ratio;

(7) on structural light strip, get arbitrarily a bit, extract it as coordinate (X _Li, Y _Li), the three-dimensional article coordinate (x that tries to achieve in conjunction with double ratio _WLi, y _WLi, z _WLi), utilize dual double ratio invariant r (D ₁, D ₂, D ₃, D ₄)=r ' (D ₁', D ₂', D ₃', D ₄') ask for this point the three-dimensional article coordinate and the storage;

(8) work of repetition (7) can obtain three-dimensional article coordinate and the corresponding two-dimensional image coordinate thereof put on the structured light optical plane of required any amount;

(9) utilize the summit that obtains in (4), to formula

$\left[\begin{array}{c}\mathrm{\ω}\·X\\ \mathrm{\ω}\·Y\\ \mathrm{\ω}\end{array}\right]=\left[\begin{array}{cccc}f& 0& 0& 0\\ 0& f& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{cc}R& t\\ 0^T& 1\end{array}\right]\·\left[\begin{array}{c}{x}_w\\ y_w\\ z_w\\ 1\end{array}\right]$ Demarcate, obtain R and t, ω represents a proportionality constant in the formula, and f represents the effective focal length of ccd video camera, and R is 3 * 3 rotation matrix, and t is 3 * 1 translation vector;

(10) point on the structured light optical plane that obtains in utilization (6), (7), (8) is to following formula ax _w+ by _w+ cz _w+ d=0 demarcates, and obtains a, b, c, d, (a, b, c, d) coefficient and the constant term of expression optic plane equations in the formula.

The advantage of the inventive method is: this method is based on dual double ratio invariance principle, structured light vision sensor is demarcated, compare with the scaling method of existing structured light vision sensor, calibration point precision height, the quantity of this method can be many arbitrarily, and the measuring accuracy height of three sensors of demarcation can reach 0.1453mm, 0.1524mm and 0.1496mm respectively.This method is fit to on-site proving.

Description of drawings

Fig. 1 is existing sawtooth target method principle schematic.Among the figure, the 1st, sawtooth target, the 2nd, laser projecting apparatus, the 3rd, ccd video camera.

Fig. 2 is a double ratio invariance principle synoptic diagram.

Fig. 3 is a kind of demarcation target of the inventive method structural representation.4 is to demarcate target body, the 5th, structural light strip among the figure.

Fig. 4 is that the inventive method is demarcated the target structural representation for second kind.6 is to demarcate target body, the 7th, structural light strip among the figure.

Fig. 5 is the third demarcation target structural representation of the inventive method.8 is to demarcate target body, the 9th, structural light strip among the figure.

Fig. 6 is the pictorial diagram of structured light vision sensor, and 10 is sensor housings among the figure, the 11st, and CCD form, the 12nd, laser emitting mouth.

Embodiment

Below the inventive method is described in further details.The inventive method based on dual double ratio invariance principle, is demarcated structured light vision sensor first.This method can obtain the high accuracy three-dimensional object point coordinate of required any amount on the structured light optical plane, is used for the demarcation of sensor.The constant principle of the dual double ratio of brief description at first.Under perspective projection transformation, the ratio between length and the length can change, but two have unchangeability about the ratio between the ratio of length.As shown in Figure 2, plane π ₁On three non-coincidence straight line A are arranged _iB _iC _i(i=1,2,3), straight line D ₁D ₂D ₃Meet at a D respectively with these three straight lines ₁, D ₂, D ₃By perspective projection center o, they are at plane π ₂On picture be respectively A _i' B _i' C _i' and D ₁' D ₂' D ₃'.According to the perspective projection theorem, straight line still is a straight line through perspective projection transformation.Therefore put A _i', B _i', C _i', D _i' (i=1,2,3) conllinear.The double ratio that conllinear is 4 is defined as: $r(A,B,C,D)=\frac{\mathrm{AC}}{\mathrm{CB}}/\frac{\mathrm{AD}}{\mathrm{DB}}---\left(1\right)$

According to double ratio invariance principle under the perspective projection transformation, have following formula to set up:

r(A _i，B _i，C _i，D _i)＝r′(A _i′，B _i′，C _i′，D _i′) (2)

At an A _i, B _i, C _iCoordinate and the some A _i', B _i', C _i', D _i' the known condition of coordinate under, utilize formula (2) can obtain D _iCoordinate.Can obtain a D respectively like this ₁, D ₂, D ₃Coordinate.

At straight line D ₁D ₂D ₃Take up an official post and get 1 D ₄, the perspective projection point that corresponding O is ordered is D ₄'.Utilize the double ratio invariance principle once more, have:

r(D ₁，D ₂，D ₃，D ₄)＝r′(D ₁′，D ₂′，D ₃′，D ₄′) (3)

By the constant D that obtains of (2) double ratios of formula ₁, D ₂, D ₃Coordinate, at its corresponding picture point D ₁', D ₂', D ₃' coordinate and the some D ₄' the known condition of coordinate under, then utilize double ratio invariant (3) can obtain D once more ₄Coordinate.The rest may be inferred, can obtain straight line D ₁D ₂D ₃Go up the coordinate of any arbitrarily.This paper calls dual double ratio unchangeability to the above-mentioned characteristic of double ratio.

The mathematical model of structured light vision sensor.

The perspective projection model of the mathematics of structured light vision sensor is made up of two parts: the perspective projection imaging model of video camera and structured light optic plane equations.

According to video camera pin-hole imaging principle and perspective projection transformation principle, the perspective projection imaging model of video camera can be described below:

$\left[\begin{array}{c}\mathrm{\ω}\·X\\ \mathrm{\ω}\·Y\\ \mathrm{\ω}\end{array}\right]=\left[\begin{array}{cccc}f& 0& 0& 0\\ 0& f& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{cc}R& t\\ 0^T& 1\end{array}\right]\·\left[\begin{array}{c}{x}_w\\ y_w\\ z_w\\ 1\end{array}\right]---\left(4\right)$

Wherein $R=\left[\begin{array}{ccc}{r}_1& r_2& r_3\\ r_4& r_5& r_6\\ r_7& r_8& r_9\end{array}\right]$ Be rotation matrix, $t=\left[\begin{array}{c}{t}_x\\ t_y\\ t_z\end{array}\right]$ Be translation vector, ω is a scale factor, is a constant.

The equation of optic plane equations can be described as: ax _w+ by _w+ cz _w+ d=0 (5)

Formula (4) and formula (5) have constituted the complete mathematical model of structured light vision sensor one.According to this model, the demarcation of structured light vision sensor was divided into for two steps: the one, and the demarcation of video camera imaging model parameter R and t, the 2nd, the demarcation of structured light optic plane equations.The R and the t that find the solution in the formula (4) need 6 non-coplanar three-dimensional world points and corresponding two-dimentional picture point thereof at least; Determine that formula (5) needs 3 non-colinear three-dimensional object points on the optical plane at least.In order to improve the precision of demarcation, often need to obtain more high-precision calibrating point.

The concrete steps of the inventive method are as follows:

(1) set target, target is made up of two planes, and its angle theta is 45 °≤θ≤135 °, on two target faces the unique point that sets in advance is arranged, and the target face is one of following structure:

A, target first have the rectangular block of the black of two row triplex row projectioies on the target face, the quantity of rectangular block is 2～100 in every row, and the spacing of rectangular block is 10～200mm, and four summits of each rectangular block are unique point;

B, target second have into the circular hole of arranged on the target face, the line number of circular hole and columns are 3～100, and the center of adopting circular hole is as unique point;

C, target the third have into the crosshair of arranged on the target face, the line number of crosshair and columns are 3～100, with the point of crossing of crosshair as unique point;

(2) sensor and target are fixed at a distance of certain distance, should guarantee that the optical plane of laser instrument emission in the sensor and two target faces intersect;

(3) open the power supply of ccd video camera in the sensor, gather the piece image of demarcating target, store in the computing machine;

(4), extract the two-dimensional image coordinate (X on all summits of black square on the target for the target first _i, Y _i) and corresponding three-dimensional article coordinate (x _Wi, y _Wi, z _Wi), store in the computing machine;

(5) open the power supply of laser instrument, make the structured light optical plane of projection and two Plane intersects of target form two-strip structure light striation;

(6) the two-dimensional image coordinate (X of the intersection point at black square edge on extraction structural light strip 5 and the target body _Li, Y _Li), utilize double ratio formula r (A one time _i, B _i, C _i, D _i)=r ' (A _i', B _i', C _i', D _i') calculate as coordinate (X _Li, Y _Li) pairing three-dimensional article coordinate (x _WLi, y _WLi, z _WLi) and storage, r (A in the formula _i, B _i, C _i, D _i) expression A _i, B _i, C _i, D _i4 double ratio, r ' (A _i', B _i', C _i', D _i') expression A _i', B _i', C _i', D _i4 double ratio;

(7) on structural light strip, get arbitrarily a bit, extract it as coordinate (X _Li, Y _Li), the three-dimensional article coordinate (x that tries to achieve in conjunction with double ratio _WLi, y _WLi, z _WLi), utilize dual double ratio invariant r (D ₁, D ₂, D ₃, D ₄)=r ' (D ₁', D ₂', D ₃', D ₄') ask for this point the three-dimensional article coordinate and the storage;

(8) work of repetition (7) can obtain three-dimensional article coordinate and the corresponding two-dimensional image coordinate thereof put on the structured light optical plane of required any amount;

(9) utilize the summit that obtains in (4), to formula

$\left[\begin{array}{c}\mathrm{\ω}\·X\\ \mathrm{\ω}\·Y\\ \mathrm{\ω}\end{array}\right]=\left[\begin{array}{cccc}f& 0& 0& 0\\ 0& f& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{cc}R& t\\ 0^T& 1\end{array}\right]\·\left[\begin{array}{c}{x}_w\\ y_w\\ z_w\\ 1\end{array}\right]$ Demarcate, obtain R and t, ω represents a proportionality constant in the formula, and f represents the effective focal length of ccd video camera, and R is 3 * 3 rotation matrix, and t is 3 * 1 translation vector;

(10) point on the structured light optical plane that obtains in utilization (6), (7), (8) is to following formula ax _w+ by _w+ cz _w+ d=0 demarcates, and obtains a, b, c, d, (a, b, c, d) coefficient and the constant term of expression optic plane equations in the formula.

Embodiment

The material object of the structured light vision sensor of actual design as shown in Figure 6.10 is the housing of sensor, and 11 is the window of CCD observing scene, and 12 is the exit portal of laser instrument.The position of CCD and laser instrument is different in the sensor, and its parameter is also different.

According to the step of narrating above, utilize target shown in Figure 3 (θ=90 °) that three concrete sensors are demarcated.

Sensor one:

Camera perspective projection imaging model parameter

$R=\left[\begin{array}{ccc}0.135832& 0.171788& 0.975725\\ 0.843045& -0.537361& -0.022753\\ 0.520408& 0.825670& -0.217816\end{array}\right]t=\left[\begin{array}{c}-61.97129\\ 8.464447\\ 601.586335\end{array}\right]$

The structured light plane equation coefficient

[a?b?c?d]＝[-0.000769?0.056868?-0.584231?10.404217]。

The reperformance test error of the sensor reconstruct three-dimensional article coordinate of this demarcation is respectively:

$3{\mathrm{\σ}}_{x_w}=0.021\mathrm{mm},3{\mathrm{\σ}}_{y_w}=0.0345\mathrm{mm},3{\mathrm{\σ}}_{z_w}=0.0033\mathrm{mm}$

The RMS error of distance between two points is in the measurement space: E _RMS=0.1453mm.

Sensor two:

Camera perspective projection imaging model parameter:

$R=\left[\begin{array}{ccc}0.212029& 0.151030& 0.965522\\ 0.713005& -0.699573& -0.047147\\ 0.668332& 0.698418& -0.256015\end{array}\right]$

$t=\left[\begin{array}{c}-68.096325\\ 6.559659\\ 609.599409\end{array}\right]$

The structured light plane equation coefficient

[a?b?c?d]＝[-0.00848?10.022128?-0.271455?7.8725557]。

The reperformance test error of the sensor reconstruct three-dimensional article coordinate of this demarcation is respectively:

$3{\mathrm{\σ}}_{x_w}=0.027\mathrm{mm},3{\mathrm{\σ}}_{y_w}=0.0445\mathrm{mm},3{\mathrm{\σ}}_{z_w}=0.0036\mathrm{mm}$

The RMS error of distance between two points is in the measurement space: E _RMS=0.1524mm

Sensor three:

Camera perspective projection imaging model parameter

$R=\left[\begin{array}{ccc}0.308163& 0.227037& -0.923907\\ -0.665353& 0.745312& -0.038520\\ -0.679944& -0.626832& -0.380535\end{array}\right]$

$t=\left[\begin{array}{c}47.889089\\ 0.686364\\ 583.643174\end{array}\right]$

The structured light plane equation coefficient

[α?b?c?d]＝[-0.013647?-0.013292?-0.190761?19.329410]。

The reperformance test error of the sensor reconstruct three-dimensional article coordinate of this demarcation is respectively:

$3{\mathrm{\σ}}_{x_w}=0.032\mathrm{mm},3{\mathrm{\σ}}_{y_w}=0.0313\mathrm{mm},3{\mathrm{\σ}}_{z_w}=0.00428\mathrm{mm}$

The RMS error of distance between two points is in the measurement space: E _RMS=0.1496mm.

Claims (1)

1, a kind of scaling method of structured light vision sensor is characterized in that,

(1) set target, target is made up of two planes, and its angle theta is 45 °≤θ≤135 °, on two target faces the unique point that sets in advance is arranged, and the target face is one of following structure:

A, target first have the rectangular block of the black of two row triplex row projectioies on the target face, the quantity of rectangular block is 2～100 in every row, and the spacing of rectangular block is 10～200mm, and four summits of each rectangular block are unique point;

B, target second have into the circular hole of arranged on the target face, the line number of circular hole and columns are 3～100, and the center of adopting circular hole is as unique point;

C, target the third have into the crosshair of arranged on the target face, crosshair line number and columns are 3～100, with the point of crossing of crosshair as unique point;

(2) sensor and target are fixed at a distance of certain distance, should guarantee that the optical plane of laser instrument emission in the sensor and two target faces intersect;

(3) open the power supply of ccd video camera in the sensor, gather the piece image of demarcating target, store in the computing machine;

(4), extract the two-dimensional image coordinate (X on all summits of black square on the target for the target first _i, Y _i) and corresponding three-dimensional article coordinate (x _Wi, y _Wi, z _Wi), store in the computing machine;

(5) open the power supply of laser instrument, make the structured light optical plane of projection and two Plane intersects of target form two-strip structure light striation;

(6) the two-dimensional image coordinate (X of the intersection point at black square edge on extraction structural light strip 5 and the target body _Li, Y _Li), utilize double ratio formula r (A one time _i, B _i, C _i, D _i)=r ' (A _i', B _i', C _i', D _i') calculate as coordinate (X _Li, Y _Li) pairing three-dimensional article coordinate (x _WLi, y _WLi, z _WLi) and storage, r (A in the formula _i, B _i, C _i, D _i) expression A _i, B _i, C _i, D _i4 double ratio, r ' (A _i', B _i', C _i', D _i') expression A _i', B _i', C _i', D _i4 double ratio;

(7) on structural light strip, get arbitrarily a bit, extract it as coordinate (X _Li, Y _Li), the three-dimensional article coordinate (x that tries to achieve in conjunction with double ratio _WLi, y _WLi, z _WLi), utilize dual double ratio invariant r (D ₁, D ₂, D ₃, D ₄)=r ' (D ₁', D ₂', D ₃', D ₄') ask for this point the three-dimensional article coordinate and the storage;

(8) work of repetition (7) can obtain three-dimensional article coordinate and the corresponding two-dimensional image coordinate thereof put on the structured light optical plane of required any amount;

(9) utilize the summit that obtains in (4), to formula

$\left[\begin{array}{c}\mathrm{\ω}\·X\\ \mathrm{\ω}\·Y\\ \mathrm{\ω}\end{array}\right]=\left[\begin{array}{cccc}f& 0& 0& 0\\ 0& f& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{cc}R& t\\ 0^T& 1\end{array}\right]\·\left[\begin{array}{c}{x}_w\\ y_w\\ z_w\\ 1\end{array}\right]$ Demarcate, obtain R and t, ω represents a proportionality constant in the formula, and f represents the effective focal length of ccd video camera, and R is 3 * 3 rotation matrix, and t is 3 * 1 translation vector;

(10) point on the structured light optical plane that obtains in utilization (6), (7), (8) is to following formula ax _w+ by _w+ cz _w+ d=0 demarcates, and obtains a, b, c, d, (a, b, c, d) coefficient and the constant term of expression optic plane equations in the formula.

Images