CN101667303B - Three-dimensional reconstruction method based on coding structured light - Google Patents

Abstract

The invention discloses a three-dimensional reconstruction method based on coding structured light, comprising the following steps: 1) projecting structured light to an object to be measured, and capturing an image modulated by the object to be measured by a camera; 2) matching an optical template, comprising: (2.1) positioning the optical strip boundary, scanning along each row of the image, determining a pixel point with strong gray variation as a candidate marginal point, and searching a local domain; and (2.2) matching the optical strip: adopting a color cluster method to build a color matching proper vector, comparing image color with a projected color, and defining Euclidean distance between the color proper vector and the cluster center to distribute the colors of red, green, blue and white of the candidate optical strip; and 3) using a calibrated system parameter for three-dimensional reconstruction of the object to be measured, determining the relation between a space point coordinate and the image coordinate point thereof by the calibrated conversion matrix parameter; and restoring three-dimensional spatial coordinate from the image coordinate of a feature point. The invention can simplify calculation process and has high matching precision and high reconstruction precision.

Description

A kind of three-dimensional rebuilding method based on coded structured light

Technical field

The present invention relates to computer vision, data processing, image processing, especially a kind of three-dimensional rebuilding method based on coded structured light.

Background technology

The purpose of computer vision is that input is expressed and understood to various images.Nearly two during the last ten years, and the object three-dimensional contour outline reconstruction is one of computer vision free-revving engine always.The implementation method of three-dimensional object profile reconstruct and measuring technique has a lot, mainly comprises contact and contactless two large classes.Non-cpntact measurement mainly is divided into two classes, and a class is optical means, and another kind of is other outer method of optics.

Optical method for measuring can be divided into two classes again: active and passive type.The former is that imaging device does not send signal by object radiation signal or body surface reflected signal synthetic image; The latter sends a branch of signal by imaging device, by receiving signal synthetic image that reflect from body surface or that penetrate object.Human eye three-dimensional perception principle is mainly simulated in the measurement of passive type, and is similar with human vision take the traditional double item stereo vision as representative, mainly is the superficial makings by target object, and half-tone information, colouring information obtain the three-dimensional appearance of target object.The passive vision method is taken two width of cloth pictures for same scene at least from different angles, and perhaps at least two viewpoint positions are taken.Unique point chooses and mates the problem that passive vision can't reliably solve always between the picture, because Feature Points Matching has regionality, it just mates for the similar area of target object, is difficult to be accurate to the point on the target object, the degree of accuracy of coupling is not high, and reconstruction precision is lower; Secondly, produce easily the mistake coupling for the coupling of the unconspicuous target object of unique point, cause the erroneous point of rebuilding more, calculation of complex, this is the problem that stereoscopic vision itself can't overcome.Therefore, stereoscopic vision can't satisfy the needs of practical application.

Active measuring method fly if having time method, phase measurement, structured light method and digital hologram method etc.Wherein, the structured light method with its intrinsic noncontact, be easy to realize and the higher advantages such as precision, be subject in recent years increasing attention.Especially since entering the eighties of last century the nineties, along with developing rapidly of industrial detection technology, reverse Engineering Technology and rapid shaping technique, the demand of three-dimensional body being carried out surface profile reconstruct and measurement is increasing, more and more higher to measuring speed and accuracy requirement, so that become the most widely used method based on the structured light method of principle of triangulation.

Summary of the invention

For the calculation of complex of the three-dimensional rebuilding method that overcomes existing Vision imaging system, the deficiency that matching precision is not high, reconstruction precision is low, the invention provides a kind of computation process, matching precision is high, reconstruction precision is high three-dimensional rebuilding method based on coded structured light simplified.

The technical solution adopted for the present invention to solve the technical problems is:

A kind of three-dimensional rebuilding method based on coded structured light may further comprise the steps:

1), to determinand projective structure light, video camera catches the image through the determinand modulation;

2), carry out optical mode plate coupling, may further comprise the steps:

(2.1), the location on striation border:

Optical mode plate to be detected is one dimension striped coding, each column scan along image, each passage to each pixel carries out pre-service with one dimension row operator, shown in Figure 8 in one dimension operator such as the accompanying drawing, determine that the strong pixel of grey scale change is candidate marginal, each lists centered by candidate marginal at image, chooses a certain size zone, its maximal value is no more than 1/2 of striation width, and the step of searching for local field is as follows:

(2.1.1). the initialization candidate marginal is as a regional center z _{I, j}

(2.1.2). along the column direction search, with z _{I-k, j}, z _{I+k, j}The pixel color space changes into tone, brightness and saturation degree (HIS) space;

(2.1.3). increase new point to this zone, as long as satisfy the threshold value H that is no more than setting with the tone value difference of neighbor pixel

h＝|z _i±k，j，h-z _{i±k±1，j，h}| (1)

H is the difference of two pixel tones, z _{I ± k, j, h}The tone value that represents the capable j row of the i ± k pixel; After operator was processed, the triple channel gray-scale value of pixel was G _Ij, the peaked pixel of regional area triple channel gray-scale value sum is defined as frontier point;

$\max \left(E\right)=\underset{j=1}{\overset{C}{\mathrm{\Σ}}}{G}_{\mathrm{ij}}(C=3)---\left(2\right)$

E is the maximal value of pixel gray-scale value sum after operator is processed;

(2.2), striation coupling: adopt the method for color cluster to set up the color-match proper vector, color of image and projection color are compared, definition color character vector distributes candidate's striation red, green, blue and white color with the Euclidean distance of cluster centre;

Different color groups is collected at specific rgb space zone, and then the cluster centre by sample point training color group mates the striation color with the distance of point-to-point;

D _ik＝d ²(P _i，C _k)(k＝1，2，3，4) (3)

D _IkBe 2 distances square, P _iThe RGB proper vector of striation center pixel, C _kBe the color cluster center, d (x, y) is x, the y distance between two points; If D=min is (D _Ik), the distribution of striation color is carried out according to (4) formula so:

s _c＝{k if D＝＝D _ik(k＝1，2，3，4) (4)

s _cExpression striation color;

After distributing color, per three adjacent striation colors form a code word s _i, find out each code word and decode exactly in the position of whole sequence; Striation coordinate figure x _iBe defined as:

$x_i=\mathrm{DdBS}\left(s_i\right)+\frac{1}{4}(j-1)---\left(5\right)$

DdBS (s _i) expression directly decodes j frame optical mode plate subsequence s _iCoordinate position;

3), utilize the systematic parameter demarcated to carry out the three-dimensional reconstruction of determinand, detailed process is:

Demarcate good transition matrix parameter and determined the relation of spatial point coordinate and its image coordinate point, recover 3 d space coordinate from the image coordinate of unique point; Take modulated image, after image is processed, obtain image coordinate

Calculate desirable normalization coordinate (x, y), there are following relation in spatial point and space under the camera coordinate system:

$\left[\begin{array}{c}{X}_c\\ Y_c\\ Z_c\end{array}\right]=R\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\end{array}\right]+t=\left[\begin{array}{c}{r}_1\\ r_2\\ r_3\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\end{array}\right]+\left[\begin{array}{c}{t}_1\\ t_2\\ t_3\end{array}\right]---\left(6\right)$

Definition X _c=xZ _c, Y _c=yZ _c, bring (6) into and get:

$\left[\begin{array}{c}{\mathrm{xZ}}_c\\ {\mathrm{yZ}}_c\\ Z_c\end{array}\right]=\left[\begin{array}{c}{r}_1\\ r_2\\ r_3\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\end{array}\right]+\left[\begin{array}{c}{t}_1\\ t_2\\ t_3\end{array}\right]---\left(7\right)$

Cancellation Z _c, abbreviation gets:

$\left[\begin{array}{c}{\mathrm{xt}}_3-{\mathrm{<figure-callout\; id="1"\; label="t"\; filenames="H2009101536034E00022.png,H2009101536034E00042.png"\; state="\{\{state\}\}">t</figure-callout>}}_1\\ {\mathrm{yt}}_3-{\mathrm{<figure-callout\; id="2"\; label="t"\; filenames="H2009101536034E00022.png,H2009101536034E00042.png"\; state="\{\{state\}\}">t</figure-callout>}}_2\end{array}\right]=\left[\begin{array}{c}{r}_1-{\mathrm{xr}}_3\\ r_2-{\mathrm{yr}}_3\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\end{array}\right]---\left(8\right)$

Obtain x by the striation decoding _dCoordinate figure:

$x_d{m}_{24}-m_{14}=[m_1-x_d{m}_2]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\end{array}\right]---\left(9\right)$

Get in conjunction with (8) formula, (9) formula:

$\left[\begin{array}{c}{\mathrm{xt}}_3-{\mathrm{<figure-callout\; id="1"\; label="t"\; filenames="H2009101536034E00022.png,H2009101536034E00042.png"\; state="\{\{state\}\}">t</figure-callout>}}_1\\ {\mathrm{yt}}_3-t_2\\ x_d{m}_{24}-m_{14}\end{array}\right]=\left[\begin{array}{c}{r}_1-{\mathrm{xr}}_3\\ r_2-{\mathrm{yr}}_3\\ m_1-x_d{m}_2\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\end{array}\right]---\left(10\right)$

Write (10) formula as matrix A X=B form, wherein X=(X _w, Y _w, Z _w), then equation linear least-squares solution is:

X＝(A ^TA) ^-1A ^TB (11)

X namely is the world coordinates that is obtained by three-dimensional computations by two dimensional image.

Further, in described step 2.1) in, the searching method of cluster centre is:

(2.1.1). initialization K=4, red, green, blue, white color class, initialization K group's center;

(2.1.2). each sample point is distributed to the group of the centre distance minimum that peels off;

(2.1.3) center of mass point among the .K group becomes new cluster centre;

(2.1.4). repeating step 2 and 3 until have a few and all tend to stably to assemble.

Further again, in described step 3) in, systematic parameter comprises camera parameters and projector parameter, scaling method may further comprise the steps:

3.1), the camera calibration process:

Adopt two-dimentional calibrating template, target is that the black and white chess lattice of standard form, unique point is that the point of chess lattice forms, video camera is taken the multiple image of diverse location, select the grid angle point as unique point to described multiple image, adopt the plane reference method that video camera is demarcated, obtain camera intrinsic parameter and demarcate K, and obtain the average focal length

It is characterized in that: described scaling method also comprises:

3.2), the projector calibrating process:

The optical mode plate that produces by coding projects on the common target, and black and white grid angle point utilizes the image that obtains as anchor point in camera calibration, extract simultaneously black and white grid angle point image coordinate, generates world coordinates;

During projection single white optical mode plate, obtain A ₁, B ₁2 image coordinate (u _A, v _A), (u _B, v _B), and world coordinates is (X _A, Y _A, Z _A), (X _B, Y _B, Z _B), ask straight line L _iIntersection point with L obtains P _iThe image coordinate of point is such as P ₁The image coordinate of point is (u _P, v _P), world coordinates is defined as (X _p, Y _P, Z _p), because A ₁, B ₁, P ₁3 coplanar, so Z _p=Z _A=Z _B, according to cross ratio invariability, obtain,

$\frac{X_A-X_P}{X_A-X_B}=\frac{v_A-v_P}{v_A-v_B},\frac{Y_A-Y_P}{Y_A-Y_B}=\frac{u_A-u_P}{u_A-u_B}---\left(12\right)$

Order

${\mathrm{\&lambda;}}_x=\frac{v_A-v_P}{v_A-v_B},$ ${\mathrm{\&lambda;}}_y=\frac{u_A-u_P}{u_A-u_B}---\left(13\right)$

Got by (12), (13) formula:

$\left\{\begin{array}{c}{X}_p=X_A+{\mathrm{\&lambda;}}_x(X_B-X_A)\\ Y_p=Y_A+{\mathrm{\&lambda;}}_y(Y_B-Y_A)\\ Z_P=Z_A=Z_B\end{array}\right.---\left(14\right).$

λ _x, λ _yThe double ratio coefficient that represents respectively X and Y-direction, (X _p, Y _P, Z _p) world coordinates of expression striation Edge Feature Points.

Further, in described step 3.2) in, the process of setting up of projector model is:

By projector coordinates x _dWith world coordinates X _wIt is as follows to set up model:

$\mathrm{\&gamma;}\left[\begin{array}{c}{\mathrm{<figure-callout\; id="1"\; label="x\; d"\; filenames="H2009101536034E00022.png,H2009101536034E00042.png"\; state="\{\{state\}\}">x</figure-callout>}}_d\\ 1\end{array}\right]=\left[\begin{array}{cc}{m}_1& m_{14}\\ m_2& m_{24}\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_{\mathrm{<figure-callout\; id="1"\; label="w\; Z\; w"\; filenames="H2009101536034E00022.png,H2009101536034E00042.png"\; state="\{\{state\}\}">w</figure-callout>}}& \\ Z_w{}_{}\\ 1\end{array}\right]=\left[\begin{array}{cccc}{m}_{11}& m_{12}& m_{13}& m_{14}\\ m_{21}& m_{22}& m_{23}& m_{24}\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]---\left(15\right)$

γ is arbitrary scale factor, eliminates γ, and conversion (15) formula obtains:

$\frac{m_1{\left[\begin{array}{ccc}{X}_w& Y_w& Z_w\end{array}\right]}^T+m_{14}}{m_2{\left[\begin{array}{ccc}{X}_w& Y_w& Z_w\end{array}\right]}^T+m_{24}}=x_d---\left(16\right)$

(16) formula of utilization structure is with m _IkBe the system of homogeneous linear equations of unknown number, then by Singular-value Decomposition Solution projector parameter;

Set up and simplify perspective cross ratio invariability geometric model, hypothesis space point A, B, C are positioned on the same straight line L, and take B as reference point, the position ratio is defined as:

PR(A，B，C)＝AB/AC (17)

In like manner, A ', B ', C ' they are under the O of centre of perspectivity effect, the imaging point of corresponding point A, B, C, and be positioned on the same point straight line L1, the position ratio on the image coordinate is:

PR(A′，B′，C′)＝A′B′/A′C′ (18)

Cross ratio invariability according to the perspective geometry principle obtains:

PR(A，B，C)＝PR(A′，B′，C′) (19)

Also be to be positioned on the same straight line L ' at image mid point A ', B ', C '.

Extract the image coordinate of unique point in the optical mode plate, obtain the world coordinates of reference point B according to (19) formula, set up space three-dimensional and come the labeling projection instrument to the mapping relations between the dimension coordinate.

Technical conceive of the present invention is: structured light three-dimensional vision is based on the triangulation principle of optics.Optical projection device (can be laser instrument, also can be projector) is projeced into the surface of object with the structured light (optical mode plate) of certain pattern, forms the striation 3-D view of being modulated by the testee surface configuration on the surface.This 3-D view is by the video camera picked-up that is in the another location, thus acquisition striation two dimension fault image, and the distortion degree of demodulation striation just can be reappeared the three-dimensional appearance of body surface.The structured light three-dimensional vision method can be divided into owing to projection light template pattern difference: dot structure light method, line-structured light method, multi-line structured light method, network light method etc., we mainly concentrate research multiline color coded structured light method, use chromatic information that striation is encoded, DeBruijn sequence optical mode plate for example, its matching problem then is obtained solution by RGB hyperchannel dynamic programming algorithm, and decoding coupling accuracy is high.

Beneficial effect of the present invention is mainly manifested in: simplify computation process, matching precision is high, reconstruction precision is high.

Description of drawings

Fig. 1 is the three-dimensional reconstruction processing flow chart.

Fig. 2 is the synoptic diagram of rgb space.

Fig. 3 is the sequence optical mode plate synoptic diagram that white waits wide interval.

Fig. 4 is that four frames are rotated counterclockwise 90 ° time shift optical mode plate synoptic diagram.

Fig. 5 is the synoptic diagram of edge detection operator.

Fig. 6 is operator edge detection result's synoptic diagram.

Fig. 7 is the synoptic diagram of the picture element scan passage gray-scale value of any row.

Fig. 8 is the one dimension row operator synoptic diagram of edge detection operator.

Fig. 9 is the synoptic diagram of striation Boundary Detection.

Figure 10 is the rgb space synoptic diagram of pixel color.

Figure 11 is perspective projection: some M projects to the synoptic diagram of the plane of delineation (u, v).

Figure 12 is the cross ratio invariability principle schematic.

Figure 13 is the synoptic diagram of high precision target.

Figure 14 is the synoptic diagram that reads in original uncalibrated image.

Figure 15 is the synoptic diagram of shooting labeling projection error.

Figure 16 is striation projection target face figure.

Figure 17 is the synoptic diagram of edge extracting and fitting a straight line.

Figure 18 is the synoptic diagram that unique point is obtained.

Figure 19 is projector calibrating error synoptic diagram.

Figure 20 is the synoptic diagram of calibration point three-dimensionalreconstruction.

Embodiment

The invention will be further described below in conjunction with accompanying drawing.

With reference to Fig. 1～Figure 20, a kind of three-dimensional rebuilding method based on coded structured light may further comprise the steps:

1), to determinand projective structure light, video camera catches the image through the determinand modulation;

2), carry out optical mode plate coupling, may further comprise the steps:

(2.1), the location on striation border:

Optical mode plate to be detected is one dimension striped coding, each column scan along image, each passage to each pixel carries out pre-service with one dimension row operator, shown in Figure 8 in one dimension operator such as the accompanying drawing, determine that the strong pixel of grey scale change is candidate marginal, each lists centered by candidate marginal at image, chooses a certain size zone, its maximal value is no more than 1/2 of striation width, and the step of searching for local field is as follows:

(2.1.1). the initialization candidate marginal is as a regional center z _{I, j}

(2.1.2). along the column direction search, with z _{I-k, j}, z _{I+k, j}The pixel color space changes into tone, brightness and saturation degree (HIS) space;

(2.1.3). increase new point to this zone, as long as satisfy the threshold value H that is no more than setting with the tone value difference of neighbor pixel

h＝z _i±k，j，h-z _{i±k±1，j，h} | (1)

H is the difference of two pixel tones, z _{I ± k, j, h}The tone value that represents the capable j row of the i ± k pixel; After operator was processed, the triple channel gray-scale value of pixel was G _Ij, the peaked pixel of regional area triple channel gray-scale value sum is defined as frontier point;

$\max \left(E\right)=\underset{j=1}{\overset{C}{\mathrm{\&Sigma;}}}{G}_{\mathrm{ij}}(C=3)---\left(2\right)$

E is the maximal value of pixel gray-scale value sum after operator is processed;

(2.2), striation coupling: adopt the method for color cluster to set up the color-match proper vector, color of image and projection color are compared, definition color character vector distributes candidate's striation red, green, blue and white color with the Euclidean distance of cluster centre;

Different color groups is collected at specific rgb space zone, and then the cluster centre by sample point training color group mates the striation color with the distance of point-to-point;

D _ik＝d ²(P _i，C _k)(k＝1，2，3，4) (3)

D _IkBe 2 distances square, P _iThe RGB proper vector of striation center pixel, C _kBe the color cluster center, d (x, y) is x, the y distance between two points; If D=min is (D _Ik), the distribution of striation color is carried out according to (4) formula so:

s _c＝{k if D＝＝D _ik(k＝1，2，3，4) (4)

s _cExpression striation color;

After distributing color, per three adjacent striation colors form a code word s _i, find out each code word and decode exactly in the position of whole sequence; Striation coordinate figure x _iBe defined as:

$x_i=\mathrm{DdBS}\left(s_i\right)+\frac{1}{4}(j-1)---\left(5\right)$

DdBS (s _i) expression directly decodes j frame optical mode plate subsequence s _iCoordinate position;

3), utilize the systematic parameter demarcated to carry out the three-dimensional reconstruction of determinand, detailed process is:

Demarcate good transition matrix parameter and determined the relation of spatial point coordinate and its image coordinate point, recover 3 d space coordinate from the image coordinate of unique point; Take modulated image, after image is processed, obtain image coordinate

Calculate desirable normalization coordinate (x, y), there are following relation in spatial point and space under the camera coordinate system:

$\left[\begin{array}{c}{X}_c\\ Y_c\\ Z_c\end{array}\right]=R\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\end{array}\right]+t=\left[\begin{array}{c}{r}_1\\ r_2\\ r_3\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\end{array}\right]+\left[\begin{array}{c}{t}_1\\ t_2\\ t_3\end{array}\right]---\left(6\right)$

Definition X _c=xZ _c, Y _c=yZ _c, bring (6) into and get:

$\left[\begin{array}{c}{\mathrm{xZ}}_c\\ {\mathrm{yZ}}_c\\ Z_c\end{array}\right]=\left[\begin{array}{c}{r}_1\\ r_2\\ r_3\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\end{array}\right]+\left[\begin{array}{c}{t}_1\\ t_2\\ t_3\end{array}\right]---\left(7\right)$

Cancellation Z _c, abbreviation gets:

$\left[\begin{array}{c}{\mathrm{xt}}_3-{\mathrm{<figure-callout\; id="1"\; label="t"\; filenames="H2009101536034E00022.png,H2009101536034E00042.png"\; state="\{\{state\}\}">t</figure-callout>}}_1\\ {\mathrm{yt}}_3-{\mathrm{<figure-callout\; id="2"\; label="t"\; filenames="H2009101536034E00022.png,H2009101536034E00042.png"\; state="\{\{state\}\}">t</figure-callout>}}_2\end{array}\right]=\left[\begin{array}{c}{r}_1-{\mathrm{xr}}_3\\ r_2-{\mathrm{yr}}_3\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\end{array}\right]---\left(8\right)$

Obtain x by the striation decoding _dCoordinate figure:

$x_d{m}_{24}-m_{14}=[m_1-x_d{m}_2]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\end{array}\right]---\left(9\right)$

Get in conjunction with (8) formula, (9) formula:

$\left[\begin{array}{c}{\mathrm{xt}}_3-{\mathrm{<figure-callout\; id="1"\; label="t"\; filenames="H2009101536034E00022.png,H2009101536034E00042.png"\; state="\{\{state\}\}">t</figure-callout>}}_1\\ {\mathrm{yt}}_3-t_2\\ x_d{m}_{24}-m_{14}\end{array}\right]=\left[\begin{array}{c}{r}_1-{\mathrm{xr}}_3\\ r_2-{\mathrm{yr}}_3\\ m_1-x_d{m}_2\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\end{array}\right]---\left(10\right)$

Write (10) formula as matrix A X=B form, wherein X=(X _w, Y _w, Z _w), then equation linear least-squares solution is:

X＝(A ^TA) ^-1A ^TB (11)

X namely is the world coordinates that is obtained by three-dimensional computations by two dimensional image.

The imaging system of present embodiment comprises experiment frame, video camera and projector.The effect of structure light imaging system is that projector is fixed on the base, and video camera is fixed on the cross bar that can move horizontally, and it is fixing that the relative position of projector and video camera can keep, and improves Systems balanth.Catch simultaneously the demand of visual field in order to satisfy different angles, the camera intrinsic horizontal rotating shaft can be regulated in 360 ° of scopes, also can adjust by the self-locking screw rod upright position of video camera on the vertical direction, the flexibility ratio that the structured-light system frame has, degree of freedom are obtained the freedom of scene greatly to improve Systems balanth.Video camera adopts the DH-SV1410FC/FM of Daheng type industrial CCD video camera, and image quality is high.The major parameter of video camera: the CCD sensing is of a size of 2/3 ", maximum frame per second 15fps, focal length of lens f=25mm, Pixel Dimensions 6.45 * 6.45um, resolution 1392 * 1040 (1447680 pixel).Adopt general happy scholar PLUS V-1100C digital projector as structured light projector, maximum display resolution is 1280 * 1024, focal distance f=23mm, but manual focusing two-way digital keystone, full color (1,677 ten thousand look), 6 inches to 200 inches of screen sizes, contrast 2000: 1, the optical mode plate that projection produces through programming.

The three-dimensional reconstruction process as shown in Figure 1, at first throw the structured light of certain pattern to determinand, video camera catches the image through the object modulation, at rim detection, striation cutting unit, the main frontier point that extracts striation, each bar striation is cut apart the location, every striation is carried out color assignment, replace with the color value in the alphabet, three striations on the continuous space are formed a code word, realize the space decoding coupling between video camera and the projector, utilize at last the systematic parameter of having demarcated to realize the three-dimensional reconstruction of object.

For what of red (green/indigo plant) component, be divided into artificially 0～255 totally 256 grades, 0 expression does not contain redness (green/indigo plant) composition, and 255 expressions contain 100% redness (green/indigo plant) composition.Just can represent 256 * 256 * 256 kinds of colors according to the various combination of red, green, blue, for example a pixel is shown as cyan when its red, green, blue composition is respectively 0,255,255.And for the pixel of gray level image, the red, green, blue composition of this pixel equates that only along with the increase of these three component values, pixel color becomes white from black.Existing most color-image forming apparatus and color display apparatus all adopt RGB (Red/Green/Blue, RGB) three primary colours represent and store, based on above-mentioned design, can adopt RGB model commonly used to set up color model, color digital image can be by the RGB color space representation, rgb space as shown in Figure 2, for simplicity, the all normalization of all colours value, namely illustrated cube is a unit cube.

In colored projection system, the striation coordinate need to detect from a sub-picture, and this paper proposes a kind of new coding strategy, and each row of projection light template is encoded.For numerous and diverse property and the raising reliability that reduces colored identification, guarantee that the color space discrimination is enough large, requirement has two Color Channel differences at least in rgb space, therefore select red, green, blue three primary colours and in vain as encoded colors, except black and white, red, green, space length between blue and white reaches maximum, this can guarantee the scene of any high saturation is all kept enough distances, and the encoded colors collection can be chosen for P={ (255,0,0), (0,255,0), (0,0,255), (255,255,255) 4 element sets }, white are as each bar color of interval, and the number of times that has satisfied white striation appearance is maximum, to obtain higher brightness of image, corresponding color set letter is mapped as:

P＝{P _i|i＝1，2，3，4}

P _iIndicate respectively red, green, blue and Bai Si color.

Based on traditional De Bruijn space encoding method, introduce wide white striation and come the interval colored bars, make adjacent striation use same shade of color value, the efficient coding rate reaches 100%.Optical mode plate is by 2n ^mThe bar striation forms, and wherein number of color is n, and window size is m, and existing De Bruijn coding techniques adopts same parameter can only generate n (n-1) ^mThe bar striation, the coded sequence of generation is:

3，4，3，4，3，4，2，4，3，4，3，4，1，4，3，4，2，4，2，4，3，4，2，4，1，4，3，

4，1，4，2，4，3，4，1，4，1，4，2，4，2，4，2，4，1，4，2，4，1，4，1，4，1

In the actual scene, the uncontinuity of body surface causes the 3D error in data to increase so that the part striation is invisible, and such striation is called the shade striation.The advantage of introducing white striation can increase decodable code striation ratio, because the white striation that corresponding shade striation is not participated in decoding substitutes.Definition Q={1,2 ..., 2n ^m, the coordinate sequence number value of corresponding every striation.The optical mode plate that generates thus as shown in Figure 3, plane space of each striped boundary demarcation.

Simultaneously, in order to increase the resolution of existing coding templet, propose an effective space-time code, each two field picture of projection is comprised of identical optical mode plate, and a rear frame moves down striation 1/4 width by former frame and forms, and four frame optical mode plates as shown in Figure 4.In view of previous existing a large amount of document the mathematics group and aspect De Bruijn sequence is studied, adopt the method for document to produce sequence.System utilizes the time response of every frame striation boundary space coding uniqueness and multiframe to increase precision and the resolution of reconstruction model.

The correct coupling of setting up between the plane of delineation and the projected light plane is the problem of a complexity.Its step comprises striation boundary alignment, striation color assignment and the position of decoding striation in every frame template.Repeat above-mentioned steps and just can mate whole unique points, and then the reconstruction of three-dimensional shape.

The accurate location on striation border: the rim detection of image mainly is tolerance, detection and the location of grey scale change, and the method for rim detection is numerous, and the wave filter that diverse ways uses also is not quite similar.In the experiment of reality piece image is assumed to be continuous function f (x, y), its directional derivative is that extreme value is arranged in edge direction, and the principle of rim detection is to realize by the extreme value of asking f (x, y) gradient.Edge detection method commonly used has the edge detection operator based on the single order differential, comprise Roberts, Sobel, Prewitt operator, based on the Laplace operator (Laplace) of second-order differential, also have the Canny operator based on optimization method, each operator template is as shown in Figure 5.

Use above-mentioned Operators piece image is carried out rim detection, first coloured image is converted into gray level image, the testing result of each operator as shown in Figure 6.

Because view data is two-dimentional, the depth information in space is lost in the perspective process, add uneven illumination and noise in the imaging process, the precision of rim detection is reduced, many pseudo-edges appear, the edge of striation presents uncontinuity, can cause losing of spatial point code word like this, can not realize decoding.Propose other rim detection scheme for these problems this paper, carry out first the image pre-service, in accurate location.Optical mode plate is comprised of the horizontal color striped, is the one-dimensional coding mode, to any row line scanning of modulated image, as shown in Figure 7.

As can be seen from Figure 6, vertically scanning, the gray scale strong variations of colored bars and white striation is obvious, and the extreme value of the edge definable triple channel gray scale of striation is determined.Colored transition finedraw color-bar with existing certain gray-scale value around the white border, the edge that catches in the image departs from actual definition template border, and accurate location, border directly affects the precision of system reconstructing.For the problems referred to above, before Edge detected, because optical mode plate is one dimension striped coding, along each column scan of image, each passage of each pixel is carried out pre-service with the one dimension row operator of Fig. 8, determine that the strong pixel of grey scale change is candidate marginal.

Consider that overall marginal point search is consuming time, this paper proposes local search approach, and each lists centered by candidate marginal at image, chooses a certain size zone, and its maximal value is no more than 1/2 of striation width, and the step of searching for local field is as follows:

1. initialization candidate marginal is as a regional center z _{I, j}

2. along the column direction search, with z _{I-k, j}, z _{I+k, j}The pixel color space changes into tone, brightness and saturation degree (HIS) space;

3. increase new point to this zone, as long as satisfy the threshold value H that is no more than setting with the tone value difference of neighbor pixel

h＝|z _i±k，j，h-z _{i±k±1，j，h}| (1)

H is the difference of two pixel tones, z _{I ± k, j, h}The tone value that represents the capable j row of the i ± k pixel.The respectively tonal difference of dialogue, the red, green, blue striation neighbor experiment that takes statistics is got threshold value H=0.06 and can be obtained desirable borderline region.After operator was processed, the triple channel gray-scale value of pixel was G _Ij, the peaked pixel of regional area triple channel gray-scale value sum may be defined as frontier point.

$\max \left(E\right)=\underset{j=1}{\overset{C}{\mathrm{\&Sigma;}}}{G}_{\mathrm{ij}}(C=3)---\left(2\right)$

E is the maximal value of pixel gray-scale value sum after operator is processed, and the precision of detection can reach Pixel-level.Fig. 9 uses said method to carry out the result of rim detection to Fig. 6 a original image, compares with above-mentioned operator, and it is more continuous that the edge becomes, and pseudo-edge point reduces.

The striation coupling: the color that every is detected in the image and template color have certain similarity.Therefore, correctly mating the striation corresponding color is to reduce the key of 3D miscount.The people such as Zhang propose dynamic programming method and mate each pixel, rather than mate every striation, very consuming time, such as document 1:L.Zhang, B.Curless.S.M.Seitz.Rapid shape acquisition using color structured light and multi-pass dynamic programming[A] .In:Proceedings First International Symposium on 3D Data Processing Visualization and Transmission[C], 2002,24-36 (L.Zhang, B.Curless, S.M.Seitz. rebuild [A] based on the quick pattern of the color structured light of hyperchannel dynamic programming. first international symposium's three-dimensional visualization data processed and transmission [C], 2002,24-26.).The people such as Fechteler adopt a complicated straight line clustering algorithm to find the parameter of each cluster straight line, degree of separation between the straight line is not obvious, such as document 2:P.Fechteler and P.Eisert.Adaptive color classification for structured light systems[A] .In:Proceedings of the 15th International Conference on Computer Vision and Pattern Recognition[C], USA, 2008,1-7, (P.Fechteler and P.Eisert. structured-light system color self-adaption cluster [A]. the 15th international computer vision and pattern-recognition meeting [C], the U.S., 2008,1-7).This paper adopts the method for color cluster to set up the color-match proper vector, and color of image and projection color are compared, and definition color character vector distributes candidate's striation red, green, blue and white color with the Euclidean distance of cluster centre.The three primary colours space length that coding adopts is enough large, with classical K mean algorithm search color class center.At first select the multiple image of shooting, consider sample space, select three width of cloth images in the experiment, and respective pixel RGB component is changed into color three-dimensional feature vector as the sample data of input, the cluster centre search is as follows:

1. initialization K=4, red, green, blue, white color class, initialization K group's center;

2. each sample point is distributed to the group of the centre distance minimum that peels off;

3.K the center of mass point among the group becomes new cluster centre;

Repeating step 2 and 3 until have a few and all tend to stably to assemble.

Experiment shows that the color of image of seizure and the template color of projector projects have various distortion and distortion.In addition, there be crosstalking between noise and color in each sensor.As shown in figure 10, describing the three primary colours rgb space, (a) is the striation color rgb space that grasps in the projector in the ideal case, (b) is the striation color rgb space that video camera is taken under working environment.

In Figure 10, (b) shown in, different color groups is collected at specific rgb space zone, then the cluster centre by sample point training color group mates the striation color with the distance of point-to-point.

D _ik＝d ²(P _i，C _k)(k＝1，2，3，4) (3)

D _IkBe 2 distances square, P _iThe RGB proper vector of striation center pixel, C _kBe the color cluster center, d (x, y) is x, the y distance between two points.If D=min is (D _Ik), the distribution of striation color is carried out according to (4) formula so

s _c＝{kifD＝＝D _ik(k＝1，2，3，4) (4)

s _cExpression striation color.For example, k=1, then the striation color is red.{ k=1,2,3,4} indicate respectively red, green, blue and white.After distributing color, per three adjacent striation colors form a code word s _i, find out each code word and decode exactly in the position of whole sequence.That adopts that Hsieh proposes directly decodes De Brujin algorithm, compares with exhaustive decoding, does not need to store extra alphabet, and time complexity is the 1/n (n is coding unit) of the method for exhaustion.Striation coordinate figure x _iBe defined as:

$x_i=\mathrm{DdBS}\left(s_i\right)+\frac{1}{4}(j-1)---\left(5\right)$

DdBS (s _i) directly decode j frame optical mode plate subsequence s _iCoordinate position.

In the present embodiment, at first select RGB model commonly used as color model, select RGB and in vain as the encoded colors collection, so that the distance of color is maximum, satisfy the exact matching of color.In the rim detection, adopt the one dimension operator to carry out pre-service, the extreme value of definition triple channel graded quadratic sum utilizes the boundary pixel point to cut apart striation, in conjunction with classical K means clustering method as frontier point simultaneously, the coupling accuracy of striation color reaches 100% in the experiment, utilize the striation color of having distributed to form code word, by directly decoding algorithm, directly locate the locus of striation, compare with the method for exhaustion, greatly improved decoding efficiency.

The calibration process of present embodiment is:

Camera model: from broadly dividing, the scaling method of video camera can be divided into traditional camera calibration method, camera self-calibration method and based on camera calibration method three classes of active vision, what this paper adopted is a kind of traditional camera marking method, the plane reference method of Zhang Zhengyou, the method utilize video camera to come all inside and outside parameter of calibrating camera from the multiple image (at least three width of cloth) of different angle shot plane templates.The dirigibility of plane reference method is can freely move between video camera and target, and kinematic parameter need not to know.Owing to have man-to-man corresponding relation between the corresponding picture point on each unique point and its image on the target, this relation can represent with a mapping matrix, so can determine a mapping matrix for every width of cloth image, for finding the solution of inner parameter provides two constraint conditions.This algorithm is at first supposed the Z of target plane place world coordinate system _w=0, calculate the initial parameter of video camera by linear model analysis, then consider that radial distortion (single order or second order) is used based on maximum-likelihood method initial parameter is carried out nonlinear optimization, utilize at last and optimize good inner parameter and the mapping matrix of target plane is obtained external parameters of cameras.The plane reference method is a kind of method between traditional scaling method and self-calibrating method.It had both avoided the classic method equipment precision requirement high, and the shortcomings such as complex operation are high than the self-calibrating method precision again, satisfied the calibration request of ordinary desktop vision system.

Figure 11 is a camera perspective projection figure, and the projection from 3d space to the 2D plane is a many-to-one mapping, and picture point all 3D points to the 3d space light mM are corresponding same 2D point m all, will lose depth information like this in imaging process.Unique point world coordinates on the target plane is designated as M=(X _w, Y _w, Z _w), the pixel coordinate of the plane of delineation is designated as m=(u, v), and corresponding homogeneous coordinates are respectively M _w=(X _w, Y _w, Z _w, 1) ^T, m _c=(u, v, 1) ^T, video camera is based on linear pin-hole model, and the perspective projection relation of putting to the image with homogeneous coordinates and matrix representation spatial point is as follows:

λm _c＝K[Rt]M _w (20)

Wherein λ is projected scale's factor of arbitrary non-zero, and rotation matrix R and translation vector t are called external parameters of cameras, and K is the intrinsic parameters of the camera matrix, is defined as:

$K=\left[\begin{array}{ccc}{\mathrm{\&alpha;}}_x& \mathrm{\&beta;}& {\mathrm{<figure-callout\; id="0"\; label="u"\; filenames="H2009101536034E00022.png,H2009101536034E00041.png"\; state="\{\{state\}\}">u</figure-callout>}}_0\\ 0& {\mathrm{\&alpha;}}_{\mathrm{<figure-callout\; id="0"\; label="y\; v"\; filenames="H2009101536034E00022.png,H2009101536034E00041.png"\; state="\{\{state\}\}">y</figure-callout>}}& v_{}{}_0\\ 0& 0& 1\end{array}\right]---\left(21\right)$

α wherein _x=f _x/ d _x, α _y=f _y/ d _y, f _x, f _y, d _x, d _yBe respectively focal length on x axle and the y direction of principal axis and the physical size of pixel, β is the out of plumb coefficient of x axle and y axle, (u ₀, v ₀) be the picture centre coordinate.The Z of target plane place world coordinate system _w=0, got by formula (20):

$\mathrm{\&lambda;}\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=K\left[\begin{array}{cccc}{r}_1& r_2& r_3& t\end{array}\right]\left[\begin{array}{c}{X}_{\mathrm{<figure-callout\; id="0"\; label="w\; Y\; w"\; filenames="H2009101536034E00022.png,H2009101536034E00041.png"\; state="\{\{state\}\}">w</figure-callout>}}& \\ Y_w{}_{}\\ 0\\ 1\end{array}\right]=K\left[\begin{array}{ccc}{r}_1& r_2& t\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ 1\end{array}\right]---\left(22\right)$

Note

Spatial point

With picture point m _cThere is a transformation matrix H, then

Note H=[h ₁h ₂h ₃], can obtain:

[h ₁ h ₂ h ₃]＝γK[r ₁ r ₂ t] (24)

γ is a constant factor, by the orthogonality (r of rotation matrix R ₁ ^Tr ₂=0, r ₁ ^Tr ₁=r ₂ ^Tr ₂), obtain two constraint conditions of intrinsic parameter K:

h ₁K ^-TK ^-1h ₂＝0 (25)

$h_1^T{K}^{-T}{K}^{-1}{\mathrm{<figure-callout\; id="1"\; label="h"\; filenames="H2009101536034E00022.png,H2009101536034E00042.png"\; state="\{\{state\}\}">h</figure-callout>}}_1=h_2^T{K}^{-T}{K}^{-1}{h}_2---\left(26\right)$

Utilize (25) (26) two constraint condition linear solution intrinsic parameters K, order

B＝K ^-TK ^-1 (27)

K ^-TK ^-1In fact described the projection of absolute conic on the plane of delineation, obtained B based on this principle, again B has been inverted, utilized SVD to decompose and try to achieve Intrinsic Matrix K.Calculate every width of cloth image with respect to outer parameter rotation matrix R and the translation vector t of camera coordinate system by K and mapping matrix H again.Adopt maximum likelihood to estimate that (Maximum likelihood estimation) carries out nonlinear optimization, further refinement to above-mentioned parameter.Demarcate for improving accuracy, this paper introduce five dimensions radially with tangential distortion coefficient vector k _c=(k ₁, k ₂, p ₁, p ₂, k ₃) ^T, k in actual the demarcation ₃=0 (because k ₅Corresponding order is higher).The image coordinate of (x, y) expression take millimeter as unit, the spatial point coordinate under the camera coordinate system is C=(X _c, Y _c, Z _c), definition normalization coordinate diagram picture:

$x_n=\left[\begin{array}{c}{X}_c/Z_c\\ Y_c/Z_c\end{array}\right]=\left[\begin{array}{c}x\\ y\end{array}\right]---\left(28\right)$

Make r ²=x ²+ y ²,

Actual normalization figure coordinate, (u, v),

The pixel coordinate of respectively expression ideal and actual observation, behind the introducing amount of distortion, the normalization coordinate may be defined as:

The actual pixels coordinate that obtains at last plane of delineation subpoint is:

The maximum likelihood estimation objective function:

Wherein

(K, k ₁, k ₂, k ₃, k ₄, R _i, t _i, M _j) representation space point M _jProject to the plane of delineation, m by the conversion of (20) and (29) formula _IjBe j calibration point in the i width of cloth image, R _iBe the rotation matrix of i width of cloth image, t _iBe the translation vector of i width of cloth image, k _cInitial value be made as 0 ^TPerhaps decompose by (29) formula least square method, with its with the camera interior and exterior parameter of trying to achieve as initial value, use the Levenberg-Marquardt nonlinear optimization algorithm estimated value of objective function is minimized, obtain at last the net result of each parameter.

Projector model is at the conceptive inversion model that is equal to video camera, because the one dimension of space encoding can be simplified to one-dimensional model here.By projector coordinates x _dWith world coordinates X _wIt is as follows to set up model:

$\mathrm{\&gamma;}\left[\begin{array}{c}{\mathrm{<figure-callout\; id="1"\; label="x\; d"\; filenames="H2009101536034E00022.png,H2009101536034E00042.png"\; state="\{\{state\}\}">x</figure-callout>}}_d\\ 1\end{array}\right]=\left[\begin{array}{cc}{m}_1& m_{14}\\ m_2& m_{24}\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_{\mathrm{<figure-callout\; id="1"\; label="w\; Z\; w"\; filenames="H2009101536034E00022.png,H2009101536034E00042.png"\; state="\{\{state\}\}">w</figure-callout>}}& \\ Z_w{}_{}\\ 1\end{array}\right]=\left[\begin{array}{cccc}{m}_{11}& m_{12}& m_{13}& m_{14}\\ m_{21}& m_{22}& m_{23}& m_{24}\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]---\left(15\right)$

γ is arbitrary scale factor, eliminates γ, and conversion (15) formula obtains:

$\frac{m_1{\left[\begin{array}{ccc}{X}_w& Y_w& Z_w\end{array}\right]}^T+m_{14}}{m_2{\left[\begin{array}{ccc}{X}_w& Y_w& Z_w\end{array}\right]}^T+m_{24}}=x_d---\left(16\right)$

(16) formula of utilization structure is with m _IkBe the system of homogeneous linear equations of unknown number, then decompose by singular value (SVD) and find the solution the projector parameter.

In three-dimensional reconstruction, the striation marginal point is as unique point, and is different from tessellated angle point, and its world coordinates can't directly obtain, and need to utilize the world coordinates of known point indirectly to ask.Set up and simplify perspective cross ratio invariability geometric model, as shown in figure 12, hypothesis space point A, B, C are positioned on the same straight line L, and take B as reference point, the position ratio is defined as:

PR(A，B，C)＝AB/AC (17)

In like manner, A ', B ', C ' they are under the O of centre of perspectivity effect, the imaging point of corresponding point A, B, C, and be positioned on the same point straight line L1, the position ratio on the image coordinate is:

PR(A′，B′，C′)＝A′B′/A′C′ (18)

Cross ratio invariability according to the perspective geometry principle can obtain:

PR(A，B，C)＝PR(A′，B′，C′) (19)

In experiment, can find A, B, C on the space line at 3, so corresponding also is to be positioned on the same straight line L ' at image mid point A ', B ', C '.

Extract the image coordinate of unique point in the optical mode plate, can obtain the world coordinates of reference point B according to (19) formula.Set up space three-dimensional and come the labeling projection instrument to the mapping relations between the dimension coordinate.

At Constructed Lighting Vision System, minute independent two steps are demarcated, camera calibration and projector calibrating, and the demarcation of each step is separate, does not rely on mutually, iteration error can not occur, and system accuracy is high.

Camera calibration: the three-dimensional coordinate accuracy requirement of unique point is high, adopts two-dimentional calibrating template, operates in order to make systematic unity, and the single white light of projector projects during camera calibration, coplanar target is as shown in figure 13 made simple and easy.

Target has 21 * 21mm standard black/white gridiron pattern to form, and gridiron pattern is printed by laser high-precision, then is close on the aluminium sheet face, and unique point is comprised of tessellated point.Video camera is taken 10 width of cloth images of diverse location, as shown in figure 14.

Select the grid angle point as unique point to 10 width of cloth images, use above-mentioned plane reference method that video camera is demarcated, the camera intrinsic parameter calibration result is:

$K=\left[\begin{array}{ccc}3872.6& 0& 692.2\\ 0& 3858.1& 443.3\\ 0& 0& 1\end{array}\right]$

Distortion factor k _c=(0.16762 ,-0.74282,0.00092 ,-0.01760,0) ^T, the physical size of used CCD is d in the experiment _x=d _y=6.45 μ m do not consider the out of plumb factor, to α _x, α _yBe averaging, can get the average focal length

With the camera standard focal distance f ₀The relative error of=25mm is:

The target unique point is carried out re-projection, and the resultant error scope is the 0.49708-0.65712 pixel, the re-projection error of each width of cloth image as shown in figure 15, x, y represent respectively the image coordinate system take pixel as unit.

Projector calibrating: projector is as the light source projects device in the structured-light system, and its demarcation relates to the precision of system, and this paper demarcates with one dimension simplified model set forth above.The demarcation of projector need to be obtained world coordinates and the striation coordinate figure of feature point for calibration.To project on the coplanar target by the optical mode plate that programming produces, black and white grid angle point as shown in figure 16, is taken four width of cloth images at depth value Z=990mm, 982mm, 974mm, 966mm place respectively as anchor point.

Marginal point carries out fitting a straight line, and the angle point on the straight line of place is also needed to carry out match.

The image that utilization is obtained in camera calibration extracts black and white grid angle point image coordinate simultaneously, and according to the grid characteristic, world coordinates can generate automatically.

As shown in figure 18, during projection single white optical mode plate, can obtain A ₁, B ₁2 image coordinate (u _A, v _A), (u _B, v _B), and world coordinates is (X _A, Y _A, Z _A), (X _B, Y _B, Z _B).Ask straight line L _iIntersection point with L obtains P _iThe image coordinate of point is such as P ₁The image coordinate of point is (u _P, v _P), world coordinates is defined as (X _p, Y _P, Z _p), because A ₁, B ₁, P ₁3 coplanar, so Z _p=Z _A=Z _B, obtained by (19) formula,

$\frac{X_A-X_P}{X_A-X_B}=\frac{v_A-v_P}{v_A-v_B},$ $\frac{Y_A-Y_P}{Y_A-Y_B}=\frac{u_A-u_P}{u_A-u_B}---\left(12\right)$

Order

${\mathrm{\&lambda;}}_x=\frac{v_A-v_P}{v_A-v_B},$ ${\mathrm{\&lambda;}}_y=\frac{u_A-u_P}{u_A-u_B}---\left(13\right)$

Got by (12), (13) formula

$\left\{\begin{array}{c}{X}_p=X_A+{\mathrm{\&lambda;}}_x(X_B-X_A)\\ Y_p=Y_A+{\mathrm{\&lambda;}}_y(Y_B-Y_A)\\ Z_P=Z_A=Z_B\end{array}\right.---\left(14\right)$

Utilize thus the world coordinates of the principle solving striation Edge Feature Points of cross ratio invariability, decode again the student number coordinate of striation, demarcate with projector model.The calibration result scope is: 0-0.2889, the error synoptic diagram as shown in figure 19.

The base length of the Constructed Lighting Vision System of present embodiment is 230mm, and four target plane of depth value Z=990mm, 982mm, 974mm, 966mm are reconstructed, and reconstructed error is as shown in table 1.As can be seen from Table 1, utilize the target of the equidistant translation of four width of cloth to demarcate, the mean absolute error of system is 1.38mm, and average relative error is 0.14%.

Z/mm	Standard deviation	Absolute error	Relative error
				990.0	1.0074	1.3209	0.13
982.2	1.0374	1.4086	0.14
				974.4	1.0293	1.3943	0.14
966.7	1.0881	1.3988	0.14

Table 1

Utilize the parameter of having demarcated respectively four plane reference points to be carried out back projection, reconstruction result as shown in Figure 9.

The precision of measuring accuracy and hardware device is closely related, and in the system calibrating experiment, the reason that error produces is a lot, mainly is summarized as the following aspects:

1. high-precision checkerboard pattern is printed and is existed certain error.

2. extract in the process that produces at angle point, some during camera calibration on the hypothesis gridiron pattern is coplanar, this plane that will seek template is definitely smooth, but in actual experiment, stencil plane can not be absolute smooth, image and the hypothesis of taking have little deviation, and this little deviation is introduced to calculate and will be caused error.

3. in the process of projector calibrating, use cross ratio invariability and ask world coordinates, introduced error when unique point is carried out fitting a straight line.

4. the error that causes of calibration system.Secondly at first the resolution of vision system and acquisition precision all can affect the error of whole system, and scaling method chooses the error that all can cause system with the combination of whole calibration system each several part.Mainly based on the plane reference method of Zhang Zhengyou, the result from demarcating based on the method for nonlinear iteration, makes the minimum of error to the camera calibration of this paper, and the decomposition of projector model also is based on error minimize.

More than four aspects are the main causes that cause calibrated error.Because experiment condition is limit, the restriction of the video camera of hardware and projector resolution and acquisition precision, in the grid angle point leaching process, the template face is not definitely smooth, causes that angle point extracts error.In addition, above-mentioned system model in theory with computation process in all can defectiveness, also can cause error.

Demarcating good transition matrix parameter and determined the relation of spatial point coordinate and its image coordinate point, when carrying out three-dimensional computations, require to recover 3 d space coordinate from the image coordinate of unique point, namely is the process of three-dimensional reconstruction.Take modulated image, after image is processed, obtain image coordinate

Calculate desirable normalization coordinate (x, y) by (29) formula and (30) formula, because there are following relation in spatial point and space under the camera coordinate system:

$\left[\begin{array}{c}{X}_c\\ Y_c\\ Z_c\end{array}\right]=R\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\end{array}\right]+t=\left[\begin{array}{c}{r}_1\\ r_2\\ r_3\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\end{array}\right]+\left[\begin{array}{c}{t}_1\\ t_2\\ t_3\end{array}\right]---\left(6\right)$

Draw X by (28) formula _c=xZ _c, Y _c=yZ _c, bring (6) into and get

$\left[\begin{array}{c}{\mathrm{xZ}}_c\\ {\mathrm{yZ}}_c\\ Z_c\end{array}\right]=\left[\begin{array}{c}{r}_1\\ r_2\\ r_3\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\end{array}\right]+\left[\begin{array}{c}{t}_1\\ t_2\\ t_3\end{array}\right]---\left(7\right)$

Cancellation Z _c, abbreviation gets

$\left[\begin{array}{c}{\mathrm{xt}}_3-{\mathrm{<figure-callout\; id="1"\; label="t"\; filenames="H2009101536034E00022.png,H2009101536034E00042.png"\; state="\{\{state\}\}">t</figure-callout>}}_1\\ {\mathrm{yt}}_3-{\mathrm{<figure-callout\; id="2"\; label="t"\; filenames="H2009101536034E00022.png,H2009101536034E00042.png"\; state="\{\{state\}\}">t</figure-callout>}}_2\end{array}\right]=\left[\begin{array}{c}{r}_1-{\mathrm{xr}}_3\\ r_2-{\mathrm{yr}}_3\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\end{array}\right]---\left(8\right)$

Obtain x by the striation decoding _dCoordinate figure,

$x_d{m}_{24}-m_{14}=[m_1-x_d{m}_2]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\end{array}\right]---\left(9\right)$

Get in conjunction with (8) formula, (9) formula

$\left[\begin{array}{c}{\mathrm{xt}}_3-{\mathrm{<figure-callout\; id="1"\; label="t"\; filenames="H2009101536034E00022.png,H2009101536034E00042.png"\; state="\{\{state\}\}">t</figure-callout>}}_1\\ {\mathrm{yt}}_3-t_2\\ x_d{m}_{24}-m_{14}\end{array}\right]=\left[\begin{array}{c}{r}_1-{\mathrm{xr}}_3\\ r_2-{\mathrm{yr}}_3\\ m_1-x_d{m}_2\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\end{array}\right]---\left(10\right)$

Write (10) formula as matrix A X=B form, wherein X=(X _w, Y _w, Z _w), then equation linear least-squares solution is:

X＝(A ^TA) ^-1A ^TB (11)

X namely is the world coordinates that is obtained by three-dimensional computations by the 2D image.

Claims (2)

1. three-dimensional rebuilding method based on coded structured light, it is characterized in that: described three-dimensional rebuilding method may further comprise the steps:

1), to determinand projective structure light, video camera catches the image through the determinand modulation;

2), carry out optical mode plate coupling, may further comprise the steps:

(2.1), the location on striation border:

Optical mode plate to be detected is one dimension striped coding, each column scan along image, each passage to each pixel carries out pre-service with one dimension row operator, determine that the strong pixel of grey scale change is candidate marginal, each lists centered by candidate marginal at image, choose a certain size zone, its maximal value is no more than 1/2 of striation width, and the step of search local neighborhood is as follows:

(2.1.1). the initialization candidate marginal is as a regional center z _{I, j}

(2.1.2). along the column direction search, with z _{I-k, j}, z _{I+k, j}The pixel color space changes into tone, brightness and saturation degree (HIS) space;

(2.1.3). increase new point to this zone, as long as satisfy the threshold value H that is no more than setting with the tone value difference of neighbor pixel

h＝|z _i±k，j，h-z _{i±k±1，j，h}| (1)

H is the difference of two pixel tones, z _{I ± k, j, h}The tone value that represents the capable j row of the i ± k pixel; After operator was processed, the triple channel gray-scale value of pixel was G _Ij, the peaked pixel of regional area triple channel gray-scale value sum is defined as frontier point;

$E=\max \left(\underset{j=1}{\overset{C}{\mathrm{\&Sigma;}}}{G}_{\mathrm{ij}}\right),C=3---\left(2\right)$

E is the maximal value of pixel gray-scale value sum after operator is processed;

(2.2), striation coupling: adopt the method for color cluster to set up the color-match proper vector, color of image and projection color are compared, definition color character vector distributes candidate's striation red, green, blue and white color with the Euclidean distance of cluster centre;

Select the red, green, blue three primary colours and in vain as encoded colors, except black and white, red, green, blue and white between space length reach maximum, the encoded colors collection is chosen for φ={ (255,0,0), (0,255,0), (0,0,255), (255,255,255) } 4 element sets, white is as each bar color of interval, and corresponding color set letter is mapped as:

Φ＝{R _i|i＝1，2，3，4}

R _iIndicate respectively the RGB proper vector of red, green, blue and Bai Si color;

Introduce wide white striation and come the interval colored bars, make adjacent striation use same shade of color value, optical mode plate is by 2n ^mThe bar striation forms, and wherein number of color is n, and window size is m;

Definition Q={1,2 ..., 2n ^m, the coordinate sequence number value of corresponding every striation, plane space of each striped boundary demarcation, each two field picture of projection is comprised of identical optical mode plate, and a rear frame moves down striation 1/4 width by former frame and forms;

Different color groups is collected at specific rgb space zone, and then the cluster centre by sample point training color group mates the striation color with the distance of point-to-point;

D _ik＝d ²(Φ _i，C _k)k＝1，2，3，4 (3)

D _IkBe 2 distances square, Φ _iThe RGB proper vector of striation center pixel, C _kBe the color cluster center, d (x, y) is x, the y distance between two points; If D=min is (D _Ik), the distribution of striation color is carried out according to (4) formula so:

s _c＝{k if D＝＝D _ik k＝1，2，3，4} (4)

s _cExpression striation color;

After distributing color, per three adjacent striation colors form a code word s _i, find out each code word and decode exactly in the position of whole sequence; Striation coordinate figure x _iBe defined as:

$x_i=\mathrm{DdBS}\left(s_i\right)+\frac{1}{4}(j-1)---\left(5\right)$

DdBS (s _i) expression directly decodes j frame optical mode plate subsequence s _iCoordinate position;

3), utilize the systematic parameter demarcated to carry out the three-dimensional reconstruction of determinand, detailed process is:

Demarcate good transition matrix parameter and determined the relation of spatial point coordinate and its image coordinate point, recover 3 d space coordinate from the image coordinate of unique point; Take modulated image, after image is processed, obtain image coordinate Calculate desirable normalization coordinate (x, y), there are following relation in spatial point and space under the camera coordinate system:

$\left[\begin{array}{c}{X}_c\\ Y_c\\ Z_c\end{array}\right]=R\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\end{array}\right]+t=\left[\begin{array}{c}{r}_1\\ r_2\\ r_3\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\end{array}\right]+\left[\begin{array}{c}{t}_1\\ t_2\\ t_3\end{array}\right]---\left(6\right)$

Definition X _c=xZ _c, Y _c=yZ _c, bring (6) into and get:

$\left[\begin{array}{c}{\mathrm{xZ}}_c\\ {\mathrm{yZ}}_c\\ Z_c\end{array}\right]=\left[\begin{array}{c}{r}_1\\ r_2\\ r_3\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\end{array}\right]+\left[\begin{array}{c}{t}_1\\ t_2\\ t_3\end{array}\right]---\left(7\right)$

Cancellation Z _c, abbreviation gets:

$\left[\begin{array}{c}{\mathrm{xt}}_3-t_1\\ {\mathrm{yt}}_3-t_2\end{array}\right]=\left[\begin{array}{c}{r}_1-{\mathrm{xr}}_3\\ r_2-{\mathrm{yr}}_3\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\end{array}\right]---\left(8\right)$

By projector coordinates x _dWith world coordinates X=(X _w, Y _w, Z _w) to set up projector model as follows:

$\mathrm{\&gamma;}\left[\begin{array}{c}{x}_d\\ 1\end{array}\right]=\left[\begin{array}{cc}{m}_1& m_{14}\\ m_2& m_{24}\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]=\left[\begin{array}{cccc}{m}_{11}& m_{12}& m_{13}& m_{14}\\ m_{21}& m_{22}& m_{23}& m_{24}\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]---\left(15\right)$

γ is arbitrary scale factor;

Obtain x by the striation decoding _dCoordinate figure:

$x_d{m}_{24}-m_{14}=[m_1-x_d{m}_2]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\end{array}\right]---\left(9\right)$

Get in conjunction with (8) formula, (9) formula:

$\left[\begin{array}{c}{\mathrm{xt}}_3-t_1\\ {\mathrm{yt}}_3-t_2\\ x_d{m}_{24}-m_{14}\end{array}\right]=\left[\begin{array}{c}{r}_1-{\mathrm{xr}}_3\\ r_2-{\mathrm{yr}}_3\\ m_1-x_d{m}_2\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\end{array}\right]---\left(10\right)$

Write (10) formula as matrix A X=B form, wherein X=(X _w, Y _w, Z _w), then equation linear least-squares solution is:

X＝(A ^TA) ^-1A ^TB (11)

X namely is the world coordinates that is obtained by three-dimensional computations by two dimensional image.

2. the three-dimensional rebuilding method based on coded structured light as claimed in claim 1 is characterized in that: in described step 3) in, systematic parameter comprises camera parameters and projector parameter, scaling method may further comprise the steps:

3.1), the camera calibration process:

Adopt two-dimentional calibrating template, target is that the black and white chess lattice of standard form, unique point is that the point of chess lattice forms, video camera is taken the multiple image of diverse location, select the grid angle point as unique point to described multiple image, adopt the plane reference method that video camera is demarcated, obtain camera intrinsic parameter and demarcate K, and obtain the average focal length It is characterized in that: described scaling method also comprises:

3.2), the projector calibrating process:

The optical mode plate that produces by coding projects on the common target, and black and white grid angle point utilizes the image that obtains as anchor point in camera calibration, extract simultaneously black and white grid angle point image coordinate, generates world coordinates;

During projection single white optical mode plate, establish that 4 corner location of each black and white grid are A on the template _i, B _iA _I+1, B _I+1, i=1,2,3 ..., two limit L of arbitrary straight line L and this grid _i(A _i, B _i) and L _I+1(A _I+1, B _I+1) intersect at respectively a P _iAnd P _I+1Directly mention A take first grid wherein as example ₁, B ₁Image coordinate (the u of two angle points _A, v _A), (u _B, v _B), and world coordinates is (X _A, Y _A, Z _A), (X _B, Y _B, Z _B), ask straight line L _iIntersection point with L obtains P _iThe image coordinate of point, P ₁The image coordinate of point is (u _P, v _P), world coordinates is defined as (X _p, Y _P, Z _p), because A ₁, B ₁, P ₁3 coplanar, so Z _p=Z _A=Z _B, according to cross ratio invariability, obtain,

$\frac{X_A-X_P}{X_A-X_B}=\frac{v_A-v_P}{v_A-v_B}$ $\frac{Y_A-Y_P}{Y_A-Y_B}=\frac{u_A-u_P}{u_A-u_B}---\left(12\right)$

Order

${\mathrm{\&lambda;}}_x=\frac{v_A-v_P}{v_A-v_B},$ ${\mathrm{\&lambda;}}_y=\frac{u_A-u_P}{u_A-u_B}---\left(13\right)$

Got by (12), (13) formula:

$\left\{\begin{array}{c}{X}_p=X_A+{\mathrm{\&lambda;}}_x(X_B-X_A)\\ Y_P=Y_A+{\mathrm{\&lambda;}}_y(Y_B-Y_A)\\ Z_P=Z_A=Z_B\end{array}\right.---\left(14\right);$

λ _x, λ _yThe double ratio coefficient that represents respectively X and Y-direction, (X _p, Y _P, Z _p) world coordinates of expression striation Edge Feature Points;

In described step 3.2) in, the process of setting up of projector model is:

By projector coordinates x _dTo set up model as follows with world coordinates X:

$\mathrm{\&gamma;}\left[\begin{array}{c}{x}_d\\ 1\end{array}\right]=\left[\begin{array}{cc}{m}_1& m_{14}\\ m_2& m_{24}\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]=\left[\begin{array}{cccc}{m}_{11}& m_{12}& m_{13}& m_{14}\\ m_{21}& m_{22}& m_{23}& m_{24}\end{array}\right]\left[\begin{array}{c}{X}_w\\ Y_w\\ Z_w\\ 1\end{array}\right]---\left(15\right)$

γ is arbitrary scale factor, eliminates γ, and conversion (15) formula obtains:

$\frac{m_1{\left[\begin{array}{ccc}{X}_w& Y_w& Z_w\end{array}\right]}^T+m_{14}}{m_2{\left[\begin{array}{ccc}{X}_w& Y_w& Z_w\end{array}\right]}^T+m_{24}}=x_d---\left(16\right)$

(16) formula of utilization structure is with m _IkBe the system of homogeneous linear equations of unknown number, then by Singular-value Decomposition Solution projector parameter;

Set up and simplify perspective cross ratio invariability geometric model, hypothesis space point A, B, C are positioned on the same straight line Γ, and take B as reference point, the position ratio is defined as:

PR(A，B，C)＝AB/AC (17)

In like manner, A ', B ', C ' they are under the O of centre of perspectivity effect, the imaging point of corresponding point A, B, C, and be positioned on the same point straight line Γ ', the position ratio on the image coordinate is:

PR(A′，B′，C′)＝A′B′/A′C′ (18)

Cross ratio invariability according to the perspective geometry principle obtains:

PR(A，B，C)＝PR(A′，B′，C′) (19)

Also be to be positioned on the same straight line Γ ' at image mid point A ', B ', C ';

Extract the image coordinate of unique point in the optical mode plate, obtain the world coordinates of reference point B according to (19) formula, set up space three-dimensional and come the labeling projection instrument to the mapping relations between the dimension coordinate.

Images