CN105066904A - Assembly line product three-dimensional surface type detection method based on phase gradient threshold - Google Patents

Abstract

The invention belongs to the on-line three-dimensional surface type detection field, and especially discloses an assembly line product three-dimensional surface type detection method based on a phase gradient threshold. The invention solves the problems of the prior art, fully takes regard of the balance between measurement speed and precision required by on-line three-dimensional surface type detection, and provides an assembly line product three-dimensional surface type real time detection method based on a phase gradient threshold. The method comprises: obtaining the whole phase information of a detected object through an Fourier transform method; using a phase gradient threshold as an image characteristic to participate in correlative operation to obtain phase shift stripes; expanding a binary phase shift gradient graph to obtain an object surface abrupt change area; and finally employing the least square method to obtain the phase distribution of the abrupt change area through iteration computation, and to modify FTP calculating results in order to improve phase obtaining precision.

Description

Based on the streamline product tri-dimensional facial type detection method of phase gradient threshold value

Technical field:

The invention belongs to online tri-dimensional facial type detection field, particularly relate to a kind of streamline product tri-dimensional facial type detection method based on phase gradient threshold value.

Background technology:

Online tri-dimensional facial type detects the category belonging to detection of dynamic, and detection of dynamic need take into full account the real-time of measuring method.Fringe projection technology of profiling passes through projection sinusoidal light in testee, calculate the three-dimensional information that deforming stripe phase place obtains object, the method have contactless, measuring accuracy is considerable, simple to operate, measure low cost and other advantages, can be used for the on-line checkingi of streamline three-dimensional size of product.

In many algorithms of fringe projection technology of profiling, Fourier transform profilometry (Fouriertransformprofilometry, be called for short FTP) be usually used in kinetic measurement, it adopts the method for Fourier analysis to obtain phase information, there is the advantage that the whole audience is analyzed, but owing to relating to frequency domain filtering, cause object plane type Sudden change region measuring accuracy to be restricted.

Phase measuring profilometer (phasemeasuringprofilometry is called for short PMP) adopts the method for the point-to-point calculating phase place of phase-shifting technique, and insensitive to the change of background, contrast and noise, measuring accuracy is high.But need to gather at least three frame deforming stripes in measuring process and calculate a frame phase information, in kinetic measurement, the motion of object causes the position of object picture in different bar graph to change, and brings error to PMP phase calculation.Overcome this problem, not only require that CCD has higher filming frequency, also require that the real-time three-dimensional detection system that the people such as projector equipment has equivalent projection switching frequency, American I owa state university Zhang Song propose can be used for online three-dimensional values, but system hardware is had higher requirements.

Sichuan University Cao Yi equality people keeps the movement characteristic of translation according to streamline product, ohject displacement is converted to the method for striped phase shift by proposition pixel matching, solve the problem that the deforming stripe figure object space that do not photograph in the same time changes, thus reduce the requirement to projection, frequency acquisition.Online PMP wherein based on Stoilov algorithm adopts metrological grating sensor to produce the deforming stripe figure of the equidistant movement of trigger pip control CCD shot object, is the PHASE DISTRIBUTION of the phase in-migration calculating object of striped after pixel matching by the converts displacement of object.

In order to reduce the complexity of measuring system control section, improving simultaneously and separating phase precision, the online PMP based on phase shift algorithm such as full cycles that they propose again.The method projection multiframe phase shift bar graph, require that movement direction of object is vertical with striped shift direction, after pixel matching, object of which movement does not produce equivalent phase shift, makes the phase-shift phase of the deforming stripe figure collected equal the phase-shift phase of projected fringe, realizes the object of the artificial coding-control of phase-shift phase.But in actual measurement process, the shift direction being difficult to accurately control projected fringe is vertical with the direction of motion of object, and off plumb systematic error produces additional phase shift amount by causing each frame deforming stripe figure after pixel matching in striped shift direction, introduce phase displacement error when adopting the phase shift algorithm such as full cycle to calculate phase place, reduce phase calculation precision.

For the problems referred to above, propose again to adopt pixel matching to obtain any phase-shift phase, adopt the method for least square method iterative computation phase place, not only increase measuring accuracy, also reduce the constraint condition of system and device locus.But least square method iterative computation phase place is consuming time longer, is unfavorable for the raising of system real time.

From above-mentioned discussion, no matter be adopt Fourier transform or adopt least square method iterative computation, all do not take into full account that tri-dimensional facial type detects the balance to measuring speed and accuracy requirement.Fourier transform profilometry real-time is good, but measuring accuracy is not high, and measuring error concentrates on the abrupt change region of measured object dignity type.Least square method iteration measuring accuracy is high, but iteration time is long, have impact on the real-time of measurement.

Summary of the invention:

The present invention overcomes the deficiency that prior art exists, solve prior art Problems existing, take into full account that online tri-dimensional facial type detects the balance to measuring speed and accuracy requirement, propose the streamline product tri-dimensional facial type real-time detection method based on phase gradient threshold value.The overall phase information of testee is obtained by the method for Fourier transform, the Grads threshold of phase place is participated in related operation as characteristics of image and obtains phase shift striped, binaryzation dephasing gradient figure is expanded and obtains body surface abrupt change region, finally adopt the PHASE DISTRIBUTION in least square method iterative computation abrupt change region, obtain precision for the result of calculation revising FTP to improve phase place.

For solving the problems of the technologies described above, the technical solution used in the present invention is: based on the streamline product tri-dimensional facial type detection method of phase gradient threshold value, testing process is carried out according to following steps:

Step 1) projector sinusoidal grating picture is in testee, and CCD gathers not deforming stripe figure in the same time;

When on travelling belt without testee time, the bar graph that CCD does not photograph in the same time does not change, and its light distribution can be expressed as: I ₀(x, y)=R (x, y) [A (x, y)+B (x, y) cos φ ₀(x, y)] (1)

In formula, R (x, y) is body surface reflectivity, and A (x, y) is background light intensity, and B (x, y) is fringe contrast; φ ₀(x, y) is fringe phase distribution;

When measuring object with conveyer belt, CCD does not collect deforming stripe figure in the same time, and its light distribution can be expressed as:

I _i(x,y)＝R(x,y)[A(x,y)+B(x,y)cosφ _i(x,y)]i＝1,2,...M(2)

φ in formula _i(x, y) is the PHASE DISTRIBUTION of body surface deforming stripe, and object of which movement makes its picture position in every frame deforming stripe figure different, and the distribution of the phase-modulation of generation is also different, distinguishes with subscript i; M represents shooting frame number;

Step 2) phase place obtain and phase only pupil filter:

A, calculate phase place that in every frame bar line, object height causes change φ by the method for Fourier transform _i(x, y):

Make Fourier transform to formula (2), leach+1 grade of fundamental component, remaking inverse Fourier transform can obtain:

$g_i(x,y)=\frac{R(x,y)\·B(x,y)}{2}\exp \[j\·{\mathrm{\φ}}_i(x,y)\]---\left(3\right)$

Do same computing to the light distribution (1) of CCD acquisition on reference surface to obtain:

$g_0(x,y)=\frac{R(x,y)\·B(x,y)}{2}\exp \[j\·{\mathrm{\φ}}_0(x,y)\]---\left(4\right)$

Therefore the phase place caused by object height is changed to:

${\mathrm{\Δ\φ}}_i(x,y)={\mathrm{\φ}}_i(x,y)-{\mathrm{\φ}}_0(x,y)=lm\{\ln \[g_i(x,y)g_0^{*}(x,y)\]\}---\left(5\right)$

Wherein lm{} represents the imaginary part of getting plural number, and * represents conjugate operation;

The gradient of b, the change of calculating phase place, according to graded by object zoning, obtains abrupt change region template;

With △ φ _ithe gradient of (x, y) is distributed as foundation, realizes the division of measuring object abrupt change region, to △ φ _ithe gradient of (x, y) carries out threshold value, obtains the phase gradient figure of binaryzation:

${{\mathrm{\&Delta;\&phi;}}^g}_i(x,y)=\left\{\begin{array}{cc}=1& grad\&lsqb;{\mathrm{\&Delta;\&phi;}}_i(x,y)\&rsqb;\&GreaterEqual;T\\ =0& grad\&lsqb;{\mathrm{\&Delta;\&phi;}}_i(x,y)\&rsqb;<T\end{array}\right.---\left(6\right)$

Wherein grad [] represents gradient algorithm, and T is system calibrating value, and the mean value being generally the greatest gradient of such testee under measurement environment subtracts one in a small amount.

Then to △ φ ^g _i(x, y) expands, and can obtain abrupt change region Prototype drawing Mask _i(x, y);

C, carrying out related calculation to binaryzation phase diagram obtains ohject displacement;

Intercept binaryzation phase gradient figure △ φ ^g _ithe characteristic area of (x, y), carries out related calculation with M frame binaryzation phase gradient figure respectively, and the coordinate difference calculating maximal correlation point can obtain ohject displacement;

D, ohject displacement is converted to the phase shift of striped, obtains phase shift bar graph; According to the phase place change that abrupt change region template adopts least square method iterative computation abrupt change region place to be caused by object height;

Abrupt change region template corresponding with it for the M frame phase shift bar graph photographed is multiplied, obtains abrupt change region bar graph:

I′ _i(x,y)＝I _i(x,y)·Mask _i(x,y)(7)

Adjacent N frame is divided into one group, abrupt change region bar graph is divided into groups, according to ohject displacement, cutting is carried out to bar graph, the pixel coordinate realizing object in each group of different distortion bar graph is consistent, also convert ohject displacement to phase shift striped simultaneously, the method for the phase shift bar graph least square method iteration obtained is calculated the phase place change in object abrupt change region;

E, by abrupt change region phase place, formula (5) to be revised, improve phase place and obtain precision;

With the phase place change at the abrupt change region place in abrupt change region phase place change alternate form (5) calculated in d, promote phase place and obtain precision;

Step 3) obtain the X of object by system calibrating, Y-direction information, and phase information is converted to depth information Z.

The present invention compared with prior art has following beneficial effect.

First, the method that the present invention proposes to combine based on Fourier transform and least square method iteration obtains the tri-dimensional facial type of testee, the overall phase information of testee is obtained by the method for Fourier transform, then by carrying out region segmentation to phase information, be divided into abrupt change region and non-abrupt change region, adopt least square method iterative computation phase place in abrupt change region, replace the phase place at abrupt change region place in the overall phase information of Fourier transformation method acquisition by above-mentioned phase place.Fourier transform ensure that measuring speed, and least square method iteration is revised abrupt change region place, ensures the measuring accuracy in abrupt change region.

The second, division with abrupt change region mild for body surface, proposes, by calculating phase gradient, to obtain abrupt change region after reasonable threshold value expansion.

3rd, for the feature extraction of deforming stripe, propose, based on the method on phase gradient threshold value searching object abrupt change border, to complete pixel matching related operation.

Accompanying drawing explanation

Fig. 1 is on-line checkingi schematic diagram.

Fig. 2 is that phase place obtains and phase correction procedure block diagram.

Fig. 3 is testee tri-dimensional facial type distribution plan.

Fig. 4 is the wherein two frame deforming stripe figure that CCD photographs.

Fig. 5 is the phase place change that the homologue height calculated causes.

Fig. 6 is the phase gradient figure of binaryzation.

Fig. 7 is three-dimensional reconstruction structural drawing, and wherein (a) is FTP three-dimensional reconstruction structure, and (b) is three-dimensional reconstruction structure of the present invention.

Fig. 8 is the distribution of corresponding root-mean-square error.

Embodiment:

As shown in Figure 1, online tri-dimensional facial type detection system principle based on fringe projection technology of profiling is, testee is with travelling belt uniform motion in X direction, computing machine controls Digital light projector projection sinusoidal grating picture in testee surface, the optical axis CO of optical axis PO and the CCD of projector meets at the O point on reference surface, and the height of object can cause striped generation deformation, and namely phase place changes, calculate the phase place change of striped, the elevation information Z of object can be obtained.

The present invention is based on the streamline product tri-dimensional facial type detection method of phase gradient threshold value, testing process is carried out according to following steps:

Step 1) projector sinusoidal grating picture is in testee, and CCD gathers not deforming stripe figure in the same time;

When on travelling belt without testee time, the bar graph that CCD does not photograph in the same time does not change, and its light distribution can be expressed as: I ₀(x, y)=R (x, y) [A (x, y)+B (x, y) cos φ ₀(x, y)] (1)

In formula, R (x, y) is body surface reflectivity, and A (x, y) is background light intensity, and B (x, y) is fringe contrast; φ ₀(x, y) is fringe phase distribution;

When measuring object with conveyer belt, CCD does not collect deforming stripe figure in the same time, and its light distribution can be expressed as:

I _i(x,y)＝R(x,y)[A(x,y)+B(x,y)cosφ _i(x,y)]i＝1,2,...M(2)

φ in formula _i(x, y) is the PHASE DISTRIBUTION of body surface deforming stripe, and object of which movement makes its picture position in every frame deforming stripe figure different, and the distribution of the phase-modulation of generation is also different, distinguishes with subscript i; M represents shooting frame number.

Step 2) phase place obtain and phase only pupil filter: whole process as shown in Figure 2,

A, calculate phase place that in every frame bar line, object height causes change φ by the method for Fourier transform _i(x, y):

Make Fourier transform to formula (2), leach+1 grade of fundamental component, remaking inverse Fourier transform can obtain:

$g_i(x,y)=\frac{R(x,y)\&CenterDot;B(x,y)}{2}\exp \&lsqb;j\&CenterDot;{\mathrm{\&phi;}}_i(x,y)\&rsqb;---\left(3\right)$

Do same computing to the light distribution (1) of CCD acquisition on reference surface to obtain:

$g_0(x,y)=\frac{R(x,y)\&CenterDot;B(x,y)}{2}\exp \&lsqb;j\&CenterDot;{\mathrm{\&phi;}}_0(x,y)\&rsqb;---\left(4\right)$

Therefore the phase place caused by object height is changed to:

${\mathrm{\&Delta;\&phi;}}_i(x,y)={\mathrm{\&phi;}}_i(x,y)-{\mathrm{\&phi;}}_0(x,y)=lm\{ln\&lsqb;g_i(x,y)g_0^{*}(x,y)\&rsqb;\}---\left(5\right)$

Wherein lm{} represents the imaginary part of getting plural number, and * represents conjugate operation;

Above-mentioned formula (5) is exactly the phase information of the testee entirety calculated by Fourier transform, but owing to relating to frequency domain filtering when Fourier transform calculates phase place, limit the precision that phase place obtains, and error mainly concentrates on the abrupt change region of object, therefore carry out revising the phase information that effectively can improve object entirety to abrupt change region phase place.

The correction of abrupt change region phase place is divided into prerequisite, due to the fringe phase change △ φ that object causes with abrupt change region _i(x, y) and object height have linear approximate relationship.Therefore the present invention is with △ φ _ithe gradient of (x, y) is distributed as foundation, realizes the division of measuring object abrupt change region.Several step emphasis introduces the division in abrupt change region and the acquisition of abrupt change region phase place below.

The gradient of b, the change of calculating phase place, according to graded by object zoning, obtains abrupt change region template;

With △ φ _ithe gradient of (x, y) is distributed as foundation, realizes the division of measuring object abrupt change region, to △ φ _ithe gradient of (x, y) carries out threshold value, obtains the phase gradient figure of binaryzation:

${{\mathrm{\&Delta;\&phi;}}^g}_i=(x,y)=\left\{\begin{array}{cc}=1& grad\&lsqb;{\mathrm{\&Delta;\&phi;}}_i(x,y)\&rsqb;\&GreaterEqual;T\\ =0& grad\&lsqb;{\mathrm{\&Delta;\&phi;}}_i(x,y)\&rsqb;<T\end{array}\right.---\left(6\right)$

Wherein grad [] represents gradient algorithm, and T is system calibrating value, and the mean value being generally the greatest gradient of such testee under measurement environment subtracts one in a small amount,

Then to △ φ ^g _i(x, y) expands, and can obtain abrupt change region Prototype drawing Mask _i(x, y);

C, maximal correlation computing is done to binaryzation phase diagram obtain ohject displacement;

Intercept binaryzation phase gradient figure △ φ ^g _ithe characteristic area of (x, y), carries out related calculation with M frame binaryzation phase gradient figure respectively, and the coordinate difference calculating maximal correlation point can obtain ohject displacement;

D, ohject displacement is converted to the phase shift of striped, obtains phase shift bar graph; According to the phase place change that abrupt change region template adopts least square method iterative computation abrupt change region place to be caused by object height;

Abrupt change region template corresponding with it for the M frame phase shift bar graph photographed is multiplied, obtains abrupt change region bar graph:

I′ _i(x,y)＝I _i(x,y)·Mask _i(x,y)(7)

Adjacent N frame is divided into one group, abrupt change region bar graph is divided into groups, according to ohject displacement, cutting is carried out to bar graph, the pixel coordinate realizing object in each group of different distortion bar graph is consistent, also convert ohject displacement to phase shift striped simultaneously, the method for the phase shift bar graph least square method iteration obtained is calculated the phase place change in object abrupt change region.

E, by abrupt change region phase place, formula (5) to be revised, improve phase place and obtain precision;

With the phase place change at the abrupt change region place in abrupt change region phase place change alternate form (5) calculated in d, promote phase place and obtain precision.

Step 3) obtain the X of object by system calibrating, Y-direction information, and phase information is converted to depth information Z.

The present invention, when obtaining phase shift striped, adopts the phase place of object to be used for related operation as feature.The feature extraction of image adopts the texture of object in scene or depth information as feature mostly, the phase information that fringe projection technology of profiling obtains with highly have linear approximate relationship, can representative image object features.When adopting Structured Illumination in measuring process, in the bar graph do not photographed in the same time, testee is without obvious textural characteristics, prior art adopts and is placed with mark without fringe area, extract mark texture information as the method for feature, this method increases the complexity of measuring process; Calculate the method for modulation of fringes and can extract the edge contour of the testee covered by striped as feature, but object edge shade can bring matching error.

The present invention is based on phase information in addition and do region segmentation, Fourier transform is adopted to obtain the overall phase information of testee, then least square method iterated revision object abrupt change region phase value is utilized, reduce the phase error that frequency domain filtering causes, compared with Fourier transformation method, measuring accuracy is higher, and compared with least square method iteration, speed is faster.The present invention has taken into full account the balance of measuring speed and measuring accuracy, is more applicable for online tri-dimensional facial type and detects.

Below by Computer Simulation example, validity of the present invention is described, is illustrated in figure 3 the tri-dimensional facial type profile diagram of testee, Fig. 4 is the wherein two frame deforming stripe figure I that CCD photographs _i(x, y), under striped, the horizontal shift of object is 40 pixels.Fig. 5 is the phase place change △ φ that the homologue height calculated according to method of the present invention causes _i(x, y).Fig. 6 is for calculating its gradient after threshold value, corresponding binaryzation phase gradient figure.After carrying out related calculation to phase gradient figure, obtaining displacement is 40 pixels, meets preset value, illustrates that abrupt change region threshold can effective recognition object abrupt change border.Expansion is carried out to Fig. 6 and can obtain abrupt change region template, to abrupt change region least square method iterative computation phase place, for revising FTP phase calculation result.After Phase-height mapping, as shown in Figure 7, wherein (a) is FTP three-dimensional reconstruction structure, and (b) is three-dimensional reconstruction structure of the present invention.(a) in Fig. 8, (b) distribute for corresponding root-mean-square error, and corresponding RMS is respectively 0.9004mm, 0.2662mm.As can be seen from Fig. 7 and Fig. 8, method of the present invention is higher relative to fourier transform method precision.

Again to the present invention put forward region segmentation phase calculation method speed evaluate, under the condition adopting the configuration of identical phase shift striped, same computer, the phase calculation time of the present invention is 5.26 seconds, if the overall situation adopts least square method iterative computation phase place to need 18.28 seconds, the FTP illustrating based on region segmentation combines with least square method and calculates phase place and can improve measuring speed.

Above embodiments of the invention are explained in detail, but the present invention is not limited to above-described embodiment, in the ken that those of ordinary skill in the art possess, various change can also be made under the prerequisite not departing from present inventive concept.

Claims (1)

1., based on the streamline product tri-dimensional facial type detection method of phase gradient threshold value, it is characterized in that, testing process is carried out according to following steps:

Step 1) projector sinusoidal grating picture is in testee, and CCD gathers not deforming stripe figure in the same time;

When on travelling belt without testee time, the bar graph that CCD does not photograph in the same time does not change, and its light distribution can be expressed as: I ₀(x, y)=R (x, y) [A (x, y)+B (x, y) cos φ ₀(x, y)] (1)

In formula, R (x, y) is body surface reflectivity, and A (x, y) is background light intensity, and B (x, y) is fringe contrast, φ ₀(x, y) is fringe phase distribution;

When testee is with conveyer belt, the not deforming stripe figure that collects of CCD in the same time, its light distribution can be expressed as:

I _i(x,y)＝R(x,y)[A(x,y)+B(x,y)cosφ _i(x,y)]i＝1,2,...M(2)

φ in formula _i(x, y) is the PHASE DISTRIBUTION of body surface deforming stripe, and object of which movement makes its picture position in every frame deforming stripe figure different, and the distribution of the phase-modulation of generation is also different, and with subscript i difference, M represents shooting frame number;

Step 2) phase place obtain and phase only pupil filter:

A, calculate phase place that in every frame bar line, object height causes change φ by the method for Fourier transform _i(x, y):

Make Fourier transform to formula (2), leach+1 grade of fundamental component, remaking inverse Fourier transform can obtain:

$g_i(x,y)=\frac{R(x,y)\&CenterDot;B(x,y)}{2}\exp \&lsqb;j\&CenterDot;{\mathrm{\&phi;}}_i\left(x,y\right)\&rsqb;---\left(3\right)$

Do same computing to the light distribution (1) of CCD acquisition on reference surface to obtain:

$g_0(x,y)=\frac{R(x,y)\&CenterDot;B(x,y)}{2}\exp \&lsqb;j\&CenterDot;{\mathrm{\&phi;}}_0\left(x,y\right)\&rsqb;---\left(4\right)$

Therefore the phase place caused by object height is changed to:

${\mathrm{\&Delta;\&phi;}}_i(x,y)={\mathrm{\&phi;}}_i(x,y)-{\mathrm{\&phi;}}_0(x,y)=lm\{ln\&lsqb;g_i\left(x,y\right)g_0^{*}\left(x,y\right)\&rsqb;\}---\left(5\right)$

Wherein lm{} represents the imaginary part of getting plural number, and * represents conjugate operation;

The gradient of b, the change of calculating phase place, according to graded by object zoning, obtains abrupt change region template;

With Δ φ _ithe gradient of (x, y) is distributed as foundation, realizes the division of measuring object abrupt change region, to Δ φ _ithe gradient of (x, y) carries out threshold value, obtains the phase gradient figure of binaryzation:

${{\mathrm{\&Delta;\&phi;}}^g}_i(x,y)=\left\{\begin{array}{cc}=1& grad\&lsqb;{\mathrm{\&Delta;\&phi;}}_i(x,y)\&rsqb;\&GreaterEqual;T\\ =0& grad\&lsqb;{\mathrm{\&Delta;\&phi;}}_i(x,y)\&rsqb;<T\end{array}\right.---\left(6\right)$

Wherein grad [] represents gradient algorithm, and T obtains according to system calibrating,

Then to Δ φ ^g _i(x, y) expands, and can obtain abrupt change region Prototype drawing Mask _i(x, y);

C, carrying out related calculation to binaryzation phase diagram obtains ohject displacement;

Intercept binaryzation phase gradient figure Δ φ ^g _ithe characteristic area of (x, y), carries out related calculation with M frame binaryzation phase gradient figure respectively, and the coordinate difference calculating maximal correlation point can obtain ohject displacement;

D, ohject displacement is converted to the phase shift of striped, obtains phase shift bar graph; According to the phase place change that abrupt change region template adopts least square method iterative computation abrupt change region place to be caused by object height;

Abrupt change region template corresponding with it for the M frame phase shift bar graph photographed is multiplied, obtains abrupt change region bar graph:

I′ _i(x,y)＝I _i(x,y)·Mask _i(x,y)(7)

Adjacent N frame is divided into one group, abrupt change region bar graph is divided into groups, according to ohject displacement, cutting is carried out to bar graph, the pixel coordinate realizing object in each group of different distortion bar graph is consistent, also convert ohject displacement to phase shift striped simultaneously, the method for the phase shift bar graph least square method iteration obtained is calculated the phase place change in object abrupt change region;

E, by abrupt change region phase place, formula (5) to be revised, improve phase place and obtain precision;

With the phase place change at the abrupt change region place in abrupt change region phase place change alternate form (5) calculated in d, promote phase place and obtain precision;

Step 3) obtain the X of object by system calibrating, Y-direction information, and phase information is converted to depth information Z.