KR101819141B1 - Method for decoding line structured light patterns by using fourier analysis - Google Patents

Abstract

The present invention relates to a method for decoding line structured light patterns, and more particularly to a method for decoding line structured light patterns using Fourier analysis in scanning and acquiring images.
A method for decoding a line structured light pattern according to the present invention, comprising the steps of: projecting line stripe light onto a target object; Capturing line stripes projected onto the object; Coordinates in the camera space of captured line stripes

For phase

And amplitude

; The phase

And amplitude

Applying a discrete Fourier transform (DFT) And triangulation, the coordinates

And the phase to which the Fourier transform is applied

And reconstructing the three-dimensional coordinates from the three-dimensional coordinates.

Description

TECHNICAL FIELD [0001] The present invention relates to a line-structured optical pattern decoding method using a Fourier analysis,

The present invention relates to a method for decoding line structured light patterns, and more particularly to a method for decoding line structured light patterns using Fourier analysis in scanning and acquiring images.

Optical 3D surfaces measured by Structured Light Illumination (SLI) have been widely applied to various fields such as inspection and scientific measurement. The SLI system comprises a projector, a camera, and a computer. The control by the computer and the projector projects a group of light patterns on the object and the corresponding illuminated scene is stored by the camera. In decoding a patterned image, correspondence between the projector and the camera is derived and is used to reconstrun the three-dimensional surface of the object through triangulation.

Among the various SLI technologies, laser stripe scanning, one of the line structured light methods, is widely used due to its simplicity and high accuracy of three-dimensional surface reformation. The laser stripe scanning technique scans an object with line stripes moving across the object. In general, the conventional method of decoding a line structured light pattern is based on the detection of the peak value of the line stripe in each grabbed frame, and the quality of the peak value detection determines the final measurement accuracy. Various algorithmic applications for complex two-dimensional image processing have been proposed for spatially peak value detection in a single image, and some of them can even achieve up to an accuracy improvement in a subpixel. However, due to the complex geometry of the scanned object as well as the arrangement of the scanning device, there may be a problem of obscurity such as the presence of two or more stripe peaks in one particular column or row. Therefore, this method based on the assumption of neighboring operations and a single peak value can not accurately detect indefinite peak values.

3-Dimensional Restoration of Projection System based on Structured Light-based Compact Camera, ICROS Conference 609 ~ 610, Park, Goo-Won, Park, Soon-

The present invention discloses a new method for decoding line structured light patterns. By applying a Fourier analysis to each pixel along the time axis, the camera-to-projector correspondence through the temporarily calculated phase is induced. Accordingly, it is an object of the present invention to propose a method for solving the problem of unclearness without detecting a peak value spatially.

According to an aspect of the present invention, there is provided a line structured light pattern decoding method using Fourier analysis,

Projecting line stripe light onto a target object; Capturing line stripes projected onto the object; Coordinates in the camera space of captured line stripes

For phase

And amplitude

; The phase

And amplitude

Applying a discrete Fourier transform (DFT) Using triangulation, the coordinates

And the phase to which the Fourier transform is applied

And reconstructing the three-dimensional coordinates from the three-dimensional coordinates.

Here, in the step of projecting the line stripe light, the light intensity of the single line stripe

Can be expressed by the following equation (1).

[Equation 1]

Here,

Is the coordinate in the projector space that projects the light, and n is the

Is an integer value in the range,

Is the resolution of the projector in the vertical direction,

Is an impulse function or a Dirac Delta function,

Is the amplitude of the projected line stripe.

In addition,

May be the maximum intensity value of the line stripe light.

Further, in the step of capturing the projected line stripes, the intensity of the captured light

Can be expressed by the following equation (2).

&Quot; (2) "

Here,

Is the amplitude of the line stripe,

Is the standard deviation of the line stripe width,

Is the intensity of ambient light.

Further, in the step of extracting the phase and the amplitude, the phase and amplitude are

(6) and (7) extracted by comparing Equations (5) and (4) obtained by substituting Equation (3) into Equation (3).

&Quot; (3) "

&Quot; (4) "

&Quot; (5) "

&Quot; (6) "

&Quot; (7) "

In addition, in the step of applying the discrete Fourier transform, the phase and amplitude to which the discrete Fourier transform is applied may be expressed by Equations (9) and (10), respectively.

&Quot; (9) "

&Quot; (10) "

Further, the step of re-forming the three-dimensional coordinates may include:

Is smaller than a predetermined value, the phase applied with the Fourier transform

Can be used for three-dimensional coordinate reformation.

Further, in the step of re-forming the three-dimensional coordinates, the phase to which the Fourier transform is applied

A plurality of phases

(k is greater than or equal to 1) to perform a Multifrequency Phase Unwrapping so that the final unwrapped phase

Lt; / RTI >

By using the present invention, complicated two-dimensional image processing technology is not used, and since all operations are pixelwise, it is possible to easily solve the problem of line stripe indetermination. Also, by using a phase to calculate the three-dimensional surface of the scanned object, the reshaped three-dimensional coordinate can naturally achieve subpixel accuracy.

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.
1 is a three-dimensional measurement system including a projector and a camera.
2 (a) to 2 (d)

,

,

And the final unwrapped phase of the < RTI ID = 0.0 >

(E) to (h) show cross sections of (a) to (d) in the 320th column, respectively.
Fig. 3 (a) is a stored image when n = 190, (b)

, (c) shows the binarized

, (d)

, (e)

, (f) shows the final unwrapped phase

FIG.
Figure 4 shows the front and side views of a reconstructed three dimensional reconstructed plaster bed according to the present invention.
5 (a) shows a scene having a complicated structure, and FIG. 5 (b) shows a stored image at n = 380.
Figs. 6 (a) and 6 (b) show front and side views of three-dimensional point clouds re-formed by the linear approximation method LA, Dimensional point clouds reconstructed by the three-dimensional point clouds.
FIG. 7 shows the relationship between the standard deviation of the phase error

FIG.

BRIEF DESCRIPTION OF THE DRAWINGS The present invention is capable of various modifications and various embodiments, and specific embodiments will be described in detail below with reference to the accompanying drawings.

The following examples are provided to aid in a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, this is merely an example and the present invention is not limited thereto.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail. The following terms are defined in consideration of the functions of the present invention, and may be changed according to the intention or custom of the user, the operator, and the like. Therefore, the definition should be based on the contents throughout this specification. The terms used in the detailed description are intended only to describe embodiments of the invention and should in no way be limiting. Unless specifically stated otherwise, the singular form of a term includes plural forms of meaning. In this description, the expressions "comprising" or "comprising" are intended to indicate certain features, numbers, steps, operations, elements, parts or combinations thereof, Should not be construed to preclude the presence or possibility of other features, numbers, steps, operations, elements, portions or combinations thereof.

It is also to be understood that the terms first, second, etc. may be used to describe various components, but the components are not limited by the terms, and the terms may be used to distinguish one component from another .

Hereinafter, exemplary embodiments of a line structured light pattern decoding method using Fourier analysis according to the present invention will be described in detail with reference to the accompanying drawings.

- Way

A projected line stripe moves vertically across an object, and a single line can be expressed by Equation 1 below.

The coordinates of the projector

Lt; / RTI > is the intensity of light at

It is an integer value in the range. remind

Is the vertical resolution of the projector. remind

Is an impulse function or a Dirac Delta function,

Is the amplitude of the line stripe projected and is generally set to the maximum intensity value of the projector. For example, it is set to 255 in an 8-bit color depth projector. Therefore,

Only the intensity value in the nth column

(white) and the other intensity value is 0 (black).

remind

Coordinates in the camera space

Lt; RTI ID = 0.0 >

Is positively related to the amplitude and surface reflectance of the line stripe,

Is the standard deviation of the line stripe width,

Is the intensity of the ambient light. In general, the camera blurring effect is limited and the width of the stored line stripes is narrow. therefore,

The value is small.

Decoding algorithms for line structured light detect peak values of line stripes in the spatial domain. They are column-wisely,

Lt; / RTI > in the peak value. if,

If there is a peak value for < RTI ID = 0.0 >

Will be used as n (

Peak value). Therefore, the three-dimensional coordinates are obtained through triangulation

As shown in FIG. However, if there is a problem of ambiguity again, the algorithm for spatially detecting the peak value may fail.

In the present invention, a decoding method using a discrete Fourier analysis is applied without using a spatial two-dimensional image processing method

The phase for each is temporarily extracted. By applying the method according to the invention along the time axis for each camera pixel,

Can be removed from the equation for simplicity. The method is described in more detail below.

According to Equation 2,

Is a one-dimensional discrete signal. In other words,

The

Strength of ambient light in the range

And a Gaussian function, and its peak value is

Lt; / RTI >

The

The coordinates of the projector corresponding to the coordinates. In the present invention, in accordance with the temporal order

A discrete Fourier transform (DFT) is performed.

The

Lt; / RTI > is a spectral component of &

to be.

Can be expressed by the following equation (4).

And

Respectively

Respectively.

Applying Equation (3) to Equation (2), the following Equation (5) can be obtained.

Comparing Equations (4) and (5)

Can be obtained as follows.

From Equation (6)

Can be observed.

First, in the range where k is 1 or more,

The

, In other words

The coordinates of the projector corresponding to the coordinates of the projector are calculated. In other words, in the range where k is 1 or more

It is possible to obtain the correspondence between the camera and the projector. Second, if k = 1

Is an absolute phase, and k > 1

Is wrapped in k periods. Third, as described in Equation (2) above,

And

Reflect the reflectance and the influence of the ambient light, respectively. According to Equation (6) above,

Quot;

And

. In other words,

Are not affected by reflectance and ambient light. According to Equation (7) above,

Includes a Gaussian function having a peak value when k = 0. The attenuation ratio of the exponential function, i. E.

Is very small, and in the range where k is 1 or more

Decreases slowly as k increases. Therefore, if k is relatively small, then in the range where k is 1 or more

Can be regarded as a fixed value, as shown in Equation (8) below.

Depending on the discrete Fourier transform (DFT)

And

The

Can be calculated from the following equations (9) and (10), respectively.

Theoretically,

For three-dimensional triangulation in

Can be used directly. However,

(

May result in significant phase errors due to random noise. According to a widely applied approach in the SLI, phase unwrapping can effectively suppress phase error. Therefore, in the present invention, in order to obtain an accurate phase,

And selected <

(k is at least 1). The maximum value of k is

. The final unwrapped phase

, The three-dimensional coordinates are obtained through triangulation

Lt; / RTI >

According to Equation (7), in the range where k is 1 or more

The

, In other words

Lt; / RTI > is proportional to the amplitude of the stored light stripe at. Since the intensity of the projected light is not stored in the shadow area,

Becomes zero. Therefore, in the present invention,

To distinguish valid areas from shaded areas. Specifically,

In

(k is 1 or more), for example,

in

Is greater than a small threshold,

in

Lt; / RTI > are valid and can be included for three dimensional reconstruction.

- Analysis of Phase Error

In practice, noise in an electronic system is an inevitable element. Therefore, in order to analyze the influence of noise in the decoding algorithm according to the present invention,

Is applied to Equation (2), it can be expressed as Equation (11) below.

remind

(STD: Standard Deviation) < RTI ID = 0.0 >

≪ / RTI > Substituting Equation (9) into Equation (11), Equation (12) can be expressed as Equation (12).

Further, if the noise value is small, the phase error

Can be calculated as shown in the following equation (13).

The phase error variance may be modeled according to a Gaussian variance with an average of zero. Therefore, the phase error standard deviation can be used as an index of phase quality, and the smaller the standard deviation, the higher the phase accuracy becomes.

Is derived as shown in Equation (14) below.

By performing multi-frequency phase unwrapping,

The absolute phase

And the standard deviation of the phase error is expressed by the following equation (15). &Quot; (15) "

Is relatively small, as shown in Equation (8) above,

Can be regarded as a fixed value. In this case,

As a result,

. Actually

As the number increases

Is gradually decreased. therefore,

Lt; RTI ID = 0.0 > significantly &

end

Can be stopped. In other words,

Does not necessarily result in an improvement in accuracy.

-Experiment result

As can be seen in FIG. 1, the 3D measurement system according to the present invention comprises an 8-bits-depth Casio XJ-A 155V projector with 800 x 600 resolution and an 8-bits-depth Prosilica GC 650 camera with 640 x 480 resolution do. All line patterns generated from Equation (1)

= 255 and n changes from 0 to 599.

- visual proof of the unwrapped phase

To visually demonstrate that phase unwrapping effectively reduces phase error, it scans the flat white plate and shows the phases before and after performing unwrapping. Using Equation 9 with k = 1,6,24,

Respectively. Then, the final unwrapped phase

Lt; RTI ID = 0.0 > phase < / RTI &

,

,

.

,

,

And

Can be seen in Figures 2 (a) - (d), respectively.

,

,

And

Cross sections in the 320 < th > column of Fig. 2 (b) are shown in Figs. 2 (e) to 2 (h). If there is no contamination by noise,

And

Will have an ideal slope cross section. Comparing FIG. 2 (h) and FIG. 2 (e)

Is distorted by the random noise, while the final unwrapped phase

Lt; RTI ID = 0.0 > accuracy. ≪ / RTI >

- Comparison of measurement accuracy

In order to compare the accuracy of the method according to the invention with the conventional spatial peak value detectors, a flat white plate is scanned and the phase error is calculated. In the process according to the invention,

= 64. Conventional peak value detectors have been adopted for comparison involving Gaussian approximation, Linear Approximation (LA) and the center of an object. These spatial detectors may include a sub-

And integer

In terms of convenience of comparison,

Lt; RTI ID = 0.0 >

This mapping changes to phase. The actual phase can be obtained from a Multifrequency Phase Measuring Profilometry (PMP). From Table 1 below, it can be seen that the accuracy of the method according to the invention is best from the standard deviation values of the phase error according to the other methods.

Three-dimensional reforming

Before three-dimensional scanning, it is necessary to calibrate the system according to the invention. A calibration target having 40 feature points that know the actual coordinates is used. First, by extracting the two-dimensional feature from the image taken from the calibration target, corresponding features can be obtained between the world coordinate and the camera coordinate. From the corresponding features, a real-

Is calculated. Secondly, not only the camera-projector equivalent obtained using the PMP but also the features extracted from the camera coordinates are placed in the project coordinates. Finally, from the matched characteristics between the world coordinates and the projector coordinates, a real-to-projector transformation matrix

Is calculated. The calibration parameters

And

Will then be used for phase-to-coordinate conversion.

The plaster of Fig. 1 above is visually reformed to demonstrate the method according to the invention. First, the gypsum phase is scanned using the line structured light pattern in equation (1). 3 (a) is an image stored when n = 190. Second, from the stored image, as can be seen in Figure 3 (b)

. Then, at a threshold of 40

And as shown in Fig. 3 (c), the binarized

Is used as a mask to remove an invalid image of the shaded area. Third, by using equation (9)

,

,

And

Respectively. From the extracted phase, the final unwrapped phase using multiple frequency phase unwrapping

. 3 (d) to 3 (f)

,

,

And

. 3-D point clouds are reconstructed by triangulation. 4 (a) and 4 (b) show a front view and a side view of a three-dimensional point cloud.

In order to illustrate the advantages of the method according to the present invention for the case where the line stripe is unclear, the scene of the complex structure shown in Fig. 5 (a) is scanned. FIG. 5 (b) is a stored image at n = 380, and there is a problem of unclearness. For example, several columns may have multiple peak values. When applying a linear approximation (LA), there are obvious drawbacks in its 3D reconstruction point cloud, as seen in the box area of Figs. 6 (a) and 6 (b). With the method according to the present invention, the problem of obscurity can be successfully solved as shown in the box regions of Figs. 6 (c) and 6 (d).

-

And Measurement Accuracy

As can be seen from equation (15) above, the accuracy of the final unwrapped phase is

.

In order to experimentally verify the measurement accuracy, a flat white plate was scanned using line structured light,

Calculate the standard deviation of the phase error while varying the value. As can be seen in Figure 7,

In the range of less than 20

The standard deviation of the phase error sharply decreases, and 20 <

&Lt; 120 &lt; / RTI &gt;

If it exceeds 120, it increases again. This result is consistent with the phase error analysis discussed above. Therefore, in order to obtain an accurate phase under the experimental conditions according to the present invention,

Is in the range of [20,120].

- conclusion

The present invention discloses a new method for decoding line structured light patterns using discrete Fourier analysis (DFT). By performing a discrete Fourier analysis on the stored images along the time axis, some phase maps of the selected spectral components are extracted. Then, the final phase used for three-dimensional reconstruction from the extracted phase map is obtained by Multifrequency Phase Unwrapping. The method according to the present invention is stable, robust, without neighboring operations during extraction of line stripes. Experimental results also show that the method according to the present invention is accurate and can solve the problem of obscurity more successfully than the conventional approach based on the detection of the stripe peak value.

Claims (9)

A method for decoding a line structured light pattern,
Projecting line stripe light onto a target object;
Capturing line stripes projected onto the object;
Coordinates in the camera space of captured line stripes

For phase

And amplitude

;
The phase

And amplitude

Applying a discrete Fourier transform (DFT)
Using triangulation, the coordinates

And the phase to which the Fourier transform is applied

And reconstructing the three-dimensional coordinates from the three-dimensional coordinates.
The method according to claim 1,
In the step of projecting the line stripe light,
Strength of light of a single line stripe

Is expressed by Equation (1). &Lt; EMI ID = 1.0 &gt;
[Equation 1]

Here,

Is the coordinate in the projector space that projects the light, and n is the

Is an integer value in the range,

Is the resolution of the projector in the vertical direction,

Is an impulse function or a Dirac Delta function,

Is the amplitude of the projected line stripe.
3. The method of claim 2,
The constant

Is a maximum intensity value of line stripe light.
3. The method of claim 2,
In the step of capturing the projected line stripes,
Strength of captured light

Is expressed by Equation (2). &Lt; EMI ID = 1.0 &gt;
&Quot; (2) &quot;

Here,

Is the amplitude of the line stripe,

Is the standard deviation of the line stripe width,

Is the intensity of ambient light.
5. The method of claim 4,
In extracting the phase and amplitude,
Wherein the phase and amplitude are

(6) and (7) extracted by comparing Equations (5) and (4) obtained by substituting Equation
here,

The

Lt; / RTI &gt; is a spectral component of &

&Lt; / RTI &gt; wherein the line structured light pattern decoding method comprises the steps of:
&Quot; (3) &quot;

&Quot; (4) &quot;

&Quot; (5) &quot;

&Quot; (6) &quot;

&Quot; (7) &quot;

6. The method of claim 5,
In applying the discrete Fourier transform,
Wherein the phase and amplitude applied to the discrete Fourier transform are respectively expressed by Equations (9) and (10).
&Quot; (9) &quot;

&Quot; (10) &quot;

The method according to claim 1,
Wherein the step of reforming the three-
Amplitude with Fourier Transform

Is larger than the predetermined value, the phase applied with the Fourier transform

Is used for three-dimensional coordinate re-formation.
The method according to claim 1,
In the step of reforming the three-dimensional coordinates,
The phase to which the Fourier transform is applied

A plurality of phases

(k is greater than or equal to 1) to perform a Multifrequency Phase Unwrapping so that the final unwrapped phase

&Lt; / RTI &gt; wherein the line structured light pattern decoding method comprises the steps of:
A program stored in a computer-readable medium for executing the steps of any one of claims 1 to 8 in a computer.

Images