JP2008190962A - Three-dimensional measurement apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a three-dimensional measurement apparatus for sufficiently reducing an effect due to a hand movement even if a projecting section and an imaging section are not operated at a high speed. <P>SOLUTION: The three-dimensional measurement apparatus comprises: a projecting section 4 for shifting a phase at a predetermined angle, and projecting a pair of stripe patterns onto an object 2; the imaging section 5 for imaging the object 2 onto which the stripe patterns are projected, and obtaining a pair of stripe pattern images; and an image processing section 6 for recovering a phase value from a pair of the stripe pattern images pixel by pixel, and measures a surface profile of the object 2. The image processing section 6 comprises: a displacement calculating means 61 for calculating a displacement between a pair of the stripe pattern images; and a displacement correcting means 62 for moving pixels by the displacement, and adding a phase shift obtained by comparing the displacement with a wavelength of the stripe patterns to the phase of the stripe patterns. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a three-dimensional measurement apparatus using a phase shift method.

  A phase shift method is known as a measuring means in a three-dimensional measuring apparatus that measures a three-dimensional shape of a subject without contact (for example, Patent Documents 1 and 2). This phase shift method is as follows. First, a process of projecting a fringe pattern whose luminance changes in a sine wave shape onto the surface of the subject and photographing the image from another direction is performed by sequentially shifting the phase of the fringe pattern by a predetermined angle (π / 2 or 2π / 3). By shifting and performing only one cycle, one set (four or three) of captured images is obtained. Next, from the brightness difference between these captured images, the phase of the fringe pattern on the subject is reproduced by performing a series of image processing such as calculating the phase value of the fringe pattern that has changed according to the surface shape of the subject, Based on this, the surface shape of the subject is measured.

  By the way, the three-dimensional measuring apparatus is used in various fields. For example, there is an apparatus that examines a shape change due to corrosion by three-dimensionally measuring a part such as a pipe in a factory facility and inspects its remaining service life. . In the previous application (Japanese Patent Application No. 2005-329448), the applicant of the present application took a set of images in a short time so that parts such as pipes in a factory facility can be easily measured in the field. The three-dimensional measurement device has been proposed in which the influence of camera shake is reduced by repeating the operation many times.

JP-A-6-66527 Japanese Patent Laid-Open No. 11-14327

  However, in the above-mentioned previous application, in order to repeatedly perform imaging for a short time such as about 1 millisecond, for example, the projection unit needs to be a high-performance one that operates at high speed, and the imaging unit operates at high speed. It must be a high-performance device or a special device that switches between a plurality of light-receiving elements, and as a result, the price has to be high.

  The present invention has been made in view of the above-described reasons, and an object of the present invention is to provide a three-dimensional measurement apparatus that can sufficiently reduce the influence of camera shake even when the projection unit and the photographing unit do not operate at high speed. There is to do.

  In order to solve the above-described problem, a three-dimensional measurement apparatus according to claim 1 includes a projection unit that shifts a phase by a predetermined angle and projects a set of stripe patterns onto a subject, and a subject on which each stripe pattern is projected. A three-dimensional measuring apparatus for measuring a surface shape of a subject, including an imaging unit that captures a pair of stripe pattern images and an image processing unit that restores a phase value for each pixel from the set of stripe pattern images In the image processing unit, a positional deviation amount calculating means for calculating a positional deviation amount between a pair of stripe pattern images, a pixel position is moved by the positional deviation amount, and the positional deviation amount is converted into a fringe pattern. And a positional deviation amount correcting means for adding the phase deviation amount obtained by comparison with the wavelength of the first to the phase of the fringe pattern.

  A three-dimensional measurement apparatus according to a second aspect is the three-dimensional measurement apparatus according to the first aspect, wherein the wavelength of the fringe pattern of the misalignment correction unit is fitted to a fringe pattern image at a plane or substantially plane portion. It is obtained by applying a sine wave.

  The three-dimensional measurement apparatus according to claim 3 is the three-dimensional measurement apparatus according to claim 1, wherein the fringe pattern wavelength of the misregistration amount correction means is a fringe pattern obtained by two-dimensional Fourier transform of the fringe pattern image and projected. It is obtained from the coordinates at which the peak value near the wavelength of the pattern appears.

  A three-dimensional measurement apparatus according to a fourth aspect is the three-dimensional measurement apparatus according to the third aspect, wherein the misregistration amount calculation means performs a two-dimensional Fourier transform of a fringe pattern image, and the misregistration amount correction means has a fringe pattern. The wavelength of the pattern is obtained using a two-dimensional Fourier transform result of the fringe pattern image performed by the positional deviation amount calculation means.

  A three-dimensional measurement apparatus according to a fifth aspect is the three-dimensional measurement apparatus according to any one of the first to fourth aspects, wherein the positional deviation amount is calculated and corrected for each pixel, and the positional deviation amount calculating means includes: The positional deviation amount of the pixel i is calculated from the stripe pattern image in the window with the pixel i as the center.

  In the three-dimensional measurement apparatus of the present invention, the positional deviation amount correcting means of the image processing unit obtains the positional deviation amount by comparing the calculated positional deviation amount with the wavelength of the fringe pattern. Since the phase shift amount to be added is added to the phase of the fringe pattern, the influence of camera shake can be sufficiently reduced without operating the projection unit and the imaging unit so fast.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, the best embodiment of the invention will be described with reference to the drawings. FIG. 1 is a perspective view showing a use state of a three-dimensional measuring apparatus 1 according to an embodiment of the present invention. The three-dimensional measurement apparatus 1 measures the surface shape of the subject 2 to be measured by the phase shift method, and has a projection unit 4, an imaging unit 5, and image processing, which will be described later, in a compact housing that can be held. Part 6 and the like are accommodated. The casing has, for example, a width of about 25 cm, a height of about 10 cm, and a depth of about 15 cm.

  FIG. 2 is a schematic diagram showing the overall configuration of the three-dimensional measuring apparatus 1. The three-dimensional measuring apparatus 1 sequentially shifts the phase by a predetermined angle and projects a set of stripe patterns onto the subject 2 and sequentially shoots the subject 2 on which each of the stripe patterns is projected and sets a set of stripes. An imaging unit 5 that obtains a pattern image (an image of the subject 2 including a fringe pattern) and an image processing unit 6 that performs a series of image processing and restores a phase value for each pixel from a set of fringe pattern images. Based on the restored phase value of each pixel, the surface shape of the subject 2 is measured.

The stripe pattern projected by the projection unit 4 has a luminance (continuous gradation) that changes in a sine wave shape. The wavelength λ ₀ of the fringe pattern to be projected is 16 pixels (length of 16 pixels) in the example of this embodiment. The photographing unit 5 includes a light receiving element (for example, a CCD), and captures and holds a single fringe pattern image from the subject 2 within the time during which the fringe pattern is projected onto the subject 2 (projection time). In the example of the present embodiment, the number of pixels of one striped pattern image is 1024 × 768.

The predetermined angle of the phase shift in the projection unit 4 can be, for example, π / 2 (90 degrees) or 2π / 3 (120 degrees), and a set of four or three fringe pattern images accordingly. It consists of a fringe pattern image. In the present embodiment, the predetermined angle is π / 2, and one set of stripe pattern images is composed of four stripe pattern images. The projection unit 4 projects one stripe pattern once in order every time the predetermined time _{TE has} elapsed. For example, if the predetermined time _TE is 60 milliseconds, the first stripe pattern is obtained by projecting the first stripe pattern, and the second stripe pattern is obtained 60 milliseconds after the start of the first projection. Is projected to obtain a second striped pattern image, and the third striped pattern is projected 60 milliseconds after the second projection starts to obtain a third striped pattern image. A fourth stripe pattern is projected 60 milliseconds after the start to obtain a fourth stripe pattern image.

The predetermined time T _E, the setting time and the projection time of the fringe pattern in the projection portion 4, obtain one fringe pattern image capturing section 5 has comprehensive and data read time of the fringe pattern image holding by the image processing section 6 Is the time needed for. The predetermined time T _E is dependent on the hardware performance of the projection unit 4 and the imaging unit 5. When the predetermined time _TE is long, the amount of positional deviation between a pair of stripe pattern images tends to increase due to the influence of camera shake.

Assuming that the luminance values of the pixel i of the first, second, third, and fourth stripe pattern images are P _1i , P _2i , P _3i , and P _4i , these are expressed by the following equations (1) to (4). The
P _1i = K _i × (I _BASE + I _S × cos (Φ _i ))
P _2i = K _i × (I _BASE + I _S × cos (Φ _i + θ _2i ))
P _3i = K _i × (I _BASE + I _S × cos (Φ _i + θ _3i ))
P _4i = K _i × (I _BASE + I _S × cos (Φ _i + θ _4i ))

Here, K _i is the reflectivity of the object 2 corresponding to the pixel i, I _BASE is the central value of the light intensity of the fringe pattern, I _S is the amplitude value of the light amount of the fringe pattern, [Phi _i is the phase to be restored pixel i Value. θ _2i , θ _3i , and θ _4i are phase shift amounts, and are constants represented by the following equations (5) to (7) assuming that there is no positional shift between a pair of stripe pattern images.
θ _2i = π / 2
θ _3i = π
θ _4i = 3π / 2

The image processing unit 6, the image reading means 60 is shooting unit 5 within a predetermined time T _E reads the data of the fringe pattern image to be maintained. Then, the positional deviation amount calculating means 61 calculates the positional deviation amount between a pair of stripe pattern images, and the positional deviation amount correcting means 62 corrects the positional deviation amount. The calculation of the amount of positional deviation and the correction thereof may be performed for the entire stripe pattern image or for each pixel. In the case of the entire fringe pattern image, it is possible to deal with a relative translational displacement between the fringe pattern images. In the case of each pixel, it is possible to cope with a relative translation and a rotational misalignment between the stripe pattern images. In addition, in the present embodiment, the first fringe pattern image is used as the reference fringe pattern image, although one set of the amount of positional deviation between the pair of fringe pattern images is the reference fringe pattern image. In addition, an n-th striped pattern image, a positional deviation amount (δ _nX , δ _nY ), a phase shift amount θ _{ni, and the} like, which will be described later, are represented by n in the notation of the present specification. And After the calculation and correction of the positional deviation amount, the phase restoring unit 63 restores the phase value of each pixel.

The amount of positional deviation (δ _nX , δ _nY ) for each entire fringe pattern image is calculated by a phase correlation method using the entire fringe pattern image or the fringe pattern image in the set window. It is preferable to set a window in order to reduce the amount of calculation when the amount of positional deviation is calculated by converting an image like the phase correlation method. The size of the window may be set in consideration of the calculation amount and the calculation accuracy, and is set to 256 × 256 pixels in the present embodiment. The part where the window is set is a part characteristic of the surface shape of the subject 2. In the phase correlation method, two images are subjected to two-dimensional Fourier transform for two images, and they are combined to perform two-dimensional inverse Fourier transform. When the two images are approximated, there is a coordinate indicating a steep peak value, and the coordinate indicates the amount of positional deviation between the two images.

It is also possible to attach the gyro to the three-dimensional measuring apparatus 1 and calculate the amount of displacement (δ _nX , δ _nY ). When the gyro is used, even when it is difficult to calculate the displacement amount from the stripe pattern image with little change in the surface shape of the subject 2, the displacement amount can be easily calculated.

Correction of the positional deviation amount (δ _nX , δ _nY ) for each of the stripe pattern images is performed in the following two stages. In the first misregistration correction, the n-th fringe pattern image is shifted by the misalignment amount (δ _nX , δ _nY ) so as to match the first fringe pattern image except for the fringe pattern. Move in the direction. FIG. 3 is an explanatory diagram of a specific example of this state. (A ₁ ) to (A ₄ ) are the stripe pattern images of the first to fourth windows before correction, and (B ₁ ) to (B ₄ ) are 1 to 4 sheets after the first positional deviation amount correction. It is a stripe pattern image in the window of the eye. The positional deviation amounts (δ _2X , δ _2Y ), (δ _3X , δ _3Y ), (δ _4X , δ _4Y ) of the second, third, and fourth striped pattern images with respect to the first striped pattern image are ( -2.96, -9.52), (-8.45, -4.04), and (-10.95, 7.44) are calculated. The second, third, and fourth stripe pattern images are moved by (2.96, 9.52), (8.45, 4.04), and (10.95, −7.44), respectively. Thus, the first misalignment correction is performed.

In the second positional deviation amount correction, the phase shift amount θ _ni of the fringe pattern of the nth fringe pattern image is corrected. The phase shift amount θ _ni need not be corrected in the Y-axis direction if the fringe pattern is projected completely (or almost completely) perpendicular to the X-axis. In this case, the amount of phase shift δ _nX in the X-axis direction is compared with the wavelength λ of the fringe pattern to obtain the phase shift amount, which is added to obtain a phase shift as shown in the following equations (5.1) to (7.1) The amount θ _ni is corrected.
θ _2i = π / 2 + δ _2X / λ × 2π
θ _3i = π + δ _3X / λ × 2π
θ _4i = 3π / 2 + δ _4X / λ × 2π
In the example of FIG. 3, the second positional deviation amount correction is expressed by the following equations (5.1 ′) to (7.1 ′).
θ _2i = π / 2 + 2.96 / 16 × 2π
θ _3i = π + 8.45 / 16 × 2π
θ _4i = 3π / 2 + 10.95 / 16 × 2π

If the fringe pattern is projected obliquely with respect to the X axis, the wavelength λ is considered to be composed of components (λ _X , λ _Y ) in the X axis direction and the Y axis direction. In this case, the amount of phase shift (δ _nX , δ _nY ) in the X-axis and Y-axis directions is compared with the fringe pattern wavelength (λ _X , λ _Y ) to obtain the phase shift amount, The phase shift amount θ _ni is corrected as shown in equations (5.2) to (7.2).
θ _2i = π / 2 + (δ _2X / λ _X + δ _2Y / λ _Y ) × 2π
θ _3i = π + (δ _3X / λ _X + δ _3Y / λ _Y ) × 2π
θ _4i = 3π / 2 + (δ _4X / λ _X + δ _4Y / λ _Y ) × 2π
Since these fringe patterns if perfectly perpendicular to the X axis lambda _Y becomes infinite, matching (5.1) - (7.1) below.

In order to improve the accuracy of the second positional deviation correction, the fringe pattern wavelength λ is corrected. This is because the angle at which the fringe pattern is projected onto the subject 2 is different from the angle at which the subject 2 is photographed, and therefore there is a wavelength λ of the captured fringe pattern in the vicinity of the wavelength λ ₀ of the projected fringe pattern.

  One method for correcting the wavelength λ of the fringe pattern utilizes the fact that only the sine wave of the projected fringe pattern is observed on the plane portion of one fringe pattern image. By applying a fitting sine wave to the observed sine wave, the value of the wavelength λ of the fringe pattern can be obtained. For this purpose, as shown in FIG. 4, a flat reference plate 7 is placed at the end of the subject 2 and is photographed together with the subject 2. Then, the value of the wavelength λ of the stripe pattern is obtained from one stripe pattern image (for example, the first stripe pattern image) on the reference plate 7. It is also possible to obtain the value of the wavelength λ of the fringe pattern by applying a sine wave that fits in a substantially plane portion of one fringe pattern image without using the reference plate 7.

Another method for correcting the wavelength λ of the fringe pattern uses the result of a two-dimensional Fourier transform. The result of the two-dimensional Fourier transform of the fringe pattern image of a predetermined window (for example, 256 × 256 pixels) corresponds to the wavelength λ ₀ of the projected fringe pattern even if the surface shape of the subject 2 is complicated. A peak value appears near the coordinates. This method is particularly effective in the case of calculating the amount of misalignment by the phase correlation method because if the two-dimensional Fourier transform is used for calculating the amount of misalignment, the result of the two-dimensional Fourier transform can be used again. is there.

FIG. 5 is an explanatory diagram of a method for obtaining the wavelength of the fringe pattern using the result of the two-dimensional Fourier transform. The specific example shown in FIG. 5 is a different example from FIG. 5A shows a projected image (256 × 256 pixels) of a stripe pattern, and FIG. 5B shows the result of the two-dimensional Fourier transform. In (B), the brightness represents the magnitude of the coefficient, and the peak value appears at the coordinates (-17, -3), (17, 3). Therefore, the fringe pattern is projected obliquely with respect to the X axis, the wavelength λ _{X in} the X axis direction is 128 ÷ 17 = 7.53 pixels, and the wavelength λ _{Y in the} Y axis direction is 128 ÷ 3 = 42.67 pixels. Is required.

It is desirable to evaluate the positional deviation amount between the calculation and correction of the positional deviation amount. FIG. 6 is a processing flowchart showing the evaluation of the amount of misalignment. After the calculation of the positional deviation amount (S101), the positional deviation amount δ _nX in the X-axis direction is _converted into the positional deviation evaluation minimum value K ₁ (for example, 1/20 pixel) and the positional deviation evaluation maximum value K ₂ (for example, 20 pixels). (S102). If positional shift amount [delta] _nX of the X-axis direction is less positional shift evaluation minimum K _1, since sufficient reconstruction accuracy can be obtained, not performed the correction of positional deviation amount for suppressing the amount of calculation (S103). If the position deviation amount [delta] _nX is above positional deviation evaluation maximum value K _2, since not perform appropriate correction is not performed three-dimensional measurement by the phase shift method. The value of K _2, when using ordinary attention in three-dimensional measurement device 1, to a value such as position shift amount [delta] _nX is than this sufficiently small. If the position deviation amount [delta] _nX is above positional deviation evaluation maximum value K _2, where the image processing can be canceled, it is also possible to perform instead a simple three-dimensional measurement. Although details are omitted in the present embodiment, three-dimensional measurement is performed by a method of spatially calculating a phase value using only the first striped pattern image (S104). When the positional deviation amount δ _nX is larger than the positional deviation evaluation minimum value K _{1 and} smaller than the positional deviation evaluation maximum value K ₂ , the positional deviation amount is corrected (S 105).

Next, calculation and correction of the positional deviation amount (δ _nXi , δ _nYi ) for each pixel will be described.

For the positional deviation amount (δ _nXi , δ _nYi ) of the pixel i in the nth striped pattern image, specifically, a window (for example, 64 × 64 pixels) centered on the pixel i is set, and the stripe of this window The positional deviation amount calculated using the pattern image is set as the positional deviation amount of the pixel i. Then, the first striped pattern image is moved in the X-axis and Y-axis directions by the amount of displacement (δ _nXi , δ _nYi ) so as to coincide with the first striped pattern image excluding the striped pattern. The amount of misalignment is corrected. Next, the wavelength (λ _nXi , λ _nYi ) of the stripe pattern in the pixel i is obtained. Specifically, the result of the two-dimensional Fourier transform of the window used in the calculation of the positional deviation amounts (δ _nXi , δ _nYi ) is used, and the calculated wavelength is set as the wavelength of the fringe pattern in the pixel i. Therefore, it is desirable to calculate the amount of displacement by a phase correlation method that can share the result of the two-dimensional Fourier transform.

Then, the positional shift amounts (δ _nXi , δ _nYi ) are _compared with the fringe pattern wavelengths (λ _nXi , λ _nYi ) to obtain a phase shift amount, and adding them, the following equations (5.3) to (7.3) The phase shift amount θ _ni is corrected as shown in FIG.
θ _2i = π / 2 + (δ _2Xi / λ _2Xi + δ _2Yi / λ _2Yi ) × 2π
θ _3i = π + (δ _3Xi / λ _3Xi + δ _3Yi / λ _3Yi ) × 2π
θ _4i = 3π / 2 + (δ _4Xi / λ _4Xi + δ _4Yi / λ _4Yi ) × 2π

When the phase shift amount θ _ni is corrected as described above, the equations (1) to (4), which are simultaneous equations with K _i , I _BASE , I _S , and Φ _i as variables, are obtained by the phase restoring means 63. To restore the phase value Φ _i of the pixel i. Specifically, the value of Φ _i can be obtained as follows. First, α _i , β _i , and γ _i are defined as in the following equations (8) to (10).
α _i = K _i × I _S × cos (Φ _i )
β _i = K _i × I _S × sin (Φ _i )
γ _i = K _i × I _BASE
Then, the equations (1) to (4) are transformed into the following equations (11) to (14).
P _1i = γ _i + α _i
P _2i = γ _i + α _i × cos (θ _2i ) −β _i × sin (θ _2i )
P _3i = γ _i + α _i × cos (θ _3i ) −β _i × sin (θ _3i )
_P4i = [gamma] _i + [alpha] _i * cos ([theta] _4i )-[beta] _i * sin ([theta] _4i ).
Α _i , β _i , and γ _i are obtained from the equations (11) to (14). And the following formula (15)
Φ _i = arctan (β _i / α _i )
Therefore, the phase value Φ _i of the pixel _i is obtained.

If there is no displacement, the displacement amounts (δ _nXi , δ _nYi ) are 0, so Φ _i is expressed by the following equation (15.1).
Φ _i = arctan ((P _4i -P _2i ) / (P _1i -P _3i ))

In this manner, the image processing unit 6 corrects the amount of positional deviation between a pair of stripe pattern images, and Φ _i that is the phase value of the pixel _i is appropriately restored. Therefore, even if the projection unit 4 and the photographing unit 5 are not hardware capable of operating at high speed and the predetermined time _TE is long, the influence of camera shake can be sufficiently reduced. Moreover, even if it is used in a situation where vibration due to mechanical vibration or the like occurs, the influence can be sufficiently reduced.

  The three-dimensional measurement apparatus according to the embodiment of the present invention has been described above. However, the present invention is not limited to the one described in the embodiment, and various designs within the scope of the matters described in the claims. It can be changed. For example, the one that calculates and corrects the amount of misalignment for each pixel described above can perform the correction with the highest accuracy, but requires a large amount of calculation. Therefore, the amount of calculation can be reduced by obtaining a common value for the entire stripe pattern image without obtaining the wavelength of the stripe pattern for each pixel.

It is a perspective view which shows the use condition of the three-dimensional measuring apparatus which concerns on embodiment of invention. It is a schematic diagram of the whole structure of a three-dimensional measuring apparatus same as the above. It is explanatory drawing of the specific example of the state of correction | amendment of the amount of positional deviation same as the above. It is a perspective view of 1 system which calculates | requires the wavelength of a fringe pattern same as the above. It is explanatory drawing of another system which calculates | requires the wavelength of a fringe pattern same as the above. It is a processing flowchart which shows evaluation of the amount of positional deviation same as the above.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 3D measuring device 2 Subject 4 Projection part 5 Imaging | photography part 6 Image processing part 60 Image reading means 61 Position shift amount calculation means 62 Position shift amount correction means 63 Phase restoration means

Claims (5)

A projection unit that shifts the phase by a predetermined angle and projects a set of stripe patterns onto a subject, a photographing unit that captures a subject on which each stripe pattern is projected and obtains a set of stripe pattern images, and a set of stripes An image processing unit that restores a phase value for each pixel from a pattern image; and a three-dimensional measurement apparatus that measures the surface shape of a subject,
The image processing unit is configured to calculate a positional deviation amount between a pair of fringe pattern images, move a pixel position by the positional deviation amount, and change the positional deviation amount to the wavelength of the fringe pattern. And a misregistration amount correction means for adding the misregistration amount obtained by comparison to the phase of the fringe pattern. The three-dimensional measuring apparatus according to claim 1,
The three-dimensional measuring apparatus according to claim 1, wherein the wavelength of the fringe pattern of the misregistration amount correcting means is obtained by applying a sine wave that fits the fringe pattern image in a plane or substantially plane portion. The three-dimensional measuring apparatus according to claim 1,
The wavelength of the fringe pattern of the misregistration amount correction means is obtained by coordinates obtained by performing a two-dimensional Fourier transform on the fringe pattern image and the peak value near the wavelength of the projected fringe pattern appears. Measuring device. In the three-dimensional measuring apparatus described in Claim 3,
The misregistration amount calculation means performs a two-dimensional Fourier transform of the fringe pattern image,
The three-dimensional measuring apparatus characterized in that the wavelength of the fringe pattern of the misregistration amount correcting means is obtained using a two-dimensional Fourier transform result of the fringe pattern image performed by the misregistration amount calculating means. The three-dimensional measurement apparatus according to any one of claims 1 to 4,
The calculation and correction of the displacement amount is performed for each pixel,
The three-dimensional measuring apparatus characterized in that the positional deviation amount calculating means calculates the positional deviation amount of the pixel i from a stripe pattern image in the window with the pixel i as the center.