JP2022130940A - Three-dimensional measuring device and three-dimensional measuring method - Google Patents

Abstract

To prevent reduction in reliability of three-dimensional measurement.SOLUTION: A three-dimensional measuring device comprises: a projection control unit that controls a projection device so that a plurality of types of stripe patterns of sinusoidal brightness distribution different in a phase from each other is projected on an object; an imaging frequency determination unit that determines a frequency of acquisition of a basic image of an object acquired in projection of one type of a stripe pattern; an imaging control unit that controls an imaging apparatus so that the basic image is acquired by the number of times equal to the acquisition frequency in projection of the plurality of types of stripe patterns; a composite image creation unit that composes the basic images in the number equal to the acquisition frequency to create a plurality of composite images corresponding to the plurality of types of stripe patterns; and a three-dimensional shape calculation unit that calculates a three-dimensional shape of the object on the basis of, the plurality of composite images.SELECTED DRAWING: Figure 2

Description

The present disclosure relates to a three-dimensional measuring device and a three-dimensional measuring method.

A phase shift method as disclosed in Patent Document 1 is known as a three-dimensional measurement method for an object. When measuring the three-dimensional shape of an object based on the phase shift method, a fringe pattern with a sinusoidal lightness distribution is projected onto the object while shifting the phase. An image of the object is obtained by capturing an image of the object on which the fringe pattern is projected.

Japanese Patent Application Laid-Open No. 2001-124534

For example, if the surface of the object has low light reflectance, the image of the object acquired by the imaging device may be unclear. If the image of the object becomes unclear, the reliability of the three-dimensional measurement may decrease.

An object of the present disclosure is to suppress deterioration in the reliability of three-dimensional measurement.

According to the present disclosure, a projection control unit that controls a projection device so that a plurality of types of fringe patterns with sinusoidal lightness distributions with mutually different phases are projected onto an object, and one type of fringe pattern obtained in projection. an imaging control unit that controls the imaging device so that the basic image is obtained by the number of acquisition times in each projection of a plurality of types of stripe patterns; and the acquisition number of times. A synthetic image generation unit that synthesizes the basic images to generate a plurality of synthetic images corresponding to each of the plurality of types of striped patterns, and a 3D shape calculation that calculates the 3D shape of the object based on the multiple synthetic images A three-dimensional measurement device is provided, comprising:

According to the present disclosure, deterioration in reliability of three-dimensional measurement is suppressed.

FIG. 1 is a diagram schematically showing a three-dimensional measuring device according to an embodiment. FIG. 2 is a functional block diagram showing the control device according to the embodiment. FIG. 3 is a diagram showing a stripe pattern according to the embodiment. FIG. 4 is a diagram illustrating parameters of a sine wave according to an embodiment; FIG. 5 is a flow chart showing a three-dimensional measurement method according to the embodiment. FIG. 6 is a schematic diagram showing a three-dimensional measurement method according to the embodiment.

Hereinafter, embodiments according to the present disclosure will be described with reference to the drawings, but the present disclosure is not limited to the embodiments. The constituent elements of the embodiments described below can be combined as appropriate. Also, some components may not be used.

In the embodiment, an XYZ orthogonal coordinate system is set, and the positional relationship of each part will be described with reference to the XYZ orthogonal coordinate system. The direction parallel to the X-axis of a predetermined surface is defined as the X-axis direction, the direction parallel to the Y-axis of a predetermined surface orthogonal to the X-axis is defined as the Y-axis direction, and the direction parallel to the Z-axis orthogonal to the predetermined surface is defined as the Z-axis direction. and The rotation or tilting direction about the X axis is the θX direction, the rotation or tilting direction about the Y axis is the θY direction, and the rotation or tilting direction about the Z axis is the θZ direction. The XY plane is a predetermined plane.

[Three-dimensional measuring device]
FIG. 1 is a diagram schematically showing a three-dimensional measuring device 1 according to an embodiment. A three-dimensional measurement apparatus 1 measures the three-dimensional shape of an object W based on the phase shift method. As shown in FIG. 1 , the three-dimensional measurement device 1 includes a stage 2 , a projection device 3 , an imaging device 4 and a control device 5 .

The stage 2 supports an object W to be measured.

Projector 3 projects a fringe pattern onto object W supported by stage 2 . The projection device 3 projects a fringe pattern with a sinusoidal lightness distribution onto the object W while shifting the phase. The projection device 3 has a light source 31 , a light modulation element 32 and a projection optical system 33 . Light source 31 generates light. The light modulation element 32 optically modulates the light emitted from the light source 31 to generate a fringe pattern. Examples of the light modulation element 32 include a digital mirror device (DMD), a transmissive liquid crystal panel, and a reflective liquid crystal panel. The projection optical system 33 projects the fringe pattern generated by the light modulation element 32 onto the object W. FIG.

The imaging device 4 images the object W on which the fringe pattern is projected. The imaging device 4 acquires an image of the object W on which the stripe pattern is projected. The imaging device 4 has an imaging optical system 41 and an imaging device 42 . The imaging optical system 41 forms an image of the fringe pattern reflected by the object W on the imaging device 42 . The imaging device 42 acquires an image of the object W via the imaging optical system 41 . As the imaging device 42, a CMOS image sensor (Complementary Metal Oxide Semiconductor Image Sensor) or a CCD image sensor (Charge Coupled Device Image Sensor) is exemplified.

Controller 5 includes a computer system. The control device 5 controls the projection device 3 and the imaging device 4 . The control device 5 has an arithmetic processing device including a processor such as a CPU (Central Processing Unit) and a storage device including memory and storage such as ROM (Read Only Memory) or RAM (Random Access Memory). The arithmetic processing unit performs arithmetic processing according to a computer program stored in a storage device.

[Control device]
FIG. 2 is a functional block diagram showing the control device 5 according to the embodiment. The control device 5 includes a projection control unit 51 , an imaging control unit 52 , a basic image acquisition unit 53 , a parameter calculation unit 54 , an imaging times determination unit 55 , a composite image generation unit 56 , and a three-dimensional shape calculation unit 57 . and

The projection controller 51 controls the projection device 3 . The projection control unit 51 controls the projection device 3 so that the sinusoidal lightness distribution fringe pattern is projected onto the object W while being phase-shifted. That is, the projection control unit 51 controls the projection device 3 so that a plurality of types of sinusoidal lightness distribution fringe patterns with mutually different phases are projected onto the object W. FIG.

The projection device 3 projects a fringe pattern with a sinusoidal lightness distribution onto the object W while shifting the phase. The projection device 3 projects onto the object W at least three types of fringe patterns with mutually different phases. In the embodiment, the projection device 3 projects onto the object W four types of fringe patterns with mutually different phases. The projection device 3 projects onto the object W four types of fringe patterns that differ in phase by π/2.

FIG. 3 is a diagram showing a stripe pattern according to the embodiment. As shown in FIG. 3, in the embodiment, the fringe pattern includes a first fringe pattern with a phase shift amount of 0° (initial phase=0) and a first fringe pattern with a phase shift amount of 90° (initial phase=π/2). a second fringe pattern, a third fringe pattern with a phase shift of 180° (initial phase=π), and a fourth fringe pattern with a phase shift of 270° (initial phase=3π/2) . The projection device 3 sequentially projects each of the first stripe pattern, the second stripe pattern, the third stripe pattern, and the fourth stripe pattern onto the object W. FIG.

The imaging control unit 52 controls the imaging device 4 . The imaging control unit 52 controls the imaging device 4 so that the object W on which the stripe pattern is projected is captured by the imaging device 4 .

The basic image acquiring unit 53 acquires from the imaging device 4 a basic image representing an image of the object W captured by the imaging device 4 . A basic image is an image of the object W acquired by the imaging device 4 in projection of one type of fringe pattern. When the imaging device 4 captures an image of the object W onto which the first stripe pattern is projected, the basic image acquisition unit 53 acquires a first basic image representing an image of the object W onto which the first stripe pattern is projected. When the imaging device 4 captures an image of the object W onto which the second stripe pattern is projected, the basic image acquisition unit 53 acquires a second basic image representing an image of the object W onto which the second stripe pattern is projected. When the imaging device 4 captures an image of the object W onto which the third stripe pattern is projected, the basic image acquisition unit 53 acquires a third basic image representing an image of the object W onto which the third stripe pattern is projected. When the imaging device 4 captures an image of the object W onto which the fourth stripe pattern is projected, the basic image acquisition unit 53 acquires a fourth basic image representing an image of the object W onto which the fourth stripe pattern is projected.

The parameter calculation unit 54 calculates a sine wave parameter for each pixel of the basic image based on the luminance value of the pixel of the basic image acquired in each projection of the plurality of types of fringe patterns.

A luminance value I _n (x, y) (n=0, 1, 2, 3) of a pixel at coordinates (x, y) in the basic image is represented by the following equation (1).

In equation (1), A(x, y) represents the contrast of the luminance value of the sine wave at the time of imaging. B(x, y) indicates the bias component of the luminance value. θ(x, y) indicates the phase value of the sine wave at the pixel.

The parameter calculator 54 calculates the phase value θ(x, y) of each pixel of the basic image based on the luminance value I _n (x, y) of each pixel of the basic image. The phase value θ(x, y) is obtained from the luminance value I _n (x, y) in the same pixel of the four basic images (first basic image, second basic image, third basic image, fourth basic image) as follows: It is calculated based on the following formula (2).

In equation (2), I ₀ (x, y), I ₁ (x, y), I ₂ (x, y), and I ₃ (x, y) are the first base image and the second base image, respectively. 4 shows the luminance values of the same pixels in each of the image, the third basic image, and the fourth basic image.

The parameter calculator 54 calculates parameters of the sine wave for each pixel of the basic image. The sinusoidal parameters include contrast A(x,y), bias component B(x,y), and phase value θ(x,y).

FIG. 4 is a diagram illustrating parameters of a sine wave according to an embodiment; _The parameter calculator _{54 calculates a sine} The wave parameters contrast A(x,y), bias component B(x,y), and phase value θ(x,y) can be calculated. Also, the parameter calculator 54 can estimate the sine wave at the pixel based on the parameters of the sine wave.

As shown in FIG. 4, the contrast A(x, y) means the amplitude value of the sine wave. The bias component B(x, y) means the central value of the amplitude of the sine wave.

The number-of-images determination unit 55 determines the number of acquisitions N of the basic image of the object W acquired by projecting one type of fringe pattern.

The imaging control unit 52 controls the imaging device 4 so that the basic image is acquired by the acquisition times N in each projection of the plurality of types of fringe patterns. The acquisition number N is plural. The acquisition count N is a natural number of 2 or more. For example, when the first stripe pattern is projected onto the object W, the imaging control unit 52 controls the imaging device 4 so that the object W onto which the first stripe pattern is projected is imaged N times. The imaging device 4 images the object W on which the first fringe pattern is projected N times. The basic image acquiring unit 53 acquires the same number of first basic images as the number of times N of acquisition. When the second stripe pattern is projected onto the object W, the imaging control unit 52 controls the imaging device 4 so that the object W onto which the second stripe pattern is projected is imaged N times. The imaging device 4 images the object W on which the second fringe pattern is projected N times. The basic image acquiring unit 53 acquires the same number of second basic images as the acquisition times N. The same applies when the third fringe pattern is projected onto the object W and when the fourth fringe pattern is projected onto the object.

The synthetic image generation unit 56 synthesizes the same number of basic images as the acquisition times N to generate a plurality of synthetic images corresponding to each of the plurality of types of striped patterns. For example, the synthetic image generation unit 56 synthesizes the same number of first basic images as the acquisition times N to generate the first synthetic image corresponding to the first striped pattern. The synthetic image generation unit 56 synthesizes the same number of second basic images as the acquisition times N to generate a second synthetic image corresponding to the second striped pattern. The synthetic image generation unit 56 synthesizes the same number of third basic images as the acquisition times N to generate a third synthetic image corresponding to the third striped pattern. The synthetic image generation unit 56 synthesizes the fourth basic images as many as the acquisition times N to generate a fourth synthetic image corresponding to the fourth striped pattern.

In an embodiment, synthesizing the plurality of base images includes adding the plurality of base images. The addition processing of multiple basic images includes adding the luminance values I _n (x, y) of the same pixel of multiple basic images. As a result, even if there is a pixel with a low luminance value I _n (x, y) in the basic image, by adding a plurality of luminance values I _n (x, y), the synthesized image has a high luminance value _Jn (x,y) pixels are obtained.

The number-of-images determination unit 55 determines the number of acquisitions N such that the luminance value J _n (x, y) of the pixels of the composite image is greater than or equal to a predetermined threshold value Th.

In the embodiment, the imaging times determination unit 55 determines the acquisition times N based on the parameters of the sine wave described above. In an embodiment, the threshold Th is set to be greater than twice the contrast A of the sine wave. That is, the threshold Th is set to a value larger than the difference between the maximum value and the minimum value of the luminance values I _n (x, y) of the pixels of the basic image. The number-of-images determination unit 55 determines the number of acquisitions N so that the number of acquisitions N is minimized under the condition that the luminance value J _n (x, y) of the pixel of the composite image is greater than twice the value of the contrast A. .

The three-dimensional shape calculator 57 calculates the three-dimensional shape of the object W based on a plurality of synthesized images (first synthesized image, second synthesized image, third synthesized image, fourth synthesized image). The three-dimensional shape calculation unit 57 calculates height data of each of a plurality of points of the object W corresponding to each pixel of the composite image based on the phase value θ of each pixel of the plurality of composite images. Calculate the three-dimensional shape of

[Three-dimensional measurement method]
Next, a three-dimensional measurement method according to the embodiment will be described. FIG. 5 is a flow chart showing a three-dimensional measurement method according to the embodiment. FIG. 6 is a schematic diagram showing a three-dimensional measurement method according to the embodiment.

The projection control unit 51 controls the projection device 3 so that a plurality of types of sinusoidal lightness distribution fringe patterns with mutually different phases are projected onto the object W. FIG. The projection device 3 sequentially projects each of the first stripe pattern, the second stripe pattern, the third stripe pattern, and the fourth stripe pattern onto the object W. FIG. The imaging control unit 52 controls the imaging device 4 so that the object W on which each of the first, second, third, and fourth stripe patterns is projected is sequentially captured by the imaging device 4. control (step S1).

The basic image acquisition unit 53 obtains a first basic image of the object W onto which the first stripe pattern is projected, a second basic image of the object W onto which the second stripe pattern is projected, and an image of the object W onto which the third stripe pattern is projected. A third basic image and a fourth basic image of the object W onto which the fourth fringe pattern is projected are obtained (step S2).

The parameter calculator 54 determines whether the luminance value I _n (x, y) of the pixel of the basic image (first basic image, second basic image, third basic image, fourth basic image) is equal to or greater than a predetermined threshold value Th. It is determined whether or not (step S3).

In step S3, when it is determined that the luminance value I _n (x, y) of the pixel of the basic image is equal to or greater than the threshold Th (step S3: Yes), the three-dimensional shape calculation unit 57 calculates the Then, the three-dimensional shape of the object W is calculated (step S4).

That is, the three-dimensional shape calculation unit 57 calculates the three-dimensional shape of the object W based on one first basic image, one second basic image, one third basic image, and one fourth basic image. Calculate the shape.

If it is determined in step S3 that the luminance value I _n (x, y) of the pixel of the basic image is not equal to or greater than the threshold value Th (step S3: No), the number-of-imaging determination unit 55 determines the luminance value J of the pixel of the composite image. The number of acquisition times N of the basic image of the object W acquired by projecting one type of fringe pattern is determined so that _n (x, y) is equal to or greater than a predetermined threshold value Th (step S5).

In the embodiment, the number-of-images determination unit 55 determines the number of times of acquisition so that the number of times of acquisition N is minimized under the condition that the brightness value J _n (x, y) of the pixel of the composite image is larger than twice the contrast A. Determine N.

The projection control unit 51 controls the projection device 3 so that a plurality of types of sinusoidal lightness distribution fringe patterns with mutually different phases are projected onto the object W. FIG. The projection device 3 sequentially projects each of the first stripe pattern, the second stripe pattern, the third stripe pattern, and the fourth stripe pattern onto the object W. FIG. The imaging control unit 52 controls the imaging device 4 so that the basic image is obtained by the number of acquisition times N in each projection of the plurality of types of fringe patterns (step S6).

In step S1, the basic image is acquired only once. If it is determined in step S3 that the acquisition count N is, for example, 4, the imaging control unit 52 controls the imaging device 4 so that the basic image is acquired only 3 times in step S6.

The basic image acquiring unit 53 acquires each of the three first basic images, the three second basic images, the three third basic images, and the three fourth basic images (step S7).

Combining the processing in step S2 and the processing in step S7, the basic image acquisition unit 53 obtains four first basic images, four second basic images, four third basic images, and four fourth basic images. I have obtained each.

In the embodiment, the amount of light emitted from the light source 31 when acquiring each of the four first basic images is the same. Similarly, the amount of light emitted from the light source 31 when acquiring each of the four second basic images and the amount of light emitted from the light source 31 when acquiring each of the four third basic images , and the amount of light emitted from the light source 31 when acquiring each of the four fourth basic images.

The synthetic image generation unit 56 synthesizes the same number of basic images as the number of acquisition times N to generate a plurality of synthetic images corresponding to each of the plurality of types of striped patterns (step S8).

In the embodiment, the synthetic image generator 56 synthesizes the four first basic images to generate the first synthetic image. The composite image generator 56 composites the four second basic images to generate a second composite image. The composite image generator 56 composites the four third basic images to generate a third composite image. The composite image generator 56 composites the four fourth basic images to generate a fourth composite image.

The three-dimensional shape calculator 57 calculates the three-dimensional shape of the object W based on the multiple composite images (step S9).

That is, the three-dimensional shape calculation unit 57 calculates the three-dimensional shape of the object W based on one first synthesized image, one second synthesized image, one third synthesized image, and one fourth synthesized image. Calculate the shape.

[effect]
As described above, according to the embodiment, for example, when the light reflectance of the surface of the object W is low and the luminance value I _n (x, y) of the pixel of the basic image acquired by one imaging is low, Furthermore, imaging is performed a plurality of times, and the basic image is acquired for the number of times N of acquisitions. After the basic images are acquired the number of acquisition times N, the same number of basic images as the number of acquisition times N are combined to generate a composite image. The brightness value J _n (x, y) of the pixels of the composite image is high. The three-dimensional shape calculator 57 can calculate the three-dimensional shape of the object W based on a composite image composed of pixels with high luminance values J _n (x, y). This suppresses deterioration in the reliability of three-dimensional measurement.

By increasing the amount of light emitted from the light source 31, the luminance value I _n (x, y) of the pixel of the basic image can be increased. However, increasing the amount of light emitted from the light source 31 increases the amount of heat generated by the light source 31 . Further, by slowing down the shutter speed of the imaging device 4, the luminance value I _n (x, y) of the pixel of the basic image can be increased. However, if the shutter speed of the imaging device 4 is slowed down, the possibility of blurring of the basic image increases. In the embodiment, when the luminance value I _n (x, y) of a pixel of one base image is low, a plurality of base images are combined to generate a composite image. As a result, the three-dimensional shape of the object W can be accurately calculated while suppressing an increase in the amount of heat generated by the light source 31 and the blurring of the basic image.

The number-of-images determination unit 55 determines the number of acquisitions N such that the luminance value J _n (x, y) of the pixels of the composite image is greater than or equal to a predetermined threshold value Th. As a result, a composite image composed of pixels with high luminance values J _n (x, y) is generated.

The parameter calculator 54 calculates the contrast A, the bias component B, and the phase value θ of the sine wave as parameters of the sine wave in each pixel of the basic image. The number-of-images determining unit 55 determines the number of acquisitions N based on the contrast A of the sine wave. As a result, a composite image composed of pixels with high luminance values J _n (x, y) is generated.

The threshold Th is greater than twice the contrast A value. That is, the threshold Th is larger than the difference between the maximum and minimum values of the luminance values I _n (x, y) of the pixels of the basic image. As a result, a composite image composed of pixels with high luminance values J _n (x, y) is generated.

The number-of-images determination unit 55 determines the number of acquisitions N so that the number of acquisitions N is minimized under the condition that the luminance value J _n (x, y) of the pixel of the composite image is greater than twice the value of the contrast A. . For example, when the number of acquisitions N is 3, [J _n (x, y)≧2×A], and when the number of acquisitions N is 4 or more, [J _n (x, y)<2×A], The acquisition number N is not determined to be 5 times, but is determined to be 4 times. As a result, the number of acquisitions N of basic images by the imaging device 4 is prevented from becoming unnecessarily large. Therefore, lengthening of the measurement time of the three-dimensional measurement apparatus 1 is suppressed.

[Other embodiments]
In the above-described embodiment, when the basic image is acquired N times, the amount of light of the light source 31 may be changed. For example, when the light source 31 emits light with the first light amount in step S1 described above and it is determined in step S3 that the luminance value I _n (x, y) of the pixel of the basic image is not equal to or greater than the threshold value Th, in step S7 In step S6, the light source 31 may emit light with a second light amount that is higher than the first light amount so that the luminance value I _n (x, y) of the pixel of the acquired basic image increases.

Note that when both a region with a low light reflectance and a region with a high light reflectance exist on the surface of the object S, the synthesized image corresponding to the region with the high light reflectance is obtained by adding the basic images instead of adding the basic images. , the basic image may be averaged. Averaging processing of a plurality of basic images includes calculating an average value of luminance values I _n (x, y) of the same pixel of a plurality of basic images. As a result, even if there is a pixel with a luminance value I _n (x, y) that is too high in the basic image, a plurality of luminance values I _n (x, y) are averaged, so that the synthesized image has an appropriate A pixel with a brightness value J _n (x, y) is obtained. In addition, random noise is reduced in the composite image by generating the composite image through the averaging process.

DESCRIPTION OF SYMBOLS 1... Three-dimensional measuring apparatus, 2... Stage, 3... Projection apparatus, 4... Imaging apparatus, 5... Control apparatus, 31... Light source, 32... Light modulation element, 33... Projection optical system, 41... Imaging optical system, 42 Imaging device 51 Projection control unit 52 Imaging control unit 53 Basic image acquisition unit 54 Parameter calculation unit 55 Imaging number determination unit 56 Composite image generation unit 57 Three-dimensional shape calculation unit , W … object.

Claims (6)

a projection control unit that controls a projection device so that a plurality of types of sinusoidal lightness distribution fringe patterns with mutually different phases are projected onto an object;
an imaging number determination unit that determines the number of acquisition times of the basic image of the object acquired in projecting one type of the fringe pattern;
an imaging control unit that controls an imaging device so that the basic image is acquired the number of acquisition times in each projection of the plurality of types of the stripe patterns;
a synthetic image generation unit that synthesizes the basic images obtained the number of times to generate a plurality of synthetic images corresponding to each of the plurality of types of stripe patterns;
a three-dimensional shape calculation unit that calculates the three-dimensional shape of the object based on the plurality of synthesized images;
3D measuring device. The number-of-images determining unit determines the number of acquisitions so that the luminance value of a pixel of the synthesized image is equal to or greater than a predetermined threshold.
The three-dimensional measuring device according to claim 1. A parameter calculation unit that calculates a sine wave parameter for each pixel of the basic image based on the luminance value of the pixel of the basic image obtained in each projection of the plurality of types of the fringe patterns,
The imaging count determination unit determines the acquisition count based on the parameter.
The three-dimensional measuring device according to claim 2. the parameter includes a contrast of the sine wave;
the threshold is greater than twice the contrast;
The three-dimensional measuring device according to claim 3. The number-of-images determining unit determines the number of times of acquisition so that the number of times of acquisition is the minimum under a condition where the brightness value of the pixel of the synthesized image is greater than twice the contrast.
The three-dimensional measuring device according to claim 4. Projecting a plurality of types of sinusoidal lightness distribution fringe patterns with mutually different phases onto an object;
Determining the number of acquisitions of the basic image of the object acquired in projecting one type of the fringe pattern;
Acquiring the basic image for the number of acquisition times in projection of each of the plurality of types of the fringe patterns;
synthesizing the basic images obtained the number of times to generate a plurality of synthesized images respectively corresponding to the plurality of types of the stripe patterns;
calculating a three-dimensional shape of the object based on a plurality of the composite images;
Three-dimensional measurement method.

Images