JP2002162215A - Three-dimensional shape measuring method and its system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable three-dimensional shape measurement, simple phase connection processing and high speed measurement without excessively taking a cost and an imaging time and without damaging the resolution of an imaging device. SOLUTION: The image of a pattern H1 is projected on an object, photographing and the shift of H1 are repeated to take in a plurality of stripe images H2 shifting a bright phase every constant amount, and the image for area classification is created. In the image for the area classification, a first value is assigned to a pixel corresponding to a pixel belonging to one half cycle in one bright cycle, a second value is assigned to the other, and a third value is assigned to the pixel corresponding to the pixel in which brightness does not change in the shift. The classification of the area in which each pixel of the first to third values continues is performed by using the image for the area classification, on the basis of the classified result the connection area of the first value and the second value mutually adjacent along the parallel direction of the stripe are classified, phase connecting processing is performed by utilizing the classified result, and the three-dimensional shaped height information of the object is extracted in accordance with the processed result.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for measuring a three-dimensional shape of an object by a phase shift method (fringe scanning method).
The present invention relates to a dimensional shape measurement method and a system thereof.

[0002]

2. Description of the Related Art Conventionally, a phase shift method (fringe scanning method) that enables high-speed and high-accuracy measurement has been used for measuring a three-dimensional shape. Here, in the phase shift method, it is necessary to deal with the phase connection problem that the height information cannot be uniquely determined unless the phase connection processing is performed correctly. −3852
In JP-A-4, a wide reference plane is arranged near the measurement target so that the entire projection pattern can be captured, and a projection pattern without distortion projected thereon is measured as a reference phase. A method is described in which the distortion of a pattern projected on a measurement target is obtained by using the method.

Japanese Patent Application Laid-Open No. H11-14327 proposes a method of applying a filter for extracting a feature of a phase discontinuity over the entire image as a method of performing a phase connection process only by image processing. I have.

Japanese Patent Laid-Open Publication No. Hei 6-66527 discloses a method of solving the phase connection problem by improving the projection method, performing measurement twice by changing the period of the pattern to be projected, and determining the maximum value determined from the period. A method is described in which a cycle deviation generated at the time of measurement in a small cycle based on the measurement height is corrected based on a measurement result in a large cycle.

[0005]

However, the above 3)
Although the methods described in the two publications can solve the phase connection problem, the following problems remain.

In the method described in Japanese Patent Publication No. 3-38524, it is necessary to capture the entire reference plane in the camera's field of view, and the measurement target can be captured only on a part of the screen.
There is a problem that the resolution of the imaging device is not fully utilized. In addition, it is necessary to set a reference plane also in terms of the device configuration, and there is a problem that application is difficult when the measurement target cannot be processed like a human body.

In the method described in Japanese Patent Application Laid-Open No. 11-14327, it is necessary to apply a large convolution filter with a large amount of calculation to the entire surface of the obtained phase estimation result image, and the process of deriving the phase takes time. There was a problem.

In the method described in Japanese Patent Application Laid-Open No. 6-66527, it is necessary to perform measurement twice when performing the measurement, and there is a problem that the measurement time is twice as long.

[0009] Also, Japanese Patent Application No. 11-304 filed by the present applicant.
Also in the application of No. 086, as an invention which can solve the phase connection problem, in order to make it easy to determine the fringe order even when the fringe is discontinuous due to a step or the like, the light intensity must be at least one cycle or more. A method is described in which a projected image of a first pattern that changes in a sine wave shape and forms a stripe pattern and a projected image of a second pattern for determining the position of the phase boundary of the first pattern are used in combination. However, some methods in this application
Since an auxiliary projection device is used to project the image of the second pattern or the number of times of projection is increased, there is a disadvantage that the cost and the imaging time are extra.

The present invention has been made in view of the above circumstances, and measures a three-dimensional shape without impairing the resolution of an imaging apparatus without adding cost and imaging time.
Enables simple phase connection processing and high-speed measurement 3
It is an object of the present invention to provide a dimensional shape measuring method and a system therefor.

[0011]

According to a first aspect of the present invention, there is provided a method for measuring a three-dimensional shape based on a phase shift method. Projects an image of a pattern that changes in a sinusoidal shape on the object, repeats photographing of the image of the pattern projected on the object and a certain amount of shift of the pattern, and the lightness phase is shifted by a fixed amount. At least three types of images are captured, and from these images, a plurality of pixels respectively corresponding to a plurality of pixels of the image are provided, and among the plurality of pixels, the plurality of pixels correspond to pixels belonging to a part of one cycle of the brightness. A pixel is assigned a first value, a pixel corresponding to a pixel belonging to the remainder of one cycle of the brightness is assigned a second value, and a pixel corresponding to a pixel whose brightness does not change due to the shift of the brightness phase. Is Assigned three values to create a region classification image comprising performs classification of each successive region of the pixels of the first to third values with the region classification image,
Based on the classification result, of the first area in which the pixels of the first value are continuous and the second area in which the pixels of the second value are continuous, first areas adjacent to each other along the direction in which the stripes are arranged And the second area are grouped, and a phase connection process is performed using the grouping result, and height information of the three-dimensional shape of the object is extracted according to the processing result. .

[0012] The invention described in claim 2 is the third invention described in claim 1.
In the dimensional shape measurement method, when creating the area classification image, among the pixels of the plurality of captured images,
Create a phase restored image in which the phase value calculated by substituting the brightness of each pixel at the same position into a predetermined arithmetic expression is assigned to each pixel as the value of the pixel corresponding to that position,
Thereafter, the area classification image corresponds to a position of a pixel whose lightness changes due to the shift of the lightness phase and a pixel whose value of a corresponding pixel in the phase-recovered image is smaller than a predetermined threshold value. A second value is assigned to a pixel corresponding to a position of a pixel whose brightness changes due to the shift of the brightness phase and whose value of the corresponding pixel in the phase-recovered image is larger than a predetermined threshold value A third value is created by assigning a third value to a pixel corresponding to a position of a pixel whose brightness does not change due to the shift of the brightness phase, and the phase connection process uses the grouping result for the phase restoration image. It is characterized by being performed.

[0013] The third aspect of the present invention provides the third aspect of the present invention.
In the dimensional shape measurement method, the area classification image includes a process of calculating a phase value by substituting the brightness of each pixel at the same position among the pixels of the plurality of captured images into a predetermined arithmetic expression. Of the captured images, the phase when the one image is used for the initial phase while the one image is used for the initial phase and the other image is used for the initial phase are executed in parallel. A difference value between the value and the phase value when the other image is used for the initial phase is determined. If the difference value is positive, the first value is changed. If the difference value is negative, the second value is changed. If not, a third value is assigned and created, and the phase connection process is performed using the grouping result for each of the phase values.

According to a fourth aspect of the present invention, there is provided the third aspect of the present invention.
In the three-dimensional shape measuring method, the certain amount is one of the lightness.
This is the amount obtained by dividing the period by a predetermined number that is an integer greater than 3, and when creating the area classification image, among the pixels of the plurality of captured images, A phase restoration image in which the phase value calculated by substituting the brightness into a predetermined arithmetic expression is assigned to each pixel as a value of a pixel corresponding to the position, among the captured images,
One image is used for the initial phase and the captured image is cyclically shifted an integer number of times smaller than the predetermined number and another image is used for the initial phase. The first image is the first value if the difference between the values of the two corresponding pixels in the two-phase restored image is positive, the second value if the difference is negative, and the third value if the brightness does not change. The phase concatenation process is created by being assigned to each pixel, and is performed using the grouping result for one of the two phase restored images.

According to a fifth aspect of the present invention, there is provided the third aspect of the present invention.
In the two-dimensional shape measurement method, among the pixels of the area classification image, the position of the pixel at which the pixel value changes from the second value to the first value along the direction in which the stripes are arranged is determined by the phase connection position. And performing the phase connection processing.

The invention according to claim 6 is the third invention according to claim 1.
In the dimensional shape measurement method, the phase connection processing includes, using the area classification image, classifying a first area in which the first value pixels are continuous and a second area in which the second value pixels are continuous. Then, a starting point and an ending point along the direction in which the stripes are arranged in the first area and the second area are obtained, respectively, and the positional relationship between the first area and the second area is evaluated. This is performed by grouping the first region and the second region adjacent to each other along the direction in which the stripes are arranged.

The invention according to claim 7 is the third invention according to claim 6.
In the dimensional shape measurement method, when each of the first area and the second area includes another small area equal to or less than a predetermined number of pixels before the grouping, a process of merging the small area with an area including the small area is performed. It is characterized by the following.

[0018] The invention described in claim 8 is the third invention described in claim 6.
In the two-dimensional shape measurement method, a third region in which the pixels of the third value are continuous is included in the first region to be an extended first region, and the third region is included in the second region to be an extended second region. The first region and the second region adjacent to each other are grouped based on the overlapping state of the first and second regions.

The ninth aspect of the present invention is the third aspect of the present invention.
In the three-dimensional shape measuring method, when the first region and the second region for a plurality of phases appear to be connected by the third region, the first region and the second region are obtained again except for the third region, and the third region is obtained. A region not divided in the direction along the stripe is extracted from the first region and the second region, and the regions divided in the direction along the stripe are combined based on this region.

According to a tenth aspect of the present invention, in the three-dimensional shape measuring method according to the second aspect, an image captured by repeating the photographing and shifting is obtained by projecting the image of the pattern onto the object from one direction. Two sets of a case and a case where the image of the pattern is projected onto the object from another direction are prepared, and two phase restored images are created from these two sets of images. When there is a region where the third value pixel is continuous on only one of the regions, the region is defined as a shadow region of a projection that forms a step in the object, and a region sandwiched between these shadow regions is defined as the projection region. It is characterized in that it is a region where a part exists.

According to an eleventh aspect of the present invention, in the three-dimensional shape measuring method according to the second aspect, among the pixels of the area classification image, the stripes in the third area in which the pixels of the third value are continuous. When the values of the pixels of the phase-recovered images at the start point and the end point along the juxtaposition direction are substantially the same, the third area is defined as a shadow area of a protrusion that forms a step in the object.

According to a twelfth aspect of the present invention, in the three-dimensional shape measuring method according to the tenth or eleventh aspect, of the pixels of the area classification image, a direction opposite to a direction in which the shadow extends from the range of the shadow. The values of the pixels of the phase-recovered image corresponding to each pixel in the direction along the stripe in the direction are examined, and when the values of these pixels are discontinuous, the projection is performed in a direction opposite to the direction in which the shadow extends. It is characterized in that the portion is continued, and when the portion is discontinuous and disappears, it is a boundary where the protrusion is interrupted.

The invention according to claim 13 is the invention according to claims 10 to 1
3. In the three-dimensional shape measuring method according to any one of 2., the length of the range of the shadow along the direction in which the stripes are arranged is obtained as a pattern shift amount, and the shadow exists in a direction opposite to the direction in which the shadow extends. A region where the pixels of the first value or the second value in the protrusion are continuous is associated with a region outside the protrusion which is shifted in the direction in which the stripes are arranged by the pattern shift amount.

According to a fourteenth aspect of the present invention, in the three-dimensional shape measuring method according to the first aspect, the phase value or the height information in the third area where the pixels of the third value are continuous is stored in the third area. It is characterized by complementing based on outer phase value or height information.

According to a fifteenth aspect of the present invention, in the three-dimensional shape measuring method according to the fourteenth aspect, the third region in which the pixels of the third value are continuous is formed by a shadow of a projection forming a step in the object. In this case, complementing the phase value or the height information in the third area based on the phase value or the height information of the area adjacent to the third area and in the direction in which the shadow of the projection extends. Features.

According to a sixteenth aspect of the present invention, there is provided a three-dimensional shape measuring system based on a phase shift method. A projection device for projecting, an imaging device for photographing an image of the pattern projected on the object, and a certain amount of shift of the pattern projected on the object is obtained repeatedly, and the phase of the lightness is obtained by a constant amount. A storage device for capturing and storing at least three types of shifted images, and a plurality of pixels respectively corresponding to a plurality of pixels of the image, from the images stored in the storage device, and among the plurality of pixels, A pixel corresponding to a pixel belonging to a part of one cycle of brightness is assigned a first value, a pixel corresponding to a pixel belonging to the remainder of one cycle of brightness is assigned a second value, and the phase of the brightness is assigned. The pixel corresponding to the pixel brightness does not change the shift creates a region classification image composed assigned third value, the using the area classification image first
To a third area in which each of the pixels of the third value is continuous, and based on the classification result, a first area in which the pixels of the first value are continuous and a second area in which the pixels of the second value are continuous The first region and the second region adjacent to each other along the direction in which the stripes are arranged are grouped, and a phase connection process is performed using the grouping result. An image processing device for extracting height information of the three-dimensional shape of the object.

According to the present invention, a region in which each of the first to third value pixels is continuous is classified using the region classification image, and based on the classification result, the regions are arranged along the direction in which the stripes are arranged. Since the first region and the second region adjacent to each other are grouped, the grouped first region and second region correspond to a phase for one cycle (hereinafter, referred to as one phase). Even when there are holes or holes, correct phase connection processing can be performed by using the grouping result. Also,
Since there is no need to use an auxiliary projection device or increase the number of projections for projecting an auxiliary pattern, three-dimensional shape measurement can be realized at low cost and at high speed.

[0028]

FIG. 1 is an explanatory diagram of a three-dimensional shape measuring system, and FIG. 2 is a flowchart showing a procedure of a three-dimensional shape measuring method. The first embodiment of the present invention will be described with reference to these drawings. I do.

The three-dimensional shape measuring method shown in FIG. 2 is a method for measuring a three-dimensional shape based on a phase shift method.
"Image number ← 1" in step S1, "Pattern switching" in step S2, "Image capture" in step S3, "Capture completed?" In step S4, and "Image number ← image number +1" in step S5. And step S6
"Image number: N", "transfer image to storage device" in step S7, "phase derivation process" in step S8, "phase boundary detection" in step S9, and "phase concatenation process" in step S10. And a procedure of “distance calculation” in step S11.

The above three-dimensional shape measuring method uses the method shown in FIG.
As shown in the example of (a), the projection device 1, the camera 2,
This is executed by a three-dimensional shape measurement system including the quantization circuit 3, the storage device 4, and the image processing device 5.

The projection apparatus 1 includes a lamp 11, a concave mirror 12 for reflecting light from the lamp 11 in one direction, a projection lens 13 disposed in front of the lamp 11, a lamp 11 and a projection lens. 13, a pattern plate 14 on which a stripe-shaped light and shade pattern H1 which changes in a sinusoidal manner in one direction with a light transmittance is formed.
4 and a pattern shift unit 15 for accurately moving the sine wave at an interval shorter than one cycle of the sine wave. In the case of this configuration, the light of the lamp 11 becomes planar light by the concave mirror 12, passes through the pattern plate 14, and projects an image of a pattern in which the brightness changes sinusoidally along the direction in which the stripes are arranged. 2 is projected onto the surface of the object 10 through the lens 13, and as a process of step S <b> 2 in FIG. 2, the phase of the brightness of the image projected on the object 10 is shifted by the pattern shift unit 15. Will be. here,
The image projected on the object 10 is 3
The image changes to an image H2 according to the dimensional shape.

It should be noted that a transmissive liquid crystal panel may be used for the pattern plate 14 and the pattern shift section 15 may be omitted. In the case of this configuration, an apparatus configuration having no mechanical movable part can be realized, and the phase of the brightness of the image projected on the object 10 can be simply and accurately shifted. Also,
The above-described projection method is an example, and may be replaced by a method of projecting a black and white grid pattern by blurring the lens with a lens, a method of using laser interference, or a method of generating fringes by moire topography.

The camera 2 is constituted by a light receiving lens, a CCD, and the like, and captures an image H2 on the surface of the object 10 from a direction different from the optical axis of the projection device 1 every time the grayscale pattern H1 shifts. In the example of FIG. 1A, the camera is installed so as to photograph the upper surface of the object 10. The quantization circuit 3 quantizes a signal obtained by photographing with the camera 2 to obtain a fringe image H3 corresponding to the image H2. The storage device 4 includes, for example, a semiconductor storage element such as a RAM or a disk device such as a hard disk, and stores a plurality of frames of the stripe image H3 obtained by the quantization of the quantization circuit 3. In this embodiment, the camera 2 and the quantization circuit 3 constitute an imaging device.

Here, the procedure of photographing by the camera 2 will be described. First, an image number for a striped image obtained by photographing is initialized to 1 (S1 in FIG. 2). Thereafter, the light and shade pattern H1 is shifted (S2), and subsequently, the stripe image H3 is captured by photographing of the camera 2 and quantization of the quantization circuit 3 (S3). Thereafter, when the capture of the stripe image H3 is completed (Yes in S4), the image number is incremented by 1 (S5), and the image number is compared with a predetermined number N (3 or more) (S6). At this time, if the image number is larger than N (Yes in S6), the captured stripe image H3 is transferred to the storage device 4 and stored (S7), otherwise (S7).
No at 6), and the process returns to step S2. Hereinafter, the light amount of the projected pattern itself is referred to as “intensity”, and the pixel value of an arbitrary coordinate point in the captured stripe image H3 is referred to as “brightness”.

Returning to FIG. 1, the image processing device 5 includes a CPU
And a plurality of pixels corresponding to a plurality of pixels of the image from the image stored in the storage device 4. Of the plurality of pixels, a part of one cycle of the lightness is included. The pixel corresponding to the pixel to which the
A second value is assigned to a pixel corresponding to a pixel belonging to the remainder of one cycle of brightness, and a third value is assigned to a pixel corresponding to a pixel whose brightness does not change due to a shift in brightness phase.
A region classification image generation function 51 for generating a region classification image to which values are assigned, and classifying a region where each of the first to third value pixels is continuous using the region classification image,
Based on this classification result, the first pixel in which
The first region and the second region, which are adjacent to each other along the direction in which the stripes are arranged, of the region and the second region in which the pixels having the second value
A height information extraction function 52 is provided for classifying the regions, performing phase connection processing using the grouping results, and extracting height information of the three-dimensional shape of the object according to the processing results. More specifically, various processes including the image processing of steps S8 to S11 in FIG. 2 are executed. For example,
A process of creating a phase restored image from the plurality of stripe images H3 stored in the storage device 4 is executed (S8). This phase restoration image is obtained by substituting the brightness of each pixel at the same position among the pixels of the plurality of stripe images H3 into a predetermined arithmetic expression as a value of a pixel corresponding to the position. It is assigned for each pixel. Further, a process of generating an area classification image having a plurality of pixels respectively corresponding to the plurality of pixels of the phase restoration image is executed (S9). In the area classification image, a first value is assigned to a pixel corresponding to a position of a pixel whose brightness changes due to the shift of the brightness phase and a value of a corresponding pixel in the phase restoration image being smaller than a predetermined threshold value. The second value is assigned to the pixel corresponding to the position of the pixel where the brightness changes due to the shift of the brightness phase and the value of the corresponding pixel in the phase restored image is larger than a predetermined threshold value, and the brightness phase is shifted. A third value is assigned to a pixel corresponding to a position of a pixel whose brightness does not change due to the shift. Further, a region (connected region) in which each of the first to third value pixels is continuous is classified using the region classification image, and a first value in which the first value pixel is continuous is determined based on the classification result. Of the connected region (also referred to as the first region) and the connected region of the second value (also referred to as the second region) in which the pixels of the second value are continuous, The second area is grouped, a phase connection process is performed using the grouping result, and the object 10 is set in accordance with the processing result.
The process of extracting the height information of the three-dimensional shape is performed (S10, S11).

Next, various processes executed by the image processing apparatus 5 will be described in detail with reference to the drawings.

FIG. 3 is a diagram for explaining the principle of the phase restoration. FIG.
In step S8, a process of estimating the phase of the projected pattern is performed by calculating the brightness change at the same coordinate point for the plurality of stripe images H3 stored in the storage device 4. For example, steps S1 to S1 in FIG.
In FIG. 7, assuming that the image was captured four times while shifting the phase by 1/4 (π / 2) of the cycle, as shown in FIG. 3, the brightness at the same coordinate point (x, y) is I0 (x, y). y) → I1 (x,
When y) → I2 (x, y) → I3 (x, y), the phase value α (x, y) at that position is derived by the following equation.

[0038]

(Equation 1)

When the phase value α is obtained for all points (x, y) and the points where the phase values are equal are connected, this corresponds to a so-called light-section line in the light-section method and reflects the cross-sectional shape of the object 10. Is formed. Then, since the positional relationship between the projection device 1 and the camera 2 and various optical parameters are known, the height information of the three-dimensional shape of the object 10 is obtained from these parameters and the shape of the equiphase lines in the same manner as in the light section method. Is obtained. As described above, when the phases are obtained for all the pixels of the captured fringe image, when the light-section method is used to scan the light-section line at a very close interval in a direction substantially orthogonal to the light-section line to obtain area-based distance information. The same measurement can be performed easily and at high speed.

FIG. 4 is a perspective view of the object, FIG. 5 is a top view and a sectional view of the object, and FIG. 6 is a stripe image obtained by photographing the object and a change in brightness along a straight line of the stripe image. FIG. 7 is a diagram showing an example, FIG. 7 is a diagram showing an example of each stripe image obtained by phase shift and an example of a brightness change along a certain straight line of each stripe image,
FIG. 8 is a diagram showing an example of a phase restored image and a phase value change along a straight line of the phase restored image, and FIG. 9 is an explanatory diagram of a phase connection process.

As shown in FIGS. 4 and 5, when the object 10 has a projection 10A having a steep step and a hole (through hole in the figure) 10B, the above equation 1 is used. The three-dimensional shape measurement method cannot accurately measure the three-dimensional shape of the target object 10. In the example of FIG. 5, since the projection optical axis of the projection device 1 has a certain inclination angle, a shadow of the projection 10A is formed, and the projection pattern is not projected on the range A of the shadow. Also, the projection pattern is not projected on the range B corresponding to the hole 10B. For this reason, as shown in FIG. 6, the brightness I _AH3 and I _BH3 of the areas A _H3 and B _H3 corresponding to the ranges A and B are affected by the projection pattern. The brightness is just illuminated by ambient light. Therefore, even if the phase shift is performed in step S2 in FIG. 2, there is a portion where the brightness of the regions A _H3 and B _H3 does not change as shown in FIG. 7, so that only the above arithmetic expression is used as shown in FIG. The phase values α _AH4 , α of the areas A _H4 , B _H4 corresponding to the ranges A, B in the phase restored image H4
_BH4 is missing, making it impossible to measure an accurate phase value.

On the other hand, when the height information of the three-dimensional shape is obtained from the above-mentioned isophase line, the possible phase value is in the range from -π to π since Equation 1 is an arctangent function. As shown in FIG. 9, in order to obtain the phase value (absolute phase value) α ′ based on the edge of the image, the fringe order n (n =
0, 1, 2,...), It is necessary to add a phase value by n × 2π. However, as shown in FIG. 8, if the phase value information is missing and an accurate phase value cannot be measured, the phase concatenation process of step S10 in FIG. 2 cannot be performed, and The absolute phase value α ′ cannot be obtained. That is, the simplest phase boundary detection process is a process of detecting a portion where the phase value changes from π to −π. In this process, as shown in FIG. 8, the phase A of the areas A _H4 and B _H4 is detected. Value α _AH4 , α
_{Since BH4} cannot be obtained, if there is a phase boundary in that area, it cannot be detected. For example, FIG.
When the phase boundary comes within the range A, when a fringe image is obtained by photographing from directly above, the portion that changes from π to −π is hidden, and the phase boundary cannot be detected. As a result, the three-dimensional shape of the object 10 cannot be measured accurately.

Therefore, in the first embodiment, as shown in FIGS. 4 and 5, as a means for solving the phase connection problem when the phase boundary is locally hidden by the protrusion 10A and the hole 10B, In order to utilize a wide range of information, each phase value is binarized into a first value and a second value, and a method is used in which a combination of regions (ranges) in which the respective values are continuous is regarded as one cycle. The boundary of the period of the restored phase value divided for each period is detected, and a correct absolute phase value is obtained.

FIG. 10 is a diagram showing an example of an area classification image and a change in brightness along a straight line of the area classification image. In FIG. 10, the threshold value is set to a phase value of 0, and is binarized into a first value (black) having a phase value smaller than 0 and a second value (white) having a phase value larger than 0, An area classification image H5 has been created. Thus, the phase restoration image H4
An image having a plurality of pixels respectively corresponding to the plurality of pixels is prepared, and among the plurality of pixels, the image corresponding to the position of the pixel whose brightness changes due to the phase shift and the phase of the corresponding pixel in the phase restored image H4 A first value is assigned to a pixel whose value is smaller than the threshold value 0, and the phase value of the corresponding pixel in the phase restored image H4 is larger than the threshold value 0 while corresponding to the position of the pixel whose brightness changes due to the phase shift. If the first value is assigned to the pixel and the area classification image H5 is created, when the first area and the second area are adjacent to each other,
A set of the first area and the second area can be handled as one phase of the projection pattern. Therefore, since the first region and the second region adjacent to each other may be set to one phase, the pixel value is examined in the phase change direction (the horizontal direction in FIG. 10), and the pixel value changes from the second value to the first value. The process of setting the position as the phase boundary is the simplest phase boundary detection process.

Then, in order to solve the above phase connection problem,
In the region classification image H5, a third value is assigned to a pixel corresponding to the position of a pixel whose brightness does not change due to the phase shift (the pixels of the regions A _H4 and B _{H4 in} FIG. 8). In the example of FIG. 10, corresponding to the region A _H4, B _H4, phase values alpha _AH5 of the region 3 value is assigned A _H5, B _H5, α _BH5 has become zero. In the first embodiment, the phase boundary detection processing is executed using the region classification image H5 created in this manner.

FIG. 11 is a flowchart showing a specific processing procedure of the phase boundary detection. In FIG. 11, first, the “ternary image generation” process of step S901 is performed by the above-described ternarization.

In the first embodiment, the area classification image H5 as a ternary image is generated using the above threshold value. However, the present invention is not limited to this, and is obtained in steps S1 to S7 in FIG. A phase value calculated by substituting the brightness of each pixel at the same position among the pixels of the plurality of fringe images into the arithmetic expression of Equation 1 above is assigned to each pixel as a value of a pixel corresponding to that position. Out of the phase-recovered image H4 obtained in step S7, one image is used for the initial phase, and the captured image is cyclically shifted by an integer smaller than N to obtain another image for the initial phase. And the first value if the difference between the values of the corresponding two pixels in the two phase restored images is positive, the second value if the difference is negative, and the brightness must change. For example, the area where the third value is assigned to each pixel It may be generated a class image for H5. That is, a similar pattern can be obtained by subtracting the two phase values restored by shifting the initial phase.

FIG. 12 is an explanatory diagram of a method for generating the area classification image. First, the projected pattern is changed from 1 → 2 → 3 → 4, and four stripe images H3 as shown in FIGS. 7A to 7D are taken in. Each pixel (x, y) of these stripe images H3 )
The lightness at is calculated by the following equation: I0 (x, y) → I1 (x, y) →
Phase restoration is performed by substituting as I2 (x, y) → I3 (x, y) to generate a phase restored image as shown in FIG. Subsequently, the four stripe images H3 are cyclically shifted to obtain a stripe image sequence in another arrangement order. Each stripe image H3 in the example of FIG.
Since the images were shot while shifting the phase of the projection pattern by π / 2 at a time, (c), (d), (a), and (c) of FIG.
If the four striped images H3 are cyclically shifted so as to have the arrangement order of (b), it is the same as the case where the image is captured twice with further phase shift from (c) and (d) of FIG. Can be obtained, and in this case, the initial phase is shifted by π. Thus, as shown in FIG. 12B, another phase restored image whose initial phase is shifted by 周期 cycle can be generated without increasing the number of times of projection and imaging.

Then, if the difference between the values of the two corresponding pixels in the two-phase restored image is positive, the first value is negative, the second value is 0, and the third value is assigned. FIG. 12 (c)
As shown in (5), the area classification image H5 can be generated. At this time, since the phase cannot be restored by the protrusion 10A and the hole 10B, an arbitrary constant (0 in FIG. 12) is substituted as the third value. Here, the deformation of the stripe shape due to the unevenness of the surface of the object 10 is the same even when the initial phase value is changed as shown in FIGS.
The difference image obtained by subtracting the pattern of FIG. 12A from the pattern of FIG. 12A has a range of -π and 0 to π when the phase value in FIG. Then π
Becomes Further, since a constant is substituted for the third value part, the difference value is 0. Therefore, in this difference image, the pixel value π is set to the first value, the pixel value −π is set to the second value, and the pixel value 0 is set to the third value, thereby easily generating a ternary image (image H5 for area classification). be able to.

When it is desired to use a memory as little as possible, for example, by performing processing for generating an area classification image by hardware, among the pixels of the plurality of captured stripe images,
The process of calculating the phase value by substituting the brightness of each pixel at the same position into a predetermined arithmetic expression is performed when one of the captured images is used for the initial phase and another image is used for the initial phase. While executing in parallel, the difference between the phase value when one image is used for the initial phase and the phase value when the other image is used for the initial phase is determined. In this case, the first value may be assigned a negative value if the value is negative, and the third value may be assigned if the brightness does not change, to create an area classification image. For the third value, for example, at the time of phase restoration, the same constant may be substituted for these phase values when the change in brightness is small, and the third value may be assigned when the difference value becomes 0. . In other words, if the initial phase is shifted for each pixel to restore the first and second phases, the sign of the difference is determined, and a ternary image is generated directly.
Since there is no need to create a phase-recovered image and perform threshold processing as an intermediate stage, the amount of memory used can be reduced.

FIG. 13 is an explanatory diagram of a classification process of each connected region using the region classification image, FIG. 14 is an explanatory diagram of a process of merging a small region into a large region (pre-processing), and FIG. It is explanatory drawing of the grouping process of a 2nd area | region. However, hereinafter, each procedure of the flowchart of FIG. 11 will be described while scanning from the left stripe to the right stripe of the area classification image of FIG. 13.

After step S901 in FIG. 11, FIG.
From the area classification image H5 of (a), the first area (N0, N
..) And the second area (P0, P1,...) Are obtained (S902, S903). At this time, as shown in FIG. 14A, if there are a large number of regions having a small area equal to or less than the predetermined number of pixels, the number of the first region and the second region increases, and the processing becomes complicated. It is desirable to apply a process of merging the small area with the surrounding large area by processing or the like, and to generate an image in which only the large area is left as shown in FIG.

Subsequently, the start point position (Sn0, Sn1,...) And end point position (En0,
.., And the start point position (Sp0, Sp1,...) And end point position (Ep0, Ep1,...) Of each second region in FIG. 13C are detected (S904). Based on the relationship between the start point position and the end point position, a set of a first area and a second area adjacent to each other is detected (FIG. 1).
1 (S906). Thereafter, the phase number i is initialized to 0 (S904). (1) Processing for First Area N0 to N2 and Second Area P0 to P2 End point En0 of first area N0 and start point Sp0 of second area P0, and end point En1 and second area P1 of first area N1 , The end point En2 of the first area N2 and the start point Sp2 of the second area P2 are adjacent to each other without intersecting each other. Therefore, any of these sets includes the protrusion 10A and the hole 10B. Therefore, the phase regions I0, I1, and I2 are generated as a set as shown in FIG. Such processing is performed in steps S906 and S90.
7, S908, S909, and S906. (2) Processing for First Area N3 and Second Area P3 The end point En3 of the first area N3 and the start point Sp3 of the second area P3 are partially adjacent to each other and separated from each other at the remaining portion (third value connection area). However, since the end point position En3 and the start point position Sp3 do not intersect with each other, FIG.
(S906, S9)
07, S908, and S909: No). (3) First region N4 and second regions P4, P5 in FIG.
Processing for 1. When Connected Region of Third Value (also referred to as Third Region) is Shorter than One Phase FIGS. 16 and 17 show explanatory views of the classification process of each connected region, and FIG. 18 shows the grouping of the first region and the second region. FIG. 4 shows an explanatory diagram of the processing.

In FIG. 13, the second region paired with the first region N4 is divided into two connection regions P4 and P5 since the third region is affected by the hole 10B.
In the measurement based on the phase shift, the phase value needs to be continuous in the direction orthogonal to the phase change direction (the direction along the stripes), and thus is divided into two connected regions like the second regions P4 and P5. And it needs to be repaired into one connected region. For this reason, in step S902, the third value is regarded as the first value as shown in FIG. 16A to form an extended first area, and in step S903, the third value is changed to the first value as shown in FIG. The extended second area is regarded as binary.

In this case, the first region N4 in FIG. 13B is fused with the adjacent first region N5 to form one extended first region N4a as shown in FIG. 16A. Can be correctly separated for each phase by using the information of the connected region (S911, S912).

In FIG. 16, when the start point position Sn4a and end point position En4a of the extended first area N4a are compared with the start point position Sp4a and end point position Ep4a of the extended second area P4a, the start point position Sp4a and the end point position Ep4a are compared.
Are located between the start point position Sn4a and the end point position En4a. From this, FIG.
It can be estimated that the first region N4 is connected by the third region over two phases. In this case, as shown in FIG. 17, at the position of the end point Ep4a of the extended second area P4a, N
4a is divided into N4b and N5, and the newly generated first
If the end point En4b of the area N4b is detected (S911),
The end point position En4b of the first region N4b is the second position shown in FIG.
Since the start point position Sp4a of the region P4a is partially separated but does not intersect, the first region N4b and the second region P4a are combined as in the case of (2), and the phase region I4 shown in FIG. Is generated (S912).

2. When Connected Region of Third Value Exceeds One Phase or More FIGS. 19 and 20 show explanatory diagrams of the classification process of each connected region, and FIG. 21 shows an explanatory diagram of the classification process of the first region and the second region. . In this case, the processing of steps S913 to S917 is performed.

In FIG. 19, when the start point position Sn4c and the end point position En4c of the extended first area N4c are compared with the start point position Sp4c and the end point position Ep4c of the extended second area P4c, the start point position Sp4c of the second area is the first area. Is located to the left of the end position En4c of the
Since p4c is on the right side of the end point position En4c of the first area, it can be seen that the first extended area N4c and the second extended area P4c intersect (S906, S910). From this, the extended first area N4c and the extended second area P
4c can be estimated to be spread over two or more phases by the third value connection region.

Therefore, regarding the extended second area P4c, if the connected area of the third value is determined again without regarding the connected area of the third value as the second value (S913), as shown in FIG. It is divided into P6 (S914). The second of these
Since the region P6 is continuous in the direction orthogonal to the phase change direction, it can be determined that the second regions P4 and P5 are separated by the third value connection region. Therefore, FIG.
As shown in (c), the second regions P4 and P5 are temporarily fused to form a connection region P4d (S915), and the connection region P4
The extended first area N4c is divided into extended first areas N4d and N5 at the end point position Ep4d of d (S916). Thereby, the end point position of the extended first area N4d becomes En4d,
FIG. 20A shows the end point position En4d of the extended first area N4d.
Since the first region N4d and the second region P4d are adjacent to the start point positions Sp4 and Sp5 of the second regions P4 and P5, the phase region I4 shown in FIG. 21 is generated (S91).
7). (4) Processing for divided first region N5 and second region N6 When the connected region of the third value is shorter than one phase: the end point position E of the divided first region N5 shown in FIG.
Since n5 and the start point position Sp6 of the second area P6 shown in FIG. 16B are adjacent to each other without intersecting, the phase area I5 is generated as a set as shown in FIG. I do.

2. When the Connected Region of the Third Value Exceeds One Phase or More The End Point En5 of the Divided Extended First Region N5 shown in FIG. 20C and the Second Region P shown in FIGS. 20A and 20B
Since the start point Sp6 of No. 6 is adjacent without intersecting, the phase area I5 is generated as a set as shown in FIG.

FIG. 22 is an explanatory diagram of a phase boundary detection result, FIG. 23 is an explanatory diagram of a phase concatenation process result, and FIG. 24 is a diagram illustrating an example of height information. The detection result of the phase boundary shown in FIG. 22 is obtained by the flow of FIG. In FIG. 22, since the end point positions Ei0 to Ei5 of each phase region are the phase connection positions, the process of adding 2π for each phase end point position from the left to the phase value shown in FIG. 8 is performed as shown in FIG. Such a phase-coupled image H6 is obtained. Then, height information as shown in FIG. 24 is obtained from the phase-coupled image H6. As described above, a connected region is obtained for each pattern obtained by binarizing the restored phase value, and the positional relationship between the connected region of the third value corresponding to a shadow or a hole for which phase recovery cannot be performed is evaluated. By obtaining, the correct height information can be obtained even when the phase boundary is concealed by a shadow or a hole.

As described above, according to the first embodiment, it is possible to examine how the positional relationship between the first value and the second value connected areas changes depending on the presence or absence of the third value connected area. The combination of the first region and the second region is extracted by removing the influence of the value connection region, and the phase boundary can be correctly determined as shown in FIG.

FIG. 25 is a perspective view of an object having a projection larger than that of FIG. 4, FIG. 26 is a view showing an example of a striped image obtained by photographing the object of FIG. 25, and FIG. FIG. 9 is a diagram showing a phase-recovered image, a region-classifying image, and the like, which are restored from the stripe image of FIG.

In the case of the object 20 shown in FIG. 25, a stripe image H3 is obtained every time the phase shift is photographed, but the image content of the stripe image H3 is different from that of FIG. are doing.

In the fringe image H3 of FIG. 26, the set of stripes G1 and G2 for one phase corresponds to the set of stripes G3 for one phase shifted to the left by the convex surface of the projection 20A. Then, as shown in FIG. 27, the first regions G corresponding to G1 to G3 respectively.
Since 1n to G3n and the second regions G1p to G3p are not continuous, in the first embodiment, the connection regions (G4p, G4p in FIG. 27) that are vertically continuous without being affected by the protrusion 20A.
G5n) is detected, and association is performed based on this. However, even if a connected region that is continuous in the vertical direction without being affected by the protrusion 20A is detected, G in FIG.
Although there are three sets G6, G7, G1 between 4 and G5, since there is only one set on the convex surface of the projection 20A, the numbers do not match as they are, and the first embodiment is applied. Can not do it. That is, in the first embodiment, measurement cannot be performed when the phase shift due to a stepped projection or the like exceeds 2π. This number mismatch is due to the fact that a stripe adjacent to the left of G3 in FIG. 26 is projected on the left side surface of the projection 20A and becomes a blind spot from the camera 2 observed from directly above. Therefore, it is necessary to estimate this by image processing.

In the second embodiment, the protrusion 20A is extracted in advance, and the first value and the second value existing in the protrusion 20A are determined based on the information of the connection area of the third value due to the shadow of the protrusion 20A. The positional shift of the value connection area is estimated and associated. FIG.
If the third value connection area A _H5 of FIG. 7B can be determined to be the shadow of the projection 20A and the projection direction is known, the set G3 of FIG. Since it can be considered that the position is shifted to the left by the width of the connection region A _{H5 in} b), it can be determined that the positions correspond to the sets G1 and G2.

Next, a method of extracting a portion (hereinafter, referred to as a slide area) in which stripes are shifted by the protrusion 20A in advance will be described. The reason why the slide area is extracted in advance is that, in the area classification image H5 shown in the example of FIG. 27B, the G3p part erroneously forms one connected area with the G6p above and below it (there is a possibility). This is because it is necessary to treat G3p and G6p above and below G3p separately.

In the extraction of the slide area, for example, a component that forms a step as a protrusion on an electronic circuit board or the like is known, and if it can be detected by pattern matching or the like, a known component is detected in advance. It is good.

If the part forming the step is unknown,
Since it is also possible to detect a slide area (a convex area of a projection forming a step) and a third value connected area which is a shadow using only the area classification image and the phase restoration image, this method will be described next. I do.

As a first method for extracting a slide area, there is a method based on the appearance of a shadow in another area classification image obtained by changing the projection direction. FIG. 28 shows a configuration diagram of a three-dimensional shape measurement system that executes the method. However, for identification, reference numeral 1 in parentheses in FIG.
L and 1R are used appropriately. It is also assumed that the object is the same as in FIG.

In the three-dimensional shape measuring system shown in FIG.
A plurality of projectors 1 are used and arranged so as to irradiate the object 10 with a stripe pattern from opposite directions.
From a plurality of images projected using one projection device 1 (1L), an area classification image H as shown in FIG.
5 are obtained, and an area classification image H5 as shown in FIG. 29B is obtained from a plurality of images projected using the other projection device 1 (1R). In FIG. 29, since the projection direction is opposite, one shadow range Ap5 occurs in the opposite direction.
Further, a third value connection area is formed in a range corresponding to the same hole 10B regardless of the projection direction. In this case, if the third-valued connected areas that are on one side and not on the other side are extracted from both the area classification images H5, only the shadow area is extracted. Then, both of the extracted connected regions A
_As shown in FIG. 30, the area between _H5 can be detected as a slide area indicating the presence of a part or the like forming a step as a protrusion.

Next, as a second method, a method of extracting a slide area by using one projection device 1 to determine whether or not the connected area of the third value is a shadow will be described. First, as to whether or not the third value connected area is a shadow, if it is known that the target object has no hole, all the third value connected areas may be determined as shadows. If not, there is a method using continuity of the phase value.
As shown in FIG. 27A, in the shadow portion of the projection 20A, the projected continuous sine wave stripe pattern is apparently divided by the shadow, and thus the shadow along the direction in which the shadow extends. Are almost the same at both ends (p1 and p2). Therefore, the restoration phase values at both ends of the shadow are examined, and if the numerical values are almost the same, it can be regarded as a shadow.

After extracting the third region due to the shadow in this manner, the slide region can be extracted by detecting the boundary of the slide region based on the phase shift of the stripe. As shown in FIG. 31, the upper left and lower edges p3 and p4 of the third region due to the shadows
At each point on the straight lines L1 and L2 extending in the direction opposite to the direction in which the shadow extends from, the change in the phase restored in a direction substantially orthogonal to L1 and L2 is examined. At this time, at the boundary (p5, p6) of the protrusion 20A, the phase value is discontinuous between the inside and the outside. Therefore, using this, p3, p4
The portion where the phase discontinuity is tracked from the left to the left, and the position where the phase discontinuity disappears is determined by the left end (p7, p7) of the projection 20A.
8). According to this method, p7-p8
A line segment between the line segment and a region surrounded by the left end line p3-p4 of the third region A _H4 can be detected as the protrusion 20A.

As described above, according to the second embodiment, the slide area corresponding to the protrusion is extracted, the length of the shadow caused by the protrusion is measured, and the pattern (first and second) existing in the slide area is measured.
The second region) is shifted in the direction opposite to the direction in which the shadow extends, and is associated with a pattern outside the slide region, so that correct phase connection processing can be performed even on an object having a step or a hole. Information is obtained.

FIG. 32 is an explanatory diagram of a method for compensating for a lack of height information, and FIG. 33 is a diagram showing a result of compensating for a lack of height information. The third embodiment of the present invention will be described with reference to these figures. An embodiment will be described.

In the first and second embodiments, since the phase cannot be restored by the connection area of the third value, the height information cannot be generated. Depending on the application, such lack of distance information may be inconvenient.In such cases, data is complemented based on surrounding height information to eliminate the lack of distance information. Can be.

For example, as shown in FIG.
When the height information in the (third value connected area) is missing, the height information in the area C adjacent to the range A and on the side where the shadow extends is extrapolated as the height information E in the range A. By doing
The missing height information can be supplemented in a pseudo manner. Further, if the lack of the height information of the range B corresponding to the hole is inconvenient, based on the height information in the regions C and D adjacent to the range B, the height is equal to or less than the height. A method of filling a hole with information F of a low predetermined height (the same height in the example of FIG. 32) can be applied.

As described above, in the third embodiment, the height information in the range to be complemented is supplemented by using the height information of the area adjacent to the range, and therefore, as shown in FIG. Information can be obtained.

[0079]

According to the first aspect of the present invention, there is provided a three-dimensional shape measuring method based on a phase shift method, wherein an image of a pattern in which the brightness changes sinusoidally along the direction in which the stripes are arranged. Is projected onto the object, and photographing of an image of the pattern projected on the object and a certain amount of shift of the pattern are repeated to capture at least three types of images in which the phase of the brightness is shifted by a fixed amount, and these are captured. From the image, having a plurality of pixels respectively corresponding to a plurality of pixels of the image,
Among the plurality of pixels, a first value is assigned to a pixel corresponding to a pixel belonging to a part of the one cycle of the brightness, and a second value is assigned to a pixel corresponding to the remaining part of the one cycle of the brightness. Is assigned, and a pixel corresponding to a pixel whose brightness does not change due to the shift of the brightness phase is created with an area classification image in which a third value is allocated. Using the area classification image, A region where each of the third value pixels is continuous is classified, and based on the classification result, a first region where the first value pixels are continuous and a second region where the second value pixels are continuous Grouping a first region and a second region adjacent to each other along the direction in which the stripes are arranged, and performing a phase connection process using the grouping result; Since the height information of the three-dimensional shape of Even if there is a step or hole, it is possible to perform processing of correct phase coupling by using the grouping results, also possible to realize a three-dimensional shape measurement at high speed with low cost. As a result, the measurement of the three-dimensional shape, the simple phase connection processing, and the high-speed measurement that do not impair the resolution of the imaging device can be performed without increasing the cost and the imaging time.

According to the second aspect of the present invention, in the three-dimensional shape measuring method according to the first aspect, when the area classification image is created, the same position among the pixels of the plurality of captured images is used. To create a phase-recovered image in which the calculated phase value is assigned to each pixel as the value of the pixel corresponding to the position by substituting the brightness of each pixel in the predetermined arithmetic expression. The image for use corresponds to the position of the pixel whose brightness changes due to the shift of the brightness phase, and the value of the corresponding pixel in the phase-recovered image is smaller than a predetermined threshold value, and a first value is assigned to the pixel. A second value is assigned to a pixel corresponding to the position of a pixel whose brightness changes due to the shift of the brightness phase and the value of the corresponding pixel in the phase-recovered image is larger than a predetermined threshold value, and the brightness phase is No The pixel corresponding to the position of the pixel whose brightness does not change in the default value is created by assigning a third value, and the process of the phase connection is performed using the grouping result for the phase restored image. The method also enables measurement of a three-dimensional shape without impairing the resolution of the imaging device, simple phase connection processing, and high-speed measurement without increasing the cost and the imaging time.

According to a third aspect of the present invention, in the three-dimensional shape measuring method according to the first aspect, the area classification image is located at the same position among the pixels of the plurality of captured images. The process of calculating the phase value by substituting the brightness of the pixel into a predetermined arithmetic expression is performed depending on whether one of the captured images is used for the initial phase and the other image is used for the initial phase. While executing in parallel, a difference value between a phase value when the one image is used for the initial phase and a phase value when the other image is used for the initial phase is obtained. If the first value is negative, the second value is assigned, and if the brightness does not change, the third value is assigned. The phase concatenation process uses the grouping result for each of the phase values. Is performed, a phase restoration image is created as an intermediate stage, and threshold processing is performed. No need to carry out, it is possible to reduce memory usage.

According to a fourth aspect of the present invention, in the three-dimensional shape measuring method according to the first aspect, the certain amount is obtained by dividing one cycle of the brightness by a predetermined number that is an integer greater than three. When creating the area classification image, the phase value calculated by substituting the brightness of each pixel at the same position among the pixels of the plurality of captured images into a predetermined arithmetic expression is A phase-recovered image assigned to each pixel as a value of a pixel corresponding to a position, of the captured images, when one image is used for an initial phase, and when the captured image is cyclically determined from the predetermined number. The other images are shifted by a small integer number of times and the other images are used for the initial phase, respectively. Thereafter, the area classification image has a positive difference value between the values of the corresponding two pixels in the two phase restoration images. If the first value is negative, A second value is created by assigning a third value to each pixel if the brightness does not change, and the phase concatenation process is performed using the grouping result for one of the two phase restored images. Therefore, it is not necessary to increase the number of projections, and high-speed measurement can be performed.

According to a fifth aspect of the present invention, in the three-dimensional shape measuring method according to the first aspect, a pixel value of each pixel of the area classification image is set along the direction in which the stripes are arranged. The phase connection processing is performed by using the position of the pixel that changes from the second value to the first value as the position of the phase connection, so that when measuring an object without holes or steps, the phase connection position can be detected at high speed. it can.

According to a sixth aspect of the present invention, in the three-dimensional shape measuring method according to the first aspect, in the phase connection processing, the pixels of the first value are continuous using the area classification image. The first area and the second area in which the pixels of the second value are continuous are classified, and the first point and the end point along the direction in which the stripes are arranged in the first area and the second area are obtained, respectively. And evaluation of the positional relationship between the second area and the second area. The first area and the second area adjacent to each other along the direction in which the stripes are arranged are grouped according to the evaluation result. In addition, the position of the phase connection can be detected based on the global area information, and the occurrence of incorrect phase connection can be prevented.

According to the seventh aspect of the present invention, in the three-dimensional shape measuring method according to the sixth aspect, before the grouping, in each of the first area and the second area, another small area less than a predetermined number of pixels is provided. If it is included, the process of merging the small area with the area including the small area is performed, so that the number of unnecessary groups is reduced,
Processing speed is improved.

According to an eighth aspect of the present invention, in the three-dimensional shape measuring method according to the sixth aspect, an extended first area including a third area in which the pixels of the third value are continuous is included in the first area. A set of the first region and the second region which are adjacent to each other based on an overlapping state of the first and second regions in the expansion, including the third region in the second region. Since the division is performed, when the first area or the second area overlaps with the third area, it is possible to prevent the area that should originally be one from being divided and becoming unrecognizable.

According to the ninth aspect of the present invention, in the three-dimensional shape measuring method according to the eighth aspect, when the first area and the second area for a plurality of phases appear to be connected by the third area, The first region and the second region are obtained again except for the three regions, and a region that is not divided in the direction along the stripe is extracted from the first region and the second region. Since the areas divided in the direction along the stripes are combined, the third
Even when the region separates the first region and the second region, correct phase connection can be performed.

According to the tenth aspect, the second aspect is provided.
In the three-dimensional shape measurement method according to the above, the image captured by repeating the photographing and shifting, the image of the pattern is projected onto the object from one direction and the image of the pattern is projected from another direction. Two sets of the phase-reconstructed image and the two sets of the phase-reconstructed image are prepared from the two sets of images, and an area where the pixel of the third value is continuous in only one of the phase-reconstructed images is prepared. If there is, the area is defined as a shadow area of the protrusion that forms a step in the object, and an area sandwiched between the shadow areas is defined as the protrusion existence area. It can be easily extracted, and it can be determined at the same time whether the third area is a shadow.

According to the eleventh aspect, according to the second aspect,
In the three-dimensional shape measurement method according to the aspect, among the pixels of the area classification image, a phase restoration image at a start point and an end point along a direction in which the stripes are arranged in a third area where the pixels of the third value are continuous. When the pixel values are substantially the same, the third area is defined as a shadow area of the protrusion that forms a step in the object, so that it is easy to determine whether the third area is a shadow. it can.

According to the twelfth aspect of the present invention, the first aspect is provided.
12. The three-dimensional shape measurement method according to 0 or 11, wherein each of the pixels of the area classification image corresponds to each pixel in a direction along the stripe in a direction opposite to a direction in which the shadow extends from the shadow range. The values of the pixels of the phase-recovered image are examined, and if the values of these pixels are discontinuous, the projections are assumed to continue in the direction opposite to the direction in which the shadow extends, and the discontinuity is eliminated. In this case, since the protrusion is a boundary where the protrusion is interrupted, the process of extracting the region of the protrusion can be easily performed.

According to the thirteenth aspect, according to the first aspect,
In the three-dimensional shape measurement method according to any one of 0 to 12, a length of the shadow range along the direction in which the stripes are arranged is determined as a pattern shift amount, and the length is determined in a direction opposite to a direction in which the shadow extends. The first value or the second value in the existing protrusion;
The area where the pixels having the same value are continuous is associated with the area outside the protrusion that is shifted in the direction in which the stripes are arranged by the pattern shift amount. Therefore, even when there is a large step exceeding one cycle, correct association can be achieved. I can do it.

According to the fourteenth aspect of the present invention, the first aspect is provided.
In the three-dimensional shape measurement method described above, phase value or height information in a third region in which the pixels of the third value are continuous is complemented based on phase value or height information outside the third region. Therefore, no loss of height information occurs.

According to the fifteenth aspect, the first aspect is provided.
4. In the three-dimensional shape measurement method according to 4, when the third region in which the pixels of the third value are continuous is caused by a shadow of a projection that forms a step in the object, the phase value or the height in the third region. The information is complemented based on the phase value or height information of the area adjacent to the third area and in the direction in which the shadow of the projection extends, so that the height information is lost without damaging the shape of the projection. Can be complemented.

The invention according to claim 16 is a three-dimensional shape measuring system based on the phase shift method, wherein an image of a pattern in which the brightness changes in a sinusoidal manner along the direction in which the stripes are arranged is applied to the object. A projection device for projecting, an imaging device for photographing an image of the pattern projected on the object, and a certain amount of shift of the pattern projected on the object is obtained repeatedly, and the phase of the lightness is obtained by a constant amount. A storage device for capturing and storing at least three types of shifted images, and a plurality of pixels respectively corresponding to a plurality of pixels of the image, from the images stored in the storage device, and among the plurality of pixels, A pixel corresponding to a pixel belonging to a part of one cycle of brightness is assigned a first value, a pixel corresponding to a pixel belonging to the remainder of one cycle of brightness is assigned a second value, and the phase of the brightness is assigned. The pixel corresponding to the pixel brightness does not change the shift creates a region classification image composed assigned third value, the using the area classification image first
To a third area in which each of the pixels of the third value is continuous, and based on the classification result, a first area in which the pixels of the first value are continuous and a second area in which the pixels of the second value are continuous The first region and the second region adjacent to each other along the direction in which the stripes are arranged are grouped, and a phase connection process is performed using the grouping result. Since it has an image processing device that extracts height information of the three-dimensional shape of the object, even if the object has a step or a hole, it is possible to perform the correct phase connection processing by using the grouping result. In addition, three-dimensional shape measurement can be realized at low cost and at high speed. As a result, the measurement of the three-dimensional shape, the simple phase connection processing, and the high-speed measurement that do not impair the resolution of the imaging device can be performed without increasing the cost and the imaging time.

[Brief description of the drawings]

FIG. 1 is an explanatory diagram of a three-dimensional shape measurement system.

FIG. 2 is a flowchart illustrating a procedure of a three-dimensional shape measurement method.

FIG. 3 is a diagram illustrating the principle of phase restoration.

FIG. 4 is a perspective view of an object.

5 is a top view and a sectional view of the object shown in FIG. 4;

FIG. 6 is a diagram illustrating an example of a stripe image obtained by capturing an object and a change in brightness along a straight line of the stripe image.

FIG. 7 is a diagram illustrating an example of each fringe image obtained by a phase shift and a change in brightness along a straight line of each fringe image.

FIG. 8 is a diagram illustrating an example of a phase restored image and a phase value change along a certain straight line of the phase restored image.

FIG. 9 is an explanatory diagram of a phase connection process.

FIG. 10 is a diagram illustrating an example of an area classification image and a change in brightness along a straight line of the area classification image.

FIG. 11 is a flowchart illustrating a specific processing procedure of phase boundary detection in FIG. 2;

FIG. 12 is an explanatory diagram of a method for generating an area classification image.

FIG. 13 is an explanatory diagram of a classification process of each connected region using a region classification image.

FIG. 14 is an explanatory diagram of a process of merging a small area into a large area.

FIG. 15 is an explanatory diagram of a grouping process of a first area and a second area.

FIG. 16 is an explanatory diagram of a classification process of each connected region.

FIG. 17 is an explanatory diagram of a classification process of each connected region.

FIG. 18 is an explanatory diagram of a grouping process of a first area and a second area.

FIG. 19 is an explanatory diagram of a classification process of each connected region.

FIG. 20 is an explanatory diagram of a classification process of each connected region.

FIG. 21 is an explanatory diagram of a grouping process of a first area and a second area.

FIG. 22 is an explanatory diagram of a detection result of a phase boundary.

FIG. 23 is an explanatory diagram of a result of the phase connection processing.

FIG. 24 is a diagram illustrating an example of height information.

FIG. 25 is a perspective view of an object having a protrusion larger than the object of FIG. 4;

26 is a diagram illustrating an example of a striped image obtained by photographing the object in FIG. 25.

FIG. 27 is a diagram showing a phase-recovered image, a region-classifying image, and the like restored from a plurality of stripe images.

FIG. 28 is a configuration diagram of a three-dimensional shape measurement system that detects a projection position of a target object by using a shadow of the projection.

FIG. 29 is an explanatory diagram of a method for detecting a projection position of a target object from an area classification image.

FIG. 30 is a diagram showing a protrusion position detected by the method of FIG. 29;

FIG. 31 is an explanatory diagram of a method for detecting the position of a projection of an object.

FIG. 32 is an explanatory diagram of a method of complementing a missing height information.

FIG. 33 is a diagram showing a result of complementing missing height information.

[Description of Signs] 1 Projection device 11 Lamp 12 Concave mirror 13 Projection lens 14 Pattern plate 15 Pattern shift unit 2 Camera 3 Quantization circuit 4 Storage device 5 Image processing device

Claims (16)

[Claims] 1. A three-dimensional shape measuring method based on a phase shift method, wherein an image of a pattern in which the brightness changes in a sinusoidal manner along a direction in which the stripes are arranged is projected on the target. By repeating the photographing of the image of the pattern projected on and the certain amount of shift of the pattern, at least three types of images in which the brightness phase is shifted by a fixed amount are taken in, and from these images, a plurality of pixels of this image are taken. Each of the plurality of pixels has a corresponding one of the plurality of pixels, and among the plurality of pixels, a pixel corresponding to a pixel belonging to a part of the one cycle of the brightness is assigned a first value, and belongs to the remainder of the one cycle of the brightness. A second value is assigned to a pixel corresponding to the pixel, and a third value is assigned to a pixel corresponding to a pixel whose lightness does not change due to the shift of the lightness phase. Using a similar image, a region in which each of the first to third value pixels is continuous is classified, and based on the classification result, a first region in which the first value pixel is continuous and the second value are classified. Out of the second area where the pixels are continuous, the first area and the second area adjacent to each other along the direction in which the stripes are arranged are grouped, and a phase connection process is performed using the grouping result. Extracting a height information of a three-dimensional shape of the object according to a result of the processing. 2. A phase value calculated by substituting the brightness of each pixel at the same position among the pixels of the plurality of captured images into a predetermined arithmetic expression when creating the area classification image. Creates a phase-recovered image that is assigned to each pixel as the value of the pixel corresponding to that position, and thereafter, the area classification image corresponds to the position of the pixel whose lightness changes due to the lightness phase shift And a first value is assigned to a pixel having a value of a corresponding pixel smaller than a predetermined threshold value in the phase-recovered image, and the pixel corresponds to a position of a pixel whose brightness changes due to the shift of the brightness phase, and A second value is assigned to a pixel whose value of a corresponding pixel in the restored image is larger than a predetermined threshold, and a pixel corresponding to a position of a pixel whose brightness does not change due to the shift of the brightness phase,
The three-dimensional shape measurement method according to claim 1, wherein a third value is created by being assigned, and the phase connection process is performed using the grouping result for the phase restoration image. 3. The region classifying image includes a process of calculating a phase value by substituting the brightness of each pixel at the same position among the pixels of the plurality of captured images into a predetermined arithmetic expression. Among the captured images, the phase value when the one image is used for the initial phase while the one image is used for the initial phase and the other image is used in parallel for the initial phase. And a difference value between the other image and the phase value when the other image is used for the initial phase. If the difference value is positive, the first value must be changed. If the difference value is negative, the second value must be changed. The three-dimensional shape measuring method according to claim 1, wherein a third value is allocated and created, and the phase connection process is performed using the grouping result for each of the phase values. 4. The fixed amount is an amount obtained by dividing one cycle of the lightness by a predetermined number that is an integer greater than 3. When creating the area classification image, the fixed amount of the plurality of captured images is Of each pixel, a phase restoration image in which a phase value calculated by substituting the brightness of each pixel at the same position into a predetermined arithmetic expression is assigned to each pixel as a value of a pixel corresponding to that position, Of the captured images, one image is used for the initial phase and the captured image is cyclically shifted by an integer number smaller than the predetermined number and the other image is used for the initial phase. Thereafter, in the area classification image, the first value is changed if the difference between the values of the corresponding two pixels in the phase-recovered images is positive, the second value is changed if the difference is negative, and the brightness is changed. Otherwise, a third value is assigned to each pixel Made is, three-dimensional shape measuring method according to claim 1, wherein said processing phase coupling is characterized by being carried out by using the groupings result in one of the two phase recovery image. 5. A position of a pixel whose pixel value changes from the second value to the first value along the direction in which the stripes are arranged in each pixel of the area classification image is set as a phase connection position. ,
The three-dimensional shape measurement method according to claim 1, wherein the phase connection processing is performed. 6. The phase connection processing classifies a first area in which the first value pixels are continuous and a second area in which the second value pixels are continuous using the area classification image. In each of the first region and the second region, a start point and an end point along the direction in which the stripes are arranged are determined to evaluate the positional relationship between the first region and the second region. 2. The three-dimensional shape measuring method according to claim 1, wherein the method is performed by grouping the first region and the second region adjacent to each other along a direction in which the stripes are arranged. 7. The method according to claim 1, wherein the first region and the second region are arranged before the grouping.
7. The three-dimensional shape measuring method according to claim 6, wherein when each of the areas includes another small area equal to or less than a predetermined number of pixels, a process of merging the small area with an area including the small area is performed. 8. An extended first area including a third area in which the pixels of the third value are continuous in the first area, and an extended second area including the third area in the second area. 7. The three-dimensional shape measuring method according to claim 6, wherein the first region and the second region adjacent to each other are classified based on the overlapping state of the first and second regions. 9. When the first region and the second region for a plurality of phases appear to be connected by the third region, the third region
The first region and the second region are obtained again except for the region,
Extracting a region that is not divided in the direction along the stripe from the first region and the second region, and combining the regions divided in the direction along the stripe based on this region; 9. The three-dimensional shape measurement method according to claim 8, wherein: 10. An image captured by repeating the photographing and shifting, wherein an image of the pattern is projected onto the object from one direction and an image of the pattern is projected onto the object from another direction. Two sets of the phase-recovered images are prepared from these two sets of images, and if only one of the phase-recovered images has an area where the pixel of the third value is continuous, 3. The three-dimensional image according to claim 2, wherein the area is defined as a shadow area of the projection that forms a step in the object, and an area sandwiched between the shadow areas is defined as an existing area of the projection. Shape measurement method. 11. Each pixel of the area classification image,
When the values of the pixels of the phase-recovered image at the start point and the end point along the direction in which the stripes are arranged in the third region in which the pixels of the third value are continuous are substantially the same, the third region is stepped in the object. 3. The three-dimensional shape measuring method according to claim 2, wherein the projection is a shadow area of the projection. 12. Each pixel of the area classification image,
Check the values of the pixels of the phase-recovered image corresponding to each pixel in the direction along the stripe in the direction opposite to the direction in which the shadow extends from the range of the shadow, and when the values of these pixels are discontinuous, 12. The projection according to claim 10, wherein the projection is continued in a direction opposite to a direction in which the shadow extends, and when the projection is discontinuous, the projection is a boundary where the projection is interrupted. The three-dimensional shape measurement method described in the above. 13. A length of a range of the shadow along the direction in which the stripes are arranged is determined as a pattern shift amount, and the first portion of the projection in the direction opposite to the direction in which the shadow extends is obtained.
13. The area according to claim 10, wherein an area in which pixels of the value or the second value are continuous is associated with an area outside the protrusion that is shifted in the direction in which the stripes are arranged by the pattern shift amount. The three-dimensional shape measurement method described in the above. 14. The method according to claim 1, wherein phase value or height information in a third region in which the pixels of the third value are continuous is complemented based on phase value or height information outside the third region. The three-dimensional shape measurement method according to claim 1. 15. When a third region in which the pixels of the third value are continuous occurs due to a shadow of a protrusion forming a step in the object, the phase value or height information in the third region is converted to the third region. 15. The three-dimensional shape measurement method according to claim 14, wherein interpolation is performed based on phase value or height information of an area adjacent to the three areas and extending in a direction in which the shadow of the protrusion extends. 16. A three-dimensional shape measurement system based on a phase shift method, comprising: a projection device for projecting an image of a pattern in which the brightness changes in a sinusoidal manner along a direction in which the stripes are arranged on a target object; An imaging device that captures an image of a pattern projected on the object; and at least 3 images obtained by repeating a certain amount of shifts of the pattern projected on the object, wherein the lightness phase is shifted by a certain amount at a time. A storage device for capturing and storing the type; and from the image stored in the storage device, the storage device includes a plurality of pixels respectively corresponding to a plurality of pixels of the image. The first value is assigned to the pixel corresponding to the pixel belonging to
A pixel corresponding to a pixel belonging to the rest of the cycle is assigned a second value, and a pixel corresponding to a pixel whose brightness does not change due to the shift in brightness phase is assigned a third value. The first area where the first value pixels are continuous is created based on the classification result, and the area where the first to third value pixels are continuous is classified using the area classification image. And a second pixel in which the pixels of the second value
Among the regions, a first region and a second region adjacent to each other along the direction in which the stripes are arranged are grouped, and a phase connection process is performed using the grouping result. An image processing device for extracting height information of the three-dimensional shape of the target object.