JP2006023178A - 3-dimensional measuring method and device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a 3-dimensional measuring method for an improvement in measuring precision and its device. <P>SOLUTION: The phase reliability determination part 43 is provided with the image processor 4 of the 3-dimensional measuring device. The phase reliability determination part 43 extracts the luminance saturation region, contrast lowered region, boundary region of the contrast lowered region, and the phase error region. For the regions, wherein the reliability of measurement of the phase value is insufficient, re-measurement is separately performed for obtaining the phase value of high reliability. Thereby, the 3-dimensional measurement of high precision is obtained as a whole. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a three-dimensional measurement method and apparatus using a phase shift method.

  As a technique for measuring the three-dimensional shape of an object in a non-contact manner, a lattice pattern projection method using a phase shift method is often used. Specifically, a lattice pattern having an illuminance distribution having a sine wave shape is projected onto an object to obtain a modified lattice image. Then, the lattice pattern is shifted laterally to obtain a plurality of images having different phases. This is a technique for obtaining a phase value of a lattice at each point for each pixel from a plurality of obtained images and calculating a shape of a measurement object using a phase connection process or the like.

As a conventional three-dimensional shape measuring method, there is a technique disclosed in Patent Document 1. In this technique, an object is irradiated with a grid pattern having color components, and processing is performed for each color component to obtain object information. Is disclosed.
Japanese Patent Laid-Open No. 9-21620

  However, in measuring a measurement object having a complicated surface, there is often a problem that the shape cannot be reproduced based on the calculated phase value. For example, when there is a luminance saturation region in the lattice image due to the material of the measurement object, or when a contrast-decreasing portion is formed due to a change in the surface of the measurement object, it is difficult to accurately measure the shape of the region. That is, in order to accurately measure the shape, it is necessary to determine the reliability of the phase value. Therefore, it is necessary to extract a portion that affects the reliability of the phase, such as a luminance saturation region and a contrast reduction region.

  Conventional methods for classifying and extracting portions that affect the reliability of phase values are not effective. For example, there is a method for extracting a low contrast area by projecting white and plain pattern light and using a captured image to determine a portion having a low luminance value as a low contrast area. However, this method has a problem that photographing other than the lattice image is necessary and the determination criterion is unclear.

  In addition, the extraction of the luminance saturation region includes a method of extracting all the regions where the luminance is saturated depending on the surface state of the object from the image obtained by projecting the pattern light, and determining the extracted region as an indefinite shape portion. is there. In this case, the extracted saturated portion is generally measured again by reducing the light amount. However, there may be a portion where the phase value is within the accuracy range even if there is a luminance saturation region in the extracted saturation region, and if such a portion is determined to be an indefinite shape, an information loss occurs. . Further, when measuring such a portion again by reducing the light amount, there is a possibility that a reduction in contrast occurs in other regions and the measurement accuracy is deteriorated.

  In the present invention, in addition to the lattice image, the phase value and amplitude value are used to extract the reliability of the phase value by extracting the luminance saturation region, the contrast reduction region, the boundary region of the contrast reduction region, and the phase error region by specular reflection. The measurement accuracy is improved by making a determination and using the reliability determination result of the phase value with a general apparatus configuration.

  An object of the present invention is to provide a three-dimensional measurement method and apparatus with improved measurement accuracy.

  In the three-dimensional measurement method of the present invention, the lattice pattern is shifted a plurality of times to irradiate the measurement object, and an image of the irradiated lattice pattern is captured every time the image is shifted, and the three-dimensional shape is measured using the image. In the three-dimensional measurement method using the phase shift method, the step of calculating the phase value and the amplitude value of each pixel of the image, and the portion where the shape cannot be accurately measured by the calculated phase value are classified and extracted And determining the reliability of the phase value of each pixel based on the extracted result, and calculating the shape of the measurement object and displaying the result using the determination result. It is characterized by.

  The three-dimensional measurement apparatus of the present invention shifts the lattice pattern a plurality of times and irradiates the measurement target, captures the image of the irradiated lattice pattern every time it shifts, and measures the three-dimensional shape using the image. In the three-dimensional measurement apparatus using the phase shift method, the calculation means for calculating the phase value and the amplitude value of each pixel of the image, and the portion where the shape cannot be measured accurately by the calculated phase value are classified. Extraction means for extraction, determination means for determining the reliability of the phase value of each pixel based on the extracted result, and shape calculation for calculating the shape of the measurement object and displaying the result using the determination result And a display means.

  According to the present invention, the reliability of a phase value can be determined through extraction of a phase saturation region, a contrast reduction region, a boundary region between contrast reduction regions, a phase error region by specular reflection, and the like, and measurement accuracy can be improved.

  In the embodiment of the present invention, a three-dimensional measurement method using a phase shift method is used. The lattice pattern is projected onto the measurement object, the phase shift of the lattice pattern is performed a plurality of times, the lattice image at each phase shift is taken in, and the phase value and amplitude value of each pixel are calculated. Then, using the phase value and the amplitude value of each of the plurality of grid images and each pixel, classify and extract a portion whose shape cannot be reproduced using the calculated phase value, and based on the extracted result, The reliability of the phase value of each pixel is determined, and using the determination result, the shape of the measurement object is calculated and the result is displayed.

The parts to be classified and extracted include a luminance saturation region, a contrast reduction region, a boundary region of the contrast reduction region, a phase error region due to specular reflection, and the like.
Further, in the extraction of the luminance saturation region, a pixel whose luminance value is the maximum luminance value is extracted based on the plurality of grid images, and a plurality of extracted pixels are determined based on the adjacent relationship between the extracted pixels. Find adjacent areas adjacent to a small area, calculate the phase value histogram for each small area and its adjacent areas, compare the calculated phase value histograms, and measure the phase value even when the brightness is saturated It is determined whether the area is within the accuracy range.

  Further, in the extraction of the contrast reduction region, the average value and the standard deviation of the amplitude value are obtained for all the pixels other than the luminance saturation region, and the pixel whose amplitude value is smaller than the average value is processed, and the moving average method is used. A pixel having a small amplitude value in the neighboring region is extracted, and an average value and a standard deviation of the amplitude value are obtained with respect to the entire extracted pixel, and a final determination is made as to whether or not the region is a contrast lowered region.

In addition, the extraction of the boundary region of the contrast reduction region is performed by performing expansion processing on the contrast reduction region and using the result of the expansion processing.
In addition, the extraction of the phase error region by the specular reflection or the like is performed by calculating the relationship between the phase value difference of adjacent pixels and the irradiation angle of the lattice pattern for pixels other than the luminance saturation region, the contrast reduction region, and the boundary region of the contrast reduction region. Done with.

  A three-dimensional measurement apparatus according to an embodiment of the present invention is a three-dimensional measurement apparatus based on a phase shift method, and performs a phase shift of the lattice pattern a plurality of times, a means for projecting the lattice pattern onto a measurement object, A phase value calculated using a means for capturing a lattice image at the time of phase shift, a means for calculating a phase value and an amplitude value of each pixel, and a plurality of lattice images and the phase value and amplitude value of each pixel Means for classifying and extracting a portion whose shape cannot be reproduced, means for determining the reliability of the phase value of each pixel based on the extracted result, and means for calculating the shape of the measurement object using the determination result And means for displaying the result.

  The classifying and extracting means extracts at least one of a luminance saturation region, a contrast reduction region, a boundary region of the contrast reduction region, and a phase error region due to specular reflection.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is an explanatory diagram showing a configuration of a system including a three-dimensional measuring apparatus according to an embodiment of the present invention.

As shown in FIG. 1, the system includes a stage 1, a lattice pattern projection mechanism 2, a photographing mechanism 3, and an image processing device 4.
In the grating pattern projection mechanism 2, the light emitted from the light source 21 and passing through the illumination optical system 22 becomes parallel light rays parallel to the optical axis direction of the illumination optical system 22, and is in the grating pattern and the scanning device 23. Irradiate a lattice pattern. By controlling the grating pattern and the scanning device 23, the grating pattern is shifted in the direction perpendicular to the optical axis of the irradiation optical system 22, and the light transmitted through the grating pattern and the scanning device 23 passes through the projection optical system 24 on the stage 1. Is projected onto the measurement object 5. The projected pattern image has a brightness that changes in a sine wave shape.

  The light reflected by the measurement object 5 enters the camera 32 via the imaging system 31 of the imaging mechanism 3. As a result, an image of the lattice pattern deformed by the measurement object 5 is taken by the camera 32. The lattice pattern is moved over several stages, and output data (grid image) from the camera 32 is stored in the data storage unit 41 of the image processing device 4 for each position. Further, the camera resolution, the incident angle, the grating pitch, and the number of phase shift steps are stored in the data storage unit 41 in advance.

  The calculation of the phase value and the amplitude value is performed by the data processing unit 42. For example, the calculation of the phase value Φ (x, y) and the amplitude value γ (x, y) of the pixel (x, y) is performed using the following Expression 1 and Expression 2, respectively.

ここで、Ｉ_i（ｘ、ｙ）は、ｉ番目のシフトステップ時の画素（ｘ、ｙ）の輝度値、δ_i＝２πｉ／Ｎ （ｉ＝０、・・・、Ｎ−１）は位相シフト量、Ｎはシフトステップ数である。

Here, I _i (x, y) is the luminance value of the pixel (x, y) at the i-th shift step, and δ _i = 2πi / N (i = 0,..., N−1) is the phase. The shift amount, N is the number of shift steps.

  The phase value reliability determination unit 43 reads a plurality of lattice image data, phase values, and amplitude values, and determines the reliability of the phase values. Specifically, extraction of a luminance saturation region, extraction of a contrast reduction region, extraction of a boundary region of the contrast reduction region, extraction of a phase error region by specular reflection or the like is performed.

The data display unit 44 is a display or the like that displays the processing results in the above units.
Hereinafter, the contents of the extraction processing for each region will be described in detail.
FIG. 2 is a flowchart showing a procedure for extracting a luminance saturation region. FIG. 3 is a diagram for explaining the state of the luminance saturation region extraction process.

  First, in step S1, all pixels whose luminance is the maximum luminance value (for example, 255) of the camera are extracted from the plurality of grid images and set as the luminance saturation region A1. Next, the process proceeds to step S2, and the extracted pixels are divided into a plurality of small areas Ei (i = 0, 1, 2,. ..)) And the adjacent area NEi (i = 0, 1, 2,...) Of each small area is obtained. NEi is an area that is not saturated in luminance, is adjacent to the small area Ei, and has more pixels than the small area Ei. For example, a region where the Euclidean distance from the small region Ei is within a predetermined value (for example, x), the luminance is not saturated, and the number of pixels is twice that of the small region Ei (double of the predetermined value) may be used. .

In the example of FIG. 3A, one saturated region Ei and its adjacent region NEi on the planar sample are shown.
Next, in step S3, phase value histograms Hi and HNi of Ei and NEi are obtained.

Hereinafter, the method of obtaining the histogram will be described with reference to FIG.
When the phase value is obtained for each pixel with respect to the planar sample of FIG. 3A, the result shown in FIG. 3B is obtained for one phase value of the scanning line of the camera. Since there is an inclination of the phase value in the region Ei and the region NEi, it is difficult to compare two histograms obtained with such a phase value. In the embodiment of the present invention, first, the tilt angle β is obtained from the CCD resolution r (horizontal size of one pixel), the grating pitch Pitch, and the incident angle α according to the following Expression 3.

そして、ＥｉとＮＥｉの位相値の傾き補正を行う。例えば、図３（ｂ）に示すように画素Ｍを軸に、位相値を時計回りに角度βだけ回転させる。図３（ｃ）に示すような傾き補正後のＥｉとＮＥｉの位相値を用いて、それぞれのヒストグラムＨｉとＨＮｉを求める。

Then, the inclination correction of the phase values of Ei and NEi is performed. For example, as shown in FIG. 3B, the phase value is rotated clockwise by an angle β around the pixel M as an axis. Using the phase values of Ei and NEi after inclination correction as shown in FIG. 3C, respective histograms Hi and HNi are obtained.

In the above processing, when there is a jump of 2π between the phase values of Ei and NEi, connection processing is performed before tilt correction.
FIG. 4 is a diagram illustrating an example of a histogram generated in the luminance saturation region extraction process.

  Next, in step S4, two histograms Hi and HNi as shown in FIG. 4 are compared. Regarding the evaluation of the comparison result, in the embodiment of the present invention, the ratio of the number of pixels of the common part of two Hi and HNi to the number of pixels of Hi is used. Finally, in step S5, if the evaluation result is larger than the preset evaluation criterion, it is determined that the phase value of the small area Ei is highly related to the neighboring area, and this small area is designated as the area A2 (the phase value is accurate). A region within the range) (a boundary region of the luminance saturation region). In step S4, an evaluation method such as a correlation degree or a similarity degree may be used.

The extraction result of the luminance saturation region composed of the region A1 and the region A2 included in the region A1 is output to the data processing unit 42 and the data display unit 44.
FIG. 5 is a flowchart showing a procedure for extracting a contrast reduction region. FIG. 6 is a diagram showing a state of the process of extracting the contrast reduction region.

  First, an average value AVE of amplitude values is obtained for all the pixels other than the luminance saturation region A1 (step S11). Next, a pixel whose amplitude value is larger than the average value AVE is extracted and removed from the subsequent processing (step S12).

  FIGS. 6A and 6B are diagrams showing a processing example up to step S12. FIG. 6A shows a state of projection of a lattice pattern, and FIG. 6B shows a region that is not subject to contrast reduction region extraction processing.

  Further, smoothing processing is performed on the pixels remaining in step S12 (hereinafter referred to as effective pixels) using an average filter (step S13). Next, using the moving average neighborhood as shown in FIG. 6B, a pixel having a small amplitude value in the neighborhood region is extracted by the moving average method (step S14). The user can adjust the neighborhood size. As a specific extraction method, a pixel whose amplitude value of the pixel (x, y) is smaller than the local average value in the vicinity thereof is extracted. Here, the calculation of the local average value of the neighborhood is performed only for the effective pixels in the neighborhood area. FIG. 6C shows the result of imaging the amplitude value of only the extracted pixel in the example of FIG. Finally, the pixels extracted in step S14 may include those whose amplitude values are not small as a whole, so such pixels are extracted and excluded from the final result (step S15). As a specific method, the average value and standard deviation σ of the amplitude values of the entire pixels extracted in step S14 are obtained, and the pixels within a predetermined amplitude value range (average value ± 3σ) are used as the final result. FIG. 6D shows an amplitude value histogram of the extraction result of FIG. A part of the extraction result in FIG. 6C is the final result of the contrast reduction region.

The extraction result of the contrast reduction region is output as the region B to the data processing unit 42 and the data display unit 44.
In the lattice image, since the luminance value of the pixel at the boundary of the contrast reduction region is unstable, the reliability of the phase value of the pixel is low. For the extraction of this region, the extracted contrast reduction region is used. The contrast reduction region is expanded using a predetermined neighborhood, and the expanded portion is set as a boundary region of the contrast decrease region.

The extraction result of the contrast reduction region is output as the region C to the data processing unit 42 and the data display unit 44.
In the embodiment of the present invention, for the pixels other than the luminance saturation region, the contrast reduction region, and the boundary region of the contrast reduction region, the reliability determination of the phase value is performed using the relationship between the phase value difference between the adjacent two pixels and the incident angle. Do.

FIG. 7 is a diagram for explaining a method of extracting a phase error region by specular reflection or the like.
7A and 7B, the horizontal axis indicates the scanning line, and the vertical axis indicates the phase value. The incident angle of the incident light with respect to the vertical direction is α. Further, a point corresponding to the phase value Pa of the pixel a is T1, and a point corresponding to the phase value Pb of the adjacent pixel b of the pixel a is T2. -2π <(Pb-Pa) <2π holds between Pa and Pb. Here, when the measurement target is a plane sample, the relationship between T1 and T2 in which no phase jump occurs is as shown in FIG. In this case, β is obtained by the above-described Expression 3. When a phase jump occurs between the pixels a and b, theoretically (Pb−Pa) has a value close to −2π, but the phase jump value actually increases due to noise or the like (the absolute value is small). Therefore, the phase jump criterion is set to a value larger than −2π, for example, −π.

  In the embodiment of the present invention, as shown in FIG. 7B, the phase skip value (Pb−Pa) is larger than a predetermined reference with respect to the incident angle α, and [tan β−tan ( If smaller than (π / 2−α)], the phase values of the pixel a and the pixel b are considered as errors. That is, when a three-dimensional shape is reproduced using such a phase value, the light beam cannot enter the pixel b.

Using this phase error discrimination method, a phase error due to specular reflection and a part of a phase error due to random noise are extracted.
In the embodiment of the present invention, such a phase error is called a phase error due to specular reflection or the like.

FIG. 8 is a diagram for explaining the cause of formation of a phase error due to specular reflection.
In addition to the incident light 1, the main incident light incident on the region E includes reflected light 2 due to specular reflection in the nearby region E <b> 1. As a result, a change in the luminance value of the pixel in the region E in each grid image cannot be predicted.

The phase error region extraction result due to specular reflection or the like is output as a region D to the data processing unit 42 and the data display unit 44.
The above is the processing performed by the phase value reliability determination unit. Next, the data processing unit 42 calculates the shape of the measurement object using the phase value and the reliability determination result of the phase value (areas A1, A2, B, C, and D). The shape calculation method may be a conventional shape calculation method using phase connection processing, or a shape calculation method using a reference plane. By using the reliability of the phase value for these processes, a more accurate result can be obtained. For example, if the phase connection process is performed while avoiding a region with low phase value reliability, it is difficult for a phase connection error to propagate.

FIG. 9 is a diagram illustrating a display example of the processing result according to the embodiment of this invention.
Finally, the data display unit 44 displays a lattice image, a phase value result, an amplitude value result, and a shape calculation result, and also displays phase value reliability determination results (regions A1, A2, B, C, and D). To do. Color display may be used so that the areas A1, A2, B, C, and D can be distinguished. In addition, when calculating the shape, whether or not the pixels in each area are masked is also displayed. FIG. 9 shows an example of the display screen of the areas A1, A2, B, C, and D. From this display screen, it can also be seen that the processing was performed without masking the areas A2 and D and masking the other areas.

  In addition, since the reliability determination result of the phase value is classified into regions A1, A2, B, C, and D, if each region is reproduced step by step, the entire measurement object can be obtained. Specifically, for reproducing these areas, the apparatus configuration is changed based on the formation cause of each area. For example, if the light amount is automatically reduced and remeasured, the luminance saturation region A1 can be reproduced. If the projection direction is increased and re-measurement is performed, a part of the boundary region C between the contrast reduction region B and the contrast reduction region can be reproduced. If the projection angle is changed and remeasured, a part of the phase error due to specular reflection or the like can be reproduced.

It is explanatory drawing which shows the structure of the system containing the three-dimensional measuring apparatus which concerns on embodiment of this invention. It is a flowchart which shows the extraction procedure of a brightness | luminance saturated area | region. It is a figure explaining the mode of the extraction process of a brightness | luminance saturated area | region. It is a figure which shows the example of the histogram produced | generated in the extraction process of a brightness | luminance saturated area | region. It is a flowchart which shows the extraction procedure of a contrast fall area | region. It is a figure which shows the mode of the extraction process of a contrast fall area | region. It is a figure explaining the extraction method of the phase error area by specular reflection etc. It is a figure explaining the formation cause of the phase error by specular reflection. It is a figure which shows the example of a display of the process result of embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Stage 2 Lattice pattern projection mechanism 3 Image pick-up mechanism 4 Image processing apparatus 5 Measurement object 21 Light source 22 Illumination optical system 23 Lattice pattern and scanning apparatus 24 Projection optical system 31 Imaging system 32 Camera 41 Data storage part 42 Data processing part 43 Phase Value reliability determination unit 44 Data display unit

Claims (8)

3D using the phase shift method that shifts the grid pattern multiple times and irradiates the measurement object, captures the image of the irradiated grid pattern every time it shifts, and measures the 3D shape using the image In the measurement method,
Calculating a phase value and an amplitude value of each pixel of the image;
In the calculated phase value, a step of classifying and extracting a portion where the shape cannot be measured accurately;
Determining the reliability of the phase value of each pixel based on the extracted result;
Calculating the shape of the measurement object and displaying the result using the determination result; and
A three-dimensional measurement method comprising:   The portion to be classified and extracted includes at least one of a luminance saturation region, a contrast reduction region, a boundary region between contrast reduction regions, or a phase error region due to specular reflection or the like. The three-dimensional measurement method described. The extraction of the luminance saturation region is as follows:
Extracting a pixel of maximum brightness from each of the plurality of images;
Integrating the extracted pixels into a plurality of small regions;
Determining an adjacent region for each small region;
Generating a phase value histogram for each small region and its neighboring regions;
Comparing the phase value histogram of the small region and the phase value histogram of the adjacent region for each small region;
Determining whether the phase value of the small region of the compared histogram is within a predetermined measurement accuracy range;
The three-dimensional measurement method according to claim 2, further comprising: The extraction of the contrast reduction region is as follows:
Obtaining an average value and standard deviation of amplitude values for all pixels other than the luminance saturation region;
Extracting pixels whose amplitude value is smaller than the average value;
For the extracted pixels, from the average value and standard deviation of the amplitude value, determining whether or not the pixel is in the contrast reduction region,
The three-dimensional measurement method according to claim 2, further comprising:   The three-dimensional measurement method according to claim 2, wherein the extraction of the boundary region of the contrast reduction region is performed by performing an expansion process on the contrast reduction region. The extraction of the phase error region by specular reflection or the like
The pixel other than the luminance saturation region, the contrast lowering region, and the boundary region of the contrast lowering region is performed using a relationship between a phase value difference between adjacent pixels and an irradiation angle of a lattice pattern. 3D measurement method. 3D using the phase shift method that shifts the grid pattern multiple times and irradiates the measurement object, captures the image of the irradiated grid pattern every time it shifts, and measures the 3D shape using the image In the measuring device,
Calculating means for calculating a phase value and an amplitude value of each pixel of the image;
In the calculated phase value, an extraction means for classifying and extracting a portion where the shape cannot be measured accurately;
Determination means for determining the reliability of the phase value of each pixel based on the extracted result;
Using the determination result, shape calculation display means for calculating the shape of the measurement object and displaying the result;
A three-dimensional measuring apparatus comprising: The portion to be classified and extracted includes at least one of a luminance saturation region, a contrast lowering region, a boundary region of the contrast lowering region, or a phase error region due to specular reflection or the like. The three-dimensional measuring apparatus described.

Images