JP2960684B2 - Three-dimensional shape detection method and device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and an apparatus for detecting a three-dimensional shape of an arbitrary object with high accuracy from a plurality of images having different focal positions. .

[0002]

2. Description of the Related Art Heretofore, a method described in Japanese Patent Application Laid-Open No. 61-124809 has been known as a method for detecting a three-dimensional shape focusing on a focus position. In this case, the focus position is changed from the concavo-convex pattern as the object to be inspected, pattern detection is performed, and the planar and three-dimensional dimensions of the pattern are calculated from the focus position information and the pattern position information. Has become. More specifically,
By taking an LSI wafer pattern as an example, an image in which the focal plane is sequentially changed by the focus adjusting mechanism is detected, and the sharpest portion of the video signal is extracted and synthesized, thereby forming the upper edge of the step pattern. , Detection of a step pattern in focus on any of the lower step edges, detection of a three-dimensional dimension (thickness) of the pattern from the focal position when the sharpest part is extracted,
In the case of a photoresist pattern, from the video signal having the maximum contrast, it is possible to detect the upper and lower end positions at the pattern step as the maximum rate of change.

[0003] On the other hand, a dissertation “Image synthesis method using variance value” (Transactions of the Institute of Electronics and Communication Engineers, '83 / 10 Vol.
6-D No. According to 10), the variance of the brightness of each small area is obtained from the images having different focal positions, and the in-focus portions are extracted from each image and combined with each other, so that the entire image is in focus. Images are obtained.

[0004]

However, according to Japanese Patent Application Laid-Open No. 61-124809, the position of a portion having a unique contrast or a unique signal waveform characteristic, such as the upper and lower end positions at a step portion of a thin film pattern. Is specified, even if the upper and lower end positions and the thickness of the step pattern portion can be detected, the entire three-dimensional shape including the flat portion of the step pattern cannot be detected.

[0005] On the other hand, in the paper "Image synthesis method using variance value", images which are entirely in focus are obtained from images having different focal positions, but consideration is not given to obtaining a three-dimensional shape from these images. It has become something.

Therefore, no three-dimensional shape has been detected so far from an image having a different focal position, and a finer than ΔZ is obtained from an image obtained by changing the focal position by, for example, ΔZ in the height direction (Z direction). Detecting a three-dimensional shape with a height resolution has not been considered at all.

SUMMARY OF THE INVENTION An object of the present invention is to provide a method and an apparatus for detecting a three-dimensional shape capable of detecting a three-dimensional shape of an object with high accuracy from images having different focal positions.

Another object of the present invention is to provide a method and an apparatus for detecting a three-dimensional shape capable of detecting a three-dimensional shape of an object with a resolution finer than a focal position difference between images.
Even for an object having a surface having no texture, such as a diffusion surface or a mirror surface, a three-dimensional shape detection method and apparatus capable of detecting the three-dimensional shape of the object, and further, the three-dimensional shape of the object at higher speed The object of the present invention is to provide a three-dimensional shape detection method and apparatus capable of detecting a three-dimensional shape, and a three-dimensional shape detection method and apparatus capable of detecting a three-dimensional shape of an object at higher speed, particularly in relation to the first object.

[0009]

The object of the present invention is to provide a method for detecting a three-dimensional shape, wherein the relative position between the focal point of the imaging means and the sample is changed in the optical axis direction of the imaging means to image the surface of the sample. A plurality of images having different focus positions are obtained, a focus position on the surface of the sample is detected in each image of the plurality of images, and interpolation is performed between the different focus positions by interpolation from the information on the focus position on the detected surface of the sample. This is achieved by calculating the in-focus position of the surface of the sample and calculating the three-dimensional shape of the sample based on the information on the in-focus position detected by imaging and the information on the in-focus position calculated by interpolation.

Further, the above object is to provide a three-dimensional shape detecting device,
Imaging means for imaging the surface of the sample; relative position changing means for changing the relative position between the focus of the imaging means and the sample in the optical axis direction of the imaging means; and the focal point of the imaging means with the relative position changing means. From a plurality of images having different focal positions obtained by imaging the surface of the sample by the imaging means while changing the relative position with respect to the sample, the in-focus position of the surface of the sample is detected in each of the plurality of images. Focusing position detection means and, based on information on the focusing position on the surface of the sample corresponding to each of the plurality of images detected by the focusing position detection means, determine the focusing position on the surface of the sample between different focusing positions. The calculation means obtained by interpolation, the information of the focus position detected by the focus position detection means and the information of the focus position on the surface of the sample between the different focus positions obtained by interpolation by the calculation means. 3D shape It is achieved by constituting a three-dimensional shape calculating means for calculating.

[0011]

[0012]

[0013]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below with reference to FIGS.

First, a three-dimensional shape detecting apparatus according to the present invention will be described. FIG. 1 shows a schematic configuration thereof together with an object 1. In this example, the object is assumed to have a random and sharp texture, such as a sintered body of metal particles, and a focus measure of the surface is detected under normal bright-field illumination conditions. It is like that.

That is, the object 1 is illuminated by the illumination light from the lamp 6 via the illumination lens 5 and the half mirror 4, and its height is moved up and down by the vertical mechanism 3 via the sample table 2. In the meantime, the image is
The image is detected by an image detector 8 as various imaging means such as a V camera and a one-dimensional image sensor. An image is similarly detected by moving the imaging lens 7 and the entire image detecting optical system including the same up and down, but in any case, a plurality of images at different focal positions are obtained.
In this example, for simplicity of description, the height of the focal point position is Z1, Z
2, Z3 (assumed to be Z3-Z2 = Z2-Z1 = ΔZ) (constant), and in the state shown, the focal position of the image detection optical system is shown as matching Z2. It has become. FIG. 2A shows video signals V1, V2 corresponding to the heights Z1, Z2, Z3, respectively.
Although V2 and V3 are illustrated, in general, the video signal is the sharpest at the focal point portion, the signal amplitude is large, and the frequency at that portion is also large. Referring back to FIG. 1, the focusing speed detecting section 9 includes the image detector 8.
From the video signals V1, V2, and V3 from the above, a focus measure can be obtained for each point on the input screen. Here, the focus measure is defined as a numerical value representative of the degree of focus at that point. The focus measure is determined, for example, as follows.

That is, the height Zi (i = 1, 1) of the focal point position
The detected image from the object at (2,..., N) is represented by Vi (x,
y), the contrast image Ci (x, y) is obtained from the detected image Vi (x, y) by (Equation 1).

[0017]

(Equation 1)

Further, from Ci (x, y), a focus measure Fi is obtained.
(X, y) is obtained by (Equation 2).

[0019]

(Equation 2)

Here, m and m 'are predetermined constants and indicate the addition area of the local image.

As is clear from (Equation 2), the focus measure Fi (x, y) in this example is a contrast image Ci (x, y).
This corresponds to local addition of y), that is, smoothing.

FIG. 3 shows a specific configuration of the focusing measure detecting section. As shown, the detected image Vi
The video signal at (x, y) is sequentially converted to a digital amount by an A / D conversion circuit 12 and sequentially passed through shift registers (having the capacity of a detected image-scanning line) 13 and 14 to be shifted. From the registers 13 and 14, video signals one scanning line before and two scanning lines before can be obtained together. Therefore, these image signals and the video signal from the A / D conversion circuit 12 are converted into a contrast extraction circuit 15.
, The contrast extraction circuit 15 obtains a contrast image Ci (x, y) in accordance with equation (1) based on each of the video signals. The contrast image Ci thus obtained
(X, y) is subsequently passed through shift registers (detected image-capacity for scanning lines) 16 and 17 in the same manner as the video signal from the A / D conversion circuit 12, and
17 together with the one from each of them, and is input to the smoothing circuit 18, where the focal amount Fi is calculated according to (Equation 2).
(X, y) is required. Since the input to the smoothing circuit 18 is as shown in the figure, the constant m 'described above is 1 in this example, and the value of m is set to 1 accordingly. Since the above processing can be performed in a pipeline manner, the processing can be performed in real time.

When the contrast image Ci (x, y) is calculated, its value is set to a predetermined threshold value T1.
In the following cases, Ci (x, y) may be processed as follows.

[0024]

(Equation 3)

This is a process for removing noise in a defocused portion. In this example, the contrast image C
Although i (x, y) is expressed as the sum of absolute values of the second derivative with respect to 3 (the number of pixels in the x direction) × 3 (the number of pixels in the y direction), the present invention is not limited to this. It may be a second derivative within a larger local image range such as × 5 pixels. In addition to the second derivative, various derivative operators such as Laplacian, Gaussian Laplacian, first derivative, and 2 × 2 pixel Roberts gradient can be applied for calculating Ci (x, y). In some cases, as shown in (Equation 4), Ci is calculated from the fractional value of the concentration.
(X, y) may be obtained.

[0026]

(Equation 4)

Here, Vμ (x, y) is an average density given by the following equation.

[0028]

(Equation 5)

Further, in (Equation 2), local addition for smoothing is performed, but this may be average value processing, intermediate value processing, or other noise processing. However, if the detected image has an extremely uniform texture and an extremely small noise component, the processing can be omitted.

The focus measure Fi is obtained as described above. FIG. 2B shows the video signal V shown in FIG.
Focusing measures F1, F2, F associated with 1, V2, V3 respectively
3 is one-dimensionally shown. As a result, the focus measure is detected as being large at the video signal position corresponding to the focused portion.

Next, a description will be given of the detection of the focus measure maximum value position. Now, as shown in FIG. 4A, Fi (x, y) (i = 1, 2,..., N) for each point on the object ) Is determined, if the focus position at which the focus measure is maximized is determined from these Fi (x, y), the three-dimensional shape Z (x, y) of the object is finally determined. Become.

[0032] than In detail, FIG. 4 (b) coordinate point with a target Butsujo shown in FIG. 4 (a) (x, y ) but is a plot of the values of F _i for, Focusing measure F
Is determined to be the maximum value at the coordinate point (x, y) if the z-coordinate Zp is determined when the maximum value Fp is obtained. FIG. 1 (c) shows the distribution of F _i in the x-direction an arbitrary coordinate position X in FIG. 1 (b), the to this distribution the maximum value Fp of the focus measure F than to determine the z-coordinate Zp to this In this case, the z coordinate Zp is obtained as the height at the arbitrary coordinate position X, and as a result, it can be seen that the three-dimensional shape of the object is detected.

As described above, it is necessary to find the z coordinate Zp with respect to the maximum value Fp of the focusing measure F.
Is obtained, for example, as follows.

That is, in this method, the data distribution of F _i (x, y) is assumed to be a Gaussian distribution as shown in (Equation 6).

[0035]

(Equation 6)

Here, σ indicates the variance of F.

In order to obtain the z coordinate Zp based on this (Equation 6), F _i (x, y) for the coordinate point (x, y)
(I = 1, 2,..., N), the maximum value Fm and its z coordinate Zm, the previous Fm _-1 (x, y) and its z coordinate Zm _-1 , and the subsequent _{Fm- 1} (x, y) and its z coordinate Z
_{m + 1} is now required. As a result, (Fm-
_{1, Zm-1), (} Fm, Zm), (F m + 1, Z m + 1) is F
_i is obtained from (x, y), but by substituting these into F and z in (Equation 6), F
p, σ, and Zp are easily determined as follows.

[0038]

(Equation 7)

Is as follows.

[0040]

(Equation 8)

[0041]

(Equation 9)

Therefore, the coordinate point (x, y) on the object
By performing the calculation according to (Equation 7) for all of them, the three-dimensional shape Z (x, y) of the target object can be obtained. (Equation 8) and (Equation 9) are not necessarily required to obtain ZP, but the value of FP is larger than a preset threshold Tz, and the value of σ is set in advance. Only when the value is smaller than the threshold value T3, the value obtained in (Equation 7) is used as it is as ZP. In other cases, a special value, for example, 0 is used as ZP.
In any of the above cases, a background portion or the like out of focus can be removed.

The above-described focus measure maximum value position is detected by the focus measure maximum value position detection unit 10, but FIG. 5 shows a specific configuration thereof (however, judgment processing by threshold values T2 and T3 is not included. ). In this case, three consecutive F _i (x, y), F _i-1 (x, y), F _i-2 (x, y
frame memories 19, 2 for storing y) as updatable
0, 21 and F are stored in these frame memories 19, 20, 21.
a selection circuit 22 for selectively writing _i (x, y);
F from any two of the frame memories 19, 20, 21
_i-2 (x, y) and F _i-1 (x, y), and F _i (x, y) from the focus measure detection unit 9 as input, and F _i-1 (x, y)
y) the maximum value extraction circuit determines whether or not the maximum is 23, any coordinate point (x, if F _i-1 (x, y) were maximum in y), F _i-1 (x, y) is written as F _m (x, y), in which case Δz × (i−1) (=
Z _m ) is written as Z _m , a frame memory 24 ′ in which F _i−2 (x, y) is written as F _{m −1} (x, y), and in that case, Fi (x, y) , The frame memory 26 which writes the data as F _{m + 1} (x, y)
And F _m-1 (x, y), F _m (x, y), F _{m + 1} (x, y
y), Z _m , Z _{m + 1} = Zm−Δz), Z _{m + 1} (= Zm + Δ
The maximum value position detection circuit 27 for obtaining Zp (x, y) based on (Equation 7) based on (z).

The operation of the focus-measurement-maximum-value detecting unit 10 will be described. In the selection circuit 22, Fi (x, y) sent from the focus-measurement-measurement detecting unit 9 in real time is the oldest one.
A new Fi (x, y) is written in any of the frame memories 19, 20, and 21 in which F _i-2 (x, y) before the frame (detection screen) is stored. The remaining two of the frame memories 19, 20, and 21 have F _i-1 (x, y) and F _i-2 (x, y) one frame before and two frames before the Fi (x, y). ) Is always stored in the maximum value extracting circuit 23 every time a detected image is obtained.
F _i−1 (x, y), F _i−2 (x, y) from any two of 20 and 21
It is determined whether or not the relationship represented by (Equation 10) is established at each coordinate point (x, y) based on y) and F _i (x, y) from the focus measure detection unit 9. It is.

[0045]

(Equation 10)

If (Equation 10) is satisfied, F
_{Assuming that i-1} (x, y) is the maximum value, next, this is the value of F-1 which has been stored in the frame memory 24 so far.
Whether it is greater than _m (x, y) is determined by (Equation 11).

[0047]

[Equation 11]

If (Equation 11) does not hold, no processing is performed.
_{Assuming that i-1} (x, y) is the maximum value obtained so far, an update process for setting this as a new _Fm (x, y) is performed. In this update processing, F _i-1 (x,
y) is written to the frame memory 24 as F _m (x, y). In the frame 24 ′, Zm (= Δz × (i−1)) based on the value of i at that point is stored in the frame memory 24 ′. 25 has F _i-2 (x, y) at that point in F
_In this case, F _i (x, y) at that point is written as F _{m + 1} (x, y) in the frame memory 26 as _m−1 (x, y).

Therefore, if the above processing is executed from i = 1 to n, the frame memories 24, 25 and 26 store the _Fm (x) related to the maximum focus measurement value for all coordinate points (x, y). , Y), F _m-1 (x, y), F _{m + 1} (x, y)
However, Zm is stored in the frame memory 24 '. Therefore, if the calculation shown in (Expression 7) is performed by the maximum value position detection circuit 27 based on these values, all the coordinate points (x, y) are obtained.
Zp (x, y) is required.

In the above processing, a Gaussian distribution is obtained based on three data points in the main part of the Gaussian distribution.
From this, Zp (x, y) is obtained, but the Gaussian distribution approximation is also obtained by the least squares approximation based on all the data points shown in FIG. 4B. Further, the calculation of Fp and Zp can be considered other than the case of using Gaussian distribution approximation. For example, as shown in FIG. 6, F _m-1 (x, y), F _{m + 1} (x, y)
Are connected to F _m (x, y) by a straight line, and the inclination angle of this straight line is α, and F _m−1 (x, y), F _m
If a straight line having an inclination angle α is drawn so as to pass through the larger one of _{m + 1} (x, y), (Fp, Zp) may be obtained as the intersection of these straight lines. Further FIG.
As shown in the above, if the intersections Z 'and Z "with the appropriately set threshold value T4 are obtained by approximating the data point sequence with a polygonal line, Zp is obtained as follows. .

[0051]

(Equation 12)

In some cases, Zp may be obtained as the position of the center of gravity of the data point sequence. By the way, Zp (x, y) is calculated by Gaussian distribution approximation (Equation 7).
Even if Zp is obtained, it is also possible to directly calculate Zp instead of performing calculation each time. Eventually place, Zp is the fact that the _{F m, F m-1,} F m + 1 of the function, as shown in FIG. 8, the coordinates _{(F m, F m-1} , F m + 1)
３ = Zp−Zm for all combinations of F _m , F _m−1 , and F _{m + 1} is previously obtained and stored in a three-dimensional memory consisting of: It is required. At that time, in _{F m, F m-1,} F m + 1 , for example, F _m, also, if to be normalized Z _m, the _{Z m + 1, Z m-} 1 in Delta] z, with a small memory capacity It is done. Further, in the above description, the focus measurement maximum value position detection focuses on obtaining Zp with a value smaller than the image detection pitch Δz, but if Zp is obtained in Δz pitch units, From (Equation 13), Zp is simply obtained.

[0053]

(Equation 13)

That is, Fi for all the coordinate points (x, y)
I at the time when (x, y) is maximized is determined, and this is Zp
(X, y).

As described above, Zp (x, y) is obtained based on the detection of the maximum value of the focus measure. Based on this, the three-dimensional shape display section 11 displays the three-dimensional shape of the object 1. I have. In the three-dimensional shape display unit 11, all the Zp (x, y) from the focus measure maximum value position detection unit 10 are temporarily stored in the memory and then displayed as a three-dimensional shape on the display tube. For display, a printer or various display means can be used in addition to the display tube. In addition to displaying a three-dimensional shape, Zp
Since (x, y) may be used for automatic inspection, shape collection, and the like, in such a case, the three-dimensional shape display unit 11 is a judgment circuit for automatic inspection or a large size for shape collection. Obviously, it may be a scale memory or a computer.

Finally, a description will be given of a modified embodiment or a more preferable embodiment of the present invention. If the texture on the surface of the object is unclear, or the object has a diffusion surface or a mirror surface having no texture as its surface. For objects, a lattice pattern or multi-spot light can be used to form a texture by illumination on the surface of the diffusion surface that does not have a texture, or to make a large number of light reflected from the mirror surface into a light source image. In this way, the three-dimensional shape can be detected even on a diffusion surface or a mirror surface. More specifically, FIG.
Then, a ride guide from the surroundings (the light source unit is not shown) 28
Forms a multi-spot light source, and an object (not shown) is illuminated by the light guides 28. FIG. 9B shows an example in which illumination is performed as a lattice light source, and the parabolic concave mirror 2 for dark field illumination is used.
A grid-like pattern is formed on the mirror surface by the illuminating light beam 30 toward the mirror 9. Further, in FIG. 9C, bright field illumination is used, but a filter 31 is inserted on the illumination-side real image plane. Filter 31
By inserting a multi-spot (many circular apertures) filter, a filter having a lattice pattern or a random texture, etc. so that they are projected with the best contrast on the focal plane, the multi-spot The three-dimensional shape of the target object can be detected by detecting an image, a lattice pattern, and a random texture by image detection.

Next, in this example, it is assumed that the vertical movement of the object is performed in a stepwise manner, and that image detection is performed at each stop position.
For example, while continuously changing the relative positional relationship between the target object and the image detection optical system at a constant speed, the image of the target object is averaged periodically or aperiodically over a certain period of time. May be detected. In the case of a storage-type sensor, such as when a TV camera is used as an image detector, the amount of charge accumulated or discharged in a light receiving unit within a certain time (accumulation time) corresponds to the amount of light received by the light receiving unit. When the image of the object is detected by the accumulation type sensor while continuously changing the relative positional relationship between the object and the image detection optical system, an average value corresponding to the amount of change within the accumulation time is used. Since no image is obtained, no problem occurs. Therefore, when performing image detection in this manner, a plurality of image detections having different focal positions are detected at higher speed, and the three-dimensional shape of the object is also obtained at higher speed.

[0058]

As described above, according to the present invention,
The three-dimensional shape of the arbitrary object is surely determined.
Further, the three-dimensional shape of the target object is determined with higher precision than the difference in the focal position, and the three-dimensional shape is detected even when the number of detected images is small. Furthermore, even if the surface is a mirror surface or a diffusion surface, a three-dimensional shape of the object is required. Furthermore, the three-dimensional shape of the object can be obtained at a higher speed, and the maximum value position can be obtained at a high speed by detecting the maximum value of the focus measure at a high resolution by interpolation.

[Brief description of the drawings]

FIG. 1 is a schematic sectional view showing a schematic configuration of a three-dimensional shape detection device according to the present invention together with an object.

FIG. 2 is a diagram for explaining an outline of processing contents according to the present invention.

FIG. 3 is a block diagram showing a specific configuration of a focus measure detection unit as a component in the apparatus.

FIG. 4 is an explanatory diagram for obtaining a focus measure maximum value position interpolated between images.

FIG. 5 is a block diagram showing a specific configuration of a focus measure maximum value position detection unit as a component in the apparatus.

FIG. 6 is a view for explaining a case where a focus measure maximum value position interpolated between images is obtained by another method.

FIG. 7 is a view for explaining a case where a focus measure maximum value position interpolated between images is obtained by another method.

FIG. 8 is a diagram for explaining matrix data that is referred to when a focus measure maximum value position interpolated between images is obtained.

FIG. 9 is a schematic sectional view for explaining various illumination systems according to the present invention.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Object, 3 ... Vertical mechanism, 4 ... Half mirror, 5 ... Illumination lens, 6 ... Lamp, 7 ... Imaging lens, 8 ... Image detector, 9 ... Focusing measure detection part, 10
... Focusing-measurement-maximum-value position detecting unit, 11: three-dimensional shape display unit, 28: light guide, 29: parabolic concave mirror for dark-field illumination, 31: filter.

──────────────────────────────────────────────────続 き Continued on the front page (58) Field surveyed (Int.Cl. ⁶ , DB name) G01B 11/00-11/24

Claims (2)

(57) [Claims] 1. A focal point of said image pickup means in an optical axis direction of said image pickup means.
And changing the relative position of the sample and the surface of the sample
Obtain a plurality of images with different focal positions by imaging,
The surface of the sample in each of the plurality of images
The in-focus position of the sample is detected, and the detected surface of the sample is focused.
Between the different focus positions by interpolation from the focus position information
Calculating the in-focus position of the surface of the sample,
The information of the detected focal point position and the focal point calculated by the interpolation
A three-dimensional shape detection method, wherein a three-dimensional shape of the sample is obtained based on information on a focal position . 2. An imaging means for imaging a surface of a sample, and the imaging means
Relative to the sample and the focal point of the imaging means in the optical axis direction of the means
Relative position changing means for changing the relative position;
The relative position between the focus of the imaging means and the sample
Imaging the surface of the sample with the imaging means while changing
Of a plurality of images having different focal positions
In each image, the focal position on the surface of the sample
In-focus position detecting means for detecting, and the in-focus position detecting means
The test corresponding to each of the plurality of images detected in
The different focal positions based on the information of the focal position on the surface of the material.
The focal position on the surface of the sample between the positions
Calculating means, and the in-focus state detected by the in-focus position detecting means.
The point position information and the difference obtained by interpolation by the arithmetic means
Information on the focal position of the surface of the sample between different focal positions
Three-dimensional shape calculation for calculating the three-dimensional shape of the sample based on
And a means for detecting a three-dimensional shape.