JP2003202214A - Shape measuring device and shape measuring method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To avoid the surface shape dependency of object's reflected light and shorten the measurement time with a simple configuration or process. <P>SOLUTION: The device irradiates a light 102 sequentially from places with the same zenithal angle and different azimuthal angles with respect to an object 107 and chooses three pictures from at least 3 pictures photographed by each light 102. Assume that the luminance values of the three pictures under identical coordinate are d<SB>1</SB>, d<SB>2</SB>, and d<SB>3</SB>respectively, the inverse matrix of the light matrix M arrayed by the vectors of the irradiation directions is M<SP>-1</SP>, and the element (i, j) of the M<SP>-1</SP>is M<SP>-1</SP><SB>ij</SB>, the azimuthal angle of the object's surface gradient θcorresponding to the pixel position is calculated by formation θ=tan<SP>-1</SP>(M<SP>-1</SP><SB>21</SB>d<SB>1</SB>+M<SP>-1</SP><SB>22</SB>d<SB>2</SB>+M<SP>-1</SP><SB>23</SB>d<SB>3</SB>)/(M<SP>-1</SP><SB>11</SB>d<SB>1</SB>+M<SP>-1</SP><SB>12</SB>d<SB>2</SB>+M<SP>-1</SP><SB>13</SB>d<SB>3</SB>). <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a shape measuring device and a shape measuring method used for shape inspection of a mold part, a soldering part, etc. of an electronic component mounted on a board.

[0002]

2. Description of the Related Art FIG. 10 and FIG.
It is a figure which shows the conventional shape measuring method shown by 304520 gazette. In FIG. 10, 2 is a chip part which is a target object, 10 is a CCD camera, and 11 to 15 are illumination groups having different zenith angles.

Next, the operation will be described. In this conventional technique, the illuminations 11 to 15 are sequentially turned on and the CCD camera 1
An image is picked up by 0. There is a method of calculating the zenith angle of the surface of the object by arranging the luminances of the same coordinates of the obtained images as shown in FIG. 11 and applying a Gaussian function to estimate the angle at which the luminance peaks.

[0004]

However, the above-mentioned conventional technique is basically a measuring method using the peak of the brightness of the reflected light of the object, that is, the specular reflected light. It is necessary to increase the data used for Gaussian function fitting by preparing a large number of different illuminations of. Moreover, in order to increase the number of data, the number of illuminations must be increased, which causes an increase in measurement time.

Further, in general, the reflected light of an object has not only a specular reflection component but also a diffusion component. Since an object such as solder has a relatively large specular reflection component, this conventional technique can be applied, but it is not appropriate to apply a Gaussian function to an object having a large diffusion component.

The present invention has been made in view of the above, and it is possible to suppress an increase in measurement time without being affected by the surface shape dependence of the reflected light of the target object while having a simple structure or process. An object is to obtain a shape measuring device and method.

[0007]

[Means for Solving the Problems]
Therefore, the shape measuring device according to the present invention
Illumination is sequentially emitted from multiple different directions to display the image of the target object.
Images are sequentially captured for each illumination, and multiple images are captured.
From the luminance of the pixel at the same coordinates in the
In a shape measuring device that measures the surface gradient of an elephant object,
From at least three positions with the same apex angle but different azimuth angles
Illumination means for sequentially illuminating the target object with illumination,
Target object illuminated by illumination from at least three positions
Image pickup means for sequentially picking up images of
Selecting means for selecting three images from the images;
The brightness value of each of the three images at the same coordinates
d₁, D₂, D₃And arranged the vectors of the illumination irradiation direction.
Let M be the illumination matrix and M be the inverse matrix of M.^-1And the inverse matrix M^-1
The (i, j) component of M^-1 _ijIs equivalent to each pixel position
The azimuth angle θ of the surface gradient of the target object θ = tan^-1((M^-1 _{twenty one}d₁+ M^-1 _{twenty two}d₂+ M^-1 _{twenty three}d₃) / (M^-1 ₁₁
d₁+ M^-1 ₁₂d₂+ M^-1 ₁₃d₃)) And a calculating means for calculating with the calculation formula
It

According to the present invention, illumination is sequentially irradiated from places having the same zenith angle and different azimuth angles, three images are selected from at least three images captured by each irradiation, and the selected images are selected. Since the azimuth angle of the surface gradient of the target object is calculated by linear processing using the brightness values at the same coordinates of the three images and the illumination matrix in which the vectors of the illumination irradiation direction are arranged, it is possible to calculate the illumination with different illumination angles. Without preparing a large number, the influence of the dark current component (DC component noise) of the image, the gain related to the brightness of the image, the change of the brightness due to the change of the diaphragm of the image pickup system, etc. It can be automatically canceled, and the azimuth angle of the surface gradient of the target object can be accurately measured.

The shape measuring apparatus according to the next invention is the shape measuring apparatus according to the above invention, when the three illuminations have different illuminances.
The calculating means is characterized in that the respective luminance values at the same coordinates of the three images are normalized by the illuminance ratios of the three illuminations to obtain the luminance values d ₁ , d ₂ and d ₃ .

According to the present invention, when the illuminances of the three lights are different, the respective brightness values at the same coordinates of the three images are normalized by the illuminance ratio of the three lights to obtain the brightness values. Therefore, even if the illuminances of the three lights are different, the azimuth angle of the surface gradient of the target object can be accurately calculated.

In the shape measuring apparatus according to the next invention, in the above invention, the selecting means selects three images from a plurality of images obtained by irradiation with four or more illuminations having the same zenith angle but different azimuth angles. A plurality of combinations of images are selected, the calculating means removes an abnormal value from a plurality of azimuth angles calculated for each combination of three images, and then performs an average calculation to calculate the azimuth of the surface gradient of the target object. The feature is that the angle is calculated.

According to the present invention, a plurality of combinations of three images are selected from a plurality of images obtained by irradiation with four or more lights having the same zenith angle and different azimuth angles, and three images are combined. After removing the abnormal values from the multiple azimuth angles calculated for each combination, the azimuth angle of the surface gradient of the target object is calculated by performing an average calculation, so stability that is not affected by erroneous measurement due to specular reflection light, etc. You can make measurements.

In the shape measuring apparatus according to the next invention, in the above invention, the illumination means includes at least three first height positions having the same zenith angle and different azimuth angles and the first height position. At least illumination can be applied to the target object from at least three second height positions having different zenith angles and different azimuth angles, and the selection means can emit the illumination of the first height position. From the plurality of obtained images and the plurality of images obtained by irradiation with the illumination at the second height position, a plurality of combinations of three images are respectively selected, and the calculation means combines the three images. The azimuth angle of the surface gradient of the target object is calculated by removing an abnormal value from the plurality of azimuth angles calculated for each of them and then performing an average calculation.

According to the present invention, at least three first height positions having the same zenith angle and different azimuth angles, and at least three zenith angles different from the first height position and having different azimuth angles. Illumination is sequentially applied to the target object from the second height position, abnormal values are removed from the obtained azimuth angles, and the remaining ones are averaged to calculate the azimuth angle. It is less affected by the components, and the width of measurable target shapes can be expanded.

A shape measuring apparatus according to the next invention is characterized in that, in the above invention, a surface light source is used for illumination.

According to the present invention, since the surface light source is used for illumination, the luminance value of the too strong specular reflection component can be blunted, and a relatively stable image can be obtained. When designing or adjusting the lighting, it is possible to easily set the lighting angle.

A shape measuring apparatus according to the next invention is characterized in that, in the above-mentioned invention, illumination is sequentially irradiated at intervals of an azimuth angle of 120 degrees.

According to the present invention, since the irradiation is performed at an azimuth angle of 120 degrees, the difference in brightness of the reflected light of the target object can be made as large as possible, and the measurement accuracy of the azimuth angle of the surface gradient of the target object can be increased. Can be increased.

The shape measuring method according to the next invention is applicable to
Illuminate objects sequentially from multiple different directions,
Images of the elephant object are sequentially captured for each irradiation, and multiple images are captured.
The brightness of the pixel at the same coordinates in the image and the irradiation direction of the illumination
Shape measurement method for measuring the surface gradient of a target object from
And at least three positions with the same zenith angle but different azimuth angles.
Step of sequentially illuminating the target object from the installation
And from the images captured by each of the above illuminations
The second step of selecting three images and the selected three images
The respective brightness values at the same coordinates of the image of d₁,
d₂, D₃, And the lighting row in which the vectors of the lighting irradiation direction are arranged.
Let M be the column and M be the inverse matrix of M.^-1And the inverse matrix M^-1Of (i,
j) the component is M^-1 _ijCorresponds to the pixel position
Azimuth θ of the surface gradient of the target object, θ = tan^-1((M^-1 _{twenty one}d₁+ M^-1 _{twenty two}d₂+ M^-1 _{twenty three}d₃) / (M^-1 ₁₁
d₁+ M^-1 ₁₂d₂+ M^-1 ₁₃d₃)) And a third step of calculating with a calculation formula of
It

According to the present invention, illumination is sequentially irradiated from locations having the same zenith angle and different azimuth angles, three images are selected from at least three images captured by each irradiation, and the selected images are selected. Since the azimuth angle of the surface gradient of the target object is calculated by linear processing using the brightness values at the same coordinates of the three images and the illumination matrix in which the vectors of the illumination irradiation direction are arranged, it is possible to calculate the illumination with different illumination angles. Without preparing a large number, the influence of the dark current component (DC component noise) of the image, the gain related to the brightness of the image, the change of the brightness due to the change of the diaphragm of the image pickup system, etc. It can be automatically canceled, and the azimuth angle of the surface gradient of the target object can be accurately measured.

A shape measuring method according to the next invention is the above-mentioned invention, wherein when the illuminances of the three illuminations are different,
The third step is characterized in that the respective brightness values at the same coordinates of the three images are normalized by the illuminance ratios of the three illuminations to obtain the brightness values d ₁ , d ₂ and d ₃ .

According to the present invention, when the illuminances of the three lights are different, the respective brightness values at the same coordinates of the three images are normalized by the illuminance ratios of the three lights to obtain the brightness values. Therefore, even if the illuminances of the three lights are different, the azimuth angle of the surface gradient of the target object can be accurately calculated.

The shape measuring method according to the next invention is, in the above-mentioned invention, three images from a plurality of images obtained by irradiation with four or more illuminations having the same zenith angle and different azimuth angles in the second step. , A plurality of image combinations are selected, and in the third step, an abnormal value is removed from the plurality of azimuth angles calculated for each of the three image combinations, and then an average calculation is performed to obtain the surface gradient of the target object. The azimuth angle is calculated.

According to the present invention, a plurality of combinations of three images are selected from a plurality of images obtained by irradiation with four or more illuminations having the same zenith angle and different azimuth angles, and the three images are combined. After removing the abnormal values from the multiple azimuth angles calculated for each combination, the azimuth angle of the surface gradient of the target object is calculated by performing an average calculation, so stability that is not affected by erroneous measurement due to specular reflection light, etc. You can make measurements.

In the shape measuring method according to the next invention, in the above invention, in the first step, at least three first height positions having the same zenith angle and different azimuth angles and the first height position are provided. At least three second height positions having different zenith angles and different azimuth angles sequentially irradiate the target object with illumination, and in the second step, irradiation with illumination at the first height position is performed. From the plurality of images obtained by the above and the plurality of images obtained by the irradiation of the illumination at the second height position, a plurality of combinations of three images are selected respectively, and in the third step, three images are selected. The azimuth of the surface gradient of the target object is calculated by removing the abnormal value from the plurality of azimuths calculated for each combination, and then performing the average calculation.

According to the present invention, at least three first height positions having the same zenith angle and different azimuth angles, and at least three zenith angles different from the first height position and different azimuth angles are provided. Illumination is sequentially applied to the target object from the second height position, abnormal values are removed from the obtained azimuth angles, and the remaining ones are averaged to calculate the azimuth angle. It is less affected by the components, and the width of measurable target shapes can be expanded.

The shape measuring method according to the next invention is characterized in that, in the above-mentioned invention, a surface light source is used for illumination in the first step.

According to the present invention, since the surface light source is used for illumination, the luminance value of the too strong specular reflection component can be blunted, and a relatively stable image can be obtained. When designing or adjusting the lighting, it is possible to easily set the lighting angle.

The shape measuring method according to the next invention is characterized in that, in the above-mentioned invention, the illumination is irradiated at an azimuth angle of 120 degrees.

According to the present invention, since the irradiation is performed at an azimuth angle of 120 degrees, the difference in brightness of the reflected light of the target object can be made as large as possible, and the measurement accuracy of the azimuth angle of the surface gradient of the target object can be increased. Can be increased.

[0031]

BEST MODE FOR CARRYING OUT THE INVENTION Preferred embodiments of the shape measuring method according to the present invention will be described in detail below with reference to the accompanying drawings.

Embodiment 1. 1 is an explanatory view of a first embodiment of a shape measuring apparatus according to the present invention. In the figure, 101 is an imaging means, 102 is an illumination, 103 is an illumination moving means for moving the illumination 102 to three different azimuth positions, and 104 is an illumination 102.
Numeral 105 denotes an image processing unit, 106 denotes an azimuth measurement result, and 107 denotes a target object. In the case of FIG. 1, one illumination 102 is moved to three positions to illuminate from the positions of the same zenith angle and three different azimuth angles, but the illumination is respectively applied to three different positions. By fixedly arranging, illumination from three different azimuth positions at the same zenith angle may be obtained.

Next, the outline of the measurement by this shape measuring apparatus will be briefly described with reference to FIG. First, the illumination means 102 is turned on by the control means 104, and the target object 10
Irradiate 7. At this time, the image of the target object is picked up by the image pickup means 101. Next, the illumination 102 is moved to the positions of the same zenith angle and different azimuth angles, and the illumination is similarly emitted to capture the second image of the target object 107. Next, the illumination 102 is moved to the position of another azimuth angle with the same zenith angle, and the illumination is similarly emitted to the target object 107.
The third image of is also acquired. The image processing means 10
5, the azimuth of the surface gradient of the target object 107 is calculated from the brightness of the pixels of the same coordinates in the obtained three images and the irradiation direction at each irradiation position of the illumination 102, and the calculated azimuth is calculated as an image. The measurement result 106 is output together with the coordinate values of

Next, the principle of measurement by this shape measuring apparatus will be described in detail with reference to the drawings, formulas and the like. In addition, in the above, when the installation position of the illumination 102 is moved,
Although two realization means for the case where 02 is installed at a plurality of positions in advance are shown, the principle of measurement is the same for both, and the latter case will be described below from the viewpoint of easy understanding of the explanation. An example will be described.

First, FIG. 2 is an explanatory diagram showing the zenith angle and azimuth angle for each illumination. In the figure, the vector r
Is the angle formed by the vector r and the z axis, and the azimuth angle θ about the vector r is the angle formed by the projection vector of the vector r on the xy plane and the x axis.

FIG. 3 shows three illuminations 102 arranged at the same zenith angle ψ and different azimuth angles as the coordinate system of the imaging surface.
It is explanatory drawing which shows the positional relationship with a, 102b, and 102c. In the figure, the target object 107 is set as the origin, and the coordinate system is set so that the imaging surface is orthogonal to the z axis.

The symbols used in the figure will be described. If the surface gradient of the target object is represented by the surface unit normal vector n, then n = (p, q, r). Since n is a unit vector, the relationship of p ² + q ² + r ² = 1 holds.
At this time, if the azimuth angle of the surface gradient of the target object is represented by θ, then θ = tan ⁻¹ (q / p).

Further, regarding the irradiation unit vector and the illumination matrix,
And explain. Illumination 102a shown in FIG.
The unit vector of the direction is called the irradiation unit vector.
M ₁It is represented by. The direction of the vector is
Direction is opposite to the direction in which light is emitted, that is, from the origin
Take in the direction toward the lighting. At this time, the irradiation unit vector
m₁Is m₁= (M₁₁, M₁₂, M₁₃) Can be expressed as Similarly, in the illumination 102b and the illumination 102c,
However, each irradiation unit vector is m₂, M₃Represented by
When, m₂= (M_{twenty one}, M_{twenty two}, M_{twenty three}) m₃= (M₃₁, M₃₂, M₃₃) Becomes This irradiation unit vector m₁, M₂, M₃Using
Illumination matrix M is defined by the following 3 × 3 matrix
It

[0039]

[Equation 1]

Next, the measurement flow of the present invention will be explained.
Reveal Fig. 4 shows the calculation of azimuth angle θ when the number of illuminations is three.
It is a flow chart which shows a dispatch procedure. First, the control hand in Fig. 1.
The step 104 allows one of the illuminations of FIG.
Select and irradiate (step S401). Next is the imaging hand
Target illuminated by step 101 in step S401
An image of the object is captured (step S402). Next picture
It is determined whether or not three types of images have been captured (step S40
3) If you are not capturing an image, select another illumination and
Repeat the body imaging. Three different types are obtained by this imaging
When the azimuth image can be captured, the image processing means 105 causes
At each coordinate of the image (that is, each coordinate of the target object)
And the following formula θ = tan^-1((M^-1 _{twenty one}d₁+ M^-1 _{twenty two}d₂+ M^-1 _{twenty three}d₃) / (M^-1 ₁₁
d₁+ M^-1 ₁₂d₂+ M^-1 ₁₃d₃)) Then, the azimuth angle θ is calculated (step S404). this
Set the azimuth angle θ for each coordinate of the image (each coordinate of the target object).
The calculation result is output (step S405).
The procedure for deriving this formula will be described in detail later.

Next, the three lights (102a, 102a,
A procedure for calculating a plurality of azimuth angles θ for one coordinate of an image by using a combination with another illumination different from (102b, 102c) will be described. FIG. 5 shows three illuminations 102 arranged at the same zenith angle ψ shown in FIG. 3 but different azimuth angles.
a, 102b and 102c have the same zenith angle ψ
FIG. 6 is an explanatory diagram of a case where the illumination 102d arranged at a different azimuth angle is added. FIG. 6 is a flowchart showing a calculation procedure in the case where four or more lights are used and a plurality of azimuth angles θ are calculated for one coordinate of an image by combining a plurality of lights. When there are four illuminations, there are four combinations of three illuminations, but even if the number of illuminations increases, the number of combinations only increases and the measurement flow is the same. A measurement flow for calculating a plurality of azimuth angles θ will be described below by taking the case of four illuminations as an example. In FIG. 6, step S40
The procedure from the selection of the illumination in 1 to the calculation of the azimuth angle θ in step S404 is the same as in the case of FIG.

When the calculation process of one azimuth angle θ by the three illuminations is completed in step S404, next,
It is determined whether to capture an image using a combination of different illuminations (step S501). In this case, since there are four illuminations, specifically, the illumination 102d is selected in step S401 and an image is captured (step S40).
2). In this case, the combination of illuminations is (102a, 102
b, 102c), (102a, 102b, 102d),
(102a, 102c, 102d) (102b, 102
c, 102d), four combinations of azimuth angles can be calculated if all combinations are used (step S404). 4 calculated in step S404
For the azimuth θ of the street, remove outliers in it,
The remaining ones are averaged to calculate the azimuth angle θ (step S502). Do this for all coordinates. As a method of removing the abnormal value in step S502, the median value is calculated for the four measured azimuth angles, and if there is a value far from the median value, it is removed. Although an abnormal value is likely to occur when the luminance of the target object becomes extremely high due to the influence of the regular reflection component, the influence of the regular reflection component can be removed by doing so. Finally, the average-processed azimuth angle θ obtained is output for each coordinate of the image (each coordinate of the target object) (step S405),
The process ends. The above measurement flow is mainly executed by the image processing unit 105.

Next, the principle of calculating the azimuth angle will be described in detail. Here, it is assumed that the illuminances of the respective illuminations on the target object are the same so that the description will not be complicated. First, as described above, the reflected light from the target object is decomposed into the specular reflection component and the diffusion component. Regarding the diffuse component, it can be assumed that Lambertian reflection is performed, and in that case, the luminance d of the diffuse component is expressed by d = c (m · n) (Equation 1). In (Equation 1), m is an irradiation unit vector, n is a normal vector of the surface of the target object, and c = c (x, y) is a coefficient including the reflectance of the surface of the target object and the gain of the camera (scalar value ). The fact that c is a function of x and y indicates that the coefficient may be different at each point of the target object due to the difference in the surface state. (•) indicates the dot product of the vector. Further, (Equation 1) is independently established at each point of the target object.

Here, assuming that the illumination is composed of three illuminations having the same zenith angle and different azimuth angles shown in FIG. 3, the brightness d ₁ obtained by irradiating these illuminations. , D ₂ , d ₃ are collectively vector d = (d _1, d _2,
As d ₃ ), if _three (Equation 1) are connected, d = cMn. If the position of the illumination is set so that M becomes a regular matrix, then n = c ^-1 M ^-1 d. M ^-1 denotes an inverse matrix of M, M ^-1 if (i, j) of M ^-1 components with M ^-1 _ij is expressed by the following equation.

[0045]

[Equation 2]

Here, since n = (p, q, r), the object
The azimuth angle θ of the body surface gradient is     θ = tan^-1(q / p)       = Tan^-1((M^-1 _{twenty one}d₁+ M^-1 _{twenty two}d₂+ M^-1 _{twenty three}d₃) / (M^-1 ₁₁d₁+ M^-1 ₁₂d₂+ M^-1 ₁₃ d₃))                                               (Formula 2) Becomes This is true at each point of the target object, so lighting 1
Image obtained from 02a (d₁Regarded as a collection of data
Image) obtained from the illumination 102b (d)₂data from
Image that is obtained from the illumination 102c.
(D₃It can be regarded as a collection of data of
Inverse matrix M of the illumination matrix M^-1(Equation 2) is used because
The surface gradient at each point (each point on the image) of the target object.
The azimuth angle θ can be calculated.

Up to this point, (Equation 2) has been derived for a general irradiation unit vector, but a special effect occurs when the zenith angles of the irradiation unit vectors are the same.
That is, the influence of the dark current component (DC component noise) of the image, which is mainly an error factor on the image pickup means side, the gain related to the brightness of the image, the change in the brightness due to the change in the aperture of the image pickup system, etc. is automatically You can cancel. The reason will be described below.

FIG. 7 is an explanatory diagram showing a case where one of the illuminations is in the zx plane. FIG. 7 shows a case where the coordinate axis is rotated around the z axis so that one of the illuminations is in the zx plane in FIG. 3, for example. Therefore, even if such an arrangement is considered, generality is not lost. Hereinafter, description will be given assuming that they are arranged as shown in FIG.

First, consider the irradiation unit vector m ₁ . m ₁
Is in the zx plane, m ₁ = (M ₁₁ , M ₁₂ , M ₁₃ ) = s (1,0, α). Where s = 1 / √ (1 + α ² ) is the scale factor,
m ₁ is set to be a unit vector. Similarly,
As for m ₂ and m ₃ , m ₂ = (M ₂₁ , M ₂₂ , M ₂₃ ) = s (β ₁ , γ ₁ , α) m ₃ = (M ₃₁ , M ₃₂ , M ₃₃ ) = s (β ₂ , γ ₂ , α). Here, to match the scale, β ₁ ² + γ ₁ ²
= 1 and β ₂ ² + γ ₂ ² = 1.

In this case, each component of the inverse matrix of the illumination matrix is
become that way. M₁₁ ^-1= T (γ₁-Γ₂) α M₁₂ ^-1= T (γ₂) α M₁₃ ^-1= T (-γ₁) α M_{twenty one} ^-1= T (β₂-Β₁) α M_{twenty two} ^-1= T (1-β₂) α M_{twenty three} ^-1= T (β₁-1) α M₃₁ ^-1= T (β₁γ₂-Β₂γ₁) M₃₂ ^-1= T (-γ₂) M₃₃ ^-1= T (γ₁) Here, t = 1 / (((γ₁-Γ₂) + (β₁γ₂-Β₂γ₁)) α)
is there. From this, find the azimuth angle θ of the surface gradient of the target object
Then,     θ = tan^-1((M^-1 _{twenty one}d₁+ M^-1 _{twenty two}d₂+ M^-1 _{twenty three}d₃) / (M^-1 ₁₁d₁+ M^-1 ₁₂d₂+ M ^-1 ₁₃ d₃))       = Tan^-1((β₂(d₁-D₂) + (d₂-D₃) + β₁(d₃-D₁)) /   (γ₁(d₁-D₃) + γ₂(d₂-D₁))) ・ ・ ・ ・ ・ (Equation 3) Becomes

Focusing on the contents of tan ⁻¹ (), (a) it does not depend on the physical quantity α depending on the zenith angle of the irradiation unit vector, and (b) by the linear combination of the difference in brightness (d _i −d _j ). (C) denominator, numerator (d _i −
It can be seen that it is composed of a linear combination of d _j ). That is, according to (a), the azimuth angle θ of the surface gradient of the target object obtained by the illumination with the same zenith angle of the irradiation unit vector.
Is independent of the zenith angle of the irradiation unit vector, and the results of (b) and (c) show that the brightness d is a conversion of d ′ = k ₁ d + k ₂ (where k ₁ and k ₂ are constants). Even if it receives, the value of the azimuth angle θ of the surface gradient of the target object calculated does not change. This is because the k ₂ component is canceled by taking the luminance difference of (b), and the k ₁ component is canceled by the numerator / denominator of (c). In this conversion, k ₁ can be considered as a gain related to the brightness of the image, a change in the brightness due to a change in the diaphragm of the imaging system, and the like, and k ₂ can be considered as a dark current component (DC component noise) of the image. Therefore, the azimuth angle θ of the surface gradient of the target object obtained by the illumination in which the zenith angle of the irradiation unit vector is the same, the surface gradient of the target object can be accurately determined without being influenced by error factors such as a camera. Azimuth can be measured.

Up to this point, the explanation has been made on the assumption that the illuminances of the respective illuminations on the target object are the same, but when the intensities of the respective illuminations are different, the respective illuminance ratios are u ₁ : When u ₂ : u ₃ , the brightnesses d ₁ , d ₂ , and d ₃ obtained when the respective illuminances are given may be used after being normalized by the illuminance ratio. That is, it becomes as follows. d ₁ ′ = d ₁ / u ₁ d ₂ ′ = d ₂ / u ₂ d ₃ ′ = d ₃ / u ₃ This d ₁ ′, d ₂ ′, d ₃ ′ is d ₁ , d ₂ in (Equation 2) , D ₃ , the azimuth angle θ of the surface gradient of the target object can be obtained.

As described above, according to the first embodiment,
Order lights located at locations with the same zenith angle but different azimuth angles
Subsequent irradiation, at least imaged by each irradiation
Select 3 images from 3 images,
The luminance value vector d at the same coordinate is d = (d₁, d₂, d₃)
And M is an illumination matrix in which the vectors of the irradiation direction are arranged,
This inverse matrix of M is M^-1And M^-1The (i, j) component of M^-1 _ij
, Then the surface gradient of the target object corresponding to the pixel position
Azimuth angle θ θ = tan^-1((M^-1 _{twenty one}d₁+ M^-1 _{twenty two}d₂+ M^-1 _{twenty three}d₃) / (M^-1 ₁₁
d₁+ M^-1 ₁₂d₂+ M^-1 ₁₃d₃)) Since it was decided to calculate with the calculation formula of
Image dark current component (DC component noise), which is a factor,
Gain related to image brightness, change in aperture of imaging system, etc.
You can automatically cancel the effects of changes in brightness due to
Shape measurement device that is not affected by error factors on the imaging device side.
Can be realized. Also, there are multiple combinations of three images.
Object selected and calculated for each combination of these three images
Removed outliers from each azimuth of body surface gradient
Then, the average calculation is performed to calculate the azimuth angle of the surface gradient of the target object.
Since it was decided to put it out, it is left to erroneous measurement due to specular reflection light etc.
It is possible to realize stable measurement that is not right.

Embodiment 2. Next, a second embodiment of the present invention will be described with reference to FIG. In the second embodiment, the illumination moving means 103 for moving one illumination 102 to the position of the same zenith angle and three different azimuth angles is configured in multiple stages. That is, in this case, the illumination moving means 103 for moving one illumination with the zenith angle ψ,
It has added illuminating moving means 103b for the other illumination lighting moving means 103a and still other moving with a zenith angle [psi _a lighting moved with a zenith angle [psi _b.

In the second embodiment, the determination in step S501 shown in FIG.
Choose one lighting combination? It should be changed to ". By doing so, it is possible to calculate a plurality of azimuth angles θ due to illumination of a plurality of zenith angles for one coordinate of the image by the measurement flow of FIG. 6. And
Out of the plurality of calculated azimuth angles θ, the abnormal value is removed, and the remaining ones are averaged to calculate the azimuth angle θ.

As shown in (Equation 3), the zenith angle ψ of the illumination is
Since there is no dependence of the azimuth θ on the measured surface gradient of the target object, even if the illumination moving means is configured in multiple stages and it is used, an error due to the difference in the zenith angle of the illumination position theoretically occurs. Absent. In the first embodiment, when removing an abnormal value, the azimuth angle of illumination is changed to increase the number of data of azimuth angles. However, it may be difficult to perform measurement by changing the azimuth angle of illumination depending on the shape of the target object. is there. In such a case, it may be possible to deal with it by changing the zenith angle of the illumination. It is often very effective to change the zenith angle of illumination when removing the influence of the regular reflection component of the target object.

As described above, according to the second embodiment,
At least three first height positions having the same zenith angle and different azimuth angles, at least three second height positions having different zenith angles and different azimuth angles from the first height position, and further Illuminating the target object sequentially from at least three third height positions having different azimuth angles and different azimuth angles from the first and second height positions, Since the abnormal value is removed and the remaining ones are averaged to calculate the azimuth angle θ, the influence of the specular reflection component is less likely to occur and the measurable width of the target shape can be widened.

Third Embodiment Next, a third embodiment of the invention will be described. In the third embodiment, a surface light source is used as the illumination 102. In the case of a point light source, the image pickup means 1
When 01 receives the specular reflection component properly, the measured luminance of the pixel of the specular reflection component becomes very large, which overlaps with the lack of the dynamic range of the image pickup means, and also for other pixels. The brightness may not be measured correctly. In this case, it becomes difficult to measure the diffused component of the reflected light of the target object. If a surface light source is used, the luminance value of the regular reflection component can be made dull, and a relatively stable image can be obtained. Further, in designing and adjusting the illumination, using a surface light source has an advantage that the illumination angle can be set more easily than a point light source.

As described above, according to the third embodiment,
Since a surface light source is used for illumination, it is possible to dull the luminance value of the specular reflection component that is too strong, and it is possible to obtain a relatively stable image, and when designing and adjusting the illumination,
It is possible to easily set the illumination angle.

Fourth Embodiment Next, a fourth embodiment of the present invention will be described with reference to FIG. FIG. 9 shows a plan view of FIG. 8, and in this case, the illumination of each stage is arranged in six directions with azimuth angles of 60 degrees. In FIG. 9, the illuminations on the same concentric circle have the same zenith angle, and what is shown by the solid line and what is shown by the broken line are
Each is a combination of three illuminations with azimuth angles of 120 degrees. Therefore, in the figure, two sets of lights at 120-degree intervals can be selected by selecting three lights shown by a solid line or three lights shown by a broken line among those having the same zenith angle. .

Attention is now paid to one of the combinations of three illuminations whose azimuth angles are 120 degrees apart. The combination of three illuminations at 120-degree intervals is the combination having the largest uniform angular difference among the three illuminations necessary for calculating the azimuth angle θ of the surface gradient of the target object. As can be understood from (Equation 3), the azimuth angle of the surface gradient of the target object is calculated by the difference in the brightness of the reflected light from each illumination.
In order to improve the measurement accuracy, it is desirable that the brightness difference be as large as possible. In order to make the difference in brightness as large as possible, it is desirable that the angle difference between the directions of the respective illuminations is as large as possible, and in this case, the azimuth angles of the illuminations are 120 degrees apart. Therefore, it is effective to select three illuminations at 120 degree intervals in order to improve the measurement accuracy.

As described above, according to the fourth embodiment,
Since the illumination is performed using at least three illuminations with azimuth angles of 120 degrees, it is possible to maximize the difference in brightness of the reflected light of the target object and improve the measurement accuracy of the azimuth angle of the surface gradient of the target object. be able to.

[0063]

As described above, according to the shape measuring apparatus according to the present invention, the illumination is sequentially irradiated from the place where the zenith angle is the same and the azimuth angle is different, and at least three images imaged by each irradiation. Select 3 images from the images,
Since the azimuth angle of the surface gradient of the target object is calculated by linear processing using the luminance value at the same coordinates of the selected three images and the illumination matrix in which the vectors of the illumination irradiation direction are arranged, Image dark current components (DC component noise), which are error factors on the image capturing side, gain related to image brightness, and changes in brightness due to changes in the aperture of the imaging system, etc., without preparing many different types of illumination. The effect of can be canceled automatically, and the azimuth angle of the surface gradient of the target object can be accurately measured.

According to the shape measuring device of the next invention, when the illuminances of the three illuminations are different, the respective luminance values at the same coordinates of the three images are normalized by the illuminance ratios of the three illuminations, respectively. Therefore, even if the illuminances of the three illuminations are different, the azimuth angle of the surface gradient of the target object can be accurately calculated.

According to the shape measuring apparatus of the next invention, a plurality of combinations of three images are selected from a plurality of images obtained by irradiation with four or more lights having the same zenith angle and different azimuth angles. After removing the abnormal value from the plurality of azimuth angles calculated for each combination of the three images, the azimuth angle of the surface gradient of the target object is calculated by averaging, so Stable measurement that is not affected by erroneous measurement can be performed.

According to the shape measuring apparatus of the next invention, at least three first height positions having the same zenith angle and different azimuth angles, and having a zenith angle different from the first height position and having an azimuth direction. The target object is sequentially illuminated from at least three second height positions with different angles, abnormal values are removed from the obtained plurality of azimuth angles, and the remaining ones are averaged to calculate the azimuth angle. Therefore, the width of the measurable target shape can be widened because it is less affected by the specular reflection component.

According to the shape measuring apparatus of the next invention, since the surface light source is used for illumination, the luminance value of the too strong specular reflection component can be blunted, and a relatively stable image can be obtained. In addition to the above, it is possible to easily set the illumination angle when designing and adjusting the illumination.

According to the shape measuring apparatus of the next invention, since the irradiation is performed at intervals of 120 degrees in azimuth,
The difference in brightness of the reflected light of the target object can be made as large as possible, and the measurement accuracy of the azimuth angle of the surface gradient of the target object can be improved.

As described above, according to the shape measuring method of the present invention, the illumination is sequentially irradiated from the locations having the same zenith angle and different azimuth angles, and at least three images captured by the respective irradiations are used. Three images are selected, and the azimuth angle of the surface gradient of the target object is calculated by a linear process using a brightness value at the same coordinates of the selected three images and an illumination matrix in which vectors of the illumination irradiation direction are arranged. Therefore, dark current components (DC component noise) of the image, which are error factors on the image pickup unit side, and gain relating to the brightness of the image, without preparing a large number of illuminations with different illumination angles,
It is possible to automatically cancel the influence of the change in the brightness due to the change in the diaphragm of the imaging system, and to accurately measure the azimuth angle of the surface gradient of the target object.

According to the shape measuring method of the next invention, when the illuminances of the three illuminations are different, the respective luminance values at the same coordinates of the three images are normalized by the illuminance ratios of the three illuminations to obtain the luminance values. Therefore, even if the illuminances of the three illuminations are different, the azimuth angle of the surface gradient of the target object can be accurately calculated.

According to the shape measuring method of the next invention, a plurality of combinations of three images are selected from a plurality of images obtained by irradiation with four or more lights having the same zenith angle and different azimuth angles. After removing the abnormal value from the plurality of azimuth angles calculated for each combination of the three images, the azimuth angle of the surface gradient of the target object is calculated by averaging, so Stable measurement that is not affected by erroneous measurement can be performed.

According to the shape measuring method of the next invention, at least three first height positions having the same zenith angle but different azimuth angles, and a zenith angle different from the first height position and having an azimuth angle The target object is sequentially illuminated from at least three second height positions with different angles, abnormal values are removed from the obtained plurality of azimuth angles, and the remaining ones are averaged to calculate the azimuth angle. Therefore, the width of the measurable target shape can be widened because it is less affected by the specular reflection component.

According to the shape measuring method of the next invention, since the surface light source is used for the illumination, the luminance value of the too strong regular reflection component can be blunted and a relatively stable image can be obtained. In addition to the above, it is possible to easily set the illumination angle when designing and adjusting the illumination.

According to the shape measuring method of the next invention, since the irradiation is performed at intervals of 120 degrees in azimuth,
The difference in brightness of the reflected light of the target object can be made as large as possible, and the measurement accuracy of the azimuth angle of the surface gradient of the target object can be improved.

[Brief description of drawings]

FIG. 1 is an explanatory diagram of a first embodiment of a shape measuring device according to the present invention.

FIG. 2 is an explanatory diagram showing a zenith angle and an azimuth angle.

FIG. 3 is an explanatory diagram showing a positional relationship between a coordinate system of an imaging surface and an illumination arranged at a location where the zenith angle ψ is the same and the azimuth angle is different.

FIG. 4 is a flowchart showing a procedure for calculating an azimuth angle θ when the number of illuminations is three.

FIG. 5 is an explanatory diagram showing a configuration including four illuminations arranged at locations having the same zenith angle ψ and different azimuth angles.

FIG. 6 is a flowchart showing a calculation procedure for calculating a plurality of azimuth angles θ by combining four or more illuminations.

FIG. 7 is an explanatory diagram showing a case where one of the illuminations is in a zx plane.

FIG. 8 is an explanatory diagram in which the illumination moving means is configured in multiple stages.

FIG. 9 is a plan view showing a configuration example in which illuminations at respective stages are arranged in six directions with azimuth angles of 60 degrees.

FIG. 10 is a diagram showing a conventional measuring method.

FIG. 11 is an explanatory diagram of a method of estimating a luminance peak angle by applying it to a Gaussian distribution in a conventional measurement method.

[Explanation of symbols]

101 image pickup means, 102, 102a, 102b, 10
2c, 102d illumination, 103, 103a, 103b
Holding means, 104 control means, 105 image processing means,
106 measurement result, 107 target object.

Claims (12)

[Claims] 1. A target object is illuminated from a plurality of different directions.
Bright light is sequentially radiated, and the image of the target object is sequentially lit for each lighting illumination.
The image is taken, and the pixels of the same coordinates in the taken multiple images
Measures the surface gradient of the target object from the brightness and the illumination direction
In the shape measuring device At least three positions with the same zenith angle but different azimuth angles
Illumination means for sequentially irradiating the target object with illumination, Objects illuminated by illumination from said at least three positions
Imaging means for sequentially picking up images of the object, Selection for selecting three images from the images picked up by this image pickup means
Means and Each of the selected three images at the same coordinates
The brightness value of₁, D₂, D₃And the direction of illumination
Let M be the lighting matrix in which^{- 1}age,
Inverse matrix M^-1The (i, j) component of M^-1 _ijAnd when each pixel
The azimuth angle θ of the surface gradient of the target object corresponding to the position, θ = tan^-1((M^-1 _{twenty one}d₁+ M^-1 _{twenty two}d₂+ M^-1 _{twenty three}d₃) / (M^-1 ₁₁
d₁+ M^-1 ₁₂d₂+ M^-1 ₁₃d₃)) A calculation means for calculating with the calculation formula of A shape measuring device comprising: 2. When the illuminances of the three lights are different, the calculating means normalizes the respective brightness values at the same coordinates of the three images by the illuminance ratios of the three lights, and the brightness values d ₁ , d. The shape measuring device according to claim 1, wherein ₂ , ₃ are obtained. 3. The selecting means selects a plurality of combinations of three images from a plurality of images obtained by irradiation with four or more lights having the same zenith angle and different azimuth angles, and the calculating means The azimuth of the surface gradient of the target object is calculated by removing an abnormal value from a plurality of azimuths calculated for each combination of three images and then performing an average calculation. The shape measuring device according to 2. 4. The lighting means comprises at least three first height positions having the same zenith angle but different azimuth angles and the first height position.
Of at least three second height positions having different zenith angles and different azimuth angles from the height position of the target object, and the selection means is capable of irradiating at least illumination. From the plurality of images obtained by the irradiation of the illumination of the above and the plurality of images obtained by the irradiation of the illumination at the second height position, a plurality of combinations of three images are respectively selected in plural, The azimuth angle of the surface gradient of the target object is calculated by removing an abnormal value from a plurality of azimuth angles calculated for each combination of images and then performing an average calculation. The shape measuring device described. 5. The shape measuring apparatus according to claim 1, wherein the lighting unit uses a surface light source for lighting. 6. The shape measuring device according to claim 1, wherein the irradiation unit sequentially irradiates the illumination at intervals of an azimuth angle of 120 degrees. 7. The target object is illuminated from a plurality of different directions.
Brightly illuminates sequentially, and images of the target object are captured sequentially for each illumination
Then, the brightness of the pixel at the same coordinates in the multiple captured images
The surface gradient of the target object from
In the shape measurement method, At least three positions with the same zenith angle but different azimuth angles
A first step of sequentially irradiating the target object with illumination, 3 images from the images taken by the illuminations
The second step of selecting the image of Each of the selected three images at the same coordinates
The brightness value of₁, D₂, D₃And the direction of illumination
Let M be the lighting matrix in which^{- 1}age,
Inverse matrix M^-1The (i, j) component of M^-1 _ijWhen the above
Azimuth angle θ of the surface gradient of the target object corresponding to the prime position
To θ = tan^-1((M^-1 _{twenty one}d₁+ M^-1 _{twenty two}d₂+ M^-1 _{twenty three}d₃) / (M^-1 ₁₁
d₁+ M^-1 ₁₂d₂+ M^-1 ₁₃d₃)) The third step of calculating with the calculation formula of A shape measuring method comprising: 8. When the illuminances of the three illuminations are different, in the third step, the respective luminance values at the same coordinates of the three images are normalized by the illuminance ratios of the three illuminations to obtain a luminance value d ₁ , The shape measuring method according to claim 7, wherein d ₂ and d ₃ are obtained. 9. In the second step, a plurality of combinations of three images are selected from a plurality of images obtained by irradiation with four or more lights having the same zenith angle and different azimuth angles, In the step, the azimuth of the surface gradient of the target object is calculated by removing an abnormal value from a plurality of azimuths calculated for each combination of three images and then performing an average calculation. The shape measuring method described in 7 or 8. 10. In the first step, at least three first height positions having the same zenith angle and different azimuth angles and at least three zenith angles different from the first height position and different azimuth angles are provided. At least illumination is sequentially applied to the target object from the three second height positions, and in the second step, a plurality of images and second heights obtained by the irradiation of the illumination at the first height position A plurality of combinations of three images are selected from each of the plurality of images obtained by irradiation with the illumination of the position, and in the third step, among the plurality of azimuth angles calculated for each combination of the three images. 9. The shape measuring method according to claim 7, wherein the azimuth of the surface gradient of the target object is calculated by removing the abnormal value and then performing an average calculation. 11. The shape measuring method according to claim 7, wherein a surface light source is used for illumination in the first step. 12. The azimuth angle 12 is set in the first step.
Irradiation is performed at intervals of 0 degree, The shape measuring method according to any one of claims 7 to 11.