JP5867123B2 - Three-dimensional shape measuring apparatus and calibration method - Google Patents

Description

  The present invention relates to a technique for measuring a three-dimensional shape of a surface of a measurement object.

  A technique for obtaining a normal (or gradient) of the surface of a measurement object by analyzing an image taken with a camera and restoring the three-dimensional shape is known.

  A color highlight method is known as one of such techniques (Patent Document 1). In the color highlight method, for example, red, blue, and green ring lights are arranged on the dome, and each color is irradiated to the measurement target. Then, by analyzing the color of the reflected light from the measurement target, the direction of the normal line (only the zenith angle component) of the measurement target surface is calculated.

  In the color highlight method, when the reflection characteristic of the measurement object is not completely specular reflection (having a specular lobe), the measurement accuracy is deteriorated due to color mixture of reflected light. On the other hand, there has been proposed a technique for accurately calculating the surface shape of a measurement object even when the reflection characteristic is not a perfect mirror surface (Patent Document 2). In this technique, the illumination pattern is devised so that the reflected light from the measurement object is the same as the regular reflection light even when the measurement object is not a perfect mirror surface.

  As shown in FIG. 15, the reflected light of light incident on an object that is not a perfect mirror surface includes light that is sharp and narrow in the direction of specular reflection (specular spike) and light that spreads in a direction that deviates from the specular reflection direction (specular surface). Robe). A specular lobe refers to the spread of specular reflection light caused by a micro uneven surface (micro facet) on the surface to be measured. As the orientation of the microfacets varies, that is, as the surface becomes rougher, the specular lobe becomes wider. Conversely, as the variation in the orientation of the microfacets becomes smaller, the mirror surface becomes closer to a perfect specular state. Here, the deviation (angle) from the regular reflection direction and the ratio of the light intensity of the lobe to the spike represent the reflection characteristics. For an object with non-uniform reflection characteristics, the shape of the specular lobe varies depending on the surface roughness at each surface position. If the surface is very rough, the reflected light will consist only of specular lobes. The ratio between the specular lobe and the specular spike is close to 1, making it difficult to distinguish between the two.

  At this time, if illumination is used so that the light from the surrounding area is canceled and the spectral characteristics are the same as in the case of a perfect mirror surface, even if the reflection characteristics are not uniform or the surface is rough, it will be completely It can be measured in the same way as a specular object. Patent Document 2 discloses an illumination pattern that uses a dome-shaped illumination device 300 as shown in FIG. 16A and that reflects light from a measurement object other than a perfect mirror surface is the same as that of a perfect mirror object. ing. Since the measurement object 200 has the specular lobe 400, the reflected light that is irradiated from the illumination device 300 to the measurement object 200 and reaches the camera 100 becomes combined light of illumination in the areas 302 and 303 of the illumination device. Here, by adopting an illumination pattern in which the illumination in the region 302 and the region 303 cancel each other, the reflected light reaching the camera 100 can be the same as the reflected light from the regular reflection direction 301.

JP-A-3-142303 International Publication No. 2010-118281

  As shown in FIG. 16A, when the inclination of the normal line of the measurement object is not so large, the entire specular lobe is within the range of the illumination device, so that the reflected light from the measurement object is the same as the regular reflection light. can do. However, when the inclination of the normal line of the measurement object is large as shown in FIG. 16B, the entire specular lobe does not fall within the range of the illumination device. In this case, the reflected light reaching the camera 100 is different from the reflected light in the regular reflection direction. That is, the principle of Patent Document 2 cannot be applied to the end portion of the illumination device (corresponding to a surface with a steep slope), and the accuracy of normal measurement is greatly reduced.

  The present invention has been made in view of the above circumstances, and the object of the present invention is to accurately measure the orientation of a surface having a normal line of inclination corresponding to the vicinity of the end of the lighting device. It is to provide the technology.

  In order to achieve the above object, the present invention predicts in advance the relationship between the characteristic amount of reflected light to be observed, the direction of the normal, and the reflection characteristic (lobe), and creates that relationship as a reference table. The feature is that a correct normal direction is obtained by using a characteristic amount of reflected light at the time of observation and a reference table.

  More specifically, the present invention is a three-dimensional shape measuring apparatus that measures the surface shape of a measurement object whose reflection characteristics are unknown, and an illumination unit that irradiates light to the measurement object with a predetermined illumination pattern; The imaging means for imaging the measurement object, and a combination of a plurality of reflection characteristics and a plurality of normal directions, the surface having the reflection characteristics and the normal directions is irradiated with light of the illumination pattern A reference table for storing feature quantities relating to reflected light from the surface; feature quantity calculating means for calculating the feature quantities at each position of the measurement object from an image captured by the imaging means; and the feature quantity calculating means. Shape calculating means for obtaining a normal direction of a surface at each position of the measurement object based on the calculated feature amount and the reference table.

  In the above description, the normal direction may be the sum of the zenith angle direction and the azimuth direction of the normal line, or may be only the zenith angle direction. Further, the characteristic amount related to the reflected light can be a brightness or a color of the reflected light, or a value obtained from these amounts.

  By adopting such a configuration, even on a surface having a normal line of inclination corresponding to the vicinity of the end of the illumination means, the direction can be measured with high accuracy. If the expression is changed, it is possible to accurately measure the normal inclination corresponding to the entire angle range of the illumination means.

  The illumination pattern is preferably a pattern in which one feature amount in the reference table is associated with one normal direction. Here, “one normal direction” may have a certain range. The range is determined by the accuracy required for measurement.

By adopting such an illumination pattern, the normal direction can be uniquely determined from the feature quantity of the reflected light with respect to the measurement target having an arbitrary reflection characteristic. Note that if one feature value employs an illumination pattern that has a one-to-one correspondence with the combination of the normal direction and the reflection characteristic, both the normal direction and the reflection characteristic are uniquely determined from the feature value of the captured reflected light. Can be determined. However, if there is no need to obtain the reflection characteristics, an illumination pattern that changes the feature value if the normal direction is different (if the normal direction is the same, even if the reflection characteristic is associated with the same feature value) By adopting a good illumination pattern), it is possible to uniquely determine the direction of the normal from the feature quantity of the reflected light that has been photographed.

  The illumination pattern in the present invention can be a combination of a plurality of illumination sub-patterns. In this case, the feature amount can be a combination of the brightness of the reflected light from the measurement object when the light of each illumination subpattern is irradiated.

  Thus, by combining a plurality of illumination sub-patterns to form one illumination pattern, the resolution in the normal direction can be improved.

  Here, the feature amount calculation means performs a process in which the illumination means irradiates the measurement object with light in each illumination sub-pattern, and the imaging means images the measurement object and measures the brightness of the reflected light. The feature amount can be obtained by repeating. In this case, imaging is performed as many times as the number of illumination sub-patterns, and the combination of the brightness of reflected light obtained from each captured image is used as the feature amount. In this configuration, the illumination color in each illumination sub-pattern can be the same.

  Further, the plurality of illumination sub-patterns are illuminations of different colors, and the feature amount calculating unit irradiates the measurement object with light superimposed on the respective illumination sub-patterns from the illumination unit, and the imaging unit The feature quantity may be obtained by imaging the measurement object and obtaining the brightness of each color. In this case, the combination of the brightness of each color becomes the feature amount. In this configuration, it is possible to acquire the feature amount by only one imaging.

  In the present invention, the illumination means has a substantially hemispherical (dome-shaped) light emission region, and each of the plurality of illumination sub-patterns does not change the light emission intensity in the azimuth direction and emits the light intensity in the zenith angle direction. Is a pattern that takes a peak at a predetermined zenith angle, and each illumination sub-pattern has a different zenith angle that takes the peak.

  According to such a configuration, the feature amount of the reflected light of each illumination sub-pattern changes continuously according to the change of the normal inclination (zenith angle). Therefore, one feature amount is associated with one normal slope (zenith angle). Further, such an illumination sub-pattern is a pattern in which the feature amount of reflected light changes sensitively according to changes in reflection characteristics (lobes) and normal slopes. Therefore, the inclination of the normal can be measured with high accuracy. Also, by combining a plurality of such illumination sub-patterns, the normal inclination measurement resolution can be improved. Since this illumination sub-pattern does not change in the azimuth angle direction, it is an illumination configuration for measuring only the zenith angle component in the normal direction.

  Further, when the illumination subpattern as described above is employed, it is preferable that one of the plurality of illumination subpatterns has a peak at the end portion of the substantially hemispherical shape.

  A part of the specular lobe on the surface having a steep inclination is out of the illumination. That is, the amount of reflected light is reduced. By adopting the configuration as described above, the amount of reflected light can be made relatively large even on a surface having a normal inclination corresponding to the illumination end, and therefore the measurement accuracy can be improved. .

  Further, the illumination pattern of the illumination means can be a pattern in which each of the plurality of illumination sub-patterns changes the light emission intensity in different directions. Alternatively, one illumination sub-pattern can have a uniform emission intensity in the entire emission region, and each of the other illumination sub-patterns can have a pattern in which the emission intensity changes in a different direction.

By adopting such an illumination pattern, the normal direction of the measurement object can be obtained for the zenith angle component and the azimuth angle component.

  The present invention can be understood as a three-dimensional shape measuring apparatus having at least a part of the above means. The present invention can also be understood as a calibration method for such a three-dimensional shape measurement apparatus, a three-dimensional shape measurement method including at least a part of the above processing, or a program for realizing such a method. Each of the above means and processes can be combined with each other as much as possible to constitute the present invention.

  According to the present invention, it is possible to accurately measure the direction of a surface having an inclination normal corresponding to the vicinity of the end of the illumination device.

It is a figure which shows the whole structure of the three-dimensional shape measuring apparatus concerning embodiment. It is a figure explaining the response | compatibility of the direction of the normal line of the surface of a measurement object, and a light emission area | region. It is a figure which shows the color pattern in the light emission area | region of an illuminating device. It is a figure which shows the functional block diagram of a table preparation process. It is a flowchart which shows the flow of a table creation process. It is a schematic diagram of the table created. It is a figure which shows the functional block diagram of a shape measurement process. It is a flowchart which shows the flow of a shape measurement process. (A) is a figure which shows an example of a normal map, (B) is a figure which shows an example of the decompress | restored shape. It is a figure which shows the modification of an illumination pattern. It is a figure which shows the modification of an illumination pattern. It is a figure which shows the modification of an illumination pattern. It is a figure which shows the structure of a shape measuring apparatus provided with a flat plate-shaped illumination apparatus. It is a figure explaining the illumination pattern in a flat plate-shaped illuminating device. It is a figure explaining a reflection characteristic. It is a figure explaining the reflected light in the (A) illumination area center vicinity and the (B) illumination area edge part.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. The three-dimensional shape measurement apparatus (hereinafter simply referred to as a shape measurement apparatus) of the present embodiment performs three-dimensional shape measurement of an incompletely diffused object by image analysis.

  Here, first, a shape measuring apparatus that takes only the normal zenith angle direction as a measurement target and does not measure the normal azimuth angle direction will be described. A configuration for measuring the azimuth direction of the normal will be described separately.

  The shape measuring device described in the present embodiment can be applied to object recognition in various automatic measuring devices, automatic inspection devices, robot visions, etc., for example, soldering quality inspection in a board visual inspection device (AOI system), It can be preferably applied to unevenness inspection on the surface of a metal workpiece.

<Overall configuration of measuring device>
With reference to FIG. 1, the whole structure of a shape measuring apparatus is demonstrated. FIG. 1 is a diagram schematically showing a hardware configuration of the shape measuring apparatus.

  The shape measuring apparatus is roughly configured to include a measuring stage 5, an inspection head H, and an information processing apparatus 6. The inspection head H is attached with an illuminating device 3 for irradiating the measurement object 4 arranged on the measurement stage 5 with measurement light, and a camera (image sensor) 1 for photographing the measurement object 4 from vertically above. It has been. The information processing device 6 includes a CPU (Central Processing Unit) 60, a memory 61, a storage device 62, an inspection head control unit 63, an image input unit 64, an illumination device control unit 66, a stage control unit 67, a user I / F 68, a display. Part 69 and the like. The inspection head control unit 63 has a function of controlling movement of the inspection head H in the Z direction (direction perpendicular to the measurement stage 5), and the stage control unit 67 has a function of controlling movement of the measurement stage 5 in the XY direction. The lighting device control unit 66 has a function of controlling lighting and extinguishing of the lighting device 3 (switching of lighting patterns as necessary). The image input unit 64 has a function of taking a digital image from the camera 1. The user I / F 68 is an input device operated by the user, and corresponds to, for example, a pointing device, a touch panel, a keyboard, and the like. The display unit 69 is a part where a measurement result or the like is displayed on the screen, and is configured by a liquid crystal display, for example.

  At the time of measurement, the inspection head H and the measurement stage 5 move relatively, and the measurement object 4 moves to a predetermined measurement position (in the example of FIG. 1, the center of the illumination device 3 (the intersection of the optical axis of the camera 1 and the measurement stage 5). )). And an image is image | photographed in the state irradiated with measurement light from the illuminating device 3. FIG. An image photographed by the camera 1 is taken into the information processing apparatus 6 via the image input unit 64 and is used for image analysis described later. Hereinafter, the configuration and processing of the shape measuring apparatus will be described in detail.

(Lighting device)
The illumination device 3 is a surface light source having a dome shape as shown in FIG. 1, and all of the dome shape is a light emitting region. Note that an opening for the camera 1 is provided at the zenith portion of the illumination device 3. Such an illuminating device 3 can be comprised from a dome-shaped color filter and the light source which irradiates white light from the exterior, for example. For example, a plurality of LED chips may be arranged inside the dome and irradiated with light through a diffusion plate. In addition, the lighting device 3 can be configured by making a liquid crystal display, an organic EL display, or the like into a dome shape.

  The shape of the light emitting area of the illumination device 3 is preferably a hemispherical dome shape so that light can be irradiated from all directions of the measurement object 4. In this way, normals in all directions can be measured. However, the light emitting region may have any shape as long as light is irradiated from a position corresponding to the normal direction to be measured. For example, if the normal direction of the surface is limited to a substantially vertical direction, there is no need to irradiate light in the horizontal direction (from a shallow angle direction).

  If an appropriate illumination pattern is employed using such an illumination device, the surface shape (normal direction) of the measurement object can be measured from only one image. This will be described with reference to FIG. It is assumed that the direction of the normal line at a certain point on the surface of the measurement object 4 is the direction of the arrow N, the zenith angle is θ, and the azimuth angle is φ. At this time, the color of the point photographed by the camera 1 becomes reflected light of light emitted from the region R of the illumination device 3 and incident on the measurement object 4. Here, the region R has a spread corresponding to the reflection characteristic (the shape of the specular lobe), and the color photographed by the camera 1 is a color obtained by weighting the light in the region R according to the reflection characteristic. Here, if an illumination pattern is used in which the color of the reflected light has a one-to-one correspondence with the combination of the normal direction and the reflection characteristic, the normal direction of the measurement object and the reflection characteristic from only one image. Can be requested. Also, an illumination pattern in which the color of the reflected light is different if the direction of the normal line is different (a pattern that allows the color of the reflected light to be the same even if the reflection direction is different if the direction of the normal line is the same). ), The reflection characteristic may not be uniquely determined, but the normal direction of the measurement object can be uniquely determined.

  The illuminating device 3 has an illumination distribution (illumination pattern) that satisfies the above conditions. However, in the present embodiment, since only the zenith angle component of the direction of the normal of the surface shape is to be measured, an illumination pattern in which the color of the reflected light and the zenith angle of the normal correspond one-to-one is employed. . The illumination pattern that can also measure the azimuth direction will be described separately.

  The illumination distribution (illumination pattern) of the light emitting region of the illumination device 3 in this embodiment is a combination of a plurality of different illumination subpatterns. Here, three illumination sub-patterns are used, and each illumination sub-pattern is realized by monochromatic light of red light (R), green light (G), and blue light (B). As shown in FIG. 3A, each illumination sub-pattern has a distribution that does not depend on the azimuth angle of the light emitting region, but changes depending only on the zenith angle. Each illumination sub-pattern is a pattern that takes a peak at a predetermined zenith angle and gradually decreases with increasing distance from the angle. As shown in FIG. 3B, the zenith angle positions at which peaks are taken for R, G, and B are different, and one illumination sub-pattern (here, G) has a peak at the end of the dome-shaped light emitting region. Is set to take.

  The illumination pattern of the illuminating device 3 is an illumination in which the characteristic quantity (RGB luminance values) of reflected light photographed by the camera 1 uniquely corresponds to the normal zenith angle with respect to the measurement object 4 having an arbitrary reflection characteristic. Any pattern can be used as long as it is a pattern. The patterns shown in FIGS. 3A and 3B described above are one of the illumination patterns that satisfy this condition. Note that this condition is satisfied even if the position where the illumination sub-pattern has a peak is changed, so that an illumination sub-pattern having a peak at the end of the illumination area may not necessarily be adopted.

(table)
Thus, the color and brightness of the pixel correspond one-to-one with the direction of the normal at that point and the angle of the light source. Therefore, in this shape measuring apparatus, a table in which the feature value values relating to the color and brightness of the pixels are associated with the angle of the light source is created in advance and registered in the storage device 62. In the present embodiment, the illumination device 3 projects light that is a combination of three component lights of red light (R), green light (G), and blue light (B). Use the ratio of each component. For example, for each component of RGB, the maximum luminance is normalized by 1, and a combination of (R, G, B) can be used as a feature amount. Further, a ratio of another color to a certain color (here, G), for example, R / (R + G), B / (B + G), or the like may be used as the feature amount.

  Hereinafter, the table creation processing will be described in detail with reference to FIGS. 4 and 5. FIG. 4 is a diagram illustrating functional blocks of functions realized by the information processing apparatus 6 that performs table creation processing. FIG. 5 is a flowchart showing the flow of table creation processing. The table creation process is realized by the CPU 60 of the information processing device 6 reading and executing a program from the storage device 62. However, some or all of these functional blocks may be configured by ASIC (ASIC), PLD (programmable logic device), or the like.

The information processing device 6 uses the mirror-like hemisphere as a teaching object, acquires the correspondence between the luminance value and the light source vector (S101), and creates illumination data (S102). Specifically, a hemispherical mirror object having a known shape is placed on the measurement stage 5 as a teaching object, and the teaching object is photographed by the camera 1 in a state where light is projected from the illumination device 3 with the illumination pattern described above. . Data captured by the camera 1 is taken into the information processing device 6 by the image input unit 64 and stored in the storage device 62 as specular ball image data 601. Further, since the shape of the teaching object is known, the direction of the surface normal at each position is also known, and this information is stored in advance in the storage device 62 as shape information 602. The illumination data creation unit 603 can acquire the correspondence between the light source vector and the feature amount (RGB luminance values) by referring to the specular sphere image data 601 and the shape information 602. The illumination data creation unit 603 converts this correspondence relationship into the illumination data 6
04 is created and stored in the storage device 62.

  The reflection characteristic information 605 is data storing a range of reflection characteristics that can be taken by a measurement object such as solder. The normal information 606 is data that stores a range of normals that can be taken by the measurement object.

  The rendering equation calculation unit 607 uses these data to calculate the feature quantity of the reflected light when the surface has an arbitrary reflection characteristic and normal inclination based on the rendering equation. Specifically, a loop (S103) within the range of the reflection characteristic σ defined in the reflection characteristic information 605 and a loop (S104) within the range of the normal direction θ defined in the normal line information 606 are shown. The following rendering equation is calculated for all combinations of the reflection characteristics σ and the normal direction θ within the range by the multiple loop (S105).

ここで、Ｅは視線ベクトル、Ｎは法線ベクトル、Ｈはハーフベクトル、Ｌは光源ベクトル、ρは鏡面反射率、σは反射特性（ローブの広がり）、θは法線ベクトルの天頂角である。 The rendering equation calculation unit 607 uses the following rendering equation based on the BRDF model. In this embodiment, a metal such as lead-free solder is the main object of measurement, so the Cook-Torrance model for specular highlights is used, but other models may be used depending on the object to be measured.

Here, E is a line-of-sight vector, N is a normal vector, H is a half vector, L is a light source vector, ρ is a specular reflectance, σ is a reflection characteristic (lobe spread), and θ is a zenith angle of the normal vector. .

  By solving this rendering equation, the characteristic amount of reflected light (a combination of RGB luminance values) captured by the camera 1 when the illumination from the illumination device 3 is projected onto the surface having the reflection characteristic σ and the normal zenith angle θ. ) Is required. The output unit 608 creates a correspondence between each reflection characteristic σ and the normal zenith angle θ and the feature quantity RGB as a reference table 609 and stores it in the storage device 62 (S106).

  FIG. 6 is a schematic diagram of the reference table obtained in this way. Here, when the combination of the reflection characteristic σ and the zenith angle θ is different, it is preferable that the feature amount RGB is also different. However, if an illumination pattern in which the feature amount RGB is different at least when the zenith angle θ is different is adopted, The zenith angle θ is obtained from the light feature RGB.

他の方法としては、近傍の複数の特徴量に対応する反射特性σと天頂角θから補間によって求めることも好ましい。 It should be noted that not all feature quantities appear in the feature quantities obtained by solving the rendering equation. For example, even if the rendering equation is solved, the feature quantity (r, g, b) is not always obtained. Accordingly, for the feature quantity RGB for which the corresponding reflection characteristic σ and zenith angle θ are not given by the rendering equation, the reflection characteristic σ and zenith angle θ corresponding to the nearest feature quantity are adopted as corresponding values (S107). Specifically, the reflection characteristic σ and the zenith angle θ given by the following equations are employed.

As another method, it is also preferable to obtain the reflection characteristic σ corresponding to a plurality of nearby feature amounts and the zenith angle θ by interpolation.

  By creating in this way, the reference table 609 associates the reflection characteristic σ and the zenith angle θ with respect to an arbitrary feature amount RGB.

  Although it is possible to use a table (general-purpose table) created by another shape measuring device, it is preferable to create a table using this device itself. By doing so, individual differences and assembly errors of the lighting device 3 and the camera 1 can be woven into the table, so that an improvement in accuracy can be expected compared to using a general-purpose table. Such table creation processing is preferably performed at the time of shipment of the apparatus. Further, when there is aged deterioration of the lighting device 3 or the camera 1, the table may be updated (calibration) periodically.

  Any shape can be used as the teaching object. For example, a polyhedron may be used, and a plate having a known shape can be tilted or rotated at various angles. However, it is preferable to use an object having a surface shape that includes as many normal (gradient) components as possible in order to reduce the number of times of photographing and reduce the labor of table creation. For a sphere (when the imaging direction is one direction, it may be a hemisphere as in this embodiment), normal information in all directions can be obtained by one imaging, and a normal vector can be easily calculated from the equation of the sphere. Therefore, it is optimal for teaching objects.

(Shape measurement)
Next, functions and processing flows related to shape measurement will be described with reference to FIGS. FIG. 7 is a diagram illustrating functional blocks of functions realized by the information processing apparatus 6 that performs shape measurement processing. FIG. 8 is a flowchart showing the flow of the shape measurement process. The shape measurement processing is also realized by the CPU 60 of the information processing device 6 reading and executing a program from the storage device 62, and part or all of the shape measurement processing is performed by an ASIC (ASIC), a PLD (programmable logic device), or the like. It may be configured.

  When the measurement object is positioned at a predetermined measurement position, the CPU 60 irradiates the measurement object with light from the illumination device 3 and performs photographing with the camera 1 (S201). The captured image is taken into the zenith angle calculation unit 611 via the image input unit 64. In addition, the zenith angle calculation unit 611 reads the reference table 609 created by the table creation process (S202).

  The zenith angle calculation unit 611 acquires the zenith angle θ and the reflection characteristic σ from the reference table 609 for each pixel of the captured image 610 using the feature amount as a key (S203, S204). Here, it is possible to obtain the reflection characteristic σ of the measurement object, but it is not necessary to acquire the reflection characteristic σ when only the direction of the normal is measured. By performing the above processing for all the pixels in the captured image, the zenith angle calculation unit 611 can obtain the direction of the normal line for the entire surface of the measurement object (S205). If the captured image includes a specular surface and a diffusing surface, only the specular surface may be extracted and the normal direction may be acquired only for the region by the above processing.

  FIG. 9A shows an example of a normal map when the hemispherical measurement object 4 is thus targeted. The normal map is a unit vector illustrating normals at points on the surface of the measurement object.

  The zenith angle calculation unit 611 may use a normal map as a final output, but can also restore a three-dimensional shape from the normal map. The three-dimensional shape is restored by converting the normals of the respective points of interest obtained in step S204 into gradients and connecting them together. This process is expressed herein as “integration”. A shape restored from the normal map of FIG. 9A is shown in FIG. According to this method, the three-dimensional shape of the surface of the measurement object can be accurately restored.

<Advantages of the embodiment>
According to the shape measuring apparatus of this embodiment, even if the surface has a steep gradient and has a normal corresponding to the end of the illumination area, the direction of the normal can be measured with high accuracy. When the expression is changed, the normal direction can be accurately measured up to the zenith angle range corresponding to the angle of the end of the illumination area. That is, in the conventional shape measuring apparatus, the measurable range of the surface gradient by the dome illumination is a range obtained by subtracting the lobe size from the illumination angle range, but according to the present embodiment, the entire illumination range is defined as the measurable range. can do.

  Moreover, according to this embodiment, even if the reflection characteristic of the measurement object is unknown, the surface shape can be measured. In the conventional method of creating a reference table based on a teaching object having a known shape, only an object having the same reflection characteristics as that of the teaching object can be measured. It can be measured. Furthermore, the reflection characteristic of the measurement object can also be obtained.

  Also, as illumination for shape measurement, each illumination sub-pattern is realized with light of different colors, and the light of the illumination pattern that overlaps them is projected onto the measurement object, so color features are acquired from only one image Therefore, the direction of the normal line of the measurement object 4 can be obtained. It is easy (high speed) for the measurement object 4 because the image is captured only once, and the calculation of the direction of the normal can be easily found by examining a table storing the correspondence between the normal and the feature quantity. It is possible to measure the surface shape.

  Each illumination sub-pattern is a pattern that changes only depending on the zenith angle, and has a bell-shaped emission intensity distribution that takes a peak at a predetermined zenith angle. Here, one illumination sub-pattern is configured to have a peak near the end of the illumination area. On the surface having a steep gradient corresponding to the end of the light emitting region, a part of the specular lobe deviates from the illumination region, so that the amount of reflected light incident on the camera 1 is reduced. Since the decrease in the amount of light leads to a decrease in measurement accuracy, the use of an illumination subpattern that has a peak at the end of the light emitting area does not reduce the amount of light even for light reflected from such a steep surface. It is configured. Thereby, even if it is a normal inclination corresponding to the edge part of an illumination area, it can measure accurately.

<Modification of lighting device>
In the above embodiment, as shown in FIG. 3, the illumination pattern having a bell-shaped emission intensity distribution in which the emission intensity of the three colors RGB depends only on the zenith angle and takes a peak at a predetermined zenith angle is adopted. . However, it is not necessary to have a bell-shaped emission intensity distribution, and a trapezoidal emission intensity distribution that takes a constant peak value in a predetermined zenith angle range may be employed as shown in FIG. In this case, it is preferable that the zenith angles taking the peak values do not overlap in each illumination sub-pattern. Further, even when such an illumination pattern is adopted, it is preferable that one illumination sub-pattern has a configuration that takes a peak at the end of illumination.

In the above-described embodiment, the purpose is to measure only the zenith angle component of the normal of the surface of the measurement object, so that the emission intensity of the RGB three colors changes depending only on the zenith angle. However, when it is intended to measure both the zenith angle component and the azimuth angle component of the normal of the measurement object surface, an illumination pattern that changes according to the azimuth angle may be employed. In this case, the reference table is a table in which the feature amount RGB is associated with the combination of the reflection characteristic σ and the normal zenith angle θ and azimuth angle φ. Since the table creation method can be performed in the same manner as in the above embodiment, a detailed description is omitted. Similarly to the above-described embodiment, if an illumination pattern in which the three combinations of the zenith angle θ, the azimuth angle φ, and the reflection characteristic σ correspond to the color of the reflected light has a one-to-one relationship is adopted, one image is taken. From the image, these three quantities can be measured. Also, an illumination pattern in which the color of the reflected light is different if at least the normal direction (θ, φ) is different (if the normal direction (θ, φ) is the same, the reflected light is reflected even if the reflection characteristics σ are different). The reflection characteristic σ may not be uniquely determined if a pattern that allows the colors to be the same is used, but the normal direction (θ, φ) of the measurement object is uniquely determined from only one image. Can be determined.

  For example, as such an illumination pattern, there is a pattern for changing the light emission intensity of each component of RGB in different directions on the dome as shown in FIG. Here, the change directions are set to 120 degrees. If such a pattern is adopted, the emission intensity changes in accordance with the zenith angle and the azimuth angle in each illumination sub-pattern, so the direction of the normal line on the surface of the measurement object is determined by the zenith angle component and the azimuth component. You will be able to ask for both.

  As another example, a combination of patterns that change in different directions, such as a pattern in which the three colors change downward, rightward, and leftward as shown in FIG. 12A, is used. You may do it. In addition, it is not necessary to change all three colors with an angle. As shown in FIG. 12B, one color emits light with uniform brightness on the entire surface, and the other two colors change with angles in different directions. A simple pattern may be adopted.

  Further, in the description so far, the configuration that makes it possible to restore the three-dimensional shape of an object by only one measurement (illumination and photographing) by using an illumination pattern in which illumination sub-patterns of different colors are superimposed. Stated. However, although the measurement time is longer than that, two or more types of illumination sub-patterns may be sequentially turned on to perform imaging, and the three-dimensional shape may be restored using a plurality of obtained images. This method can also obtain the same restoration result. In the case of sequentially photographing with different illumination sub-patterns, the light emission color of each illumination sub-pattern can be the same color (for example, green). In this case, the feature value of the reflected light employs a combination (G1, G2, G3) of the reflected light luminance values by the respective illumination sub-patterns instead of the combination of the luminance values of the respective colors of the reflected light (R, G, B). do it. The method of repeating the measurement a plurality of times as described above has a demerit that the measurement time becomes longer, but has a merit that the configuration can be simplified and the manufacturing cost can be reduced because a monochromatic light source may be used. Further, since the number of illumination sub-patterns can be easily increased, there is an advantage that the normal measurement resolution can be further improved by employing four or more illumination sub-patterns.

  Moreover, the shape of the illuminating device 3 is not limited to a dome shape (hemisphere), and may be a flat plate shape as shown in FIG. Moreover, the shape which curved the flat plate in the arc shape may be sufficient. If the shape is to be restored, the illumination may be uniquely determined from the observed image of the position of the light source.

  Further, as in the example of FIG. 14A, a red light (R) pattern whose emission intensity increases toward the right, a green light (G) pattern whose emission intensity increases toward the left, Blue light (B) patterns whose emission intensity increases in the direction may be superimposed. In this case, as shown in FIG. 14B, the light emission intensity may be changed in accordance with the angle θ in each illumination subpattern.

  1: Camera, 3: Lighting device, 4: Measurement object, 5: Measurement stage, 6: Information processing device, 11: Inspection head

Claims (12)

A three-dimensional shape measuring apparatus for measuring the surface shape of an object to be measured whose reflection characteristic is a ratio of the light intensity of the specular lobe to the specular spike and the angle from the regular reflection direction ,
Illuminating means for irradiating the measurement object with light with a predetermined illumination pattern;
Imaging means for imaging the measurement object;
For a combination of a plurality of reflection characteristics and a plurality of normal directions, a reference for storing a feature amount related to reflected light from the surface when the surface having the reflection characteristics and the normal direction is irradiated with light of the illumination pattern Table,
Feature quantity calculating means for calculating the feature quantity at each position of the measurement object from an image taken by the imaging means;
Based on the feature amount calculated by the feature amount calculation unit and the reference table, a shape calculation unit for obtaining a normal direction of a surface at each position of the measurement object;
A three-dimensional shape measuring apparatus. The illumination pattern is a pattern in which one feature amount in the reference table is associated with one normal direction.
The three-dimensional shape measuring apparatus according to claim 1. The illumination pattern is a combination of a plurality of illumination sub-patterns,
The feature amount is a combination of the brightness of the reflected light from the measurement object when the light of each illumination sub-pattern is irradiated.
The three-dimensional shape measuring apparatus according to claim 1 or 2. The feature amount calculating means repeats a process in which the illumination means irradiates light on the measurement object with each illumination sub-pattern, and the imaging means images the measurement object and measures the brightness of reflected light. , Obtaining the feature amount,
The three-dimensional shape measuring apparatus according to claim 3. The plurality of illumination sub-patterns are illuminations of different colors,
The feature amount calculating means irradiates the measurement object with light superposed with respective illumination sub-patterns from the illumination means, and the imaging means images the measurement object and obtains the brightness of each color. , Obtaining the feature amount,
The three-dimensional shape measuring apparatus according to claim 3. The illumination means has a substantially hemispherical light emitting region,
Each of the plurality of illumination sub-patterns is a pattern in which the emission intensity does not change in the azimuth angle direction, the emission intensity changes in the zenith angle direction, and takes a peak at a predetermined zenith angle,
In each illumination sub-pattern, the zenith angle taking the peak is different,
The three-dimensional shape measuring apparatus according to any one of claims 3 to 5. One of the plurality of illumination sub-patterns has a peak at the substantially hemispherical end,
The three-dimensional shape measuring apparatus according to claim 6. The illumination means has a substantially hemispherical light emitting region,
Each of the plurality of illumination sub-patterns is a pattern whose emission intensity changes in a different direction from each other.
The three-dimensional shape measuring apparatus according to any one of claims 3 to 5. The illumination means has a substantially hemispherical light emitting region,
One illumination sub-pattern is a pattern having uniform light emission intensity in the entire light emission region,
Each of the other illumination sub-patterns is a pattern whose emission intensity changes in a different direction from each other.
The three-dimensional shape measuring apparatus according to any one of claims 3 to 5. The measurement object is irradiated with light by an illuminating device, and the measurement object is imaged by an imaging unit in a state where the light is irradiated. From the feature amount of reflected light at each position of the captured image, the surface of the measurement object is measured. A calibration method for a three-dimensional shape measuring apparatus for obtaining a normal direction,
Obtaining a light source vector of an illumination pattern of the illumination device;
A surface having a plurality of reflection characteristics (the angle from the specular reflection direction and the ratio of the light intensity of the specular lobe to the specular spike; the same applies hereinafter) and a plurality of normal directions in combination . The feature quantity of the reflected light from the surface when the illumination pattern is irradiated with light is calculated by solving a rendering equation,
Create a reference table that correlates the reflection characteristics and normal direction with feature values.
Calibration method. The acquisition of the light source vector is performed by arranging an object whose surface inclination is known, irradiating light with the illumination device, capturing an image with the imaging unit, and acquiring a feature amount at each position of the object.
The calibration method according to claim 10. A three-dimensional shape measurement method for measuring a surface shape of a measurement object whose reflection characteristics are unknown,
An imaging step of imaging the measurement object in a state where the measurement object is irradiated with light using illumination means for irradiating the measurement object with light with a predetermined illumination pattern;
A surface having a plurality of reflection characteristics (the angle from the specular reflection direction and the ratio of the light intensity of the specular lobe to the specular spike; the same applies hereinafter) and a combination of the plurality of normal directions . Storing in advance a reference table for storing feature quantities relating to reflected light from the surface when the illumination pattern is irradiated with light;
A feature amount calculation step of calculating the feature amount at each position of the measurement object from the image captured in the imaging step;
Based on the feature amount calculated in the feature amount calculation step and the reference table, a shape calculation step for obtaining a normal direction of the surface at each position of the measurement object;
A three-dimensional shape measuring method.

Images