JP2015232487A - Image inspection device, image inspection method, image inspection program and computer readable recording medium, and apparatus having image inspection program recorded therein - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image inspection device that applies a photometric stereo method enabling accurate and quick estimation of the azimuth and zenith angle of illumination, and a method of detecting an illumination direction.SOLUTION: An image inspection device includes: imaging means including imaging means 11 for photographing a workpiece WK from a fixed direction; illumination means including first illumination means 21 to fourth illumination means 24 for illuminating the workpiece WK from different directions at least three times; illumination control means for lighting the first illumination means 21 to the fourth illumination means 24 one by one in order; source image creation means for creating a plurality of source images by driving the imaging means 11 at respective illumination timings; and illumination direction estimation means 41g for detecting illumination directions that best match the actual illumination directions of the first illumination means 21 to the fourth illumination means 24, from the plurality of source images SP based on the plurality of source images created by the source image creation means.

Description

  The present invention relates to an image inspection apparatus, an image inspection method, an image inspection program, and a computer-readable recording medium or recorded apparatus using a photometric stereo method.

  2. Description of the Related Art Image inspection apparatuses that perform inspections such as the presence or absence of scratches on the surface of a workpiece (inspection object or subject), appearance shape, and reading of printed characters are used. Such an image inspection apparatus performs necessary illumination on a workpiece, captures an image, performs image processing such as edge detection on the obtained image data, and determines pass / fail judgment based on the result. Had gone.

  However, in such an image inspection apparatus, depending on the type of work, there are cases in which the visibility changes depending on the type of illumination and the direction of application. For this reason, sufficient knowledge and experience are required to perform an appropriate inspection on such a workpiece.

  Further, in the conventional image inspection apparatus, there is a problem that erroneous detection is likely to occur only when the illumination condition, installation condition, and the like are slightly changed, and it is difficult to detect stably. Furthermore, in the appearance inspection of workpieces, both information on the shape of the workpiece such as scratches and edges, in other words, three-dimensional information, and planar information such as printing and stains are subject to inspection. As a result, there was a problem that it could not be detected well.

  As a technique for solving such a problem, an image processing apparatus that acquires height information using a photometric stereo method is known (see, for example, Patent Document 1). The photometric stereo method (photometric stereo) is also called illuminance difference stereo method, which is a three-dimensional measurement method that captures images illuminated from different illumination directions and calculates the normal vector of the workpiece from the shadow information. One of the methods. An image processing apparatus using such a photometric stereo method is an image obtained by replacing the X-axis and Y-axis components with luminance values from a normal vector (equivalent to an inclination image), or a reflectance image (equivalent to an albedo image). Is applied to image inspection. Here, in order to accurately perform the three-dimensional measurement by the photometric stereo method, studies are mainly made on the installation method of the illumination device and the illumination light irradiation method.

JP 2007-206797 A

  In the photometric stereo method, in order to determine the inclination of the workpiece (subject) surface, albedo (reflectance), and the like, it is necessary to perform a plurality of images from a plurality of illumination directions with different illumination directions. Here, the illumination direction can be represented by an azimuth angle (FIG. 12B) and a zenith angle (FIG. 12A) with respect to the subject. Therefore, in the photometric stereo method, processing is normally performed on the assumption that the illumination directions of each of a plurality of captured images are known.

  Here, the image processing apparatus disclosed in Patent Document 1 is an integral type in which the camera and illumination used in the photometric stereo method cannot be separated. Here, when the camera and the illumination are separated, the illumination direction with respect to the camera can be freely changed according to the positional relationship of the installation. Also, the separation increases the degree of freedom of installation, and the illumination can be installed so that the subject is not physically interfered by obstacles other than the subject. It is also possible to select a zenith angle or azimuth angle at which specular reflection light that adversely affects the results does not enter the camera.

  However, as described above, the technique using the photometric stereo method is implemented on the assumption that the illumination direction is known. Here, a case is considered in which the azimuth angles of the respective illumination directions are different, but the zenith angle can be regarded as not largely changing.

  When the zenith angle given to the photometric stereo processing is different from the actual value, the obtained inclination of the surface is a value obtained by multiplying the actual value by a certain ratio. On the other hand, if the azimuth angle given to the photometric stereo processing is different from the actual value under a condition where the zenith angle is constant, it is not a simple change in proportional relationship, and the actual subject surface inclination is obtained. The correlation with the inclination of the surface is greatly deviated.

  From the above, the photometric stereo method has little influence on the recognition performance of the surface shape of the workpiece even if the accuracy for the zenith angle is poor, when the illumination is installed so that each zenith angle is almost constant, If the accuracy with respect to the azimuth is poor, the surface shape recognition performance is greatly adversely affected. That is, when the preset illumination direction and the actual illumination direction are significantly different, there is a possibility that the shape of the subject surface is mistaken.

  Therefore, a technique using a conventional photometric stereo method is a method in which the user individually sets the azimuth angle and zenith angle with respect to the subject for each illumination.

  However, if the user individually sets the azimuth angle and the zenith angle with respect to the subject for each illumination having a large number of illumination directions, a complicated setting operation is imposed due to a complicated development operation. Even if (Interface) is implemented, the possibility of causing a setting error cannot be denied, and there is still a possibility that the actual shape of the subject surface is mistaken. In particular, since the feature of the surface of the subject is displayed even if the setting is slightly wrong, it is difficult for the user to notice the mistake.

  In order to solve this problem, a method of automatically estimating the direction of installed illumination without forcing the user to perform a setting operation can be considered. However, when the photometric stereo method is considered to be applied to various inspections assumed in FA equipment, the calculation amount becomes enormous due to the fact that it is based on a plurality of images taken from different illumination directions. This process may be delayed, making real-time inspection difficult. In addition, it is also required to improve the direction estimation processing speed and accuracy so as to meet the constraints of installation of cameras and lighting when applied to FA (Factory Automation) equipment.

  The present invention has been made in view of such circumstances, and its main purpose is an image applying a photometric stereo method that makes it possible to accurately estimate the azimuth angle and zenith angle of illumination at high speed. It is an object to provide an inspection apparatus, an image inspection method, an image inspection program, and a computer-readable recording medium or recorded device.

Means for Solving the Problems and Effects of the Invention

  According to the image inspection apparatus according to the first aspect of the present invention, there is provided an image inspection apparatus for inspecting the appearance of a work, and three or more illumination means for illuminating the work from different actual illumination directions. The lighting control means for turning on the three or more lighting means one by one in a predetermined lighting order, and the lighting timing at which each lighting means is turned on by the lighting control means, to image the workpiece from a certain direction, Photometric using pixel values for each pixel having a correspondence relationship between an imaging unit for capturing a plurality of partial illumination images with different actual illumination directions and a plurality of partial illumination images captured by the imaging unit Based on the stereo method, a source image is generated to generate a source image of the workpiece surface in the assumed illumination direction assuming the illumination direction for each of a plurality of different assumed illumination directions. And an illumination direction estimating means for detecting an estimated illumination direction that most closely matches an actual illumination direction in the illumination means based on a plurality of source images generated for each assumed illumination direction by the source image generation means. Can be provided. With the above configuration, even when the actual illumination direction is unknown on the image inspection apparatus side, the actual illumination direction can be estimated as the estimated illumination direction based on a plurality of source images with different assumed illumination directions. Become. As a result, it is possible to perform image inspection based on the photometric stereo method without strictly arranging the illumination means, and there is an advantage that the installation work of the illumination means, which has conventionally been troublesome, can be greatly simplified. In particular, since the actual illumination direction installed by the user can be automatically estimated, even when the user performs image inspection, even if the imaging means and illumination means are roughly installed, a fine setting operation is forced. Therefore, it is possible to perform image inspection under optimum illumination conditions.

  Further, according to the image inspection apparatus according to the second aspect, the three or more illuminating units are integrally configured, and the actual illuminating is performed on the virtual rotation plane centered on the optical axis of the imaging unit. The illumination direction estimation means can estimate an estimated illumination direction from a plurality of assumed illumination directions assumed to be located on the virtual rotation plane. With the above configuration, the actual illumination direction that can be taken on the virtual rotation surface can be estimated by the illumination direction estimation means.

  Furthermore, according to the image inspection apparatus according to the third aspect, the three or more illumination units are configured to be arranged independently of each other, and on a virtual rotation plane centered on the optical axis of the imaging unit. The illumination direction estimating means estimates the estimated illumination direction from a plurality of assumed illumination directions assumed to be located on the virtual rotation plane, and the three or more illuminations Each installation position of the means can be estimated. With the above configuration, even when a plurality of illumination means are replaced and arranged, the arrangement position of each illumination means and the actual illumination direction can be estimated by the illumination direction estimation means. .

  Furthermore, according to the image inspection apparatus according to the fourth aspect, the illumination direction estimating means can estimate the illumination illumination direction assuming that the surface shape of the workpiece is convex.

  Still further, according to the image inspection apparatus according to the fifth aspect, the source image generation means generates the plurality of source images using a zenith angle and an azimuth angle that specify an illumination direction as variables, The illumination direction estimation means determines a source image evaluated as the highest in accordance with a first evaluation function indicating a degree of matching with the source image in the actual illumination direction among the source images generated by the source image generation means. Once selected, the zenith angle and azimuth variable values in the source image can be estimated as the estimated illumination direction. With the above configuration, it is possible to select the source image that is evaluated as the highest rank, and to detect the zenith angle and azimuth angle variable values in the source image as the actual illumination direction.

  Furthermore, according to the image inspection apparatus according to the sixth aspect, the first evaluation function is calculated from the source image calculated by using the zenith angle and the azimuth as variables, with respect to the zenith angle and the azimuth. A function obtained by quantifying the amount of deviation between the surface shape of the workpiece in the source image and the actual surface shape in the source image can be obtained. With the above configuration, when the value of the first evaluation function is minimized, the degree of matching with the source image in the actual illumination direction is maximized. Therefore, the variable values of the zenith angle and azimuth angle in the source image at this time are actually Can be detected as the illumination direction.

  Furthermore, according to the image inspection apparatus according to the seventh aspect, the source image generation means uses the zenith angle as a constant among the zenith angle and the azimuth angle that specify the illumination direction. A second evaluation function that indicates a degree of matching with the source image according to an actual illumination direction among the source images generated by the source image generation unit; Thus, it is possible to select the source image evaluated as the highest rank and estimate the azimuth angle variable value and the zenith angle constant value in the source image as the estimated illumination direction. With the above configuration, it is possible to select the source image that is evaluated as the highest rank, and to detect the zenith angle and azimuth angle variable values in the source image as the actual illumination direction. Also, since the difference in zenith angle is not greatly reflected in the source image, assuming a constant value, the azimuth angle and zenith angle of the illumination direction are estimated with high accuracy and high speed, and photometric stereo processing is performed at high speed. It becomes possible.

  Furthermore, according to the image inspection apparatus according to the eighth aspect, the second evaluation function has a zenith angle as a constant and a azimuth angle as a variable. The expression ratio of the convex / concave component of the workpiece surface can be expressed as a function. With the above configuration, when the value of the second evaluation function is maximized, the degree of matching with the source image in the actual illumination direction is maximized. Therefore, the variable values of the zenith angle and azimuth angle in the source image at this time are actually Can be detected as the illumination direction.

  Furthermore, according to the image inspection apparatus according to the ninth aspect, the azimuth angle as a variable can be set from a predetermined combination. With the above configuration, the azimuth angle as a variable is set from predetermined combinations, so that photometric stereo processing can be performed at higher speed.

  Furthermore, according to the image inspection apparatus according to the tenth aspect, the azimuth angle as a variable can be set with a uniform angular width. With the above configuration, the azimuth angle as a variable is set with a uniform angular width, so that photometric stereo processing can be performed at higher speed.

  Furthermore, according to the image inspection apparatus according to the eleventh aspect, the illumination direction estimating means estimates the workpiece as a hemisphere including a specular reflection component, and further, for each source image, in the source image of the workpiece. From the center coordinates of the base point, obtain a vector of halation coordinates in the center of the specular reflection component image from the workpiece, calculate the direction of the vector as the azimuth of the illumination direction when generating the source image, From the ratio of the number of pixels of the vector and the number of pixels of the workpiece size, the zenith angle in the illumination direction when the source image is generated can be calculated. With the above configuration, calculation processing using a simple algorithm is sufficient without performing calculation processing using the photometric stereo method, so the azimuth angle and zenith angle of the illumination direction can be estimated with high accuracy and high speed. The subsequent photometric stereo processing can be quickly transferred.

  Still further, according to the image inspection apparatus according to the twelfth aspect, the illumination direction estimating means sets the workpiece height as known, and for each source image, uses the center coordinates in the source image of the workpiece as a base point. The vector of the shadow coordinate at the tip of the shadow in the source image is calculated, the direction of the vector is calculated as the azimuth angle of the illumination direction when the source image is generated, and the number of pixels in the vector and the height of the workpiece From the ratio to the number of pixels, the zenith angle in the illumination direction when the source image is generated can be calculated. With the above configuration, calculation processing using a simple algorithm is sufficient without performing calculation processing using the photometric stereo method, so the azimuth angle and zenith angle of the illumination direction can be estimated with high accuracy and high speed. The subsequent photometric stereo processing can be quickly transferred.

  Furthermore, according to the image inspection apparatus according to the thirteenth aspect, the illumination direction estimation means has a preset zenith angle and / or azimuth angle in a state where the imaging means and the illumination means are arranged. The setting range can be compared with the estimated zenith angle and / or azimuth angle, and a warning can be generated if the setting range is not met. With the above configuration, the user can notice an error in the installation or setting of the illumination or imaging means, and usability is improved.

  Furthermore, according to the image inspection apparatus according to the fourteenth aspect, the illumination control unit can arbitrarily change the lighting order when generating the source image for each different assumed illumination direction.

  Furthermore, according to the image inspection apparatus according to the fifteenth aspect, the source image generation unit uses a pixel value for each pixel having a correspondence relationship between the plurality of partial illumination images captured by the imaging unit. Based on the photometric stereo method, a normal vector calculating means for calculating a normal vector for the surface of the work of each pixel, and a normal vector of each pixel calculated by the normal vector calculating means, And a contour image generating means for performing a differentiation process in the X direction and the Y direction to generate a contour image indicating a contour of the inclination of the surface of the workpiece. With the above configuration, the normal vector is subjected to differentiation processing to generate a contour image indicating the contour of the workpiece surface tilt, and this contour image is applied as an image to be inspected, thereby affecting the influence of the workpiece or illumination tilt. Image inspection that is harder to receive and more robust to the environment than conventional height measurement using photometric stereo can be realized. In particular, the first-order inclination image is generated using the photometric stereo method, and the obtained first-order inclination image is differentiated in the X direction and the Y direction to obtain the second order. Since it is configured to create a tilt image, that is, a contour extraction image, it is possible to more easily and robustly inspect for scratches and prints on the workpiece. In addition to the effects of applying such a photometric stereo method, even when the user performs image inspection using the photometric stereo method, even if imaging means and illumination are roughly installed, detailed setting operations There is also an effect that the image inspection can be performed under the optimal illumination condition without being forced to perform the above. Furthermore, since it is usually difficult to distinguish between the original scratch and the original contour, it is required to search for an area that should not have a scratch as an inspection area. After the inspection area is determined, the scratch inspection can be performed. Also, when performing OCR, it is possible to perform OCR after performing a search on a texture extracted image and determining a target area for performing OCR.

  Furthermore, according to the image inspection apparatus according to the sixteenth aspect, the inspection area specifying means for specifying the position of the inspection area to be inspected with respect to the generated contour image, and the inspection area specifying means Image processing means for performing image processing for detecting a flaw in the specified inspection area, and determination means for judging the presence or absence of a flaw on the workpiece surface based on the processing result of the image processing means Can be provided. With the above configuration, by providing a flaw inspection tool necessary for flaw inspection after generating the contour extraction image, it is a common common perception that photometric stereo technology is one method of three-dimensional measurement, It is possible to provide an image inspection apparatus that is positioned as a practical product that applies photometric stereo technology to scratch inspection.

  The image inspection method according to the seventeenth aspect is an inspection method for inspecting an appearance by capturing an image of a workpiece, wherein the workpiece is subjected to three or more different illumination means one by one in a predetermined lighting order. And illuminating at a predetermined lighting timing from different actual illumination directions, and using a common imaging means, one partial illumination image is captured for each actual illumination direction, and the actual illumination direction is determined. Based on the photometric stereo method, using a pixel value for each pixel in a correspondence relationship between a step of obtaining a plurality of different partial illumination images and a plurality of partial illumination images obtained by illumination with different illumination means, Generating a source image of the surface of the workpiece in the assumed illumination direction assuming the illumination direction for each of a plurality of different assumed illumination directions by the source image generating means; and Based on the plurality of source images generated for each constant illumination direction, with the illumination direction estimation means may include a step of detecting an estimated illumination direction most consistent with the actual illumination direction in the illumination means. Accordingly, even when the actual illumination direction is unknown on the image inspection apparatus side, the actual illumination direction can be estimated as the estimated illumination direction based on a plurality of source images having different assumed illumination directions. . As a result, it is possible to perform image inspection based on the photometric stereo method without strictly arranging the illumination means, and there is an advantage that the installation work of the illumination means, which has conventionally been troublesome, can be greatly simplified.

  Furthermore, the image inspection program according to the eighteenth aspect is an inspection program for inspecting an appearance by capturing an image of a workpiece, and the computer is provided with three or more illumination means different from each other. Illuminate in a predetermined lighting order one by one and illuminate at a predetermined lighting timing from different actual illumination directions, and use a common imaging means to capture one partial illumination image for each actual illumination direction, Photometric using the pixel value of each pixel that has a correspondence relationship between the function of acquiring multiple partial illumination images with different actual illumination directions and the multiple partial illumination images acquired by illuminating with different illumination means Function to generate source image of workpiece surface in hypothetical illumination direction assuming illumination direction based on stereo method for each of multiple different hypothetical illumination directions by source image generation means A function of detecting an estimated illumination direction that most closely matches an actual illumination direction in the illumination means, based on a plurality of source images generated for each assumed illumination direction by the source image generation means. Can be realized. Thereby, even if the actual illumination direction is unknown on the image inspection apparatus side, the actual illumination direction can be estimated based on a plurality of source images having different assumed illumination directions. As a result, it is possible to perform image inspection based on the photometric stereo method without strictly arranging the illumination means, and there is an advantage that the installation work of the illumination means, which has conventionally been troublesome, can be greatly simplified.

  A computer-readable recording medium or a stored device according to the nineteenth aspect of the present invention stores the image inspection program. CD-ROM, CD-R, CD-RW, flexible disk, magnetic tape, MO, DVD-ROM, DVD-RAM, DVD-R, DVD + R, DVD-RW, DVD + RW, Blu-ray (product) Name), HD DVD (AOD), and other magnetic disks, optical disks, magneto-optical disks, semiconductor memories, and other media that can store programs. The program includes a program distributed in a download manner through a network line such as the Internet, in addition to a program stored and distributed in the recording medium. Furthermore, the stored devices include general-purpose or dedicated devices in which the program is implemented in a state where it can be executed in the form of software, firmware, or the like. Furthermore, each process and function included in the program may be executed by computer-executable program software, or each part of the process or hardware may be executed by hardware such as a predetermined gate array (FPGA, ASIC), or program software. And a partial hardware module that realizes a part of hardware elements may be mixed.

It is a block diagram which shows the structure of the image inspection apparatus which concerns on Embodiment 1 of this invention. It is a schematic plan view which shows the positional relationship of the imaging means and illumination means of the image inspection apparatus of FIG. It is a model side view which shows the positional relationship of the imaging means and illumination means of the image inspection apparatus of FIG. It is a schematic diagram which shows the illumination means which implement | achieves the four illumination blocks which concern on Embodiment 2. FIG. It is a schematic plan view which shows a mode that five illumination blocks are implement | achieved with the illumination means of FIG. It is a schematic plan view which shows the illumination means which concerns on Embodiment 3. It is a schematic plan view which shows the illumination means which concerns on Embodiment 4. It is a schematic plan view which shows the illumination means which concerns on a modification. It is a schematic plan view which shows the illumination means which concerns on another modification. FIG. 10 is a schematic diagram illustrating a configuration of an image inspection apparatus according to a fifth embodiment. FIG. 10 is a schematic diagram illustrating a configuration of an image inspection apparatus according to a sixth embodiment. It is a block diagram with which it uses for description of the signal processing system of an image processing part. FIG. 13A is a diagram showing a positional relationship between the diffuse reflection surface S and illumination for explaining the basic principle of the photometric stereo method, FIG. 13B is a diagram showing a state in which light is irradiated from L1, and FIG. 13C is a diagram from L2. The figure which shows the state which irradiated the light, FIG. 13D is a figure for demonstrating estimating the direction of the surface from the combination of an irradiation direction and the brightness of reflected light. FIG. 14A is a diagram showing an example of a tilt image differentiated in the vertical direction, FIG. 14B is a diagram showing an example of a tilt image differentiated in the horizontal direction, and FIG. It is a figure which shows an example. 15A is a diagram showing surface information for explaining a calculation method of δ ² s / δx ² and δ ² s / δy ² , FIG. 15B is a diagram showing an inclination image, and FIG. 15C is a diagram showing a forward difference, FIG. 15D is a diagram showing the central difference. FIG. 16A is a diagram showing a source image for explaining a generation method of a texture extraction image, FIG. 16B is a diagram showing an image processed by the averaging method, and FIG. 16C shows an image processed by the halation removal method FIG. It is a figure where it uses for description of angle noise reduction. It is a schematic cross section which shows a mode that it interferes with an obstruction, when illumination and a camera are integrated. It is a schematic diagram which shows a mode that a camera blocks a part of illumination light. It is a schematic cross section which shows the structure which made the illumination and the camera a separated type. FIG. 21A is a diagram illustrating a case where the LWD is set small, and FIG. 21B is a diagram illustrating a case where the LWD is set large, for explaining the merit that the illumination and the camera are separated. It is a schematic diagram which shows a mode that ring-shaped illumination is arrange | positioned. It is a schematic diagram which shows a mode that ring-shaped illumination and the rotation position of a camera are made to correspond. It is a figure for demonstrating the definition of a zenith angle and an azimuth. FIG. 25A is a contour extraction image obtained with a zenith angle and an azimuth angle that matches the assumed illumination direction, FIG. 25B is a contour extraction image obtained with an incorrect zenith angle and a correct azimuth angle, and FIG. 25C is a correct zenith angle and error. FIG. 25D is an image diagram showing a contour extraction image obtained with a correct zenith angle and an incorrect azimuth angle (shuffle). It is a figure for demonstrating that there exists rotation as a way of making a mistake in lighting installation. 27A is obtained by rotating at an angular width of 45 °, assuming an illumination direction that matches the assumed illumination direction as 0 °, 0 °, FIG. 27B is 45 °, FIG. 27C is 90 °, FIG. 27D is 135 °, FIG. 27E is an image diagram showing a contour extraction image obtained at an azimuth angle of 180 °, FIG. 27F at 225 °, FIG. 27G at 270 °, and FIG. 27H at 315 °. It is a figure for demonstrating that there is shuffle as a way of making a mistake in lighting installation. FIG. 29A is a contour extraction image with an illumination direction that matches the assumed illumination direction, and FIGS. 29B to 29F are image diagrams illustrating the contour extraction images generated in all combinations of shuffles. It is the graph which calculated and averaged the height with respect to a rotation angle from the source image obtained by rotating about various workpieces. It is the graph which computed and averaged the height with respect to a rotation angle from the source image acquired with respect to all the combinations of shuffle. It is a figure for demonstrating that there exists a shift | offset | difference from a cross axis (xy axis | shaft) as a way of making a mistake in lighting installation. It is a figure for demonstrating that there exists a zenith angle asymmetrical deviation x2 as a mistake of lighting installation. It is a figure for demonstrating that there exists a zenith angle symmetrical change x2 as a mistake of lighting installation. FIG. 35A is a diagram for explaining how to determine the zenith angle θ for explaining a method for detecting the actual illumination direction when the work is hemispherical and specularly reflects, and FIG. 35B is a diagram for explaining how to obtain the azimuth angle. is there. FIG. 36A is a diagram for explaining how to find the zenith angle θ for explaining a method for detecting the actual illumination direction when the height of the workpiece is known, and FIG. 36B is a diagram for explaining how to find the azimuth angle. is there. It is a flowchart which shows the flow of the detection process of the illumination direction using the photometric stereo method. It is a flowchart which shows the flow of the detection process of the illumination direction which does not utilize a photometric stereo method. It is a flowchart which shows the image inspection method which concerns on an Example. It is a flowchart which shows the image inspection method which concerns on a modification.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the embodiment described below is an example for embodying the technical idea of the present invention, and the present invention is not limited to the following. Further, the present specification by no means specifies the members shown in the claims to the members of the embodiments. In particular, the dimensions, materials, shapes, relative arrangements, and the like of the component parts described in the embodiments are not intended to limit the scope of the present invention unless otherwise specified, and are merely explanations. It is just an example. Note that the size, positional relationship, and the like of the members shown in each drawing may be exaggerated for clarity of explanation. Furthermore, in the following description, the same name and symbol indicate the same or the same members, and detailed description thereof will be omitted as appropriate. Furthermore, each element constituting the present invention may be configured such that a plurality of elements are constituted by the same member and the plurality of elements are shared by one member, and conversely, the function of one member is constituted by a plurality of members. It can also be realized by sharing.
(1. Configuration of the image inspection apparatus 1)

  FIG. 1 shows a block diagram of an image inspection apparatus according to Embodiment 1 of the present invention. The image inspection apparatus 1 shown in this figure includes an imaging unit 11 that images a workpiece (hereinafter also referred to as “work WK”) from a certain direction, and an illumination unit that illuminates the workpiece WK from three or more different illumination directions. A lighting control unit 31 for lighting each lighting unit one by one in the lighting order, and an image processing unit 41 connected to the lighting control unit and the imaging unit to control them. The image processing unit 41 and the imaging unit are connected via the imaging cable 12, and the image processing unit 41 and the illumination control unit 31 are connected via the illumination cable 32, respectively. The image processing unit connects the display unit 51 and the operation unit 61. Furthermore, the image processing unit can be connected to an external device such as a PLC (Programmable Logic Controller) or a computer as necessary.

  The imaging unit captures a plurality of partial illumination images having different illumination directions by imaging the workpiece WK from a certain direction at an illumination timing at which each illumination unit is turned on by the illumination control unit.

  The image processing unit uses a pixel value for each pixel having a correspondence relationship between a plurality of partial illumination images captured by the imaging unit, and based on the photometric stereo method, the image processing unit It functions as source image generation means for generating a source image of the surface for each of a plurality of different assumed illumination directions. Specifically, the source image generation means includes a normal vector calculation means 41a, a contour image generation means 41b, a texture extraction image generation means 41c, an inspection region specification means 41d, an image processing means 41e, and a determination means 41f. And the function of the illumination direction estimation means 41g is implement | achieved. The normal vector calculation unit 41a calculates a normal vector n with respect to the surface of the work WK of each pixel by using pixel values for each pixel having a correspondence relationship among the plurality of partial illumination images captured by the imaging unit. . The contour image generation unit 41b performs a differentiation process on the calculated normal vector n of each pixel in the X direction and the Y direction to generate a contour image indicating the contour of the inclination of the workpiece WK surface. The texture extraction image generating unit 41c calculates the albedo of each pixel from the calculated normal vector n of each pixel that is illuminated by the illuminating unit, and removes the influence of the surface inclination of the workpiece WK from the albedo. A texture extraction image showing the processed pattern is generated. The inspection area specifying unit 41d specifies the position of the inspection area to be inspected with respect to the generated contour image. The image processing unit 41e performs image processing for detecting a flaw in the specified inspection area. The determination unit 41f determines the presence or absence of a scratch on the surface of the workpiece WK based on the processing result. The illumination direction estimation unit 41g detects an illumination direction that most closely matches the actual illumination direction in the illumination unit from a plurality of source images SP (described later) based on the plurality of images generated by the source image generation unit.

  The imaging unit and the illumination unit can be arranged as separate members. This enables a layout with a high degree of freedom. As an example, as shown in the schematic plan view of FIG. 2 and the schematic side view of FIG. 3, the imaging unit 11 with the optical axis directed in the vertical direction is disposed directly above the work WK placed on the stage SG. . Further, the four illumination means, that is, the first illumination means 21, the second illumination means 22, the third illumination means 23, and the fourth illumination means 24 are arranged at the same height in the east, west, north, and south directions around the imaging means 11. Deploy. The positional relationship between the imaging unit arranged and each illumination unit is recorded on the image inspection apparatus side. Each illumination means is sequentially turned on at a predetermined illumination timing by the illumination control means, and the workpiece is imaged from a certain direction with the common imaging means to obtain a partial illumination image.

Note that the imaging unit and the illumination unit are not limited to being configured as separate members, and may be configured integrally via an arm or the like. In this case, since the positional relationship between the imaging unit and the illumination unit is fixed in advance, it is possible to eliminate the adjustment work of matching the optical axes. However, the degree of freedom is lost.
(Imaging means)

As the imaging means 11, for example, an imaging element such as a CCD (Charge Coupled Device) camera or a CMOS (Complementary Metal Oxide Semiconductor) imager can be used. The image of the subject is photoelectrically converted by the imaging device to output an image signal, and the output image signal is converted into a luminance signal and a color difference signal by the signal processing block, and the image processing unit 41 connected by the imaging cable 12 is connected. Output.
(Lighting means)

  The illumination means 21, 22, 23, and 24 are arranged so as to surround the work WK as shown in the schematic plan view of FIG. 2 so that the work WK can be irradiated with illumination light from different illumination directions. . Further, as shown in the schematic side view of FIG. 3, each illuminating means is installed with its optical axis obliquely downward. It is preferable that the optical axis of the imaging unit and the central axis of the plane (virtual rotation plane) provided with the illumination unit coincide with each other so that the workpiece illuminated by each illumination unit can be imaged by the common imaging unit. Moreover, it is preferable to arrange | position 360 degrees equally as the space | interval (azimuth angle from a central axis) between illumination means, dividing | segmenting equally by the number of illumination means. Furthermore, the zenith angle is preferably constant for all illumination means. Furthermore, it is preferable that the distance between each illumination means and the workpiece is also constant. By doing so, it is possible to simplify the input of azimuth and zenith angle information necessary for the calculation of photometric stereo processing. As will be described later, when capturing the all-lamp image MC, since all the illumination means are imaged in the all-lamp state, the above-described state reduces the intensity of all the illuminations evenly and reduces unevenness. Images can be taken under illumination.

In the example of FIG. 1, the first illumination unit 21, the second illumination unit 22, the third illumination unit 23, and the fourth illumination unit 24 are configured. Each illumination means can use an incandescent bulb or a fluorescent lamp. In particular, a semiconductor light emitting element such as a light emitting diode (LED) is preferable because it consumes less power and has a long life and excellent responsiveness. As shown in FIG. 1, each lighting unit is connected to the lighting branching unit 75 via cables 71, 72, 73, 74 corresponding to each lighting unit, and further connected to the lighting control unit 31 via a cable 76. ing.
(Lighting branch unit)

  The illumination branching unit is an interface for connecting each illumination unit to the illumination control unit. Specifically, an illumination connection connector for connecting an illumination cable extended from the illumination means is provided. In the example of FIG. 1, four illumination connection connectors are provided so as to connect the four illumination means. At this time, a mark or the like may be provided as installation assistance means so that the illumination cable is correctly connected to the illumination connection connector. By correctly connecting the illumination table of each illumination means to the illumination connection connector of the illumination branch unit, the illumination means can be lit in the correct direction in the correct order by the illumination control means, and the imaging means can be synchronized with each illumination timing. A partial illumination image can be taken by operating. The illumination branching unit is provided separately from the illumination control unit in the example of FIG. 1, but is not limited to this configuration. For example, the illumination branching unit may be incorporated in the illumination control unit.

  The illumination colors of the illumination means 21, 22, 23, and 24 can be changed according to the type of the work WK. For example, when it is desired to inspect a fine scratch, blue having a short wavelength is suitable. When inspecting a colored workpiece, it is preferable to use white illumination so that the illumination color does not get in the way. Furthermore, when oil is attached to the workpiece, red illumination may be employed to prevent the influence.

  In the example of FIGS. 1 to 3 described above, the number of illumination means is four. However, at least three illumination means are sufficient so that the workpiece WK can be illuminated from three or more different illumination directions. When the number of illumination means is increased, partial illumination images can be obtained from more illumination directions, so that the accuracy of image inspection can be improved. For example, you may arrange | position eight in total including the northeast, northwest, southeast, and southwest direction. Moreover, it is preferable to arrange | position 360 degrees equally as the space | interval (azimuth angle from a central axis) between illumination means, dividing | segmenting equally by the number of illumination means. Furthermore, the zenith angle is preferably constant for all illumination means. Note that the amount of processing increases as the number of images to be processed increases, and the processing time is delayed. In the present embodiment, the number of illumination means is set to four as described above in consideration of the balance between processing speed, ease of arithmetic processing, accuracy, and the like.

  Further, the illumination means can be constituted by a plurality of light emitting elements arranged in an annular shape. For example, in the ring illumination according to the second embodiment illustrated in FIG. 4, the annular light emitting element is divided into four illumination blocks. The illumination control means uses the first illumination block as the first illumination means, the second illumination block as the second illumination means, the third illumination block as the third illumination means, and the fourth illumination block as the fourth illumination means. By making the illumination timing of the blocks different, it can be controlled in the same way as there are four separate illumination means.

  Further, with this configuration, there is an advantage that the number of illumination means can be arbitrarily changed using the same ring illumination. That is, when the lighting control means can arbitrarily control the lighting of each light emitting element, as shown in FIG. 5, the illumination block is divided so that the circumference of the annularly arranged light emitting elements is divided into 4 sections to 5 sections. The partial illumination images from the five illumination directions can be acquired by controlling the illumination control unit so that the partial illumination images are captured by shifting the illumination timing for lighting the five illumination blocks. Similarly, if the annular circumference is changed to 6 divisions and 7 divisions, the illumination direction can be further increased. Moreover, you may change an illumination block for every period, not only the structure which acquires a partial illumination image with a constant illumination block at all times. In this way, by adjusting the lighting pattern of each light emitting element, the same effect as the case of virtually changing the number of illumination blocks and adjusting the number of illumination means using the same ring illumination can be obtained. Can do. In other words, common hardware can handle different accuracies.

  Furthermore, the illumination means is arranged in an annular shape, and as shown in the schematic plan view of FIG. 6 as Embodiment 3, the illumination means configured in a bar shape is arranged in a rectangular shape, or as Embodiment 4 in FIG. As shown in the schematic plan view, they can be arranged in a polygonal shape.

  Alternatively, the illumination means can be arranged in a planar shape in addition to the circular or polygonal ring shape. For example, different illumination directions can be realized by arranging a large number of light emitting elements in a plane and changing the illumination block to be lit. Specifically, like the illumination unit according to the modification shown in FIG. 8, the illumination unit 20 ′ is configured by stacking a plurality of concentric rings, and the illumination unit is configured with each ring having a different radius as an illumination block. Configure. Alternatively, as in the illumination unit according to another modified example shown in FIG. 9, the illumination unit 20 ″ is a lighting unit in which light emitting elements are arranged in a dot matrix, and the illumination unit is divided by a plurality of line segments passing through the center. Thus, in the present invention, the illumination means and the illumination direction are not necessarily limited to physically separated illumination, and illumination is performed with an illumination block in which one illumination means is divided into a plurality of parts. It is used in the meaning including the structure.

In the present embodiment, processing is performed on the assumption that the partial illumination light from each illumination means is parallel light within the imaging range. As long as the partial illumination light is parallel light, only the direction of the illumination light (for example, any one of east, west, south, and north) becomes a problem, and its detailed position, for example, the coordinate position of the light source of the illumination means, does not need to be considered.
(Lighting control means)

  The illumination control means turns on each of the illumination means and the imaging means so that the three or more illumination means are turned on one by one in the order of lighting, and the workpiece is imaged from a certain direction by the imaging means at the illumination timing at which each illumination means is turned on. Control to synchronize. In other words, the illumination control means synchronizes the illumination timing by the illumination means and the imaging timing by the imaging means. Further, the lighting control means turns on each lighting means by lighting the lighting means arranged so as to surround the work in the order of clockwise or counterclockwise, or in a discrete order, for example, one It may be in the order of crossing every other. Normal vector images based on the photometric stereo method can be used as long as the partial illumination images captured at each illumination timing are captured in which illumination direction in any order. Can be built.

In Embodiment 1 of FIG. 1, the illumination control unit 31 is provided as a separate body from the image processing unit 41, but is not limited to this configuration. For example, it may be integrated with the image processing unit 41 as in the fifth embodiment shown in FIG. 10, or may be built in the illumination means 25 as in the sixth embodiment shown in FIG.
(Image processing unit)

  The image processing unit 41 controls the operations of the imaging unit 11 and the illumination units 21, 22, 23, and 24, and uses the image signals Q <b> 1 to Q <b> 4 of the four partial illumination images input from the imaging unit 11. A normal vector image (hereinafter referred to as an “inclined image”) of the surface at, and a second-order inclined image (hereinafter referred to as an “outline extracted image”) in the X and Y directions of the inclined image, or an albedo. (Albedo, meaning reflectance) An image (hereinafter also referred to as “texture extraction image”) is created, and processing for performing scratch inspection, character detection, and the like is performed using these images. Note that the processing for performing a flaw inspection, character detection, and the like is not limited to the configuration processed by the image processing unit 41, and can be executed by an external device such as the PLC 81, for example.

  FIG. 12 shows a signal processing system 42 of the image processing unit 41. The signal processing system 42 includes a CPU 43, a memory 44, a ROM 45, a display unit 51, an operation unit 61 such as a pointing device, an imaging unit 11, an illumination control unit 31, and a PLC (programmable logic controller) 81 that executes a result output process. These are connected to each other via a bus 46 and cables 12, 32, 52, 62, 82. The ROM 45 may be a portable medium. Further, the display unit 51 may be combined with the operation unit 61 such as a touch panel.

  The CPU 43 controls transmission / reception of data between the memory 44, the ROM 45, the display unit 51, the operation unit 61, the imaging unit 11, the illumination control unit 31, and the PLC 81 based on a program stored in the ROM 45, Control of the display means 51, the imaging means 11, and the illumination control means 31 is performed.

For example, the image processing unit 41 is assumed to be a single computer in which a program is stored. However, each unit may be configured by a combination of a plurality of computers, or some of the units may be dedicated. You may comprise by a circuit. Or it can also be set as the member designed exclusively like ASIC.
(Judgment means)

As described above, the image processing unit 41 realizes the function of the determination unit. The determination means inspects the presence / absence and size of the scratch based on the obtained texture extraction image. For example, if it is equal to or greater than a predetermined threshold, it is determined that it is a scratch. In addition, the determination unit can perform OCR based on the contour extraction image as necessary and output a recognized character string.
(Basic principle)

  Next, the basic principle of performing an appearance inspection of a workpiece using the above-described image inspection apparatus will be described in comparison with the technique of Patent Document 1 which is a conventional technique. First, the basic principle of the technique disclosed in Patent Document 1 is based on the principle of the photometric stereo method. Light is applied to an unknown surface from various directions, and the difference in reflected light of the workpiece is used. It is intended to estimate the shape of the workpiece. The reflected light of the work reflects the incident angle of the illumination, the distance of the illumination, etc., and has the property that the portion where the incident angle is 90 ° is brightest and becomes darker as the distance from the illumination increases.

  Using this property, prepare multiple illuminations with known brightness and position, switch the lighting on in order, and the brightness of the reflected light when illuminated from each direction of illumination at each point Use the difference to estimate which direction the surface is facing. Specifically, an X component image in which the X component Y component of the tilt image is replaced with the luminance of the X component and a Y component image in which the luminance of the Y component is replaced are created and applied to the inspection.

  However, this method has robustness such as a slight change in the lighting or workpiece installation surface, or an error in input information such as the position of the originally input lighting, resulting in a large change in the obtained inspection image. There was a problem of being inferior. For example, a work that has an uneven image with no actual unevenness, or that appears to be bright in the vicinity of lighting, but appears to be an image where the center of the work is raised due to changes in brightness. However, there is a drawback that an inspection image conforming to the actual shape cannot always be obtained.

  On the other hand, the basic principle of the image inspection technique according to the present embodiment is that the obtained first order is obtained while generating a first-order tilt image using the photometric stereo method. The second tilt image, that is, the contour extraction image, is generated by differentiating the tilt image in the X direction and the Y direction, and the image is inspected for scratches and the like. Even in the case where the above-mentioned drawbacks occur, the second-order tilt image has a small effect, and the portion where the change in the surface inclination is large is darkened while the portion where the change in the surface inclination is small is bright. By setting the gradation, an image suitable for extracting scratches, contours, and the like that greatly change the inclination of the surface of the workpiece is obtained.

  Further, with the technique disclosed in Patent Document 1, the reflectance image (corresponding to a texture extracted image) generated by using the photometric stereo method may cause halation and it may be difficult to detect characters. there were. On the other hand, the image inspection technique according to the present embodiment basically uses the basic principle that halation is performed at the same location with only two illuminations. By adopting the third pixel value from the top, the influence of halation is removed.

In addition, the image processing apparatus disclosed in Patent Document 1 integrally includes a camera and a light source for illumination. However, in this configuration, the camera and the light source are enlarged, and the size is restricted during installation. There was also a problem. On the other hand, in the image inspection apparatus according to the present embodiment, since the imaging unit and the illumination unit can be separate members, more flexible installation is possible in consideration of the arrangement space, which is advantageous in terms of usability. .
(Basic principle of photometric stereo method)

  Here, the basic principle of the photometric stereo method will be described with reference to FIGS. 13A to 13D. First, as shown in FIG. 13A, it is assumed that there is an unknown diffuse reflection surface S and a plurality of illuminations whose brightness and position are known (in this example, two of the first illumination unit L1 and the second illumination unit L2). To do. For example, as shown in FIG. 13B, when light is irradiated from the first illumination means L1, the diffuse reflection light on the surface of the diffuse reflection surface S is (1) brightness of illumination (known), (2) direction of illumination (known) ), (3) surface orientation of workpiece WK (normal vector n), and (4) surface albedo parameter of workpiece WK.

Therefore, as shown in FIG. 13B and FIG. 13C, each of the partial illumination images composed of the diffuse reflected light when the illumination light is projected from a plurality of different illumination directions, specifically, three or more illumination directions, is respectively imaged. Shoot with. Then, as shown in FIG. 13D, by using three or more partial illumination images as input images, the unknown (3) orientation of the workpiece WK surface (normal vector n), (4) the albedo of the workpiece WK surface, It can be calculated based on the following relational expression.
I = ρLSn

In the above formula,
ρ: albedo L: illumination brightness S: illumination direction matrix n: surface normal vector I: image gradation value

  From the above formula, when there are three illumination means, it can be expressed by the following formula.

  Moreover, when there are four illumination means, it can be expressed by the following equation.

(Normal vector n)

From the above equation, the normal vector n can be expressed by the following equation.
n = 1 / ρL · S ⁺ I

In the above formula,
If S ⁺ : square matrix, ordinary inverse matrix S ⁺ : inverse matrix of vertical matrix is Moore-Penrose pseudo inverse matrix S ⁺ = (S ^t S) ^{−1 St}
Ask for.
(Albedo)

Further, the albedo ρ can be expressed by the following equation.
ρ = | I | / | LSn |
(2-2. Outline extraction image)

Next, a method of generating a tilt image by the photometric stereo method and obtaining surface information of a workpiece such as a scratch or a contour from the obtained tilt image will be described.
(Tilt image)

First, a tilt image generation method will be described. When the curved surface of the workpiece is S, the tilt image is given by the following equation.
x direction: δs / δx, y direction: δs / δy

  Here, as an example of the tilt image, an example in which a one-yen coin is used as a work is shown in FIGS. 14A and 14B. FIG. 14A is a Y coordinate component image in the normal direction, and FIG. 14B is an X coordinate component image in the normal direction. Here, using the partial illumination images captured from the four illumination directions, the inclination image shown in FIG. 14A is differentiated in the X direction (horizontal direction in the figure) by differentiating in the Y direction (vertical direction in the figure). Thus, the tilt image shown in FIG. 14B is obtained as shown in FIG. 14B.

Here, since the scratches, contours, and the like are places where the tilt of the workpiece surface changes, the tilt image is differentiated in each direction. The second-order tilt image is given by the following equation.
x direction: δ ² s / δx ² , y direction: δ ² s / δy ²
(Outline extraction image)

From the above, the portions δ ² s / δx ² and δ ² s / δy ² of the tilt image in the x direction and the y direction are synthesized to generate a contour extracted image including the contour of the workpiece and flaw information. The contour extraction image E is given by the following equation.
E = δ ² s / δx ² + δ ² s / δy ²

In the above equation, E indicates contour information, and S indicates the curved surface of the workpiece. An example of the contour extraction image calculated from FIGS. 14A and 14B is shown in FIG. 14C. In the contour extracted image, the height is expressed by shading (brightness) of the image so that the high portion is white and the low portion is black.
(Differential synthesis method)

Examples of the differential synthesis method performed when generating the contour extracted image as described above include (1) simple addition, (2) multi-resolution, and (3) sum of squares.
(1: Simple addition)

Here, the simple addition in (1) is the sum of the differentiation of the X and Y tilt images at each pixel.
(2: Multi-resolution)

For the multi-resolution of (2), a plurality of reduced inclination images obtained by reducing the inclination image with different reduction ratios are created, and the strength of the contour is obtained by the method of (1) in each reduced inclination image. The reduction ratio is, for example, 1/1, 1/2, 1/4, 1/8, 1/16, 1/32, or the like. The plurality of reduced contour images obtained in this way are subjected to predetermined weighting and enlargement processing, and the sum of all the enlarged reduced contour images is defined as a contour extraction image. If weighting is changed here, it becomes possible to extract a flaw, outline, etc. of arbitrary thickness.
(3: sum of squares)

  Further, in the sum of squares of (3), a contour extraction image is created in which the sum of the squares of the derivatives of the X and Y tilt images is the strength of the contour. In the present embodiment, the multi-resolution (2) is adopted.

  What size is determined as a scratch varies depending on the use of the user. For example, there may be a case where a thing straddling 10 pixels is determined as a scratch, and a case where a concave part straddling 100 pixels is determined as a scratch. In some cases, only sharp edges may be extracted as edges.

  If the number of pixels in the tilt image is large, the processing results in a large scratch. Therefore, if a large scratch is to be extracted, the tilt image is reduced and then the contour strength is obtained by the method (1) and then enlarged. On the other hand, if it is desired to extract small scratches, differential synthesis may be performed by the method (1) without weighting.

  That is, for weighting, a set of weights determined in advance is prepared, all the above-mentioned types of reduced tilt images are created, and if you want to see large scratches, the results from the more reduced images are weighted. If you want to see small scratches, weight the results from images with reduced reduction.

Here, the result of adding all the enlarged reduced contour images as the contour extraction image is that the scratch is usually detected across a plurality of frequencies. This is because only the flaws detected at the limited frequency are extracted and blurred as a whole.
(Feature size)

  The weight set described above is provided with a parameter called feature size, for example, so that the finest scratches can be detected when this value is 1, and large scratches can be detected as this value is increased. When the feature size is increased and larger scratches are easily detected, the unevenness of the workpiece surface becomes clearer. Therefore, a predetermined threshold value may be provided for the feature size, and the case where the threshold value is equal to or greater than the threshold value may be used as the concavo-convex mode, depending on the feature size of the contour extraction image.

Next, a method for calculating δ ² s / δx ² and δ ² s / δy ² will be described. Examples of the calculation method include (1) forward difference, (2) center difference, and the like.
(1: Forward difference)

  In the forward difference, the horizontal gradient image Gh and the vertical gradient image Gv are input, and the pixel G (x, y) at the coordinates (x, y) of the contour image E is calculated by the following equation.

  E (x, y) = Gh (x-1, y) -Gh (x, y) + Gv (x, y-1) -Gv (x, y)

Here, the information on the scratches appearing in the contour image is shown in FIGS. 15A to 15D as schematic profiles. In these drawings, FIG. 15A shows the surface information of the workpiece, FIG. 15B shows the tilt image, FIG. 15C shows the contour image based on the forerunner difference, and FIG. 15D shows the profile of the contour image based on the center difference. In the forward difference of (1), there is an advantage that scratches in units of one pixel can be clearly seen as shown in FIGS. 15A and 15C. On the other hand, there is a demerit that an image shifted by 0.5 pixels from the original image is obtained.
(2: Central difference)

Next, a method for calculating δ ² s / δx ² and δ ² s / δy ² by the central difference will be described. The pixel G (x, y) at the coordinates (x, y) of the contour image E is calculated by the following formula using the horizontal tilt image Gh and the vertical tilt image Gv as inputs.

  E (x, y) = Gh (x-1, y) -Gh (x + 1, y) + Gv (x, y-1) -Gv (x, y + 1)

As shown in FIGS. 15A and 15D, the central difference of (2) has a merit that the coordinates are not shifted from the original image, but also has a demerit that the result is slightly blurred.
(2-3. Texture extraction image)

  Next, a method for obtaining a texture extracted image suitable for character detection or the like by removing the surface state of the workpiece from the tilt image obtained by the photometric stereo method will be described. First, the texture information is calculated from the albedo ρ on the surface of the workpiece. The albedo ρ is given by the following equation.

  ρ = | I | / | LSn |

In the above formula,
ρ: albedo L: illumination brightness S: illumination direction matrix n: surface normal vector I: image gradation value

  Note that one texture extraction image (albedo) can be obtained by the above equation ρ = | I | / | LSn |. From the normal vector obtained by this equation and N input images (partial illumination images). It is also possible to obtain N texture extracted images (albedo) and synthesize them to obtain one texture extracted image (albedo). Specific methods of synthesis include an average method, a halation removal method, and the like.

16A to 16C show examples of texture output images. In these figures, FIG. 16A shows four texture extracted images as input images, FIG. 16B shows a texture extracted image obtained by applying an average method to these images, and FIG. 16C shows a texture extracted image obtained by applying a halation removal method. Each image is shown.
(1: Average method)

In the averaging method, the average value of N ρ for each pixel is used as the pixel value in that pixel. As shown in FIG. 16B, although the entire shadow has disappeared, the portion where halation has occurred in the input image cannot be erased by the photometric stereo method, and thus the image remains affected by halation. That is, among the four input images (partial illumination images) in FIG. 16A, the whitened portions are the locations where halation occurs, and when averaged by the averaging method, unevenness is removed to some extent as shown in FIG. 16B. Although it is easy to read, some unevenness remains on the ground.
(2: Halation removal method)

  Because of the limitation of the dynamic range of the camera that is the imaging means and the variety of reflectivity of the work surface, the above equation of ρ = | I | / | LSn | It is. A halation removal method can be used to correct this error.

  Since the location where halation occurs is determined by the position of illumination, basically, it is considered that no halation occurs at the same location in the four partial illumination images. Strictly speaking, although halation may straddle two places between two directions, it can basically be said that halation does not occur at the same place with only two illuminations.

  In the halation removal method, when a texture-extracted image for each illumination direction calculated from N pieces of partial illumination images is synthesized, the partial illumination image having the highest pixel value or the first to Nth highest pixel values in each pixel. The partial illumination image is considered to have a lot of halation, and is synthesized after removing them.

  Specifically, in the present embodiment, when each pixel of the four illumination direction-specific texture extracted images is synthesized by adopting the third largest pixel value (for example, albedo value or luminance), an image shown in FIG. 16C is obtained. Thus, the influence of halation can be removed. If the fourth largest pixel value is adopted, the image is slightly dark because of the influence of shadows. Conversely, when the second largest pixel value is adopted, the influence of halation remains slightly.

  When the illumination means is in eight directions, the fifth pixel value from the top is adopted on the assumption that the influence of halation does not occur on the fifth and subsequent pixel values. In fact, according to tests conducted by the inventors, it was confirmed that the image was best when the fifth pixel value was adopted. It has also been found that the influence of shadows comes out when the sixth and subsequent pixel values are adopted.

Note that the synthesis method and averaging are not limited to these, and various methods can be used. For example, the above-described halation removal method and averaging method may be combined to sort the albedo values, adopt values in a specific order from the top, and average the third and fourth, for example.
(Feature size)

Next, details of the setting will be described. As described above, the feature size can be set when the contour extraction image is created. The contour extraction image can be an image suitable for OCR by setting the feature size to a predetermined value or more.
(3-2. Gain)

  When creating a contour extraction image or a texture extraction image, the pixel values of the original image can be multiplied by a gain in the process of generating these images.

  The gain at the time of creating the contour extraction image refers to a constant when the pixel values calculated by the calculation process are distributed in shades of 0 to 255. For example, when the value of this gain is increased when the scratches, contours, etc. are too shallow and it is difficult to grasp the scratches, contours, etc., the shading, contours, etc. are easily grasped because the change in the pixel value becomes large.

  Further, if the pixel value calculated by the calculation process exceeds the range of 0 to 255, the pixel value is adjusted to fall within the range, and if the pixel value is small, the adjustment is performed so that the pixel value is expanded to the range of 0 to 255. Will be easier to grasp.

  In the halation removal method described above, since the albedo values are sorted and, for example, the third one from the top is adopted, the brightness of the generated image cannot be predicted. For this reason, as a result of removing regular reflection, it may become darker than expected. Therefore, in order to adjust the brightness, a predetermined gain is multiplied by the pixel value when creating the texture extracted image.

Note that when calculating the tilt image, the gain can be similarly adjusted so that the pixel value falls within the range of 0 to 255.
(3-3. Noise removal filter)

When creating an inclination image, etc., calculation is performed using simultaneous equations using a plurality of images, but in reality, a differential calculation is performed. At this time, since the image data captured by the image capturing means includes noise at the time of raw data, the noise component may be emphasized and the outline may be rough when creating the tilt image. In order to reduce such noise, a noise removal filter such as a guided filter is used. With a general low-pass filter, not only noise but also information on scratches may be lost or lost, but with a guided filter, the edge can be removed by applying a guided filter when obtaining a tilt image. Noise can be removed while maintaining it, which is preferable.
(3-4. Angle noise reduction)

Next, the principle of angle noise reduction will be described based on the schematic diagram of FIG. As shown in this figure, assuming that there are two illuminations with incident angles α and β, and the workpiece WK is formed of a diffuse reflection surface, the inclination of the workpiece WK with respect to the reference plane is γ, and the direction perpendicular to the reference plane is When the angle is θ, the brightness of the reflected light from the illumination α is I _α , and the brightness of the reflected light from the illumination β is I _β , γ is given by the following equation.

γ = arctan (A · | I β -I α | / | I β + I α |), A = cotθ

The angle noise reduction is forcibly setting the slope γ to 0 when | I _β + I _α | is small to some extent.

Even if I _beta I _alpha is extremely dark, for _{example, I β = 2, I α} = 1 _{Datosuruto, | I β -I α | /} | I β + I α | = a large value of 1/3. On the other hand, bright nor I _beta I _alpha, for _{example, I β = 300, I α} = 200 _{Datosuruto, | I β -I α | /} | I β + I α | = a small value of 1/5. I _β = 2 and I _α = 1 are largely reflected in the inclination despite the possibility of simply noise, so there is angle noise reduction to reduce the influence of the noise, A threshold value of | I _β + I _α | forcibly setting the inclination to 0 can be set.
(Separate structure of illumination means and imaging means)

  In the image inspection using the photometric stereo method, it is necessary to strictly define the corresponding positions of the illumination unit and the imaging unit in advance. For this reason, in the conventional image inspection apparatus, the illumination unit and the imaging unit are configured integrally. In other words, the photometric stereo method is a measurement method that performs accurate three-dimensional measurement after strictly positioning the relative position of the imaging means and the illumination means, so it was originally installed when the illumination means and the imaging means were installed. Giving position freedom has never been done before. However, in the configuration in which the illumination unit and the imaging unit are fixed in advance, the imaging illumination unit in which these members are integrated inevitably increases in size, so that handling becomes worse.

  For example, if there is an obstacle at the inspection position, it may interfere with the imaging illumination unit and may not be installed. In particular, there are lenses of a large size, such as a line camera, a zoom lens, and a large macro lens, which are attached to a camera that is a form of the imaging means. As the imaging means becomes longer, such as mounting such a large lens on the imaging means, as shown by the arrows in FIG. 18, the obstacle OS and the camera e1 that are present around the work WK are arranged around the surroundings. The risk of interference with the illumination light sources e21 to e24 increases. Further, when the camera e1 which is an imaging means becomes longer, as shown in FIG. 19, the illumination light is partially blocked by the camera e1 or a part of the lens, and a shadow is cast on the work WK. It is also possible. In order to avoid such a situation, if the imaging illumination unit is arranged away from a position where it does not interfere with an obstacle, the distance between the illumination means and the work (Light Working Distance (LWD)) is too long. As a result, the amount of illumination light decreases, leading to a decrease in inspection accuracy.

  On the other hand, if the imaging means and the illumination means can be separated, it will be easy to arrange them at positions that do not interfere with the obstacles. For example, even when the obstacle OS shown in FIG. 18 described above is present, the first illumination means 21 to the fourth illumination means 24 are separated from the imaging means 11 so that the first illumination means 21 is not interfered as shown in FIG. The illumination means 21 to the fourth illumination means 24 can be installed. Similarly, even when a large lens is attached to the image pickup means, adjustment can be made so as to avoid physical interference between the lens 12 and the first illumination means 21 to the fourth illumination means 24. In this way, the first illumination means 21 to the fourth illumination means 24 and the imaging means 11 are separated, thereby increasing the degree of freedom of installation, such as adjustment to a position that does not interfere with obstacles, in various environments. The work WK can be inspected.

  It is also possible to adjust the mounting position of the illumination means so that the imaging means does not block the illumination light. Similarly, it is possible to adjust the arrangement position of the illumination means so as to suppress the influence of halation and shadows. In particular, since the photometric stereo method assumes that the surface of the work is a diffuse reflection surface, the normal vector of the surface in the tilt image obtained by the photometric stereo method is generally different from the actual work surface. End up. Therefore, measures such as performing second-order differential processing have been taken, but in addition to such measures, when installing the illumination means, such deviation is reduced by adjusting the posture to avoid halation. be able to. In addition, as in the case of the mirror surface, halation has an adverse effect on large unevenness where the brightness changes. For example, when the workpiece is cylindrical, specular reflection occurs when illumination light is applied, but if there is unevenness, the influence of the specular reflection is received to some extent. Therefore, giving a degree of freedom to the installation position and angle of the illumination light makes it possible to make adjustments in advance so as to reduce such specular reflection, and as a result, has the advantage of improving inspection accuracy.

  Furthermore, in the photometric stereo method, it is necessary to consider the influence of shadows. In the photometric stereo method, since a normal vector is calculated from reflected light of illumination, detection of reflected light is essential. However, for example, when the workpiece is complex and it is easy to get a shadow, if a lighting means is installed in the vicinity of this workpiece, appropriate reflected light cannot be obtained from a place where light does not reach, and normal vector calculation is performed. May cause problems. Even in such a case, the first illumination unit 21 to the fourth illumination unit 24 and the imaging unit 11 are separated from each other, so that the first illumination unit 21 to the fourth illumination unit 24 are set to the optimum positions and angles. This effect can be suppressed by installing it.

In addition, by separating the illumination unit from the imaging unit, the height of the illumination light can be changed. As a result, the distance LWD (Light Working Distance) between the work WK and the illumination can be reduced or conversely increased, and an appropriate arrangement according to the inspection application such as low-angle or multi-angle illumination can be selected. It becomes like this.
(1: When LWD is reduced)

  When the LWD is reduced, as shown in FIG. 21A, a large amount of light from the first illumination means 21 to the fourth illumination means 24 strikes from the lateral direction. In general, halation occurs when the incident angle and the reflection angle are the same. Therefore, by causing the first illumination means 21 to the fourth illumination means 24 and the work WK to approach each other to reduce LWD, halation occurs. The position can be near the outside of the workpiece WK. In other words, it is possible to intentionally generate halation on the outside of the work WK and suppress halation near the surface of the work WK. Thereby, the effective visual field of the workpiece | work WK can be increased.

  In addition, when the LWD is small, illumination is performed at a low angle. For example, even if unevenness on the workpiece surface is difficult to recognize due to light diffusing by direct illumination from above, illuminating from an oblique side makes it possible to capture changes in shallow irregularities by changing the contrast greatly even with small changes in tilt. it can. Thus, by emphasizing the contrast at a low angle, there is an advantage that the tilt image and the contour extraction image can be made clearer in the photometric stereo method.

On the other hand, as a demerit when the LWD is made small, shadows are easily generated by irradiating illumination light at a shallow angle, so that the effective visual field is reduced and photometric stereo processing may be difficult. In addition, since the first illuminating means 21 to the fourth illuminating means 24 and the work WK easily interfere with each other, the use of the work WK having a large height may be limited.
(2: Large LWD)

  Conversely, if the LWD is increased, a large amount of illumination light from the first illumination means 21 to the fourth illumination means 24 is irradiated from above as shown in FIG. 21B. This makes it difficult for shadows to occur, reducing the invisible areas and improving the accuracy of photometric stereo processing. Moreover, since it becomes difficult for the 1st illumination means 21-the 4th illumination means 24 and the workpiece | work WK to interfere, it can utilize suitably also to the workpiece | work WK with large height.

  On the other hand, since the position where halation occurs is inside the work WK, the effective visual field may be reduced. In addition, when the LWD is large, it corresponds to the case where the LWD is far in multi-angle illumination, and it is difficult to capture the change in shallow unevenness on the workpiece surface because it is difficult to obtain contrast. On the other hand, when the LWD is far away due to multi-angle illumination, the first illumination means 21 to the fourth illumination means 24 create a uniform imaging state with little illuminance unevenness and the surface state of the workpiece WK itself. A clearly captured image can be obtained.

  As described above, by separating the imaging unit and the illumination unit, there can be obtained an advantage that it is possible to cope with inspections for various purposes and applications and to increase the degree of freedom of arrangement. On the other hand, when the illumination unit and the imaging unit are configured separately and image inspection is performed by the photometric stereo method, it is necessary to clearly set the relative positional relationship between the illumination unit and the imaging unit. That is, the photometric stereo method is executed on the assumption that the illumination direction is known. If the virtual illumination direction set for the photometric stereo method is different from the actual illumination direction, there is a possibility that the inspection accuracy is lowered or an erroneous result is output.

  Here, as a method of adjusting the illumination direction, as shown in FIGS. 22 and 23, a method of physically correcting the position of the illumination means, and a photometric stereo at the current position of the illumination means, that is, the illumination direction, are used. In order to execute the method, there is a method of adjusting the illumination direction on the image processing side so as to coincide with that of the illumination means actually installed.

  In the former method, for example, as shown in FIG. 23, the relative rotation position is adjusted so that the direction of each illumination light emitted by the illumination unit integrated in a ring shape matches the rotation angle of the imaging means. Is necessary and very time consuming. In addition, as shown in FIG. 22, in the configuration in which each illumination unit is separated and can be arranged independently at any position, in addition to the relative rotation angle with the imaging unit described above, Since mistakes also occur, the adjustment work is further complicated.

On the other hand, the latter method is simple and excellent in convenience because it can be handled on the software side without performing troublesome work such as adjusting the angle and position of the physical illumination means. Therefore, in the present embodiment, a procedure for performing such adjustment of the illumination direction using the illumination direction estimating means 41g will be described.
(Illumination direction estimation means 41g)

The illumination direction estimation means 41g is mainly configured by the image processing unit 41 and processed by software. Here, (1) the influence when each of the zenith angle and the azimuth angle is shifted, (2) the principle that the illumination direction can be detected, and (3) the viewpoint of processing by software will be described.
(1) Effects when the zenith angle and azimuth angle shift

  First, the influence when each of the zenith angle and the azimuth angle is shifted will be described. Here, as shown in the schematic side view of FIG. 24A, the zenith angle is a vector from the imaging means 11 to the work WK and a vector from each of the first illumination means 21 to the fourth illumination means 24 to the work WK. It is an angle to make. The azimuth is a vector indicating the reference direction of the image (upward in the example of FIG. 24B), the first illuminating means 21 to the fourth illuminating means 24 from the imaging means 11, as shown in the schematic plan view of FIG. 24B. It is the angle that the vector to each of makes. In FIG. 24B, only the first illumination means 21 is shown for convenience of explanation, and the other illumination means 22 to 24 are not shown.

  In the photometric stereo method, processing is performed on the premise that the illumination direction is accurately known, that is, that the zenith angle and the azimuth angle that define the illumination direction are known. In this case, if the position of each lighting means installed by the user, that is, the actual lighting direction, and the position of the lighting assumed in the processing by the photometric stereo method, that is, the assumed lighting direction, are different. It becomes.

  The theoretical difference between the virtual zenith angle set for the photometric stereo method and the actual zenith angle is theoretically the relative position of the imaging means 11 and the first illumination means 21 to the fourth illumination means 24 in the vertical direction. Meaning the difference. Here, in order to explain the influence when the zenith angle and the azimuth angle are deviated, the description will be made based on the contour extracted images of FIGS. 25A to 25D. In these figures, FIG. 25A is a correct contour extraction image obtained with an illumination direction arranged to coincide with the assumed illumination direction (zenith angle and azimuth angle), and FIG. 25B is an incorrect zenith angle and correct azimuth angle. The obtained contour extraction image, FIG. 25C is a contour extraction image obtained with a correct zenith angle and an incorrect azimuth angle (when the whole is shifted by 45 °), and FIG. 25D is a correct zenith angle and an incorrect azimuth angle ( The contour extraction images obtained in the shuffle state in which the illumination means are replaced are shown respectively.

  As is clear from FIGS. 25A and 25B, the change in the zenith angle appears in such a manner that the contrast is proportional, such that the unevenness of the contour extraction image is doubled or halved, for example. In this case, it becomes a problem in an application in which accurate height information is to be measured strictly, but in an application in which only the shape change is to be grasped instead of the absolute value (numerical value) of the height, there is not much problem. From this, it can be said that, in applications where the photometric stereo method is used for image inspection, the influence of the difference in the zenith angle on the detection accuracy such as scratches and prints to be inspected is not so great.

On the other hand, the difference between the virtual azimuth angle and the actual azimuth angle is expressed in a form in which the shape extraction performance deteriorates as shown in FIGS. 25A, 25C, and 25D, for example. For this reason, when the photometric stereo method is used for image inspection, if the azimuth angle is different, it can be said that the influence on the detection accuracy of scratches and prints to be inspected is great.
(2) Principle of detecting the illumination direction (2-1) When the shape of the workpiece WK is unknown

  It is conceivable that the illumination unit and the imaging unit are installed in a state where various physical restrictions are provided in the mutual positional relationship. For example, when installing ring illumination in which each illumination means is arranged in a ring shape, or when installing each illumination means with the same angular width, for example, illumination to be connected so as to match the assumed illumination direction on the image inspection apparatus side For example, a common mark may be provided on the lighting connection connector of the lighting control means and the lighting control means so that the lighting means can be accurately installed without being mistaken.

In addition, under the physical constraints between the illumination unit and the imaging unit, there are cases where the way in which the illumination unit is disposed is finite. For example, four first illuminating means 21 to fourth illuminating means 24 are arranged on the same circumference at intervals of 90 °, and the workpiece WK and the imaging means 11 are arranged at the center in plan view of these four illuminating means. Consider the constraints (conditions) of placement. First, when one illumination means is fixed, there are ₃ P ₃ = 6 ways of arranging the remaining three illumination means. And there are 4 lightings to be fixed, that is, 4 ways to determine the lighting, and there is 1 way to set the correct answer, so there are 6 × 4−1 = 23 ways to make mistakes. is there.

Generalizing this idea, consider a case where n illumination means are arranged with an angular width of 360 ° / n on the same circumference centered on the imaging means in plan view. When one illumination means is fixed, there are _n-1 P _n-1 ways of arranging the remaining illumination means. Further, there are 360 ° / θ ways of arranging the illumination means that are supposed to be fixed by rotating and shifting them by the angular width θ.

Therefore, the illuminating means that is supposed to be fixed has azimuth angles shifted by an angle θ of 0 °, θ, 2θ,..., (360 ° −2θ) / θ, (360 ° −θ) / θ. If there is a physical restriction that the light source is disposed on the same circumference at any position and the remaining n-1 illumination means are disposed at an angular width of 360 ° / n, _n-1 P Among the combinations of _n-1 × 360 ° / θ illumination arrangements, there is an illumination arrangement that matches the assumed illumination direction, and _n-1 P _n-1 × 360 ° / θ-1 ways does not match the assumed illumination direction. This is a lighting arrangement.

For example, although the actual azimuth angle of each of the illumination means is unknown, the physicality that n illumination means are arranged with an angular width of 360 ° / n on the same circumference centered on the imaging means in plan view. If there is a general restriction (hereinafter referred to as “Case 1”), if the angle θ is reduced as much as possible, the above _- mentioned combinations of _n−1 P _n−1 × 360 ° / θ are assumed illuminations. An illumination arrangement that perfectly matches the azimuth of the direction should be found in theory.

  If an illumination arrangement that perfectly matches the azimuth angle of the assumed illumination direction is found, the illumination direction can be detected. Several methods are conceivable.

For example, in case 1, the _n-1 P _n-1 × 360 ° / θ method is performed according to the photometric stereo method, assuming the illumination direction from n images captured by illuminating from each illumination direction. A line vector image or a contour extraction image is generated as a source image SP to be subjected to an evaluation function, and the source image SP evaluated as the highest is selected by the evaluation function, and variable values of zenith angle and azimuth angle in the source image SP are selected. Detect as the actual illumination direction.
(Rotation of lighting direction)

First, as shown in the schematic plan view of FIG. 26, when an illumination unit (here, ring illumination) integrated with a plurality of illumination means is used as the illumination means, in other words, the illumination unit. A method for specifying the rotation angle will be described. In the example of FIG. 26, the illumination unit is composed of the first illumination means 21, the second illumination means 22, the third illumination means 23, and the fourth illumination means 24. It is arranged at a position of 270 °. These illumination means have an actual illumination direction on a virtual rotation plane centered on the optical axis of the imaging means. With respect to the actual illumination direction AL defined in this manner, the assumed illumination direction IL is rotated clockwise from the position of 0 ° by an angle width of 45 °, and the contour extraction image (source image in each illumination direction) is rotated. An example of generating SP) is shown in FIGS. 27A to 27H. In these figures, FIG. 27A is a contour extraction image in which the actual illumination direction is matched to match the assumed illumination direction. In other words, FIG. 27B is a contour extraction image in which the difference between the assumed illumination direction and the actual illumination direction is 0 °. The difference is 45 °, FIG. 27C is 90 °, FIG. 27D is 135 °, FIG. 27E is 180 °, FIG. 27F is 225 °, FIG. 27G is 270 °, and FIG. (Extracted image). For example, when the assumed illumination direction is rotated by 180 ° and shifted, as shown in FIG. 27E, the unevenness of the obtained contour extraction image is reversed. Further, when the angle is shifted to 90 ° in FIG. 27C or 270 ° in FIG. 27G, the outline becomes very unclear. Furthermore, at 45 ° in FIG. 27B, 135 ° in FIG. 27D, 225 ° in FIG. 27F, and 315 ° in FIG.
(Shuffle of lighting means)

  In the above, an example in which a lighting unit in which a plurality of lighting units are integrated is used corresponding to the rotation angle of the lighting unit has been described. In this case, since the 1st illumination means 21, the 2nd illumination means 22, the 3rd illumination means 23, and the 4th illumination means 24 are being fixed to the illumination unit, it is not necessary to consider the situation where each illumination means changes. On the other hand, as shown in FIG. 28, when each lighting unit is individually configured, it is necessary to consider the replacement of the lighting unit (referred to as “shuffle” in this specification). In FIG. 28, for the sake of explanation, any one of the illumination means (here, the third illumination means 23) is fixed, and the other illumination means (the first illumination means 21, the second illumination means 22, and the third illumination means 23) are fixed. A state of arbitrary replacement is shown. Hereinafter, a procedure for specifying each illumination unit in a shuffle state in which such illumination units are replaced will be described.

  Here, as shown in FIG. 28, FIGS. 29A to 29F show how the contour extraction image changes depending on the shuffle state of the illumination means. In these figures, FIG. 29A is a correct contour extraction image (source image SP) with an illumination direction that matches the assumed illumination direction, and FIGS. 29B to 29F show the first illumination means 21, the second illumination means 22, Source images SP (contour extraction images) generated in all combinations (3 × 2 × 1 = 6) in which the third illumination unit 23 is shuffled are shown.

  As shown in FIGS. 27 and 29, the correct contour extraction images shown in FIGS. 27A and 29A are shaded. On the other hand, when the illumination direction does not match the assumed illumination direction, as shown in FIGS. 27B to 27H and FIGS. 29B to 29F, the contour becomes smooth or unclear. On the other hand, as the shape of the workpiece to be inspected, there are few concave shapes, and a convex shape is generally considered. In the case of a convex workpiece, when the correct parameter of the photometric stereo method is selected, an image having the largest overall convexity is obtained. From this, using the average convexity as an evaluation value, the angle (rotation angle) in the illumination direction and a parameter indicating the maximum value for shuffle are searched. For example, if a function for obtaining an average height is used as an evaluation function, this function value becomes the largest when the convex / concave component expressed as a shading difference is the largest. By utilizing this, the values of the zenith angle and azimuth angle in the source image SP obtained at this time can be detected as the actual illumination direction.

  Note that the evaluation function is not limited to the configuration having the average value of the height, and other functions such as the square of the average value can be used as appropriate. Further, the method for narrowing the maximum value of the evaluation values such as the average of the obtained height is not particularly limited. For example, step search, binary search, golden section search, steepest descent method, etc. Can be used as appropriate.

  Next, the average height of the entire image is obtained from the source image SP obtained by rotating the assumed illumination direction IL as shown in FIG. The obtained graph is shown in FIG. Here, hard disks, tweezers, one-yen coins, substrates, hooks, hooks with tilted zenith angles, flat plates, calculators, eye drops, SD cards, and consoles were used as workpieces. As a result, in any workpiece, an image having the largest average value of the height is obtained when the rotation angle is around 0 ° (360 °). Here, 0 ° (360 °) is a position where the assumed illumination direction IL overlaps the actual illumination direction AL, that is, the correct illumination direction. Therefore, it was confirmed that if the rotation angle at which the average height is maximized was obtained, this position could be treated as the estimated illumination direction. In particular, even in workpieces of various shapes, the tendency to have one peak could be confirmed if the overall shape was convex. It can also be seen that the more convex components, the better the calculation of the estimated illumination direction.

  Further, as shown in FIG. 28, the average of the height of the entire image is similarly obtained from the source images SP obtained for all combinations (six combinations including the original) in which the imaging means is shuffled, and the rotation angle is obtained. The graph plotted for each is shown in FIG. Also in this figure, an image having the largest average value of the height is obtained when the rotation angle is around 0 ° (360 °). In other words, it was confirmed that if the rotation angle with the maximum height average value was calculated, this position could be treated as the estimated illumination direction.

  From the above, considering experimental errors in installation of the imaging means and the illumination direction, the average value of the height is used as the evaluation function, so that the value of the zenith angle and azimuth angle in the source image SP at the time when this function value becomes maximum The estimated illumination direction can be used to detect the actual illumination direction.

  Here, various forms are conceivable as physical restrictions between the illumination unit and the imaging unit. For example, the illumination unit and the imaging unit arranged at equal intervals are completely integrated, or the illumination unit is integrated and separated from the imaging unit at an arbitrary rotation angle with respect to the periphery of the imaging unit. There are examples, or examples of free illumination in which each illumination means can be arranged independently. In addition, various variations can be conceived as to how to install the illumination means in accordance with such physical constraints. For example, in addition to the rotation direction error shown in FIG. 26, the shuffle, that is, the illumination means shown in FIG. 28, the deviation from the cross axis (xy axis) as shown in FIG. 32, or the zenith angle asymmetric deviation shown in FIG. Examples include x2 and zenith angle symmetry change x2 shown in FIG. Here, with respect to variations in physical constraints between the illuminating means and the imaging means, Table 1 summarizes the possibility of mistakes occurring for each manner of illuminating means installation mistakes.

  In addition, Table 2 summarizes the effects of failure in estimation, the constraint conditions necessary for the estimation, and factors that adversely affect the estimation for each mode of installation of such illumination means.

(2-2) When the shape of the workpiece WK is known

  When the shape of the workpiece WK is known, the surface shape and the source of the workpiece WK in the source image SP with respect to the zenith angle and the azimuth angle are calculated from the source image SP calculated by using the zenith angle and the azimuth angle as variables. A function obtained by quantifying the amount of deviation from the actual surface shape in the image SP is defined as an evaluation function. According to this evaluation function, when the value of the evaluation function is minimized, the degree of matching with the source image SP in the actual illumination direction is maximized. Therefore, the variable values of the zenith angle and azimuth angle in the source image SP when the value of the evaluation function is minimized can be detected as the actual illumination direction.

  Further, when the workpiece WK is a hemispherical workpiece having a strong specular reflection component, as shown in FIG. Occur. From the relationship between the magnitude of the direction vector to the center coordinates of the halation with the center coordinate of the workpiece WK as the base point, the radius of the workpiece WK, and the zenith angle θ, the zenith angle θ is obtained as follows.

  The calculation may be performed as follows.

  The azimuth angle in this case is obtained from the coordinates of the xy plane of the direction vector as shown in FIG. 35B.

  Further, when the workpiece WK is a workpiece having a known height, as shown in FIG. 36A, a shadow of the workpiece WK by illumination light can be formed. From the relationship between the size of the shadow of the light from the illumination means by the work WK, the height of the work WK, and the zenith angle θ, the zenith angle θ is obtained as follows.

The azimuth angle in this case is obtained from the coordinates of the shadow vector in the xy plane, as shown in FIG. 36B.
(3) Processing by software

The process by the software in the illumination direction estimation means 41g is demonstrated with reference to the flowchart shown to FIGS. FIG. 37 is a flowchart showing the flow of the illumination direction detection process using the photometric stereo method. FIG. 38 is a flowchart showing the flow of the illumination direction detection process not using the photometric stereo method. FIG. 39 is a flowchart showing the overall flow of processing of the image inspection apparatus 1.
(3-1) Illumination direction detection method using photometric stereo method

  First, an illumination direction detection method using the photometric stereo method will be described with reference to a flowchart shown in FIG. In step ST1, the image processing unit 41 and the imaging unit 11 execute the processes in steps ST101 to ST105 in the flowchart shown in FIG.

  In step ST2, the user selects an evaluation function according to whether the shape of the workpiece WK is unknown or is known, determines whether the zenith angle is a constant or a variable, Parameters are set for the angle width and the azimuth angle width.

  In step ST3, the image processing unit 41 executes the process in step ST108 when the contour extraction mode is selected by the user, and generates the source images SP for all the variable values.

  When the shape of the workpiece WK is known, in step ST4, the image processing unit 41 refers to the actual surface shape information as the source image SP stored in the storage unit 44. If the shape of the workpiece WK is unknown, this process is not executed.

  In step ST5, the image processing unit 41 performs an evaluation function on the zenith angle and azimuth angle from the source image SP generated by calculating the zenith angle and azimuth angle as variables (in some cases constants) in step ST3. Accordingly, the source image SP evaluated as the highest is selected.

  In step ST5, the image processing unit 41 returns to step ST2 and resets the parameters when the numerical analysis does not converge.

In step ST6, the image processing unit 41 uses the zenith angle and azimuth variable values (constant values in some cases) in the source image SP evaluated as the highest in step ST5 in accordance with the evaluation function in the actual illumination direction. Detect as.
(3-2) Detection method of illumination direction without using photometric stereo method

  Next, an illumination direction detection method that does not use the photometric stereo method will be described with reference to the flowchart shown in FIG. In step ST11, the image processing unit 41 and the imaging unit 11 execute the processes in steps ST101 to ST103 in the flowchart shown in FIG. 39, and the image processing unit 41 has four image signals Q1 to Q4 obtained by photographing the workpiece WK. Is stored in the storage means 44 as actual surface shape information.

  In step ST <b> 12, the image processing unit 41 refers to actual surface shape information stored in the storage unit 44.

  In step ST13, a zenith angle and an azimuth angle are calculated from the actual surface shape information referred in step ST12.

  In step ST14, the image processing unit 41 detects the values of the calculated zenith angle and azimuth as the actual illumination direction in step ST13.

Thus, since the actual illumination direction set by the user can be estimated automatically and with high accuracy, when combined with the photometric stereo method, the illumination unit and the imaging unit are separated from each other. There are merits that can be done. That is, if the illumination unit and the imaging unit are separated and the user can freely change the position of the illumination, the tilt image (normal vector) has a characteristic that changes according to the position of the illumination. In an image inspection apparatus that allows such characteristics, the presence of an illumination direction estimation unit that estimates the actual illumination direction installed by the user is extremely useful.
(Image inspection method)

  Here, a procedure of an image inspection method for performing an appearance inspection of a workpiece using the image inspection apparatus 1 will be described based on a flowchart of FIG.

  In step ST101, the image processing unit 41 issues a trigger signal to the illumination units 21, 22, 23, and 24 via the illumination control unit 31, and the illumination units 21, 22, 23, and 24 respond to the trigger signal. Turn on one by one.

  In step ST102, the imaging unit 11 operates each time the lighting units 21, 22, 23, and 24 are turned on, and images the workpiece WK.

  In step ST103, the imaging unit 11 transfers the four image signals Q1 to Q4 obtained by photographing the work WK to the image processing unit 41.

  In step ST <b> 104, the image processing unit 41 uses the four image signals Q <b> 1 to Q <b> 4 input from the imaging unit 11 to calculate the normal vector of the surface in each pixel for each image signal Q <b> 1 to Q <b> 4.

  In step ST105, the image processing unit 41 reduces the image signals Q1 to Q4 to 1/1, 1/2, 1/4, 1/8, 1/16, and 1/32, which are necessary for subsequent processing. Create a reduced tilt image with each reduced size. In addition, the image processing unit 41 creates a texture extraction image that is necessary for subsequent processing for each of the image signals Q1 to Q4.

  In step ST106, the feature size is adjusted as necessary. By changing the feature size, the size of the unevenness extracted in the unevenness extraction image changes. Specifically, when the feature size is increased, a concavo-convex extracted image from which large-sized concavo-convex has been extracted is obtained. On the other hand, when the feature size is reduced, small-sized irregularities are extracted. Therefore, the user adjusts the feature size according to the size of the wound to be extracted. Alternatively, in the case of OCR use, an unevenness extraction image suitable for OCR can be obtained by increasing the feature size.

  In step ST107, unevenness extraction image calculation is performed. In this example, the image processing unit 41 generates a concavo-convex extracted image obtained by extracting the concavo-convex according to the feature size set in step ST <b> 106, and displays it on the display unit 51.

  In steps ST110 to ST112, the image processing unit 41 or the PLC 81 performs a flaw detection process for detecting a flaw using a flaw inspection tool on the contour extraction image, and determines whether or not the flaw is a flaw. do.

  First, in step ST110, the image processing unit 41 or the PLC 81 specifies the position of the inspection region to be inspected with respect to the generated contour image. When setting the inspection area, for example, an image of the work WK is extracted by extracting an edge or the like. When the work WK does not greatly deviate, the set position of the inspection area may be registered in advance.

  In step ST111, the image processing unit 41 or the PLC 81 performs image processing for detecting a flaw in the specified inspection area. The image processing unit 41 or the PLC 81 calculates a reference density value by a stored calculation method, and calculates a difference between the density value of each pixel in the inspection area and the reference density value for each pixel. Then, the image processing unit 41 or the PLC 81 performs labeling processing based on a threshold value that is set and stored (a threshold value that is a scratch amount is determined) (“0, 1, 2,. .. ”is executed, and a feature amount is calculated for each specified flaw. The feature quantity to be calculated is, for example, positive / negative information regarding the positive / negative of the difference, the sum of the differences, the maximum value of the differences, the average value of the differences, or the standard deviation of the differences.

In step ST112, the image processing unit 41 or the PLC 81 performs a scratch determination process on the scratch identified in step ST111 based on a determination criterion used for scratch determination. If it is determined as a scratch, marking is made on the scratched area on the display means 51, and the process is terminated.
(Modification)

  In the above example, the example in which the feature size can be adjusted by the user to generate the unevenness extraction image in which the unevenness of the size desired by the user is extracted using the feature size parameter has been described. However, the present invention is not limited to this configuration, and a plurality of observation modes corresponding to the user's observation application and purpose are prepared in advance, and a desired image is generated by allowing the user to select an observation mode. You can also. Such an example will be described based on the flowchart of FIG. Steps ST ″ 1 to ST ″ 5 are the same as those in FIG. 39 described above, and thus detailed description thereof is omitted.

  In step ST ″ 6, the user is made to select an observation mode. In this example, any one of a contour extraction mode, a texture extraction mode, and an uneven mode can be selected. Each observation mode has a feature size suitable for each observation application. It should be noted that the feature size can be manually adjusted by the user after selection of the observation mode, and the next transition step differs according to the selected observation mode, ie, contour extraction. If the mode is selected, the process proceeds to step ST ″ 7, if the texture extraction mode is selected, the process proceeds to step ST ″ 8, and if the uneven mode is selected, the process proceeds to step ST ″ 9. After the processing of each step, the process proceeds to step ST ″ 10. Thus, the unevenness extraction image generation unit, the contour image generation unit 41b, and the texture extraction image generation unit 41c can be switched.

  In step ST ″ 7, the image processing unit 41 performs the unevenness extraction image calculation. That is, the image processing unit 41 displays on the display means 51 a contour extraction image having a feature size on which the unevenness extraction image is displayed.

  In step ST ″ 8, the image processing unit 41 executes a process when the contour extraction mode is selected by the user. That is, the image processing unit 41 is based on the reduced tilt image created in step ST ″ 5. The contour extraction image is calculated, and the contour extraction image is displayed on the display means 51.

  In step ST ″ 9, the image processing unit 41 executes processing in the case where the texture extraction mode is selected by the user. That is, the image processing unit 41 is based on the texture extracted image created in step ST ″ 5. Then, the texture extraction image calculation is performed, and the texture extraction image is displayed on the display means 51.

  Subsequent steps ST ″ 10 to 12 can use the same procedure as in FIG. 39 described above, and a detailed description thereof will be omitted.

  According to the image inspection apparatus described above, the first-order tilt image obtained in the X direction and the Y-direction is generated while generating the first-order tilt image using the photometric stereo method. A second-order tilt image, that is, a contour extraction image is generated by performing differentiation. This process greatly changes the inspection image to be obtained due to errors in the input information such as the slight inclination of the illumination or workpiece installation surface or the position of the illumination that was originally input. Inspection that matches the real thing, such as the phenomenon that the image becomes uneven although it has not been done, and the phenomenon that the center of the workpiece WK is raised due to the tendency to become brighter near the illumination. The disadvantage of the conventional photometric stereo method that an image cannot be obtained can be reduced. By setting the gradation where the slope change is large in the concave direction of the surface to darken, while setting the gradation where the small change in the convex direction of the surface is brighter, scratches and contours that greatly change the workpiece surface inclination are set. The image is suitable for extraction.

  Further, the contour image generation means creates a plurality of reduced inclination images with different reduction ratios from the calculated normal vectors of the respective pixels, and performs a differentiation process in each of the reduced inclination images in the X direction and the Y direction. The reduced outline image is weighted so that the reduced outline image with a predetermined reduction ratio is largely reflected, enlarged to the original size, and then added to all the enlarged reduced outline images. A contour extracted image can be obtained. According to this configuration, what size is determined as a flaw depends on the user's application, and weighting can be performed so that a reduced contour image with a predetermined reduction ratio is largely reflected in the contour extracted image. According to the user's application, a contour extraction image in which a reduced contour image having a reduction ratio desired by the user is emphasized is obtained. As user needs, for example, there is a case where it is desired to determine a scratch extending over 10 pixels as a scratch, and a case where a concave portion extending over 100 pixels is determined as a scratch. In some cases, only sharp edges may be extracted as edges.

  Also, scratches are usually detected across a plurality of frequencies, but the sum of all the enlarged reduced contour images is used as the contour extraction image, so when limited to a reduced contour image of one frequency Compared to the above, scratches, contours, and the like can be clearly detected without blurring.

  Furthermore, weighting is performed by preparing a set of predetermined weights, and by multiplying the reduced contour image by the weight set, and by dividing the adoption ratio of the reduced contour image of each reduction ratio. Can do. According to this, since a set of weights (feature size) determined in advance when combining with the contour extraction image is preset, the user can easily and instantly switch to the desired contour extraction image.

  Furthermore, the set of weights can include a set in which the adoption ratio of the reduced contour image from which a contour extraction image with clear irregularities on the surface of the workpiece is obtained is increased. For example, by using the uneven mode for this usage, it can be used separately from the contour extraction mode.

  Furthermore, the set of weights can include a set in which the adopted ratio of the reduced contour image from which a contour extraction image suitable for OCR is obtained is increased, so that OCR is performed on a casting mark, for example. Therefore, it is possible to create a suitable image.

  Furthermore, the albedo of the same number of pixels as the input image is calculated from the calculated normal vector of each pixel that is illuminated by the illumination means, and the tilt state of the workpiece surface is removed from the albedo. Since the contour image generation means and the texture extraction image generation means are configured to be switchable so as to generate a texture extraction image indicating the above, it is usually difficult to distinguish the original scratch from the original contour. Although it is required to search for an area where there should not be an inspection area, it is possible to perform a scratch inspection after searching the texture extracted image and determining the inspection area. Also, when performing OCR, it is possible to perform OCR after performing a search on a texture extracted image and determining a target area for performing OCR.

  The texture extracted image generation means can sort the albedo values of the same number of pixels as the input image and adopt the pixel values in a specific order from the top as the texture extracted image. As a result, a pixel value having high luminance causing halation is not adopted, and a texture extracted image from which the influence of halation is removed is obtained.

  In addition, it is common knowledge that photometric stereo technology is a method of three-dimensional measurement by providing a flaw inspection tool necessary for flaw inspection after generating contour extraction images. Photometric stereo technology It is possible to provide an image inspection apparatus that is positioned as a practical product that is applied to scratch inspection.

  Further, the image inspection apparatus includes an imaging unit that images the workpiece from a certain direction, an illumination unit that illuminates the workpiece at least three times from different directions, an illumination control unit that sequentially turns on the illumination unit one by one, and each illumination timing. By driving the image pickup means, a normal vector with respect to the surface of the work of each pixel is calculated using the image generation means for generating a plurality of images and the pixel values for each pixel in a corresponding relationship in the plurality of images. Calculate the albedo of each pixel as many as the normal vector from the normal vector calculation means and the normal vector of each pixel that has been illuminated by the illumination means, and the inclination of the surface of the workpiece from the albedo Texture extraction image generation means for generating a texture extraction image showing the pattern from which the state has been removed, and the texture extraction image generation means has the same number as the normal vector. It can be a material obtained by employing the pixel values of a specific order from the top to sort the values of albedo of each pixel of the texture extracted image. According to this, the texture extraction image generation means sorts the albedo values of the same number of pixels as the normal vector and adopts a pixel value in a specific order from the top as the texture extraction image. Therefore, pixel values with high luminance causing halation are not adopted, and a texture extracted image from which the influence of halation is removed is obtained.

  As described above, according to the image inspection apparatus according to the embodiment, it is possible to more easily and robustly inspect the scratches and prints of the work by using the photometric stereo method.

  The image inspection apparatus, the image inspection method, the image inspection program, and the computer-readable recording medium or the recorded device of the present invention can be suitably used for an inspection apparatus or a digitizer using a photometric stereo.

DESCRIPTION OF SYMBOLS 1 ... Image inspection apparatus 11 ... Imaging means 12, 32, 52, 62, 82 ... Cable 71, 72, 73, 74, 76 ... Cable 20, 20 ', 20 "... Annular illumination unit 21, 22, 23, 24, 25 ... illumination means 31 ... illumination control means 41 ... image processing unit; 41a ... normal vector calculation means; 41b ... contour image generation means 41c ... texture extraction image generation means; 41d ... inspection region specification means 41e ... image processing means; ... Determination means; 41g ... Illumination direction estimation means 42 ... Signal processing system 43 ... CPU
44 ... Memory 45 ... ROM
46 ... Bus 51 ... Display means 61 ... Operation means 75 ... Illumination branch unit 81 ... PLC
e1 ... cameras e21 to e24 ... illumination light source WK ... work SG ... stage L1 ... first illumination means; L2 ... second illumination means n ... normal vector S ... diffuse reflection surface OS ... obstacle AL ... actual illumination direction IL: Assumed illumination direction

Claims (19)

An image inspection apparatus for inspecting the appearance of a workpiece,
Three or more illumination means for illuminating the workpiece from different actual illumination directions;
Illumination control means for lighting the three or more illumination means one by one in a predetermined lighting order;
Imaging means for capturing a plurality of partial illumination images having different actual illumination directions by imaging the workpiece from a certain direction at an illumination timing at which each illumination means is turned on by the illumination control means;
The source image of the surface of the workpiece in the assumed illumination direction assuming the illumination direction based on the photometric stereo method using the pixel values for each pixel having a correspondence relationship among the plurality of partial illumination images captured by the imaging unit Source image generating means for generating a plurality of different assumed illumination directions,
Illumination direction estimation means for detecting an estimated illumination direction that most closely matches the actual illumination direction in the illumination means, based on a plurality of source images generated for each assumed illumination direction by the source image generation means. A characteristic image inspection apparatus. The image inspection apparatus according to claim 1,
The three or more illumination means are integrally configured, and have an actual illumination direction on a virtual rotation plane centered on the optical axis of the imaging means,
An image inspection apparatus, wherein the illumination direction estimation means estimates an estimated illumination direction from a plurality of assumed illumination directions assumed to be located on a virtual rotation plane. The image inspection apparatus according to claim 1,
The three or more illumination means are configured to be arranged independently of each other, and have an actual illumination direction on a virtual rotation plane centered on the optical axis of the imaging means,
The illumination direction estimating means is configured to estimate an estimated illumination direction from a plurality of assumed illumination directions assumed to be located on a virtual rotation surface, and to estimate each installation position of the three or more illumination means. An image inspection apparatus characterized by comprising: The image inspection apparatus according to claim 1,
The image inspection apparatus, wherein the illumination direction estimating means estimates the illumination illumination direction on the assumption that the surface shape of the workpiece is convex. The image inspection apparatus according to any one of claims 1 to 4,
The source image generation means generates the plurality of source images with a zenith angle and an azimuth angle that specify an illumination direction as variables,
The illumination direction estimating means is a source image evaluated as the highest in accordance with a first evaluation function indicating a degree of matching with the source image in the actual illumination direction among the source images generated by the source image generating means. The image inspection apparatus is characterized in that the zenith angle and azimuth angle variable values in the source image are estimated as the estimated illumination direction. The image inspection apparatus according to claim 5,
The first evaluation function is obtained by calculating the surface shape of the workpiece in the source image and the actual surface of the source image with respect to the zenith angle and azimuth angle from the source image calculated by using the zenith angle and azimuth as variables. An image inspection apparatus, which is a function obtained by quantifying the amount of deviation from a shape. The image inspection apparatus according to any one of claims 1 to 4,
The source image generation means generates the plurality of source images with a zenith angle as a constant and a azimuth as a variable among zenith angles and azimuth angles that specify an illumination direction,
The illumination direction estimation means is a source image evaluated as the highest by a second evaluation function indicating a degree of matching with the source image in the actual illumination direction among the source images generated by the source image generation means. An image inspection apparatus characterized by selecting the variable value of the azimuth angle and the constant value of the zenith angle in the source image as the estimated illumination direction. The image inspection apparatus according to claim 7,
The second evaluation function quantifies the expression ratio of the uneven component of the surface of the workpiece with respect to the azimuth angle from the source image calculated by using the zenith angle as a constant and the azimuth angle as a variable. An image inspection apparatus characterized by being a function. The image inspection apparatus according to any one of claims 5 to 8,
An image inspection apparatus characterized in that a value is set from a predetermined combination for an azimuth angle as a variable. The image inspection apparatus according to any one of claims 5 to 8,
An image inspection apparatus in which an azimuth angle as a variable is set with a uniform angular width. The image inspection apparatus according to claim 5, wherein
The illumination direction estimating means estimates the workpiece as a hemisphere including a specular reflection component, and further, for each source image, uses a center coordinate in the source image of the workpiece as a base point, as an image of the specular reflection component from the workpiece. A vector of the halation coordinates at the center of the image, and the direction of the vector is calculated as the azimuth angle of the illumination direction when the source image is generated, and the ratio between the number of pixels of the vector and the number of pixels of the workpiece size The zenith angle in the illumination direction when the source image is generated is calculated from the image inspection apparatus. The image inspection apparatus according to claim 5, wherein
The illumination direction estimating means obtains a vector of shadow coordinates at the tip of a shadow in the work source image, with the height of the work being known and for each source image, based on the center coordinate in the work source image. The direction of the vector is calculated as the azimuth angle of the illumination direction when the source image is generated, and the ratio of the number of pixels of the vector to the number of pixels of the workpiece height is used to generate the source image. An image inspection apparatus characterized by calculating a zenith angle in an illumination direction. The image inspection apparatus according to claim 5, wherein
The illumination direction estimating means compares a preset setting range of the zenith angle and / or azimuth with the imaging means and the illumination means arranged with the estimated zenith angle and / or azimuth. An image inspection apparatus configured to issue a warning when the set range is not satisfied. An image inspection apparatus according to any one of claims 1 to 13,
The image inspection apparatus, wherein the illumination control unit is configured to be able to arbitrarily change a lighting order when generating a source image for each different assumed illumination direction. An image inspection apparatus according to any one of claims 1 to 14,
The source image generation means includes
For calculating a normal vector with respect to the surface of the workpiece of each pixel based on the photometric stereo method using pixel values for each pixel having a correspondence relationship among a plurality of partial illumination images captured by the imaging unit Normal vector calculation means;
Contour image generation for performing differential processing in the X direction and Y direction on the normal vector of each pixel calculated by the normal vector calculation means to generate a contour image indicating the contour of the workpiece surface inclination Means,
An image inspection apparatus comprising: The image inspection apparatus according to claim 1, further comprising:
Inspection area specifying means for specifying the position of the inspection area to be inspected with respect to the generated contour image;
Image processing means for performing image processing for detecting a flaw in the inspection area specified by the inspection area specifying means;
An image inspection apparatus comprising: determination means for determining the presence or absence of scratches on the workpiece surface based on the processing result of the image processing means. An inspection method for inspecting the appearance by taking an image of a workpiece,
Three or more lighting means are lighted one by one in a predetermined lighting order, and the work is illuminated at a predetermined lighting timing from different actual lighting directions, and a common imaging means is used for each actual lighting direction. Capturing one partial illumination image and obtaining a plurality of partial illumination images having different actual illumination directions; and
Based on the photometric stereo method, the surface of the workpiece in the assumed illumination direction is assumed based on the photometric stereo method, using pixel values for each pixel in a correspondence relationship between multiple partial illumination images acquired by illumination with different illumination means. Generating a source image for each of a plurality of different assumed illumination directions by the source image generating means;
Based on a plurality of source images generated for each assumed illumination direction by the source image generation means, with the illumination direction estimation means, detecting an estimated illumination direction that most closely matches the actual illumination direction in the illumination means;
An image inspection method comprising: An inspection program for inspecting the appearance by taking an image of a workpiece,
Three or more lighting means are lighted one by one in a predetermined lighting order, and the work is illuminated at a predetermined lighting timing from different actual lighting directions, and a common imaging means is used for each actual lighting direction. A function of capturing one partial illumination image and acquiring a plurality of partial illumination images with different actual illumination directions;
Based on the photometric stereo method, the surface of the workpiece in the assumed illumination direction is assumed based on the photometric stereo method, using pixel values for each pixel in a correspondence relationship between multiple partial illumination images acquired by illumination with different illumination means. A function of generating a source image for each of a plurality of different assumed illumination directions by the source image generation means;
Based on a plurality of source images generated for each assumed illumination direction by the source image generation means, a function for detecting an estimated illumination direction that most closely matches an actual illumination direction in the illumination means with an illumination direction estimation means;
An image inspection program characterized by realizing the above.   A computer-readable recording medium or a recorded device on which the image inspection program according to claim 18 is recorded.