JP6812118B2 - Defect detector, defect detection method and program - Google Patents

Description

The present invention relates to a technique for detecting surface defects of an object.

Conventionally, metal parts used for driving parts of automobiles are manufactured by forging or casting. After manufacturing the metal parts, a so-called visual inspection is performed to check whether the obtained metal parts have defects such as scratches. Conventionally, the visual inspection of such metal parts has been performed visually by an operator. However, depending on the visual inspection of the worker, the time required for the inspection and the inspection result vary among the plurality of workers. In addition, even if the same worker is in poor physical condition or has poor concentration, the inspection may take a long time or defects may be overlooked. Therefore, there is a demand for the development of an apparatus capable of automatically and at high speed in visual inspection of metal parts.

Here, conventionally, an apparatus has been used in which an object is irradiated with light to take an image and the appearance of the object is inspected based on the captured image. In particular, a technique (so-called self-comparison technique) is known in which a difference process is performed on an captured image before and after the expansion / contraction process to extract defect candidates.

For example, in the visual inspection apparatus for inspecting the appearance of an electronic component of Patent Document 1, after binarizing the captured image, expansion processing and contraction processing are performed to generate a contracted image in which small defects are eliminated. By taking the difference between this and the captured image, a defect with a small size is extracted. Further, in Patent Document 1, if a defect having a large size is also extracted by the difference processing between the captured image and the reference image and it is determined that even one of the difference images is NG (defect), the captured image is obtained. It is described that the determination result of the defect is output (see paragraphs 0061-0074, FIGS. 5 and 6).

JP 2015-137921

The above self-comparison inspection does not require the preparation of an image (hereinafter referred to as "reference image") as a reference for the inspection, and can extract various defects existing in the captured image only from the captured image. it can. However, on the surface of the object included in the captured image, fine lines and fine irregularities that are not defects are also detected as defects, so it is necessary to suppress over-detection.

The present invention has been made in view of the above problems, and over-detection of defects is suppressed by removing defects that do not actually correspond to defects (false defects) from the defect candidates extracted by self-comparison with high accuracy. The purpose is.

The first invention of the present application for solving the above-mentioned problems is a defect detection device that detects defects on the surface of an object, and includes an imaging unit that captures an image of the object and acquires an captured image, and the captured image. A storage unit that stores the corresponding reference image, the value of each pixel of the captured image, and one of the expansion processing using the maximum value filter and the contraction processing using the minimum value filter are performed on the captured image, and further. A defect candidate region is detected based on at least one of a difference or a ratio between the value of the pixel corresponding to the captured image of the image obtained by performing the other processing different from the one processing of the expansion treatment and the contraction treatment. , The difference or ratio between the value of each pixel of the reference image and the value of the pixel corresponding to the reference image of the image obtained by subjecting the reference image to the one-sided processing and further performing the other-processing. A mask region is detected based on at least one of them, and among the defect candidate regions, a region overlapping the mask region is excluded from the defect candidates, and the alignment is performed when the defect candidate region and the mask region are overlapped. After accurately aligning the captured image with the reference image, if the one-sided process is an expansion process, the reference image is imaged based on the difference image or the ratio image of these images. When a region darker in the image is acquired as another defect candidate region and the one-sided processing is a shrinkage processing, the captured image is based on the difference image or the ratio image of these images with respect to the reference image. A brighter region is acquired as another defect candidate region, an region where the defect candidate region and the other defect candidate region overlap is acquired as a new defect candidate region, and then based on the new defect candidate region. It also includes a defect acquisition unit that acquires the presence of defects in the captured image.

According to the first invention, among the defect candidates detected by the self-comparison of the captured images, those that overlap with the self-comparison result of the reference image can be excluded from the defect candidates, and as a result, the defects included in the captured image. It is possible to suppress over-detection by removing a portion brighter or darker than the surroundings from the defect candidates. In addition, a new defect candidate is defined as a region where the defect candidate detected by the so-called self-comparison and the defect candidate detected by the difference image between the captured image and the reference image (the defect candidate detected by the so-called comparison with others) overlap. By doing so, overdetection can be suppressed more reliably.

The second invention of the present application is the defect detection device of the first invention, and the defect acquisition unit acquires the presence of the first defect in the captured image and the second defect different from the first defect, and the captured image. Based on at least one of the difference or ratio between the value of each pixel of the above and the value of the pixel corresponding to the captured image of the image obtained by subjecting the captured image to the expansion process and then the contraction process. The value of each pixel of the reference image after detecting the first defect candidate region, and the value of the pixel corresponding to the reference image of the image obtained by subjecting the reference image to expansion processing and then contraction processing. The first mask region is detected based on at least one of the difference or the ratio of, and the region of the first defect candidate region that overlaps with the first mask region is excluded from the first defect candidate, and the first defect candidate region is excluded . After aligning the captured image and the reference image more accurately than the alignment when the defect candidate region and the first mask region are overlapped, the reference is based on the difference image or the ratio image of these images. The region where the captured image is darker than the image is acquired as the third defect candidate region, and the region where the first defect candidate region and the third defect candidate region overlap is acquired as a new first defect candidate region. Then, with respect to the first defect acquisition unit that acquires the presence of the first defect in the captured image based on the new first defect candidate region, the value of each pixel of the captured image, and the captured image. The second defect candidate region is detected based on at least one of the difference or the ratio between the value of the pixel corresponding to the captured image of the image obtained by performing the shrinkage treatment and then the expansion treatment, and the reference image is described. The first is based on at least one of the difference or the ratio between the value of each pixel and the value of the pixel corresponding to the reference image of the image obtained by subjecting the reference image to the contraction process and then the expansion process. When detecting 2 mask regions, excluding the region overlapping the second mask region from the second defect candidate region from the second defect candidate, and overlapping the second defect candidate region with the second mask region. After aligning the captured image and the reference image more accurately than the alignment of the above, the captured image has a brighter region with respect to the reference image based on the difference image or the ratio image of these images. It is acquired as a fourth defect candidate area, and an area where the second defect candidate area and the fourth defect candidate area overlap is acquired as a new second defect candidate area, and then based on the new two defect candidate area. 2nd defect to acquire the presence of the 2nd defect in the captured image It is characterized by having a depression acquisition unit.

According to the second invention, the defect acquisition unit can acquire the first defect and the second defect different from the first defect, that is, a different kind of defect by changing the order of the expansion treatment and the contraction treatment. .. As a result, it is possible to more reliably prevent omission of detection of defects included in the captured image.

The third invention of the present application for solving the above-mentioned problems is a defect detection method for detecting defects on the surface of an object, and a) an imaging step of capturing an image of the object with an imaging unit to acquire an image. And b) The value of each pixel of the captured image and one of the expansion process using the maximum value filter and the contraction process using the minimum value filter are performed on the captured image, and further, among the expansion process and the contraction process. A defect candidate region detection step of detecting a defect candidate region based on at least one of a difference or a ratio between the value of the pixel corresponding to the captured image of the image obtained by performing the other processing different from the one processing. c) A reference image corresponding to the captured image is prepared, and the value of each pixel of the reference image and the reference image are subjected to the one-sided processing and further the other-processed image. A mask area detection step of detecting a mask area based on at least one of the difference or ratio with the value of the pixel corresponding to the reference image, and d) a defect in the defect candidate area that overlaps with the mask area. The one-sided processing is performed after the defect candidate exclusion step of excluding from the candidates and e) the alignment of the captured image and the reference image are performed more accurately than the alignment when the defect candidate region and the mask region are overlapped. When is an expansion process, a region darker in the captured image than the reference image is acquired as another defect candidate region based on the difference image or the ratio image of these images, and the one-sided process is contracted. In the case of processing, a region in which the captured image is brighter than the reference image is acquired as another defect candidate region based on the difference image or the ratio image of these images, and the defect candidate region and the other A defect acquisition step is provided in which a region overlapping the defect candidate region of the above is acquired as a new defect candidate region, and then the presence of a defect in the captured image is acquired based on the new defect candidate region.

According to the third invention, among the defect candidates detected by the self-comparison of the captured images, the region overlapping with the self-comparison result of the reference image can be excluded from the defect candidates, and as a result, the defects included in the captured image. It is possible to suppress over-detection by removing a portion brighter or darker than the surroundings from the defect candidates. In addition, a new defect candidate is defined as a region where the defect candidate detected by the so-called self-comparison and the defect candidate detected by the difference image between the captured image and the reference image (the defect candidate detected by the so-called comparison with others) overlap. By doing so, overdetection can be suppressed more reliably.

The fourth invention of the present application for solving the above-mentioned problems is a program for causing a computer to detect a defect in the target area from a plurality of images of the target area on the surface of the object, and the computer executes the program. In the computer, a) a step of preparing an captured image acquired by imaging the target area and a corresponding reference image, and b) a value of each pixel of the captured image and a maximum with respect to the captured image. subjected to expansion processing and contrast processing in the contraction process using the minimum value filter uses a value filter, corresponding to the captured image of the image obtained by performing different other processing further the one processing in the expansion processing and contraction processing A defect candidate region detection step of detecting a defect candidate region based on at least one of a difference or a ratio between the pixel value and c) a reference image corresponding to the captured image is prepared, and each of the reference images Based on at least one of the difference or ratio between the pixel value and the pixel value corresponding to the reference image of the image obtained by subjecting the reference image to the one-sided processing and further performing the other-processing. A mask area detection step of detecting a mask area, d) a defect candidate exclusion step of excluding an area overlapping the mask area from the defect candidates, and e) the defect candidate area and the mask area. After aligning the captured image and the reference image more accurately than the alignment at the time of superimposing, if the one-sided processing is the expansion processing, it is based on the difference image or the ratio image of these images. When the region in which the captured image is darker than the reference image is acquired as another defect candidate region and the one-side processing is the shrinkage processing, the reference is based on the difference image or the ratio image of these images. A region in which the captured image is brighter than the image is acquired as another defect candidate region, and an region in which the defect candidate region and the other defect candidate region overlap is acquired as a new defect candidate region. A defect acquisition step of acquiring the presence of a defect in the captured image is executed based on the new defect candidate region.

By installing the program of the fourth invention on a computer, it is possible to obtain a defect detection device that excludes a region overlapping with the self-comparison result of the reference image from the defect candidates detected by the self-comparison of the captured images. As a result, over-detection can be suppressed by removing a portion brighter or darker than the surroundings from the defect candidates, although it is not a defect contained in the captured image. In addition, a new defect candidate is defined as a region where the defect candidate detected by the so-called self-comparison and the defect candidate detected by the difference image between the captured image and the reference image (the defect candidate detected by the so-called comparison with others) overlap. By doing so, overdetection can be suppressed more reliably.

According to the present invention, over-detection of defects can be suppressed.

It is a figure which shows the structure of the defect detection apparatus. It is a top view which shows the main body of a defect detection apparatus. It is a figure which shows the structure of a computer. It is a block diagram which shows the functional structure realized by a computer. It is a figure which shows the structure of the 1st defect acquisition part. It is a figure which shows the structure of the 1st defect candidate acquisition part. It is a figure which shows the structure of the 3rd defect candidate acquisition part. It is a figure which shows the structure of the defect candidate limitation part in FIG. It is a figure which shows the structure of the 2nd defect acquisition part. It is a figure which shows the structure of the 2nd defect candidate acquisition part. It is a figure which shows the structure of the 4th defect candidate acquisition part. It is a figure which shows the structure of the defect candidate limitation part in FIG. It is a figure which shows the flow of operation of a defect detection apparatus. It is a figure which shows the flow of the 1st defect acquisition process in FIG. It is a figure which shows the flow of the 2nd defect acquisition process in FIG. It is a figure which shows the flow of the 1st defect candidate acquisition process in FIG. It is a figure which shows the flow of the 2nd defect candidate acquisition process in FIG. It is an image example for demonstrating the processing content of FIG. It is an image example for demonstrating the processing content of the 3rd defect candidate acquisition process of FIG. It is an image example for demonstrating the processing content of the defect candidate limiting process of FIG. It is an image example for demonstrating the processing content of FIG. It is an image example for demonstrating the processing content of the 4th defect candidate acquisition process of FIG. It is an image example for demonstrating the processing content of the defect candidate limiting process of FIG.

FIG. 1 is a diagram showing a configuration of a defect detection device 1 according to an embodiment of the present invention. FIG. 2 is a plan view showing the main body 11 of the defect detection device 1. The defect detection device 1 is a device for inspecting the appearance of a three-dimensional object 9 having a non-mirror surface, and detects defects on the surface of the object 9.

<About the object to be inspected>
The object 9 is, for example, a metal part formed by forging or casting. The surface of the object 9 is satin-like with minute irregularities, that is, in a matte state. In other words, the surface of the object 9 has a gloss while reflecting light diffusely. The glossiness of the surface of the object 9 is preferably about 20 to about 60. The glossiness is an index that one-dimensionally expresses the degree of glossiness on the surface of an object by paying attention to the ratio of specularly reflected light and the directional distribution of diffusely reflected light. The glossiness is measured using a glossometer compliant with the glossiness measurement method of an industrial standard such as JIS.

The surface of the object 9 was treated by shot blasting using a predetermined projecting material such as sandblasting or steel shot blasting, and corners and burrs at the edges and boundaries between the upper and lower surfaces and the side surfaces were removed. It has a shape (so-called R shape). The object 9 is, for example, various parts used for a universal joint, for example, a metal part used for a driving part of a vehicle, an aircraft, or a generator.

A defect on the surface of the object 9 is a portion that is concave or convex with respect to an ideal shape. Defects include, for example, dents, scratches, processing defects, and the like. The defect may be a foreign object adhering to the surface.

<Configuration of defect detection device>
As shown in FIG. 1, the defect detection device 1 includes a main body 11 and a computer 12. The main body 11 includes a holding unit 2, a plurality of imaging units 3 (in FIG. 1, reference numerals 3a, 3b, 3c are attached, but when these are not distinguished, a code 3 is attached), and a light irradiation unit 4. .. The object 9 is held by the holding unit 2. The main body 11 is provided with a light-shielding cover (not shown) for preventing external light from reaching the holding unit 2, and the holding unit 2, the imaging unit 3, and the light irradiation unit 4 are provided in the light-shielding cover.

When the entire surface of the object 9 is automatically inspected, another main body 11 is provided. Then, a mechanism for transporting the object 9 by inverting the object 9 upside down is provided between the two main bodies 11.

As shown in FIGS. 1 and 2, the plurality of imaging units 3 include one upper imaging unit 3a, eight oblique imaging units 3b, and eight side imaging units 3c. The upper imaging unit 3a is arranged above the holding unit 2. An image of the object 9 on the holding unit 2 captured from directly above can be acquired by the upper imaging unit 3a.

As shown in FIG. 2, when the main body 11 is viewed from the upper side to the lower side (that is, when the main body 11 is viewed in a plan view), eight oblique imaging units 3b are arranged around the holding unit 2. To. The eight oblique imaging units 3b pass through the center of the holding unit 2 and are arranged at an angular interval (angle pitch) of 45 ° in the circumferential direction centered on the central axis J1 facing in the vertical direction. As shown in FIG. 1, the angle θ2 formed by the optical axis K2 and the central axis J1 is approximately 45 ° on the surface of each oblique imaging unit 3b including the optical axis K2 and the central axis J1. Each oblique imaging unit 3b can acquire an image obtained by capturing an object 9 on the holding unit 2 from diagonally above. If the object 9 is imaged from diagonally above, the angle θ2 is not limited to 45 °, and is preferably set arbitrarily in the range of 15 to 75 °.

When the main body 11 is viewed in a plan view, the eight lateral imaging units 3c are also arranged around the holding unit 2 in the same manner as the eight oblique imaging units 3b. The eight lateral imaging units 3c are arranged at an angular interval of 45 ° in the circumferential direction. The angle θ3 formed by the optical axis K3 and the central axis J1 on the surface of each side imaging unit 3c including the optical axis K3 and the central axis J1 is approximately 90 °. An image obtained by capturing an object 9 on the holding unit 2 from the side can be acquired by each side imaging unit 3c.

The upper imaging unit 3a, the oblique imaging unit 3b, and the side imaging unit 3c have, for example, a two-dimensional image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor), and have a multi-gradation image. Is obtained. The upper image pickup unit 3a, the oblique image pickup unit 3b, and the side image pickup unit 3c are supported by a support portion (not shown).

The light irradiation unit 4 includes one upper light source unit 4a, eight oblique light source units 4b, and eight side light source units 4c. The upper light source unit 4a is adjacent to the upper imaging unit 3a. In the upper light source unit 4a, a plurality of LEDs (light emitting diodes) are arranged perpendicularly to the central axis J1, that is, horizontally. The upper light source unit 4a irradiates the object 9 on the holding unit 2 with light from substantially directly above.

When the main body 11 is viewed in a plan view, the eight oblique light source portions 4b are arranged around the holding portion 2. Each of the oblique light source units 4b is adjacent to the oblique imaging unit 3b. The eight oblique light source units 4b are arranged at an angular interval of 45 ° in the circumferential direction. In each oblique light source unit 4b, a plurality of LEDs are arranged substantially perpendicular to the optical axis K2. In each oblique light source unit 4b, light can be applied to the object 9 on the holding unit 2 from diagonally above.

When the main body 11 is viewed in a plan view, the eight side light source portions 4c are arranged around the holding portion 2. The side light source units 4c are adjacent to the side image pickup units 3c, respectively. The eight side light source units 4c are arranged at an angular interval of 45 ° in the circumferential direction. In each side light source unit 4c, a plurality of LEDs are arranged substantially perpendicular to the optical axis K3 and in the horizontal direction. Therefore, in a plan view, the eight side light source portions 4c form a substantially octagon. In each side light source unit 4c, light can be irradiated from the side to the object 9 on the holding unit 2. A type of light source other than the LED may be used in the upper light source unit 4a, the oblique light source unit 4b, and the side light source unit 4c. In the present embodiment, the color of light is white, but the color of light and the wavelength band may be changed in various ways. The light irradiation unit 4 can irradiate the object 9 with diffused light from various directions. Hereinafter, each state in which light is emitted from at least one specific light source unit and the object 9 is illuminated with biased light is referred to as an “illumination state”. The light irradiation unit 4 can irradiate the object 9 with light in a plurality of biased illumination states different from each other.

FIG. 3 is a diagram showing the configuration of the computer 12. The computer 12 has a general computer system configuration including a CPU 121 that performs various arithmetic processes, a ROM 122 that stores basic programs, and a RAM 123 that stores various information. The computer 12 includes a fixed disk 124 for storing information, a display 125 for displaying various information such as images, a keyboard 126a and a mouse 126b for receiving input from an operator (hereinafter collectively referred to as "input unit 126"). Further, a reading device 127 that reads information from a computer-readable recording medium 8 such as an optical disk, a magnetic disk, or a magneto-optical disk, and a communication unit 128 that transmits / receives a signal to / from another configuration of the defect detecting device 1. Including.

In the computer 12, the program 80 is read in advance from the recording medium 8 via the reading device 127 and stored in the fixed disk 124. The CPU 121 executes arithmetic processing while using the RAM 123 and the fixed disk 124 according to the program 80. The CPU 121 functions as a calculation unit in the computer 12. In addition to the CPU 121, another configuration that functions as a calculation unit may be adopted.

FIG. 4 is a diagram showing a function realized by the computer 12 executing arithmetic processing according to the program 80. In FIG. 4, the image pickup control unit 51, the defect acquisition unit 52, and the storage unit 53 correspond to the functions realized by the computer 12. All or part of these functions may be realized by a dedicated electric circuit. Further, these functions may be realized by a plurality of computers.

The image pickup control unit 51 controls the image pickup unit 3 and the light irradiation unit 4, and acquires an image of the object 9 (more accurately, data indicating the image). The image data is stored in the storage unit 53. In FIG. 4, the image pickup unit 3 is shown by one block, but in reality, the upper image pickup unit 3a, the oblique image pickup unit 3b, and the side image pickup unit 3c are connected to the image pickup control unit 51.

As will be described later, each time the image pickup control unit 51 controls each light source unit of the light irradiation unit 4 to change the illumination state, image data is acquired by at least one of the 17 image pickup units 3. Hereinafter, the image acquired by imaging is referred to as "captured image", and the data is referred to as "captured image data". The captured image data 911 is stored in the storage unit 53. In the storage unit 53, the image data of the ideal object 9 under each illumination state is stored as the reference image data 912. That is, ideal image data corresponding to each lighting state of each imaging unit 3 is prepared in the storage unit 53 as reference image data 912.

The light illumination state by the light irradiation unit 4 refers to a state in which the object 9 is irradiated with light from a specific irradiation direction. The irradiation direction is not strictly defined and means an approximate light irradiation direction. The irradiation direction is not limited to parallel light when the light is irradiated only from that direction. When light is emitted from a certain direction, it means that the light is emitted unevenly from that direction. Further, the number of light irradiation directions in one shooting is not limited to one. For example, light may be emitted from a plurality of light source units separated from each other at the same time.

The defect acquisition unit 52 includes a first defect acquisition unit 521 and a second defect acquisition unit 522. The "first defect" means a defect that appears dark in the image in the present embodiment, that is, a "dark defect". The "second defect" means a defect that appears brightly in an image in the present embodiment, that is, a "bright defect".

Hereinafter, various configurations of the first defect acquisition unit 521 will be described with reference to FIGS. 5 to 8.

FIG. 5 is a diagram showing the configuration of the first defect acquisition unit 521. The first defect acquisition unit 521 includes a first defect candidate acquisition unit 541, a third defect candidate acquisition unit 551, and a defect candidate limitation unit 581. Based on the input captured image data 911 and reference image data 912, the first defect acquisition unit 521 performs processing to acquire the first defect data 931 which is the image data indicating the first defect region. The first defect region is a region that is darker than the reference image in the captured image due to various processes described later (a region that is not originally a first defect in the captured image and is used as a first defect candidate region described later. This is the area from which the detected area) has been removed.

FIG. 6 is a diagram showing the configuration of the first defect candidate acquisition unit 541. The first defect candidate acquisition unit 541 includes two filter processing units 542, a prealignment unit 543, an expansion processing unit 544, a contraction processing unit 545, a comparison unit 546, a binarization unit 547, and an area filter unit. Includes 548 and. Based on the input captured image data 911 and reference image data 912, the first defect candidate acquisition unit 541 performs processing to acquire the first defect candidate data 921, which is image data indicating the first defect candidate region. To. The first defect candidate region is a region that is darker than the reference image in the captured image.

FIG. 7 is a diagram showing the configuration of the third defect candidate acquisition unit 551. The third defect candidate acquisition unit 551 includes two filter processing units 552, a prealignment unit 553, a shaking comparison unit 554, a binarization unit 555, and an area filter unit 556. Based on the input captured image data 911 and reference image data 912, the third defect candidate acquisition unit 551 performs processing to acquire the third defect candidate data 923, which is image data indicating the third defect candidate region. To. The third defect candidate region is a region that is darker than the reference image in the captured image. As will be described later, the method of acquiring the third defect candidate region is different from the method of acquiring the first defect candidate region.

FIG. 8 is a diagram showing the configuration of the defect candidate limiting unit 581. The defect candidate limiting unit 581 includes two or more logical product calculation units 582 and 583 and an area selection unit 584. The first defect data 931 is acquired by the defect candidate limiting unit 581 performing processing based on the input first defect candidate data 921 and the third defect candidate data 923.

Next, various configurations of the second defect acquisition unit 522 will be described with reference to FIGS. 9 to 12.

FIG. 9 is a diagram showing the configuration of the second defect acquisition unit 522. The second defect acquisition unit 522 includes a second defect candidate acquisition unit 561, a fourth defect candidate acquisition unit 571, and a defect candidate limitation unit 591. Based on the input captured image data 911 and reference image data 912, the second defect acquisition unit 522 performs processing to acquire the second defect data 932, which is image data indicating the second defect region. The second defect region is a region that is bright with respect to the reference image in the captured image and is noise due to various processes described later (in the captured image, it is a region that is not originally a second defect and is used as a second defect candidate region described later. This is the area from which the detected area) has been removed.

FIG. 10 is a diagram showing the configuration of the second defect candidate acquisition unit 561. The second defect candidate acquisition unit 561 includes two filter processing units 562, a prealignment unit 563, a contraction processing unit 564, an expansion processing unit 565, a comparison unit 566, a binarization unit 567, and an area filter unit. Includes 568 and. The second defect candidate acquisition unit 561 performs processing based on the input captured image data 911 and reference image data 912 to acquire the second defect candidate data 922, which is image data indicating the second defect candidate region. To. The second defect candidate region is a region that is brighter than the reference image in the captured image.

FIG. 11 is a diagram showing the configuration of the fourth defect candidate acquisition unit 571. The fourth defect candidate acquisition unit 571 includes two filter processing units 572, a prealignment unit 573, a shaking comparison unit 574, a binarization unit 575, and an area filter unit 576. Based on the input captured image data 911 and reference image data 912, the fourth defect candidate acquisition unit 571 performs processing to acquire the fourth defect candidate data 924, which is image data indicating the fourth defect candidate region. To. The fourth defect candidate region is a region that is brighter than the reference image in the captured image. As will be described later, the method of acquiring the fourth defect candidate region is different from the method of acquiring the second defect candidate region.

FIG. 12 is a diagram showing the configuration of the defect candidate limiting unit 591. The defect candidate limiting unit 591 includes two or more logical product calculation units 592, 593 and an area selection unit 594. The second defect data 932 is acquired by the defect candidate limiting unit 591 performing processing based on the input second defect candidate data 922 and the fourth defect candidate data 924.

Here, in the case of high-speed processing, a large number of first defect candidate acquisition units 541 and a third defect candidate acquisition unit 551 are provided in the first defect acquisition unit 521, and a large number of second defect candidate acquisition units 561. A fourth defect candidate acquisition unit 571 is provided in the second defect acquisition unit 522, and processing is performed in parallel for a plurality of captured images.

<Operation of defect detection device>
FIG. 13 is a diagram showing an operation flow of the defect detection device 1. First, the object 9 to be inspected is held on the holding unit 2. The holding portion 2 is provided with, for example, a contact portion for alignment, and when the contact portion comes into contact with a predetermined portion of the object 9, the object 9 is oriented at a predetermined position. It is placed at. The holding portion 2 may be a stage provided with a positioning pin.

Next, the image pickup control unit 51 performs imaging by the selected image pickup unit 3 while changing the illumination state by changing the light source unit to be lit (steps S11 to S13). That is, during the light irradiation step (step S11) in which the image pickup control unit 51 issues a lighting command to the light irradiation unit 4 and the selected light irradiation unit 4 lights up, and during the lighting of the light irradiation unit 4, the image pickup control unit 51 An imaging step (step S12) of capturing an image of an object 9 by the imaging unit 3 selected by the above and acquiring an captured image, and after the imaging step, the imaging control unit 51 determines whether or not to perform the next imaging. The imaging continuation determination step (step S13) is executed.

Specifically, one side imaging unit 3c is selected, and one of five horizontal light source units 4c that are continuous in the horizontal direction with the side imaging unit 3c as the center is selected and lit in order, and each lighting is performed. The side imaging unit 3c acquires an image. The above operation is repeated while changing the side imaging unit 3c. Actually, the operation time can be shortened by performing imaging with a plurality of side imaging units 3c in each illumination state. Further, all the side light source units 4c are turned on, and all the side image pickup units 3c perform imaging. As a result, six images are acquired by each side imaging unit 3c.

In the case of the oblique imaging unit 3b, one oblique imaging unit 3b is selected, one of the eight oblique light source units 4b is selected and lit in order, and the oblique imaging unit 3b acquires an image each time it is lit. To do. The above operation is repeated while changing the oblique imaging unit 3b. Actually, the operation time can be shortened by performing imaging with all the oblique imaging units 3b in each illumination state. Further, all the oblique light source units 4b are turned on, and all the oblique imaging units 3b perform imaging. Even when only the upper light source unit 4a is lit, all the oblique imaging units 3b perform imaging. As a result, 10 images are acquired by each oblique imaging unit 3b.

In the case of the upper imaging unit 3a, the illumination state is changed and 10 images are acquired in the same manner as in the oblique imaging unit 3b. In the actual operation, the operation time can be shortened by performing the image pickup by the upper image pickup unit 3a at the time of the image pickup of the oblique image pickup unit 3b.

The image data of the captured image is stored in the storage unit 53 as the captured image data 911. As described above, the storage unit 53 prepares the data of the reference image corresponding to each captured image as the reference image data 912. The reference image shows an object 9 (so-called non-defective product) extracted as a reference for defect inspection under the same lighting conditions as the captured image. The reference image data 912 may be acquired by imaging an object 9 having no defects, or may be acquired as data of an average image of images of a large number of objects 9.

When all the imaging steps scheduled by the imaging control unit 51 are completed, the defect acquisition unit 52 then executes the first defect acquisition step (step S14) and the second defect acquisition step (step S15). The first defect acquisition step S14 is a step of acquiring the first defect included in the captured image. The second defect acquisition step S15 is a step of acquiring the second defect included in the captured image. In the present embodiment, the first defect acquisition step S14 and the second defect acquisition step S15 are executed in parallel by the defect acquisition unit 52, but the implementation of the present invention is not limited to this, and the first defect acquisition step S14 may be executed before the second defect acquisition step S15, or may be in the reverse order.

Hereinafter, in order to simplify the expression, the processing for the image data may be expressed simply as the processing for the image. Further, only the processing focusing on one imaging unit 3 will be described. The same processing is performed on the other imaging unit 3. The defect detection device 1 can detect the presence of the first defect (dark defect) and the presence of the second defect (bright defect). However, in the following description, the first defect will be focused on and the first defect will be described. 2. The points common to the first defect detection process in the defect detection process will be omitted as appropriate.

FIG. 14 is a diagram showing details of the first defect acquisition step S14. The first defect acquisition step S14 includes a first defect candidate acquisition step S141, a third defect candidate acquisition step S142, and a defect candidate limiting step S143.

FIG. 15 is a diagram showing details of the second defect acquisition step S15. The second defect acquisition step S15 includes a second defect candidate acquisition step S151, a fourth defect candidate acquisition step S152, and a defect candidate limiting step S153.

FIG. 16 is a diagram showing details of the first defect candidate acquisition step S141. The first defect candidate acquisition step S141 includes the first defect candidate region detection step S1411 for detecting the first defect candidate region, the first mask region detection step S1412 for detecting the first mask region, and the first defect candidate region. , The defect candidate exclusion step S1413 for excluding those overlapping the first mask region by a predetermined condition or more from the first defect candidates, and the post-processing step S1414 are included.

FIG. 17 is a diagram showing details of the second defect candidate acquisition step S151. The second defect candidate acquisition step S151 includes the second defect candidate region detection step S1511 for detecting the second defect candidate region, the second mask region detection step S1512 for detecting the second mask region, and the second defect candidate region. , The defect candidate exclusion step S1513 for excluding those overlapping the second mask region by a predetermined condition or more from the second defect candidate, and the post-processing step S1514 are included.

Hereinafter, the first defect acquisition step S14 will be described with reference to FIGS. 5 to 20 as appropriate.

When the first defect acquisition step S14 is started, first, the first defect candidate acquisition step S141 is executed. In the first defect candidate acquisition step S141, first, one captured image is selected by the first defect acquisition unit 521, and a reference image corresponding to the captured image is selected. As shown in FIG. 6, in the first defect candidate acquisition unit 541, the captured image data 911 of the captured image is input to the filter processing unit 542 (captured image data input step S201), and the reference image data 912 of the reference image is filtered. It is input to the unit 542 (reference image data input step S211).

See FIG. Next, the filter processing unit 542 and the prealignment unit 543 execute the preprocessing steps S202 and S212 for performing the filter processing and the prealignment processing on the captured image and the reference image. In the two filter processing units 542, filter processing for reducing noise such as a median filter and a Gaussian filter is performed on the captured image and the reference image, respectively. The filtered image and the reference image are output to the prealignment unit 543. In the pre-alignment unit 543, the amount of deviation of the position and angle of the reference image relative to the captured image is specified by pattern matching using a predetermined pattern. Then, by translating and rotating the reference image with respect to the captured image by the amount of displacement of the position and angle between the two images, the approximate position and angle of the reference image are adjusted to the captured image. As a result, pre-alignment is performed for both images.

See FIG. FIG. 18 is an image example showing the content of the process in the first defect candidate acquisition step S141. The captured image input by the captured image data input step S201 and executed by the preprocessing step S202 is exemplified as the captured image 811. Further, a reference image input by the reference image data input step S211 and executed by the preprocessing step S212 is exemplified as the reference image 812. Of the surfaces of the object 9 to be inspected, the region appearing in the captured image 811 and the reference image 812 is referred to as a “target region 70”, and is indicated by a rectangle in the captured image 811 and the reference image 812. The imaging unit 3 and the target area 70 have a one-to-one correspondence, and each imaging unit 3 always acquires the same image of the target area 70.

In the target area 70 of the captured image 811, a first defect 711 that appears darker than the surroundings and a second defect 721 that appears brighter than the surroundings can be confirmed. These are, for example, dents and scratches. The target area 70 of the captured image 811 includes pattern portions 712 and 722. The pattern portion 712 is a concave or convex portion that appears brighter than the surroundings, and the pattern portion 722 is a concave or convex portion that appears darker than the surroundings. These pattern portions 712 and 722 are derived from the shape conventionally formed on the object 9, and are not defects.

In the target area 70 of the reference image 812, a second defect 721 that appears brighter than the surroundings can be confirmed. Normally, it is ideal to extract and acquire a defect-free object 9 as a reference image, but there is a possibility that the object 9 extracted for the reference image also contains defects. Further, the target area 70 of the reference image 812 includes the pattern portions 712 and 722 at the same positions as the captured image 811.

The captured image 811 and the reference image 812 shown in FIG. 18 are exaggerated representations of the first defect, the second defect, the pattern portion, and the like for the sake of explanation, and the actual defects and the like are expressed by smaller pixels. It may also be represented. Further, in the captured image 811 and the reference image 812, the defect and the pattern portion are separately described for convenience, but in the actual image, the dark region is either the first defect or the pattern portion, or is bright. Whether the reflected portion is the second defect or the pattern portion is clarified for the first time through the process described later, and it is unknown at the stage of the captured image 811 and the reference image 812 whether the defect or the pattern portion is distinguished.

See FIG. Next, the expansion processing unit 544 executes expansion processing steps S203 and S213 to perform expansion processing on the prealigned captured image and the reference image.

As shown in FIG. 6, in the first defect candidate acquisition unit 541, the captured image data 911 and the reference image data 912 are input from the prealignment unit 543 to the expansion processing unit 544, and the captured image and the reference image are expanded. .. The expansion process here is a process for expanding a bright region in a multi-valued image. In the expansion treatment, a known maximum value filter is used. The maximum value filter is a filter that extracts the pixel having the maximum pixel value from the pixels within the kernel size centered on the pixel of interest and changes the pixel value of another pixel to the maximum pixel value. By passing through the maximum value filter, a pixel having a high pixel value (that is, bright) of 1 pixel is expanded to 3 pixels × 3 pixels.

As a result, the expanded image 811a and the expanded image 812a of FIG. 18 are generated. The expanded image 811a and the expanded image 812a are multi-valued images. In the expanded image 811a and the expanded image 812a, the bright region 721a and the pattern portion 712a are confirmed in the expanded state, while the small dark region disappears.

Regarding the implementation of the present invention, the kernel size of the maximum value filter is not limited to 3 pixels × 3 pixels, and various other sizes may be adopted.

See FIG. Next, the contraction processing unit 545 executes the contraction processing steps S204 and S214 for performing the contraction processing on the captured image and the reference image that have been expanded.

As shown in FIG. 6, the data of the captured image and the reference image that have been expanded are input to the contraction processing unit 545, and the captured image and the reference image are subjected to the contraction processing. The shrinkage process here is a process of shrinking a bright region in a multi-valued image. In the shrinkage treatment, for example, a minimum value filter having the same size as the maximum value filter used in the expansion treatment is used, and the bright region is returned to almost the original size. The minimum value filter is a filter that extracts the pixel having the smallest pixel value from the pixels in the kernel size centered on the pixel of interest and changes the pixel value of the other pixel to the minimum pixel value.

As a result, the contraction image 811b and the contraction image 812b shown in FIG. 18 are generated. The contraction image 811b and the contraction image 812b are multi-valued images. In the contracted image 811b and the contracted image 812b, the second defect 721b and the pattern portion 712b, which are bright regions, are confirmed in a state of being substantially returned to their original sizes. By the expansion treatment and the contraction treatment, the large dark region in the original captured image and the reference image is maintained in the original state, and the small dark region disappears. In the image example of FIG. 18, all the dark areas disappear by the expansion process.

See FIG. Next, the comparison unit 546 executes comparison processing steps S205 and S215 that perform comparison processing on the image captured image and the reference image that have undergone shrinkage processing.

As shown in FIG. 6, the data of the captured image and the reference image that have been contracted are input to the comparison unit 546, and the comparison unit 546 first combines the captured image and the contracted image obtained by expanding / contracting the captured image. A comparative image is generated based on at least one of the difference and the ratio (comparison processing step S205), and in parallel with this, at least one of the difference and the ratio between the reference image and the contracted image obtained by expanding / contracting the reference image. A comparison image is generated based on (comparison processing step S215).

The comparison processing step S205 generates a comparative image based on at least one of the difference and the ratio between the captured image and the contracted image obtained by expanding / contracting the captured image. In the defect inspection apparatus 1 of the present application, whether it is based on the difference or the ratio is determined based on the parameters predetermined by the program 80 or the parameters instructed by the user using the input unit 126. Further, various predetermined values, reference values and threshold values used in the defect inspection device 1 of the present application are determined based on parameters predetermined by the program 80 or parameters instructed by the user using the input unit 126. The generation of the comparison image based on the difference and the generation of the comparison image based on the ratio will be described below.

As one of the specific examples of the comparison processing step S205, a comparison image is acquired by the difference (for example, subtraction processing). Specifically, a difference image showing the difference between the two images by subtracting the pixel value in the region overlapping the captured image of the contracted image obtained by expanding / contracting the captured image from the value of each pixel of the captured image. Is obtained. Then, a comparative image obtained by extracting a region in which the absolute value of the pixel is larger than a predetermined threshold value as the first defect candidate region in the difference image is acquired.

In the present embodiment, the value of the pixel whose absolute value of the pixel is equal to or less than a predetermined threshold value in the difference image is set to "0", and the region where the pixel value is larger than 0 is extracted as the first defect candidate region. The method for extracting the first defect candidate region in the present invention is not limited to this, and the first defect candidate region may be extracted by executing a binarization process on the difference image according to a predetermined threshold value.

Generally speaking, in the comparative processing step S205, the brightness of the captured image is lower than that of the contracted image obtained by expanding / contracting the captured image, and the absolute value of the difference in brightness is equal to or higher than the first reference value. The region is represented in the comparison image as the first defect candidate region. The first reference value is a positive value. In other words, in the captured image, a region having a brightness lower than a predetermined value as compared with the contracted image obtained by expanding / contracting the captured image is represented in the comparative image as a first defect candidate region. When the image is monochrome, the pixel value may be regarded as the brightness, and in the case of a color image, the value obtained by performing a predetermined calculation on the pixel value for each color component is treated as the brightness.

Further, as another specific example of the comparison processing step S205, the first defect candidate region is obtained from the ratio of the value of each pixel of the captured image and the value of the corresponding pixel of the contracted image obtained by expanding / contracting the captured image. May be done. Specifically, the pixel value of the ratio image is obtained by dividing the value of each pixel of the contracted image obtained by expanding / contracting the captured image by the value of the corresponding pixel of the captured image. A first reference value larger than 1 is prepared in advance, and a region composed of pixels having a value equal to or higher than the first reference value in the ratio image is extracted as a first defect candidate region, and the first defect candidate region is extracted. A comparative image including is acquired.

Of course, the pixel value of the ratio image may be obtained by dividing the value of each pixel of the captured image by the value of the corresponding pixel of the contracted image obtained by expanding / contracting the captured image. In this case, in the ratio image, a region composed of pixels having a value equal to or less than the first reference value smaller than 1 is extracted as the first defect candidate region, and a comparative image including the first defect candidate region is acquired.

The first reference value does not have to be a constant. The first reference value may be a function of the brightness or pixel value of the captured image. The first reference value may be determined by using the difference and ratio of the brightness or pixel value of the captured image, and other calculations may be used. The point that the first reference value may be set in various ways is the same for the second, third, and fourth reference values described later. The first to fourth reference values do not have to be the same value, and the calculation method may be different. Generally speaking, in the captured image, a region whose brightness is lower than the brightness of the reference image and lower than a value satisfying a predetermined condition is acquired as the first defect candidate region. The "predetermined conditions" may be set individually for each captured image. Further, a plurality of "predetermined conditions" may be used for one captured image. For example, at a position where the pixel value is likely to change with each imaging, such as an edge in a captured image, a first reference value may be set so that it is difficult to detect as a defect candidate region. The above description is the same for the extraction of the second defect candidate region, the third defect candidate region, and the fourth defect candidate region below.

The comparison processing step S215 generates a comparative image based on at least one of the difference and the ratio between the reference image and the contracted image obtained by expanding / contracting the reference image. Regarding the acquisition of the difference image or the ratio image, the same processing as in the comparison processing step S205 for the captured image is performed. In the comparison processing step S215, the first mask region is extracted from the reference image in place of the first defect candidate region by the same processing as in the comparison processing step S205.

As one of the specific examples of the comparison processing step S215, a comparison image is acquired by the difference (for example, subtraction processing). Specifically, a difference image showing the difference between the two images by subtracting the pixel value in the region overlapping the reference image of the contracted image obtained by expanding / contracting the reference image from the value of each pixel of the reference image. Is obtained. Then, a comparison image obtained by extracting a region in which the absolute value of the pixels is larger than a predetermined threshold value as the first mask region in the difference image is acquired.

Further, as another specific example of the comparison processing step S215, the first mask region is obtained from the ratio of the value of each pixel of the reference image to the value of the corresponding pixel of the contracted image obtained by expanding / contracting the reference image. You may. Specifically, the pixel value of the ratio image is obtained by dividing the value of each pixel of the contracted image obtained by expanding / contracting the reference image by the value of the corresponding pixel of the reference image. A first reference value larger than 1 is prepared in advance, and a region composed of pixels having a value equal to or higher than the first reference value in the ratio image is extracted as the first mask region, and a comparison including the first mask region is performed. The image is acquired.

At this time, the predetermined threshold value and the first reference value in the comparison processing step S205 and the predetermined threshold value and the first reference value in the comparison processing step S215 may be the same or different. If they are the same, the processing can be shared, which contributes to the reduction of processing memory and the reduction of processing time.

By the above processing, the comparative image 811c and the comparative image 812c shown in FIG. 18 are acquired. In the comparative image 811c, the first defect candidate regions 711c and 722c are extracted as regions darker than the captured image 811. In addition, 711c corresponds to the first defect (true defect), and 722c corresponds to the pattern portion (false defect). Further, in the comparative image 812c, the first mask region 722c is extracted as a region darker than the reference image 812. Note that 722c corresponds to a pattern portion (false defect).

The captured image data input step S201, the pretreatment step S202, the expansion processing step S203, the shrinkage processing step S204, and the comparison processing step S205 described above are the first defect candidate region detection step S1411 in the present embodiment. To configure. The preprocessing step S202 can be omitted depending on the state of the captured image and the inspection conditions.

Further, the reference image data input step S211, the pretreatment step S212, the expansion treatment step S213, the contraction treatment step S214, and the comparison processing step S215 described above are the first mask region detection steps in the present embodiment. It constitutes S1412. The pretreatment step S212 may be omitted depending on the state of the reference image and the inspection conditions.

See FIG. Next, the comparison unit 546 executes the defect candidate exclusion step S1413 that performs the defect candidate exclusion process on the comparison image based on the captured image.

When the defect candidate exclusion step S1413 is started, the comparison unit 546 first executes a thickening process for thickening the first mask region. Specifically, the comparative image 812c is subjected to an expansion process as a thickening process by using a maximum value filter in the same manner as in the expansion process step S213.

The implementation of the present invention is not limited to the thickening treatment using the maximum value filter, and various thickening treatments can be adopted.

Here, there may be a tolerance between the object reflected in the reference image and the object reflected in the captured image. In addition, there may be a misalignment between the posture of the object when capturing the reference image and the posture of the object when capturing the captured image. Therefore, as will be described later, if the region where the first mask region and the first defect candidate region overlap is simply excluded, the non-defect region (that is, the pattern portion) of the first defect candidate region becomes the first mask region. For the portion that does not overlap well and is misaligned, the region that is not a defect remains as it is as the first defect candidate region. Therefore, the first mask region is thickened to reduce the non-overlapping portion between the first defect candidate region and the first mask region due to the misalignment. As a result, over-detection can be further suppressed.

Subsequently, in the defect candidate exclusion step S1413, a first defect candidate image is generated in which a part of the first defect candidate region in the comparison image of the captured image is excluded from the first defect candidate based on the first mask region. Execute defect candidate exclusion processing. More specifically, among the first defect candidate regions of the comparison image based on the captured image generated by the comparison unit 546, those that overlap the first mask region of the comparison image based on the reference image by a predetermined condition or more are the first defects. A first defect candidate image excluded from the candidates is generated.

Here, as a method of evaluating the overlap between the first defect candidate region and the first mask region, a predetermined difference or sum is taken between each pixel in the comparison image of the captured image and each pixel in the comparison image of the reference image. A method of extracting the first defect candidate region in comparison with the threshold value is used. Which of the difference and sum extraction methods is used is determined based on the parameters predetermined by the program 80 or the parameters instructed by the user using the input unit 126.

When the first defect candidate image is acquired based on the difference, specifically, the value of the pixel at the corresponding position of the comparison image of the reference image is subtracted from the value of each pixel in the comparison image of the captured image. Acquire a difference image showing the difference between the two images. Then, among the difference images, the region where the pixel value is larger than the predetermined threshold value is left as the first defect candidate, and the first defect candidate image is acquired with the other regions set to "0".

When acquiring the first defect candidate image based on the sum, specifically, by adding the value of each pixel in the comparison image of the captured image and the value of the pixel at the corresponding position in the comparison image of the reference image, Acquire a sum image showing the sum of both images. Then, in the sum image, a region in which the pixel value is larger than a predetermined threshold value is set to the pixel value "0", so that the first mask region and the first defect candidate region are excluded as overlapping regions, and the first The first defect candidate image is acquired while leaving the first defect candidate area that does not overlap with the 1 mask area.

The implementation of the present invention is not limited to this, and for the comparison image of the captured image and the comparison image of the reference image, the pixel having a pixel value equal to or higher than a predetermined threshold value is set to "1", and the pixel value lower than the threshold value is set to "1". After performing a binarization process in which the pixel having the above is set to "0", the comparison image of the reference image is subjected to a process of inverting the pixel values "1" and "0", and the pixels of each pixel of the comparison image of the captured image. The first defect candidate image in which the region where the pixel value is "1" is set as the first defect candidate region by taking the logical product of the value and the pixel value of each corresponding pixel in the inverted image of the comparison image of the reference image is obtained. You may get it. In this case, in the post-processing step S1414 described later, the binarization process by the binarization unit 547 can be omitted.

Further, in the present embodiment, the fattening process is performed in the defect candidate exclusion step S1413, but the embodiment of the present invention is not limited to this, the fattening process is omitted, and only the defect candidate exclusion process is performed in the defect candidate exclusion step S1413. May be executed.

See FIG. Next, the binarization unit 547 and the area filter unit 548 execute the post-processing step S1414 that performs post-processing on the first defect candidate image.

As shown in FIG. 6, the first defect candidate image output from the comparison unit 546 is binarized by the binarization unit 547 at a predetermined threshold value. Here, in the first defect candidate image, the first defect candidate region is represented by multiple values, and the other regions are represented by "0". By the binarization process in the binarization unit 547, the first defect candidate region is set to "1", and the other regions are set to "0".

Subsequently, the first defect candidate image that has been binarized is input to the area filter unit 548. The area filter unit 548 deletes the first defect candidate region whose area is smaller than the predetermined value, and the image showing the remaining first defect candidate region is the final first defect candidate image (to be exact, the first defect candidate image). It is acquired as first defect candidate data 921), which is image data indicating one defect candidate area.

By the above processing, the first defect candidate image 821 shown in FIG. 18 is acquired. In the first defect candidate image 821, the first defect candidate region 722c extracted in the comparison image 811c is excluded by taking a difference from the comparison image 812c. As a result, false news caused by the pattern portion 722, which is not originally a defect, can be suppressed. However, in the image example of FIG. 18, due to the tolerance of the object 9 between the captured image 811 and the reference image 812, a part of the first defect candidate region 722c cannot be completely excluded by the first mask region. It remains as the first defect candidate region 722d in the first defect candidate image 821.

The plurality of captured images acquired by the imaging unit 3 are sequentially selected as processing targets, and a number of first defect candidate data 921 equal to the number of captured images is acquired.

See FIG. Next, the third defect candidate acquisition unit 551 executes the third defect candidate acquisition step S142.

As shown in FIG. 7, when the third defect candidate acquisition step S142 is started, first, one captured image selected in the first defect candidate acquisition step S141 and a reference image corresponding to the captured image are selected. To. Then, in the third defect candidate acquisition unit 551, the captured image data 911 of the captured image is input to the filter processing unit 552, and the reference image data 912 of the reference image is input to the filter processing unit 552.

Next, the filtering unit 5 5 2 and the pre-alignment unit 5 5 3 performs the filtering and pre-alignment process on the captured image and the reference image. Since these processes are the same as the pretreatment steps S202 and S212 in the first defect candidate acquisition step S141, the description thereof will be omitted.

Subsequently, the shaking comparison unit 554 performs the shaking comparison processing using the preprocessed captured image and the reference image, and the binarization unit 555 performs the binarization processing on the shake comparison processed image. By doing so, a third defect candidate image is generated. In the shaking comparison unit 554, an evaluation value indicating the difference between the captured image and the reference image is obtained while moving the reference image from the prealigned position little by little up, down, left and right. As the evaluation value, for example, the sum of the absolute values of the (signed) differences of the pixel values in the region where the two images overlap is obtained. Then, an image showing the signed difference between the pixel values of both images at the position where the evaluation value is the minimum is generated. The signed difference image is binarized at a predetermined value by the binarization unit 555, and a third defect candidate image showing the third defect candidate region is generated.

In practice, a signed difference image is not required to simplify the process. Specifically, the value of the pixel of the difference image is obtained by subtracting the value of the corresponding pixel of the captured image from the value of each pixel of the reference image and setting it to 0 when the value is negative. A positive value is prepared in advance, and a region composed of pixels having a value equal to or greater than the positive value in the difference image is acquired as a third defect candidate region.

Generally speaking, a region in the captured image whose brightness is lower than that of the reference image and whose absolute value of the difference in brightness is equal to or greater than the second reference value is acquired as the third defect candidate region. The second reference value is a positive value. In other words, in the captured image, a region having a brightness lower than a predetermined value than the reference image is acquired as a third defect candidate region. When the image is monochrome, the pixel value may be regarded as the brightness, and in the case of a color image, the value obtained by performing a predetermined calculation on the pixel value for each color component is treated as the brightness.

The third defect candidate region may be obtained from the ratio of the value of each pixel of the reference image to the value of the corresponding pixel of the captured image. Specifically, the value of the pixel of the ratio image is obtained by dividing the value of each pixel of the reference image by the value of the corresponding pixel of the captured image. A second reference value larger than 1 is prepared in advance, and a region composed of pixels having a value equal to or higher than the second reference value in the ratio image is acquired as a third defect candidate region. Of course, the pixel value of the ratio image may be obtained by dividing the value of each pixel of the captured image by the value of the corresponding pixel of the reference image. In this case, a region composed of pixels having a value equal to or less than a second reference value smaller than 1 in the ratio image is acquired as a third defect candidate region.

The second reference value does not have to be a constant. The second reference value may be a function of the brightness or pixel value of the reference image and / or the captured image. The second reference value may be determined using the difference and ratio of the brightness or pixel value of the reference image and the captured image, and other calculations may be used.

FIG. 19 is an image example showing the content of the process in the third defect candidate acquisition step S142. The input captured image 811 and reference image 812 are the same as the images illustrated in FIG. The third defect candidate image generated as a result of executing the shaking comparison processing by the shaking comparison unit 554 and the binarization processing by the binarization unit 555 is exemplified as the third defect candidate image 823.

In the third defect candidate image 823, regions of the captured image 811 that are darker than the reference image 812 are extracted as the third defect candidate regions 711i and 721i by the shaking comparison process. Of these, the third defect candidate region 711i is a true defect corresponding to the first defect 711 existing in the captured image 811, while the third defect candidate region 721i is not present in the captured image 811 but is a reference image. Since it exists as a second defect 721 (that is, a defect that appears brighter than the surroundings) in 812, it is a so-called false defect that has been extracted as a region darker than the reference image 812 by the shaking comparison process. As described above, in the shaking comparison process, if a defect region is included in the reference image, the defect region not included in the captured image may be extracted as noise due to the defect region.

See FIG. 7. When the third defect candidate areas 711i and 721i are acquired by the shaking comparison process and the binarization process, the area filter unit 556 deletes the third defect candidate area whose area is smaller than the predetermined value, and the rest. The image showing the third defect candidate area of is acquired as the final third defect candidate image (more accurately, the third defect candidate data 923 which is the image data showing the third defect candidate area).

The plurality of captured images acquired by the imaging unit 3 are sequentially selected as processing targets, and a number of third defect candidate data 923 equal to the number of captured images is acquired.

See FIG. When the first defect candidate data 921 and the third defect candidate data 923 are acquired, the defect candidate limiting step S143 is then executed, and the first defect is executed based on the first defect candidate data 921 and the third defect candidate data 923. Data 931 is acquired.

As shown in FIG. 8, when the defect candidate limiting step S143 is started, the first defect candidate data 921 and the third defect candidate data 923 are input to the logical product calculation unit 582. Here, the first defect candidate data 921 input to the logical product calculation unit 582 is one captured image acquired by the predetermined first imaging unit 3 under the predetermined first lighting condition and the captured image. It is the data generated from the reference image corresponding to the image. Then, the third defect candidate data 923 input to the same AND calculation unit 582 is the same captured image acquired by the first imaging unit 3 under the first lighting condition and the reference image corresponding to the captured image. It is the data generated from.

Then, the first defect candidate data 921 and the third defect candidate data 923 generated from another captured image and the reference image corresponding to the captured image are input to the logical product calculation unit 583 different from the logical product calculation unit 582. Will be done. Here, the first defect candidate data 921 input to the logical product calculation unit 583 has the same first imaging unit 3 as that of the logical product calculation unit 582, but is different from that of the logical product calculation unit 582. It is data generated from one captured image acquired under the conditions and a reference image corresponding to the captured image. Then, the third defect candidate data 923 input to the same AND calculation unit 583 is obtained from the captured image acquired by the first imaging unit 3 under the second lighting condition and the reference image corresponding to the captured image. This is the generated data.

Then, the data of the logical product image obtained from each combination of the first defect candidate data 921 and the third defect candidate data 923 is input to the area selection unit 584. In other words, each AND image is an image generated from a captured image and a reference image acquired by the same imaging unit under different lighting conditions. The first logical product image obtained by the logical product calculation unit 582 is generated based on the image acquired by the first imaging unit 3 under the first lighting condition, and is obtained by the logical product calculation unit 583. The logical product image of 2 is generated based on the image acquired by the first imaging unit 3 under the second illumination condition.

Each logical product image is an image showing a first defect region generated from the first defect candidate region and the third defect candidate region.

FIG. 20 is a diagram illustrating a first defect image 831 showing a first defect region 731 generated from the first defect candidate regions 711d and 722d of FIG. 18 and the third defect candidate regions 711i and 721i of FIG. ..

The first defect candidate regions 711d and 722d are the first mask regions in the comparative image 812c obtained by extracting the first defect candidate regions 711c and 722c in the comparative image 811c extracted by the so-called self-comparison of the captured images by the so-called self-comparison of the reference image. Since it is obtained by masking with 722c, it is unlikely to detect noise caused by a defect contained in the reference image 812. However, due to the convenience of processing time, there is a possibility of detecting a false defect (first defect candidate region 722d in FIG. 20) due to improper alignment.

On the other hand, the third defect candidate regions 711i and 721i are obtained from the difference images of these images after the captured image 811 and the reference image 812 are aligned, so that the reliability indicating the existence of the defect is high. However, the third defect candidate region may detect the defect included in the reference image 812 as a false defect (third defect candidate region 721i in FIG. 20).

Therefore, in the present embodiment, by obtaining the logical product image of the first defect candidate image 821 and the third defect candidate image 823, the first defect image 831 showing the first defect region 731 with improved reliability is obtained.

In the area selection unit 584, when a plurality of first defect images 831 are overlapped, the area where a predetermined number or more of the first defect areas overlap is maintained as the first defect area, and the area less than the predetermined number is the first defect area. Exclude from. Here, the "overlapping region" may be a logical sum of pixel values in a plurality of overlapping regions, or may be a logical product of the pixel values. In the present embodiment, when the number of overlapping regions is 2 or more, it is maintained as the first defect region. The region selection unit 584 acquires one first defect image showing a limited first defect region from a plurality of first defect images 831 showing the first defect region 731, and the corresponding first defect data 931. Is output from the area selection unit 584. As a result, a dark region with respect to the reference image is extracted as the first defect region from the plurality of captured images acquired under the plurality of illumination states while referring to the corresponding reference image.

The first defect data 931 output from the area selection unit 584 is output from the defect acquisition unit 52 to the storage unit 53 and stored in the storage unit 53.

As a result, the first defect acquisition step S14 for acquiring the presence of the first defect in the captured image is completed.

In parallel with the first defect acquisition step S14, the second defect acquisition step S15 for acquiring the presence of the second defect in the captured image is executed. The second defect is a defect that appears brightly with respect to the reference image in the captured image. The second defect acquisition step S15 is different from the first defect acquisition step S14 in that the expansion processing step and the contraction processing are performed in order to detect a bright region of the captured image as compared with the reference image and other regions inside the captured image. Since the order of the steps is reversed and the order of subtraction and division is reversed, and the other parts are common to the first defect acquisition step S14, the description of the common parts will be omitted as appropriate.

Hereinafter, the second defect acquisition step S15 will be described with reference to FIGS. 9 to 23 as appropriate.

When the second defect acquisition step S15 is started, first, the second defect candidate acquisition step S151 is executed. In the second defect candidate acquisition step S151, first, one captured image is selected by the second defect acquisition unit 522, and the reference image corresponding to the captured image is selected. As shown in FIG. 10 , in the second defect candidate acquisition unit 561, the captured image data 911 of the captured image is input to the filter processing unit 562 (captured image data input step S301), and the reference image data 912 of the reference image is filtered. It is input to the unit 562 (reference image data input step S311).

See FIG. The pretreatment steps S302 and S312 are performed in the same manner as the pretreatment steps S201 and S211 (FIG. 16). Next, the shrinkage processing unit 564 executes the shrinkage processing steps S303 and S313 that perform the shrinkage processing on the prealigned captured image and the reference image.

As shown in FIG. 10, in the second defect candidate acquisition unit 561, the captured image data 911 and the reference image data 912 are input from the prealignment unit 563 to the contraction processing unit 564, and the captured image and the reference image are contracted. .. The shrinkage process here is a process of shrinking a bright region in a multi-valued image, and is also a process of expanding a dark region. This eliminates the small bright areas. In the shrinkage process, for example, a known minimum value filter is used, and a pixel having a low pixel value (that is, dark) of 1 pixel is expanded to 3 pixels × 3 pixels.

FIG. 21 is an image example showing the content of the process in the second defect candidate acquisition step S151, and the input captured image 811 and the reference image 812 are the same images as those in FIG.

The contraction process produces the contraction image 811e and the contraction image 812e of FIG. In the contracted image 811e and the contracted image 812e, the dark region, the first defect 711e and the pattern portion 722e, are confirmed in an expanded state, while the small bright region disappears.

Regarding the implementation of the present invention, the kernel size of the minimum value filter is not limited to 3 pixels × 3 pixels, and various other sizes may be adopted.

See FIG. Next, the expansion processing unit 565 executes expansion processing steps S304 and S314 for performing expansion processing on the captured image and the reference image that have undergone contraction processing.

As shown in FIG. 10, the data of the captured image and the reference image that have been shrink-processed are input to the expansion processing unit 565, and the captured image and the reference image are expanded. The expansion process here is a process for expanding a bright region in a multi-valued image. In the expansion treatment, for example, a maximum value filter having the same size as the minimum value filter used in the contraction treatment is used, and the expanded dark region is returned to almost the original size.

As a result, the expansion image 811f and the expansion image 812f shown in FIG. 21 are generated. In the expanded image 811f and the expanded image 812f, the first defect 711f and the pattern portion 722f, which are dark regions, are confirmed in a state of being substantially returned to their original sizes. By the contraction treatment and the expansion treatment, the large bright region in the original captured image and the reference image is maintained in the original state, and the small bright region disappears. In the image example of FIG. 21, all the bright regions disappear by the shrinkage treatment.

See FIG. Next, the comparison unit 566 executes the comparison processing steps S305 and S315 for performing the comparison processing on the expanded image and the reference image.

As shown in FIG. 10, the data of the captured image and the reference image that have been expanded are input to the comparison unit 566, and the comparison unit 566 first combines the captured image and the expanded image obtained by shrinking / expanding the captured image. A comparative image is generated based on at least one of the difference and the ratio (comparison processing step S305), and in parallel with this, at least one of the difference and the ratio between the reference image and the expanded image obtained by shrinking / expanding the reference image. A comparison image is generated based on (comparison processing step S315).

The processing in the comparison processing steps S305 and S315 is common because it is common except that the reference value used as the threshold value and the order of the numerator denominator when generating the comparison image based on the ratio are different from those in the comparison processing steps S205 and S215. The points to be described will be omitted as appropriate.

As one of the specific examples of the comparison processing step S305, a comparison image is acquired by the difference (for example, subtraction processing). Specifically, a difference image showing the difference between the two images by subtracting the pixel value in the region overlapping the captured image of the expanded image obtained by shrinking / expanding the captured image from the value of each pixel of the captured image. Is obtained. Then, a comparison image obtained by extracting a region in which the absolute value of the pixel is larger than a predetermined threshold value as a second defect candidate region in the difference image is acquired.

Generally speaking, in the comparative processing step S305, in the captured image, the brightness is higher than that of the expanded image obtained by shrinking / expanding the captured image, and the absolute value of the difference in brightness is equal to or higher than the third reference value. The region is represented in the comparative image as a second defect candidate region. The third reference value is a positive value. In other words, in the captured image, a region having a brightness higher than a predetermined value or more than the expanded image obtained by shrinking / expanding the captured image is represented in the comparative image as a second defect candidate region. When the image is monochrome, the pixel value may be regarded as the brightness, and in the case of a color image, the value obtained by performing a predetermined calculation on the pixel value for each color component is treated as the brightness.

Further, as another specific example of the comparison processing step S305, the second defect candidate region is obtained from the ratio of the value of each pixel of the captured image and the value of the corresponding pixel of the expanded image obtained by shrinking / expanding the captured image. May be done. Specifically, the pixel value of the ratio image is obtained by dividing the value of the corresponding pixel of the captured image by the value of each pixel of the expanded image obtained by shrinking / expanding the captured image. A third reference value larger than 1 is prepared in advance, and a region composed of pixels having a value equal to or higher than the third reference value in the ratio image is extracted as a second defect candidate region, and the second defect candidate region is extracted. A comparative image including is acquired.

Of course, the pixel value of the ratio image may be obtained by dividing the pixel value of the expanded image obtained by shrinking / expanding the captured image by the value of each corresponding pixel of the captured image. In this case, in the ratio image, a region composed of pixels having a value equal to or less than a third reference value smaller than 1 is extracted as a second defect candidate region, and a comparative image including the second defect candidate region is acquired.

The comparison processing step S315 generates a comparative image based on at least one of the difference and the ratio between the reference image and the expanded image obtained by shrinking / expanding the reference image. Regarding the acquisition of the difference image or the ratio image, the same processing as in the comparison processing step S305 for the captured image is performed. In the comparison processing step S315, a bright region of the reference image is extracted as the second mask region in place of the second defect candidate region by the same processing as in the comparison processing step S305.

As one of the specific examples of the comparison processing step S315, a comparison image is acquired by the difference (for example, subtraction processing). Specifically, a difference image showing the difference between the two images by subtracting the pixel value in the region overlapping the reference image of the expanded image obtained by shrinking / expanding the reference image from the value of each pixel of the reference image. Is obtained. Then, a comparison image obtained by extracting a region in which the absolute value of the pixels is larger than a predetermined threshold value as a second mask region in the difference image is acquired.

Further, as another specific example of the comparison processing step S315, the second mask region is obtained from the ratio of the value of each pixel of the reference image to the value of the corresponding pixel of the expanded image obtained by shrinking / expanding the reference image. You may.

By the above processing, the comparative image 811 g and the comparative image 812 g shown in FIG. 21 are acquired. In the comparative image 811g, the second defect candidate regions 712g and 721g are extracted as regions brighter than the captured image 811. Further, in the comparative image 812 g, the second mask regions 712 g and 721 g are extracted as regions brighter than the reference image 812. In addition, 721 g corresponds to the second defect (true defect), and 712 g corresponds to the pattern portion (false defect).

The captured image data input step S301, the pretreatment step S302, the shrinkage treatment step S303, the expansion treatment step S304, and the comparison processing step S305 described above are the second defect candidate region detection step S1511 in the present embodiment. To configure. The preprocessing step S302 can be omitted depending on the state of the captured image and the inspection conditions.

Further, the reference image data input step S311, the pretreatment step S312, the shrinkage treatment step S313, the expansion treatment step S314, and the comparison treatment step S315 described above are the second mask region detection steps in the present embodiment. It constitutes S1512. The pretreatment step S312 can be omitted depending on the state of the reference image and the inspection conditions.

See FIG. Next, the comparison unit 566 executes the defect candidate exclusion step S1513 that performs the defect candidate exclusion process on the comparison image based on the captured image.

In the defect candidate exclusion step S1513, a second defect candidate image is generated in which a part of the second defect candidate region in the comparison image of the captured image is excluded from the second defect candidate based on the second mask region. More specifically, similarly to the defect candidate exclusion step S1413, among the second defect candidate regions of the comparison image based on the captured image generated by the comparison unit 566, the second mask region of the comparison image based on the reference image is defined. A second defect candidate image is generated in which those that overlap more than the conditions are excluded from the second defect candidates.

Next, the binarization unit 567 and the area filter unit 568 execute the post-processing step S1514 that performs post-processing on the second defect candidate image, and the final second defect candidate image (to be exact, the second defect candidate image). The second defect candidate data 922), which is image data indicating the defect candidate area, is acquired. Since the post-treatment step S1514 is the same process as the post-treatment step S1414, the description thereof will be omitted.

By the above processing, the second defect candidate image 822 shown in FIG. 21 is acquired. In the second defect candidate image 822, the second defect candidate region 712g extracted from the comparison image 811g is excluded by taking a difference from the comparison image 812g, and only the second defect candidate region 721h remains. As a result, over-detection caused by the pattern portion 712, which is not originally a defect, can be suppressed.

Further, in the comparative image 812g, even if the second defect 721 included in the reference image 812 remains as the second mask region 721g, it is prevented from affecting the second defect candidate image 822. Since the second defect candidate region 721h is extracted from the second defect candidate regions 712g and 721g of the comparative image 811g, noise is generated in the second defect candidate image 822 due to the second defect 721 generated only in the reference image 812. (Ie, false defects) do not occur.

The plurality of captured images acquired by the imaging unit 3 are sequentially selected as processing targets, and a number of second defect candidate data 922 equal to the number of captured images is acquired.

See FIG. Next, the fourth defect candidate acquisition unit 571 executes the fourth defect candidate acquisition step S152.

As shown in FIG. 11, when the fourth defect candidate acquisition step S152 is started, first, one captured image selected in the second defect candidate acquisition step S151 and a reference image corresponding to the captured image are selected. To. Then, in the fourth defect candidate acquisition unit 571, the captured image data 911 of the captured image is input to the filter processing unit 572, and the reference image data 912 of the reference image is input to the filter processing unit 572.

Next, the filter processing unit 572 and the prealignment unit 573 perform filter processing and prealignment processing on the captured image and the reference image. Since these processes are the same as the pretreatment steps S302 and S312 in the second defect candidate acquisition step S151, the description thereof will be omitted.

Subsequently, the shaking comparison unit 574 performs the shaking comparison processing using the preprocessed captured image and the reference image, and the binarization unit 575 performs the binarization processing on the shake comparison processed image. By doing so, a fourth defect candidate image is generated. In the shaking comparison unit 574, an evaluation value indicating the difference between the captured image and the reference image is obtained while moving the reference image from the prealigned position little by little up, down, left and right. As the evaluation value, for example, the sum of the absolute values of the (signed) differences of the pixel values in the region where the two images overlap is obtained. Then, an image showing the signed difference between the pixel values of both images at the position where the evaluation value is the minimum is generated. The signed difference image is binarized at a predetermined value by the binarization unit 575, and a fourth defect candidate image showing the fourth defect candidate region is generated.

In practice, a signed difference image is not required to simplify the process. Specifically, the value of the pixel of the difference image is obtained by subtracting the value of the corresponding pixel of the reference image from the value of each pixel of the captured image and setting it to 0 when the value is negative. A positive value is prepared in advance, and a region composed of pixels having a value equal to or greater than the positive value in the difference image is acquired as a fourth defect candidate region.

Generally speaking, a region in the captured image having a higher brightness than the reference image and an absolute value of the difference in brightness of which is equal to or greater than the fourth reference value is acquired as the fourth defect candidate region. The fourth reference value is a positive value. In other words, in the captured image, a region having a brightness higher than the reference image by a predetermined value or more is acquired as the fourth defect candidate region. When the image is monochrome, the pixel value may be regarded as the brightness, and in the case of a color image, the value obtained by performing a predetermined calculation on the pixel value for each color component is treated as the brightness.

The fourth defect candidate region may be obtained from the ratio of the value of each pixel of the reference image to the value of the corresponding pixel of the captured image. Specifically, the value of the pixel of the ratio image is obtained by dividing the value of each pixel of the captured image by the value of the corresponding pixel of the reference image. A fourth reference value larger than 1 is prepared in advance, and a region composed of pixels having a value equal to or higher than the fourth reference value in the ratio image is acquired as the fourth defect candidate region. Of course, the value of the pixel of the ratio image may be obtained by dividing the value of each pixel of the reference image by the value of the corresponding pixel of the captured image. In this case, a region composed of pixels having a value equal to or less than the fourth reference value smaller than 1 in the ratio image is acquired as the fourth defect candidate region.

The fourth reference value does not have to be a constant. The fourth reference value may be a function of the brightness or pixel value of the reference image and / or the captured image. The fourth reference value may be determined using the difference and ratio of the brightness or pixel value of the reference image and the captured image, and other calculations may be used.

FIG. 22 is an image example showing the content of the process in the fourth defect candidate acquisition step S152. The input captured image 811 and reference image 812 are the same as the images illustrated in FIG. The fourth defect candidate image generated as a result of executing the shaking comparison processing by the shaking comparison unit 574 and the binarization processing by the binarization unit 575 is exemplified as the fourth defect candidate image 824.

In the fourth defect candidate image 824, a region of the captured image 811 that is brighter than the reference image 812 is extracted as the fourth defect candidate region 721j by the shaking comparison process.

Referring to FIG. 1 1. When the fourth defect candidate region 721j is acquired by the shaking comparison processing and the binarization processing, the area filter unit 576 deletes the fourth defect candidate region whose area is smaller than the predetermined value, and the remaining third defect candidate region is deleted. The image showing the four defect candidate areas is acquired as the final fourth defect candidate image (more accurately, the fourth defect candidate data 924 which is the image data showing the fourth defect candidate area).

The plurality of captured images acquired by the imaging unit 3 are sequentially selected as processing targets, and a number of fourth defect candidate data 924 equal to the number of captured images is acquired.

See FIG. When the second defect candidate data 922 and the fourth defect candidate data 924 are acquired, the defect candidate limiting step S153 is then executed, and the second defect is executed based on the second defect candidate data 922 and the fourth defect candidate data 924. Data 932 is acquired.

As shown in FIG. 12, when the defect candidate limiting step S153 is started, the second defect candidate data 922 and the fourth defect candidate data 924 are input to the logical product calculation units 592 and 593. The relationship between the input second defect candidate data 922 and the fourth defect candidate data 924 and the logical product calculation units 592 and 593 (that is, the relationship between the imaging unit and the lighting conditions) is the same as the relationship in the defect candidate limiting step S143. Therefore, the description thereof will be omitted.

The logical product image data obtained from each combination of the second defect candidate data 922 and the fourth defect candidate data 924 is input to the area selection unit 594.

FIG. 23 is a diagram illustrating a second defect image 832 showing a second defect region 732 generated from the second defect candidate region 721h of FIG. 21 and the fourth defect candidate region 721j of FIG. 22.

In the defect candidate exclusion step S1513, a second defect image 832 showing the second defect region 732 with improved reliability is obtained by obtaining a logical product image of the second defect candidate image 822 and the fourth defect candidate image 824. Can be done.

In the area selection unit 594, when a plurality of second defect images 832 are overlapped, the area where a predetermined number or more of the second defect areas overlap is maintained as the second defect area, and the area less than the predetermined number is the second defect area. Exclude from. Here, the "overlapping region" may be a logical sum of a plurality of overlapping regions, or may be a logical product. In the present embodiment, when the number of overlapping regions is 2 or more, it is maintained as a second defect region. The region selection unit 594 acquires one second defect image showing a limited second defect region from a plurality of second defect images 832 showing the second defect region 732, and the corresponding second defect data 932. Is output from the area selection unit 594. As a result, a region brighter than the reference image is extracted as the second defect region from the plurality of captured images acquired under the plurality of illumination states while referring to the corresponding reference image.

The second defect data 932 output from the area selection unit 594 is output from the defect acquisition unit 52 to the storage unit 53 and stored in the storage unit 53.

As a result, the second defect acquisition step S15 for acquiring the presence of the second defect in the captured image is completed.

By the above processing, the position of the defect when the defect is present is detected in the target area 70 as seen from one imaging unit 3.

One captured image is displayed on the display unit of the computer 12, and the first defect region and the second defect region are displayed together as regions with different colors on the target region 70. The implementation of the present invention is not limited to this, and one of the first defect area and the second defect area is displayed on the display unit, and the other is displayed so as to be switchable based on the input of the user. Is also good.

The minimum number of captured images acquired by one imaging unit 3 is 2, but preferably 3 or more. That is, the light irradiation unit 4 can be in three or more different illumination states, for example, the object 9 can be irradiated with light from three or more directions, and three or more can be controlled by the image pickup control unit 51. The imaging unit 3 acquires an image during the illumination state. When a preferable lighting state is known, at least one of three or more captured images acquired based on the information may be selected as the captured image to be processed. By preparing three or more captured images, more appropriate defect detection can be easily performed.

In general, the first defect candidate and the second defect candidate in the above description can be expressed as "defect candidate", and the first defect candidate area and the second defect candidate area are expressed as "defect candidate area". The first mask area and the second mask area can be expressed as a "mask area", and the first defect candidate image and the second defect candidate image can be expressed as a "defect candidate image".

The defect detection device 1 can be modified in various ways.

The arrangement and number of the light source units 4a, 4b, 4c and the imaging unit 3 may be changed as appropriate. The illumination state by the light irradiation unit 4 may be changed in various ways. Two of the plurality of light source units may be lit, or three may be lit. In the light irradiation unit 4, the state of state may be changed by moving the light source unit.

Further, in the defect detection method, the third defect candidate acquisition step S142 and the defect candidate limiting step S143 in the first defect acquisition step S14 are omitted, and the first defect candidate region acquired in the first defect candidate acquisition step S141 is used. It may be acquired as one defect area. Also in this case, in the first defect candidate acquisition step S141, among the first defect candidate regions, those that overlap the first mask region by a predetermined condition or more are excluded from the first defect candidates, and are based on the first defect candidate region. Since the presence of defects in the captured image is acquired, over-detection can be suppressed.

Similarly, the fourth defect candidate acquisition step S152 and the defect candidate limiting step S153 in the second defect acquisition step S15 are omitted, and the second defect candidate region acquired in the second defect candidate acquisition step S151 is used as the second defect region. You may get it. Also in this case, in the second defect candidate acquisition step S151, among the second defect candidate regions, those that overlap the second mask region by a predetermined condition or more are excluded from the second defect candidates, and are based on the second defect candidate region. Since the presence of defects in the captured image is acquired, over-detection can be suppressed.

In this case, the defect detection device 1 does not particularly provide the third defect candidate acquisition unit 551, the fourth defect candidate acquisition unit 571, the defect candidate limiting unit 581 and 591, and the first defect candidate acquisition unit 541 outputs the first defect candidate unit 541. The defect candidate data 921 may be acquired as the first defect data 931, or the second defect candidate data 922 output by the second defect candidate acquisition unit 561 may be acquired as the second defect data 932.

The computer 12 may be dedicated hardware, or partially dedicated hardware may be used. When performing visual inspection at high speed, it is preferable that parallel processing is performed by a computer or dedicated hardware.

If substantially the same processing is performed, the processing order can be changed as appropriate. The processing sequence described in the above embodiment is only an example.

The defect detection device 1 may be used for detecting defects on the surface of other objects such as various substrates and films on which patterns are formed. The defect detection device 1 is particularly suitable for inspecting an object in which over-detection is likely to occur due to having a satin-like region (not limited to a metal surface) on the surface.

The above-described embodiments and configurations in the respective modifications may be appropriately combined as long as they do not conflict with each other.

1 Defect detection device 2 Holding unit 3 Imaging unit (3a Upper imaging unit, 3b Orthorhombic imaging unit, 3c Side imaging unit)
4 Light irradiation unit (4a upper light source unit, 4b oblique light source unit, 4c side light source unit)
8 Recording medium 9 Object 11 Main unit 12 Computer 51 Imaging control unit 52 Defect acquisition unit 53 Storage unit 70 Target area 80 Program 124 Fixed disk 125 Display 126 Input unit (126a keyboard, 126b mouse)
127 Reader 128 Communication unit 521 First defect acquisition unit 522 Second defect acquisition unit 541 First defect candidate acquisition unit 544 Expansion processing unit 545 Shrinkage processing unit 546 Comparison unit 551 Third defect candidate acquisition unit 554 Comparison unit 561 Second defect Candidate acquisition unit 564 Shrinkage processing unit 565 Expansion processing unit 566 Comparison unit 571 Fourth defect candidate acquisition unit 574 Comparison unit 581 Defect candidate limiting unit 582,583 Logical product calculation unit 584 Area selection unit 591 Defect candidate limiting unit 592,593 Logical product Calculation unit 711 First defect 721 Second defect S12 Imaging process S14 First defect acquisition process S141 First defect candidate acquisition process S14111 First defect candidate area detection process S1412 First mask area detection process S1413 Defect candidate exclusion process S142 Third defect Candidate acquisition process S143 Defect candidate limiting process S15 Second defect acquisition process S151 Second defect candidate acquisition process S1511 Second defect candidate area detection process S1512 Second mask area detection process S1513 Defect candidate exclusion process S152 Third defect candidate acquisition process S153 Defects Candidate limited process S203, S213, S304, S314 Expansion process S204, S214, S303, S313 Shrinkage process S205, S215, S305, S315 Comparison process

Claims (4)

A defect detection device that detects defects on the surface of an object.
An imaging unit that captures an object and acquires an captured image,
A storage unit that stores a reference image corresponding to the captured image,
The value of each pixel of the captured image and one of the expansion treatment using the maximum value filter and the shrinkage treatment using the minimum value filter are performed on the captured image, and further, the one-sided treatment of the expansion treatment and the shrinkage treatment is performed. Detects a defect candidate region based on at least one of the difference or ratio between the value of the pixel corresponding to the captured image of the image obtained by performing different other processing, and the value of each pixel of the reference image. The mask region is detected based on at least one of the difference or the ratio of the reference image to the pixel value corresponding to the reference image of the image obtained by subjecting the one-sided processing to the other-processing. , Of the defect candidate regions, the region that overlaps with the mask region is excluded from the defect candidates, and the captured image and the reference image are more accurately aligned than when the defect candidate region and the mask region are overlapped. After the alignment is performed, when the one-sided processing is the expansion processing, the region where the captured image is darker than the reference image is set as another defect candidate based on the difference image or the ratio image of these images. When the region is acquired as a region and the one-sided processing is a contraction processing, a region in which the captured image is brighter than the reference image is used as another defect candidate region based on the difference image or the ratio image of these images. The region where the defect candidate region and the other defect candidate region overlap is acquired as a new defect candidate region, and then the presence of a defect in the captured image is acquired based on the new defect candidate region. Defect acquisition department and
A defect detection device. The defect detection device according to claim 1.
The defect acquisition unit acquires the existence of the first defect in the captured image and the second defect different from the first defect.
At least one of the difference or ratio between the value of each pixel of the captured image and the value of the pixel corresponding to the captured image of the image obtained by subjecting the captured image to expansion processing and then contraction processing. Based on this, the first defect candidate region is detected, and the value of each pixel of the reference image and the pixel corresponding to the reference image of the image obtained by subjecting the reference image to expansion processing and then contraction processing. The first mask region is detected based on at least one of the difference or the ratio with the value of, and the region of the first defect candidate region that overlaps with the first mask region is excluded from the first defect candidate. After aligning the captured image and the reference image more accurately than the alignment when the first defect candidate region and the first mask region are overlapped, the difference image or the ratio image of these images is used. The region where the captured image is darker than the reference image is acquired as the third defect candidate region, and the region where the first defect candidate region and the third defect candidate region overlap is a new first defect candidate. A first defect acquisition unit that acquires the presence of the first defect in the captured image based on the new first defect candidate region after acquiring the region.
At least one of the difference or ratio between the value of each pixel of the captured image and the value of the pixel corresponding to the captured image of the image obtained by subjecting the captured image to the contraction process and then the expansion process. Based on this, the second defect candidate region is detected, and the value of each pixel of the reference image and the pixel corresponding to the reference image of the image obtained by subjecting the reference image to shrinkage processing and then expansion processing. The second mask region is detected based on at least one of the difference or the ratio with the value of, and the region of the second defect candidate region that overlaps with the second mask region is excluded from the second defect candidate. After aligning the captured image and the reference image more accurately than the alignment when the second defect candidate region and the second mask region are overlapped, the difference image or the ratio image of these images is used. The region where the captured image is brighter than the reference image is acquired as the fourth defect candidate region, and the region where the second defect candidate region and the fourth defect candidate region overlap is a new second defect candidate. A second defect acquisition unit that acquires the presence of the second defect in the captured image based on the new second defect candidate region after acquiring the region.
A defect detection device, characterized in that it has. A defect detection method that detects defects on the surface of an object.
a) An imaging process in which an object is imaged by an imaging unit to acquire an captured image,
b) The value of each pixel of the captured image and one of the expansion treatment using the maximum value filter and the shrinkage treatment using the minimum value filter are performed on the captured image, and one of the expansion treatment and the shrinkage treatment is further performed. A defect candidate region detection step of detecting a defect candidate region based on at least one of a difference or a ratio between the pixel values corresponding to the captured image of the image obtained by performing the other processing different from the processing.
c) A reference image corresponding to the captured image is prepared, and the value of each pixel of the reference image and the reference image are subjected to the one-sided processing and then the other-processed image. A mask area detection step that detects a mask area based on at least one of the differences or ratios of the pixel values corresponding to the reference image.
d) A defect candidate exclusion step of excluding the region overlapping the mask region from the defect candidates among the defect candidate regions.
e) After aligning the captured image and the reference image more accurately than the alignment when the defect candidate region and the mask region are overlapped, if the one-sided processing is an expansion processing, these When the captured image is darker than the reference image as another defect candidate region based on the difference image or the ratio image of the image, and the one-sided processing is the shrinkage processing, these images A region in which the captured image is brighter than the reference image is acquired as another defect candidate region based on the difference image or the ratio image of the above, and a region in which the defect candidate region and the other defect candidate region overlap is obtained. A defect acquisition step of acquiring the presence of a defect in the captured image based on the new defect candidate region after acquiring it as a new defect candidate region.
A defect detection method. A program that causes a computer to detect defects in the target area from a plurality of images of the target area on the surface of the object, and execution of the program by the computer causes the computer to execute the program.
a) A step of preparing an captured image and a corresponding reference image obtained by imaging the target area, and
b) The value of each pixel of the captured image and one of the expansion treatment using the maximum value filter and the shrinkage treatment using the minimum value filter are performed on the captured image, and one of the expansion treatment and the shrinkage treatment is further performed. A defect candidate region detection step of detecting a defect candidate region based on at least one of a difference or a ratio between the pixel values corresponding to the captured image of the image obtained by performing the other processing different from the processing.
c) A reference image corresponding to the captured image is prepared, and the value of each pixel of the reference image and the reference image are subjected to the one-sided processing and then the other-processed image. A mask area detection step that detects a mask area based on at least one of the differences or ratios of the pixel values corresponding to the reference image.
d) A defect candidate exclusion step of excluding the region overlapping the mask region from the defect candidates among the defect candidate regions.
e) After aligning the captured image and the reference image more accurately than the alignment when the defect candidate region and the mask region are overlapped, if the one-sided processing is an expansion processing, these When the captured image is darker than the reference image as another defect candidate region based on the difference image or the ratio image of the image, and the one-sided processing is the shrinkage processing, these images A region in which the captured image is brighter than the reference image is acquired as another defect candidate region based on the difference image or the ratio image of the above, and a region in which the defect candidate region and the other defect candidate region overlap is obtained. A defect acquisition step of acquiring the presence of a defect in the captured image based on the new defect candidate region after acquiring it as a new defect candidate region.
A program characterized by executing.