JP2019090643A - Inspection method, and inspection device - Google Patents

Abstract

To provide an inspection method and an inspection device each of which enables a defect of an inspection object to be detected with higher degree of accuracy.SOLUTION: The inspection method for detecting a defect on the surface of an inspection object includes the steps of: acquiring a captured image of the inspection object; comparing, for each pixel of the captured image, the pixel value of each pixel adjacent to the pixel with the pixel value of the pixel, thereby calculating the gradient direction of the pixel value of the pixel; differentiating, for each pixel of the captured image, the pixel n orders or more (n=natural number 2 or greater) in the gradient direction of the pixel value of the pixel as the direction of differentiation, thereby calculating the n-order differential value of the pixel; and extracting a pixel, among the pixels of the captured image, whose absolute value of calculated n-order differential value is equal to or greater than a threshold, as the contour peripheral pixel of a defect of the inspection object.SELECTED DRAWING: Figure 2

Description

  The present disclosure relates to an inspection method and an inspection apparatus for detecting a defect on the surface of an inspection object.

  2. Description of the Related Art Conventionally, when a defect is inspected for quality assurance during a manufacturing process of a mass-produced product or after the product is completed, an automatic inspection is performed by visual inspection with human eyes or image recognition such as a camera. Automatic appearance inspection involves an initial cost such as equipment price and setting of inspection conditions compared to visual inspection, but elimination of personalities, improvement of inspection repeatability, shortening of inspection time, education of inspection personnel and management cost It is widely used in large-scale manufacturing sites where the initial cost can be expected to be profitable, because the merits such as the reduction of In particular, inspections using more advanced calculations have become possible with realistic tact times due to the recent improvement of the arithmetic processing capability and communication speed of computers and the improvement of camera performance.

  As an example of an inspection method for detecting a defect to be inspected by image recognition, a defect inspection method according to Japanese Unexamined Patent Publication No. 2010-197176 (Patent Document 1) is disclosed. In the defect inspection method, the differential absolute value of each pixel is calculated by differential operation based on the density of each pixel in the captured image, and the pixels whose differential absolute value is equal to or greater than a specified value are extracted as edge pixels and edges adjacent to each other A set of pixels is extracted as a position detection land, and by binarizing each pixel of the captured image, defect candidate pixels are extracted, and a set of defect candidate pixels adjacent to each other is extracted as a defect candidate land, The number of pixels of the defect candidate land is measured, and when the number of pixels of the defect candidate land is less than the pixel number threshold, it is determined that the inspection object is a non-defective item.

Unexamined-Japanese-Patent No. 2010-197176

  In the technique according to Patent Document 1, the density change ΔX in the X direction (horizontal direction) and the vertical direction (Y direction) are made based on the density values of eight pixels adjacent to each pixel in the image by differential arithmetic processing using a differential filter. And the derivative absolute value and the derivative direction value are calculated based on .DELTA.X and .DELTA.Y.

  However, according to the technique according to Patent Document 1, the differential operation is performed on the pixel of interest using the density change in a predetermined direction (here, the X direction and the Y direction), so that If there is variation and the difference between the density of the defect and the background is small, the outline of the defect can not be detected accurately. In addition, the magnitude of the density gradient of the background varies, for example, when the density value of the pixel is higher in the order of the maximum density of the captured image, the density of the defect, and the minimum density of the captured image, or When the maximum value of the differential value of the defect is as large as the derivative value of the density of the defect, the difference between the background and the outline of the defect can not be distinguished, and the outline of the defect can not be detected with high accuracy.

  An object in one aspect of the present disclosure is an inspection method capable of detecting a defect of an inspection object more accurately by performing second or higher order differential operation based on the gradient direction of the pixel value of each pixel, and It is providing a test apparatus.

  According to one embodiment, an inspection method for detecting a defect on the surface of an inspection object is provided. The inspection method is a step of acquiring a captured image of the inspection object, and for each pixel of the captured image, comparing the pixel value of each adjacent pixel of the pixel with the pixel value of the pixel, In the step of calculating the gradient direction, for each pixel of the captured image, the pixel is graded n or more (n is a natural number of 2 or more) with the gradient direction of the pixel value of the pixel as the derivative direction A step of calculating a value, and a step of extracting a pixel having an absolute value of the calculated nth-order derivative value equal to or more than a threshold among pixels of the captured image as an outline peripheral pixel of a defect of the inspection object.

  According to another embodiment, an inspection device for detecting a defect on the surface of an inspection object is provided. The inspection apparatus compares the pixel value of each pixel adjacent to the pixel with the pixel value of the pixel by comparing the pixel value of each pixel adjacent to the pixel with the image acquisition unit that acquires the imaged image of the inspection object. A direction calculation unit that calculates the gradient direction of the value, and for each pixel of the captured image, the pixel is differentiated by n (n is a natural number of 2 or more) floors with the gradient direction of the pixel value of the pixel as the differentiation direction Of the differential operation unit for calculating the nth-order differential value of the pixel, and pixels among the pixels of the captured image whose absolute value of the calculated n-order differential value is equal to or greater than the threshold are extracted as outline pixels of the defect of the inspection object And an extraction unit.

  According to the present disclosure, it is possible to detect a defect of the inspection object more accurately by executing the second or higher order differential operation based on the gradient direction of the pixel value of each pixel.

FIG. 1 is a diagram showing an example of the entire configuration of an inspection system according to a first embodiment. 7 is a flowchart showing an example of an inspection method according to the first embodiment. FIG. 6 is a diagram for describing a direction calculation process according to the first embodiment. FIG. 7 is a diagram showing an example of a differential filter according to the first embodiment. 5 is a diagram for illustrating an extraction process according to Embodiment 1. FIG. FIG. 5 is a schematic view showing an example of a functional configuration of the inspection apparatus according to the first embodiment. It is the schematic which shows a power semiconductor chip. It is the schematic for demonstrating the brightness | luminance gradient of a power semiconductor chip. It is a figure which shows the captured image of the main electrode 11. FIG. It is a figure which shows the image which emphasized the contrast of the captured image of FIG. It is a figure which shows the brightness | luminance of the pixel on line segment K1 K2 of FIG. It is a figure which shows the result of having converted the value after the differential processing which applied the Sobel filter to the X direction with respect to the captured image shown in FIG. 3 to brightness | luminance. It is a figure which shows the differential value of the pixel on line segment K1 K2 of FIG. It is a figure which shows the result of having converted the value after the differential processing which applied 8-direction Laplacian filter with respect to the captured image shown in FIG. 9 to brightness | luminance. It is a figure which shows the differential value of the pixel on line segment K1 K2 of FIG. It is a figure which shows the image at the time of performing a smoothing process to the captured image shown in FIG. It is a figure which shows the result of having converted the 1st-order differential value of each pixel of the smooth image shown in FIG. 10 into luminance. It is a figure which shows the first-order differential value of the pixel on line segment K1 K2 of FIG. It is a figure which shows the result of having converted the 2nd-order differential value of each pixel of the smooth image shown in FIG. 17 into a brightness | luminance. FIG. 20 is a diagram showing second derivative values of pixels on line segment K1K2 in FIG. 19; FIG. 7 is a diagram showing a plurality of differential filters according to a modification of the first embodiment. It is a figure which shows the result of having converted the 3rd-order differential value of each pixel of the captured image shown in FIG. 9 into a brightness | luminance. It is a figure which shows the 3rd-order differential value of the pixel on line segment K1 K2 of FIG. FIG. 17 is a diagram for describing an example of an extraction method of a group of outline peripheral pixels according to the second embodiment. FIG. 17 is a diagram for explaining another example of a method of extracting a set of contour peripheral pixels according to the second embodiment. It is a figure which shows the derivative value of the captured image of a circular defect, and a captured image. It is a figure which shows typically the contour periphery pixel shown in FIG.26 (c). It is a figure for demonstrating the extraction system of the aggregate | assembly of an outline periphery pixel. It is a figure for demonstrating the extraction system of the aggregate | assembly of an outline periphery pixel.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same components are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description about them will not be repeated.

Embodiment 1
<System configuration>
FIG. 1 is a diagram showing an example of the entire configuration of inspection system 1000 according to the first embodiment. Referring to FIG. 1, inspection system 1000 includes, as main components, inspection apparatus 100 and camera 8 connected to inspection apparatus 100.

  As an example, the camera 8 is configured to include an imaging device divided into a plurality of pixels, such as a CCD (Coupled Charged Device) or a CMOS (Complementary Metal Oxide Semiconductor) sensor, in addition to an optical system such as a lens. The camera 8 sequentially images the inspection object 10 conveyed by the conveyance mechanism 9. An input image acquired by photographing with the camera 8 is transmitted to the inspection apparatus 100.

  The inspection apparatus 100 inspects whether or not there is a defect on the surface of the inspection object 10 by executing an inspection program. In the inspection apparatus 100, the pixel value of each pixel of the captured image obtained by imaging the inspection object 10 is converted into numerical data, and calculation is performed in association with position coordinates. The pixel values correspond to, for example, gradations of any color such as a green component even if they correspond to, for example, luminance gradations and monochrome (for example, black and white) density gradations. It is also good. The number of gradations may be, for example, 512 or 256. In the following, for ease of explanation, it is assumed that the pixel value is luminance.

  The inspection result by the inspection device 100 is displayed on the display 114 of the inspection device 100. The inspection apparatus 100 typically has a structure according to a general-purpose computer architecture, and the processor executes a pre-installed program to realize various processes to be described later.

  Specifically, the inspection apparatus 100 includes a processor 110, a memory 112, a display 114, a camera interface 116, an input interface 118, a communication interface 120, and a memory interface 122. These units are communicably connected to each other.

  The processor 110 is typically an arithmetic processing unit such as a central processing unit (CPU) or a multi processing unit (MPU). The processor 110 controls the operation of each part of the inspection apparatus 100 by reading and executing the program stored in the memory 112. More specifically, the processor 110 implements each of the processing (steps) of the inspection apparatus 100 described later by executing the program.

  The memory 112 is realized by a random access memory (RAM), a read-only memory (ROM), a flash memory, a hard disk, or the like. The memory 112 stores a program executed by the processor 110, an input image acquired by the camera 8, a processing result on the input image, and the like.

  The display 114 is, for example, a liquid crystal display, an organic EL (Electro Luminescence) display, or the like. The display 114 may be configured integrally with the inspection apparatus 100 or may be configured separately from the inspection apparatus 100.

  The camera interface 116 corresponds to an input unit that receives image data generated by imaging the inspection object 10, and mediates data transmission between the processor 110 and the camera 8. More specifically, the camera interface 116 can be connected to one or more cameras 8, and an imaging instruction is output from the processor 110 to the camera 8 via the camera interface 116. Thus, the camera 8 captures an object and outputs the generated image to the processor 110 via the camera interface 116.

  The input interface 118 mediates data transmission between the processor 110 and input devices such as a keyboard, a mouse, a touch panel, and a dedicated console. That is, the input interface 118 receives an operation command given by the user operating the input device.

  The communication interface 120 mediates data transmission between the processor 110 and other personal computers and server devices (not shown). The communication interface 120 is typically made of Ethernet (registered trademark), USB (Universal Serial Bus), or the like.

  The memory interface 122 mediates data transmission between the processor 110 and an external recording medium. The recording medium is distributed in a state in which the inspection program to be executed by the inspection apparatus 100 is stored, and the memory interface 122 reads the inspection program from the recording medium. Further, in response to the internal command of the processor 110, the memory interface 122 writes the input image acquired by the camera 8, the processing result in the inspection apparatus 100, and the like to the recording medium. As the recording medium, a general-purpose semiconductor storage device such as SD (Secure Digital), a magnetic recording medium such as a flexible disk (Flexible Disk), an optical recording medium such as a CD-ROM (Compact Disk Read Only Memory), etc. It can be mentioned.

<Overview of inspection method>
FIG. 2 is a flowchart showing an example of the inspection method according to the first embodiment. Typically, the flowchart shown in FIG. 2 is executed by the processor 110 of the inspection apparatus 100.

  Referring to FIG. 2, the inspection method includes step S <b> 100 (hereinafter, also referred to as “acquisition process”) of acquiring a captured image of inspection object 10.

  The inspection method includes step S200 (hereinafter, also referred to as a “standardization step”) of standardizing the acquired captured image.

  The inspection method calculates the gradient direction of the pixel value of the pixel by comparing the pixel value of each pixel adjacent to the pixel with the pixel value of the pixel for each pixel of the standardized captured image (see below. , Also referred to as “direction calculation step”.

  In the inspection method, for each pixel of the captured image, second-order differentiation of the pixel is performed with the gradient direction of the pixel value of the pixel as a differentiation direction to calculate a second-order derivative value of the pixel (hereinafter, “differentiation operation step Also referred to as

  In the inspection method, a pixel having a calculated second derivative value equal to or greater than a predetermined threshold value among the pixels of the captured image is extracted as an outline peripheral pixel of the defect of the inspection object 10 (hereinafter referred to as “ Also referred to as “extraction step”.

  The inspection method includes step S600 (hereinafter, also referred to as “determination step”) of determining whether a defect is present in the inspection object 10 based on the extracted outline peripheral pixels.

Hereinafter, the inspection method according to the first embodiment will be specifically described step by step.
<Details of each step of inspection method>
(Acquisition process)
In the acquisition step corresponding to step S100, typically, the processor 110 first outputs an imaging instruction via the camera interface 116. The camera 8 images the inspection object 10 in accordance with the imaging instruction, and outputs a captured image of the inspection object 10. The processor 110 receives an input of the captured image through the camera interface 116 (that is, acquires the captured image), and stores the captured image in the memory 112.

(Standardization process)
In the standardization step corresponding to step S200, typically, the processor 110 executes a smoothing process on the acquired captured image to generate a smoothed image corresponding to the captured image after standardization. The smoothing filter used for the smoothing process is, for example, a Gaussian filter, a median filter, a moving average filter, or the like.

  As another example, in the standardization step, the pickup image acquired in the acquisition step may be standardized using the pickup image of at least one inspection object 10 prepared in advance. For example, a captured image of at least one inspection object 10 is stored in the memory 112.

  Specifically, the processor 110 calculates an average value of pixel values (for example, luminance) of each pixel in a plurality of captured images, and generates a standard image ST1 with the average value as the luminance of each pixel. Specifically, the processor 110 superimposes the at least one captured image and the captured image acquired in the acquisition step, and calculates an average value of the luminance of each pixel present at the same coordinate position in the plurality of captured images. . The processor 110 generates a standard image ST1 in which the calculated average value of the luminance of each pixel is used as the luminance of each pixel. The standard image ST1 is an image obtained by standardizing the captured image acquired in the acquisition step.

  Also, the processor 110 may calculate a median of the luminance of each pixel in a plurality of captured images, and generate a standard image ST2 in which the median is the luminance of each pixel. Specifically, the processor 110 superimposes the at least one captured image and the captured image acquired in the acquiring step, and calculates the median value of the luminance of each pixel present at the same coordinate position in the plurality of captured images. . The processor 110 generates a standard image ST2 in which the calculated median value of the luminance of each pixel is used as the luminance of each pixel. The standard image ST2 is an image obtained by standardizing the captured image acquired in the acquisition step.

  As described above, three types of standardization steps have been described. For example, in the case where there is one captured image or in the case where there is no tendency of the brightness gradient for each individual product and it is random, standardization using a smoothing process is preferable.

  Alternatively, in the case where the tendency of variation in luminance is the same among products to be inspected, standardization using an average value or standardization using a median value is preferable. The reason is that by processing the same tendency in an overlapping manner, it is possible to prevent a situation in which the brightness gradient is shifted due to the influence of the inherent variation of the product.

  In particular, when the difference in luminance between the background of the captured image and the defect is small, standardization using an average value is preferable. This is because the difference in luminance below the decimal point can be used to calculate the luminance gradient direction. On the other hand, when the number of samples of the captured image is large (for example, several tens or more) and the difference between the luminance of the background and the defect is a predetermined value or more (for example, 5 cd / m 2), the median is used. Standardization is preferred. This is because the influence of variations in luminance can be suppressed by using the central value of luminance.

(Direction calculation process)
In the direction calculation step corresponding to step S300, the processor 110 calculates, for each pixel of the captured image standardized in the standardization step, a difference value between the brightness of the pixel and the brightness of each of the eight pixels adjacent to the pixel. Do.

  FIG. 3 is a diagram for explaining a direction calculation process according to the first embodiment. Referring to FIG. 3, assuming that the pixel P5 is a pixel of interest, eight pixels P1 to P4 and P6 to P9 are adjacent pixels to the pixel P5. The processor 110 calculates, for each of the adjacent pixels P1 to P4 and P6 to P9, a difference value between the luminance of the adjacent pixel and the luminance of the pixel P5. The processor 110 calculates the direction from the pixel P5 to the adjacent pixel corresponding to the largest difference value among the eight difference values as the brightness gradient direction of the pixel P5.

  For example, in the processor 110, the difference value between the luminance of the adjacent pixel P7 among the adjacent pixels P1 to P4 and P6 to P9 of the pixel P5 and the luminance of the pixel P5 is the difference between the adjacent pixels P1 to P4 and P6 to P9. The direction 300 from the pixel P5 to the adjacent pixel P7 is calculated as the brightness gradient direction of the pixel P5 when the difference value between the brightness of the remaining adjacent pixels P1 to P4, P6, P8 and P9 and the brightness of the pixel P5 is larger. Do.

  If the plurality of difference values among the eight difference values are the same and the largest difference value is not determined uniquely, the pixel P5 is changed to the adjacent pixel corresponding to any difference value of the plurality of difference values. Is calculated as the luminance gradient direction of the pixel P5. In this case, priorities may be provided in the order of predetermined directions.

  Here, the direction from the pixel P5 to the adjacent pixel P8, the direction from the pixel P5 to the adjacent pixel P7, the direction from the pixel P5 to the adjacent pixel P4, the direction from the pixel P5 to the adjacent pixel P1, and the direction from the pixel P5 to the adjacent pixel P2. (1,0), the direction from pixel P5 to adjacent pixel P2, the direction from pixel P5 to adjacent pixel P3, the direction from pixel P5 to adjacent pixel P6, and the direction from pixel P5 to adjacent pixel P9 (1,0) , (1, 1), (0, 1), (-1, 1), (-1, 0), (-1, -1), (0, -1), (1, 1) .

  For example, from the direction (0,1) to the clockwise order, that is, the direction (0,1), the direction (1,1), the direction (1,0), the direction (1, -1), the direction (0, -1) The priority is set to be higher in the order of (1), direction (−1, −1), direction (−1, 0), direction (−1, 1). If the largest difference value is not uniquely determined, the processor 110 calculates the direction with the highest priority as the luminance gradient direction of the pixel P5.

(Differential operation process)
In the differentiation operation process corresponding to step S400, first, processor 110 sets the luminance gradient direction of each pixel calculated in the direction calculation process as the differentiation direction, and uses the differentiation filter shown in FIG. Calculate

FIG. 4 is a diagram showing an example of the differential filter according to the first embodiment. FIG. 4A shows the luminance C _{x, y} of the pixel at coordinates (x, y). FIG. 4B is a diagram showing an example of the differential filter. FIG. 4C shows another example of the differential filter.

For example, in the direction calculation step, when the luminance gradient direction of the pixel at the coordinate (x, y) is the direction (1, 1), the luminance C _{x, y} is calculated using the differential filter 401 of FIG. The first order differential value D _{x, y of} is expressed by the following equation (1).

D _{x, y} = 2 x C _{x +1, y + 1} + C _{x +1, y} + C _{x, y + 1-} (2 x C _{x -1, y -1} + C _{x -1, y} + C _{x, y-1} ) (1)
Also, for example, in the direction calculation step, when the brightness gradient direction of the pixel at the coordinates (x, y) is the direction (1, 0), the brightness C _x is obtained using the differential filter 402 of FIG. _, first order differential value _{D x} of _{y, y} is expressed by the following equation (2).

D _{x, y} = 2 x C _{x +1, y} + C _{x +1, y-1} + C _{x, y + 1-} (2 x C _{x -1, y} + C _{x -1, y + 1} + C _{x, y-1} ) (2)
When the luminance gradient direction of the pixel at the coordinates (x, y) is the direction (-1, 1), the differential filter 401 is rotated 90 degrees (that is, rotated 90 degrees counterclockwise). Differential operations are performed using this. Similarly, when the luminance gradient direction is the direction (-1, -1) or the direction (1, -1), the differential operation is performed using the differential filter obtained by rotating the differential filter 401 by 180 degrees and 270 degrees, respectively. Is executed.

  When the luminance gradient direction is the direction (0, 1), the differential operation is performed using a differential filter obtained by rotating the differential filter 402 by 90 degrees. Similarly, when the brightness gradient direction is the direction (-1, 0) or the direction (0, -1), the differential operation is performed using the differential filter obtained by rotating the differential filter 402 by 180 degrees and 270 degrees, respectively. To be executed.

  Next, the processor 110 further differentiates the first order differential value of each pixel calculated as described above in the brightness gradient direction to calculate the second order differential value of each pixel.

For example, when the luminance gradient direction of the pixel is the direction (1, 1), the second order differential value DD _{x obtained} by differentiating the first order differential value D _{x, y} using the differential filter 401 of FIG. _{, Y} is represented by the following formula (3).

DD _{x, y} = 2 x D _{x +1, y + 1} + D _{x +1, y} + D _{x, y + 1-} (2 x D _{x-1, y-1} + D _{x-1, y} + D _{x, y-1} ) (3)
(Extraction process)
In the extraction process corresponding to step S500, the processor 110 determines whether the second-order differential value of each pixel calculated in the differential calculation process is equal to or more than the threshold value Th. The processor 110 extracts a pixel whose second derivative value is equal to or more than the threshold value Th among the pixels as a contour peripheral pixel of a defect on the surface of the inspection object 10.

  FIG. 5 is a diagram for explaining an extraction process according to the first embodiment. Specifically, FIG. 5A is an image view of a captured image including the defect 51 of the inspection object 10. FIG. 5B is a diagram showing the luminance of the pixel corresponding to the coordinates present on the line segment Q1Q2 connecting the point Q1 and the point Q2 of the captured image. FIG. 5C is a diagram showing an absolute value (hereinafter, also referred to as “first-order derivative absolute value”) of the first-order derivative value of the pixel corresponding to the coordinates present on the line segment Q1Q2. FIG. 5D is a diagram showing an absolute value (hereinafter also referred to as "second-order derivative absolute value") of a second-order derivative value of a pixel corresponding to a coordinate existing on the line segment Q1Q2.

  Referring to FIGS. 5A and 5B, in the captured image, the luminance is generally high (ie bright) from point Q1 to point Q2, but there is a defect 51. The brightness is almost constant in the part where the Referring to FIG. 5C, the peak value of the first-order differential absolute value is calculated at point Qa and point Qb on line segment Q1Q2. The difference between the luminance of the contour pixel of the defect 51 and the luminance of the background is large, so the first-order differential absolute value of the contour pixel is large. Therefore, points Qa and Qb corresponding to the peak position of the first order differential absolute value indicate the coordinates of the contour pixel.

  Referring to FIG. 5D, at points Qc, Qd, Qe, and Qf on line segment Q1Q2, peak values of second-order differential absolute values are calculated. At points Qa and Qb indicating the coordinates of the contour pixel, the first derivative absolute value is a peak value, and the second derivative absolute value is zero. From this, a pixel having a second derivative absolute value equal to or larger than the threshold value Th is extracted as a pixel existing around the contour pixel, that is, as a contour peripheral pixel of the defect 51. The threshold Th is arbitrarily determined by the user according to the determination criterion of the presence or absence of the defect of the inspection object 10. For example, in the case of applying a stricter determination criterion, the threshold value Th may be reduced to facilitate extraction of contour peripheral pixels.

(Judgment process)
In the process of determining the presence or absence of a defect corresponding to step S600, the processor 110 determines whether or not a defect is detected based on the contour peripheral pixels extracted in the extraction process. In one aspect, processor 110 may determine that a defect is present on the surface of inspection object 10 (ie, a defect has been detected) if contour peripheral pixels are extracted.

  In another aspect, the processor 110 may determine that a defect is present if the number of pixels of a collection of contour peripheral pixels adjacent to each other is equal to or greater than the reference pixel number. In this case, a collection of outline peripheral pixels is detected as a defect. In yet another aspect, the processor 110 may determine that a defect is present if the number of pixels included in the area surrounded by the collection of neighboring contour peripheral pixels is equal to or greater than the reference number of pixels. In this case, a region surrounded by a set of contour peripheral pixels is detected as a defect.

<Functional Configuration of Inspection Device 100>
FIG. 6 is a schematic diagram showing an example of a functional configuration of inspection apparatus 100 according to the first embodiment. Referring to FIG. 6, inspection apparatus 100 includes an image acquisition unit 202, a standardization processing unit 204, a direction calculation unit 206, a differential operation unit 208, an extraction unit 210, and a determination unit 212. Each of these functions is implemented by, for example, the processor 110 executing a program stored in the memory 112. Note that some or all of these functions may be configured to be realized by hardware.

  The image acquisition unit 202 acquires a captured image of the inspection object 10 captured by the camera 8 via the camera interface 116, and stores the captured image in the memory 112. The image acquisition unit 202 is a functional configuration for executing the above-described acquisition process.

  The standardization processing unit 204 performs standardization processing on the captured image acquired by the image acquisition unit 202. Typically, the standardization processing unit 204 performs smoothing processing on the captured image to generate a smoothed image corresponding to the captured image after standardization. The standardization processing unit 204 may execute a standardization process using an average value using a plurality of captured images or a standardization process using a median value. The standardization processing unit 204 is a functional configuration for executing the above-described standardization process.

  The direction calculation unit 206 calculates, for each pixel of the standardized captured image, the luminance gradient direction of the pixel by comparing the luminance of each adjacent pixel of the pixel with the luminance of the pixel. Specifically, for each pixel of the captured image after standardization, the direction calculation unit 206 selects one of the adjacent pixels (for example, adjacent pixels P1 to P4 and P6 to P9) of the pixel (for example, the pixel P5). The difference between the luminance of one adjacent pixel (for example, the adjacent pixel P7) and the luminance of the corresponding pixel is the difference between the luminance of the corresponding pixel and the luminance of the remaining adjacent pixels (for example, adjacent pixels P1 to P4, P6, P8, P9) among the adjacent pixels. If it is larger than the difference with the luminance of the pixel, the direction (for example, the direction 300) from the pixel to the first adjacent pixel is calculated as the luminance gradient direction of the pixel. The direction calculation unit 206 is a functional configuration for executing the above-described direction calculation process.

  The differential operation unit 208 calculates, for each pixel of the imaged image after standardization, second-order differentiation of the pixel by using the luminance gradient direction of the pixel as a differentiation direction to calculate a second-order differential value of the pixel. Typically, the differential operation unit 208 executes the differential operation using the differential filter shown in FIG. 4 according to the luminance gradient direction (that is, the differential direction). The differential operation unit 208 is a functional configuration for executing the above-described differential operation process.

  The extraction unit 210 extracts a pixel having a second-order differential absolute value calculated by the differential operation unit 208 greater than or equal to the threshold value Th among pixels of the captured image as a contour peripheral pixel of the defect of the inspection object 10. The extraction unit 210 is a functional configuration for executing the above-described extraction process.

  The determination unit 212 determines whether a defect is detected based on the contour peripheral pixels extracted in the extraction step. For example, when a contour peripheral pixel is extracted, the determination unit 212 may determine that a defect is present on the surface of the inspection object 10 (that is, a defect is detected). Typically, the determination result of the determination unit 212 is displayed on the display 114. The determination unit 212 is a functional configuration for executing the above-described determination process.

<Example>
Here, an embodiment in which the inspection object 10 is inspected based on a monochrome image of the inspection object 10 will be described. In the case of a monochrome image, the number of image sensors can be reduced to one-fourth that in the case of a color image, so that a high-definition captured image can be obtained.

  FIG. 7 is a schematic view showing a power semiconductor chip. The power semiconductors are IGBTs (Insulated Gate Bipolar Transistors), MOSFETs (metal-oxide-semiconductor field-effect transistors), and the like. Referring to FIG. 7, a power semiconductor chip 10A, which is an example of the inspection object 10, includes a main electrode 11, a termination structure 12, and a gate electrode 13.

  When manufacturing the power semiconductor chip 10A, visual inspection is often performed during the manufacturing process or before shipment to guarantee the quality. By removing defective chips in which defects such as scratches, foreign particles, stains, and deformations have been detected by visual inspection, it is possible to suppress the post-process outflow of defective products and the market outflow. The shapes and colors of the defects are various, and there are also undetectable defects having the same appearance as the background.

  For example, metals such as an aluminum sputtered film and a gold plating film are used for the main electrode 11 and the gate electrode 13. Due to the unevenness of the uneven shape due to the fine grains on the surface of the film or the difference in the in-plane deposition rate at the time of plating, even in the case of a non-defective product, there may be variations in color tone. Furthermore, this variation often has a brightness gradient in the in-plane direction.

  FIG. 8 is a schematic diagram for explaining the brightness gradient of the power semiconductor chip. Referring to FIG. 8, it can be seen that the end portions of the main electrode 11 and the gate electrode 13 are deeply discolored, and the hue is different between the center and the periphery of the wafer. For example, when an aluminum sputtered film is formed on a wafer by magnetron sputtering, since the magnets are arranged concentrically, the central portion tends to be thin due to the difference in magnetic field, and the film quality tends to be rough. Center and perimeter may differ. Alternatively, when a gold plating film is formed as the main electrode 11, the degree of agitation of the plating solution differs between the vicinity of the step and the center of the electrode depending on the step shape of the termination structure 12 etc. The brightness may be different. In such a case, the variation in the color tone of the main electrode 11 and the gate electrode 13 does not have to be detected.

  In defect inspection based on image recognition, the variation in color tone of the background is reflected in the captured image as the variation in luminance value of the captured image. In addition, when the main electrode 11 and the gate electrode 13 are inspected, in order to detect a minute difference in luminance, the inspection by the same algorithm is not performed on the inspection area for the other areas (for example, the termination structure 12 etc.) It should not be performed, and it is desirable to inspect with the main electrode 11 or the gate electrode 13 alone. Examples of the inspection object 10 are a wafer state, a chip state, or a power chip in a state of being bonded to a substrate circuit, such as an IGBT or a free wheeling diode (FWD).

  FIG. 9 is a view showing a captured image of the main electrode 11. FIG. 10 is a view showing an image in which the contrast of the captured image of FIG. 9 is emphasized.

  Here, if the luminance values of the defect and the background are only comparable, even with the method according to Patent Document 1, there is a possibility that the contour of the defect can be detected by setting the threshold used for detection strictly. is there. However, for example, as illustrated in FIG. 9, in the case of a captured image in which the luminance of the main electrode 11 as the background has a gradient, detection of a defect becomes difficult. Referring to FIG. 10 in which the contrast of the captured image shown in FIG. 9 is emphasized, it can be seen that the defect 20 is present on the line segment K1K2.

  FIG. 11 is a diagram showing the luminance of the pixel on the line segment K1K2 of FIG. Referring to FIG. 11, it can be seen that the luminance of the portion where the defect 20 is present is constant.

  FIG. 12 is a diagram showing the result of converting the value after differential processing in which the Sobel filter is applied in the X direction to the captured image shown in FIG. 3 into luminance. The Sobel filter is a differential filter in which a differential direction is specified, and is used to extract a contour.

  FIG. 13 is a diagram showing differential values of pixels on line segment K1K2 of FIG. Referring to FIG. 13, even after Sobel filter processing, the derivative values are generally dispersed, and the S / N ratio is about 1.3, so it is difficult to detect contour pixels of defect 20 by threshold setting. It is. For example, when the threshold is set large, missed defects 20 frequently occur, and when the threshold is set small, false detection frequently occurs.

  FIG. 14 is a diagram showing the result of converting the value after differential processing in which the 8-directional Laplacian filter is applied to the captured image shown in FIG. 9 into luminance. The Laplacian filter is a differential filter for performing second-order differential processing in all directions that does not specify a differential direction, and can extract a point where the value is 0 (that is, a zero cross point) as an outline.

  FIG. 15 is a diagram showing differential values of pixels on the line segment K1K2 of FIG. Referring to FIG. 15, even after the Laplacian filter processing, the differential values are generally dispersed, and the zero cross point is present in the entire image, so detection of the outline pixel of the defect 20 is difficult.

  On the other hand, according to inspection apparatus 100 in accordance with the first embodiment, as shown in FIG. 9, there is a luminance gradient throughout the captured image, and defect 20 is accurately detected even when the difference between the luminance of defect 20 and the luminance of the background is small. be able to.

  FIG. 16 is a diagram showing an image in the case where smoothing processing is performed on the captured image shown in FIG. The image shown in FIG. 16 is a smoothed image when smoothing processing is performed by applying an operating average filter to the captured image shown in FIG. 9 in the above-described standardization process. The inspection apparatus 100 calculates, for each pixel of the smoothed image shown in FIG. 16, the brightness gradient direction of the pixel by comparing the brightness of the pixel with the brightness of each adjacent pixel, and Set in the differential direction.

  Subsequently, the inspection apparatus 100 calculates, for each pixel of the smoothed image shown in FIG. 16, a first-order derivative value of the pixel with the luminance gradient direction of the pixel as the differentiation direction.

  FIG. 17 is a diagram showing the result of converting the first-order differential value of each pixel of the smoothed image shown in FIG. 10 into luminance. FIG. 18 is a diagram showing a first-order differential value of the pixel on the line segment K1K2 of FIG. FIG. 19 is a diagram showing the result of converting the second-order differential value of each pixel of the smoothed image shown in FIG. 17 into luminance. Referring to FIG. 19, in the regions 901 and 902, when viewed in the vertical direction, a portion with high and low density alternately exists in a cycle of about 15 pixels, and a portion with high density is wavy in a line. I can see it. FIG. 20 is a diagram showing second derivative values of pixels on line segment K1K2 in FIG.

  As shown in FIG. 18, even if the differential direction is set, it is difficult to detect the outline of the defect 20 because the S / N ratio is low and it is difficult to detect the outline of the defect 20 only by executing the first-order differentiation process. The S / N ratio improves to 1.9 by performing the second-order differentiation process. Therefore, by setting the threshold value, a pixel having a large second derivative value can be extracted as an edge peripheral pixel of the defect 20. That is, the defect 20 can be detected with high accuracy.

  As described above, although the gradient of the luminance of the background is noise only by the first-order differentiation process, it is possible to reduce the noise by the second-order differentiation process. In addition, by setting the differentiation direction to the gradient direction of the luminance, it is possible to differentiate in the direction in which the difference between the luminance of the defect 20 and the luminance of the background becomes maximum.

<Modification of differential operation process>
(Differential filter)
A modification of the differential filter used in the above-described differential operation process will be described with reference to FIG. FIG. 21 shows a plurality of differential filters according to the modification of the first embodiment.

  FIGS. 21 (a) and 21 (b) are diagrams showing differential filters 411 and 412, respectively.

When the luminance gradient direction of the pixel at the coordinate (x, y) is the direction (1, 1), the first-order differential value D _{x of the} luminance C _{x, y} is obtained using the differential filter 411 of FIG. _{, Y} is represented by the following equation (4).

D _{x, y} = C _{x + 1, y + 1} + C _{x + 1, y} + C _{x, y + 1-} (C _{x -1, y -1} + C _{x -1, y} + C _{x, y -1} ) (4)
When the luminance gradient direction of the pixel at the coordinate (x, y) is the direction (1, 0), the first-order differential value of the luminance C _{x, y} is obtained using the differential filter 412 of FIG. D _{x, y} is expressed by the following equation (5).

D _{x, y} = C _{x +1, y} + C _{x +1, y-1} + C _{x +1, y + 1-} (C _{x -1, y} + C _{x -1, y + 1} + C _{x -1, y-1} ) (5)
When the luminance gradient direction of the pixel at the coordinates (x, y) is the direction (−1, 1), the differential operation is performed using a differential filter obtained by rotating the differential filter 411 by 90 degrees. Similarly, when the luminance gradient direction is the direction (-1, -1) or the direction (1, -1), the differential operation is performed using the differential filter obtained by rotating the differential filter 411 by 180 degrees and 270 degrees, respectively. Is executed.

  Further, when the luminance gradient direction is the direction (0, 1), the differential operation is performed using a differential filter obtained by rotating the differential filter 412 by 90 degrees. Similarly, when the luminance gradient direction is the direction (-1, 0) or the direction (0, -1), the differential operation is performed using the differential filter obtained by rotating the differential filter 412 by 180 degrees and 270 degrees, respectively. To be executed.

  Here, as shown in FIG. 11, since the luminance of the pixel on line segment K1K2 of the captured image shown in FIG. 9 increases from point K1 to point K2, the increasing direction of the luminance is from point K1 to point It corresponds to the direction towards K2, ie the direction (1, 0). The decreasing direction of the luminance corresponds to the direction from the point K2 to the point K1, that is, the direction (−1, 0).

  Referring to FIG. 4, in differential filter 402 in the case where the brightness gradient direction is direction (1, 0) in the first embodiment, coefficients corresponding to direction (1, 0) and direction (−1, 0) Is two. Therefore, when the differential filter 402 is applied to the captured image shown in FIG. 9, the influence of the two adjacent pixels in the brightness gradient direction is strong. On the other hand, as shown in FIG. 21B, in the differential filter 412 where the luminance gradient direction is the direction (1, 0), the coefficients corresponding to the direction (1, 0) and the direction (−1, 0) are Since it is 1, the said influence is suppressed.

  As shown in FIG. 19, when there are alternately high and low density portions in a cycle of about 15 pixels (that is, in the case where density undulations exist), the differential value and differential direction between adjacent pixels are Although the change is relatively large, for example, when light and shade exist in a cycle of about 30 pixels, the differential value and the differential direction of adjacent pixels do not change significantly. Therefore, by using the differential filter 412 whose coefficient is reduced to 1, the differential value (for example, background noise) caused by something other than the defect outline is reduced. For example, when the luminances of the six adjacent pixels P1 to P3 and P7 to P9 on the left and right of the central pixel P5 shown in FIG. 3 slightly fluctuate, the differential values of the two adjacent pixels P4 and P6 above and below the pixel P5 are fluctuate. In this case, unlike the differential filter 402, since the differential filter 412 does not weight the coefficients, the noise of the differential value becomes smaller by averaging, and the S / N ratio increases. That is, in the case where undulations of light and shade exist in a cycle of about 30 pixels, it is preferable to use the differential filters 411 and 412 shown in FIGS. 21 (a) and (b).

  Note that, as shown in FIG. 19, when there are alternately high and low density portions in a cycle of about 15 pixels, the differential value and the differential direction of adjacent pixels change relatively large. Therefore, in order to extract local values more accurately, it is preferable to use the differential filter 402 in which the coefficient corresponding to the luminance gradient direction is set to 2. In this case, even if the brightness of the six adjacent pixels P1 to P3 and P7 to P9 on the left and right of the pixel P5 fluctuates minutely, the ratio of the fluctuation caused by the undulation of the gray level rather than the relatively uniform background is large. By setting the factor to 2, the S / N ratio is increased. That is, in the case where an undulation of light and shade exists in a cycle of about 15 pixels, it is preferable to use the differential filter shown in FIG.

  FIGS. 21 (c) and 21 (d) are diagrams showing the differential filters 413 and 414, respectively.

When the luminance gradient direction of the pixel at the coordinate (x, y) is the direction (1, 1), the first-order differential value D _{x of the} luminance C _{x, y} is obtained using the differential filter 413 of FIG. _{, Y} is represented by the following equation (6).

D _{x, y} = C _{x + 1, y + 1-} C _{x-1, y-1} (6)
When the luminance gradient direction of the pixel at the coordinates (x, y) is the direction (1, 0), the first-order differential value of the luminance C _{x, y} is obtained using the differential filter 414 of FIG. D _{x, y} is expressed by the following equation (7).

D _{x, y} = C _{x +1 , y} -C _{x -1, y} (7)
When the luminance gradient direction of the pixel at the coordinates (x, y) is the direction (-1, 1), the direction (-1, -1), or the direction (1, -1), the differential filter 413 is used. The differential operation is performed using a differential filter obtained by rotating 90 degrees, 180 degrees, and 270 degrees.

  When the luminance gradient direction is direction (0, 1), direction (-1, 0), direction (0, -1), rotate differential filter 414 by 90 degrees, 180 degrees, 270 degrees, respectively. The differential operation is performed using the differential filter.

  Here, the differential filters 413 and 414 are useful, for example, when there is a gradation of light and dark with a period of 3 pixels or less. For example, in the differential filter 414 where the luminance gradient direction is the direction (1, 0), four oblique directions (1, 1), (1, -1), (-1, -1), (-1) , 1) is 0. When the period of the waviness of the density becomes short, the values of the pixels corresponding to the four diagonal directions become noise because they include the fluctuation due to the waviness of the density. Therefore, the S / N ratio is improved by removing the information of the pixel caused by the noise and setting it as the information of only two pixels corresponding to the direction (1, 0) and the direction (-1, 0).

  21 (e) and 21 (f) are diagrams showing differential filters 415 and 416, respectively.

When the luminance gradient direction of the pixel at the coordinate (x, y) is the direction (1, 1), the first-order differential value D _{x of the} luminance C _{x, y} is obtained using the differential filter 415 of FIG. _{, Y} is expressed by the following equation (8).

D _{x, y} = C _{x + 1, y + 1} −C _{x, y} (8)
When the luminance gradient direction of the pixel at the coordinate (x, y) is the direction (1, 0), the first-order differential value of the luminance C _{x, y} can be obtained using the differential filter 416 of FIG. D _{x, y} is expressed by the following equation (9).

D _{x, y} = C _{x +1 , y} -C _{x, y} (9)
When the luminance gradient direction of the pixel at the coordinates (x, y) is the direction (-1, 1), the direction (-1, -1), or the direction (1, -1), the differential filter 415 is used. The differential operation is performed using a differential filter obtained by rotating 90 degrees, 180 degrees, and 270 degrees.

  When the luminance gradient direction is direction (0, 1), direction (-1, 0), direction (0, -1), rotate differential filter 416 by 90 degrees, 180 degrees, 270 degrees, respectively. The differential operation is performed using the differential filter.

  The differential filters 415 and 416 are effective, for example, when there is a gradation of light and dark with a period of 2 pixels or less. When the period of the waviness of the light and dark is 2 pixels or less, it is not possible to confirm the waviness of light and darkness at random. That is, it can not follow due to the relationship of the pixel size, and it is in a state of being viewed randomly on the data. In such a case, if information of two pixels corresponding to the direction (1, 0) and the direction (−1, 0) as in the differential filter 413 is used, the range used for differential processing becomes three pixels. , Including noise. Therefore, as in the differential filter 416, S / N is improved by differential processing with two pixels.

(Derivative rank)
Although the configuration for calculating the second-order derivative value of each pixel of the captured image in the differentiation operation step has been described above, the present invention is not limited thereto, and the third-order derivative value may be calculated.

  FIG. 22 is a diagram showing the result of converting the third order differential value of each pixel of the captured image shown in FIG. 9 into luminance. FIG. 23 is a diagram showing the third order differential value of the pixel on the line segment K1K2 of FIG.

  As shown in FIG. 23, even when the third-order differentiation process is performed, a high S / N ratio is obtained as in the case where the second-order differentiation process is performed. Thus, by setting the threshold value, it is possible to extract a pixel having a large third-order derivative value as an edge peripheral pixel of the defect 20. That is, the extraction unit 210 extracts a pixel having a third-order derivative absolute value equal to or more than the threshold among the pixels of the captured image as the outline peripheral pixel of the defect of the inspection object 10. When the luminance gradient of the background luminance is minute, the noise caused by the background is reduced by increasing the differential rank, and the differential value caused by the outline of the defect 20 can be more easily detected.

  In the differentiation operation step, each pixel of the captured image is differentiated n or more (n is a natural number of 2 or more) so that the S / N ratio is maximized according to the inspection object 10, and the nth order differential value of each pixel Any configuration may be used as long as it calculates. However, since the peripheral pixels of the outline of the defect to be detected increase in the direction of the luminance gradient from the position of the contour pixel of the actual defect every time the differential rank increases, the accuracy of the positional information of the defect decreases. Therefore, the user may determine the value of n for each inspection object 10 in consideration of the accuracy of the position information and the detection accuracy of the defect.

<Advantage>
According to the first embodiment, even in the case where there is a luminance gradient in the background and the difference between the density of the background and the defect is small, pixels around the outline of the defect can be extracted with high accuracy. Therefore, the defect can be detected more accurately. For example, by applying the inspection method according to the present embodiment in the appearance inspection in the manufacturing process, it is possible to suppress the outflow of defects to the post process and reduce the loss cost. In addition, by applying the inspection method according to the present embodiment in the appearance inspection of the inspection before shipment, it is possible to reduce market defects and improve the reliability.

Second Embodiment
In the second embodiment, more detailed processing of the extraction process according to the above-described first embodiment will be described. Specifically, a method of extracting an aggregate in which a plurality of contour peripheral pixels are integrated will be described.

  FIG. 24 is a diagram for describing an example of an extraction method of a group of outline peripheral pixels according to the second embodiment. Referring to FIG. 24, the extraction unit 210 extracts a pixel having a second derivative absolute value equal to or more than the threshold value Th among pixels of the captured image as a contour peripheral pixel of a defect of the inspection object 10, and the contour A binarization process is performed in which peripheral pixels are represented as black pixels and non-contour peripheral pixels are represented as white pixels. The extraction unit 210 extracts a set of contour peripheral pixels adjacent to each other as a contour peripheral aggregate (hereinafter, also simply referred to as an “aggregate”). In the example of FIG. 24, aggregates 81, 82, 83 are extracted.

  Next, the extraction unit 210 acquires the lengths L1, L2, and L3 in the longitudinal direction of the aggregates 81, 82, and 83. For example, focusing on the aggregate 81, the extraction unit 210 determines whether the distance DS1 between the aggregate 81 and another aggregate 82 is less than the threshold value TD1. The threshold value TD1 is set, for example, within a length L1 × r1 times in the longitudinal direction, within a predetermined number of pixels r2, or the like. The values of r1 and r2 are arbitrarily set by the user. The appearance of pixels around the outline of the defect varies depending on the material of the inspection object, surface condition, illumination conditions at inspection, differential parameters, threshold values, etc. Therefore, the user changes r1 and r2 while checking the detection result give feedback.

  When the distance DS1 is less than the threshold value TD1, the extraction unit 210 integrates the assembly 81 and the assembly 82 and extracts it as one assembly AG. Similarly, when the distance DS2 between the aggregate AG and the aggregate 83 is less than the threshold value TD1, the extraction unit 210 integrates the aggregate AG and the aggregate 83 and extracts it as one aggregate 84. .

  In summary, the extraction unit 210 obtains the length in the longitudinal direction of an assembly of contour peripheral pixels adjacent to each other, and in the longitudinal direction of the assembly, the periphery of another contour is within a predetermined distance from the assembly. It is determined whether a set of pixels exists. When it is determined that there is another aggregate, the extraction unit 210 extracts the aggregate and the other aggregate as an aggregate of one contour peripheral pixel. In the following description, the extraction method is also referred to as “extraction method EM1”. According to the extraction method EM1, even if the aggregate is disconnected from another aggregate in the longitudinal direction of the aggregate of peripheral pixels, it is assumed that the aggregate is a contour neighboring pixel derived from the same defect It can be judged. Therefore, the count number of defects or the shape of defects can be recognized more accurately.

  FIG. 25 is a diagram for explaining another example of a method of extracting a set of contour peripheral pixels according to the second embodiment. Referring to FIG. 25, the extraction unit 210 calculates the centroid pixel H1 and the length L5 in the longitudinal direction of the aggregate 91, and the centroid pixel H2 and the length L6 in the longitudinal direction of the aggregate 92.

  Subsequently, the extraction unit 210 determines whether or not the distance from the gravity center pixel H1 to the aggregate 92 in the lateral direction is less than the threshold value TD2 focusing on the aggregate 91. The threshold value TD2 is set, for example, within a length L5 × r3 times in the longitudinal direction and within a predetermined number of pixels r4. The values of r3 and r4 are arbitrarily set by the user. When the distance is less than the threshold value TD2, the extraction unit 210 labels (that is, extracts) the aggregate 91 and the aggregate 92 as an aggregate 93 of contour peripheral pixels derived from the same defect.

  In summary, the extraction unit 210 acquires the center of gravity and the short side direction of the aggregate, and in the short side direction of the group, an aggregate of other contour peripheral pixels within a predetermined distance from the center of gravity of the group It is determined whether or not exists. When it is determined that there is another aggregate, the extraction unit 210 extracts the aggregate and the other aggregate as an aggregate of one contour peripheral pixel. In the following description, the extraction method is referred to as “extraction method EM2”.

  By the extraction method EM2, even if the aggregate is disconnected from another aggregate in the short direction of the aggregate of peripheral pixels, the aggregate is a contour neighboring pixel derived from the same defect It can be judged. Therefore, the count number of defects or the shape of defects can be recognized more accurately.

  The extraction unit 210 may also determine whether the distance from the end pixel present at one end in the longitudinal direction of the aggregate 91 to the aggregate 92 is less than the threshold value TD3 in the short side direction. The threshold value TD3 is set, for example, within a length L5 × r5 times in the longitudinal direction and within a predetermined number of pixels r6. The values of r5 and r6 are arbitrarily set by the user. When the distance is less than the threshold value TD3, the extraction unit 210 labels the set 91 and the set 92 as a set 93 of outline peripheral pixels derived from the same defect.

  In summary, the extraction unit 210 determines whether or not there is a set of other contour peripheral pixels within a predetermined distance from the pixel at one end in the longitudinal direction of the set in the short direction of the set. Do. When it is determined that the other aggregate exists, the extraction unit 210 extracts the aggregate and the other aggregate as an aggregate of one contour peripheral pixel. In the following description, the extraction method is referred to as “extraction method EM3”.

  By the extraction method EM3, even in the case where the aggregate is disconnected from another aggregate in the short direction of the aggregate of peripheral pixels, the aggregate is a contour neighboring pixel derived from the same defect It can be judged. In particular, when the shape of the defect is a triangle or the like, although it is difficult to extract an aggregate of pixels near the outline of the defect in the extraction method EM2, the aggregate can be extracted according to the extraction method EM3. Therefore, the count number of defects or the shape of defects can be recognized more accurately.

  By combining the above extraction methods EM1 to EM3, for example, circular defects can be detected with high accuracy. FIG. 26 is a view showing a captured image of a circular defect and a differential value of the captured image. Specifically, FIG. 26 (a) shows a captured image of a circular defect. FIG. 26B is a diagram showing a first-order differential value obtained by performing first-order differentiation of the captured image of FIG. 26A in the X direction. FIG. 26C shows second derivative values obtained by differentiating the captured image of FIG. 26A in the X direction.

  Referring to FIG. 26B, when each pixel of a captured image of a circular defect is subjected to first-order differentiation, outline pixels (white portions in the drawing) of a torus are extracted. Referring to FIG. 26C, when each pixel of a captured image of a circular defect is second-order differentiated, a contour peripheral pixel (white portion in the drawing) is extracted in the shape of a double line. This is also understood from the fact that there are two peak values of second derivative values on both sides of the defect contour as shown in FIG. 5 (d).

  FIG. 27 is a view schematically showing the contour peripheral pixels shown in FIG. Referring to FIG. 27, circle 801 schematically indicates the position of the outline pixel. Around the circle 801, outline peripheral pixels indicated by black pixels are scattered, and have a double line shape as a whole.

  FIGS. 28 and 29 are diagrams for explaining a method of extracting a set of contour peripheral pixels. Specifically, FIG. 28 (a) is a diagram showing an example of a set extracted by the extraction method EM1. FIG. 28 (b) is a diagram showing an example of an assembly extracted by the extraction method EM 2. FIG. 29 is a diagram showing an example of a set extracted by the extraction method EM3.

  Referring to FIG. 28A, by applying the extraction method EM1, aggregates 501, 502, 503, and 504 are extracted. Assemblies 501 and 503 are U-shaped, and indicate assemblies corresponding to lines outside the double line. Assemblies 502 and 504 indicate assemblies that are rectangular and correspond to lines inside the double line. As described above, when the extraction method EM1 is applied, double lines corresponding to both sides of the defect are extracted as separate aggregates, and there are four aggregates as a whole.

  Referring to FIG. 28 (b), by further applying extraction method EM2, aggregate 501 and aggregate 502 are extracted as one aggregate 510, and aggregate 503 and aggregate 504 as one aggregate 520. It has been extracted. Thus, when the extraction method EM2 is applied, double lines corresponding to one side of the defect are extracted as one assembly, and there are two assemblies as a whole.

  Referring to FIG. 29, by further applying extraction method EM3, an aggregate 510 and an aggregate 520 shown in FIG. 28 (b) are extracted as one aggregate 550. As described above, after the application of the extraction method EM3, aggregates existing on both sides of the defect are extracted as one aggregate.

  As described above, by applying the extraction methods EM1 to EM3, the number of defects can be recognized as the same defect instead of recognizing the outline peripheral pixels scattered in the double linear shape as a plurality of defects. The accuracy of the count and the recognition of the shape of the defect is increased

  Here, when the number of pixels included in the area in the aggregate extracted as described above is equal to or more than the reference pixel number, the determination unit 212 determines that a defect exists, and detects the area as a defect. Further, the determination unit 212 may count defects present in the inspection object 10 and perform the quality determination according to the count number. For example, the determination unit 212 determines the inspection object 10 as a defective product when the count number is equal to or more than the specified number, and determines the inspection object 10 as a non-defective product when the count number is less than the specified number. It is also good.

<Advantage>
According to the second embodiment, in addition to the advantages of the first embodiment, the number of defects present in the inspection object 10 and the shape of the defects can be grasped more accurately.

Other Embodiments
(1) In the embodiment described above, the configuration for detecting a defect using luminance as a pixel value of each pixel of a monochrome image has been described, but using a lightness as a pixel value of each pixel of a monochrome image It may be configured to detect. The luminance is calculated by multiplying the RGB (Red, Green, Blue) values of each pixel by an appropriate coefficient and adding them, so the blue property tends to be thin and the green property tends to appear. Inspection is performed using the lightness of each pixel to detect defects that are difficult to detect by luminance by equalizing the influence of the R value indicating the red component, the G value indicating the green component, and the B value indicating the blue component. be able to. For example, in the case of using a monochrome image with a large difference between the background and the defect in the R value and the B value, it is preferable to use a configuration in which the defect is detected using lightness. On the other hand, when the difference between the background and the defect is large in the G value, it is preferable to use a configuration to detect the defect using the luminance.

  Further, in the embodiment described above, the configuration for detecting a defect using each pixel of a monochrome image has been described, but the configuration may be such that a defect is detected using each pixel of a color image. If there is no difference in luminance or lightness between the background and the defect and there is a difference in RGB values, it is desirable to use a configuration in which each pixel of the color image is used to detect the defect. For example, in the case of lightness, the RGB values of the background are RGB values (100, 50, 30) and the RGB values of the defect are not different when they are RGB values (30, 100, 50). , Color images are used.

  From this, the pixel value may be a gradation value when the captured image is a gray scale method, and may be a value calculated from three components of R, G, and B when it is an RGB method. Examples of values calculated from three components of R, G, and B include an average value or maximum value of three components, a value converted into a lightness component, and a value converted into a luminance component.

  (2) In the above-described embodiment, the gradient direction of the pixel value of each pixel of the inspection object 10 is calculated by using a non-defective imaging image known to have no defect imaged before the inspection. It may be. Specifically, as pre-processing before inspection, the processor 110 of the inspection apparatus 100 acquires a non-defective imaged image of the inspection object 10 and standardizes the non-defective imaged image. The processor 110 calculates the gradient direction of the pixel value of the pixel by comparing the pixel value of each adjacent pixel of the pixel with the pixel value of the pixel for each pixel of the non-defective captured image. The gradient direction of the pixel value of each pixel is stored in the memory 112.

  Subsequently, as processing at the time of inspection, the processor 110 acquires a captured image of the inspection object 10, and for each pixel of the captured image, the gradient of the pixel value of the pixel of the non-defective captured image corresponding to the pixel The direction is differentiated, and the pixel is differentiated n or more times to calculate the n-th order differential absolute value of the pixel. That is, the processor 110 differentiates the gradient direction of the pixel value of the pixel at the coordinate G of the non-defective captured image as n pixels or more of the pixel as the differential direction of the pixel at the coordinate G of the captured image of the inspection object 10 Calculate the nth-order derivative absolute value of the pixel. Then, the processor 110 extracts a pixel having a calculated nth-order differential absolute value equal to or more than a threshold among the pixels of the captured image as a contour peripheral pixel of the defect of the inspection object.

  As a result, since the differentiation direction of each pixel is known at the time of inspection, it is not necessary to perform the standardization process and the direction calculation process for the inspection object 10, so the inspection time can be shortened.

  (3) In the above (2), although the configuration for calculating the brightness gradient direction of each pixel of the inspection object 10 using the non-defective pickup image has been described, a plurality of inspection objects for which it is not known The brightness gradient direction of each pixel of the inspection object 10 may be calculated using the captured image of the above.

  For example, the processor 110 calculates, for each pixel, an average value of luminance in captured images of a plurality of inspection objects, and generates an averaged image with the average value as the luminance of each pixel as a reference image. Alternatively, the processor 110 calculates, for each pixel, the median value of luminance in the captured images of a plurality of inspection objects, and generates a median value image in which the median value is the luminance of each pixel as a reference image. Then, the processor 110 calculates the brightness gradient direction for each pixel of the reference image (that is, the averaged image or the median image), and stores the calculated brightness gradient direction of each pixel in the memory 112. The subsequent processing is the same as the above (2). As a result, since the differentiation direction of each pixel is known at the time of inspection, it is not necessary to perform the standardization process and the direction calculation process for the inspection object 10, so the inspection time can be shortened.

  In particular, when it is known that the color tone of the outer peripheral portion tends to be dark due to the manufacturing process as in the power semiconductor chip 10A described above, the color density of the outer peripheral portion is different for each chip (For example, in the case where chips of light color are mixed) or the like, it is possible to calculate the brightness gradient direction with high accuracy by averaging the variations in the gradient direction of individual products.

  (4) In the embodiment described above, in the reference image, when the brightness gradient direction is not uniform with respect to the entire image or the inspection area and minutely changes for each adjacent pixel, the smoothing process is not performed. The luminance gradient direction may be acquired. This is because, in such a case, even if the luminance gradient direction tends to be random on a pixel basis in actuality, since the luminance gradient directions of adjacent pixels will be similar if smoothed, non-existent light and shade This is because swells occur and become noise. Therefore, the S / N ratio is increased by reflecting the actual brightness gradient direction without executing the smoothing process.

  When the defect size is smaller than the pixel, the brightness of the pixel includes both the component derived from the background and the component derived from the defect. In this case, since the pixel has the luminance averaged for the area larger than the defect size, the state is the same as that after the smoothing. Therefore, even when the size of the defect to be inspected is small or the resolution of the camera is low, for example, when the pixel size is larger than the defect, the actual brightness gradient direction is reflected without smoothing processing. Increase the / N ratio.

  (5) In the above-described embodiment, the smoothing processing of the captured image may be performed for each pixel block of m × m (m is a natural number) matrix. Specifically, the processor 110 divides the captured image into a plurality of pixel blocks in a matrix as smoothing processing, and for each pixel block, an average value of pixel values of a plurality of pixels constituting the pixel block (or A median value is calculated, and the average value (or median value) is set to the pixel value of the pixel block (that is, the pixel value of a plurality of pixels). The value of m above is arbitrarily set by the user.

  The above smoothing process is useful when the size of the defect of the inspection object is known and the size of the defect is equal to or less than m pixels. This is because, when the defect size is known, the influence of small-sized light and shade waves (ie, noise) can be reduced by treating pixel blocks larger than the defect size like one pseudo pixel and smoothing them. It is for.

  (6) The configuration exemplified as the above-described embodiment is an example of the configuration of the present invention, and can be combined with another known technique, and a part of the configuration can be made without departing from the scope of the present invention. It is also possible to change and configure, such as omitting.

  In the embodiment described above, the processes and configurations described in the other embodiments may be adopted appropriately and implemented.

  It should be understood that the embodiments disclosed herein are illustrative and non-restrictive in every respect. The scope of the present invention is indicated not by the above description but by the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims.

  DESCRIPTION OF SYMBOLS 8 camera, 9 conveyance mechanism, 10 inspection object, 10A power semiconductor chip, 11 main electrode, 12 termination structure part, 13 gate electrode, 20, 51 defects, 81 to 84, 91 to 93, 501 to 504, 510, 520 , 550 assembly, 100 inspection apparatus, 110 processor, 112 memory, 114 display, 116 camera interface, 118 input interface, 120 communication interface, 122 memory interface, 202 image acquisition unit, 204 standardization processing unit, 206 direction calculation unit, 208 Differential operation unit, 210 extraction units, 212 determination units, 401, 402, 411 to 416 differential filters, 1000 inspection systems.

Claims (11)

An inspection method for detecting a defect on the surface of an inspection object, comprising:
Acquiring a captured image of the inspection object;
Calculating, for each pixel of the captured image, the gradient direction of the pixel value of the pixel by comparing the pixel value of each adjacent pixel of the pixel with the pixel value of the pixel;
Calculating, for each pixel of the captured image, an nth-order derivative value of the pixel by differentiating the pixel by n (n is a natural number of 2 or more) floor or more with the gradient direction of the pixel value of the pixel as a differentiation direction;
Extracting a pixel having an absolute value of the calculated nth-order derivative value equal to or greater than a threshold among each pixel of the captured image as an outline peripheral pixel of a defect of the inspection object. Further including the step of standardizing the captured image;
The inspection method according to claim 1, wherein the step of calculating the gradient direction includes calculating, for each pixel of the standardized captured image, a gradient direction of a pixel value of the pixel.   The step of standardizing includes calculating an average value of pixel values of each pixel in a plurality of the captured images, and generating a first standard image having the average value as a pixel value as the standardized captured image. The inspection method according to claim 1 or claim 2.   The step of standardizing includes calculating a median of pixel values of each pixel in a plurality of the captured images, and generating a second standard image having the median as a pixel value as the standardized captured image. The inspection method according to claim 1 or claim 2.   The inspection method according to claim 1 or 2, wherein the step of standardizing includes performing smoothing processing on the captured image to generate a smoothed image as the standardized captured image. .   The smoothing process divides the captured image into a plurality of pixel blocks in a matrix, calculates an average value of pixel values of a plurality of pixels forming the pixel block for each pixel block, and calculates the average value The inspection method according to claim 5, comprising setting to the pixel value of the pixel of.   In the step of calculating the gradient direction, for each pixel of the captured image, the difference value between the pixel value of the first adjacent pixel among the adjacent pixels of the pixel and the pixel value of the pixel is Calculating the direction from the pixel to the first adjacent pixel as the gradient direction of the pixel value of the pixel if the difference value between the pixel value of the remaining adjacent pixel and the pixel value of the pixel is larger The inspection method according to any one of claims 1 to 6. Obtaining the longitudinal direction of the first set of neighboring pixels of the contour periphery adjacent to each other;
Determining whether there is a second set of other contour peripheral pixels within a predetermined distance from the first set in the longitudinal direction of the first set;
8. The method according to claim 1, further comprising the step of extracting the first and second aggregates as a collection of one of the contour peripheral pixels when the second collection exists. The inspection method according to any one of the items. Acquiring a center of gravity and a lateral direction of the first assembly;
Determining whether there is a third aggregate of other contour peripheral pixels within a predetermined distance from the center of gravity of the first aggregate in the lateral direction of the first aggregate;
9. The method according to claim 8, further comprising the step of extracting the first and third aggregates as a collection of one of the contour peripheral pixels, when the third collection is present. . It is determined whether or not a fourth aggregate of other contour peripheral pixels is present within a predetermined distance from a pixel at one end in the longitudinal direction of the first aggregate in the lateral direction of the first aggregate. Step to
The method according to claim 9, further comprising the step of extracting the first aggregate and the fourth aggregate as an aggregate of one of the contour peripheral pixels, when the fourth aggregate is present. . An inspection apparatus for detecting a defect on the surface of an inspection object, comprising:
An image acquisition unit that acquires a captured image of the inspection object;
A direction calculation unit that calculates, for each pixel of the captured image, the gradient direction of the pixel value of the pixel by comparing the pixel value of each pixel adjacent to the pixel with the pixel value of the pixel;
A differential operation unit that differentiates the pixel by n (where n is a natural number of 2 or more) ranks or more with respect to each pixel of the captured image with the gradient direction of the pixel value of the pixel as the differentiation direction When,
An inspection unit comprising: an extraction unit which extracts, of each pixel of the captured image, a pixel whose absolute value of the calculated nth-order derivative value is equal to or greater than a threshold as an outline peripheral pixel of a defect of the inspection object.