JP2004069698A - Image-processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To precisely extract a defect in an inspection image without recognizing erroneously as the defect deformation of the inspection image caused by fluctuation of an imaging environment. <P>SOLUTION: The least square method is applied for linear relations of respective outlines of an instruction image registered preliminarily and the inspection image of a concentration image, and a conversion parameter is thereby calculated to express the deformation of the inspection image with respect to the instruction image (S2-S5). A concentration difference of corresponding picture elements in a deformed image of the instruction image and the inspection image is found based on the found conversion parameter, and the picture element of which the concentration difference exceeds a preset threshold value is extracted (S8,S9). The extracted picture element is used for appearance inspection for the inspection object. An influence of the deformation of the inspection image caused by fluctuation of the image picking-up environment is removed to discriminate the inherent defect in the inspection image. <P>COPYRIGHT: (C)2004,JPO

Description

The present invention detects an inspection image, which is an image of an inspection object, from a captured image obtained by imaging the inspection object, compares the inspection image with a preset teaching image, and teaches the inspection image. The present invention relates to an image processing method for performing a visual inspection of an inspection object by extracting a part that does not match an image.

In general, the type of an inspection object is determined by extracting an inspection image, which is an image of the inspection object, from a captured image in an imaging space including the inspection object, and comparing the inspection image with a preset teaching image. There is known an image processing technique for judging the quality of an inspection object (for example, see Patent Document 1).

In this type of image processing, the position of the inspection image in the captured image, the rotation angle of the inspection image with respect to the teaching image, and the enlargement / reduction ratio with respect to the teaching image are obtained. After performing, the difference between the area of the teaching image and the teaching image is obtained by superimposing the teaching image, and the presence / absence of a defect (scratch, chipping, dirt, etc.) on the appearance of the inspection object is extracted according to the obtained difference. ing.
JP 2001-175865 A

However, by simply determining the difference between the number of pixels between the inspection image and the teaching image to determine the quality of the inspection object, the primary deformation caused by the movement of the inspection object with respect to the image input unit and the image deformation by the image input unit The distortion of the inspection image due to the secondary deformation caused by the positional relationship between the optical axis of the lens to be used and the inspection object cannot be removed, and a defect such as a scratch, chip, or dirt in the inspection image and a distortion of the inspection image are erroneously recognized. Erroneous judgment may be made.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an image processing method capable of accurately extracting a defect in an inspection image without erroneously recognizing a deformation of the inspection image due to a change in an imaging environment as a defect. To provide.

According to the first aspect of the present invention, contours of an inspection image as an image of an inspection object and a reference teaching image are obtained, and the inspection image is regarded as an image obtained by adding a deformation represented by a linear expression to the teaching image. By applying the least squares method to pixels on the contour line between the inspection image and the teaching image, a conversion parameter representing the deformation of the inspection image with respect to the teaching image is calculated. The method is characterized in that a density difference between a corresponding pixel and an image is obtained, a pixel in which the density difference exceeds a preset threshold value is extracted, and the extracted pixel is used for an appearance inspection of an inspection object.

According to this method, by using the contour of the inspection image and the teaching image, the inspection image is regarded as an image obtained by adding a deformation represented by a linear expression to the teaching image, and the least square method is applied to the contour. Since the conversion parameters are calculated and the conversion parameters are extracted as information on the primary deformation, the influence of the primary deformation caused by the influence of the imaging environment when the inspection object is imaged by the image input means is removed, and the inspection image is obtained. There is an advantage that it is possible to accurately detect only the defective portion and perform an accurate appearance inspection.

According to a second aspect of the present invention, the contours of the inspection image, which is the image of the inspection object, and the reference teaching image are obtained, and the inspection image is regarded as an image obtained by adding a deformation represented by a quadratic expression to the teaching image. By applying the least squares method to pixels on the contour line between the inspection image and the teaching image, a conversion parameter representing the deformation of the inspection image with respect to the teaching image is calculated. The method is characterized in that a density difference between a corresponding pixel and an image is obtained, a pixel in which the density difference exceeds a preset threshold value is extracted, and the extracted pixel is used for an appearance inspection of an inspection object.

According to this method, by using the contour of the inspection image and the teaching image, the inspection image is regarded as an image obtained by adding a deformation represented by a quadratic expression to the teaching image, and the least square method is applied to the contour. Since the conversion parameter is calculated and the conversion parameter is extracted as information of the secondary deformation, the influence of the secondary deformation caused by the influence of the imaging environment when the inspection object is imaged by the image input means is removed, and the inspection image is obtained. There is an advantage that it is possible to accurately detect only the defective portion and perform an accurate appearance inspection.

According to a third aspect of the present invention, in the first or second aspect of the present invention, the inspection image is included in a captured image obtained by capturing an image of a spatial region including the inspection target object by an image input unit, and the teaching image is prepared as a template in advance. By comparing the teaching image registered in the storage unit with the captured image, an image modified so as to match the position, rotation angle, and enlargement / reduction ratio of the teaching image with the inspection image with low accuracy is used. .

According to this method, since the preprocessing for obtaining the approximate position, rotation angle, and enlargement / reduction ratio of the inspection image in the captured image with respect to the teaching image is performed, the teaching image can be roughly matched with the inspection image. Since the error when applying the multiplication method is reduced, it is possible to accurately compare the teaching image with the inspection image. That is, the reliability of the result of the appearance inspection increases.

According to a fourth aspect of the present invention, in the first or second aspect of the invention, when extracting the contour, a pixel having a maximum value obtained by applying a Sobel filter to the inspection image and the teaching image is tracked. It is characterized in that a line is extracted as a contour line.

(4) According to this method, a contour line can be accurately extracted from an inspection image and a teaching image, and much information can be used on distortion of the inspection image.

According to a fifth aspect of the present invention, in the invention of the fourth aspect, when the contour is traced, the angle difference between the direction in which the differential value of the next pixel to be traced becomes a local maximum value and the normal of the contour already traced. Is within ± 45 degrees, a pixel to be tracked next is set as a pixel on the contour line.

According to this method, only pixels that are likely to be contours are selected at the time of tracking the contours, so that the possibility that unnecessary pixels are extracted as contours is reduced, The reliability of the process of extracting a contour line is improved.

According to a sixth aspect of the present invention, in the first or second aspect of the present invention, when extracting the contour line, a pixel in the first area which is close to each pixel of the inspection image and the teaching image, respectively. Generating a first smoothed image having an average value of the density values of the pixels as pixel values, and generating a second smoothed image having a pixel value of the average value of the density values of the pixels in the second area wider than the first area. Then, after weighting the pixel value of each pixel of the first smoothed image by one or more weighting factors, subtraction is performed from the pixel value of the corresponding pixel of the second smoothed image to obtain a difference image. When the sign of the pixel value of each pair of pixels changes, one pixel is extracted as a pixel on the contour line.

According to this method, it is possible to reduce the influence of background noise and the influence of minute fluctuation of the outline by smoothing, thereby improving the robustness of detection and reducing the amount of information to be processed, thereby increasing the processing speed. Will be possible. That is, in general, when noise is included in the background around the inspection image, a part of the noise may be determined as the inspection image and the quality may be erroneously determined. By adopting this method, Erroneous determination due to the influence of noise can be suppressed.

According to a seventh aspect of the present invention, in the first to sixth aspects of the present invention, the inspection image for obtaining a density difference between a corresponding pixel and an image obtained by deforming the teaching image using the conversion parameter is a density value of each pixel of the teaching image. When the difference from the density value of the corresponding pixel of the original inspection image is obtained by performing weighting on the original inspection image, only the pixels formed in the region formed by the pixel having the difference value exceeding the specified threshold value are included.

According to this method, each pixel of the teaching image is weighted, so that pixels having low importance in the teaching image are removed from the comparison target with the inspection image, so that when comparing the inspection image with the teaching image, Can be reduced. In addition, since it is possible not to consider pixels having low importance, accurate determination can be made for a region having high importance.

According to an eighth aspect of the present invention, in the first to seventh aspects, at the time of the appearance inspection, a connected region of the extracted pixels is obtained, and a distribution of the connected region on the teaching image, a density distribution in the connected region, and It is characterized in that the type of a defect on the appearance of the inspection object is determined using the shape of the connection area.

That is, in the conventional configuration, the quality is determined using only the difference between the area of the inspection image and the area of the teaching image. Therefore, the type of the defect cannot be determined. Also, the type of defect can be determined.

According to the method of the present invention, the contour of the inspection image and the teaching image is used, and the inspection image is regarded as an image obtained by adding a deformation represented by a linear expression or a quadratic expression to the teaching image, and the contour line is minimized by two. The conversion parameter is calculated by applying the multiplication method, and the conversion parameter is extracted as the information of the primary deformation or the secondary deformation. Therefore, 1 is generated due to the influence of the imaging environment when the inspection object is imaged by the image input means. There is an advantage that the influence of the secondary deformation and the secondary deformation can be removed, and only the defective portion of the inspection image can be accurately detected, and an accurate appearance inspection can be performed.

(Embodiment 1)
As shown in FIG. 2, the presence or absence of a defect in the appearance of the inspection object Ob is extracted using an image obtained by imaging the imaging space including the inspection object Ob by the image input unit 1 such as a TV camera. I do. Although not shown, the image input means 1 includes an A / D converter, and the captured image is a grayscale image in which each pixel is associated with a digital density value. The grayscale image is stored in the image memory 2. Note that the image input means 1 is not limited to a TV camera, and it is also possible to use a scanner or the like, and there is no particular limitation as long as a grayscale image can be obtained.

Also, in the present invention, similarly to the conventional configuration, a teaching image for comparison with a captured image is set in advance, and the teaching image is stored in the template storage unit 3. As the teaching image stored in the template storage unit 3, a grayscale image of the inspection object Ob, which is known to have no defect, captured by the image input unit 1 under appropriate illumination is used. As a pre-processing, a Sobel filter (for a 3 × 3 local image area) is applied as a pre-process to the teaching image in the contour extraction unit 4 to extract the contour of the teaching image. When extracting the contour, a line connecting the pixels having the maximum value obtained by applying the Sobel filter is traced as the contour, and the differential value of the pixel to be traced is 0, 45, 90, or 135. Degrees, 180 degrees, 225 degrees, 270 degrees, and 315 degrees (for example, the horizontal right direction of the image is 0 degree, and the counterclockwise direction is a positive angle), and the maximum value is obtained. Only when the angle difference between the direction to be taken and the normal of the already traced contour is within ± 45 degrees, the pixel to be tracked next is extracted as a pixel on the contour. By this processing, the possibility of erroneously detecting a contour line can be reduced.

As shown in FIG. 1, the inspection image stored in the image memory 2 is first subjected to preprocessing by the preprocessing unit 5 (S1), and the approximate position / rotation angle and teaching of the inspection image in the captured image are obtained. An approximate scaling factor with the image is determined. The position, rotation angle, and enlargement / reduction ratio of the inspection image with respect to the teaching image are obtained with low accuracy. The pre-processing unit 5 uses a conventionally known technique such as “generalized Hough transform” and “normalized correlation” in order to obtain the approximate position / rotation angle / enlargement / reduction ratio of the inspection image with respect to the teaching image. The position, rotation angle, and enlargement / reduction ratio obtained by the pre-processing unit 5 each include a relatively large error. However, since the range of the error is almost fixed, the range of the position, rotation angle, and the enlargement / reduction ratio is narrowed down. Will be. The error ranges of the position, the rotation angle, and the enlargement / reduction ratio are, for example, about ± 4 pixels, ± 9 degrees, and ± 10%. The pre-processing unit 5 can roughly determine the approximate range of the position, the rotation angle, and the enlargement / reduction ratio. However, since the pre-processing unit 5 does not cope with the fluctuation of the imaging environment for the inspection object Ob, the pre-processing unit 5 calculates The following processing is performed so that the teaching image is deformed by applying the position, the rotation angle, and the enlargement / reduction ratio to the teaching image so as to be able to cope with a change in the imaging environment. That is, the teaching image in the following processing is a teaching image deformed by the preprocessing unit 5 so as to match the inspection image.

First, a technique similar to the teaching image is applied to the inspection image in the vicinity of the approximate position determined in the preprocessing to extract the contour line of the inspection image (S2). Next, the conversion parameter calculation unit 6 obtains a conversion parameter by the following processing. That is, the conversion parameter calculation unit 6 first performs coordinate conversion on the outline of the teaching image by using, as parameters, the approximate position and rotation angle of the inspection image obtained by the preprocessing and the approximate scaling ratio with the teaching image. Superimpose on the inspection image (S3). Here, as shown in FIG. 3, the inspection image P2 is searched from each point (hereinafter, referred to as “contour point”) on the contour L1 of the teaching image P1 in the normal direction of the contour L1 of the teaching image P1. Then, the contour points on the contour line L2 of the inspection image P2 that are respectively first found when each contour point is searched as a starting point are set as the corresponding points of the contour points of the teaching image P1 (S4). Corresponding points are obtained for all the contour points of the teaching image P1, and an error function (cumulative square error) Q shown as Expression (1) is obtained. The error function Q is expressed by a linear relationship between the contour lines of the teaching image and the inspection image as shown by equations (2) and (3). (A relationship in which the deformed image represented by で is regarded as an inspection image). Next, the error function Q is partially differentiated with each of the conversion parameters A to F (described later) included in the error function Q to set dQ = 0, and a simultaneous equation in which each of the conversion parameters A to F is an unknown is solved to solve each of the equations. When the conversion parameters A to F are obtained, the conversion parameters including the position, the rotation angle, and the enlargement / reduction ratio are obtained (S5).
Q = Σ (Qx ² + Qy ² ) (1)
However,
Qx = αn (Xn− (Axn + Byn + C)) (2)
Qy = αn (Yn− (Dxn + Eyn + F)) (3)
xn, yn: contour point Xn, Yn of teaching image P1: contour point A = β · cos θ of inspection image P2
B = −γ · sinφ
C = dx
D = β · sin θ
E = γ · cosφ
F = dy
It is. Also, αn: weight, β: scaling ratio in x direction, γ: scaling ratio in y direction, θ: rotation angle around x axis, φ: rotation angle around y axis, dx: displacement in x direction (position ), Dy: Displacement (position) in the y direction.

In equation (1), the sum total of (Qx ² + Qy ² ) is obtained for all contour points of the teaching image P1. The conversion parameters A to F are obtained as values satisfying ∂Q / ∂A = 0, ∂Q / ∂B = 0,..., ∂Q / ∂E = 0, and ∂Q / ∂F = 0. The weight αn can be variously set. The inner product of the normal line between the teaching image P1 and the inspection image P2, the ratio of each distance to the maximum value of the distance between the contour points between the teaching image P1 and the inspection image P2, the teaching The reciprocal of the distance between the contour points of the image P1 and the inspection image P2 can be used. It is also possible to set the weight αn to 1 and not to perform weighting. Further, the weight αn can be set for each pixel.

When the values of β, γ, θ, φ, dx, and dy obtained as described above become equal to or less than half of the set detection accuracy (S6), the processing ends and the position / rotation angle is determined.・ Output the scaling ratio. When the values of β, γ, θ, φ, dx, and dy exceed half of the set detection accuracy, the coordinates of the contour points of the teaching image P1 are converted based on the obtained conversion parameters AF. The same processing is performed again, and the same processing is repeated as a condition for determining the end of the calculation (S3 to S6). When this process ends (S7). After coordinate conversion of the teaching image P1 based on the obtained conversion parameters A to F (S8), the defect extracting unit 8 obtains a difference between the converted teaching image P1 and the inspection image P2 (S9), thereby obtaining an imaging environment. Can remove the influence of the primary deformation of the inspection image P2 due to the fluctuation of the inspection image P2, and it is possible to accurately detect only the defective portion of the inspection image P2. The extracted defect portion is input to the defect type determination unit 9, and the type of the defect is determined by applying the technology of the fourth embodiment described later. 2 except for the image input unit 1 are realized by causing a computer device to execute an appropriate program.

For example, if a TV camera having no shutter function, such as a CMOS camera, is used for the image input means 1 and the inspection object Ob is moving on a line, a teaching image shown in FIG. The inspection image P2 is deformed as shown in FIGS. 4B and 4C with respect to P1 (in the illustrated example, the inspection image P2 is deformed as if a shear force acts in the horizontal direction). Such a deformation can be represented by a linear relationship as described above. Therefore, by using the above-described procedure of the present embodiment, it is possible to extract the position, the rotation angle, and the enlargement / reduction ratio related to the primary relation, and thus, ignoring the apparent deformation, only the essential defect of the inspection image P2 is determined. Inspection becomes possible.

(Embodiment 2)
In the present embodiment, as in the first embodiment, first, the correspondence between the contour points of the teaching image P1 and the inspection image P2 is determined. That is, after the pre-processing, when the inspection image P2 is searched in the normal direction of the contour L1 of the teaching image P1 from each contour point on the contour L1 of the teaching image P1, and the search is performed using each contour point as a starting point. The contour points on the contour line L2 of the inspection image P2, which are found first, are set as the corresponding points of the contour points of the teaching image P1. Further, corresponding points are obtained for all the contour points of the teaching image P1. In the present embodiment, a value represented by Expression (4) is used as the error function Q. The error function Q is a quadratic relationship between the teaching image P1 and the inspection image P2 (if an image obtained by adding a deformation represented by a quadratic expression such as Expressions (5) and (6) to the teaching image is an inspection image). Relationship).
Q = Σ (Qx ² + Qy ² ) (4)
However,
Qx = αn (Xn− (Gxn ² + Hxn · yn + Iyn ² + Axn + Byn + C)) (5)
Qy = αn (Yn− (Jxn ² + Kxn · yn + Lyn ² + Exn + Fyn + G)) (6)
xn, yn: contour points Xn, Yn: contour points A to F of the inspection image P2 are the same as those of the equation (1) shown in the first embodiment. Further, .alpha.n: Weights, G: contribution to the deformation amount in the x direction of xn ^2, H: contribution to the deformation amount in the x direction of xn · yn, I: contribution to the deformation amount in the x direction of yn ² whenever, J: contribution to the amount of deformation of the y direction of xn ^2, K: contribution to the amount of deformation of the y direction of xn · yn, L: is a contribution to the y-direction deformation amount of yn ^2.

In equation (4), the sum total of (Qx ² + Qy ² ) is obtained for all the contour points of the teaching image P1. The conversion parameters A to L are obtained as values satisfying ∂Q / ∂A = 0, ∂Q / ∂B = 0,..., ∂Q / ∂K = 0, and ∂Q / ∂L = 0. Further, the same weight as in the first embodiment is used for the weight αn. By obtaining the conversion parameters A to L in the above procedure, the position, rotation angle, and enlargement / reduction ratio in the case of the quadratic relationship can be obtained.

When the conversion parameters A to L are obtained, the coordinates of the teaching image P1 are converted based on the conversion parameters A to L in the same manner as in the first embodiment, and then the difference between the converted teaching image P1 and the inspection image P2 is obtained. As a result, it is possible to remove the influence of the secondary deformation of the inspection image P2 due to the change in the imaging environment, and it is possible to accurately detect only the defective portion of the inspection image P2. Note that, as in the first embodiment, the end conditions of the process are that the values of β, γ, θ, φ, dx, and dy are equal to or less than half of the set detection accuracy. Do not use.

For example, when the inspection image P2 is rotated in the optical axis direction (depth direction) of the camera with respect to the teaching image P1 shown in FIG. 5A, deformation as shown in FIG. If there is, a deformation as shown in FIG. That is, the deformation can be represented by the above-described quadratic relation. Therefore, by applying the technology of the present embodiment described above, it is possible to inspect only the essential defects of the inspection image P2 ignoring the apparent deformation.

Now, assuming that a moving image is tracked, the movement of the inspection object Ob becomes three-dimensional, and the distortion of the lens differs for each location. Therefore, the inspection image P2 is as shown in FIGS. Deformation will occur. Here, by applying the technique of the present embodiment, this type of deformation can be removed, and tracking and inspection of the inspection image P2 can be performed. Further, in the moving image, the position of the inspection image P2 is near the detection position on the previous screen, so that the preprocessing is not required for images other than the first one screen.

(Embodiment 3)
In each of the above-described embodiments, the contour lines of the teaching image P1 and the inspection image P2 are extracted by using the Sobel filter in the contour line extraction unit 4. However, in this embodiment, when extracting the contour lines, Japanese Patent Application Laid-Open No. 2002-183713. The technology disclosed in Japanese Unexamined Patent Publication (Kokai) No. H10-209400 is used. That is, first, the teaching image P1 and the inspection image P2 are smoothed so that the average value of the pixel values (density values) of the neighboring area of each pixel is set to the value of each pixel, and the sizes of the neighboring areas differ. Two types of smooth images are obtained. That is, a first smoothed image in which the average value of the density values of the pixels in the first area which is the neighboring area is the pixel value of each pixel, and an average value of the density values of the pixels in the second area wider than the first area And a second smoothed image having the pixel value of each pixel. Next, after appropriately weighting the pixel value of each pixel of the first smoothed image (the weighting factor is 1 or more), a difference image is obtained by subtracting from the pixel value of each pixel of the second smoothed image. When the sign of the pixel value of each pair of adjacent pixels changes in the difference image thus obtained, one pixel is extracted as a pixel on the contour line. Although the technique disclosed in Japanese Patent Application Laid-Open No. 2002-183713 does not extract a contour line, it extracts a pixel having a positive pixel value as a pixel of a target portion, so that the pixel value changes from positive to negative. The changing point corresponds to the contour line. Here, by appropriately selecting a smoothing range and a weighting coefficient, it is possible to extract only global information regarding the contour. The processing for obtaining the smoothing, the weighting, and the difference image may be performed by different means.

Now, when a contour is extracted by applying a Sobel filter to the teaching image P1 shown in FIG. 6, the contour is extracted with relatively high accuracy as shown in FIGS. 7 (a) to 7 (c). Therefore, the amount of information is increased, and a background noise component and a minute fluctuation component of an outline are also extracted. On the other hand, in the case of extracting a contour line by obtaining a difference between smoothed images as described above, the effect of smoothing by making the neighboring area to be smoothed relatively small and the weight coefficient relatively small. 8A to 8C, it is possible to reduce the information on the contour line as compared with the case where the Sobel filter is applied, as shown in FIGS. In addition, by performing the smoothing, the background noise component and the minute fluctuation component of the contour line are also smoothed, and the influence is reduced. If the neighborhood area to be smoothed is relatively large and the weighting coefficient is also relatively large, the information on the contour line can be further reduced as shown in FIGS.

Even if the information amount regarding the contour is reduced as described above, the purpose of extracting the contour is to roughly extract the position, the rotation angle, and the enlargement / reduction ratio between the teaching image P1 and the inspection image P2. The purpose can be sufficiently achieved, and the processing time can be shortened by reducing the amount of information. For example, assuming that a processing time of 80 [ms] is required in the example of FIG. 7 to which the Sobel filter is applied, the processing time is 20 [ms] in the example of FIG. 8, 10 [ms] in the example of FIG. Can be shortened. By the way, by adopting the present embodiment, it is possible to increase the speed 4 to 8 times as compared with the case where the Sobel filter is used. Moreover, the robustness with respect to the detection of the deformation is improved by being less affected by the noise component and the fluctuation component of the outline. Other configurations and functions are the same as those of the first and second embodiments.

(Embodiment 4)
In the present embodiment, when calculating the difference between the teaching image P1 and the inspection image P2, the captured image is smoothed in two levels of strength, and one of the two types of obtained smooth images is weighted to calculate the difference. Take. By performing such processing, an image obtained by extracting only the inspection image from the captured image can be obtained. This procedure is specifically described in JP-A-2002-183713.

If the difference between the pixels in this area and the pixels in the teaching image P1 is determined after extracting the area of the inspection image P2 as described above, unnecessary area information is removed, which is effective in removing background noise. . For example, when the object to be inspected is a character and another character exists behind the character to be inspected in the captured image as shown in FIG. 10A, the inspection is performed as shown in FIG. An area corresponding to the image P2 (an area surrounded by a thin black line in FIG. 10B) can be extracted. Only the pixels in this area are different from the teaching image P1 in FIG. 10C. As described above, it is possible to extract a defective area by removing background noise. That is, the area Dx corresponding to the defect can be extracted only from the inspection image P2, and the appearance inspection for determining the type of the defect as described later becomes easy.

Moreover, according to the method of the present embodiment, only the global shape is extracted due to the averaging effect, so that minute variations on the outline and the character surface are deleted, and only the global defective portion is extracted. This makes it possible to inspect only essential defects on the inspection image. Other configurations and operations are the same as those of the first to third embodiments.

(Embodiment 5)
In each of the embodiments described above, the technique of extracting a part that does not match the teaching image P1 from the inspection image P2 as a defective pixel has been described. In the present embodiment, the type of the defect is determined from the pixel extracted as the defective pixel. The technology will be described. That is, the processing in the defect type determination unit 9 shown in FIG. 2 will be described. In order to determine the type of defect, in the present embodiment, a density difference between pixels corresponding to each other in the inspection image P2 and the teaching image P1 is determined, and a connected region (a pixel) in which the density difference exceeds a predetermined threshold value. (Difference area). That is, an area where pixels whose density differences exceed the threshold value is adjacent is extracted as a connected area. Next, it is determined whether the connected region is within the range of the inspection image or the range of the background region.

First, processing for the inside of the inspection image P2 will be described. The following four types of processing are performed on the inspection image P2 to determine the density distribution and shape of the connected region.
(1) The contour of the connected area is tracked, and it is verified whether or not the contour of the teaching image P1 is included.
(2) When the outline of the teaching image P1 is included in the outline of the connected region, it is verified whether the included outline of the teaching image P1 is divided into a plurality of parts.
(3) The density distribution (variance) in the connected area is obtained, and it is verified whether the obtained value of the variance is within a specified range (a value specified by a user, a value determined in a previous test, and the like). This is a process for verifying whether a gradation has occurred in the connected area.
(4) The proportion of the portion of the contour of each connected region that matches the contour of the teaching image P1 (= the length of the portion that matches the contour of the teaching image P1 / the total length of the contour of the connecting region) is defined. It verifies whether the value is within the specified range (the value specified by the user, the value determined in the previous inspection, etc.).

Next, by combining the results of the above-described four types of processing, the types of defects are classified into four types of “scratch”, “chip”, “blurred”, and “thin”. The conditions for determining each defect are as follows.
(A) When the connected area does not include the contour line of the teaching image P1, it is determined to be “scratch”.
(B) When the portion including the outline of the teaching image P1 in the outline of the connected region is divided into a plurality of parts and the density distribution in the connected region is constant (the variance is equal to or less than the threshold), or the outline of the connected region , The portion including the outline of the teaching image P1 is not divided, and the value of (length of the portion that matches the outline of the teaching image P1 / total length of the outline of the connection area) is smaller than the specified threshold value and is connected. If the density distribution in the area is constant (the variance is equal to or less than the threshold), it is determined as “missing”.
(C) In the case where the portion including the outline of the teaching image P1 in the outline of the connected region is divided into a plurality of parts and gradation occurs in the density distribution in the connected region, or the case where the teaching image P1 is included in the outline of the connected region Is not divided, and the value of (length of the part that matches the contour of the teaching image P1 / total length of the contour of the connected area) is smaller than a specified value, and the density distribution in the connected area If there is a gradation in the image, it is determined that the image is "faint".
(D) In cases other than "scratch", "chip", and "blurring", it is determined to be "thin".

Next, processing on the outside of the inspection image P2 (that is, the background side) will be described. The following five types of processing are performed on the background side to determine the density distribution and shape of the connected region.
(1) The contour of the connected area is tracked, and it is verified whether or not the contour of the teaching image P1 is included.
(2) When the outline of the teaching image P1 is included in the outline of the connected region, it is verified whether the included outline of the teaching image P1 is divided into a plurality of parts.
(3) Each closed curve of the outline of the teaching image P1 is tracked, and (the length of a portion included in the connected area in the closed curve of the outline of the teaching image P1 / the total length of the closed curve of the teaching image P1) is within a prescribed range. (Eg, values specified by the user, values determined in previous tests, etc.).
(4) The density distribution (variance) in the connected area is obtained, and it is verified whether the value of the obtained variance is within a specified range (a value specified by a user, a value determined in a previous test, and the like). This is a process for verifying whether gradation has occurred in the connected region.
(5) The proportion of the part of the contour of each connected area that matches the contour of the teaching image P1 (= the length of the part that matches the contour of the teaching image P1 / the total length of the contour of the connecting area) is defined. It verifies whether the value is within the specified range (the value specified by the user, the value determined in the previous inspection, etc.).

Next, by combining the results of the above-described five types of processing, the types of defects are classified into five types of “background noise”, “crushed”, “extra”, “bleeding”, and “fat”. The conditions for determining each defect are as follows.
(A) If the connected region does not include the outline of the teaching image P1, it is determined as "background noise".
(B) The outline of the teaching image P1 is not divided by the outline of the connected region, and the value of the length of the portion of the outline of the teaching image P1 included in the connected region / the total length of the closed curve of the teaching image P1) is If it is larger than the specified threshold value, it is determined as “crush”.
(C) When the portion of the outline of the connected region including the outline of the teaching image P1 is divided into a plurality of parts and the density distribution in the connected region is constant (variance is equal to or less than a threshold), or the outline of the connected region , The portion including the outline of the teaching image P1 is not divided, and the value of (length of the portion that matches the outline of the teaching image P1 / total length of the outline of the connection region) is larger than the specified value, and the connection region If the density distribution inside is constant (the variance is equal to or less than the threshold), it is determined to be “extra”.
(D) When the portion including the outline of the teaching image P1 in the outline of the connected region is divided into a plurality of parts and gradation occurs in the density distribution in the connected region, or the teaching image P1 is generated in the outline of the connected region. Is not divided, and the value of (length of the part that matches the contour of the teaching image P1 / total length of the contour of the connected area) is larger than a specified value, and the density distribution in the connected area Is determined to be "smeared" when gradation occurs.
(E) In cases other than “background noise”, “crush”, “extra”, and “bleeding”, it is determined to be “fat”.

FIG. 11 shows an example in which a defect is determined as described above. FIG. 11A shows the teaching image P1, and the outline of the teaching image P1 can be extracted as shown in FIG. 11B. Here, assuming that the inspection image P2 shown in FIG. 12A is obtained, as shown in FIG. 12B, the portion including the outline of the teaching image P1 in the outline of the connected region is not divided and Since the ratio of the length of the portion included in the connected region to the total length of the closed curve of the image P1 is large, it is determined that the image is “crushed”. Further, assuming that the inspection image P2 shown in FIG. 13A is obtained, as shown in FIG. 13B, the portion including the outline of the teaching image P1 in the outline of the connected region is not divided, and Since the value of the length of the portion that matches the contour of the image P1 / the total length of the contour of the connected area) is smaller than the specified threshold value and the variance in the connected area is equal to or less than the threshold value, it is determined to be “missing”. . Further, as is clear from FIG. 14B, it is determined that an inspection image P2 such as that shown in FIG. 14A has defects such as "chip", "fat" and "extra". it can.

Note that “scratch”, “chip”, “blurring”, and “thinning” may be determined by the following combinations. In the following, logical symbols are used, ∧ represents logical product, ∨ represents logical sum, and ¬ represents negation. The processes (1) to (4) in the following (A) to (D) correspond to the four types of processes for the inside of the teaching image P1 described above.
(A) ¬ (1) is “scratch”.
(B) (1) ∧ (((2) ∧¬ (3)) ∨ (¬ (3) ∧ (4)) is “missing”.
(C) (1) ∧ (((2) ∧ (3)) ∨ ((3) ∧ (4)) is “blurred”.
(D) Combination "thin" other than (A) to (C).

In this case, the determination of “background noise”, “crush”, “extra”, “bleeding”, and “fat” is made by the following combinations. The processes (1) to (5) in the following (A) to (E) correspond to the above-described five types of processes on the outside of the teaching image P1.
(A) ¬ (1) is “background noise”.
(B) (1) ∧ (2) is “crushed”.
(C) (1) ∧¬ (2) ∧ (((3) ∧¬ (4)) ∨ (¬ (4) ∧ (5)) is “extra”.
(D) (1) ∧¬ (2) ∧ (((3) ∧ (4)) ∨ ((4) ∧ (5)) is “bleeding”.
(E) Combination other than (A) to (D) "Fat".

結果 The result of such a determination is shown in FIG. FIG. 15A shows a teaching image P1, and FIGS. 15B to 15E show an inspection image P2 and a determination result, respectively.

As described above, by using nine types of information on the density distribution and the shape in the connected region in combination, it is possible to perform the inspection such as “scratch”, “chip”, “blurring”, “thinning” within the range of the inspection image P2. It is possible to determine a total of nine types of defects such as “background noise”, “crush”, “extra”, “bleeding”, and “fat” on the background side of the image P2. This embodiment can be applied to any of the technologies of the first to third embodiments.

In the above-described example, an example has been described in which the conversion parameters are obtained after the teaching image stored in the template storage unit 3 is deformed by the preprocessing. However, the position of the inspection target is fixed with respect to the image input unit 1. When the deviation between the inspection image and the teaching image is small, as in the case where there is, the conversion parameter may be obtained without performing the preprocessing.

Further, in the above-described example, the density difference of the corresponding pixel between the image obtained by deforming the teaching image by the conversion parameter and the inspection image included in the captured image is obtained. However, the density value of each pixel of the teaching image is weighted. The difference between the pixel value of each pixel of the weighted teaching image and the density value of the corresponding pixel of the original inspection image included in the captured image is determined, and the difference value at this time is formed of pixels exceeding a specified threshold. A density difference between an image obtained by deforming the teaching image and the image obtained by transforming the teaching image using the conversion parameter may be obtained for pixels only in the region to be processed.

FIG. 4 is an operation explanatory diagram of the first embodiment of the present invention. It is a block diagram same as the above. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing of Embodiment 2 of this invention. FIG. 13 is an operation explanatory diagram of the third embodiment of the present invention. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing of Embodiment 4 of this invention. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing same as the above.

Explanation of reference numerals

DESCRIPTION OF SYMBOLS 1 Image input means 2 Image memory 3 Template storage part 4 Contour extraction part 5 Preprocessing part 6 Conversion parameter calculation part 7 Judgment processing part 8 Defect extraction part 9 Defect type discrimination part Ob Inspection object L1, L2 Contour line P1 Teaching image P2 inspection image

Claims (8)

The contours of the inspection image, which is the image of the inspection object, and the reference teaching image are obtained, and the inspection image is regarded as an image obtained by adding a deformation represented by a linear expression to the teaching image, and the inspection image, the teaching image, and By applying the least-squares method to the pixels on the contour line, a conversion parameter representing the deformation of the inspection image with respect to the teaching image is calculated, and the density of the corresponding pixel between the image obtained by deforming the teaching image with the obtained conversion parameter and the inspection image An image processing method comprising: obtaining a difference, extracting a pixel having a density difference exceeding a preset threshold value, and using the extracted pixel for a visual inspection of an inspection object. The contours of the inspection image, which is the image of the inspection object, and the reference teaching image are obtained, and the inspection image is regarded as an image obtained by adding a deformation represented by a quadratic expression to the teaching image, and the inspection image and the teaching image are used. By applying the least-squares method to the pixels on the contour line, a conversion parameter representing the deformation of the inspection image with respect to the teaching image is calculated, and the density of the corresponding pixel between the image obtained by deforming the teaching image with the obtained conversion parameter and the inspection image An image processing method comprising: obtaining a difference, extracting a pixel having a density difference exceeding a preset threshold value, and using the extracted pixel for a visual inspection of an inspection object. The inspection image is included in a captured image obtained by capturing an image of a spatial region including an inspection target by an image input unit, and is taught by comparing a teaching image registered in a template storage unit in advance with the captured image as the teaching image. 3. The image processing method according to claim 1, wherein an image deformed so that the position, rotation angle, and enlargement / reduction ratio of the image are adjusted to the inspection image with low accuracy is used. 2. The method according to claim 1, wherein, when extracting the contour, a line that traces a pixel having a maximum value obtained by applying a Sobel filter to the inspection image and the teaching image is extracted as the contour. 2. The image processing method according to 2. When tracking the contour, the pixel to be tracked next when the angle difference between the direction in which the differential value of the pixel to be tracked next becomes a local maximum and the normal of the already tracked contour is within ± 45 degrees. 5. The image processing method according to claim 4, wherein is a pixel on the contour line. When extracting the contour, a first smoothed image is generated for each pixel of the inspection image and the teaching image, where the average value of the density values of the pixels in the first region that is in the vicinity is used as the pixel value. Generates a second smoothed image having an average value of the density values of the pixels in the second area wider than the first area as a pixel value, and then assigns a weight of one or more to each pixel value of the first smoothed image. After performing weighting by the coefficient, the difference image is obtained by subtracting from the pixel value of the corresponding pixel of the second smoothed image, and when the sign of the pixel value of each pair of pixels adjacent to the difference image changes, one of the pixels is changed. 3. The image processing method according to claim 1, wherein the image is extracted as a pixel on an outline. The inspection image for calculating the density difference of the corresponding pixel from the image obtained by deforming the teaching image by the conversion parameter is obtained by weighting the density value of each pixel of the teaching image and the density value of the corresponding pixel of the original inspection image. The image processing method according to any one of claims 1 to 6, wherein the image processing method includes pixels only in an area formed by pixels having a difference value exceeding a predetermined threshold value when the difference is obtained. At the time of the appearance inspection, a connected region of the extracted pixels is obtained, and the distribution of the connected region on the teaching image, the density distribution in the connected region, and the shape of the connected region are used to determine the type of defect in the appearance of the inspection object. The image processing method according to claim 1, wherein the determination is performed.

Images