JP6126290B1 - Appearance inspection device - Google Patents

Abstract

The present invention makes it possible to correctly determine the presence of a dent / blowing defect or a crack defect even when a portion having a low reflection intensity exists in an inspection object. A visual inspection apparatus according to the present invention generates a time-series image V (x, y, t) obtained by photographing an inspection object under a plurality of different optical conditions. 10a and the time series image V (x, y, t) are normalized based on the extended time series image obtained by connecting the normalization suppression constant and its reciprocal number to the time series image V (x, y, t). Based on the time-series normalized image generation unit 11a that generates the time-series normalized image I (x, y, t), the time-series normalized image I (x, y, t), and the imaging result of the normal product. A correlation distance image generation unit 12a that generates a correlation distance image Icd (x, y) indicating a correlation distance for each pixel between the generated time series normalized images is provided. [Selection] Figure 7

Description

  The present invention relates to an appearance inspection apparatus, and more particularly to an appearance inspection apparatus suitable for detecting blister / dent defects and crack defects.

  In the manufacturing process of an electronic component, it is necessary to inspect that the manufactured electronic component is free from scratches, dirt, foreign matter, and the like. This inspection is generally called an appearance inspection, and in recent years, it has been automatically performed using an image recognition device. Patent Documents 1 to 4 disclose examples of such appearance inspection.

  In Patent Document 1, a reference density value (median value, average value, etc. of luminance values) is calculated from a multi-value image to be inspected by a predetermined method, and the difference between the density of each pixel and the reference density value is calculated for each pixel. A technique of calculating and identifying a blob (a block defect) based on the result is disclosed. However, this technique can only inspect a uniform density surface such as the surface of a film without a pattern, and cannot inspect the appearance of a portion having structures of various shapes on the surface.

  Japanese Patent Laid-Open No. 2004-228561 targets printed matter instead of electronic components, but calculates a difference between the reference image and the inspection image for each pixel and determines that there is an abnormality when the calculated difference exceeds a threshold value. The technique of doing is disclosed. In this technique, the flatness is calculated for each predetermined rectangular area, and a threshold value is determined for each rectangular area based on the calculated flatness.

  In Patent Document 3, after the difference between the reference image and the inspection target image is converted into a numerical value called a defect degree by a fuzzy membership function created for each pixel, an integral value of the defect degree is calculated for each local region of an arbitrary size. A technique for recognizing a region where the calculated integral value is equal to or greater than a certain determination reference value as a defect is disclosed. The fuzzy membership function is a luminance difference value defect degree conversion filter, which reduces the influence of noise and plays a role of making it possible to distinguish between large defects and small defects.

  Patent Document 4 discloses a technique for detecting a defect from an image based on a plurality of optical conditions. The paragraph [0030] of the document describes that an average value of two images with different illumination conditions is used as a feature amount.

  Non-Patent Document 1 discloses a technique for restoring the shape of an object using illuminance difference stereo (photometric stereo). This technique obtains the luminance history of each pixel (a vector obtained by arranging the luminance observed under each light source and normalized so that the length becomes 1. Refer to Equation (1) in the same document) The normal vector of each pixel is estimated based on the geodesic distance between any two points obtained from the acquired luminance history (= absolute value of the difference in luminance history). This document shows that the geodesic distance and the inverse cosine of the inner product of the luminance history are approximately equal (see equation (9) of the document).

  Furthermore, Patent Document 5 discloses a technique for extracting an animal body other than the background from an image. This technique divides an image into a plurality of small regions, treats pixel groups in each small region as a one-dimensional vector, performs normalization (see paragraph [0006] of the same document), and then performs normalization. The variation in the time direction of the vector is calculated for each small region (see Equation (6) of the same document), and the moving object region is extracted based on the result.

JP 2012-167963 A JP 2012-103225 A JP 2004-038885 A Japanese Patent No. 5174535 JP 2002-032760 A

Yoichi Sato, “Object shape restoration based on similarity of luminance change accompanying illumination change”, online], [searched on September 15, 2016], Internet <URL: http://www.osakac.ac.jp/ viri / symposium07 / sato.pdf>

  By the way, one of the important purposes of visual inspection of electronic components is detection of blister / dent defects or crack defects. Since both are defects that can significantly impair the quality of the product, detection by visual inspection is essential.

  In practice, however, it is often difficult to detect blister / dent and crack defects. This is because these defects have the property of finally appearing when they are observed while changing the line-of-sight direction in various directions. In automatic inspection, it is difficult to change the line-of-sight direction freely from the viewpoints of cost and inspection time, and therefore invisible defects may occur depending on the degree and direction of blister / dent defects and crack defects.

  The inventor of the present application examined the application of photometric stereo as described in Non-Patent Document 1 to appearance inspection as a method for enabling detection of blisters / dents and crack defects even a little. ing. It is considered that abnormalities occurring in the inspection object can be found by obtaining the normal vector of each pixel by photometric stereo in each of the normal product and the inspection object and comparing the obtained normal vectors. .

  However, as a result of research conducted by the inventor of the present application, the method using photometric stereo as described above has a problem that correct measurement cannot be performed if there is a portion having a low reflection intensity on the surface of the inspection object. found. For example, in the case of a part having a hole in the center, reflected light cannot be obtained at the center, so that only camera noise is recorded. As a result, the hole portion cannot be inspected. In addition, in the case where a hole is formed only in the defective product, it is desired to recognize this hole as abnormal, but this is also impossible for inspection for the same reason. This problem is not limited to holes, and becomes apparent when there are regions with greatly different reflection intensities on the surface of the object. For example, color difference.

  Accordingly, one of the objects of the present invention is to provide an appearance inspection apparatus that can correctly determine the presence of a dent / blowing defect or a crack defect regardless of the reflection intensity of the inspection object.

An appearance inspection apparatus according to an aspect of the present invention includes a first time-series image generation unit that generates a first time-series image obtained by photographing a test object under a plurality of different optical conditions, A first time-series normalized image obtained by normalizing the first time-series image based on a first extended time-series image obtained by connecting a normalization suppression constant and its reciprocal to the first time-series image. A first time-series normalized image generation unit that generates a second time-series image that generates a second time-series image obtained by photographing a normal product under the plurality of different optical conditions And a second time obtained by normalizing the second time series image based on a second extended time series image obtained by connecting a normalization suppression constant and its reciprocal to the second time series image. A second time-series normalized image generating unit for generating a series-normalized image; A first correlation distance image generation unit configured to generate a first correlation distance image indicating a correlation distance for each pixel between the first time series normalized image and the second time series normalized image; It is characterized by that.

  According to the present invention, since a weak signal such as camera noise is not amplified in normalizing the first and second time-series images, the expected inspection is independent of the reflection intensity of the inspection object. Can be carried out.

  In the appearance inspection apparatus, the first extended time-series image includes a first connected image obtained by adding the normalization suppression constant to an average value in a time direction of the first time-series image for each pixel; An image formed by connecting, to each of the first time-series images, a second connected image obtained by subtracting the normalization suppression constant from an average value in the time direction of the first time-series image for each pixel; The first time-series normalized image generation unit generates the first time-series normalized image based on an average value and a standard deviation in a time direction of the first extended time-series image, and the second time-series normalized image is generated. The extended time-series image includes a third connected image obtained by adding the normalization suppression constant to an average value in the time direction of the second time-series image for each pixel, and the second time-series image for each pixel. A fourth connected image obtained by subtracting the normalization suppression constant from an average value in the time direction of The second time-series image is an image connected to the second time-series image, and the second time-series normalized image generator generates the second time-series image based on an average value and a standard deviation in a time direction of the second extended time-series image. The second time-series normalized image may be generated. By doing so, it is possible to obtain a time-series image suitable for detection of blister / dent defects.

  In the above appearance inspection apparatus, a third time-series image generation unit that generates a plurality of third time-series images obtained by repeating a plurality of times of photographing normal products under a plurality of different optical conditions is further provided. The second time-series image may be an image configured by an average value for each pixel of the plurality of third time-series images. In this way, it is possible to reduce the error of the result that may be caused by the variation between normal products.

  In the appearance inspection apparatus, for each of the plurality of third time series images, based on a third extended time series image formed by connecting a normalization suppression constant and its reciprocal number to the third time series image, For each of the third time-series normalized image generating unit that generates a third time-series normalized image obtained by normalizing the third time-series image, and the plurality of third time-series normalized images, Generated by a second correlation distance image generation unit that generates a second correlation distance image indicating a correlation distance for each pixel between the second time-series normalized image and the second correlation distance image generation unit A first average image constituted by an average value for each pixel of the plurality of second correlation distance images, and a first deviation constituted by a standard deviation for each pixel of the plurality of second correlation distance images. A first probabilistic behavior to generate a standard deviation image of And the first correlation distance image and the plurality of second correlation distance images based on the first description distance section, the first correlation distance image, the first average image, and the first standard deviation image. A first probabilistic distance image generation unit that generates a first probabilistic distance image indicating the probabilistic distance for each pixel may be further provided. In this way, it is possible to detect defects on the surface of the inspection object using a so-called stochastic distance image.

  In each of the above appearance inspection apparatuses, the third extended time-series image is a fifth connection obtained by adding the normalization suppression constant to the average value in the time direction of the third time-series image corresponding to each pixel. The image and the sixth connected image obtained by subtracting the normalization suppression constant from the average value in the time direction of the third time-series image corresponding to each pixel are connected to the corresponding third time-series image. The third time-series normalized image generation unit (11c) for each of the plurality of third time-series images in the time direction of the corresponding third extended time-series image The third time-series normalized image may be generated based on the average value and the standard deviation. In this way, it is possible to obtain a time series image suitable for detection of blister / dent defects for each of the plurality of third time series images.

  In each of the appearance inspection apparatuses, a first time-direction average image generating unit that generates a first time-direction average image indicating an average of the first time-series images in the time direction, and the plurality of third time series For each of the images, a second time direction average image generating unit that generates a second time direction average image indicating an average of the time direction, and the plurality of second time direction average image generating units generated by the second time direction average image generating unit. 2nd average image comprised by the average value for every pixel of 2 time direction average images, and 2nd standard deviation image comprised by the standard deviation for every pixel of these 2nd time direction average images Based on the second stochastic behavior description unit that generates the first temporal direction average image, the second average image, and the second standard deviation image, and the first temporal direction average image and the second standard deviation image Multiple second time directions It may further include a second stochastic distance image generation unit for generating a second probabilistic range image showing a probabilistic distance for each pixel with the average image. In this way, since the time direction average image can be further used, it is possible to detect not only a blister / dent defect but also a general defect such as dirt or foreign matter.

  In each of the appearance inspection apparatuses, a first time-direction standard deviation image generation unit that generates a first time-direction standard deviation image indicating a time-direction standard deviation of the first time-series image, and the plurality of third For each of the time-series images, a second time-direction standard deviation image generating unit that generates a second time-direction standard deviation image indicating a standard deviation in the time direction, and the second time-direction standard deviation image generating unit By a third average image composed of average values for each pixel of the plurality of second time direction standard deviation images to be generated, and by a standard deviation for each pixel of the plurality of second time direction standard deviation images Based on a third stochastic behavior description part that generates a third standard deviation image that is configured, the first time-direction standard deviation image, the third average image, and the third standard deviation image, The first time A third probabilistic distance image generating unit that generates a third probabilistic distance image indicating a probabilistic distance for each pixel between the direction standard deviation image and the plurality of second time direction standard deviation images; It is good. This makes it possible to further use the time-direction standard deviation image, so that it is possible to more reliably detect defects on the surface of the inspection object.

  In the appearance inspection apparatus, a fourth time-series image obtained by photographing an inspection object under a plurality of different optical conditions is displayed in a first direction and a second direction that intersects the first direction. A first time-series differential image generation unit that generates first and second time-series differential images by differentiating each of the first time-series differential images, and a plurality of differential images constituting the first time-series differential image. An average value in the time direction of one time-series differential image is subtracted, and further, an average value in the time direction of the second time-series differential image from each of the plurality of differential images constituting the second time-series differential image Is obtained by photographing a normal product under the plurality of different optical conditions, and a first time-series gradient vector image generation unit that generates a first time-series gradient vector image. Time series A second time-series differential image generator for generating third and fourth time-series differential images by differentiating the image in the first and second directions, respectively, and the third time-series differential image An average value in the time direction of the third time-series differential image is subtracted from each of the plurality of differential images constituting the fourth differential image, and the fourth differential image constituting the fourth time-series differential image is further subtracted from the fourth differential image. A second time-series gradient vector image generating unit that generates a second time-series gradient vector image by subtracting an average value in the time direction of the time-series differential image of the first time-series differential image May be the first time-series gradient vector image, and the second time-series gradient image may be the second time-series gradient vector image. This makes it possible to use the present invention for detecting crack defects.

  In the appearance inspection apparatus, the first extended time-series image includes a seventh connected image in which the density of each pixel is equal to the normalization suppression constant, and the density of each pixel is equal to a reciprocal of the normalization suppression constant. An eighth connected image connected to the first time-series gradient vector image, and the first time-series normalized image generation unit is configured to generate a time-direction image of the first extended time-series image. The first time-series normalized image is generated based on a length as a vector, and the second extended time-series image includes a ninth connected image in which the density of each pixel is equal to the normalization suppression constant; The tenth connected image in which the density of each pixel is equal to the reciprocal of the normalization suppression constant is connected to the second time-series gradient vector image, and the second time-series normalized image is generated. A vector in the time direction of the second extended time-series image It is also possible to generate the second time series normalized image based on the length of the. This makes it possible to obtain a time-series image suitable for detecting crack defects.

  According to the present invention, since a weak signal such as camera noise is not amplified in normalizing the first and second time-series images, the expected inspection is independent of the reflection intensity of the inspection object. Can be carried out.

It is a figure which shows the external appearance and functional block of the external appearance inspection apparatus 1 by embodiment of this invention. (A) is a top view which shows an example of arrangement | positioning of the illuminating device 3 shown in FIG. 1, (b) is a side view corresponding to (a). It is a figure which shows another example of arrangement | positioning of the illuminating device 3 shown in FIG. 1, (a) shows the state which turned on the partial illumination 3c, (b) showed the state which turned on the partial illumination 3b, respectively. Yes. It is a figure which shows the image formed by imaging the test target T which has a dent defect in the state which all the partial illumination 3a-3e shown in FIG. 3 was lighted. (A)-(e) is a figure which shows the image formed by imaging the same test target T as FIG. 4 in the state which each lit the partial illumination 3a-3e shown in FIG. 3 independently. It is a figure which shows the part regarding the production | generation of the model information for detecting a blister / dent defect among the functional blocks of the image processing apparatus 5 shown in FIG. It is a figure which shows the part regarding the image process (including a comparison process with model information) of the test target T for detecting a blister / dent defect among the functional blocks of the image processing apparatus 5 shown in FIG. (A) is a figure which shows an example of the time-sequential image _TV (x, y, t) of the test target T which has a dent defect, (b) is the time-sequential image V (x, y) of (a). It is a figure which shows the time series normalization image I (x, y, t) produced | generated based on y, t). (A) is a diagram showing an example of a case of the normal product series image _{S V (x, y, t} ), (b) , the time-series images V (x of the test object T with a dent defect, y is a diagram showing an example of a t), (c), the time-series images _S V of (a) (x, y, t) and time-series images _T V (x in (b), y, t) in the based probabilistic range image _SFD 1 which is generated by (x, y) is a diagram showing a time series of (d) is time-series images _S V of (a) (x, y, t) and (b) image V (x, y, t) the probability is generated based on the distance image _SFD 2 (x, y) is a diagram showing, (c), the time-series images _S V (x in (a), y , T) and a probabilistic distance image SFD ₃ (x, y) generated based on the time-series images V (x, y, t) of (b). It is a figure which shows the part regarding the production | generation of the model information for detecting a crack defect among the functional blocks of the image processing apparatus 5 shown in FIG. It is a figure which shows the part regarding the image process (a comparison process with model information) of the test target T for detecting a crack defect among the functional blocks of the image processing apparatus 5 shown in FIG. (A) is a figure which shows an example of the time series image V (x, y, t) of the test target T which has a crack defect, (b) is the time series image V (x, y of (a). , T) is a diagram showing a time-series normalized gradient vector image J (x, y, u) generated based on. (A) is a diagram showing an example of a case of the normal product series image _{S V (x, y, t} ), (b) , the time-series images V (x of the test object T with a crack defects, y is a diagram showing an example of a t), based on (c) are (time-series images _S V (x in a), y, t) and time-series images V in (b) (x, y, t) generated Te stochastic distance image _SFD 4 (x, y) is a diagram showing, (d), the time-series images _S V of (a) (x, y, t) and time-series images of (b) V is a diagram showing (x, y, t) probabilistic range image _SFD 5 generated based on the (x, y) and, (c), the time-series images _S V (x, y of (a), t) and is a diagram showing time-series images V (x, y, probabilistic distance image _SFD 6 generated on the basis of t) to (x, y) of (b).

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a diagram showing an appearance and functional blocks of an appearance inspection apparatus 1 according to an embodiment of the present invention. As shown in the figure, the appearance inspection apparatus 1 includes a transport device 2, an illumination device 3, a camera 4, an image processing device 5, an illumination control device 6, an output device 7, and an input device 8. Composed.

  The transport device 2 is a device configured to be able to transport the inspection target T placed thereon, and plays a role of transporting the inspection target T so as to pass under the camera 4. The operation of the conveying device 2 is controlled by the image processing device 5. The inspection target T is an electronic component such as an IC.

  The illuminating device 3 is an illuminating device for illuminating the inspection target T on the transport device 2, and is arranged so that the inspection target T can be illuminated from an oblique direction. Moreover, the illuminating device 3 is comprised by the some partial illumination which can be lighted independently, and is comprised so that a some optical condition can be formed by which partial illumination is lighted.

  FIG. 2A is a top view showing an example of the arrangement of the illumination device 3, and FIG. 2B is a side view corresponding to FIG. The illuminating device 3 by this example is comprised including four partial illumination 3a-3d. The inspection target T to be illuminated by the illuminating device 3 has, for example, a quadrangular pyramid shape as illustrated, and the partial illuminations 3a to 3d each have four different side surfaces of the inspection target T, It arrange | positions in the position which can be illuminated from each diagonal upper direction.

  FIG. 3 is a diagram illustrating another example of the arrangement of the illumination device 3. The illumination device 3 according to this example includes five partial illuminations 3 a to 3 e and a half mirror 9. 3A shows a state in which the partial illumination 3c is turned on, and FIG. 3B shows a state in which the partial illumination 3b is turned on.

  The partial illuminations 3a to 3e are arranged side by side at equal intervals in the vertical direction, obliquely above the inspection target T. The irradiation directions of the partial illuminations 3 a to 3 e are parallel to the conveyance direction of the inspection target T by the conveyance device 2. The half mirror 9 is a mirror disposed above the inspection target T and plays a role of reflecting the light irradiated from each of the partial illuminations 3a to 3e and illuminating the inspection target T from above. Since the partial illuminations 3a to 3e are arranged in the vertical direction, the light emitted from each of the partial illuminations 3a to 3e is centered as understood from, for example, comparing FIG. 3 (a) and FIG. 3 (b). The inspection object T is irradiated with the points being offset little by little.

  Returning to FIG. The camera 4 is an imaging unit that is arranged, for example, directly above the inspection target T and images the inspection target T. The camera 4 may be a line sensor camera or an area camera. The camera 4 is configured to perform a photographing operation according to the control of the image processing device 5 and output an image obtained as a result of the operation to the image processing device 5.

  The image processing apparatus 5 is a personal computer having a CPU and a storage device therein, for example, and is configured to execute each operation described below when the CPU executes a program stored in the storage device. The output device 7 and the input device 8 are man-machine interfaces normally provided in a personal computer, and the illumination control device 6 is a control device that controls the lighting state of the illumination device 3 in accordance with instructions from the image processing device 5. Specific examples of the output device 7 include a display and a speaker. Specific examples of the input device 8 include a keyboard and a mouse.

  The image processing apparatus 5 temporarily stops the inspection target T at the timing when the inspection target T enters the imaging range of the camera 4. While the inspection target T is stopped, there is one partial illumination constituting the illumination device 3. The transport device 2, the illumination control device 6, and the camera 4 are controlled so that the light is sequentially turned on one by one and the photographing by the camera 4 is executed each time. With this control, for example, if the lighting device 3 has the configuration shown in FIG. 2, four images captured when each of the partial lightings 3 a to 3 d are turned on are sequentially taken from the camera 4. It is supplied to the processing device 5. Hereinafter, the plurality of images thus supplied in time series are referred to as “time series images”.

  FIG. 4 is a diagram illustrating an image (bright field image) obtained by imaging the inspection target T having a dent defect in a state where all of the partial illuminations 3a to 3e illustrated in FIG. 3 are turned on. 5A to 5E show images obtained by capturing the same inspection target T as in FIG. 4 in a state where each of the partial illuminations 3a to 3e shown in FIG. FIG.

  As can be understood by comparing FIG. 4 and FIG. 5, a dent defect that cannot be seen at all when the partial lights 3 a to 3 e are turned on becomes visible when only one of the partial lights 3 a to 3 e is turned on. There is a case. For example, in the example of FIG. 5, the dent defect is most noticeably observed in FIG. 5 (d) showing the state where the partial illumination 3 d is turned on, and the dent defect is also present at the same position in the other FIGS. Observed. As can be understood from this result, in some cases, a dent defect cannot be observed by using a method of irradiating an object uniformly from all directions, such as a general bright field illumination method or dark field illumination method. The same applies to blister defects and crack defects. In order to visualize these defects, it is necessary to irradiate light from a specific direction suitable for the defects.

  The appearance inspection apparatus 1 according to the present embodiment is configured to take an image of the inspection object T by sequentially turning on the partial illuminations 3a to 3e one by one in view of such properties of the blister / dent and crack defects. It is a thing. The image processing device 5 is configured to convert a time-series image obtained as a result of the photographing into six types of probabilistic distance images, and determine the presence or absence of a defect based on the result.

  Hereinafter, processing performed by the image processing device 5 to obtain six types of probabilistic distance images will be described in detail with reference to functional blocks of the image processing device 5. Since the processing is slightly different between the blister / dent defect and the crack defect, detection of the blister / dent defect will be described first, and then detection of the crack defect will be described.

  FIG. 6 is a diagram illustrating a part related to generation of model information for detecting a blister / dent defect among the functional blocks of the image processing apparatus 5. FIG. 7 is a diagram showing a part related to image processing (including comparison processing with model information) of the inspection object T for detecting a blister / dent defect in the functional blocks of the image processing apparatus 5.

  As shown in FIGS. 6 and 7, the image processing apparatus 5 detects time-series image generation units 10a and 10c, an average time-series image generation unit 10b, a time-series normalized image generation unit 11a, 11c, average time-series normalized image generation unit 11b, correlation distance image generation units 12a and 12b, probabilistic behavior description units 13a to 13c, probabilistic distance image generation units 14a to 14c, time direction average image generation units 20a and 20b, The time direction standard deviation image generation units 21 a and 21 b and the model information storage unit 50 are configured.

  First, referring to FIG. 7, the time-series image generation unit 10a obtains a time-series image V (x, y, t) (first) obtained by photographing the inspection object T under a plurality of different optical conditions. Is a functional unit (first time-series image generation unit). The plurality of different optical conditions here means, for example, a state in which different ones of the four partial illuminations 3a to 3d shown in FIG. 2 (or the five partial illuminations 3a to 3e shown in FIG. 3) are lit. This is realized by the illumination control device 6 shown in FIG. Further, x and y in parentheses of the time-series image V (x, y, t) indicate coordinates in the image, and t is a serial number of T partial illuminations used for photographing (t = 0 to T−1). ). These points are the same in the following description.

The time-series normalized image generation unit 11a connects the time-series image V (x, y, t) to the normalized time-series image V _E (x, y, t) formed by connecting the normalization suppression constant Th and its reciprocal -Th. ) (T = 0 to T + 1, based on the first extended time-series image), the time-series normalized image I (x, y, t) (the first time-series image V (x, y, t)) 1 is a functional unit (first time-series normalized image generating unit) that generates one time-series normalized image). Hereinafter, the processing of the time-series normalized image generation unit 11a will be described in detail.

First, in the extended time-series image V _E (x, y, t), specifically, the normalized suppression constant Th is added to the average value in the time direction of the time-series image V (x, y, t) for each pixel. The normalized suppression constant Th is calculated from the connected image V _E (x, y, T) (first connected image) and the time-series average value of the time-series image V (x, y, t) for each pixel. This is an image formed by connecting the subtracted connection image V _E (x, y, T + 1) (second connection image) to the time-series image V (x, y, t). When the connected images V _E (x, y, T) and V _E (x, y, T + 1) are written by mathematical formulas, the formula (1) is obtained. However, V _avr (x, y) in Expression (1) is an average value in the time direction of the time-series image V (x, y, t), and is expressed by Expression (2).

The time-series normalized image generation unit 11a generates a time-series normalized image I (x, y, t) based on the average value and standard deviation in the time direction of the extended time-series image V _E (x, y, t). Generate. Specifically, a time-series normalized image I (x, y, t) is generated by the following equation (3). However, V _Eavr (x, y) in Expression (3) is an average value in the time direction of the extended time-series image V _E (x, y, t), and is expressed as Expression (4). In addition, V _Estddev (x, y) in Expression (3) is a standard deviation in the time direction of the extended time-series image V _E (x, y, t), and is expressed as Expression (5). In Equation (3), the denominator on the right side is multiplied by the square root of T + 2. The time-series normalized images [I (x, y, 0), I (x, y, 1),. .., I (x, y, T + 1)] is set to 1.

Equation (3) can be rewritten as the following equation (6). However, Nv (x, y) in equation (6) is a divisor for normalization and is represented by equation (7). When the time-series normalized image I (x, y, t) is generated using Expression (6), it is not necessary to generate the extended time-series image V _E (x, y, t) as an intermediate product. The time-series normalized image generation unit 11a actually generates the time-series normalized image I (x, y, t) using Expression (6).

  When the time-series normalized image generation unit 11a performs the normalization as described above, in the time-series normalized image I (x, y, t), the presence of a dent / blowing defect regardless of the reflection intensity of the inspection target. Can be determined correctly. Hereinafter, this point will be described in detail.

  If the time-series normalized image generation unit 11a normalizes the time-series image V (x, y, t) by a normal method not based on the method of the present invention, the time-series normalized image I (x , Y, t) is calculated as the following equation (8).

  The time-series normalized image I (x, y, t) calculated by Expression (8) has a problem that only camera noise may be emphasized at a specific pixel. That is, if there is a pixel (for example, a pixel corresponding to a portion having a deep hole) that does not cause a significant variation in light and shade even when the optical condition is changed, a time-series image V (x, y, The standard deviation of t) is a very small value. Then, in the time-series normalized image I (x, y, t), a slight shading variation is extremely emphasized. Since this slight shading variation is often caused by camera noise, in the time-series normalized image I (x, y, t) calculated by Equation (8), false information due to camera noise occurs. Is more likely to do. In addition, when there is a hole in the inspection target that is not in a normal product, it is difficult to correctly detect the hole as a defect.

On the other hand, in the above-described extended time-series image V _E (x, y, t), as can be understood from the equation (1), a shade variation of at least 2 Th width is secured. As a result, extended time-series image _{V E (x, y, t} ) the standard deviation _{V Estddev} (x, y) of the so it is no longer becomes extremely small value, extended time-series image _V E (x, y, In the time-series normalized image I (x, y, t) generated by the expression (3) or the expression (6) based on t), the possibility that a slight shading variation is extremely emphasized is eliminated. The Therefore, according to the normalization method of the present invention, in the time-series normalized image I (x, y, t), it is possible to prevent the occurrence of false information due to camera noise and to detect a hole that is not in a normal product. It can be said that the hole can be correctly detected as a defect when it is in the object.

  FIG. 8A is a diagram illustrating an example of a time series image V (x, y, t) of the inspection target T having a dent defect, and FIG. 8B is a time series of FIG. 8A. It is a figure which shows the time series normalization image I (x, y, t) produced | generated based on the image V (x, y, t). As shown in these figures, in the time-series normalized image I (x, y, t), a dent defect that is emphasized in the time-series image V (x, y, t) by using oblique illumination. Is further emphasized. On the other hand, in the time-series normalized image I (x, y, t), compared to the time-series image V (x, y, t), variations in light and shade due to individual differences in the inspection target T and the illumination device 3 are suppressed. The

Turning to FIG. 6, the time-series image generation unit 10 c has a plurality of different optical conditions (however, the same one used in the generation of the time-series image V (x, y, t) by the time-series image generation unit 10 a). S time-series images _s V (x, y, t) (third time series) obtained by repeating the photographing of normal products under S for S times from s = 0 to s = S−1. (Function) (third time-series image generation unit). Note that the lowercase letter s attached to the head of the code of the time-series image _s V (x, y, t) represents the number of repetitions. The maximum number S of repetitions may be an integer of 1 or more.

The normal product is a component having the same shape as the inspection object T, but it is confirmed in advance by other means such as visual observation that there are no defects (particularly blister / dent and crack defects). Therefore, none of the S time-series images _s V (x, y, t) includes a defect. Note that the time-series image generation unit 10c may generate S time-series images _s V (x, y, t) by repeatedly photographing the same normal product, or one or a plurality of different normal products. S time-series images _s V (x, y, t) may be generated by capturing images one by one.

For each of the S time-series images _s V (x, y, t), the time-series normalized image generation unit 11c adds the normalization suppression constant Th and its time series image _s V (x, y, t) Based on the extended time-series image _s V _E (x, y, t) (t = 0 to T + 1, third extended time-series image) formed by concatenating the reciprocal number −Th, the time-series image _s V (x, Function unit (third time-series normalized image generation unit) that generates time-series normalized image _s I (x, y, t) (third time-series normalized image) obtained by normalizing y, t) It is. The extended time-series image _s V _E (x, y, t) is a connected image obtained by adding a normalization suppression constant Th to the average value in the time direction of the time-series image _s V (x, y, t) for each pixel. _The normalization suppression constant Th is subtracted from the average value in the time direction of _s V _E (x, y, T) (fifth connected image) and the time-series image _s V (x, y, t) for each pixel. A connected image _s V _E (x, y, T + 1) (sixth connected image) to be connected to a time-series image _s V (x, y, t), and a time-series normalized image generation unit 11c, for each of the S time-series images _s V (x, y, t), based on the average value and standard deviation in the time direction of the corresponding extended time-series image _s V _E (x, y, t). Then, a time-series normalized image _sI (x, y, t) is generated.

Since the normalization method by the time series normalized image generation unit 11c is basically the same as that by the time series normalized image generation unit 11a, detailed description thereof is omitted. The time-series normalized image generation method corresponding to Expression (6) is represented by the following Expression (9) in the time-series normalized image generation unit 11c. However, _s V _avr (x, y) in Equation (9) is an average value in the time direction of the time-series image _s V (x, y, t), and is represented by Equation (10). Further, _s Nv (x, y) in the equation (9) is a divisor for normalization and is represented by the equation (11). By adopting such a normalization method, it is possible to prevent generation of false information due to camera noise even in S time-series normalized images _sI (x, y, t).

The average time-series image generation unit 10b generates an average time-series image aV (x, y, t) (an image composed of average values for each pixel of the S time-series images _s V (x, y, t) ( It is a functional unit (second time-series image generation unit) that generates a second time-series image. Specifically, the average time series image aV (x, y, t) is generated by the following equation (12).

The average time-series normalized image generation unit 11b connects the average time-series image aV (x, y, t) to the normalization suppression constant Th and its reciprocal -Th to expand the time-series image aV _E (x, y , T) (t = 0 to T + 1, the second extended time-series image), the average time-series normalized image aI (x, y, t) obtained by normalizing the average time-series image aV (x, y, t). t) A functional unit (second time-series normalized image generation unit) that generates (second time-series normalized image). The expanded time series image aV _E (x, y, t) is a connected image aV formed by adding a normalization suppression constant Th to the average value in the time direction of the average time series image aV (x, y, t) for each pixel. _E (x, y, T) (third connected image) and a connected image obtained by subtracting the normalization suppression constant Th from the average value in the time direction of the time-series image aV (x, y, t) for each pixel. aV _E (x, y, T + 1) (fourth connected image) and an average time-series image aV (x, y, t), and the average time-series normalized image generation unit 11b An average time series normalized image aI (x, y, t) is generated based on the average value and standard deviation in the time direction of the extended time series image aV _E (x, y, t).

Since the normalization method by the average time-series normalized image generation unit 11b is basically the same as that by the time-series normalized image generation unit 11a, detailed description is omitted. The time-series normalized image generation method corresponding to Expression (6) is expressed by the following Expression (13) in the average time-series normalized image generation unit 11b. However, aV _avr (x, y) in Expression (13) is an average value in the time direction of the time-series image aV (x, y, t), and is expressed by Expression (14). Further, aNv (x, y) in equation (13) is a normalization divisor and is represented by equation (15). By adopting such a normalization method, it is possible to prevent generation of false information due to camera noise even in the average time-series normalized image aI (x, y, t).

  The average time series normalized image aI (x, y, t) generated by the average time series normalized image generation unit 11b is stored in the model information storage unit 50 as a part of model information representing a normal product.

Returning to FIG. 7, the correlation distance image generation unit 12 a performs the correlation distance for each pixel between the time-series normalized image I (x, y, t) and the average time-series normalized image aI (x, y, t). _Is a functional unit (first correlation distance image generation unit) that generates a correlation distance image I _cd (x, y) (first correlation distance image). Specifically, the correlation distance image I _cd (x, y) is generated by the following equation (16). As understood from the equation (16), the image processing apparatus 5 calculates the correlation distance between two images using the inner product calculation.

Returning to FIG. 6, the correlation distance image generation unit 12b calculates the average time-series normalized image aI (x, y, t) for each of the S time-series normalized images _s I (x, y, t). a correlation distance image _{s I} cd showing the correlation distance for each pixel between _(x, y) function unit for generating a (second correlation distance image) (second correlation distance image generation unit). Specifically, the correlation distance image _s I _cd (x, y) is generated by the following equation (17).

The probabilistic behavior description unit 13a includes an average image F _avr1 (x, y) configured by an average value for each pixel of the S correlation distance images _s I _cd (x, y) generated by the correlation distance image generation unit 12b. ) (First average image) and the standard deviation image F _stddev1 (x, y) (the first average image) composed of the standard deviation for each pixel of the S correlation distance images _s I _cd (x, y) It is a functional part (first probabilistic behavior description part) that generates a standard deviation image. Specifically, in the following formulas (18) and (19), _s F (x, y), F _avr (x, y), and F _stddev (x, y) are respectively represented as correlation distance images _s I _cd (x , Y), the average image F _avr1 (x, y), and the standard deviation image F _stddev1 (x, y) are replaced with the average image F _avr1 (x, y) and the standard deviation image F _stddev1 (x, y). Generate. The generated average image F _avr1 (x, y) and standard deviation image F _stddev1 (x, y) are stored in the model information storage unit 50 as part of model information representing a normal product.

Returning to FIG. 7, the stochastic distance image generation unit 14 a, the correlation distance image I _cd (x, y), the average image F _avr1 (x, y) and the standard deviation image F _stddev1 stored in the model information storage unit 50. Based on (x, y), a stochastic distance image SFD ₁ indicating a stochastic distance for each pixel between the correlation distance image I _cd (x, y) and the S correlation distance images _s I _cd (x, y). It is a function part (1st stochastic distance image generation part) which produces | generates (x, y) (1st stochastic distance image). Specifically, in the following formulas (20) and (21), F (x, y), F _avr (x, y), F _stddev (x, y), and SFD (x, y) are respectively correlated distances. By _{replacing the} image I _cd (x, y), the average image F _avr1 (x, y), the standard deviation image F _stddev1 (x, y), and the stochastic distance image SFD ₁ (x, y), the stochastic distance image SFD ₁ (x, y) is generated. However, the function max (A, B) in the equation (20) is a function that outputs the larger of the variable A and the variable B. Amp is a given coefficient.

Next, the time direction average image generation unit 20a displays the time direction average image I _avr (x, y) indicating the time direction average of the time series image V (x, y, t) generated by the time series image generation unit 10a. ) (First time direction average image generation unit) for generating (first time direction average image generation unit). Specifically, the time direction average image I _avr (x, y) is calculated by the following equation (22). The time direction average image I _avr (x, y) is the time direction average value V _avr (x, y) of the time series image V (x, y, t) calculated by the above-described equation (2). Since they are the same, this average value V _avr (x, y) may be used as the time direction average image I _avr (x, y).

Returning to FIG. 6, the time-direction average image generation unit 20 b is a time indicating an average in the time direction for each of the S time-series images _s V (x, y, t) generated by the time-series image generation unit 10 c. the direction average image _s I _avr (x, y) function unit for generating a (second time direction average image) (second time direction average image generating unit). Specifically, S time direction average images _s I _avr (x, y) are calculated by the following equation (23). The time direction average image _s I _avr (x, y) is the time direction average value _s V _avr (x, y) of the time-series image _s V (x, y, t) calculated by the above-described equation (10). Since this is the same as y), the average value _s V _avr (x, y) may be used as the time direction average image _s I _avr (x, y).

The probabilistic behavior description unit 13b is configured to generate an average image F _avr2 (x that is configured by an average value for each pixel of the S time direction average images _s I _avr (x, y) generated by the time direction average image generation unit 20b. , Y) (second average image) and the standard deviation image F _stddev2 (x, y) (the standard deviation for each pixel of the S time-direction average images _s I _avr (x, y)) It is a functional part (second stochastic behavior description part) that generates a second standard deviation image). Specifically, _s F (x, y), F _avr (x, y), and F _stddev (x, y) in the above-described equations (18) and (19) are respectively calculated in the time direction average image _s I _avr ( x, y), the average image F _avr2 (x, y), and the standard deviation image F _stddev2 (x, y) to replace the average image F _avr2 (x, y) and the standard deviation image F _stddev2 (x, y) Is generated. The generated average image F _avr2 (x, y) and standard deviation image F _stddev2 (x, y) are stored in the model information storage unit 50 as part of model information representing a normal product.

Returning to FIG. 7, the stochastic distance image generation unit 14 b includes the time direction average image I _avr (x, y), the average image F _avr2 (x, y) stored in the model information storage unit 50, and the standard deviation image F. _stddev2 (x, y) based on the probability distance indicating the probabilistic distance each pixel of the time direction average image _I avr (x, y) and the S time direction average image _s I _avr (x, y) It is a functional unit (second probabilistic distance image generation unit) that generates an image SFD ₂ (x, y) (second probabilistic distance image). Specifically, in the following formulas (24) and (25), F (x, y), F _avr (x, y), F _stddev (x, y), and SFD (x, y) are respectively set in the time direction. By _{replacing the} average image I _avr (x, y), the average image F _avr2 (x, y), the standard deviation image F _stddev2 (x, y), and the stochastic distance image SFD ₂ (x, y), the stochastic distance An image SFD ₂ (x, y) is generated. However, the function abs (A) in the expression (25) is a function that outputs the absolute value of the variable A.

Next, the time-direction standard deviation image generation unit 21a displays a time-direction standard deviation image I _stddev () indicating the time-direction standard deviation of the time-series image V (x, y, t) generated by the time-series image generation unit 10a. x, y) is a functional unit (first time direction standard deviation image generation unit) that generates (first time direction standard deviation image). Specifically, the time direction standard deviation image I _stddev (x, y) is calculated by the following equation (26).

Returning to FIG. 6, the time-direction standard deviation image generation unit 21b calculates the time-direction standard deviation for each of the S time-series images _s V (x, y, t) generated by the time-series image generation unit 10c. the time direction standard deviation image _s I _stddev shown (x, y) function unit for generating a (second time direction standard deviation image) (second temporal standard deviation image generation unit). Specifically, S time-direction standard deviation images _s I _stddev (x, y) are calculated by the following equation (27).

The probabilistic behavior description unit 13c is an average image F _avr3 configured by an average value for each pixel of the S time direction standard deviation images _s I _stddev (x, y) generated by the time direction standard deviation image generation unit 21b. (X, y) (third average image) and the standard deviation image F _stddev3 (x, y) composed of the S-time standard deviation images _s I _stddev (x, y) for each pixel. y) A functional unit (third stochastic behavior description unit) that generates (third standard deviation image). Specifically, _s F (x, y), F _avr (x, y), and F _stddev (x, y) in the above-described equations (18) and (19) are respectively time direction standard deviation images _s I _stddev. (X, y), the average image F _avr3 (x, y), and the standard deviation image F _stddev3 (x, y) are replaced with the average image F _avr3 (x, y) and the standard deviation image F _stddev3 (x, y). ) Is generated. The generated average image F _avr3 (x, y) and standard deviation image F _stddev3 (x, y) are stored in the model information storage unit 50 as part of model information representing a normal product.

Returning to FIG. 7, the stochastic distance image generation unit 14 c includes the time-direction standard deviation image I _stddev (x, y), the average image F _avr3 (x, y) and the standard deviation image stored in the model information storage unit 50. Based on F _stddev3 (x, y), indicates the stochastic distance for each pixel between the time-direction standard deviation image I _stddev (x, y) and the S time-direction standard deviation images _s I _stddev (x, y). stochastic distance image SFD 3 _(x, y) (third probabilistic distance image) function unit for generating a (third probabilistic distance image generation unit). Specifically, F (x, y), F _avr (x, y), F _stddev (x, y), and SFD (x, y) in the time direction in the above-described equations (24) and (25), respectively. By _{replacing the} standard deviation image I _stddev (x, y), the average image F _avr3 (x, y), the standard deviation image F _stddev3 (x, y), and the stochastic distance image SFD ₃ (x, y) A distance image SFD ₃ (x, y) is generated.

The image processing device 5 determines the inspection object T based on any one or combination of the three types of probabilistic distance images SFD ₁ (x, y) to SFD ₃ (x, y) calculated as described above. A determination is made as to whether there are blister / dent defects on the surface. Specifically, at least one of known threshold processing and blob analysis is performed on each of the three types of probabilistic distance images SFD ₁ (x, y) to SFD ₃ (x, y), and based on the results. Determine if there are blister / dent defects.

9 (a) is a diagram showing an example of a case of the normal product series image _{S V (x, y, t} ), FIG. 9 (b), the time series of the test object T with a dent defect image V It is a figure which shows an example of (x, y, t). Further, FIG. 9 (c), generated based on the time-series images _{S V (x, y, t} ) and time-series images _T V in FIG. 9 (b) (x, y , t) shown in FIG. 9 (a) is a diagram illustrating the probabilistic range image _SFD 1 (x, y), FIG. 9 (d) when shown in FIG. 9 (a) series image _{S V (x, y, t} ) and FIG. 9 (b) FIG. 9C is a diagram showing a probabilistic distance image SFD ₂ (x, y) generated based on the time series image V (x, y, t) of FIG. 9, and FIG. 9C is a time series of FIG. 9A. is a diagram illustrating an image _{S V (x, y, t} ) and 9 time-series images V in (b) (x, y, t) the probability is generated based on the distance image _SFD 3 (x, y) . In addition, each stochastic distance image of FIG. 9 is generated by setting the number of times of capturing a normal product by the time-series image generation unit 10c as 1 (that is, S = 1).

In the example of FIG. 9, the dent defect appearing in the two images V (x, y, 2) and V (x, y, 3) is reflected as a white area in the stochastic distance image SFD ₁ (x, y). ing. The image processing device 5 detects this white area by a known threshold processing or blob analysis, and detects a dent defect present on the surface of the inspection object T based on the result. In this example, no defect appears in the stochastic distance images SFD ₂ (x, y) and SFD ₃ (x, y), but the image processing device 5 has three types of probabilistic distance images SFD ₁ (x, y). Since y) to SFD ₃ (x, y) are used, a defect appears in the stochastic distance image SFD ₁ (x, y), which is one of them, so that a dent defect can be detected normally. Is done. Which of the probabilistic distance images SFD ₁ (x, y) to SFD ₃ (x, y) appears depends on the content of the defect, and thus the three types of probabilistic distance images SFD ₁ (x, y) to SFD _{By using 3} (x, y), it is possible to improve the detection probability. Further, by using the stochastic distance image SFD ₂ (x, y) generated based on the average image in the time direction, not only blisters / dents but also general defects such as adhesion of dirt and foreign matters are combined. It becomes possible to detect.

  Next, detection of crack defects will be described.

  FIG. 10 is a diagram illustrating a part related to generation of model information for detecting a crack defect in the functional blocks of the image processing apparatus 5. Moreover, FIG. 11 is a figure which shows the part regarding the image process (a comparison process with model information) of the test target T for detecting a crack defect among the functional blocks of the image processing apparatus 5.

  As shown in FIGS. 10 and 11, the image processing device 5 detects time-series image generation units 10a and 10c, average time-series image generation unit 10b, time-series differential image generation units 31a and 31c, and averages. Time-series differential image generator 31b, time-series gradient vector image generators 32a and 32c, average time-series gradient vector image generator 32b, time-series normalized gradient vector image generators 33a and 33c, average time-series normalized gradient vector image Generation unit 33b, gradient correlation distance image generation units 34a and 34b, probabilistic behavior description units 35a to 35c, probabilistic distance image generation units 36a to 36c, time direction average gradient vector image generation units 40a and 40b, time direction gradient norm image The generation unit 41a, 41b and the model information storage unit 50 are configured. Hereinafter, the explanation will be advanced while omitting the same points as the detection of the blister / dent defect as appropriate.

  First, referring to FIG. 11, the time-series differential image generation unit 31a generates a time-series image V (x, y, t) (fourth time-series image) generated by the time-series image generation unit 10a in the x direction ( Differentiating each of the first direction) and the y direction (second direction) intersecting (orthogonal) with the x direction, two types of time-series differential images Gx (x, y, t), Gy (x, y, t) is a functional unit (first time-series differential image generation unit) that generates (first and second time-series differential images). Specifically, for example, the time-series differential images Gx (x, y, t) and Gy (x, y, t) are generated by calculating the difference between two pixels adjacent in the differential direction. That's fine.

The time-series gradient vector image generation unit 32a averages the time-series differential images Gx (x, y, t) from each of a plurality of differential images constituting the time-series differential images Gx (x, y, t). The value Gx _avr (x, y) is subtracted, and the time-series differential image Gy (x, y, t) is further _extracted from each of a plurality of differential images constituting the time-series differential image Gy (x, y, t). A time-series gradient vector image G (x, y, u) (first time-series gradient vector image = first time-series image) is generated by subtracting the average value Gy _avr (x, y) in the time direction. Functional unit (first time-series gradient vector image generation unit). Here, u is an integer of 0 to 2T-1. The time series gradient vector image G (x, y, u) is specifically expressed by the following equation (28). In addition, the average values Gx _avr (x, y) and Gy _avr (x, y) are respectively expressed as Equation (29) and Equation (30).

Moving to FIG. 10, the time-series differential image generation unit 31c converts each of the S time-series images _s V (x, y, t) generated by the time-series image generation unit 10c in the x direction and the y direction, respectively. It is a functional unit that generates two types of time-series differential images _s Gx (x, y, t) and _s Gy (x, y, t) by differentiating each. Specifically, for example, by calculating the difference between two pixels adjacent in the differential direction, the time-series differential images _s Gx (x, y, t) and _s Gy (x, y, t) are generated. And it is sufficient.

The time-series gradient vector image generation unit 32c generates a time-series differential image _s Gx (x, y, t) from each of a plurality of differential images constituting the time-series differential image _s Gx (x, y, t). mean _s Gx _avr (x, y) of the subtracted, further, the time-series differential image _{s Gy (x, y, t} ) time series differential image from each of the plurality of differential images constituting the _s Gy (x, This is a functional unit that generates a time-series gradient vector image _s G (x, y, u) by subtracting the average value _s Gy _avr (x, y) in the time direction of y, t). The time series gradient vector image _s G (x, y, u) is specifically expressed by the following equation (31). In addition, the average values _s Gx _avr (x, y) and _s Gy _avr (x, y) are respectively expressed as Equation (32) and Equation (33).

  The average time-series differential image generation unit 31b generates the average time-series image aV (x, y, t) (fifth time-series image) generated by the average time-series image generation unit 10b in each of the x direction and the y direction. A functional unit (second second) for generating two types of time-series differential images aGx (x, y, t) and aGy (x, y, t) (third and fourth time-series differential images) by differentiating. A time-series differential image generation unit). Specifically, the time-series differential images aGx (x, y, t) and aGy (x, y, t) are generated by calculating the difference between two pixels adjacent in the differential direction, for example. That's fine.

The average time-series gradient vector image generation unit 32b generates a time-series differential image aGx (x, y, t) in the time direction from each of a plurality of differential images constituting the time-series differential image aGx (x, y, t). The average value aGx _avr (x, y) is subtracted, and the time-series differential image aGy (x, y, t) is further obtained from each of the plurality of differential images constituting the time-series differential image aGy (x, y, t). Are subtracted from the average value aGy _avr (x, y) in the time direction of the average time-series gradient vector image aG (x, y, u) (second time-series gradient vector image = second time-series image) Is a functional unit (second time-series gradient vector image generation unit). The average time series gradient vector image aG (x, y, u) is specifically expressed by the following equation (34). Further, the average values aGx _avr (x, y) and aGy _avr (x, y) are respectively expressed as Expression (35) and Expression (36).

Returning to FIG. 11, the time-series normalized gradient vector image generation unit 33a connects the time-series gradient vector image G (x, y, u) to the normalization suppression constant Th and its reciprocal number −Th. Time series normalization obtained by normalizing the time series gradient vector image G (x, y, u) based on the image G _E (x, y, u) (u = 0 to 2T + 1, first extended time series image). It is a function part (1st time series normalization image generation part) which produces | generates gradient vector image J (x, y, u) (1st time series normalization image).

Specifically, the extended time-series image G _E (x, y, u) is a connected image G _E (x, y, 2T) (seventh connected image) in which the density of each pixel is a normalized suppression constant Th. And a connected image G _E (x, y, 2T + 1) (eighth connected image) in which the density of each pixel is a reciprocal -Th of the normalization suppression constant is used as a time-series gradient vector image G (x, y, u). ). When the concatenated images G _E (x, y, 2T) and G _E (x, y, 2T + 1) are written by mathematical formulas, the formula (37) is obtained.

The time-series normalized gradient vector image generation unit 33a generates a time-series normalized gradient vector image J (x, y) based on the length of the extended time-series image G _E (x, y, u) as a vector in the time direction. , U). Specifically, the time series normalized gradient vector image J (x, y, u) is generated by the following equation (38). In Expression (38), the denominator on the right side corresponds to the length of the extended time-series image G _E (x, y, u) as a vector in the time direction.

Expression (38) can be rewritten as the following Expression (39). However, Ng (x, y) in the equation (39) is a normalization divisor and is expressed by the equation (40). When generating the time-series normalized gradient vector image J (x, y, u) using the equation (39), it is unnecessary to generate the extended time-series image G _E (x, y, u) as an intermediate product. Therefore, the time-series normalized gradient vector image generation unit 33a actually generates the time-series normalized gradient vector image J (x, y, u) using Expression (39).

  The time-series normalized gradient vector image generation unit 33a performs the above normalization, thereby preventing the occurrence of false information due to camera noise in the time-series normalized gradient vector image J (x, y, u). It becomes possible. In particular, in a time-series normalized gradient vector image, differential processing is performed prior to normalization, and therefore, other than a portion where the brightness changes greatly, such as a boundary portion between a character and a background in the image shown in FIG. 12 described below. This part (bright and dark flat part) is in a state where only camera noise remains. Therefore, since the influence of the false information due to the camera noise on the inspection is further increased, the suppression of the false information by performing the above normalization becomes more important.

  FIG. 12A is a diagram illustrating an example of a time series image V (x, y, t) of the inspection target T having a crack defect, and FIG. 12B is a time series of FIG. It is a figure which shows the time series normalization gradient vector image J (x, y, u) produced | generated based on the image V (x, y, t). As shown in these figures, the time-series normalized gradient vector image J (x, y, u) is emphasized in the time-series image V (x, y, t) by using oblique illumination. Crack defects are further emphasized. On the other hand, in the time-series normalized gradient vector image J (x, y, u), as compared with the time-series image V (x, y, t), the light and shade fluctuation due to the individual difference of the inspection target T and the illumination device 3 occurs. It is suppressed.

Returning to FIG. 10, the time-series normalized gradient vector image generating unit 33c, S number of time-series gradient vector image _{s G (x, y, u} ) for each, time series gradient vector image _s G (x, y , extended time-series images obtained by connecting the normalized inhibition constants Th and its inverse number -Th to _{u) s G E (x,} y, u) (u = 0~2T + 1) based on the time-series gradient vector image _s This is a functional unit that generates a time-series normalized gradient vector image _s J (x, y, u) obtained by normalizing G (x, y, u). The extended time-series image _s G _E (x, y, u) includes a connected image _s G _E (x, y, 2T) in which the density of each pixel is a normalization suppression constant Th, and the density of each pixel is normalized and suppressed. A connected image _s G _E (x, y, 2T + 1) that is a constant reciprocal -Th is connected to a time-series gradient vector image _s G (x, y, u), and is a time-series normalized gradient. The vector image generation unit 33c calculates the length of each of the S time-series gradient vector images _s G (x, y, u) as a vector in the time direction of the extended time-series image _s G _E (x, y, u). Based on this, a time-series normalized gradient vector image _s J (x, y, u) is generated.

Since the normalization method by the time series normalized gradient vector image generation unit 33c is basically the same as that by the time series normalization gradient vector image generation unit 33a, detailed description thereof is omitted. A time-series normalized image generation method corresponding to Expression (39) is expressed by the following Expression (41) in the time-series normalized gradient vector image generation unit 33c. However, _s Ng (x, y) in the equation (41) is a divisor for normalization and is represented by the equation (42). By adopting such a normalization method, it is possible to prevent generation of false information due to camera noise even in S time-series normalized gradient vector images _s J (x, y, u). Become.

Mean time series normalized gradient vector image generating unit 33b, an average time series gradient vector image aG (x, y, u) formed by connecting the normalized inhibition constants Th and its inverse number -Th in the extended time-series images aG _E Average time-series normalized gradient obtained by normalizing the average time-series gradient vector image aG (x, y, u) based on (x, y, u) (u = 0 to 2T + 1, second extended time-series image). It is a function part (2nd time series normalization image generation part) which produces | generates vector image aJ (x, y, u) (2nd time series normalization image). The average extended time series image aG _E (x, y, u) includes a connected image aG _E (x, y, 2T) (the ninth connected image) in which the density of each pixel is a normalized suppression constant Th, and each pixel. The connected image aG _E (x, y, 2T + 1) (tenth connected image) whose density is the reciprocal -Th of the normalized suppression constant is connected to the average time series gradient vector image aG (x, y, u) The average time-series normalization gradient vector image generation unit 33b generates an average time-series normalization based on the length of the extended time-series image aG _E (x, y, u) as a vector in the time direction. A normalized gradient vector image aJ (x, y, u) is generated.

  Since the normalization method by the average time series normalized gradient vector image generation unit 33b is basically the same as that by the time series normalization gradient vector image generation unit 33a, detailed description thereof is omitted. The time-series normalized image generation method corresponding to Expression (39) is expressed by the following Expression (43) in the average time-series normalized gradient vector image generation unit 33b. However, aNg (x, y) in equation (43) is a divisor for normalization and is represented by equation (44). By employing such a normalization method, it is possible to prevent generation of false information due to camera noise even in the average time-series normalized gradient vector image aJ (x, y, u).

Returning to FIG. 11, the gradient correlation distance image generation unit 34 a performs a time series normalized gradient vector image J (x, y, u) and an average time series normalized gradient vector image by the same processing as the correlation distance image generation unit 12 a. A gradient correlation distance image J _cd (x, y) indicating a correlation distance for each pixel between aJ (x, y, u) is generated. Specifically, the gradient correlation distance image J _cd (x, y) is generated by the following equation (45).

Returning to FIG. 10, the gradient correlation distance image generation unit 34 b calculates the average time-series normalized gradient vector image aJ (x, y) for each of the S time-series normalized gradient vector images _s J (x, y, u). , U) is a functional unit that generates a gradient correlation distance image _s J _cd (x, y) indicating a correlation distance for each pixel. Specifically, the gradient correlation distance image _s J _cd (x, y) is generated by the following equation (46).

The probabilistic behavior description unit 35a is configured to generate an average image F _avr4 (x that is configured by an average value for each pixel of the S gradient correlation distance images _s J _cd (x, y) generated by the gradient correlation distance image generation unit 34b. , Y) and the standard deviation image F _stddev4 (x, y) composed of the standard deviation for each pixel of the S gradient correlation distance images _s J _cd (x, y). Specifically, _s F (x, y), F _avr (x, y), and F _stddev (x, y) in the above-described equations (18) and (19) are respectively represented by gradient correlation distance images _s J _cd ( x, y), the average image F _avr4 (x, y), and the standard deviation image F _stddev4 (x, y) to replace the average image F _avr4 (x, y) and the standard deviation image F _stddev4 (x, y) Is generated. The generated average image F _avr4 (x, y) and standard deviation image F _stddev4 (x, y) are stored in the model information storage unit 50 as part of model information representing a normal product.

Returning to FIG. 11, the stochastic distance image generation unit 36 a, the gradient correlation distance image J _cd (x, y), the average image F _avr4 (x, y) and the standard deviation image F stored in the model information storage unit 50. A stochastic distance indicating a stochastic distance for each pixel between the gradient correlation distance image J _cd (x, y) and the S gradient correlation distance images _s J _cd (x, y) based on _stddev4 (x, y). An image SFD ₄ (x, y) is generated. Specifically, F (x, y), F _avr (x, y), F _stddev (x, y), and SFD (x, y) in the above equations (20) and (21) are gradient correlations, respectively. By _{replacing the} distance image J _cd (x, y), the average image F _avr4 (x, y), the standard deviation image F _stddev4 (x, y), and the stochastic distance image SFD ₄ (x, y), the stochastic distance An image SFD ₄ (x, y) is generated.

Next, the time direction average gradient vector image generation unit 40a indicates the time direction average gradient indicating the time direction average of the time series gradient vector image G (x, y, u) generated by the time series gradient vector image generation unit 32a. A vector image J _avr (x, y) is generated. Specifically, the time direction average gradient vector image J _avr (x, y) is calculated by the following equation (47).

Returning to FIG. 10, the time-direction average gradient vector image generation unit 40 b calculates the time-series gradient vector image _s G (x, y, t) generated by the time-series gradient vector image generation unit 32 c for each time. A time direction average gradient vector image _s J _avr (x, y) indicating the average of directions is generated. Specifically, S time direction average gradient vector images _s J _avr (x, y) are calculated by the following equation (48).

The stochastic behavior description unit 35b is an average image configured by an average value for each pixel of the S time direction average gradient vector images _s J _avr (x, y) generated by the time direction average gradient vector image generation unit 40b. _F avr5 (x, y), and, generate a standard deviation image _{F stddev5} (x, y) formed by the standard deviation of each pixel of the S pieces of time direction average gradient vector image _s J _avr (x, y) To do. Specifically, _s F (x, y), F _avr (x, y), and F _stddev (x, y) in the above-described equations (18) and (19) are respectively calculated in the time direction average gradient vector image _s J. _{By replacing avr} (x, y), average image F _avr5 (x, y), and standard deviation image F _stddev5 (x, y), average image F _avr5 (x, y) and standard deviation image F _stddev5 (x, y) y) is generated. The generated average image F _avr5 (x, y) and standard deviation image F _stddev5 (x, y) are stored in the model information storage unit 50 as part of model information representing a normal product.

Returning to FIG. 11, the stochastic distance image generation unit 36 b includes the time direction average gradient vector image J _avr (x, y), the average image F _avr5 (x, y) stored in the model information storage unit 50, and the standard deviation. Based on the image F _stddev5 (x, y), the probability for each pixel of the time direction average gradient vector image J _avr (x, y) and the S time direction average gradient vector images _s J _avr (x, y). Probabilistic distance image SFD ₅ (x, y) indicating the distance is generated. Specifically, F (x, y), F _avr (x, y), F _stddev (x, y), and SFD (x, y) in the time direction in the above-described equations (24) and (25), respectively. By _{replacing the} average gradient vector image J _avr (x, y), the average image F _avr5 (x, y), the standard deviation image F _stddev5 (x, y), and the stochastic distance image SFD ₅ (x, y), the probability A target distance image SFD ₅ (x, y) is generated.

Next, the time direction gradient norm image generation unit 41a is a time direction average gradient indicating the length (norm) of the time series gradient vector image G (x, y, u) generated by the time series gradient vector image generation unit 32a. A norm image J _stddev (x, y) is generated. Specifically, the time direction average gradient norm image J _stddev (x, y) is calculated by the following equation (49).

Returning to FIG. 10, the time-direction gradient norm image generation unit 41 b calculates the length of each of the S time-series gradient vector images _s G (x, y, t) generated by the time-series gradient vector image generation unit 32 c. A time direction average gradient norm image _s J _stddev (x, y) _{indicating the above} is generated. Specifically, S time direction average gradient norm images _s J _stddev (x, y) are calculated by the following equation (50).

The probabilistic behavior description unit 35c is an average image F configured by an average value for each pixel of the S time-direction average gradient norm images _s J _stddev (x, y) generated by the time-direction gradient norm image generation unit 41b. _avr6 (x, y), and generates the S number of the time-direction average gradient norm image _s J _stddev (x, y) the standard deviation image _F constituted by the standard deviation of each pixel of _stddev6 (x, y) . Specifically, _s F (x, y), F _avr (x, y), and F _stddev (x, y) in the above-described equations (18) and (19) are respectively time direction average gradient norm images _s J. _{By replacing the stddev} (x, y), the average image F _avr6 (x, y), and the standard deviation image F _stddev6 (x, y), the average image F _avr6 (x, y) and the standard deviation image F _stddev6 (x, y) y) is generated. The generated average image F _avr6 (x, y) and standard deviation image F _stddev6 (x, y) are stored in the model information storage unit 50 as part of model information representing a normal product.

Returning to FIG. 11, the probabilistic distance image generation unit 36 c includes the time-direction average gradient norm image J _stddev (x, y), the average image F _avr6 (x, y) and the standard deviation stored in the model information storage unit 50. Based on the image F _stddev6 (x, y), the probability of each pixel of the time direction average gradient norm image J _stddev (x, y) and the S time direction average gradient norm images _s J _stddev (x, y). Probabilistic distance image SFD ₆ (x, y) indicating the distance is generated. Specifically, F (x, y), F _avr (x, y), F _stddev (x, y), and SFD (x, y) in the time direction in the above-described equations (24) and (25), respectively. By _{replacing the} mean gradient norm image J _stddev (x, y), the mean image F _avr6 (x, y), the standard deviation image F _stddev6 (x, y), and the stochastic distance image SFD ₆ (x, y), the probability A target distance image SFD ₆ (x, y) is generated.

Based on the three types of probabilistic distance images SFD ₄ (x, y) to SFD ₆ (x, y) calculated as described above, the image processing apparatus 5 causes crack defects on the surface of the inspection target T. It is determined whether or not there is. Specifically, at least one of known threshold processing and blob analysis is performed on each of the three types of stochastic distance images SFD ₄ (x, y) to SFD ₆ (x, y), and based on the results. The presence or absence of crack defects is determined.

13 (a) is a diagram showing an example of a case of the normal product series image _{S V (x, y, t} ), FIG. 13 (b), the time-series image V of the test object T with a crack defect (x, y, t) is a diagram showing an example of FIG. 13 (c), time-series images _S V in FIG. 13 (a) (x, y, t) and time-series images shown in FIG. 13 (b) V is a diagram showing (x, y, t) based probabilistic distance image _SFD 4 (x, y) generated by the, FIG. 13 (d) is time-series images _S V in FIG. 13 (a) ( x, y, t) and a probabilistic distance image SFD ₅ (x, y) generated based on the time-series image V (x, y, t) in FIG. 13B. c) the time-series images _S V (x in FIG. 13 (a), y, t ) and 13 time-series images V in (b) (x, y, t) stochastic distance image generated on the basis of SFD ₆ (x , Y). In addition, each stochastic distance image of FIG. 9 is generated by setting the number of times of capturing a normal product by the time-series image generation unit 10c as 1 (that is, S = 1).

In the example of FIG. 13, crack defects appearing in two images V (x, y, 2) and V (x, y, 3) are three probabilistic distance images SFD ₄ (x, y) to SFD ₆ ( x, y) are reflected as white lines. The image processing device 5 detects this white line by known threshold processing or blob analysis, and detects a crack defect present on the surface of the inspection object T based on the result. In this example, a defect appears in all of the three probabilistic distance images SFD ₄ (x, y) to SFD ₆ (x, y), but even if no defect appears in any two of the images, the image Since the processing device 5 uses three types of probabilistic distance images SFD ₄ (x, y) to SFD ₆ (x, y), a defect appears in one of them, so that a crack defect is normally detected. Can be detected. Which of the probabilistic distance images SFD ₄ (x, y) to SFD ₆ (x, y) appears depends on the content of the defect, and thus the three types of probabilistic distance images SFD ₄ (x, y) to SFD _{By using 6} (x, y), it is possible to improve the detection probability.

As described above, according to the appearance inspection apparatus 1 according to the present embodiment, the time-series images V (x, y, t), _s V (x, y, t), aV (x, y, t), Since G (x, y, u), _s G (x, y, u), and aG (x, y, u) are forcibly caused to change in shade, the same effect is obtained. Normalized images I (x, y, t), _s I (x, y, t), aI (x, y, t), J (x, y, u), _s J (x, In y, u) and aJ (x, y, u), camera noise is not greatly emphasized. Therefore, it is possible to correctly determine the presence of a dent / blowing defect or a crack defect regardless of the reflection intensity of the inspection object.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to such embodiment at all, and this invention can be implemented in various aspects in the range which does not deviate from the summary. Of course.

DESCRIPTION OF SYMBOLS 1 Appearance inspection apparatus 2 Conveyance apparatus 3 Illumination apparatus 3a-3e Partial illumination 4 Camera 5 Image processing apparatus 6 Illumination control apparatus 7 Output apparatus 8 Input apparatus 9 Half mirror 10a, 10c Time series image generation part 10b Average time series image generation part 11a , 11c Time series normalized image generation unit 11b Average time series normalized image generation unit 12a, 12b Correlation distance image generation units 13a-13c, 35a-35c Probabilistic behavior description units 14a-14c, 36a-36c Probabilistic distance image generation 20a, 20b Time direction average image generation unit 21a, 21b Time direction standard deviation image generation unit 31a, 31c Time series differential image generation unit 31b Average time series differential image generation unit 32a, 32c Time series gradient vector image generation unit 32b Average time Sequence gradient vector image generation unit 33a, 33c Time series normalized gradient vector image generation unit 33 Mean time series normalized gradient vector image generating unit 34a, 34b slope correlation distance image generation unit 40a, 40b time direction average gradient vector image generating unit 41a, 41b temporal gradient norm image generating unit 50 model information storage unit

Claims (9)

A first time-series image generating unit that generates a first time-series image obtained by photographing an inspection object under a plurality of different optical conditions;
A first time series normalization obtained by normalizing the first time series image based on a first extended time series image obtained by connecting a normalization suppression constant and its reciprocal to the first time series image. A first time-series normalized image generation unit for generating an image;
A second time-series image generating unit that generates a second time-series image obtained by photographing a normal product under the plurality of different optical conditions;
A second time series normalization obtained by normalizing the second time series image based on a second extended time series image obtained by connecting a normalization suppression constant and its reciprocal to the second time series image. A second time-series normalized image generation unit for generating an image;
A first correlation distance image generation unit configured to generate a first correlation distance image indicating a correlation distance for each pixel between the first time-series normalized image and the second time-series normalized image. An appearance inspection apparatus characterized by that. The first extended time-series image includes a first connected image obtained by adding the normalization suppression constant to an average value in the time direction of the first time-series image for each pixel, and the first for each pixel. A second connected image obtained by subtracting the normalization suppression constant from an average value in the time direction of the time series image of the first time series image,
The first time-series normalized image generation unit generates the first time-series normalized image based on an average value and a standard deviation in the time direction of the first extended time-series image,
The second extended time-series image includes a third connected image obtained by adding the normalization suppression constant to an average value in the time direction of the second time-series image for each pixel, and the second connected image for each pixel. A fourth connected image obtained by subtracting the normalization suppression constant from the average value in the time direction of the time-series image and the second time-series image,
The second time-series normalized image generation unit generates the second time-series normalized image based on an average value and a standard deviation in a time direction of the second extended time-series image. The appearance inspection apparatus according to claim 1. A third time-series image generating unit that generates a plurality of third time-series images obtained by repeating a plurality of normal image capturing operations under a plurality of different optical conditions;
The appearance inspection apparatus according to claim 1, wherein the second time-series image is an image configured by an average value for each pixel of the plurality of third time-series images. For each of the plurality of third time-series images, the third time-series image is based on a third extended time-series image obtained by connecting a normalization suppression constant and its reciprocal to the third time-series image. A third time-series normalized image generator for generating a third time-series normalized image obtained by normalizing the image;
A second correlation distance image for generating a second correlation distance image indicating a correlation distance for each pixel between each of the plurality of third time series normalized images and the second time series normalized image. A generator,
A first average image composed of an average value for each pixel of the plurality of second correlation distance images generated by the second correlation distance image generation unit, and a plurality of second correlation distance images A first probabilistic behavior description part for generating a first standard deviation image composed of standard deviations for each pixel;
Probabilistic for each pixel of the first correlation distance image and the plurality of second correlation distance images based on the first correlation distance image, the first average image, and the first standard deviation image. The visual inspection apparatus according to claim 3, further comprising: a first probabilistic distance image generation unit that generates a first probabilistic distance image indicating a distance. The third extended time-series image corresponds to the fifth connected image obtained by adding the normalization suppression constant to the average value in the time direction of the third time-series image corresponding to each pixel, and corresponds to each pixel. An image obtained by connecting the sixth connected image obtained by subtracting the normalization suppression constant from the average value in the time direction of the third time series image to the corresponding third time series image,
The third time-series normalized image generation unit, for each of the plurality of third time-series images, based on the time-direction average value and standard deviation of the corresponding third extended time-series image. The visual inspection apparatus according to claim 4, wherein three time-series normalized images are generated. A first time-direction average image generating unit that generates a first time-direction average image indicating an average of the first time-series images in the time direction;
A second time-direction average image generating unit that generates a second time-direction average image indicating an average in the time direction for each of the plurality of third time-series images;
A second average image configured by an average value for each pixel of the plurality of second time direction average images generated by the second time direction average image generation unit; and the plurality of second time directions A second probabilistic behavior description part for generating a second standard deviation image constituted by the standard deviation for each pixel of the average image;
For each pixel of the first time-direction average image and the plurality of second time-direction average images based on the first time-direction average image, the second average image, and the second standard deviation image The visual inspection apparatus according to claim 3, further comprising: a second probabilistic distance image generation unit that generates a second probabilistic distance image indicating the probabilistic distance. . A first time-direction standard deviation image generating unit that generates a first time-direction standard deviation image indicating a time-direction standard deviation of the first time-series image;
A second time-direction standard deviation image generating unit that generates a second time-direction standard deviation image indicating a standard deviation in the time direction for each of the plurality of third time-series images;
A third average image constituted by an average value for each pixel of the plurality of second time direction standard deviation images generated by the second time direction standard deviation image generation unit; A third probabilistic behavior description part for generating a third standard deviation image constituted by the standard deviation for each pixel of the time direction standard deviation image;
Based on the first time direction standard deviation image, the third average image, and the third standard deviation image, the first time direction standard deviation image and the plurality of second time direction standard deviation images, 7. A third stochastic distance image generation unit that generates a third probabilistic distance image indicating a stochastic distance for each pixel of claim 3, further comprising: Appearance inspection device. By differentiating a fourth time-series image obtained by photographing an inspection object under a plurality of different optical conditions in each of a first direction and a second direction intersecting with the first direction. A first time-series differential image generation unit that generates first and second time-series differential images;
An average value in the time direction of the first time-series differential image is subtracted from each of the plurality of differential images constituting the first time-series differential image, and further, a plurality of pieces constituting the second time-series differential image. A first time-series gradient vector image generating unit that generates a first time-series gradient vector image by subtracting an average value in the time direction of the second time-series differential image from each of the differential images;
Third and fourth time series differentiation by differentiating a fifth time series image obtained by photographing a normal product under the plurality of different optical conditions in each of the first and second directions. A second time-series differential image generation unit for generating an image;
The average value in the time direction of the third time-series differential image is subtracted from each of the plurality of differential images constituting the third time-series differential image, and further, the plurality of times constituting the fourth time-series differential image A second time-series gradient vector image generation unit that generates a second time-series gradient vector image by subtracting an average value in the time direction of the fourth time-series differential image from each of the differential images. Prepared,
The first time-series image is the first time-series gradient vector image;
The appearance inspection apparatus according to claim 1, wherein the second time-series image is the second time-series gradient vector image. The first extended time-series image includes a seventh connected image in which the density of each pixel is equal to the normalized suppression constant, and an eighth connected image in which the density of each pixel is equal to a reciprocal of the normalized suppression constant. Is connected to the first time-series gradient vector image,
The first time-series normalized image generating unit generates the first time-series normalized image based on a length of the first extended time-series image as a vector in a time direction;
The second extended time-series image includes a ninth connected image in which the density of each pixel is equal to the normalized suppression constant, and a tenth connected image in which the density of each pixel is equal to the reciprocal of the normalized suppression constant. Is connected to the second time-series gradient vector image,
The second time-series normalized image generator generates the second time-series normalized image based on a length of the second extended time-series image as a vector in the time direction. The appearance inspection apparatus according to claim 8.