JPH0869533A - Image inspection and recognition method, and reference data generating method used for same and device for them - Google Patents

Abstract

PURPOSE: To inspect a component is normal or defective from its image without being affected by lighting unevenness, stains, etc. CONSTITUTION: Featured images of predetermined density, edges, etc., are extracted from an inspection image (202) and quantized (203); and weighted generalized Hough transformation which previously finds a quantized featured image from a learnt image and adds and votes featured point weighting by referring to the weight of each feature point by feature values of the image is performed (204). Similarity is found from the transformation result (205) and it is decided to be normal or defective depending on whether or not the similarity is larger than a threshold value (206).

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention uses an image to visually inspect an object to be inspected, for example, whether it is good or bad, or classifies an object.
Alternatively, the present invention relates to an image inspection / recognition method for recognizing and a reference data creating device used therefor.

[0002]

2. Description of the Related Art First, a typical conventional image inspection method will be described below. Figure 1 shows the conventional inspection method by binarization.
Will be described with reference to. Consider the problem of the inspection in which the projected image of the inspection object is obtained as shown in FIG. 1A and the degree of bulging or bulging of the central rectangular area 11a is determined.

In FIG. 1A, areas 11b, 11c, 11
The part where the density is higher in the order of d and 11a is shown. The density change on the dotted line 12 passing through the centers of the areas 11a and 11d in the image is a curve 15 obtained by adding the stepwise change shown by the curve 13 in FIG. 1B and the variation shown by the curve 14, and this curve 15 Is binarized (patterned) with a threshold value of 16 or less and a threshold value of 16 or more. Threshold value 16 for performing this binarization processing
Is the lowest value 18 of the two peaks of the density histogram curves 17a and 17d in the areas 11a and 11d as shown in FIG. 1C. Normally, an ideal step-like pattern (curve 13) is not input, and a density change occurs due to dirt or a change in lighting conditions, and the density change is superimposed to form a pattern 15. Process. Such binarization processing is performed on the entire rectangular area 11a of interest, and the good or bad of the inspection object is determined by whether or not the resulting distribution state has a predetermined shape. However, it is sufficient if the threshold value 16 can be determined by predicting the method of the concentration fluctuation 14, but in many cases it is difficult. If the threshold value 16 is set incorrectly, the binarized shape of each part of the areas 11a and 11b deviates from the original shape of the rectangular area 11a, as shown in FIG. However, a stable shape cannot be detected due to a variation component such as the shaded portion 19. As a result, it becomes impossible to determine a defective state such as a crack or a bulge that should be actually detected.

Next, FIG. 2 shows a conventional inspection method based on density-normalized correlation. 2B, the density distribution M _kl (k = 1, ..., K; l = 1, ..., L) of each pixel in the reference window 22 to be inspected in the reference image 21 shown in FIG. 2A is obtained, This is registered as a density pattern. Figure 2
A region 24 corresponding to the reference window 22 in the inspection image 23 shown in C is scanned from the upper left corner to the lower right corner, and the following calculation is performed for each scanning point (i, j) to _{obtain the} correlation coefficient γ _ij Calculate (−1 ≦ γ _ij ≦ 1).

[0005]

[Equation 1] Here, M ₀ is the density average value in the reference window 22, T
₀ is the average density value within the size of the area 24 in the inspection image. The correlation coefficient γ _ij thus defined for the scanning point
Among them, the maximum value γ _max is detected, and the closer γ _max is to 1, the better the product. This correlation coefficient is invariant to the offset variation of the density, but as shown in FIG. 2D, the reference density pattern 25 and the inspection density pattern 26 are not affected by the non-uniform density fluctuation such as dirt and change of illumination conditions. A fluctuation component (hatched portion) 27 surrounded by is generated, and the correlation coefficient greatly fluctuates even for non-defective products, which causes erroneous determination.

As described above, in order to realize the image inspection method, what image feature is extracted (image feature extraction method), which image feature is used (image feature selection method), and the selected image feature is selected. How to use image features (matching method) is important. As the image feature extraction method, various methods are disclosed in Japanese Patent Laid-Open No. 63-226785, "Image feature extraction apparatus". That is, from the target image, a region feature image that reflects the properties of the region such as density statistic, complexity, and spread, and a contour feature image that reflects the properties of the contour such as edge and edge bending degree are extracted. Here, the region in the region feature image is not a region obtained as a result of segmentation but a local region to be processed by local filtering.

The density statistic is a distribution of density values within a certain viewing radius from each pixel as a characteristic value of the pixel. The complexity is calculated by accumulating the number of edges on the scanning line existing within the visual field radius when scanning radially from each pixel within a certain visual field radius, that is, counting for each scanning direction and the average value between the scanning directions. Is defined as a feature value of each pixel.
However, to increase the sharpness of the output, w _t (r) = a / r _k
(a and k are constants, r is the address of the scanning line), and the true edge density is calculated. Therefore, not the simple cumulative value of the edges, but the calculation of accumulating the number of times of crossing the edges. The degree of spread is an amount corresponding to the area of the region, and when scanning radially from each pixel within a certain visual field radius, the distance until it collides with the first edge is counted for each scanning direction, and between the scanning directions. Is defined as the feature value of each pixel. The edge bending degree is calculated as a variation amount (square norm) in the peripheral edge direction with respect to the edge direction of the pixel at the center of the visual field within a certain visual field radius from each pixel.
This feature has the property that even if there is a break in the edge due to a disturbance, it is unlikely to be affected by the integration effect.

The area characteristic image and the contour characteristic image are image characteristics that are complementary to each other, and it is difficult to sufficiently express the shape only by the area characteristic, and it is possible to sufficiently express the difference in texture only by the contour characteristic. difficult. Furthermore, the sensitivity to noise differs between the area feature image and the contour feature image,
Even if the contour is discontinuous to some extent due to noise, there is relatively little variation in the feature value in the area feature. In this way, applying image features having different properties to inspection, recognition, and the like means expressing images from various angles. If many feature images with such complementary relationships are used, inspection and recognition will be performed. The reliability of can be improved.

As a method of using the image feature selection method and the matching method, Japanese Patent Laid-Open No. 4-175885 “Image feature selection method in generalized Hough transform” has already been proposed. FIG. 3 shows a flowchart of processing in the "image feature selection method in generalized Hough transform". First, in step S1, a reference sample is selected for each category, that is, a good product and a defective product in the case of inspection, and a plurality of objects to be distinguished from each other in classification and recognition. Feature images with different properties are extracted and used as reference feature images. A feature point is selected for each reference feature image, that is, when the feature image is an edge image, each pixel of each point (feature point) indicating each edge is selected from the edge image, and the feature image is a density image. If so, for each quantized density value,
Each pixel (feature point) taking the density value (feature value) is selected from the density image. In step S2, a reference table is created for each category and each feature with respect to the selected feature point. In the reference table, as shown as a reference table 27 in FIG. 4, each feature value f _i is used as an index, and the position of each pixel having that feature value (feature point of that feature value of that feature image) starts from the reference point. Polar coordinate vector (r _i , α _i ), i = 1, ..., N _p (N _p is the number of feature points)
Retained as. The reference point may be anywhere in principle, but it is known that the conversion error of the generalized Hough transform is minimized when the reference point is placed at the center of gravity of the object.

In step S3, the feature image of each feature of M samples to which each category and each disturbance is added is extracted,
The characteristic image is subjected to generalized Hough transform using the reference table created in step S2. The derivation principle of the generalized Hough transform will be described with reference to FIG. 4 by taking an edge image as an example. The edge image (reference feature image) 28 of the reference image 21 shows the contour points of the object. Therefore, the feature point of the edge image is a contour point, and the feature value, that is, the edge value is 1, and the edge value is 0 except the contour point. Therefore, step S2
Then, in the reference table 27, only the edge point value 1 is stored as an index in the index column of the feature value f _i of the feature point, and each image (feature image) of the contour image (edge image) 29 of the edge image 28 of the reference image is stored. An address (r _i , α _i ), i = 1, ..., Np (Np is the number of contour points, that is, the number of feature points) expressed in polar coordinates of point p _i is stored in the address field. Next, an unknown input image 31 using this reference table 27
The processing in step S3 for will be described. The parameters to be calculated here are the position and orientation (rotation angle) of the object in the input image 31. The position of the object is the reference feature image 2
This is the position of the reference point (polar coordinate origin) already determined in 8, that is, the reference point of the object in the input image. The same feature image as the reference feature image 28, that is, the edge image 32 is generated for the input image 31, and the edge value of each pixel of the edge image 32 (its orthogonal coordinates are (x, y)) is stored in the reference table 27. Only when the value is 1 (feature value), the polar coordinate addresses (r _i , α _i ) for the feature value stored in the reference table 27 are read one by one, and the following address conversion is performed.

X_G= X + r_icos (αi + θ) y_G= Y + r_isin (αi + θ) where θ is the rotation angle of the object. Now for the sake of simplicity,
The rotation angle θ of the body is known. For address translation like this
The coordinates (x_Gy_G) Is the predicted object
It is a position that indicates the possibility of a focal point. An edge value of 1 was obtained
Each conversion address x for one pixel position (x, y)
_G, Y_GAs shown in FIG.
X on the cumulator array 34_G, Y_G1 for the position value
Just add votes. That is, for example, the unknown edge image 32
For example, all addresses in the reference table 27 for point A
Are read in order, the address is converted, and the converted address is read.
Les x_G, Y_GX of the accumulator array 34_G, Y
_GWhen you vote for the value of
Trajectory with a possible reference point on the data array 34 (value of 1
(Trajectory) 35a is drawn. Unknown edge
The same processing is applied to points B and C in image 32,
Traces 35b and 35c are obtained. Of the accumulator array 34
Each position x_G, Y_GEach value of is an additional vote (cumulative addition)
Therefore, when the loci 35b and 35c are drawn, the locus 35
At the position where it overlaps with a, the value becomes 2 when it overlaps once.
The value becomes 3 by overlapping. Edge image of unknown edge image 32
Is at least A with the edge image 29 of the reference edge image 28.
The relative positional relationship with the points B, C, and
Then, the three loci 35a, 35b, 35c are one
It will intersect at intersection 36. This process only 3 points
Instead of all the edge points on the unknown edge image 32
Reference on the accumulator array 34
Regarding the locus of the possibility of points, the object of the input image 31 is the reference image 2
If it is exactly the same as the one object, it will intersect Np times at the same point.
I will make a difference. The number of intersections, that is, the cumulative voting value distribution
Is obtained, for example, as shown by the curve 37 in FIG.
Maximum peak point (position) of cloth and maximum peak value (maximum
Finding the frequency value is the position of the reference point in the input image 31.
And the degree of shape preservation for the reference image can be calculated.
And If the rotation angle θ is unknown, the range of θ that can rotate is
By changing θ by the angle increment Δθ pitch by the circled portion, this time,
The accumulator array 34 is (x_G,y_G) In two dimensions
Without (x_G,y_G, θ) and the same for each θ.
The maximum peak value is given (x
_G ^* _,y_G ^* _,θ^*) Is calculated. And this θ ^*Is wanted
It is the rotation angle of the object to be struck. For example, in the case of inspection, the object
For example, the variation range of the rotation angle θ depends on the accuracy of the jig when manufacturing
For example, it is predicted to be within ± 10 °, and the angle θ is incremented within that range.
Just change it.

As described above, the address x _G ^* _, y _{G at which} the voting value on the accumulator array 34 in the parameter space becomes the maximum.
_^*, Theta ^* x Medium _{G ^*,} y _G ^* is a reference point position, the rotation angle of theta ^* is the object, the amount to which the vote value maximum value (maximum frequency value) indicates the proximity of the reference . In step 3 in FIG. 3, this maximum frequency value is calculated for samples in each category, a maximum frequency value distribution in that category is created, and for samples between certain categories, for example, defective sample images for non-defective items. To create a maximum frequency value distribution between categories.
The distance between the maximum frequency value distribution within a category and the maximum frequency value distribution between categories is calculated for each category / feature. The weight for each category / feature is determined by this distance. In step 4, the weight of each feature is sorted for each category, and each feature is ranked. In step S5, for each category, the number of reference tables to be combined is increased in order from the one with the highest feature rank, that is, the number of features to be combined is increased, and how much the categories can be distinguished for each category. evaluate. The resolution of the input image is changed in step S6, and steps S1 to S5 are repeated until the error rate becomes sufficiently small, that is, until the good product and the defective product are sufficiently distinguished, or the number of unidentifiable categories becomes zero. .

With the above processing, the generalized Hough transform output of each characteristic image obtained from the input image and the contents of the reference table of each characteristic image obtained from the standard image is stored in the accumulator of the generalized Hough transform output. The distance determined by the relationship between the intra-category variation of the frequency distribution on the array and the inter-category variation of the frequency distribution on the accumulator array of the generalized Hough transform output is used as a weight. The distances are evaluated, and the weighted distances are combined with each other until the weighted distance exceeds a certain value or no further improvement is made. This conventional method proposes a method of weighting feature types and selecting a combination of feature types.

[0014]

However, the method of selecting the feature points is limited to the method of selecting the feature point having the maximum mutual distance for each feature image obtained from the reference image. In this feature point selection method, if the feature points of the reference image to be selected include points that are contaminated with dust or other noise, the reference will be incomplete, so the inspection capability will be reduced. It causes to decrease. Also, even if the reference image is manually selected, there is no guarantee that it is the reference image. Therefore, feature points are automatically weighted from a series of non-defective images, and feature points with high weight, that is, high reliability, have more influence on feature type selection and matching. In addition, for a feature point having a low weight, that is, a low reliability, it is necessary to reduce the influence on the feature type selection and matching.

FIG. 6 shows an image example of the inspection pattern. Figure 6
The image of a non-defective product actually obtained with respect to the ideal pattern shown in A is an image in which a noise component is superimposed due to dust, dirt, and the like. For example, as shown in FIGS. Or rotation is occurring. Figure 6E
The image of the defective product shown in FIG. 6 is slightly flattened with respect to the ideal pattern of FIG. 6A, and similarly noise is superimposed, resulting in displacement / rotation. In such a situation, it is necessary to distinguish good products from defective products. This problem similarly occurs when an object is classified or recognized using an image.

The object of the present invention is to solve the above problems, and in the situation as shown in FIG. 6, a non-defective product which is generated due to a change in the illumination environment or the reflection characteristic of the sample surface, or due to the attachment of dust or the like. An inspection / recognition method capable of reliably distinguishing a non-defective product from a defective product, or correctly classifying or recognizing the same even with respect to a density variation of a reference sample image.

[0017]

According to the present invention, image features are extracted from an input image to obtain a feature image in a first step, and the feature image is quantized in a second step to obtain a quantized feature image. , A method of performing image inspection or recognition by obtaining the similarity or difference degree of the feature points of the quantized feature image in the third step by using the learning data of the corresponding features obtained in advance by the plurality of learning images, The weight of the feature point is calculated in advance from the feature value histogram at each feature point of the quantized feature image for the learning image, and the similarity or difference in the third step is used as the weighted similarity or Obtained as a weighted dissimilarity. In the third step, the weighted similarity or the weighted dissimilarity obtained for a plurality of mutually different feature images is subjected to feature weighting combination to obtain the weighted similarity or the weighted dissimilarity.

The weighted similarity in the third step is obtained by a weighted generalized Hough transform operation in which the weights of feature points are added and voted in the parameter space. Alternatively, the normalized correlation coefficient between the feature point feature value series of the feature image for the input image and the feature point feature value series of the feature image for the learning image, and the weighted normalized correlation operation obtained by weighting with the feature point weight sequence Go and ask. The weighted dissimilarity in the third step is
The weighted Hough transform is performed to add and vote the weights of the feature points to the parameter space for each quantization level of the quantized feature image to obtain a Huff plane. Calculate and obtain.

For one characteristic image, the total weight of each characteristic point is normalized by the number of the characteristic points. If the contrasts of the plurality of learning images are larger than the predetermined value, the region characteristic image is extracted as the characteristic image in the first step, the normalized correlation calculation is performed in the third step, and if the contrast is smaller than the predetermined value, the first step. Then, the contour feature image is extracted as the feature image, and the weighted generalized Hough transform calculation is performed in the third step. Alternatively, the weighted generalized Hough transform calculation and the weighted normalized correlation calculation are performed on each of a plurality of learning images of different categories such as non-defective products and defective products, and the inter-distribution distance of each similarity distribution of each category is obtained. , The one with the larger distance between distributions is calculated.

The quantization of the feature image is performed by setting the quantization width that maximizes the mutual distribution distance between the similarity or dissimilarity distributions for each category such as non-defective products and defective products for a plurality of learning images. The quantization width that minimizes the mean square error of the feature value histogram of the feature points in the feature images of the plurality of learning images is used. As for the weight of the feature point, a quantized feature image is obtained from a plurality of learning images, a histogram of feature values at each feature point of the quantized feature image is obtained, and the weight is obtained according to the degree of variation of the histogram.

According to another method of the present invention, reference data obtained by obtaining a quantized feature image of an input image as in the previous invention and obtaining the degree of similarity or difference of feature points of the quantized feature image from a learning image. In the same way, but in this case, the quantized feature image is made into a multi-bit plane consisting of bit planes for each quantization level, and each bit plane is generalized Hough transform or simple Hough transform using reference data. The conversion is performed to calculate the similarity from the generalized Hough transform, or the difference is obtained by calculating the difference between the Hough transformed and the reference multiple Hough plane. In either case, the weighted similarity calculation or the weighted difference calculation using the weight given to the bit plane is performed.

A plurality of quantized feature images are obtained from a plurality of learning images of a plurality of categories, and the learning data used in the third step is used to calculate the weighted similarity of the feature points of each feature image of the plurality of learning images or The weighted dissimilarity is calculated by the same method as the calculation performed in the third step, the similarity or the dissimilarity for each feature image is calculated from the calculation result, and the weight between the features is calculated from the similarity or the dissimilarity. , The features are combined, the distance between the similarity or dissimilarity within a category and the similarity or dissimilarity between that category and other categories is calculated, and the feature combinations are changed until this distance satisfies a predetermined condition. Thus, the combination of characteristic images used in the calculation of the third step is determined.

A threshold for discriminating between the categories is determined between the similarity or the dissimilarity within the category and the similarity or the dissimilarity between the categories in the determined combination. That is, this threshold value is used to determine whether the product is a good product or a defective product by comparing with the similarity or the difference calculated in the third step, and is used for the determination for classification or recognition. Further, the positional deviation amount, the rotation amount, or the scale change amount of the target origin, which has the maximum weighted similarity or the minimum weighted dissimilarity obtained in the third step, is obtained, and the amount is used as an inspection scale.

According to the image inspection / recognition device of the present invention,
An input image is stored in an image memory, means for extracting a feature of the image from the input image to generate a feature image, means for quantizing the feature image are provided, and the quantized feature image is stored in the feature image memory. Stored, the feature value and its feature points and weights for each feature image are stored in a reference table memory as a reference table, and for the quantized feature image,
There is provided a calculating means for calculating the weighted similarity or difference using the corresponding reference table in the reference table memory and a control means for controlling each of the above means.

According to the image inspection / recognition reference data generating device of the present invention, the input learning image is stored in the image memory, the feature of the image is extracted from the learning image, and the feature image is generated. Means for quantizing an image, the quantized characteristic image is stored in a characteristic image memory, means for determining a characteristic value distribution of the characteristic points from the quantized characteristic image is provided, and each characteristic image The feature point feature value distribution is stored in the feature point feature value distribution memory, means for determining the weight of each feature point from the feature value distribution is provided, and the feature point for each feature and its weight are stored in the reference table memory. A weighted similarity (difference) calculation means for obtaining a weighted similarity or difference by performing a correlation calculation on the feature image using the feature point weight of the reference table memory is provided. , Distribution of the weighted similarity or dissimilarity similarity (dissimilarity) stored in the distribution memory,
A means for obtaining the weight between features from the distribution of the above-mentioned similarity or dissimilarity corresponding to the category, and determining the combination between the features, and the weighted similarity within the category or the weighted dissimilarity between the features and the category Means for obtaining the distance from the weighted similarity or the weighted dissimilarity, and changing the combination of the features until the distance satisfies a predetermined condition, and the feature points of each feature image of the combination at that time and the weight thereof are calculated. Means for making reference data are provided.

In order to obtain the weight of the feature point determined from the variation of multi-cell feature (multi-resolution feature such as region feature and contour feature) of a plurality of image samples to be learned, the position shift of the feature point of the learning feature image is obtained. The correction is performed, the variation of the feature value of each feature point is measured, and the weight of each feature point is determined based on the result. The weight of each feature point is calculated for each feature. The weight of this feature point is given as a coefficient of the weighted similarity calculation and the weighted dissimilarity calculation, and the distance determined from the relationship between the intra-category variation and the inter-category variation of the obtained similarity or dissimilarity distribution is calculated between the feature types. And the weight of
The features are combined and applied in order from the highest weight, and the features are combined until the distance becomes a certain value or more or no more improvement.

As a result, with respect to fluctuations in lighting conditions and surface contamination, feature points for stably distinguishing good products from defective products are automatically selected from a set of multidimensional features (feature types are also automatically selected at the same time. ), The inspection reliability can be improved.

[0028]

【Example】

First Embodiment FIG. 7 shows a flow chart of a first embodiment of the present invention. This processing is divided into an image inspection reference data creation processing unit 100 and an inspection processing unit 200. In the description of the embodiments, the weighted similarity calculation calculation and the weighted dissimilarity calculation calculation, which are features of the present invention, will be described in detail. FIG. 7 shows a case where a weighted generalized Hough transform is performed in the weighted similarity calculation calculation. In the first embodiment, particularly, the learning feature image generating step 102 to the reference data storing step 10
7. The parts of the weighted generalized Hough transform step 113 and the weighted generalized Hough transform step 204 are the most characteristic parts relevant to the present invention. In learning image input step 101, a plurality of non-defective learning images are input. It is assumed that an image is quantized and input in multiple gradations (usually 256 gradations). In the learning feature image generation step 102, a learning feature image is generated by performing multi-cell feature extraction processing on a non-defective learning image. The multi-cell feature extraction is shown in FIG. That is, 4 of the density distribution 41, the direction distribution 42, the edge density distribution 43, and the edge space distribution 45 of the input learning image.
Focus on one basic expression. From the density distribution 41, a density statistic image 45 such as density dispersion is generated by local filtering. From the direction distribution 42, the curvature image (edge bending degree image) 46 and the image 4 of the amount (bar feature) indicating parallelism are obtained.
7 is generated. From the edge density distribution 43, the complexity image 4 showing the number of lines crossing the edge when radially scanned
8. From the edge space distribution 44, the spread degree image 49 showing the distance (radial distance) until the first collision with the edge when radially scanned, and the “shape of the shape” calculated from the distance ratio of the radial distance in the diagonal direction. The blob characteristic image 50 indicating “cohesiveness” is generated. These characteristic images 45 to 5
The extraction method of 0 is, for example, Japanese Patent Laid-Open No. 63-21308.
Japanese Patent Laid-Open No. 62-62, "Image feature extraction calculation method"
No. 213093, “Image feature extraction method”, and Japanese Patent Laid-Open No. 62-226785, “Image feature extraction device”. These characteristic images 45 to 50 are area characteristic images that reflect characteristics of areas such as density statistics, complexity, and spread (here, the area is not an area obtained as a result of segmentation, but is processed by local filtering. The target local area is shown) and the contour feature image reflecting the characteristics of the contour such as edge and curvature. In addition to such differences in feature image types including the original density image and edge image, the above-mentioned various features due to differences in spatial frequency when filtering and differences in resolution (difference in unit (cell) expressing pattern) Generate a new feature image for.

As an example, let us consider extracting a zero-cross point image, which is a contour feature, from the image of the inspection pattern shown in FIG. The zero-cross point image is a point at which the secondary differential value crosses zero with respect to the secondary differential image of the original image calculated by convolution integration of the original image and the Laplacian Gaussian filter (the gray value of the original image changes. Equivalent to the inflection point) is calculated and defined as the zero cross point. The Gaussian filter has (G (x, y, σ) = (1 / (2
πσ ² )) exp (-(x ² + y ² ) / (2σ ² ))) is defined by the second derivative G "of Laplacian Gaussian, and the zero cross points are extracted for different σ. .

9A-D show zero-cross point images extracted from the images of FIGS. 6B-E, respectively. Figure 9A
The uppermost characteristic image in D is a case where σ is small, and although the spatial frequency is high and a fine contour structure can be extracted, noise is also detected. The feature image of the bottom image in FIG. 9 has a large σ and has a low spatial frequency and does not react to noise, but only a rough outline structure can be extracted.

As described above, cells with multiple resolutions and different resolutions (multi-source cell characteristics) are collectively managed by the multilayer feature plane 51 (FIG. 8) having a pyramid structure. In this way, based on the four basic expressions (41 to 44 in FIG. 8), and by extracting the multi-cell feature generated by the difference in spatial frequency and the difference in resolution, even if noise is present, it is determined as a good product. It is possible to stably distinguish defective products. In the learning feature image generation step 102 in FIG. 7, a part of the multi-source cell features which are considered to be effective depending on the target is usually extracted in terms of system efficiency.

Next, learning feature point position deviation correction step 1
In 03, misalignment / rotation / scale correction is performed on the plurality of learning feature images obtained in the learning feature image generation step 102. Here, the feature point is a pixel point that directly represents the property of the target from the feature image, and for example, in the zero-cross point image, the pixel point having a value of 1 is the feature point. Since the positional deviation / rotation / scale correction is merely positional correction, any method may be used, such as calculating the deviation of the feature points by least-square fitting and performing affine transformation to correct the deviation amount. A simple method is preferable.

In this embodiment, a method using a generalized Hough transform which is a matching process will be described. One of the learning feature images is used as the reference feature image, and the feature point in the target (object of the learning image) and the target origin (reference point) position to be corrected are designated. The target origin position for this correction is preferably set at the center of the target in order to minimize the correction error. The feature point position from the target origin in the reference feature image is stored in the reference table using the feature value of the target feature point as an index, as in step S2 of FIG. Using this reference table, the generalized Hough transform is performed on other learning feature images. here,
In the weighted generalized Hough transform, which will be described in detail later,
Since the feature point weight w has not been determined yet, w is set to the same value among the feature points. That is, the increment value to the cumulative histogram is not w, but 1 as in the conventional generalized Hough transform.
Is.

In this generalized Hough transform, when the feature value of each feature point of the learning feature image matches the feature value index of the reference table, the feature stored in the matched feature value index as in the case of FIG. Address conversion is performed using the point position, voting is performed in the parameter space of the position / rotation angle of the target origin, and the position / rotation angle / scale change of the target origin is calculated according to the principle of majority voting. Using the determined position / rotation angle / scale change of the target origin, correction processing is performed by affine transformation for the amount of positional deviation / rotation change / scale change from the reference feature image, and the learning feature correction image is obtained as a result. calculate. FIGS. 10A to 10C show images obtained as a result of the correction processing on the learning feature images of high, medium, and low spatial frequencies in FIGS. 9A to 9D, respectively.

Next, when the feature value has multiple tones, the image feature quantization step 104 quantizes the image feature with a preferred quantization width as described later. If the image feature is F (x, y), the quantized image feature is G (x, y), and the number of quantization levels is Iq, then G (x, y) = (F (x, y) -F _min / R) × I _q 0 ≦ G (x, y) ≦ I _q −1. However, F _min is the minimum value of F, and R is the dynamic range of F. The quantization width is defined as R / I _q . Here, a quantum that maximizes the mutual distribution distance from the similarity distribution of good products (maximum frequency distribution within categories in the description of FIG. 3) and the similarity distribution of defective products (maximum frequency distribution between categories) defined later. It is preferable to quantize using the conversion width. If the quantization width is not optimal, the hit rate of the generalized Hough transform is reduced, which causes the inspection performance to be reduced. To calculate the optimum quantization width, it is appropriate to evaluate the distance between distributions of the similarity distribution. Therefore, even if it is a learning process to gradually change the quantization width to obtain the optimum value, the processing time It cannot be ignored. Therefore, it is preferable in terms of efficiency as a learning system if the existence range of the approximate optimum quantization width can be limited from the distribution of image characteristics of non-defective products without calculating the distance between distributions of similarity.

As an evaluation measure for limiting the existence range of the optimum quantization width, attention is paid to the mean integral squared error of the feature value histogram of the learning image of a good product, and the quantization width that minimizes this error is roughly optimized. It is used as a width, and the quantization width that maximizes the distance between distributions of similarity is searched for in the existing range. The average integrated square error of the feature value histogram is formulated by the following equation.

H (k), k = 1, ..., I
q, H is the ideal feature value histogram _idealk)
And error (I_q) = Σ^iq _{k = 1}(H_ideal(k) -H (k))² Minimize the integral squared error defined by H_ideal
Since (k) is unknown, the maximum likelihood estimation method
The expected difference value, that is, the mean squared error is calculated by the following formula.
It

E _error (I _q ) = (I _q / (nR)) [1
-((N + 1) / (n-1)) (Σ ^iq _{k = 1} H (k) ² /
n-1)] Here, n is the sum of the frequency values of the feature value histogram. By determining I _q that minimizes this E _error (I _q ), it is possible to limit the existence range of the optimum number of quantization levels (quantization width).

Further, as a simpler measure, the existence range of the optimum number of quantization levels (quantization width) is obtained by setting I _q ^* ≦ (2n) ^1/3 . Next, learning feature point feature value distribution measurement step 1
At 05, a feature value histogram is created for each feature point from a series of learning feature correction images (quantized images) stored for each feature. In the feature value histogram creation, there is usually a position correction error, so consider that error,
Pixels around the feature points are taken in by the allowable correction error to create a histogram. Alternatively, the image blocks may be divided in advance into overlapping image blocks by the allowable correction error, and a histogram may be created for each block.
Fig. 11 shows a zero-cross point characteristic image (1/0 at the zero-cross point).
A feature value histogram for a series of 0) 52 other than the zero-cross point is shown. That is, the feature value histogram 54 near the boundary 53 shows that the frequency of the zero-cross points 1 is high and the likelihood of the zero-cross points (feature points) is high. , The frequency of zero-cross points 1 is low, and the likelihood of zero-cross points is low.

In the learning feature point weight determination step 106,
Based on the learning feature point feature value histogram distribution state for each feature point, a weight indicating the degree of the feature is set for each feature point. Here, the number of learning samples is M. FIG.
In the case of 1, the weight is determined in proportion to the magnitude of the frequency ratio R (= frequency / M) of the zero cross points (points where the value of the zero cross point characteristic image is 1). A point with a value of 0 is not a zero-cross point and therefore is not a target for weighting. The point of low weight is
It is preferable to remove it at this stage from the viewpoint of calculation cost. For example, if the frequency ratio R is around ½ or more, it is considered that the decrease in the frequency ratio is caused by the quantization error. Therefore, it may be regarded as a zero cross point, but if R is extremely low, It is considered to be a noise component, so it is better not to consider it as a zero-cross point. In this way, the frequency ratio R is used for thresholding to remove noise-like points. The remaining zero-cross points are weighted by the frequency ratio R.
Using the weight distribution for each of the remaining zero-cross points,
Each point (feature point) of the reference zero-cross point set (reference feature point set) is weighted. In addition, as a special case, it is possible to uniformly set the weight for each remaining point. To find the reference zero-cross point set, 1) Search the peaks of each distribution from the weight distribution of the zero-cross points, set these as new zero-cross points, and set these as the reference zero-cross point set, 2) A method of focusing on the most average zero-cross point image of the zero-cross point learning characteristic correction image (zero-cross point characteristic correction image) and using the zero-cross points of the zero-cross point image as the reference zero-cross point set can be considered. In any method, the weight value w _i (i = 1, ..., N _p ) of the zero-cross point weight distribution at the same position is given to each zero-cross point of the reference zero-cross point set. Here, each weight value is normalized such that the sum of the weights of the zero-cross points in the reference zero-cross point set is the reference zero-cross point number N _p . Therefore, if each weight has the same value, the weight becomes 1 / N _p . In this way, the feature point weights are generalized regardless of the type of each feature.

It is assumed that the distribution of feature values in the example of the densified image is as shown in FIGS. 12A to 12D, for example. 12
As shown in B, the more the distribution is concentrated on a certain characteristic value (density), that is, the smaller the variation of the distribution is, the larger the weight is. In this case, the frequency ratio R obtained by dividing the maximum frequency by M
Can be regarded as a measure of weight. Alternatively, the weight can be set in inverse proportion to the magnitude of the variance of the distribution. Here, as in the previous case, the frequency ratio R or the variance value of the distribution is thresholded to eliminate noise-like points. The set of remaining feature points is set as a reference feature point set. The feature value of each feature point of the reference feature point set is, for example, a feature value that gives the maximum frequency or a feature value that gives the average value of the distribution. The weight values w _i (i = 1, ..., N _p ) for the set of reference feature points are normalized so that the total becomes N _p . Or, as this special case, with respect to each point remaining, uniformly set the weight, it is also possible to sum of all the weights are set such that N _p.

Further, the operator can directly specify a feature point on the screen by means of a mouse, a cursor or the like to remove a background image or the like which is not to be weighted in advance. Alternatively, the CAD (Computer A
id Design) Data can be read and unnecessary inspection points can be removed. In this way, the operator or
By prioritizing the feature points directly specified by the CAD data and determining the weights of the learning feature points, more detailed weighting of the feature points becomes possible.

In the reference data storing step 107 in FIG. 7, the origin (object) origin C _G (x _G, y _G ) of the calculation of the characteristic image 52 is determined as shown in FIG.
The positional relationship with each feature point Pi obtained in step 5 is expressed as a song coordinate (r
_i , α _i ) (i = 1, ..., N _p ). These feature points Pi are the polar coordinates (r _i , α _i ) of the feature point having the feature value and the weight w _{i of the} feature point obtained in step 106, using the feature value obtained in step 105 or 106 as an index. Is stored in the reference table 61. In FIG. 13, the reference tables 61a and 61 for the zero-cross point characteristic images 52 having high, medium, and low spatial frequencies are shown.
The structural example of b and 61c is shown. Zero-cross point feature image 52
Then, the characteristic value of the characteristic point is only 1, so that the index column is omitted in each of the tables 61a, 61b, and 61c.

FIG. 14 shows an example of the reference table 61 for N characteristic images. For the feature image f (1), the feature values I ₀ (1), I ₁ (1), ... Are used as indexes, and the position information and weight information (r _{i 0} (1), α
_{i 0} (1), w _{i 0} (1) (i = 1, ..., N _p0 (1))), (r _i1
(1), α _i1 (1), w _i1 (1) (i = 1, ..., N _p1 (1)) ... are stored in the reference table 61 ₁ and the same applies to the reference table 61 _k (k = 1). , ..., N) have characteristic values I ₀ (k), I ₁ (k) ...
Position information and weight information (r
_i0 (k), α _i0 (k), w _i0 (k)), (r _i1 (k), α _i1 (k
), W _i1 (k), ...) are stored. next,
The weights W _k , k = 1, ..., N between the feature images are determined, feature combination processing is performed, and the optimum combination is determined. Regarding the case of using the weight between the feature images in this feature combination processing, Japanese Patent Laid-Open No. 17588/1992
No. 5, "Image feature selection method in generalized Hough transform", is described in detail. On the other hand, in the first embodiment of the present invention, the feature point weight (analog value) is determined from a plurality of images as described above, and the feature point weight is generalized within the framework of feature image weights. It is extended to apply to Hough transform.

Therefore, first, in the learning image input step 109 in FIG. 7, both the learning image of a good product and the learning image of a defective product are input. In the learning feature combination processing step 110, a learning feature image of a non-defective product and a non-defective product is created from these learning images in the learning feature image generation step 111, similarly to step 102 (FIG. 8), and similarly, multi-cell feature extraction is performed. A learning quantized feature image is generated in the image feature quantization calculation step 112 for each of the feature images. This quantization width is the same as that of step 104.

In the weighted generalized Hough transform step 113, each feature point (x,
The weighted generalized Hough transform process is performed on y) as follows. For a non-defective image that has already been subjected to positional deviation correction, only the point sequence within the circumscribed rectangular frame of the reference object is targeted. The rotation angle θ may be fixed to 0 and the scale change rate s may be fixed to 1. On the other hand, for the non-defective image and the defective image that have not been subjected to the positional deviation correction, the target point sequence is free of all points in the learning image, the rotation angle, and the scale change rate. Although misregistration correction may be forcibly executed regardless of whether the product is a good product or a defective product, in general, for an image in which defective products or noise are mixed, a half-angle characteristic image such as an edge image or a zero-cross point image is not displayed. It is generally difficult to learn with correct positional deviation correction because the performance of positional deviation correction differs depending on the type of image feature because it is easily affected by noise and the image quality deterioration due to positional deviation correction is large. Therefore, here, the positional deviation correction is not intentionally performed. However, as described above, in the generalized Hough transform, matching with respect to the positional deviation is automatically performed.

The general formula of the weighted generalized Hough transform is shown below. When the feature value of each point (x, y) of the feature image of the image input in step 109 matches the feature value index stored in the reference table 61, the feature stored in the matched feature value index Point position (r,
α) is used to perform the next address conversion.

X ′ = x + s × r × cos (θ + α) (2) y ′ = y + s × r × sin (θ + α) Here, x ′ and y ′ are predicted target origins (reference points) in the learning image. The position, θ is the predicted rotation angle, and s is the predicted scale. The weight w of the feature point is cumulatively added to the address position of the parameter space (x ′, y ′, θ, s) formed of these predicted parameters. That is, if A (x ', y', θ, s) is a cumulative histogram in the parameter space (x ', y', θ, s), then A (x ', y', θ, s): = A ( x ′, y ′, θ, s) + w (3)

The above processing is performed in accordance with the predicted variation range of θ and s, for example, the variation range of ± 10 °, which is determined by the accuracy of the jig when manufacturing the object in the image, the position of ± 10%, and the scale S. Repeated processing is performed by slightly changing the fluctuation component by the fluctuation range. In normal usage,
With s fixed, A in the parameter space of (x ′, y ′, θ)
In many cases, a cumulative histogram of (x ', y', θ) is constructed.

In the similarity determination step 114, the optimum origin position / rotation that gives the maximum frequency to the cumulative histogram A (x ', y', θ, s) calculated by the weighted generalized Hough transform process described above. Angle / scale parameter (x
^* , Y ^* , θ ^* , s ^* ) is calculated, the maximum frequency H _m is normalized by the number of feature points N _p , and the value is calculated as the similarity S.

In the inter-feature weight measurement step 115, the N feature images for each of the non-defective product and the defective product are
Weighted generalized Hough transform and similarity determination. That is, assuming that the number of good sample images is M _ok and the number of defective sample images is M _ng , the similarity to the kth feature image is
In the case of non-defective products, the similarity series is S _ok (k, m), m = 1, ..., M _ok , and in the case of defective products, S _ng (k, m), m = 1, ..., M _ng . Thus, for the kth feature image,
As shown in FIG. 5, a non-defective product similarity distribution 63 and a defective product similarity distribution 64 are generated. The weight for the kth feature image is
It is calculated by the degree of difference between the similarity distribution 63 of non-defective products and the similarity distribution 64 of defective products. When the number of samples is large, the degree of opening is calculated using a distance measure such as the Batacharia distance using the average and variance of both distributions, but since it is expected that there are few learning samples, a simple statistical measure is used. For example, the distance W (k) between the worst value of a good product, that is, the value farthest from the reference feature image in the good product and the best value of the defective product, that is, the closest value of the good product in the defective product is set to the difference between both distributions. The degree is used as the weighting coefficient. That is, the kth
Weight coefficient W for the feature images (k) becomes W (k) = MIN (S ok (k, m)) -MAX (S ng (k, m)) + c (4). Here, c is a constant adjusted so that W (k) ≧ 0. The weighting factors W (1) ..., W (N) are
As shown in FIG. 16, it is stored in the header part of each feature of the reference data tables 61 _{1 to} 61 _N shown in FIG.

In the feature combination step 116, features are combined in order from the largest weighting coefficient W (k).
First, the weighting factors W (k) are sorted in descending order of size. After that, the feature combination is performed, and the OR combination and the AND connection can be considered as the feature combination method. The AND combination performs increment in the weighted generalized Hough transform, that is, cumulative addition of weights for the feature points only when the values of the same feature points on the combined feature images are satisfied at the same time. Here, as w (feature point weight) to be added, for example, a value averaged between the features may be used. In the OR combination, the cumulative histogram of the generalized Hough transform for the kth feature image is A (k | x ′,
y ′, θ, s), and combining up to N _c high order, Acc (N _c | x ′, y ′, θ, s) = ΣW (k) A (k | x ′, y ′, θ, s) (5) (where Σ is the sum of N _c pieces), and a weighted addition cumulative histogram Acc (N _c | x ′, y ′, θ, s) is calculated. Here, the weighted addition process of the cumulative histogram is not executed for all the values in the parameter space, but is executed only for the vicinity of the peak value for efficiency.

In the non-defective / defective product distance measuring step 117, the degree of separating the non-defective product / defective product having the feature of combining the upper N _c pieces (non-defective product / defective product distance) is measured.
The maximum frequency value S (N _c ) is calculated for Acc (N _c | x ′, y ′, θ, s), and the maximum frequency value S (N _c ) is calculated as N _p (N
_c ), that is, the similarity S normalized by the sum of the number of feature points in the reference table of the combined features (the standard number of feature points is also rearranged in the descending order of the weighting factor of the feature).
Calculate (N _c ). That is, S (N _c ): = S (N _c ) / (N _p (N _c ) ΣW (k)) (6) (where Σ is the sum of N _c pieces). Here, the normalized similarity distribution for the non-defective sample is S _ok (N _c , m),
, m _ok , the normalized similarity distribution for defective samples is S _ng (N _c , m), m = 1, ..., M _ng, and the degree of opening D (N _c ) for both distributions is , As a simple statistical measure, the minimum value of S _ok (N _c , m) and Sng
(N _c, m) difference between the maximum, that is calculated by _{D (N c) = MIN (} S ok (N c, m)) -MAX (S ng (N c, m)) (7).

In the feature selection end determination step 118, it is determined whether the two distributions do not intersect with each other with a margin, that is, whether the feature combination end condition of D (N _c ) -ε> 0 (8) is satisfied. ε (≧ 0) is a parameter for adjusting the margin.
Although ε varies depending on the target image, it should be re-evaluated as gradually increasing as a large value to make the system stable. Usually 0.1
The value is about 0.2.

If the end condition is satisfied, a pass / fail judgment threshold value
At decision step 119, it is determined whether the good product and the bad product are separated.
Threshold T_hTo determine. T_hAs a method of determining
How to set the limit value for non-defective products according to the degree of constant severity
Limit value MIN (S _OK(N_c, M)) and defective products
Limit value MAX (S_ng(N_c, M))
There is a possible method. On the other hand, the termination condition must be met.
For example, returning to the feature combination step 116, the feature combination is performed.
Number of joints N_cIncrease the number by 1 and execute the feature combination process
To do.

Next, the processing of the inspection processing section 200 will be described. In the inspection image input step 201, an inspection image is input. By the inspection feature image generation step 202,
An inspection feature image is generated by performing a multi-source cell feature extraction calculation. This inspection feature image has a termination condition ((8)
It is generated for the combination of features when (formula) holds. In the image feature quantization calculation step 203, image feature quantization processing is performed for each feature with the optimum number of quantization levels calculated in the learning process. With respect to the calculated check quantized feature image, in the weighted generalized Hough transform step 204, the reference table 61 storing the feature combination and feature point weight information determined by the learning process is used (3) Performs a weighted generalized Hough transform of the expression.

In the similarity determination step 205, the number of combinations N _c determined by the learning processing is used to perform the processing of the equations (5) and (6) to calculate the inspection similarity S _t . In the quality determination step 206, a quality determination threshold value T
_{Using h} , it is determined that the product is a good product if S _t ≧ T _{h and a} defective product if S _t <T _h . Second Embodiment Next, a second embodiment of the method of the present invention will be described with reference to FIG. In FIG. 17, parts corresponding to those in FIG. 7 are designated by the same reference numerals, and the description will focus on the parts different from the first embodiment.

A learning image of a non-defective product is input, a learning characteristic image is generated in the learning characteristic image generating step 102 in the same manner as the processing in FIG. 7, and the learning characteristic point position deviation correcting step 1
Position / rotation / scale correction at 03 is performed. This correction may use a weighted normalized correlation operation described later. Since the feature point weight w has not been determined yet, w in the weighted normalized correlation calculation is set to the same value between the feature points.
For example, let w _i (k) = 1, i = 1, ..., N _p (k). The position-corrected learning feature image is quantized with the optimum quantization width as in step 104 in FIG.

Further, a feature value histogram of each feature point is created, and the weight of the feature point is determined from its distribution. The table structure of the reference table 71 stored in the learning data storing step 307 is different from that of the first embodiment. For example, in FIG.
As shown in, the number of feature points for each feature image,
A series of x, y coordinate values, feature value f, and feature point weight w is stored. That is, for the first feature image f (1), the reference table 71 ₁ has ((x _i (1), y _i
(1), f _i (1), w _i (1)), i = 1, ..., N _p (1))
For each of the feature points of the Nth feature image f (k) ((x _i (N), y _i (N), f _i (N), w _i
(N)), i = 1, ..., N _p (N)) Reference table 71 _N
To be stored.

The learning feature image generator step 111 in the learning feature combination processing 110 is the same as in the first embodiment. For the characteristic image that has already undergone the positional deviation correction, only the point sequence within the circumscribed rectangular frame of the reference object is the target. In this case, the rotation angle θ may be fixed to 0 and the scale change rate s may be fixed to 1. On the other hand, for non-defective images and defective images that have not been subjected to misregistration correction, the target point sequence is all points in the learning image,
The rotation angle and scale change rate are free. The misregistration correction may be forcibly executed regardless of whether it is a good product or a defective product, but in general, for images with defective products or noise, the performance of misregistration correction differs depending on the type of image feature. It is necessary to select the features including the positional deviation correction error.
Therefore, here, the positional deviation correction is not intentionally performed.

The learning feature image generated in step 111 is
Quantize with the quantization width used in step 104 and step
At 309, weighted normalized correlation calculation is performed. This step
In 309, each θ, each scale s, and each point (x, y) is paired.
Then, the following processing is performed. Each point (x ', y') is predicted
When the target origin position is set to
_i(K), i = 1, ..., N_p(K), k = 1, ..., N
(Feature point weight sequence: w _i(K), i = 1, ..., N
_p(K)) and the same relative position (x ′ + x) as the learning feature point _i
(K), y '+ y_i(K)) inspection image feature value g_i
(K), i = 1, ..., N_pWeighted normalization phase with (k)
Perform a seki. Here, x of the learning feature point feature value_i(K),
y_i(K) Address is predicted rotation angle θ, scale
Address conversion (affine conversion) was performed on the value of s
Apply things. Prediction parameters (x ', y', θ,
The weighted normalized correlation coefficient for s) is A (k | x ′,
y ′, θ, s), A (k | x ′, y ′, θ, s) = Σ (w_i(K) (f_i(K) -f '_m(K)) x (g_i(K) -g '_m(K))) / ((Σw_i(K)) (Σ (f_i(K) -f '_m(K))²× Σ (g_i(K) -g '_m(K))²)^1/2) (9). Where f ′_m(K), g '_m(K) is that
The average value of each feature value series, f ′_m(K) = (1 / N_p(K)) Σf_m(K) (10) g '_m(K) = (1 / N_p(K)) Σg_m(K). Σ is all i = 1 to N_pIn total up to (k)
is there. The calculated A (k | x ', y', θ, s) is
The parameter space (x ′,
y ′, θ, s) is a cumulative histogram
In the case of the weighted normalized correlation, the parameter space (x ′,
y ′, θ, s) is the normalized correlation coefficient distribution.

The processing procedure for the following A (k | x ', y', θ, s) is the same as the processing in the first embodiment. The method of storing the weight value for the feature image in the inter-feature weight measurement step 115 is also the same as in the first embodiment, and as shown in FIG. (K) is stored.

Also in the inspection processing 200, instead of the weighted generalized Hough transform step 204 in the first embodiment, the weighted normalized correlation calculation step 310 (9)
First except that the weighted normalized correlation coefficient is calculated by the formula
This is the same as the embodiment. Next, the contrast Contrast = R / σ ² defined as the ratio of the dynamic range R of the density image and the variation σ ² of the density image is measured in advance for a plurality of learning images of good products, and the contrast of the learning images of the good products is measured. By selecting a matching method depending on the size of, it becomes possible to flexibly deal with the inspection target. That is, the weighted generalized Hough transform based on the contour feature image such as the edge image has an extremely excellent effect on an image with a lot of noise and a low contrast. On the other hand, a weighted normalized correlation based on a region feature image such as a density feature image is suitable for an image with relatively low noise and high contrast. Therefore, as shown in FIG. 29A, the contrast of the learning image is measured and threshold processing is performed, and if the contrast is smaller than the threshold, the weighted generalized Hough transform shown in FIG. 7 is performed using the contour feature image. Learning data is generated, and if the contrast is larger than the threshold value, the region characteristic image is used to perform the weighted normalized correlation calculation shown in FIG. 17 to generate learning data. In this way, the matching method is automatically selected, and more flexible inspection control becomes possible. The threshold value in this case varies depending on the target image and is experimentally determined.

More strictly, the weighted generalized Hough transform shown in FIG. 7 is performed on the contour feature image of the learning image to measure the inter-distribution distance of the similarity distribution of good / defective products, and By performing the weighted normalization correlation calculation shown in FIG. 17 on the characteristic image to measure the inter-distribution distance of the similarity distribution of the good / defective products, and selecting the learning data having the larger inter-distribution distance. It becomes possible to adopt the optimum matching method.

Third Embodiment Next, as an example of performing inspection by weighted dissimilarity calculation,
FIG. 2 shows an example of inspection based on the weighted difference calculation of the Huff plane.
0, it demonstrates using FIG. In this embodiment, the spread degree is adopted as a feature that incorporates the phase relationship around the edge points. As for the spread degree, as shown in FIG. 22, when scanning radially from each pixel within a certain visual field range, the distance d _i until the collision with the first edge is counted for each scanning direction, and the distance between the scanning directions is determined. the distance mean ^{_{Σ n i = 1 d i /}} n are as defined as a feature value. On the edge, the degree of spread is 0, which includes the property of the edge. The edge point here is a zero-cross point that is not easily affected by the density fluctuation. In this embodiment, the feature sets of both the spread degree distribution and the density distribution are applied to improve the recognition ability of the target image. These image features are
It is described in a multi-resolution framework (multi-resolution plane) so that an efficient recognition strategy of -to-Fine can be taken.

This embodiment is an inspection process based on "feature multidimensional Hough transform", and learning feature image generation step 1
In 02, the plurality of non-defective learning feature images are compressed by the smoothing / resampling process, and the features are extracted for each designated resolution level and stored in the multi-resolution / multi-layer feature plane. Here, the feature set Ω extracted for each resolution is Ω = (((F _k (x, y), x = 1,., D _x ), y = 1,., D _y ), k = 1, ., N _f ). However, N _f is the number of features, and D _x × D _y is the resolution.

As in the first and second embodiments, step 10
Image feature quantization in step 4 and steps 105 and 10
6 The weight of each feature point is determined from the feature value histogram distribution of each learning feature point. It is assumed that the weights are normalized so that the total number of characteristic points becomes a predetermined number of characteristic points N _p . At the same time, in the reference image generation step 401, from the feature value histogram of each learning feature point, the feature value or the average value giving the maximum frequency is calculated, and the calculated feature value is added to each feature point to obtain the reference feature image. To generate.

The following processing is performed on this reference feature image. In the step 104, feature quantization is performed on the reference feature image for the same efficiency as described in the first embodiment and the efficiency of matching by the Hough transform. Feature value Fk (x, y) at feature point (x, y) for each feature image k
G is the characteristic value when is quantized by the number of quantization levels Iq (k).
If k (x, y), then G _k (x, y) = ((F _k (x, y) -F _min )) / (R _k )) × I _q (k) 0 ≤ G _k (x, y) ≤ I _q (k) -1. However, F _min is the minimum value of F _k , and R _k is the dynamic range of F _k .

Here, the multiple bit plane generation step 40
In step 2, G _k (x, y) is mapped to a multiple feature bit plane B _k that represents ON / OFF of the (x, y) plane for each quantization level (feature value level). That is, B _k = [B _ku (x, y), u = 0, ..., I _q (k) -1] For example, quantization is performed at I _q (k) = 4, and at the point (x, y) If the quantized characteristic value G _k (x, y) = 2, then in B _ku (x, y), the bit is ON only when u = 2, that is, B
_{k 2} (x, y) = 1 and the bit is OFF when u ≠ 2, that is, B _{k 0} (x, y) = B _{k 1} (x, y) = B _{k 3} (x, y) = 0 Becomes For example, a feature image (G _k (x, y)) 73 quantized into a ternary value of a cell as shown in FIG. 23A is a feature bit plane (B _k0 (x, y) having a quantized value (feature value level) 0. )) 74
And a feature bit plane (B of a quantized value (feature value level) 1 (B
_{k 1} (x, y)) 75 and a quantized value (feature value level) 2 feature bit plane (B _{k 2} (x, y)) 76.

This mapping is done for each feature: elements are combined in a cascade to define a multi-feature bit plane B for all features. That is, [[B _lu (x, y), u = 0, ..., I _q (1) -1], ... [B _ku (x, y), u = 0, ..., I _q (k) -1], ・ ・ ・ [B _Nfu (x, y), u = 0, ..., I _q (Nf) -1]] --- >>> B = [B _v x, y), v = 0, ..., V-1] Here, V is the multiplicity of the multi-feature bit plane B, and V = Σ ^Nf _{K = 1} I _q (k).

Next, the weighted Hough transform calculation step 40
In 3, the weighted Hough transform is performed. That is, the point (x, y)
Θ = (t−1) Δθ, only when the bit at is ON

[0072]

[Equation 2] To ρ = (x−x ₀ ) cos θ + (y−y ₀ ) sin θ, and (ρ, θ)

[0073]

(Equation 3) The weight w of (x, y) is cumulatively added to the point (ρ, θ) at. Here, r = (ρ + ρ ₀ ) / Δρ, where ρ ₀ = √ (D _x ² + D _y ² ) / 2

[0074]

[Equation 4] (x ₀ , y ₀ ): Origin coordinates Perform this operation at all (x, y) points (x = 1, .., D _x , y = 1, .., D _y ).
Run against. In the multiple Huff plane generation step 404, the above operation is performed for each bit plane B of the multiple feature bit planes.
By executing for _v (v = 0, ..., V-1), the multiple Hough plane HM HM ＝ (((h _v (r, t), r = 1,., D (ρ))) , t = 1,., D (θ)), v = 0,., V-
1) is generated.

The weight information of each feature point and the information of the multiple Hough plane calculated as described above are stored in the reference table for each image feature in step 107. Further, a reference table as shown in FIG. 14 is created for the position and weight of each feature point for each bit plane and stored in the table. In this case, the weight of each feature point may be commonly used for each bit plane for the same feature image, or the weight of the feature point may be obtained for each bit plane. In FIG. 23, the multiple bit plane 7
This is a case in which the common feature point weight plane 74 is used for 3. These and the reference multiplex Hough plane 75 are stored in the reference memory.

In the learning feature combination processing 110, first, a learning feature image is generated in step 111 from the learning images of non-defective products and defective products, and the learning feature point position offset value is corrected in step 405. The misregistration correction is performed to facilitate the correspondence with the address of the feature point of the reference image when the weighted Hough transform is performed.

The feature image displaced in step 172 is image feature quantized, a multi-bit plane is created in step 406, and weighted Hough transform is performed in step 407 using the weight value of the feature point of the reference image. As a result, a multiple Hough plane is generated in step 408, and the following difference degree calculation is performed between the multiple Huff plane and the multiple Huff plane for the standard image stored in the reference table in step 409.

In the calculation of the dissimilarity, since the positions of the multiple Hough planes have already been corrected, the dissimilarity d between the two Huff planes h and h 'regarding the characteristic value level (quantization value) v is calculated.
ist (v) is defined as follows.

[0079]

(Equation 5) Here, when the feature point weights of the reference image are all the same, the position correction can be performed from the distribution of the multiple Hough planes. For rotation correction, for example, the array element for t is the maximum frequency direction

[0080]

(Equation 6) It is realized by rearranging the elements of the multiple Hough planes so that they coincide with each other. Also, the correction of the center of gravity position is
This is realized by calculating the shift amount that minimizes the difference.

In the bit plane weight measurement step 410, the degree of difference is weighted, but the feature value level v, that is, the weighting between bit planes is performed. Statistical weighting factor is the within-category distribution regarding dissimilarity (for example, non-defective distribution)
And distribution between categories (for example, distribution of defective products to non-defective products)
If follows a normal distribution, the Bhattacharyya distance and the like are known, but since there are often few learning samples, another scale is required. In this embodiment, the worst value of the simplest intra-category distribution (for example, the worst value among good products)
The distance between the worst value of the inter-category distribution (for example, the closest value to the good product among the defective products) is used as a scale.

When the number of learning samples for each category is M, the series of distributions in the non-defective product of the difference degree of the vth feature value level with respect to the non-defective product is dist _W (v) = {dist _W (1, v) ,. , dist _W (m, v), .., dist
Let _W (M, v)} be N and the number of defective samples be N, and the series of distributions between non-defective products and defective products of the degree of difference in the vth feature value level between non-defective products and defective products is dist _B (v) = { dist _B (1, v), .., dist _B (n ', v), .., dist _B (N,
v)}. Here, the weight for the vth feature value level is W (v) = MIN (dist _B (v))-MAX (dist _W (v)) + c. However, c is an offset value for setting W (v) ≧ 0.

Generally, the inspection ability should be evaluated for all combinations of each feature value level, but from the viewpoint of efficiency of learning, the feature value levels having larger weights are sequentially combined in the feature combination step 411. Apply the shape feature selection procedure. That is, the ordered new weight sequence is set as W (v) = SORT [v to W (v)], and the non-defective distribution dist _w (Dw) m, V _r ) and the distribution between good and defective products dist _B (n ', V _r ) are dist _W (m, V (n)) = Σ ^Vr _{v = 1} (W (v) dist _W (m, v) ) / Σ ^Vr _{v = 1} W (v) (11) dist _B (n ', V (n)) ＝ Σ ^Vr _{v = 1} (W (v) dist _B (n', v)) / Σ ^Vr _{v =} It becomes ₁ W (v) (12). Therefore, in step 412, the distribution of dissimilarity within good product dist _W (m, V _r ) and the distribution between good and defective products dist
_As a simple measure that gives the mutual difference of _B (n ', V _r ), the number of counts in which the series of good product and defective product distribution overlaps the series of good product distribution, that is,

[0084]

(Equation 7) Count of dist _B (n ', V _r ) overlap (n, V _r )
To calculate. This count number generally approaches asymptotically 0 as the number of combinations V _r of feature value levels increases. (Here, ε (≧ 0) is a safety parameter that increases the number of overlaps, and consequently the number of combinations of feature value levels, forcing the system to work on the safety side.) If the count does not decrease even if the number of combinations V _r is increased, the combination of feature value levels is terminated in step 413. The number of combinations at that time is Lr ^* , and the value obtained by dividing the limit count number by the sample number N of defective products is N _U ^* (0 ≤ N _U ^*
≦ 1) That, Lr ^* a _{(n) = OPTMIN [V r} ~ overlap (V r)] N U * (n) = overlap (Lr *) / N (13) N U * , it is a large Inspection capacity smaller value It is an evaluation scale shown. The rating scale, placing a measure of converting such value is high inspection capability increases [psi and ^{_{Ψ = 1- N U * (14}} ).

In step 414, the pass / fail judgment threshold value is set to the center of the difference distribution within the category (inside the good product) and the difference distribution between the categories (between the good and defective products), or the limit value of the good product (safety parameter ε). A value with a margin) is used as the judgment threshold. On the other hand, for the inspection image, step 20
1, an inspection feature image is generated, the positional deviation is corrected in step 420, the inspection feature image is quantized in step 203,
Generate multiple bit planes in step 421, step 4
The weighted Hough transform is performed in step 22, a multiple Hough plane is generated in step 423, and misregistration correction is performed if necessary.
Further, using the optimum feature point weights and optimum feature value level combinations constructed in the learning process in step 427, the degree of difference from the reference multiplex Hough plane obtained in step 404 is calculated by (1
It is calculated by the equation 2), and in step 206, the quality is determined based on the quality determination threshold value.

Fourth Embodiment In the third embodiment, the weighted Hough transform using the feature point weight is performed. However, even if the Hough transform is simply performed, there are noise influences, changes in the illuminance environment, and density variations. Also has a high reliability in distinguishing good products from defective products, has a high recognition rate, and can reliably perform classification. The flow of processing in this case is shown in FIGS.
, And the parts corresponding to those in FIG. 20 and FIG. 21 are denoted by the same reference numerals, and most of the processes are the same, so the differences will be mainly described. In addition, this embodiment also applies the feature sets of both the spread degree distribution and the density distribution shown in FIG. 22 as the features incorporating the phase relationship around the edge points. In this example, the multiple bit plane B is generated in the multiple bit plane generation step 402 from the quantized learning feature image. For each bit plane of this multiple bit plane B,
In the Hough transform calculation step 501, Hough transform is performed. Figure 2
0, at the weighted Hough transform calculation step in FIG. 21, the point (ρ, θ) corresponding to the point (x, y) where the bit is on
Cumulative addition of feature point weight W of point (x, y) (additional voting)
However, in step 501, 1 is additionally voted. Such a multiple Hough plane is generated, and in the reference data storing step 107, the multiple Hough plane is stored in the reference table.

In the positional deviation correction step 405 in the learning characteristic combination processing 110, the center of gravity of the characteristic image of the image input in step 109 is aligned with the coordinate origin (x _0, y ₀ ), and the attitude is i) the principal axis direction by moment calculation. , Ii) Average direction calculation, iii) Maximum frequency direction, etc. can be considered, but from the viewpoint of affinity between robustness and Hough transform, posture rotation correction is performed according to maximum frequency direction.
That is, the same correction as in step 405 of FIGS. 20 and 21 is performed. In step 502, the multi-bit plane obtained in step 406 is not weighted but simply subjected to Hough transform calculation. This Hough transform calculation is based on the same method as the calculation in step 501. In this way, the multiple Hough plane of the learning image input in step 109 is calculated in step 409 with respect to the reference Hough plane stored in step 107, the weight between bit planes is determined in step 408,
The combination of the characteristic value levels in step 411, the number in which the series of non-defective product / defective product distribution in step 412 becomes smaller than the worst value in the non-defective product distribution, that is, the number of overlaps is counted, and the count number is decreased in step 413. The number of combinations of feature value levels is increased until no more, and at that time, an evaluation scale for inspection and recognition is obtained from the count value, and step 414
The determination of the pass / fail judgment threshold value is performed by the same processing as that of the embodiment shown in FIGS.

Further, in the inspection processing 200, until the multi-bit plane is generated for the input image, the process shown in FIGS.
The fourth embodiment is similar to the fourth embodiment, except that in this embodiment, the Hough transform is performed on each of the multiple bit planes in step 503, that is, the feature in step 422 in FIGS. The Hough transform with the point weight of 1 is performed to generate a multiple Hough plane for a predetermined feature value level in step 423. Thereafter, this multiple Hough plane is used as the reference multiple Hough plane obtained in step 107. The difference is calculated in step 424, and the quality judgment is performed in step 206 by the same processing as in the embodiments of FIGS.

Fifth Embodiment When such a multiple Hough plane is used, image inspection / recognition can be performed by obtaining the similarity instead of the difference.
In this case, as shown in FIG. 28 in which parts corresponding to those in FIGS. 7, 20, and 21 are assigned the same reference numerals, the quantized characteristic image obtained in step 164 is generated in the multi-bit plane in step 402, In step 107, a reference table is stored for each bit plane in which the addresses of the characteristic points are stored using the characteristic value level as an index. In the learning feature image combination processing, the feature image quantized in step 112 is made into a multi-bit plane in step 406, and in step 131, the generalized Hough transform is performed for each bit plane using the correspondence reference table. That is, the addition vote in the processing in step 113 in FIG. 7 is always 1 instead of the weight W of the feature point. In step 114, the similarity is calculated for each feature value level, that is, for each bit plane by the same method as in step 114 of FIG. 7, and in step 115, the bit plane ( The weights of the characteristic value levels are determined, and in step 117, for the combined bit planes (feature value levels), the good product / defective product distance is obtained by the same processing as in step 117 in FIG. 7, and in step 118. Similar to the case of FIG. 7, the combination of feature value levels is determined, and similarly, the threshold value is determined in step 119. In the inspection process 200, a multi-bit plane is generated in step 421 from the quantized feature image obtained in step, and step 13
In step 2, generalized Hough transform is performed for each bit plane using the reference table stored in step 107, and in step 205, the similarity is calculated in the same manner as in step 205 in FIG. Is compared in step 206 to judge pass / fail.

The inspection scale of the quality judgment described above is the degree of similarity, but when the mounting position inspection is performed, the amount of positional deviation with respect to the reference, the amount of rotation change, and the amount of scale change are important. Both the weighted generalized Hough transform and the weighted normalized correlation calculation can calculate the position and rotation amount of the target origin that gives the maximum similarity, and can also detect the scale change amount, so it can be applied to the mounting position inspection. Becomes When the above various embodiments are used for classification or recognition, for example, to determine which of the first to nth categories the input image falls into, the i-th category (i = 1, 2, ..., N) is used. ) Is the non-defective product, and each category other than that is the non-defective product.
Learning data about categories, that is, a reference table (reference Hough surface) in the embodiment is created, and using this, a similarity distribution in the i-th category or a dissimilarity distribution and a similarity distribution between the i-th category and other categories. Or, the dissimilarity distribution is obtained, the optimum feature combination (feature value level combination) is determined from each distance between the two distributions, and the obtained n n-th It classifies or recognizes that the input image belongs to the category corresponding to the maximum in the similarity or the minimum in the difference.

Also in the case of performing the weighted normalized correlation calculation shown in FIG. 17, multiple bit planes are created as shown in FIGS. 20 and 21, and the weighted normalized correlation calculation is performed between the corresponding bit planes. Furthermore, the weights between the feature value levels (in the bit plane) are obtained, the combination of the feature value levels is determined, and the combination of the feature image multiple bit plane of the input image and the feature image multiple bit plane of the learning image is weighted for that combination. Normalized correlation calculation may be performed. 24 and 2
Similar to the relationship between FIG. 5 and FIG. 28, with respect to FIGS.
The method shown in can be applied to multiple bit planes.

In the case of recognizing N-value categories in the third embodiment, the evaluation measure MEAN (N _u ^* ) for discriminating all N categories on average is expressed by the equation (12) as MEAN (N _u ^* ) = Σ _{n. =} a ^{_{1 N N u * (n)}} / N. The following equation can be used as this evaluation scale as a scale Ψ that increases the discrimination ability as the value increases.

Ψ = 1−MEAN (N _u ^* ) _Let MEAN (L _r ^* ) be the number of average feature value level combinations of all N categories at this time. That is, MEAN (L _r ^* ) = Σ _{n = 1} ^N L _r ^* (n) / N This Ψ is used as an evaluation scale for selecting feature value level (bit plane) combinations.

The degree of difference between the unknown input image and each category is
Using the optimal feature value level combination determined in advance by learning, the following equation can be expressed from equation (11). dis (n, L _r ^* (n)) = Σ (W (n, v) dis
t (v)) / ΣW (n, v) Σ is from v = 1 to L _r ^* (n) The category n that minimizes the difference is the recognition result of the unknown input image, but the difference between the categories is When comparing
It is necessary to normalize the degree of difference in advance. Using the intermediate value Dm (n) between the intra-category distribution (10) formula and the inter-category distribution (11) formula of the dissimilarity determined at the time of selecting the feature value level combination, the margin shown by the following formula V
Define a.

[0095]

[Equation 8] However, this intermediate value D _m (n) takes into consideration the safety parameter ε. The category in which Va> 0 and the maximum Va is determined to be the category to be determined as the recognition result. Next, a configuration example of an apparatus that implements the above method will be described with reference to FIG. In the common bus 81, a frame memory 82, an image feature extraction unit 83, a feature image memory 84, an image feature quantization unit 85, a weighted similarity degree (difference degree), a calculation unit 86, a reference table memory 8
7, a quality determination unit 88, a control unit 89 such as a personal computer are connected, and memories 82, 8 are further connected.
4, 87 and units 83, 85, 86, 88 are mutually connected by a dedicated bus 90.

An object to be inspected (not shown) is photographed by the video camera unit 91, and an input image (inspection image) from this is stored in the frame memory 82 as a digital signal by an AD converter (not shown). In the feature extraction unit 83, for each feature of the reference table that is predetermined from the input image by the processing performed in step 202 in FIGS. 7, 17, 20, and 21, that is, stored in the reference table memory 87. A characteristic image is extracted and stored in the characteristic image memory 84. In the image feature quantization unit 85 for the extracted feature image, as shown in FIG.
The quantization characteristic image is quantized by the processing performed in step 203 in FIGS. 17, 20, and 21, and the quantized characteristic image is stored in the characteristic image memory 84. In the reference table 87, in the method of FIG. 7, step 118 in the reference table shown in FIG.
The characteristics of which the selection is completed are stored in the same manner as in FIG.
The reference table shown in FIG. 9 is the reference multiplex Hough plane and the reference multiplex Hough plane of the feature point weight selected in step 413 in the method of FIGS. In the method of FIG. 28, feature points and their weights are stored, and corresponding to these, each step 18, FIG. 20, FIG. 21, FIG. 24, FIG. 25, FIG.
Each threshold value determined in step 414 is stored. The weighted similarity (difference) calculation unit 86 compares the quantized image in the memory 84 with the reference table memory 87.
In the method of FIG.
The weighted generalized Hough transform calculation of 04 is performed, the weighted normalized correlation calculation of step 310 is performed in the method of FIG. 17, the generalized Hough transform is performed in the method of FIG. 28, and the similarity calculation is further performed. 20, the method of FIG. 21 generates multiple bit planes in steps 421, 422, 424, performs the weighted Hough transform operation, and further calculates the dissimilarity. In the methods of FIGS. 24 and 25, steps 421, 50.
Generation of multiple bit planes at 3, 424, Hough transform calculation, and difference degree calculation are performed. The calculation result of the weighted similarity (dissimilarity) calculation unit 86 is compared with the threshold value of the reference table memory 87 by the quality determination unit 88, and the determination result is output to the outside through the control unit 89, which is not shown in the figure. Displayed on the display. The quality determination unit 88 may use the function of the control unit 89.

For example, when inspecting non-defective / defective products of various parts, a lot of reference data is required, and the reference data and the threshold value are stored in a large-capacity memory such as an external magnetic disk. Input the part number to be inspected, the control unit 89 reads the corresponding set of reference tables and threshold values from the large capacity memory,
The reference table memory 87 is loaded and the inspection is executed.
At this time, the threshold value is held in the control unit 89, and the final pass / fail judgment is performed by the control unit 89.

Next, an example of the device configuration of the reference data creating portion will be described with reference to FIG. This apparatus has most of the functions of the image inspection apparatus shown in FIG. 26, and often serves as the image inspection apparatus. Therefore, parts corresponding to those in FIG. 26 are designated by the same reference numerals. In this apparatus, the learning feature point feature value distribution memory 93 and the feature value distribution measuring unit 9 are further included.
4, a feature point weight determination unit 95, a similarity (dissimilarity) distribution memory 96, an inter-feature weight measurement unit 97, a feature weight matching unit 98, and a pass / fail threshold determination unit 99, and a feature image memory 84, a feature The point feature value distribution memory 93 and the similarity (difference) distribution memory 96 have a multi-faceted structure so as to store a large amount of data as compared with the inspection apparatus of FIG.

In this case, as described above, the learning images are input to the frame memory 82, various characteristic images of them are extracted by the extraction unit 83, and stored in the characteristic image memory 84. These characteristic images are quantized. Each is quantized in the unit 85 and stored again in the memory 84. The feature value distribution measuring unit 94 obtains the feature point feature value distribution in step 105 for the same feature image, and the feature point feature value distribution is stored in the memory 93.
Using this feature point feature value distribution, the feature point weight determining unit 95 determines the feature point weight in step 106, and a reference table including each feature point and its weight is stored in the memory 87 for each feature value. . However, in the method of FIGS. 20 and 21, the standard image of step 401, the multiple bit plane of step 402, and the weighted Hough transform operation of step 403 are performed to generate the multiple Hough plane. It is stored in 87.

Next, a plurality of non-defective product learning images and defective product learning images are input to generate characteristic images for these images, and these are quantized and stored in the characteristic image memory 84. These quantized feature images are subjected to the weighted similarity (difference) calculation unit 86 in the weighted generalized Hough transform calculation in step 113, the weighted normalized correlation calculation in step 309, the multiple bit plane generation in steps 406 to 408, The weighted Hough transform calculation and the multiple Hough plane generation are performed, and the similarity (dissimilarity) is obtained from these calculation results and the distribution thereof is stored in the memory 9
6 is stored. The inter-feature weight measuring unit 97 calculates inter-feature weights (inter-bit-plane weights) in step 115 (step 410) and stores them in the memory 96. Further, the feature combination unit 98 combines the features in step 116 (step 411) (combines bit planes) in step 117 (step 412) to obtain the distance between the non-defective product distribution and the defective product distribution, and characterizes until the distance satisfies the above condition. (Bit surface) combination is performed. Then, the pass / fail threshold determination unit 99 determines the threshold.

In each of FIGS. 26 and 27, one or more of each unit can be configured by a DSP (digital signal processor).

[0102]

According to the first embodiment, as shown in FIG. 9, a zero cross point is extracted as an outline feature in an edge image, the spatial frequency of the edge is changed, and the most suitable zero cross point is obtained as a generalized huff. By doing the conversion,
A feature extraction function that is not easily affected by density offset fluctuations and relative sensitivity reduction, and a matching function that does not change the similarity degree are realized by the majority decision logic of the generalized Hough transform even if noise is present. Furthermore, stable feature point extraction for noise is guaranteed by weight learning of each feature point for multiple learning images, and the change in resolution is also considered by using complementary features such as region features and contour features. By applying the function of multi-cell feature extraction to the weighted generalized Hough transform, the weight value for that feature is automatically determined according to the image properties by learning processing, and the features are combined and applied to create a lighting environment or sample. It is possible to realize a highly reliable image inspection with a small error rate in distinguishing a good product from a defective product in an environment in which there is a density variation of a sample image of a good product caused by a change in the reflection characteristic of the surface and adhesion of dust or the like. It also enables image recognition with high recognition rate and image classification with low error rate.

According to the second embodiment, by the weight learning of each feature point for a plurality of learning images, stable feature point extraction with respect to noise is guaranteed, and complementary features such as a region feature and a contour feature are obtained. Applying the function of multi-cell feature extraction that also considers the change of resolution in combination with weighted normalized correlation calculation, the weight value for that feature is automatically determined according to the image property by learning processing, and the features are combined. By applying this, the error rate of distinguishing good / defective products is small and the reliability is high in the environment where there is a change in the density of the sample image of a good product caused by the change of the lighting environment or the reflection characteristics of the sample surface and the adhesion of dust. A high image inspection method, an image recognition method with a high recognition rate, and an image classification method with a low error rate can be realized.

According to the third and fourth embodiments, the selective feature extraction system for a large number of feature sets such as the degree of spread showing complementary properties to each other even in the presence of density changes and noise caused by illumination changes. Applied to the Hough transform framework,
Recognition is performed based on the dissimilarity or similarity to the multiple Hough planes obtained as a result, and an efficient image recognition system, image classification system, or highly reliable inspection system can be configured without reducing the discrimination ability. .

[Brief description of drawings]

1A is a diagram showing an example of an inspection image, FIG. 1B is a diagram showing examples of an ideal density change curve 13 and a density fluctuation curve 14 on a solid line 12 in FIG. 1A, and a rigidity curve 15 of these curves, and FIG. The figure which shows the example of the density histogram of the inside area | region 11a, D is FIG.
It is a figure which shows the extraction result of the conventional inspection method of the area | region 11a in A.

FIG. 2A is a diagram showing a reference window in a reference image, B
2A is a diagram showing an example of a density pattern in the reference window in FIG. 2A, C is a diagram showing a search region in the inspection image, and D is a diagram showing an example of the relationship between the density pattern of the search region and the density pattern of the corresponding reference window. Is.

FIG. 3 is a flowchart showing a processing procedure in an image inspection method using a conventional generalized Hough transform.

FIG. 4 is a diagram for explaining a conventional generalized Hough transform.

FIG. 5 is a diagram showing how voting is performed on an accumulator array (parameter space) when a general Hough transform is performed on an input edge image with respect to a reference edge image.

6A is a diagram showing an ideal pattern, and FIGS. 6A to 6D are FIG. 6A.
3B is a diagram showing a non-defective product pattern including some noise and rotation with respect to the ideal pattern of FIG.

FIG. 7 is a flowchart showing the processing procedure of the first embodiment of the method of the present invention.

FIG. 8 is a diagram for explaining the concept of multi-cell feature extraction from an input image.

9A to 9D are views showing images having different spatial frequencies from FIGS. 6B to 6E, respectively.

FIG. 10 is a diagram showing an example of a learning feature correction image using the image of FIG. 9A as an example.

FIG. 11 is a diagram showing an example of a feature point feature position distribution in a learning zero-cross point image.

12A to 12D are diagrams showing various examples of learning feature point feature value distributions using a density image as an example.

FIG. 13 is a diagram showing an example of a reference table of zero-cross point images in the first embodiment.

FIG. 14 is a diagram showing an example of a reference table in the case of N features.

FIG. 15 Defective product similarity distribution (similarity distribution between categories)
It is a figure which shows the concept of the degree of difference (distance) between a non-defective item similarity distribution (similarity distribution within a category).

FIG. 16 is a diagram showing a state in which feature weights are registered in a reference table in the first embodiment.

FIG. 17 is a flow chart showing a processing procedure of a second embodiment of the method of the present invention.

FIG. 18 is a diagram showing an example of a reference table in the second embodiment.

19 is a diagram showing a state in which feature weights are registered in the reference table of FIG.

FIG. 20 is a flowchart showing a processing procedure in reference data generation of the third embodiment.

FIG. 21 is a diagram showing a procedure in an inspection process of the third embodiment.

FIG. 22 is a diagram showing an example of spread characteristics.

FIG. 23 is a diagram conceptually showing a relationship between a characteristic image and its multiple bit plane and a reference table.

FIG. 24 is a flowchart showing a processing procedure in reference data generation according to the fourth embodiment.

FIG. 25 is a flowchart showing a processing procedure in reference data generation according to the fourth embodiment.

FIG. 26 is a block diagram showing an embodiment of the image inspection / recognition device of the present invention.

FIG. 27 is a block diagram showing an embodiment of a reference data generation device of the present invention.

FIG. 28 is a flowchart showing the processing procedure of the fifth embodiment of the present invention.

29A and 29B are flowcharts showing a procedure of selecting a characteristic image for each of learning images.

 ─────────────────────────────────────────────────── ─── Continuation of front page (72) Inventor Shoji Sahashi 1-1-6 Uchisaiwaicho, Chiyoda-ku, Tokyo Nihon Telegraph and Telephone Corporation (72) Inventor Shuichi Ohara 1-1-6 Uchisaiwaicho, Chiyoda-ku, Tokyo Nippon Telegraph and Telephone Corp. (72) Inventor Eiji Mitsuya 1-1-6 Uchisaiwaicho, Chiyoda-ku, Tokyo Nihon Telegraph and Telephone Corp.

Claims (28)

[Claims] 1. A first step of extracting a feature image from an input image to obtain a feature image, a second step of quantizing the feature image to obtain a quantized feature image, and a step of extracting feature points of the quantized feature image. A third step of obtaining the degree of similarity or the degree of difference using reference data of corresponding features previously obtained from a plurality of learning images, wherein each of the quantized feature images of the plurality of learning images is A feature is that the weight of the feature point is calculated in advance from the feature value histogram at the feature point, and the similarity or dissimilarity in the third step is obtained as the weighted similarity or the weighted dissimilarity using the weight. Image inspection / recognition method 2. The image inspection / recognition method according to claim 1, wherein the third step weights the weighted similarity or the weighted dissimilarity for a plurality of mutually different feature images by performing feature weighting combination. Find similarity or weighted dissimilarity. 3. The image inspection / recognition method according to claim 2, wherein the third step is to obtain the weighted similarity degree by a weighted generalized Hough transform operation for adding and voting the weights of the feature points to a parameter space. It is characterized by being. 4. The image inspection / recognition method according to claim 2, wherein in the third step, the feature point feature value series of the feature image for the input image and the feature point feature value series of the feature image for the learning image are normalized. It is characterized in that the weighted similarity coefficient is obtained by a weighted normalized correlation operation for obtaining a weighted correlation coefficient by weighting the weighted sequence of the feature points. 5. The image inspection / recognition method according to claim 2, wherein in the third step, the quantized feature image is weighted by Hough transform for additionally voting the weights of the feature points in a parameter space for each quantization level. The Huff plane is obtained, and the weighted dissimilarity is obtained by calculating the difference between the Huff plane and the reference Huff plane obtained from the feature image of the learning image. 6. The image inspection / recognition method according to claim 2, wherein the total weight of each of the feature points is normalized by the number of the feature points for the one feature image. 7. The image inspection / recognition method according to claim 2, wherein the frequency ratio of the feature values in each pixel of the same type of feature images obtained from the plurality of learning images is thresholded to identify noise-like pixels. The remaining pixels that have been removed are the characteristic points of the characteristic value. 8. The image inspection / recognition method according to claim 2, wherein when the contrasts of the plurality of learning images in the same category are larger than a predetermined value, the area feature is set as the characteristic image in the first step. An image is extracted, a weighted normalized correlation operation is performed in the third step, and when the contrast is smaller than the predetermined value, a contour feature image is extracted as the feature image in the first step, and in the third step. A weighted general Hough transform calculation is performed to obtain the weighted similarity. 9. The image inspection / recognition method according to claim 2, wherein a weighted generalized Hough transform calculation, a weighted normalized correlation calculation, and a quantization level are performed for each of a plurality of learning images in different categories. The difference calculation is performed on the Huff plane for each, and the similarity or dissimilarity distribution in each category,
The inter-distribution distance between each category and the similarity or dissimilarity distribution is obtained, and the operation that has the largest distribution distance is performed in the third step. 10. The image inspection according to any one of claims 2 to 7.
In the recognition method, the second step uses a quantization width that maximizes the inter-distribution distance between the intra-category distribution of the similarity or the dissimilarity for each category and the inter-category distribution for the plurality of learning images. Is to perform a quantization operation. 11. Image inspection according to any one of claims 2 to 7.
In the recognition method, the second step is to perform a quantization operation using a quantization width that minimizes an average integration square error of feature value histograms of feature points in the feature images of the plurality of learning images of one category. Is. 12. The image inspection / recognition method according to claim 10, wherein the learning image is obtained from a non-defective product, and the inter-distribution distance is between a distribution of non-defective products and a distribution of non-defective products. Fourth, comparing the degree of similarity or the degree of difference with weight with a threshold value to judge the quality of the object corresponding to the input image.
Including steps. 13. The image inspection / recognition method according to claim 12, wherein a displacement amount, a rotation amount, or a scale change amount of a target origin having the maximum weighted similarity or the minimum weighted dissimilarity is calculated, It includes a fourth step in which the amount is used as a test scale. 14. The image inspection / recognition method according to claim 11, wherein the learning image is obtained from a non-defective item, and the mean integral squared error is for a non-defective item, and the weighted similarity degree or the weighted dissimilarity degree. And a threshold value are compared, and a fourth step of determining quality of the object corresponding to the input image is included. 15. The image inspection / recognition method according to claim 14, wherein a displacement amount, a rotation amount, or a scale change amount of a target origin point having the maximum weighted similarity or the minimum weighted dissimilarity is obtained, It includes a fourth step in which the amount is used as a test scale. 16. The image inspection according to any one of claims 2 to 7.
In the recognition method, a fifth step of extracting the feature image from the plurality of learning images, a sixth step of quantizing the learning feature image, and a histogram of feature values at each feature point of the quantized learning feature image. And the eighth step of obtaining the weight according to the degree of variation of the histogram obtained in the seventh step.
The weights of the feature points are obtained by the steps. 17. The image inspection / recognition method according to claim 16, wherein a ninth step of obtaining a plurality of quantized feature images from a plurality of learning images of a plurality of categories, and the learning data used in the third step are used. Then, the tenth step of obtaining the weighted similarity degree or the weighted dissimilarity degree of the feature points of each feature image of the plurality of learning images by the same method as the operation performed in the third step, and the tenth step Eleventh step of obtaining the similarity or dissimilarity for each feature image from the calculation result, twelfth step of obtaining the weight between the features from the obtained similarity or dissimilarity, and the similarity in the category by combining the above features Alternatively, a thirteenth step of obtaining a distance between the dissimilarity and the similarity or the dissimilarity between the category and another category, and the above distance satisfying a predetermined condition. By the fourteenth step of changing the combination of the features to determine the plurality of different feature images in the third step. 18. The image inspection / recognition method according to claim 17, wherein the degree of similarity or dissimilarity within the category and the degree of similarity or dissimilarity between the categories in the combination determined in the fourteenth step are applied to these categories. 14th step of determining a threshold value for distinguishing 19. A first step of extracting a plurality of image features from a learning image to generate a feature image, a second step of quantizing each of these feature images into a quantized feature image, and said quantization. A third step of obtaining a feature value distribution of each feature point in the same type of feature images, a fourth step of determining a weight of each feature point from the feature point feature value distribution, and a second step of the same category as the learning image. A fifth step of generating a quantized second characteristic image and a third characteristic image from the learning image and the third learning image of another category, respectively, and the second and third characteristic images, the characteristic points and their weights. The sixth step of obtaining the degree of similarity or the degree of difference between the learning image and the characteristic image by the weighted correlation calculation of the characteristic points, and the similarity degree corresponding to both the categories or Is the seventh step of determining the weight between the features from the degree of dissimilarity, and the eighth step of obtaining the corresponding similarity or the distance between the dissimilarities of the above two categories by combining the features, and the distance is sufficiently large. Up to the ninth step of changing the combination of the features up to and including the final combination of the feature points and the weight thereof as reference data. 20. The reference data generating method according to claim 19, wherein the correlation calculation in the sixth step is a weighted generalized Hough transform calculation for additionally voting the feature point weights in a parameter space. 21. The reference data generating method according to claim 19, wherein the correlation calculation in the sixth step uses a sequence of weights of the feature image sequence, the second or third feature image sequence, and the feature point as weights. It is a weighted normalized correlation operation. 22. The reference data generation method according to claim 19, wherein in the correlation operation of the sixth step, a weight for adding and voting the weight of the feature point in a parameter space for each quantization level of the quantized feature image. A Hough plane is obtained by the Hough transform, and a difference operation between the Hough plane and the corresponding one of the characteristic images is performed. 23. An image memory in which an input image is stored, means for extracting a feature of an image from the input image to generate a feature image, means for quantizing the feature image, and the quantized feature image. , A reference table memory in which a feature value, its feature points, and weights are stored as a reference table for each feature image, and a corresponding reference in the reference table memory for the quantized feature image An image inspection / recognition apparatus comprising: a calculating unit that calculates a weighted similarity or a difference using a table; and a control unit that controls each unit. 24. An image memory in which an input learning image is stored, a means for extracting a feature of an image from the learning image to generate a feature image, a means for quantizing the feature image, and the quantized feature. A feature image memory for storing images, a means for obtaining a feature value distribution of the feature points from the quantized feature image, a feature point feature value distribution memory for storing the feature point feature value distribution for each feature image, the feature A means for determining the weight of each feature point from the value distribution, a reference table memory for storing the feature point for each feature and its weight, and a correlation calculation for the feature image using the feature point weight of the reference table memory And a similarity (dissimilarity) in which the distribution of the weighted similarity or dissimilarity is stored. Distribution memory, means for finding weights between features from distribution of similarities or dissimilarities corresponding to categories, and determining combination between features, and weighted similarity or weighted dissimilarity within category for that combination And means for obtaining the distance between the weighted similarity between categories or the weighted dissimilarity, and changing the combination of the above features until the distance satisfies a predetermined condition, and the feature points of each feature image of the combination at that time. An image inspection / recognition reference data generation device comprising: and means for using the weight as reference data. 25. A first step of extracting a feature of an image from an input image to obtain a feature image; a second step of quantizing the feature image to obtain a quantized feature image; An image retrieval / recognition method comprising: a third step of obtaining the degree of difference using reference data of corresponding features previously obtained from a learning image; and the third step, A fourth step of generating a multi-bit plane consisting of bit planes for, and a fifth step of Hough-transforming each bit plane of the multi-bit plane to generate a multi-Huff plane, and the multi-Huff plane and the reference data. An image inspection / recognition method comprising the sixth step of obtaining the above difference by calculating the difference from the reference multiple Hough plane. 26. A first step of extracting an image feature from an input image to obtain a feature image, a second step of quantizing the feature image to obtain a quantized feature image, and a step of extracting feature points of the quantized feature image. An image inspection / recognition method comprising: a third step of obtaining a degree of similarity using reference data of corresponding features obtained in advance from a learning image; A fourth step of generating a multiple bit plane consisting of bit planes, a fifth step of performing a generalized Hough transform on each of the predetermined ones of the bit planes using the above reference data, and the Hough transformed And an image inspection / recognition method comprising the sixth step of calculating the similarity. 27. The image inspection / recognition method according to claim 25 or 26, wherein there are a plurality of the characteristic images, and the operation is performed on a predetermined bit plane among the multiple bit planes obtained from the characteristic images. , The weighted calculation is performed using the weight given to the bit plane. 28. The image inspection / recognition method according to claim 27, wherein a seventh step of obtaining a plurality of quantized feature images from a plurality of learning images of a plurality of categories, and a multi-bit plane from each of the quantized feature images. The eighth step of generating each and the similarity or difference of each bit plane of the multiplex bit planes are calculated by the same method as the calculation performed in the third step using the learning data used in the third step. The ninth step of obtaining, the tenth step of obtaining the weight between bit planes of the feature value level from the degree of similarity or difference, and the above bit planes are combined, and the degree of similarity or difference in the category, the category and other Eleventh step of obtaining the distance between the similarity or the difference between the categories, and combining the bit planes until the distance satisfies a predetermined condition. The twelfth step of changing, determines the bit planes used in the third step.