JP2009157701A - Method and unit for image processing - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform accurate and high-speed positioning between a reference image and an inspection target image in template matching. <P>SOLUTION: In regard to a reference image R, an edge, which is a boundary of contrast, is extracted, and an initial ROI (Region Of Interest) having a predetermined ROI width centering the above edge is set. Then, sparse points uniformly distributed with a predetermined dot density among the initial ROI are set, and each sparse point is made to be a sparse ROI. By moving an inspection target image P in each of x axis direction, y axis direction and θ rotation direction, the degree of coincidence relative to the reference image R and the inspection target image P is calculated using a pixel value at the sparse point position, so that a positioning parameter (x, y, θ) producing the maximum degree of coincidence is obtained. Because the edge appears particularly in an industrial product, even when the sparse point is limited to the vicinity of the edge, sufficient positioning accuracy can be secured, and a high speed is obtainable because of a small data amount to obtain the degree of coincidence. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing method and image processing apparatus for image recognition and image identification used for various purposes such as object (person) search, defect detection, and authentication, and more specifically, pattern matching between two images is performed. The present invention relates to an image processing method and an image processing apparatus.

  In defect inspection equipment that visually inspects defects and defects of products manufactured at the factory, the inspection target image obtained by actually photographing the inspection target product is compared with the reference image, and the degree of coincidence The process of evaluating is performed. In such processing, in order to check the degree of coincidence between the inspection target image and the reference image, it is necessary to first align the images.

  As a general technique for aligning two images, a method called template matching is known. In conventional general template matching, the position of the reference image is shifted little by little, and the whole or a specific area in it is compared with the target area in the image to be inspected, and the position where both are considered to be the best match is determined. Try to find it. However, the conventional general template matching technique has a problem that the calculation amount becomes enormous as the image size increases, and it takes time until the result is obtained. In addition, there is a problem that the position is likely to be displaced during the alignment due to the influence of noise.

  On the other hand, a technique of setting a region of interest (ROI = Region Of Interest) of an appropriate size in an image and comparing both images only for this ROI has been known (Patent Documents 1 and 2). Etc.) According to such a method, since the calculation amount is smaller than that of the conventional method, the time until the result is obtained can be shortened. Further, the influence of noise can be reduced by narrowing the ROI to a region significant for alignment.

  However, for example, when in-line defect inspection is performed in a factory production line or the like, if the image processing takes time as described above, this inspection will limit the speed of the line. Is done. In the conventional technology as described above, since the amount of data to be calculated is reduced by simply narrowing the ROI range (area), it is possible to increase the speed to some extent. However, when the ROI is simply narrowed, there is a problem in that the accuracy and reproducibility of the alignment deteriorate.

  On the other hand, Patent Literature 3 discloses a technique called sparse correlation. This is to set a sparse, that is, a sparsely distributed point (sparse ROI), and to compare both images only for the sparse ROI. In such sparse correlation, it is important to select a sparse ROI so that the position of an object of interest existing in an image can be appropriately evaluated. In Patent Document 3, a sparse ROI is set by automatic learning. It has become. However, with such a method, the amount of calculation for setting the sparse ROI increases, which is contrary to the demand for image processing at high speed as described above.

Japanese Patent Laid-Open No. 11-3425 JP 2002-319022 A Special table 2002-516440 gazette

  The present invention has been made to solve the above-described problems, and the object of the present invention is to increase the speed by reducing the amount of calculation for pattern matching while ensuring high alignment accuracy and reproducibility. Another object of the present invention is to provide an image processing method and an image processing apparatus.

An image processing method according to the present invention made to solve the above problems is an image processing method for performing pattern matching between an inspection object image obtained for an inspection object and a reference image as a reference,
a) an edge extraction step of extracting an edge in the reference image;
b) an initial region of interest setting step for setting an initial region of interest having a predetermined width for the extracted edge;
c) a sparse region-of-interest setting step of setting points dispersed with a predetermined density in the initial region of interest as a sparse region of interest;
d) using the pixel values in the reference image and the pixel values in the inspection target image for a plurality of pixel positions corresponding to the sparse region of interest to determine the coincidence of both images, and based on that, the position between the images A positional deviation evaluation step for evaluating the deviation amount;
It is characterized by having.

The image processing apparatus according to the present invention is an apparatus for carrying out the image processing method according to the present invention, and performs image pattern matching between an inspection target image obtained for an inspection target and a reference image as a reference. Because
a) edge extraction means for extracting an edge in the reference image;
b) initial region of interest setting means for setting an initial region of interest having a predetermined width for the extracted edge;
c) Sparse region-of-interest setting means for setting, as sparse regions of interest, points dispersed at a predetermined density in the initial region of interest;
d) using the pixel values in the reference image and the pixel values in the inspection target image for a plurality of pixel positions corresponding to the sparse region of interest to determine the coincidence of both images, and based on that, the position between the images A positional deviation evaluation means for evaluating the deviation amount;
It is characterized by having.

  Since a specific object, for example, a product such as an industrial part manufactured in a factory, has a comparatively clear shading boundary on an image, an edge can be extracted with high accuracy. Therefore, in the image processing method according to the present invention, edge extraction is first performed on the reference image. Next, an initial region of interest having a predetermined width is set around the extracted edge. This initial region of interest can be regarded as having been widened so that the edges have a predetermined width. Thereafter, for example, points that are uniformly distributed at a predetermined density are set in the initial region of interest, and each point is set as a sparse region of interest. That is, the sparse region of interest is limited to the vicinity of the edge extracted from the reference image, not the entire reference image.

  The area of the initial region of interest set across the edges extracted from the reference image is much smaller than the area of the entire image, and the total area of the sparse region of interest set sparsely in the initial region of interest is Even smaller. Therefore, the amount of data used for the calculation for evaluating the misalignment between the reference image and the image to be inspected is much smaller than the data amount of the entire image, and a calculation for checking the coincidence of both images. The amount is much smaller than before. Thereby, the time required for pattern matching can be shortened.

  Further, unlike the conventional technique in which the area of the region of interest is simply narrowed, the sparse region of interest is spread and distributed in the initial region of interest set to a certain extent. Therefore, it is easy to combine with so-called coarse / fine search, in which coarse alignment is first performed with low accuracy, and then the accuracy is gradually increased while narrowing the movable range of alignment. In general, it is known that such a coarse / fine search requires less time on average than the high-precision dense search from the beginning, and this coarse / fine search and the above-described setting of the sparse region of interest. Further speeding up can be achieved by the combination.

  Therefore, in the image processing method according to the present invention, preferably, the predetermined width in the initial region-of-interest setting step is gradually reduced and / or the predetermined density in the sparse region-of-interest setting step is increased stepwise. However, it is preferable to improve the evaluation accuracy of the positional deviation amount by repeating the processes of the initial region of interest setting step, the sparse region of interest setting step, and the positional deviation evaluation step a plurality of times.

  The calculation of the amount of misalignment between the reference image and the inspection object image, in other words, the accuracy of the alignment of both images varies depending on the inspection purpose, the mechanical accuracy of the inspection object itself, and the like. In other words, high alignment accuracy may be required, and low alignment accuracy may be sufficient. Therefore, in the image processing method according to the present invention, it is preferable to determine a predetermined width in the initial region of interest setting step and / or a predetermined density in the sparse region of interest setting step according to required accuracy. .

  Thereby, when the alignment accuracy may be low, the processing can be performed at a higher speed. In addition, when high alignment accuracy is required, the necessary accuracy can be reliably ensured without reducing the speed as much as possible.

  First, the principle of template matching in the image processing method according to the present invention will be described with reference to FIG.

  Here, it is assumed that the inspection target image P is a grayscale image as shown in FIG. 1E obtained by photographing the inspection target with an imaging device such as a CCD camera. On the other hand, the reference image R is a grayscale image of the reference object of the inspection object, as shown in FIG. In order to simplify the description, it is assumed that the size of the inspection object is the same in both images P and R, that is, there is no enlargement, reduction, or deformation, linear movement in the x-axis direction, straight line in the y-axis direction. It is assumed that there is only a target movement and a rotational movement in the θ angle axis direction, that is, only a position difference.

An alignment parameter (x, y, θ), which is a displacement amount between the reference image R and the inspection target image P, is obtained by the following procedure.
First, with respect to the reference image R, a light and dark boundary line on the image is extracted as an edge. Various known methods can be used as an edge extraction method. FIG. 1B is an edge extraction image with respect to the reference image R. In general, unlike objects obtained in the natural world such as agricultural products and fishery products, industrial products have clear boundaries between light and shade, so that the extraction of edges is easy and the reliability of the extracted edges is high.

  Next, an initial region of interest (initial ROI) in which the region is widened by a predetermined ROI width around the extracted edge line is set. As shown in FIG. 1C, this initial ROI is equivalent to the one in which the line width of the edge is increased. Thereafter, sparse points that are distributed almost uniformly at a predetermined density are set in the set initial ROI. As shown in FIG. 1D, the initial ROI is formed by a set of sparse points. Each sparse point is a sparse ROI, and one sparse point corresponds to one pixel on the image. What is characteristic here is that the sparse ROI is not set widely in the entire image, but only in the vicinity of the edge of the inspection object. Such setting is possible because of the high reliability of edge extraction as described above.

  When the sparse ROI is determined, the degree of coincidence between the images R and P is calculated using the pixel value corresponding to each sparse point on the reference image R and the pixel value at the same position of the inspection target image P. However, the inspection target image P is moved by predetermined pixel steps in the x-axis direction, the y-axis direction, and the θ-angle axis direction, and the degree of coincidence is calculated for each movement. For the calculation of the degree of coincidence, correlation, difference, or the like can be used. Then, the movement amounts x, y, and θ at which the degree of coincidence between the images R and P is the highest are obtained as alignment parameters. In this way, the positional deviation for comparing the images R and P is obtained, and therefore, the difference between the inspection object and the reference object, specifically, from the degree of coincidence obtained in the state where the positional deviation is corrected, Can determine the presence or absence of appearance defects or defects of the inspection object.

  FIG. 5 shows an example in which the initial ROI and sparse ROI for the reference image R are actually set. When the shading boundary is relatively clear as shown in FIG. 5A, an initial ROI is appropriately set for a well-extracted edge as shown in FIG. (C) It can be seen that the sparse ROI is also set appropriately.

  The above is the template matching procedure in the image processing method according to the present invention. Here, what is important is the ROI width and the density of sparse points. FIG. 2 is an explanatory diagram of a sparse ROI setting method. As shown in FIG. 2, since the sparse points are dispersed in the range determined by the ROI width, the theoretical alignment accuracy (error) is such that the edge can move within the gap range where the sparse points do not exist (rotational movement is also possible). Equivalent). Therefore, the denser the sparse points, the better the alignment accuracy. However, in this case, the amount of data for evaluating the degree of coincidence increases, so that calculation processing takes time. On the other hand, if the sparse point is sparse, the alignment accuracy is lowered, but the calculation speed can be increased. For this reason, the alignment accuracy is roughly determined by the ROI width and the number of points with respect to the edge length. In other words, in order to achieve a certain required accuracy, the ROI width with respect to the edge length and the number of points or the density of the points may be defined.

  Next, the configuration and operation of an embodiment of an image processing apparatus for carrying out the image processing method according to the present invention will be described. FIG. 3 is a schematic configuration diagram of the image processing apparatus according to the present embodiment, and FIG. 4 is a flowchart showing a procedure of image alignment processing in the image processing apparatus. In the apparatus of this embodiment, speeding-up and accuracy improvement are achieved by combining the above-described alignment parameter calculation method and multistage coarse / fine search.

  In FIG. 3, the reference image storage unit 12 stores image data constituting a reference image in advance. The inspection target image input unit 10 is, for example, a CCD camera or a CMOS camera, captures an inspection target, and captures the captured image. This image data is stored in the target image temporary storage unit 11. The arithmetic processing unit 20 functions as an edge extraction unit 21, an initial ROI setting unit 22, a sparse ROI setting unit 23, a matching degree calculation unit 24, a matching degree determination unit 25, an ROI width / point density setting unit 26, and a defect detection unit. 27 and the like. Each function of the arithmetic processing unit 20 can be realized by a computer centered on a CPU. However, in reality, when all of the functions are processed by the CPU, it often takes time. For this reason, it is desirable to process specific functions with dedicated hardware. The output unit 13 displays a defect detection result.

A typical operation of the image processing apparatus having the above configuration will be described with reference to FIG.
When the processing is started, the arithmetic processing unit 20 reads the reference image R stored in the reference image storage unit 12 and sets it in the edge extraction unit 21 (step S1). The edge extraction unit 21 extracts a light / dark boundary as an edge in the reference image R (step S2). The ROI width / point density setting unit 26 sets the initial value of the ROI width in the initial ROI setting unit 22 (step S3). Here, in order to search at high speed with coarse accuracy, for example, a search for alignment is performed while skipping 10 pixels at a time, but it is necessary to set an ROI width that can sufficiently cover it.

  The initial ROI setting unit 22 sets the initial ROI spreading on both sides of the extracted edge using the ROI width set as described above (step S4). Thereafter, the sparse ROI setting unit 23 sets the sparse ROI by determining the sparse points almost uniformly so as to obtain a predetermined point density within the set initial ROI (step S5). The point density at this time is also specified by the ROI width / point density setting unit 26. However, since fine accuracy is not required at the beginning, the point density is extremely lowered and sparse points are sparsely distributed.

  When such a sparse ROI is determined, positional information of each sparse point is given to the coincidence calculation unit 24. Further, the image data of the reference image R read from the reference image storage unit 12 and the image data of the inspection target image P stored in the target image temporary storage unit 11 are also input to the coincidence degree calculation unit 24. And only about the pixel in the position based on the positional information of the sparse ROI, both the image R, from the pixel value of the reference image R and the pixel value of the inspection target image P moved about the x axis, the y axis, and the θ angle axis, The degree of coincidence of P is calculated (step S6). When the inspection target image P is moved, for example, as described above, the movement is performed while skipping 10 pixels in the range of the ROI width, and the degree of coincidence is obtained for each movement.

  If the calculation accuracy of the alignment parameter at this time does not yet satisfy the required accuracy set in advance (No in step S7), the coincidence determination unit 25 determines the coincidence calculated as described above. A predetermined number is selected in descending order of the degree, and alignment parameters (x, y, θ) corresponding thereto are acquired. Then, the predetermined number of inspection target images P whose positions are corrected by different alignment parameters are stored in the target image temporary storage unit 11 with (corrected inspection target image P ′) as candidates (step S8). Thereafter, a new ROI width and point density are set by the ROI width / point density setting unit 26 (step S9), and the process returns to step S4. After returning to step S4, a search that is denser than that in step S6 described above, that is, with a reduced number of pixels to be skipped, is executed, but the ROI width is narrowed and the point density is increased accordingly.

  After returning to step S4 in this way, a new initial ROI is set again for the edge extracted from the reference image R, and a sparse point is set within the initial ROI. Then, by using the pixel value at the position of each set sparse point, the degree of coincidence between each inspection object image P ′ whose position has been corrected as described above and the reference image R is examined, and the alignment with improved accuracy is performed. Find the parameters. Usually, the ROI width, the point density, etc. are determined such that the alignment accuracy is lower than the required accuracy by repeating the processing of steps S4 to S9. When it is determined Yes in step S7, An alignment parameter (x, y, θ) that maximizes the degree of coincidence is acquired (step S10). Since the positional deviation of the original inspection target image is obtained in this way, the defect detection unit 27 compares the inspection target image whose positional deviation is corrected with the reference image, and determines the inspection target object based on the degree of coincidence. The presence or absence of a defect or a defect is determined, and the result is output from the output unit 13.

  Note that, since the required accuracy of alignment differs depending on the purpose of image processing and the like, it is not always necessary to perform the fine search step by step from the rough search as described above.

  Moreover, the said Example is an example of this invention, and even if it changes suitably in the range of the meaning of this invention, correction, and addition, it is clear that it is included by the claim of this application.

Explanatory drawing of the principle of template matching in the image processing method which concerns on this invention. Explanatory drawing of the setting method of sparse ROI in the image processing method which concerns on this invention. 1 is a schematic configuration diagram of an embodiment of an image processing apparatus for carrying out an image processing method according to the present invention. 3 is a flowchart showing a processing procedure of an embodiment of an image processing method according to the present invention. The figure which shows the image of the Example of the image processing method which concerns on this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Inspection object image input part 11 ... Target image temporary storage part 12 ... Reference | standard image storage part 13 ... Output part 20 ... Operation processing part 21 ... Edge extraction part 22 ... ROI setting part 23 ... Sparse ROI setting part 24 ... Matching degree calculation Unit 25 ... coincidence degree determination unit 26 ... ROI width / point density setting unit 27 ... defect detection unit

Claims (6)

An image processing method for performing pattern matching between an inspection object image obtained for an inspection object and a reference image serving as a reference,
a) an edge extraction step of extracting an edge in the reference image;
b) an initial region of interest setting step for setting an initial region of interest having a predetermined width for the extracted edge;
c) a sparse region-of-interest setting step of setting points dispersed with a predetermined density in the initial region of interest as a sparse region of interest;
d) using the pixel values in the reference image and the pixel values in the inspection target image for a plurality of pixel positions corresponding to the sparse region of interest to determine the coincidence of both images, and based on that, the position between the images A positional deviation evaluation step for evaluating the deviation amount;
An image processing method comprising:   The image processing method according to claim 1, wherein the predetermined width in the initial region of interest setting step is gradually reduced and / or the predetermined density in the sparse region of interest setting step is increased stepwise. An image processing method characterized by improving the evaluation accuracy of the amount of misalignment by repeating the initial region of interest setting step, the sparse region of interest setting step, and the misregistration evaluation step a plurality of times.   The image processing method according to claim 1 or 2, wherein a predetermined width in the initial region of interest setting step and / or a predetermined density in the sparse region of interest setting step are determined according to required accuracy. A featured image processing method. An image processing apparatus that performs pattern matching between an inspection object image obtained for an inspection object and a reference image serving as a reference,
a) edge extraction means for extracting an edge in the reference image;
b) initial region of interest setting means for setting an initial region of interest having a predetermined width for the extracted edge;
c) Sparse region-of-interest setting means for setting, as sparse regions of interest, points dispersed at a predetermined density in the initial region of interest;
d) using the pixel values in the reference image and the pixel values in the inspection target image for a plurality of pixel positions corresponding to the sparse region of interest to determine the coincidence of both images, and based on that, the position between the images A positional deviation evaluation means for evaluating the deviation amount;
An image processing apparatus comprising:   5. The image processing apparatus according to claim 4, wherein the predetermined width in the initial region-of-interest setting unit is gradually reduced and / or the predetermined density in the sparse region-of-interest setting unit is increased stepwise. An image processing apparatus characterized in that evaluation accuracy of a positional deviation amount is improved by repeating the processes of an initial region of interest setting means, the sparse interest area setting means, and the positional deviation evaluation means a plurality of times.   6. The image processing apparatus according to claim 4, wherein the calculation parameter determines a predetermined width in the initial region-of-interest setting unit and / or a predetermined density in the sparse region-of-interest setting unit according to required accuracy. An image processing apparatus further comprising setting means.