JP2001297325A - Image processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem that special know-how is required and a processing time can not be shortened without lowering recognition accuracy. SOLUTION: The image processor for performing recognition by means of template matching using prepared template data is provided with a variance calculating part for calculating a variance inside the same area as a template image in the template data of an inputted source image and a variance deciding part for deciding whether a target exists or not by comparing the variance found by the valiance calculating part wit a threshold.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to an image processing device in a visual device used for position control and inspection of a robot or an industrial machine, and more particularly to generation of a template image in a template matching method.

[0002]

2. Description of the Related Art Before starting a description of a specific conventional image processing apparatus, a basic template matching method will be described first. In the template matching method (Template Matching = TM method), teaching-by-showing, that is, the object can be automatically re-recognized simply by registering the image of the object without setting a program for each object Such a function is a feature of the method, and a user who has no knowledge of image processing can easily use the function easily. Further, as a recent demand for an image processing apparatus in the field of factory automation (hereinafter referred to as FA), it is required to perform complicated inspection and positioning at high speed and with high precision which cannot be dealt with only by binarization processing. There is great expectation for light and shade template matching that performs positioning based on the generalized cross-correlation coefficient. However, gray-scale template matching has heretofore been impossible to implement without dedicated hardware due to its enormous amount of calculation. For this reason, the image processing apparatus has become expensive and has been an obstacle to market development.

Next, an outline of the density template matching processing will be described with reference to FIG. In the gray-scale template matching _, the correlation value M _x, between the template image G _{i, j} and the search image F _{i, j} is shifted while sequentially shifting the local area S _{x, y} having the same size as the template starting from the position (x, y) _{. y} is calculated and the position (x,
y) is output as the position where the template is found. The correlation value M _{x, y} is given by the following equation (1).

[0004]

(Equation 1)

In the above equation (1), F and G are the average values of the brightness of the image in the local area of the search image or the template image, and n is the number of pixels included in the local area _{Sx, y} . Further, as can be seen from the fact that the expression (1) includes a term representing the luminance variance of the image on the denominator side, the correlation value M _{x, y} is represented by a linear variation F of the brightness of the image. It can be seen that ' _{i, j} = F _{i, j} × k1 + k2' is not affected at all. Therefore, the contrast between the template image and the inspection image is changed by using the gray-scale template matching, and the margin for the change in brightness of the entire image and noise is large.

However, such gray-scale template matching takes a long time to calculate a correlation value, and when a high speed is required, such as in the FA field, it cannot be practically coped with due to time limitations. For example, when searching for a 128 × 128 template image in a 512 × 512 inspection image, the product-sum operation of the number of times according to the following equation (2) is required.
This requires a calculation time of 0 second or more.

128 × 128 × (512-128) × (512-128) = 2.4 × 10 ⁹ (2)

On the other hand, there is a coarse search method as a well-known high-speed technique. The coarse search method is, as shown in FIG. 7, as shown in FIG.
In this method, an image (coarse image) degenerated to 8 is created, a rough image is searched for a template image, and an approximate position of the rough image is searched for in the original image to obtain an accurate position. The amount of calculation required to obtain the same result as the previous example by this method can be the number of times given by the following equation (3), provided that the ratio of coarse / fine search is 1: N.

[0009]

(Equation 2)

[0010] When N is changed in this manner, there is an optimum coarse / fine ratio as shown in FIG. 8.
0.1 which is set as a standard for practical use in the FA line
To achieve recognition within seconds, a time reduction of several hundred times must still be achieved.

As described above, the conventional template matching method using the coarse / fine search has a problem that the processing speed is slow and impractical.

Several techniques have been developed for such template matching techniques. FIG. 9 is an explanatory diagram showing a conventional partial template matching method disclosed in Japanese Patent Publication No. 2-642, for example. In the figure, reference numeral 11 denotes a circuit element forming region of a semiconductor pellet to be subjected to pattern position detection, and 12 denotes connection pads regularly arranged on two sides thereof. Reference numerals 13a and 13b denote pattern search areas (target patterns) searched for pattern position detection.

In this case, a template that matches each of the target patterns is provided, and a true peak is to be detected using the positional relationship taught in advance. In this case, even if a plurality of patterns having the same characteristics are arranged at regular intervals by dividing the template to be targeted into parts and performing template matching with the pattern search areas 13a and 13b in each part,
The target pattern can be identified. In addition, since the target pattern can be imaged by being divided into portions, the resolution can be increased.

However, in this method, the user has to determine which part should be used as a template, and know-how is required.

On the other hand, as a method for self-evaluating a template, Japanese Patent Application Laid-Open No. 61-74082 proposes the following method. Next, this will be described with reference to FIG. In the figure, reference numeral 14 denotes an image pickup means for picking up an image of a detection target, 15 denotes a binarization circuit for binarizing imaged image data, 16 denotes an image memory for storing binarized image data, and 17 denotes a shift memory 18. A matching circuit for matching a partial pattern cut out from image data with a standard pattern, and a pattern matching circuit 22.
Is a minimum value detection circuit for detecting the minimum value of the output of the above, 23 is an area limiting circuit, 24 is a timing generation circuit for giving a predetermined timing to them, and 25 is a computer for performing overall control of the system.

In this method, a partial pattern having a size of a standard pattern to be obtained is cut out from an image of a detection target imaged by the image pickup means 14 and is set as a standard pattern candidate.
An operation is performed to obtain an evaluation value indicating suitability as a standard pattern from the sequentially extracted partial pattern or the partial pattern and the image to be detected, and determine the standard pattern based on the value. As described above, according to this method, a standard pattern suitable for pattern matching can be automatically selected from within a screen, so that user know-how is not required.

However, in this method, in order to select a template from the input image, the user must determine the size of the template in advance, and the self-evaluation is performed to automatically register the optimal template. Even if it does, it cannot be evaluated whether the optimal size is selected.

In this method, the entire input image must be sequentially cut out as template candidates and evaluated for each template, which takes time. When performing recognition using many template matchings, the object that the user wants to recognize is often self-explanatory, but the user's knowledge cannot be used, and the template is determined only by the evaluation value, which is unnecessary. However, there is a problem that it takes a lot of time to perform various processes, and it is difficult to reflect the intention of the user.

Further, Japanese Patent Application Laid-Open No. 4-359388 proposes the following system. Next, this will be described with reference to FIG. This apparatus includes a search image storage device 26 for storing a search image, a template image storage device 27 for storing a template image, a 0-value data processing circuit 28 for processing 0 values of both images, and a ratio of both image values. The divider 29 to be obtained, a calculated value storage device 30 for storing the calculated value of the divider 29, and the absolute value of the difference between the output value of the divider 29 and the stored value of the counted value storage device 30 is obtained. An absolute value calculating circuit 31, an adding circuit 32 for adding the absolute value obtained this time to the absolute value obtained last time, a threshold value storing circuit 33 for storing the threshold value, and comparing the added value by the adding circuit 32 with the threshold value And a comparison circuit 34. According to this method, the sequential sequential test method (Sequential Sim
ilality Detection Algorit
hm = SSDA method) is modified so that it can be achieved even if there is a difference in brightness between images. Therefore, there is no restriction on the brightness of the surroundings at the time of acquiring an image, and the present invention can be used for an application in a situation where the environment constantly changes.

Although this method can cope with a change in the width of the contrast of the brightness of the entire input image, when the brightness of the entire image is shifted up or down, the ratio of the two images is reduced. Cannot be handled because it changes depending on the position of the template. Further, threshold processing is performed, and know-how is required to determine the threshold.

Further, as a method of obtaining the accuracy of the sub-pixel, the following method is proposed in Japanese Patent Application Laid-Open No. Hei 5-120436. Next, this will be described with reference to FIG. This method calculates a correlation value between the detected image and the template image while shifting the pixel position in pixel units, and searches for a pixel position having the maximum correlation value (first stage).
The position of a neighboring pixel and a correlation value are obtained (second stage), and a multivariable polynomial regression surface is determined from these coordinates and the correlation value (third stage).
Step), and a peak of sub-pixel accuracy is detected from the peak of the curved surface (fourth step). According to this method, since an estimated peak position that is continuous with a change in the peak position can be obtained with high accuracy from the multivariable polynomial regression surface, the position can be detected with sub-pixel accuracy beyond the pixel unit.

However, in this method, the correlation values of all 9 points near the center and 8 must be calculated first, which takes a long processing time. Also, in this method, it is assumed that the relationship between the XY plane and the correlation value forms a quadratic surface, but the actual correlation value peak and the shape in the vicinity thereof draw a sharper peak than the quadratic surface. Therefore, a peak cannot be accurately obtained by approximation of a quadratic surface.

[0023]

Since the conventional image processing apparatus is constructed as described above, it takes a lot of time to calculate the correlation value in the ordinary shading template matching, so that it is not practical, and it is not practical. The processing time cannot be sufficiently shortened even by template matching by the method described above, and it is not practically possible to cope with the field requiring high-speed processing such as FA due to time constraints. In such a case, the user needs know-how to determine which part to use as a template, and in the method of performing self-evaluation of the template, the template is determined only by the evaluation value.
SSDs do not take advantage of the user's knowledge, take a lot of time for unnecessary processing, and hardly reflect the user's intention.
The method A cannot cope with the shift up or down of the brightness of the entire image, and has a problem that know-how is required to determine a threshold for threshold processing.

The present invention has been made to solve the above-described problems, and has as its object to provide an image processing apparatus which does not require special know-how and can reduce the processing time without reducing the recognition accuracy. I do.

[0025]

An image processing apparatus according to the present invention is an image processing apparatus for performing recognition by a template matching method using template data prepared in advance. A variance value calculation unit that calculates a variance value in the same region as the image, and a variance value determination unit that determines whether an object exists by comparing the variance value obtained by the variance value calculation unit with a threshold value Is provided.

[0026]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described below. Embodiment 1 FIG. FIG. 1 is a diagram showing a configuration of an image processing apparatus according to Embodiment 1 of the present invention. In the figure, reference numeral 40 denotes an object such as a work registered by a user, and reference numeral 41 denotes an image input unit such as a television camera which captures the object and inputs an original image. Reference numeral 53 denotes an image memory for storing the original image input by the image input unit 41, and reference numeral 54 denotes a template image, a variance value of the template image, and a threshold value of the correlation value, which are prepared in advance for template matching. , Search area, search illumination state parameters, and the like. Reference numeral 55 denotes a variance value calculation unit that calculates a variance value in the same area as the template image in the template data 54 in the original image stored in the image memory 53, and 56 denotes a variance value calculation unit. A variance value determination unit that determines whether an object exists based on the variance value obtained. Reference numeral 57 denotes a correlation value calculation unit that calculates a correlation value between the original image stored in the image memory 53 and the template image in the template data 54. Reference numeral 58 denotes a correlation value calculated by the correlation value calculation unit 57. It is a correlation value determining unit for determining. Reference numeral 59 denotes a recognition result recording unit that records a recognition result based on the determination results of the variance value determination unit 56 and the correlation value determination unit 58.

Next, the operation will be described. Here, FIG.
4 is a flowchart showing a flow of processing of the image processing apparatus according to the first embodiment. The user generates the template data 54 in advance.
As described above, 4 includes a template image, a variance value of the template image, a threshold value of a correlation value, a range of a search area (start coordinates, image size), an illumination state parameter at the time of a search, and the like. When the process is started, the template data 54 is loaded first (step ST51), and an input image of the search area is stored in the image memory 53 (step ST52). Next, the sum of the variances of the input image in the same area as the size of the template image in the template data 54 is calculated by the variance value calculator 55 using the following equation (4) (step ST53). .

[0028]

(Equation 3)

In the equation (4), F is the average value of the luminance of the image in the local area, and n is the local area S _{x, y}
Is the number of pixels included in.

Next, if the sum of the variance values Vx _{, y} calculated by the variance value calculation unit 55 is smaller than the threshold value, it is determined that no object exists at that position, and the subsequent processing is performed. Censor. On the other hand, if the sum of the variances is larger than the threshold, the position is left as a candidate for the object recognition position, and the final accurate position is calculated by the correlation value calculation unit 57 by calculating a correlation value by template matching. (Step ST5
5) Compare the correlation value with the threshold value (step ST
56) Decide, and record the result in the recognition result recording unit 59 (step ST57).

The threshold value of the variance value is determined in accordance with a condition at the time of inspection, such as a change in illumination or a change in color of the object 40, which is input by a user as a parameter. That is, when the fluctuation of the illumination condition is large, the variance of the template image and the variance of the search image are significantly different. Conversely, when the illumination condition is stable,
Take close values. By utilizing this fact, the user is left as a candidate as an illumination state parameter only when the variance of the search image is larger than the variance of the template image when the illumination condition is stable. Further, when the illumination condition is unstable or easily affected by disturbance, a threshold value of a fraction of the template image, for example, 1/5 is set as a threshold value, and when the variance of the search image is larger than the threshold value, it is determined as a candidate. The final decision is made based on a conventionally used correlation value.

Reference Example 1 Although not directly related to the present invention,
Reference Example 1 to help the understanding is shown here. FIG. 3 is a block diagram showing the image processing apparatus according to the first embodiment. The corresponding parts are denoted by the same reference numerals as in FIG. In the figure, reference numeral 60 denotes a peak estimating unit for estimating a peak position and a value of a correlation value which is obtained by the correlation value calculating unit 57 and is compared with a threshold value by the correlation value determining unit 58 and remains as a candidate. is there.

Next, the operation will be described. Here, FIG.
FIG. 5 is a flowchart showing the flow of processing of the image processing apparatus according to the first embodiment, and FIG. 5 is an explanatory diagram conceptually showing the principle of peak estimation according to the first embodiment. Also in this case, the user generates the template data 54 in advance. The template data 54 includes a template image, a threshold value of a correlation value, a start coordinate of a search area, an image size, and the like. When the process is started, template data is loaded in the same manner as in the first embodiment (step ST61), and an input image of the search area is input to the image memory 53 (step ST62). Next, the correlation value of the input image with the template image in the template data 54 is calculated by the correlation value calculation unit 57 for every several pixels (Step S).
T63), the maximum value is detected (step ST64)
The comparison with the threshold value is performed in the correlation value determination section 58 (step ST65). As a result of the comparison, when the correlation value at that coordinate is higher than the surrounding correlation value, it is determined that there is a possibility that a true peak exists in the vicinity thereof, and it is left as a candidate for the object recognition position (step ST66). ). When the processing is completed for all the search areas, the peak estimating unit 60 obtains the peripheral correlation values and coordinates (step ST67), and determines the peak position from the obtained peripheral correlation values and coordinates, and the own coordinates and correlation values. And a value (step ST6)
8).

The estimation of the peak is performed for each of the X axis and the Y axis. The X axis will be described with reference to 5 in the figure. Now, at a certain coordinate (sO in FIG. 5), it is assumed that sO is near the peak since it is higher than the correlation value on the left and right. Next, s_min
Since us1 and s_plus1 are compared and s_minus1 is larger, s_minus2 is obtained, and a peak is estimated from the coordinates of the four points and the correlation value. The estimation is similarly performed in the Y-axis direction. As a result of the estimation, when the estimated coordinate of the peak exceeds the range or the correlation value of the peak becomes negative, the peak cannot be estimated and is excluded from the candidates (step ST6).
9) Record only appropriate ones in the recognition result recording unit 59 (step ST70).

In the above reference example, the peak was estimated from the intersection of the two straight lines. However, a quadratic or cubic curve was applied by Lagrange interpolation using the correlation values of three or four neighboring points to obtain a peak. May be obtained.

[0036]

As described above, according to the present invention, the presence of an object is determined by calculating the sum of the variance values in the same area as the size of the template for the input image obtained by capturing the search area. Since it is possible to narrow down the candidate points where the object is present from the values, and it is not necessary to calculate the correlation values of all the regions, there is an effect that the processing time can be shortened, and especially the background Can greatly reduce the search time for search images in which only the target object is installed, and detect the position where the target object does not exist based on the variance value, so that highly reliable recognition can be performed. effective.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of an image processing device according to a first embodiment of the present invention.

FIG. 2 is a flowchart showing a flow of processing in the image processing apparatus according to the first embodiment.

FIG. 3 is a diagram illustrating a configuration of an image processing apparatus according to Embodiment 1 of the present invention;

FIG. 4 is a flowchart illustrating a flow of processing in the image processing apparatus according to the first embodiment.

FIG. 5 is an explanatory diagram illustrating the principle of peak detection in Reference Example 1.

FIG. 6 is a conceptual diagram showing a conventional basic template matching method.

FIG. 7 is a conceptual diagram showing a conventional coarse / fine search method.

FIG. 8 is an explanatory diagram showing a speed-up effect by a conventional coarse / fine search.

FIG. 9 is an explanatory diagram showing a template matching method by a conventional image processing apparatus.

FIG. 10 is a configuration diagram showing another conventional image processing apparatus.

FIG. 11 is a configuration diagram showing still another conventional image processing apparatus.

FIG. 12 is a flowchart illustrating a method for obtaining pixel accuracy in still another conventional image processing apparatus.

[Description of Signs] 40 objects, 41 image input unit, 53 image memory,
54 template data, 55 variance calculator, 56
A variance value determination unit, 57 a correlation value calculation unit, 58 a correlation value determination unit, 59 a recognition result recording unit;

 ────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Manabu Hashimoto 8-1-1, Tsukaguchi-Honmachi, Amagasaki-shi F-term (reference) 5L096 AA06 FA33 FA34 JA03 JA09 JA11 JA26

Claims (1)

[Claims] An image processing apparatus for performing recognition by a template matching method using template data prepared in advance, wherein a variance for calculating a variance value of an input original image in the same area as the template image in the template data is provided. Image processing, comprising: a value calculation unit; and a variance value determination unit that determines whether an object is present by comparing the variance value obtained by the variance value calculation unit with a threshold value. apparatus.