JPH0420502B2 - - Google Patents

Description

[Detailed Description of the Invention] Purpose of the Invention (Field of Industrial Application) The present invention relates to image pattern matching for performing image recognition for positioning parts, detecting defects, etc. in automatic assembly of industrial products and product inspection work. It is about the method.

(Prior Art) A typical conventional image pattern matching method was as follows.

FIG. 1 is a diagram showing an example of an input image, and FIG. 2 is a diagram showing the result of labeling by connected component labeling processing.

First, connected component labeling processing is applied to an input image as shown in FIG. 1, and labeled regions as shown in FIG. 2 are extracted.

Furthermore, a plurality of geometric features such as area and perimeter are calculated for this label region, and differences from the same features extracted in advance from an image of a standard object are determined. This method determines the label region where the sum of the differences between the features is the minimum as the region where the recognized object exists (object region). (For example, Kura, “Current status of commercially available visual devices for robots”, Journal of the Robotics Society of Japan, Vol. 1, No. 4, pp. 309-311, 1983) (Problems to be solved by the invention) However, optical The input image obtained by the imaging means is often subject to noise contamination or image deformation depending on the shooting conditions such as illumination, and the labeled object area is deformed as shown in Figure 2. The image feature values such as area and perimeter that are compared with the standard pattern are affected.
Correct discrimination is often difficult.

Moreover, FIG. 3 is a diagram showing two examples of target objects.

FIG. 4 is a diagram showing an example of an input image when objects overlap.

In the case of an input image in which both objects A and B shown in Fig. 3 overlap as shown in Fig. 4, the existing regions of the two objects will have the same label and will be mistakenly recognized as the existing regions of different objects. You will be excluded. Furthermore, the labeling process for connected components has problems such as high computational time costs.

Structure of the Invention (Means for Solving the Problems) In order to solve these drawbacks, the present invention does not extract object regions by labeling connected components as preprocessing for object discrimination, but performs recognition. Obtain the upper and lower limits of multiple image features from a predetermined mask image representing the target object, and determine the number of images that produce image features that fall within the feature definition range determined by the upper and lower limits. The object image area to be detected is counted, the degree of matching between the mask image and the object area is calculated from the counted value, and matching is performed using this matching.

(Function) In this way, even if the input image contains noise or deformation, or if objects overlap, it is possible to stably identify and position objects using a simple algorithm. It is something to do.

(Embodiment) FIG. 5 is a block diagram of an embodiment showing the configuration of an apparatus for carrying out the present invention, in which 1 is a television camera, 2 is an AD conversion section, 3 is an input image storage section, and 4 is a block diagram of an embodiment of the present invention.
5 is a feature extraction section, 5 is an image feature storage section, 6 is a standard feature pattern storage section, 7 is a matching processing section, and 8 is an output terminal. Furthermore, the matching processing section 7-
1 feature determination unit, 7-2 determination value storage unit, 7-3
It consists of a judgment value evaluation section.

The television camera 1 takes an image of an object (a photograph or a drawing may be used instead of the actual object) and outputs an electric signal having a value corresponding to the brightness and color of the object as a raster scanning signal. The AD conversion section 2 converts the raster scanning signal into a digital signal and writes the image data into the input image storage section 3.

Although the image is generated by a television camera here, the present invention is also applicable to image data generated by other image generation means such as a VTR device and a facsimile.

A case where an image as shown in FIG. 1 is stored in the input image storage section 3 will be described below.

In the standard feature pattern storage section 6, standard feature patterns that serve as standards for object discrimination are set as follows.

FIG. 6 is a diagram showing an example of a mask image.

That is, as shown in FIG. 6, a plurality of image features f ₁ (x, y), ..., _fo are extracted from a mask image specified by a rectangular area M (area size Lx×Ly) surrounding the object.
(x, y) is calculated.

For example, the following known image features, namely, the brightness feature in which the brightness level value is normalized by a certain value, and the three primary color signal components of red (R), green (G), and blue (B) are used. Level values, color component ratios, normalized edge intensity features obtained by edge extraction processing, etc.
Various image features of the image are calculated.

Here, upper and lower limit values within the rectangular area M are set for the brightness feature and the edge strength feature.

FIG. 7 is a diagram showing an example in which upper and lower limit values are set for edge intensity features obtained from edge images.

Generally, a series of upper limit values (u ₁ ,..., u _o ) and a series of lower limit values (l ₁ ,..., _lo ) for each pixel feature within a rectangular area M, and each pixel feature within the area M. A series of representative values of the number of pixels (s ₁ , . . . , s _o ) and area sizes Lx, Ly within the range of the upper and lower limits of are stored in the standard feature pattern storage unit.

Here, each value of the series of representative values (s ₁ ,..., s _o ) is set to a value that does not change with the area size, so even if each element is replaced with the value divided by the area Lx × Ly of the area size, good.

Using this standard feature pattern, object discrimination and positioning are performed below.

The feature extraction unit 4 extracts the same image features f ₁ (x, y),..., _fo (x,
y) is calculated from the input image and output to the image feature storage section 5.

Next, the matching processing section 7 will be explained.

FIG. 8 is a diagram showing scanning of a standard feature pattern of a mask image of an input image using a rectangular area R.

The feature determination unit 7-1 first uses a series of upper limit values (u ₁ ,..., u _o ) and a series of lower limit values (l ₁ , u o ) of mask image features stored in the standard feature pattern storage unit 6 .
…, l _o ) and a series of representative values of the number of pixels (s ₁ , …, s _o )
and area sizes Lx, Ly, and set a rectangular area R with the same size as the mask image in the input image as shown in FIG. 8, and image features f ₁ (x, y),...,
Pixels in region R where f _o (x, y) satisfies l ₁ ≦ f ₁ (x, y) ≦ u ₁ , ..., l _o ≦ f _o (x, y) ≦ u _o ... (1) Count numbers.

Let the series of values counted for each image feature be (N ₁ , ..., _No ).

Here, each image feature fk (x, y) (k=1,...,
Regarding n), the degree of coincidence between the rectangular region R and the mask image is calculated. This degree of coincidence can be expressed using, for example, the following degree of coincidence function h(Nk) (k=1, . . . , n).

h(Nk) = 1-α | Nk-Sk | ...(2) or = 1-α(Nk-Sk) ^2Here , h(Nk) takes a value between 0 and 1, and Nk
It is assumed that the maximum matching degree is 1 when is equal to Sk.

On the other hand, h(Nk) decreases as Nk moves away from Sk. However, it is assumed that the value of α is set appropriately.

In addition, the standard feature pattern storage unit 6 stores the series (s ₁ ,..., s _o ) as a value that does not change to the mask image area size.
When reading from , h(Nk) can be calculated by dividing Nk (k=1, . . . , n) by the rectangular area area Lx×Ly.

Next, using the matching degree for each image feature, all image features f ₁ (x, y),..., f _o (x, y)
The degree of coincidence between the rectangular region R and the mask image when applying is calculated.

This degree of matching can be expressed as, for example, each pixel feature fk (x, y)
Weighted linear sum _〓〓 using matching function h(Nk) for
It is possible to set w _k h (Nk).

Here, the position of the rectangular region R is sequentially shifted from the upper left to the lower right at intervals of one pixel or several pixels, and the input image is scanned over the range where the object region may exist. Repeat the above matching process.

Then, the degree of coincidence calculated for each movement and the representative point coordinates (Px, Py) of the position of the rectangular region R are stored in the determination value storage section 7-2. However, storing all the degrees of coincidence here requires a huge amount of memory, so a threshold value of the degree of coincidence is usually set, and if it is less than the threshold, no storage is performed.

Next, the judgment value evaluation unit 7-3 detects the representative point (Px ⁰ , Py ⁰ ) at the position of the rectangular area R that gives the maximum value of the count value stored in the judgment value storage unit, and outputs it to the output terminal 8. Representative point of detected area (Px ⁰ , Py ⁰ )
and output the area sizes Lx and Ly.

Here, it is possible to determine that the area that gives the highest degree of matching is the area where the object to be determined exists.

FIG. 9 is a diagram showing the processing flow of the present invention,
A flowchart of the above matching process is shown.

Next, a method of dividing the mask image to define partial mask images (small areas of the mask image) and performing more detailed matching processing using the partial mask images will be described.

In this case, the processing contents of the standard feature pattern storage section 6 and the matching processing section 7 are different from the case where the image is not divided into partial mask images.

FIG. 10 is a diagram showing a mask image divided into partial mask images.

Here, for simplicity, as shown in FIG. 10, the mask image region M (region size Lx, Ly) is divided into 2×2, and the region size
A case will be described in which partial mask images corresponding to four areas M ₁₁ , M ₂₁ , M ₁₂ , and M ₂₂ of ×Ly/2 are set. Generally, m×n division is performed.

The standard feature pattern storage unit 6 stores image features f ₁ in each partial mask image region Mij (i, j=1, 2).
(x, y), ..., f _o (x, y) upper limit value series (u ₁ ,
…, u _o ), lower limit series (l ₁ , …, _lo ) and area
A series of representative values of the number of pixels within the range of the upper and lower limits of each image feature in Mij (s ₁ ,..., s _o ), region sizes Lx, Ly, partial region sizes Lx/2,
Ly/2 is stored.

On the other hand, the image feature storage unit 5 stores the image feature f ₁ (x,
y),..., f _o (x, y) are stored.

FIG. 11 is a diagram showing scanning of a partial rectangular area of an input image.

Here, the feature determining unit 7-1 in the matching processing unit 7 determines the image feature f ₁ as shown in FIG.
(x, y),..., f _o (x, y) within a rectangular area R of size Lx×Ly (i=1,
2) Each partial region Rij (i=,2) equal to the size of
, the number of pixels satisfying l ₁ ≦f ₁ (x, y) ≦u ₁ , . . . , l _o ≦ _fo (x, y) ≦u _o (3) is counted.

Here, the series of values counted for each image feature is assumed to be (N ₁ , . . . , _No ).

Using this series of count values, the degree of coincidence between the rectangular area R and the mask image area M is calculated. This degree of matching can be set, for example, as follows.

First, each image feature f _k (x, y) (i=,...,n)
For each partial region Rij and partial mask image region
Calculate the degree of match with Mij. This degree of coincidence can be set by calculating, for example, replacing Nk and Sk with Nk and Sk in equation (2), respectively.

Next, using this degree of matching, all image features f ₁
The degree of coincidence between each partial region Rij and the partial mask image region Mij when (x, y), ..., _fo (x, y) is applied is calculated.

This degree of matching can be set, for example, as the weighted linear sum of each image feature 〓〓w _k h (Nk) (where w _k is a weighted coefficient).

Finally, the degree of coincidence between the rectangular area R and the mask image area M is set, for example, as follows.

That is, the partial region Rij and the partial mask region Mij
Matching degree H (Nk) and each partial mask image area
Using the weighting coefficient w _ij for Mij, it is possible to set the weighted linear sum 〓〓〓w _ij H(Nk).

As described above, by calculating the degree of coincidence between region R and mask image region M each time the position of region R is moved from the upper left to the lower right at intervals of one pixel or several pixels, it is the same as when not dividing into partial mask images. matching processing is possible.

In the above embodiment, a method of counting the degree of matching on a pixel-by-pixel basis has been described, but as the input image size increases, it is expected that the number of repeated operations for performing matching will increase significantly.

This is done by dividing an image into small areas as shown below, and calculating the average value of brightness, the number of pixels in the small area (edge density) that has an edge intensity above a certain threshold, etc. By calculating the features and counting the small areas that satisfy the domain of the image features,
The amount of repetitive calculations can be reduced.

FIG. 12 is a diagram showing an example of a mask image divided into small regions.

For example, as shown in FIG. 12, a case where a mask image (mask image area size Lx×Ly) is divided into 4×4 to define 16 small areas will be described.

Examples of image features calculated from this small region include the average value of brightness, the number of pixels in the small region (edge strength) having an edge strength equal to or higher than a certain threshold value, and the like.

A series of upper and lower limit values for each image feature calculated in advance for each small region of the mask image region M, and a representative of small regions within the range of the upper and lower limits of each image feature within the region M. The value series and mask image area sizes Lx and Ly are calculated in advance.

FIG. 13 is a diagram showing scanning of a rectangular area on an input image divided into small areas.

As shown in FIG. 13, the input image is divided into parts having the same size as the small regions set in the mask image, and a plurality of image features of the same type as the mask image are calculated for each small region.

Furthermore, a rectangular region R of the same size as the mask image in the input image is set, and the number of small regions between the upper and lower limits of the corresponding feature value of the mask image is counted in the rectangular region R. Similar to the case where the screen is not divided into small areas using the count value, the degree of coincidence between the mask image and the rectangular area R is calculated.

Furthermore, the rectangular area R is sequentially shifted from the upper left to the lower right in the input image at intervals of one small area or several small areas,
The degree of coincidence is calculated for each rectangular region R, and the region where the object to be determined exists is determined.

Here, the amount of calculation to calculate the number of small areas between the upper and lower limits of the feature value corresponding to the mask image is compared to the amount of calculation to calculate the number of pixels, and is calculated by {1/(Lx /4)}×{1/(Ly/4)}, and the larger Lx and Ly are, the more the amount of calculation is reduced. Furthermore, the amount of calculation required to scan the rectangular region R in the input image is also reduced, and the amount of repeated calculations required for matching is greatly reduced as a whole.

FIG. 14 is a diagram showing an example of a target object with a complicated pattern on its surface.

This considers objects whose brightness differs from one object to another, and furthermore, the complexity of the pattern on the object surface also differs from one object to another.

When conventional connected component labeling processing is applied to this object, it becomes difficult to set a threshold to distinguish between the object and the background and binarize it, and the labeling processing becomes difficult depending on the complexity of the pattern. There were some problems.

According to the present invention, for example, the upper and lower limits of the brightness feature and the edge intensity feature are set, and the number of pixels at a certain brightness level and the number of pixels within a certain edge intensity range are determined in the object region. Discrimination and positioning can be performed simply by determining the .

In this way, by setting a very simple standard feature pattern consisting of a series of upper and lower limit values for image features, objects can be identified and positioned.Furthermore, when you want to add another image feature, you can use the image that has already been set. This is an image pattern matching method that only requires adding the upper and lower limit values of new image features to the series of upper and lower limit values of the features, and can easily improve the discrimination ability even for complex objects. ,
Further, it is easy to implement the hardware and can be constructed at low cost.

FIG. 15 is a diagram showing an example of an input image when objects overlap.

Even if objects overlap, as shown in FIG. 15, by counting the degree of coincidence for the rectangular region R, it is possible to distinguish, although the degree of coincidence is somewhat reduced.

Next, the effect of the matching method when the mask image is divided into partial mask images will be explained.

FIG. 16 is a diagram showing an example of a target object having different brightness levels within the object region.

FIG. 17 is a diagram showing an example in which a mask image is divided into partial mask images.

As shown in Figure 16, the brightness level differs depending on the area within the same object area (brightness characteristics
Consider the cases I ₁ , I ₂ , I ₃ ).

In this case, it is impossible to simply set the brightness range of the entire mask image area;
Four partial mask images (M ₁₁ , M ₂₁ , M ₁₂ ,
Discrimination is possible by dividing into M ₂₂ ).

That is, partial mask images M ₁₁ , M ₂₁ , M ₁₂ , M ₂₂
A series of upper limit values (u _I1 , u _I2 , u _I3 , u _I3 ) and a series of lower limit values (l _I1 , l _I2 , l _I3 , l _I3 ) of the brightness feature are set as standard feature patterns for each. However, it becomes possible to distinguish by applying this method.

FIG. 18 is a diagram showing an example of a mask image of a target object with a hole.

According to the present invention, when an object whose hole matches the upper left corner of the object shown in FIG. 16 is identified as shown in FIG _.
The presence or absence of a hole is detected by detecting the level area M ₁₁ and using the edge intensity feature for the detected area M ₁₁ in the next step and counting the number of pixels within the range of a certain edge intensity feature. It is also possible to perform a stepwise matching process, which improves the efficiency of the matching process.

Next, the image pattern matching method according to the present invention can be expected to perform stable matching processing even against noise and image deformation caused by fluctuations in imaging conditions.

For example, consider a case where an object is determined based on the circumference of the object.

Conventionally, methods for determining the perimeter of an object include methods that perform edge extraction processing on an input image and track the edges of the object,
There is a method of finding the boundaries of closed regions obtained by labeling connected components.
However, in this method of tracking edges or boundaries, if the input image is subjected to noise or deformation, the perimeter changes depending on the amount of noise or deformation.

19-1 and 19-2 are diagrams showing examples of input images including noise and deformation, and FIG. 20-1,
FIG. 20-2 is a diagram showing the boundaries of an object obtained by tracking edges or boundaries.

When distinguishing between objects with and without a slight depression in the upper center as shown in Figs. 19-1 and 19-2, as shown in Figs. 20-1 and 20-2, There is almost no difference in circumference between the two objects, and the two objects may become indistinguishable.

In the image pattern matching method according to the present invention, the degree of matching is evaluated based on the number of pixels whose edge strength is within a certain range, so even if noise and deformation are included, the proportion of the total number of matched pixels is limited. It is being Therefore, it can be said that the influence of noise deformation is less compared to the method of tracking edges and boundaries and calculating the perimeter.

Effects of the Invention As explained above, the present invention does not extract an object region by labeling processing of connected components as preprocessing for object discrimination, but extracts a plurality of object regions from a predetermined mask image representing the object to be recognized. Obtain the upper and lower limit values of the image features, count the number of images that produce image features that are within the feature definition range determined by the upper and lower limit values for the object image area to be discriminated, and calculate the number of images using the counted values. This method calculates the degree of coincidence between the mask image and the object region, and performs matching using this coincidence, so it is stable even when the input image contains noise or deformation, or when there is overlap between objects. , and has the advantage that objects can be identified and positioned using a simple algorithm.

[Brief explanation of drawings]

FIG. 1 is a diagram showing an example of an input image, FIG. 2 is a diagram showing a result of labeling by connected component labeling processing, and FIG. 3 is a diagram showing an example of two target objects.
Fig. 4 is a diagram showing an example of an input image when objects overlap, Fig. 5 is a block diagram of an embodiment showing the configuration of an apparatus for implementing the present invention, and Fig. 6 is an example of a mask image. Figure 7 is a diagram showing an example of setting upper and lower limit values for the edge intensity feature obtained from an edge image, and Figure 8 is a scan of a standard feature pattern of a mask image of an input image using a rectangular area R. 9 is a diagram showing the processing flow of the present invention, FIG. 10 is a diagram showing a mask image divided into partial mask images, and FIG. 11 is a diagram showing scanning of an input image by a partial rectangular area. , FIG. 12 is a diagram showing an example of a mask image divided into small regions, FIG. 13 is a diagram showing scanning of a rectangular region on an input image divided into small regions, and FIG. 14 is a diagram showing an example of a mask image divided into small regions. FIG. 15 is a diagram showing an example of an input image when objects overlap, and FIG. 16 is a diagram showing an example of a target object with different brightness levels within the object area. Fig. 17 is a diagram showing an example of dividing a mask image into partial mask images, Fig. 18 is a diagram showing an example of a mask image of a target object with a hole, Fig. 19-1,
FIG. 19-2 is a diagram showing an example of an input image including noise and deformation, and FIGS. 20-1 and 20-2 are diagrams showing boundaries of objects obtained by tracking edges or boundaries. 1...TV camera, 2...AD conversion unit, 3...
...Input image storage unit, 4...Feature extraction unit, 5...Image feature storage unit, 6...Standard feature pattern storage unit,
7...Matching processing unit, 7-1...Characteristic determination unit, 7-2...Judgement value storage unit, 7-3...Judgement value evaluation unit, 8...Output terminal.

Claims (1)

[Scope of Claims] 1. An image of a rectangular area surrounding an object to be discriminated is used as a mask image, pixels or small areas of the mask image are used as a partial mask image, and a plurality of types of image feature values are set for each partial mask image. Along with asking,
For each image feature type, an upper limit value and a lower limit value of the image feature value are set in advance, and the number S of partial mask images included in the range of the image feature value included in the mask image is set; Regarding the input image, the area with the same size as the mask image is a rectangular area, the area with the same size as the partial mask image is a rectangular partial area, and as in the case of the partial mask image, for each image feature type, the rectangle is The image feature value of the partial area is calculated, the image feature value is within the range between the upper limit value and the lower limit value, and the number N of partial mask images within the rectangular area is calculated, and the number N of partial mask images and the number N of partial mask images are calculated. Obtaining a value of the degree of coincidence of the rectangular area using a degree of coincidence function that takes a value of 1 when the difference with the number of partial mask images S is equal, and determining a weighted linear sum of the degree of coincidence for the plurality of types of image features; An image pattern matching method characterized in that the rectangular area is sequentially scanned within the input image, and the image within the rectangular area at the scanning position where the weighted linear sum is the largest is determined as the area where the object to be determined exists.