JPH06100448B2 - Image pattern matching method and apparatus - Google Patents

Description

The present invention relates to an image pattern matching method for visual recognition such as discrimination of parts in an automatic robot for industrial product automatic assembly equipment and inspection / maintenance work. And the device.

(Prior Art) Conventionally, as this kind of image pattern matching method,
There was a mask pattern matching method. In the mask pattern matching method, as shown in FIG. 9, when the mask pattern (b) is overlaid on the input image (a) captured by a TV camera or the like, and the input image and the mask pattern have the same shape, This is a method for discriminating or positioning an object in an input image by utilizing the fact that the degree of overlap is high and the degree of overlap is low when the shapes are different.

(Problems to be Solved by the Invention) However, in an input image obtained by an optical image pickup means such as a television camera, depending on shooting conditions such as lighting,
Noise is often mixed and the image is deformed, and the degree of overlap between the input image and the mask pattern is reduced according to the deformation of the input image and the amount of noise. Therefore, as shown in Fig. 10, the input image containing noise and deformation is subjected to overlay matching with two mask patterns of similar shapes A and B, and which mask pattern and the input image match. When discriminating, there is a small difference in the overlap of the input images with respect to the two mask patterns, and there is a drawback that correct discrimination becomes difficult.

The present invention aims to remedy this drawback.

(Means for Solving Problems) In order to solve these problems, the present invention defines a small area for dividing an image on an input image and an edge image created from the input image, and divides the divided small area. An average brightness feature, an edge feature, and an edge distance feature are calculated for each area, and these features are combined to perform matching processing.

(Operation) By performing the above-described processing, correct matching can be obtained by the complementary property of each feature, and the correct discrimination of the object can be performed even when the input image contains noise or deformation or the shape is similar. It is possible.

(Example) The present invention will be described below with reference to Figs. 1 to 8. FIG. 1 is a block diagram showing an example of an apparatus for carrying out the present invention, where 1 is a television camera, 2 is an AD converter, and 3 is
Is an input image storage unit, 4 is an edge image extraction unit, 5 is an edge image storage unit, 6 is an input image division address generation unit, 7 is an edge image division address generation unit, 8 is an average brightness feature extraction unit, and 9 is an edge. A feature extraction unit, 10 an edge distance feature extraction unit, 11 an image feature storage unit, 12 a mask image feature storage unit,
13 is a matching processing unit, and 14 is an output terminal.

The television camera 1 captures an object image and outputs an electric signal having a value corresponding to the brightness as a raster scanning signal. AD
The conversion unit 2 converts the raster scanning signal into a digital signal and writes the image data in the input image storage unit 3.

Hereinafter, a case where an image as shown in FIG. 2 is stored in the image storage unit 3 will be described. An edge image extraction unit 4 creates an edge image as shown in FIG. 3 on the input image by a known edge extraction process as one of image processing techniques, and outputs the result to the edge image storage unit 5. .

The input image division address generation unit 6 calculates an address of a small area formed by dividing the input image into a plurality of pieces, and supplies an address signal to the input image storage unit 3 to calculate the average brightness of the image data in each small area. The data is sequentially read out to the feature extraction unit 8. It is appropriate to define the small areas so that adjacent small areas overlap each other little by little, but for simplicity, here, the fourth area is used.
As shown in the figure, the small areas do not overlap each other, and 4 ×
A case will be described in which 16 small areas C ₁₁ , C ₂₁ , ..., C ₄₄ are defined by division of 4. The edge image division address generation unit 7 calculates the address of the small area into which the edge image is divided in the same manner as the input image division address generation unit 6, supplies the address signal to the edge image storage unit 5, and The edge image is sequentially read to the edge feature extraction unit 9. Here are shown in FIG. 5 such _{C '11, C' 21,} ..., 16 pieces of small areas C _'44 is determined. If the setting of the small area is the same for the input image and the edge image, the image division address generators 6 and 7 can be shared. An average brightness feature extraction unit 8 obtains an average value of image brightness for each of the small areas into which the input image is divided, and outputs the result to the image feature storage unit 11. That is, the brightness distribution of one small area Cij is Iij (x,
y) and the number of pixels in the small area Cij is n pixels and is approximately m pixels, the average brightness Mij of the pixels for the small area Cij is Calculate by Similar processing is applied to all small areas C ₁₁ , C ₂₁ ,
By doing this for C ₄₄ , an average brightness feature consisting of M ₁₁ , M ₂₁ , ..., M ₄₄ is calculated.

Reference numeral 9 denotes an edge feature extraction unit that obtains, as an edge feature, the total number of pixels having an edge strength equal to or greater than a predetermined threshold T ₁ included in a small region of a divided edge image, or a ratio to all pixels in the small region, The result is output to the image feature storage unit 11. Now, if the edge intensity distribution of one small area C′ij of the edge image is represented by I′ij (x, y), the edge feature Eij for the small area C′ij having vertical n pixels and horizontal m pixels is shown.
To Or Ask by. All small regions C the same process _'11, C'
_21, ..., C 'by performing the _44, edge characteristic E _11, E
₂₁ , ..., E ₄₄ is calculated. The edge feature is a feature amount that expresses the complexity of the contour shape of the object.

An edge distance feature extraction unit 10 reads the edge features of each small area from the image feature storage unit 11, and for each small region, exists in eight directions around the small region and has edge features of a threshold value T ₂ or more. Calculate the distance to the closest small area,
It is output to the image feature storage unit 11 as an edge distance feature. For example 'E'ijT ₂ become small areas C around 8 direction focusing on _{22' 21} small regions C in FIG. 6, C _'31,
C and was _'42, C' _44. Now the distance l between the two small areas C'ij and C'kl Determining _{if, C '21, C' 31} , C '42, C'' distance l (C up to _{21' 22,} C closest small area C in _{44 '21)} = 1 C' edge distance in ₂₂ by Calculate as feature L ₂₂ . By performing the same process for all C'ij, the edge distance features L ₁₁ , L ₂₁ , ..., L ₄₄ are obtained. The edge distance feature is
It is a feature quantity expressing whether or not an edge exists near the small area.

In this embodiment, the edge distance feature is obtained from the small areas existing in the eight surrounding directions, but more generally, the edge distance feature is selected from the small areas existing in the N direction around the small area C'ij of interest. It is also possible to obtain the closest small area having a feature of T ₂ or more and the closest. Instead of using the distance to the closest small area as the edge distance feature, the distance to the farthest small area or the sum or average of the distances to the small areas whose edge features are T ₂ or more in the N direction is used as the edge distance feature. It is also possible to use

An image feature storage unit 11 includes average brightness features M ₁₁ , M ₂₁ , ..., M ₄₄ calculated by the average brightness feature extraction unit 8 and edge features E ₁₁ , E calculated by the edge feature extraction unit 9. ₂₁ ,…, E
₄₄ and the edge distance features L ₁₁ , L ₂₁ , ... L ₄₄ calculated by the edge distance feature extraction unit 10 are stored. Hereinafter, the average brightness feature, the edge feature, and the edge distance feature are collectively referred to as an image feature.

Reference numeral 12 denotes a mask image feature storage unit, which corresponds to an object in an input image and which is an image feature calculated from a mask image (corresponding to a mask pattern in a conventional method) of an object set as a criterion for discrimination. Memorize as.
The mask image feature is a dictionary for discriminating an object in the input image and is created in advance.

A matching processing unit 13 performs matching processing between the image feature obtained from the input image and the mask image feature, and determines whether or not the input image feature and the mask image feature match. There are several possible methods for matching processing. For example, if the input image features are Mij, Eij, Li
j, the mask image features are M'ij, E'ij, L'ij Calculate the sum of the absolute values of the differences represented by. For predetermined thresholds T _M , T _E , T _L , d _M ≤ = T _M , d _E ≤T _E , d _L
It is possible to determine that the input image feature and the mask image feature match when ≦ _TL is satisfied at the same time, and to determine that they do not match otherwise.

In the present embodiment, the case where the input image and the mask image have the same size and the image feature is calculated in both 4 × 4 small areas has been described, but the input image is large and the object to be discriminated in the input image The present invention can be applied even when the corresponding mask image is small. In this case, as shown in FIG. 7, the number of divisions of the small area is determined so that the size of the small area determined by the division of the input image is equal to the size of the small area determined by the division of the mask image, and the mask image is displayed on the input image. By performing the image feature matching process while sequentially shifting the positions of, it is possible to find the position where the input image and the mask image overlap each other as the position of the object to be discriminated.

Further, in the present embodiment, in order to determine whether the input image and the mask image match, the average brightness feature, the edge feature,
The matching process is performed using all the edge distance features, but depending on the shape of the object to be discriminated, the matching process is performed using only two appropriate features of the average brightness feature, the edge feature, and the edge distance feature. It is also possible.
In this case, the amount of calculation involved in the matching process can be reduced. FIG. 8 shows an example of a processing flow chart. 8th
In the representation of the figure, the storage of the input image and the extraction / storage of the edge image, the extraction of the average brightness feature and the extraction of the edge feature are
Although shown as being performed sequentially, each can be performed in parallel.

(Effects of the Invention) As described above, in the present invention, the input image is divided into small areas, and the average brightness feature expressing the brightness of the object for each small area and the complexity of the contour shape of the object are expressed. The edge distance feature expressing the spatial spread of the image feature is calculated as the image feature by the edge feature and the distance to the edge region existing in the vicinity of the small region, and the feature is stored in advance. Performs matching processing with mask image features. Since the average brightness feature, the edge feature, and the distance feature represent the shape of the object from different angles, an object that cannot be identified only by the brightness of the object can be identified by the edge feature and the edge distance feature. In addition, even if the edge feature and the edge distance feature cannot be used alone, the average brightness feature can be used together to make the distinction. For example, the complementary nature of each feature enables highly accurate image pattern matching processing.

In order to realize highly accurate discrimination of an object, the ability to correctly discriminate an object having a similar shape is required even when the input image contains noise or deformation. The example of FIG. 10 is a problem of determining which of the two mask images (mask patterns) A and B the input image including the deformation matches.
The difference between the mask images A and B is the presence or absence of a hole in the center part of the image, but since the noise and deformation included in the input image are large compared to the area of the hole, conventional methods, especially matching using only the average brightness feature In processing, correct discrimination is difficult.
However, in the present invention, the mask image A is a small area.
By has a hole in the C ₂₂ (and C _'22) within,
Edge characteristic E ₂₂ for C _'22 becomes a large value. 'Surrounding the small region C of _{22' 21} C Thus, C _'12, C' _23,
Are all edge distance characteristics 1 in C _'32. That is, the edge distance feature has a large change in the value of the same feature in the surrounding small regions depending on whether or not there is a small region having an edge nearby, so that the mask images A and B have the same overall shape, Even if the shape in one area differs depending on the presence or absence of a hole, the edge distance characteristics of the mask image A and the mask image B are significantly different, and correct discrimination is possible even if the input image contains noise or deformation.

When the shape of the object to be discriminated is complicated, a small area that divides the input image and the mask image into smaller areas is defined, and by calculating the image characteristics according to the shape of the target object, high-precision discrimination ability can be obtained. Can be realized.

It should be noted that according to the present invention, the image feature is calculated for a small area determined by the division of the image, so that the capacity thereof is extremely small compared to the case where the image is stored as the data as it is. Therefore, the memory for storing the input image feature and the mask image feature has a small capacity, and the calculation amount required for the matching process is reduced.

[Brief description of drawings]

FIG. 1 is an example of a block diagram of a device of the present invention, FIG. 2 is an input image example for explaining the present invention, FIG. 3 is an edge image example, FIG. 4 is a small area example for the input image, FIG. 5 is an example of a small area for an edge image, FIG. 6 is an explanatory diagram of edge distance feature calculation, FIG. 7 is an example of matching processing by scanning a mask image, and FIG. 8 is a flowchart showing a flow of processing. FIG. 9 shows a conventional mask pattern matching method, and FIG. 10 shows an example of mask pattern matching with an input image containing noise and deformation. 1 ... TV camera, 2 ... AD conversion unit, 3 ... input image storage unit, 4 ... edge image extraction unit, 5 ... edge image storage unit, 6 ... input image division address generation unit, 7 ... Edge image division address generation unit, 8 ... Average brightness feature extraction unit, 9 ... Edge feature extraction unit, 10 ... Edge distance feature extraction unit, 11 ... Image feature storage unit, 12 ... Mask image feature storage unit , 13 …… Matching processing block, 14 …… Output terminal.

Claims (2)

[Claims] 1. A method for matching image data of an object obtained by an imaging device with a standard pattern, wherein a small area is formed by dividing the image data and an edge image obtained by subjecting the image data to edge extraction processing. From the small areas defined by dividing the image data, the average brightness feature that is the average value of the brightness of the images included in each small area is calculated. Is an edge feature that is the total number or the ratio of the pixels of the edge image that are included in each small area and whose edge strength is equal to or greater than a predetermined threshold, and other small features whose edge features are greater than or equal to the defined threshold. An edge distance feature, which is the distance to the area, is calculated, and the average brightness feature, the edge feature, and the edge distance feature are combined to form a predetermined standard pattern. Image pattern matching method and performing a matching process between. 2. An image pickup apparatus for inputting an object image, an AD conversion section for converting an image signal output from the image pickup apparatus into image data of a digital signal, an input image storage section for storing the image data, and an input image An edge image extraction unit for creating an edge image, an edge image storage unit for storing the edge image, an image division address generation unit for generating the addresses of the input image and a small region for dividing the edge image, and an average for each small region Average brightness feature extraction unit for calculating brightness feature, edge feature extraction unit for calculating edge feature for each small region from edge image, edge distance feature for calculating edge distance feature for each small region from edge feature Extraction unit, image feature storage unit for storing average brightness features, edge features, and edge distance features for the input image, preset as a standard pattern Mask image feature storage unit for storing a feature quantity with respect to the mask image, an image pattern matching apparatus characterized by having a matching unit that performs matching processing with the image features calculated from the mask image features of the input image.