JPS63226785A - Image feature extracting device - Google Patents

Abstract

PURPOSE:To easily transmit/receive I/O feature images through a feature image memory by connecting a feature extracting part for extracting a feature indicating the property of a boundary with a feature extracting part for extracting various attributes through common video buses and a system bus. CONSTITUTION:The feature extracting part 6 extracts an important feature indicating the property of a boundary such as a boundary curvature feature based on convolutional arithmetic and high-order statistic arithmetic. On the other hand, the feature extracting part 7 extracts various attributes in areas for complexity, expansion or the like. These feature extracting parts 6, 7 are connected with each other through common video buses 9-11 and the system bus 12. The feature extracting parts 6, 7 can access a feature image storing part 5 consisting of a feature image memory having multi-layer structure in common.

Description

DETAILED DESCRIPTION OF THE INVENTION (Technical Field of the Invention) The present invention relates to a device for extracting image features necessary for single character recognition and image recognition of mechanical parts, tools, and the like.

(Prior Art) The present inventors previously proposed the following image feature extraction method.

One is "Image feature extraction calculation method" (patent pending) by Meguro, Sano 9 Ishii. In this method, three different image memories A are selected from among the image memories.

Designate B and C to execute a weighted sum-of-products operation of the power of the difference between the pixel on the first image memory B and the pixel in the neighboring area in the upper part of the image memory, and store the result in the image memory C. Furthermore, a new image memory 6 is added, and only when the pixel value of the image memory 6 has a set value, the power of the difference between the pixel on the image memory 6 and the pixel in the upper neighboring area in the image memory 6 is calculated. A configuration is also adopted in which a weighted product-sum operation is executed and the result is stored in image memory C. This conventional method will be referred to as method A hereinafter.

The other is "Image feature extraction method" (patent pending) by Sano and Meguro 2 Ishii. With this method.

Two feature images calculated from an open image are used together as an input feature image for image feature extraction calculations and a feature image for calculation control, respectively, and calculation stop conditions and points of interest are defined for the feature values of the feature images for calculation control. Set the size of the field of view that gives a range of radial scanning centered on The scanning area is controlled by scanning, and feature extraction calculations are performed on the feature values on the scanning line of the input feature image in each scanning direction. This conventional method will be referred to as method B hereinafter.

In order to represent an object, both features related to the region of the object (region feature) and features related to the boundary surrounding it (boundary feature) are important. This is because region features are necessary when performing global object recognition in region units, and cotton boundary features are necessary when performing local object recognition in line segment units that determine the constituent elements of the region. Here, the extraction method using method B is suitable for region feature extraction, and the extraction method using method A is suitable for boundary feature extraction. Method B
, it is possible to calculate the integral amount and distance from a point within the region to a point on the boundary until the collision occurs, as well as the feature value of the collision point. Due to this property, the Institute of Electronics and Communication Engineers Technical Research Report P RU36
-90.1987: Sano, Meguro 2 As shown in Ishii's "Model matching using reference points that accumulate tentacle features," various area features such as spread, complexity, and aspect ratio are calculated. Ru. On the other hand, in method A, the amount of variation in direction (amount equivalent to curvature) can be calculated by using a feature image indicating whether it is a boundary or not as a control feature image and taking the quadratic statistics ff1 (dispersion) in the direction of the boundary. .

In addition, method A is capable of calculating various statistics including high-order statistics, but only scalar amounts are calculated because the product-sum operation is performed for the entire neighborhood region. On the other hand, in method B, multidimensional quantities are calculated for each scanning direction, but higher-order statistics are not calculated. Both multidimensional features and high-order statistical features are necessary to increase the throughput of image recognition systems.

In this way, method A and method B have mutually complementary properties, and if applied alone, it is not possible to completely extract the feature amounts necessary for image recognition.

Furthermore, in method B, if the boundary is partially interrupted and incompletely extracted due to the influence of illumination conditions, it may not be possible to stop scanning at the boundary depending on the scanning direction. Also,
If there is a noise component in the middle of scanning and its magnitude exceeds a threshold value that provides a scanning stop condition, scanning may be stopped at the position of the noise without reaching the boundary. As described above, method B has the drawback of being unstable due to noise and the like.

Furthermore, when the size of the weight mask in method A and the size of the field of view in method B become larger, the calculation cost becomes O(
There is also a problem that the number increases on the order of n'').

(Problems to be Solved by the Invention) The purpose of the present invention is to solve the problem of the limit of the feature extraction ability of the conventional method in which method A and method B were applied alone, and the problem of instability due to noise etc. in method B. By solving the problem of calculation cost when increasing the weight mask size and field of view size that are common to both methods, and by integrating both methods, we have developed an image feature extraction device that can calculate image features that are more stable and have high descriptive ability. It is about providing.

(Means for Solving the Problems) The present invention provides input feature image memories A and B, a reference feature image memory D, and an output feature image memory C of a first image feature extraction unit realized by method A; The input feature image memory 18 of the feature extraction unit B realized by method B and the output feature image memory are stored in the first and second
The most important feature is that the feature image storage section consists of a multi-layered feature image memory that can be commonly accessed by the image feature extraction section of the image feature extraction section, and that this is placed under the control of a single control section. It has the following configuration.

The present invention includes an input image storage unit that stores input image data; and stores a feature image obtained by a feature extraction process on the input image data or a feature image obtained by further performing a feature extraction process on the obtained feature image. a characteristic image storage section consisting of a plurality of characteristic image memories; specifying two image storage areas A and B for the characteristic image storage section or the input image storage section; Specify two characteristic image memories, and perform a weighted sum-of-products operation of the power of the difference in pixel value between the pixel at the target point on the image storage area B and the pixel in the corresponding neighboring area on the middle of the image storage area. , a first image feature extraction unit that executes the process only when the value of a pixel in the feature image memory C corresponding to the pixel in the neighboring region is equal to a preset value, and stores the result in the feature image memory C; 2 of the input images or feature images stored in the input image storage unit or feature image storage unit
The two images are respectively designated as the input feature image and the control feature image, and the scan stop conditions defined for the size of the field of view that gives the range of radial scanning and the feature value of the control feature image are set, and the The scanning area is controlled by scanning the image radially from a point of interest at the same position to a stopping point determined by either the field of view size or the scanning stop condition, and the scan area is scanned on the scanning line of the input feature image in each scanning direction. A feature image is generated from the feature values at each point of interest obtained by performing a feature extraction operation on the feature values of and sequentially moving the position of the point of interest on both feature images; a second image feature extraction unit for storing in a feature image memory specified by;
The input image storage unit and the feature image storage unit are connected to the first and second
This feature is characterized in that it has a configuration that can be used for Hongtong from the feature extraction section.

(Operation) Conventionally, the first image feature extraction section and the second image feature extraction section each have a feature image storage section.

Naturally, there is no data exchange between the first image feature extraction section and the second image extraction section. However, in the configuration according to the present invention, since a common feature image storage unit is set for the first image feature extraction unit and the second image feature extraction unit,
The output feature image of the first image feature extraction section can be treated as the input feature image or control feature image of the second image feature extraction section, and conversely, the output feature image of the second image feature extraction section can be used as the first image feature It can be used as the input feature images A and B of the extraction unit or as a reference feature image. The biggest advantage of this configuration is that by exchanging input/output information between the first image feature extraction section and the second image feature extraction section via the feature image storage section, each can be realized independently. For example, new features that could not be calculated and are effective for object recognition and image processing6
The image feature extraction unit takes each pixel point as a point of interest, scans radially from the point of interest, and calculates the total amount of distance reached in each scanning direction to the boundary for each pixel point, thereby calculating the amount of spatial expansion. A feature image representing the distribution of is obtained, but
The feature image of this spread amount distribution is used as the input feature image of the first image feature extraction section.

By calculating the molecule il, a new feature of the non-uniformity of the amount of spatial spread is calculated. This feature is a quantity that exhibits a high value in non-uniform pattern components, and is effective when detecting boundaries between different patterns. Furthermore, as a new feature extraction example, the first image feature extraction section calculates the variance amount in the edge direction at each pixel point to obtain a distribution of directional uniformity, but the second image feature extraction section calculates the dispersion amount in the edge direction. Let the feature image of the distribution of heterogeneity of be the input feature image,
By scanning this radially at each pixel point and calculating the reached product sum value for each scanning direction up to the boundary, the edge direction nonuniformity amount distribution for each scanning direction can be obtained. This feature is a multidimensional feature that expresses the amount of nonuniformity in the direction of pattern distribution within an object at each pixel point and in each scanning direction, and can realize highly efficient recognition in an image recognition system.

An additional advantage of the configuration of the present invention is that
The advantage is that the problem of instability due to noise and the like in the second image feature extraction section can be improved.

This is because the first image feature extraction section can realize the smoothing process as a first-order statistic (average value) calculation operation, and the first image feature extraction section smooths the smoothing process as a control feature image for the second image feature extraction section. This is because the problem of instability due to noise etc. can be improved by using the transformed feature image.

Furthermore, in the present invention, by adopting a configuration in which a data compression unit is newly added that performs smoothing processing on the input image or feature image, samples the result at a set sampling interval, and compresses the data. , the capacity of the feature image storage unit and the processing cost of feature extraction calculations can be reduced. Furthermore, the compressed feature image can reduce the influence of noise caused by illumination conditions and mid-scanning, which conventionally occurred in the second image feature extraction section, due to the smoothing effect.

(Embodiment) FIG. 1 is a block diagram illustrating a detailed embodiment of the present invention.

1 is an imaging device such as a television camera or scanner for inputting images; 2 is an input image converting device from analog to digital; This is an A/D converter that outputs data. Here, the reason why the input image storage unit 3 is composed of a plurality of input image memories is that, for example, when performing processing for detecting depth information of an object or processing for recognition of a three-dimensional object, it is necessary to This is because it is necessary to save the input image.

Reference numeral 5 denotes a feature image storage unit that stores the image feature extraction calculation results shown below, and is composed of a plurality of feature image memories from feature image memory (1) to feature image memory (N). At least three feature image memories are required in the present invention, and each feature image memory is coupled to at least three video buses (9 to 11) and a system bus 12 used for controlling operations. Similarly, each input image memory is also coupled to the video bus and the system bus.

Reference numeral 6 denotes a first image feature extraction unit (20 to 40 indicate constituent elements) which performs statistical feature calculation operations, and 7 a second image feature extraction unit which calculates multidimensional shape and phase features ( 4
1 to 59 indicate constituent elements).

Although this embodiment has a configuration in which a data compression section 4 is added, this embodiment can be executed even if the data compression section 4 is not added. Hereinafter, the operation of this embodiment will first be described for a configuration in which a data compression section is not added, and finally the operation will be described for an embodiment in which a data compression section is added.

First image feature extraction unit 6 and second image feature extraction unit 7
specifies an arbitrary memory from 1M input image memories and N feature image memories, and writes the output result to the feature image memory. 8 is a control unit for controlling the overall operation; 13 is a threshold processing unit that binarizes the contents of the input image memory and feature image memory using a threshold value and writes the results to the feature image memory; An inter-pixel calculation unit performs addition, subtraction, multiplication, division, logical addition, and logical addition for each pixel between the input image memory or the feature image memory, and writes the output to the feature image memory; 15 is one or two input image memories; Alternatively, a data conversion unit converts data using a lookup table using the contents of the characteristic image memory as input, and writes the result to the characteristic image memory. 16 is a data conversion section that converts one of two input image memories or the contents of the characteristic image memory into an input characteristic image. , the other is taken as a control image, and the designated area of the input feature image corresponding to the pixels equal to the predetermined setting value of the control image is added to the histogram, and the maximum frequency value, variance value, etc. are added to the obtained histogram. This is a histogram processing unit that performs calculation operations and writes the results to the feature image memory.

In the present invention, at least two image memories from a plurality of input image memories or feature image memories are input, and one
In order to selectively use the images as output, at least three video paths are prepared, and the feature extraction section is configured to be able to access three image memories simultaneously.

61 is a matching processing unit, which performs feature extraction processing.

This section receives the characteristic image stored in the characteristic image storage section obtained from the above as an input, and performs recognition processing such as matching processing with a standard pattern and processing of correspondence between characteristic images obtained at different camera positions.

This is the part that utilizes the output results according to the present invention.

62 is an address and synchronization signal generation section, and A/D conversion section 2. Input image storage unit 3. Data compression section 4, first image feature extraction section 5. Second image feature extraction unit6. I! I value processing unit 132 inter-pixel calculation unit 14. Data converter 15. Histogram processing unit 16. The address signal and synchronization signal necessary for operation are supplied to each part of the matching processing section 61.

Next, the operation of each feature extraction section will be explained.

First, in the first image feature extraction section, a linear sum-of-products calculation is performed.

However, F(i, j, ni, Q)=(f'(i+klj+Q)−
〇f''(itj))''(f'(i+to*J+l2)=C
') or = O (when f'(i+, j+12)≠C') where S is a constant and W(k, fl) is the neighborhood area of the operation similar to the conventional convolution operation. A mask relationship (two-dimensional data table) that specifies, f'(1+j) and fq(i, j) are input images, f'(x+j) is a control image for controlling the calculation, f''(i, j) is an output image showing the calculation result. Also, n is a natural number 0.1, 2.
..., C is a calculation parameter with a value of O or 1, and C' is a control parameter with an arbitrary constant value. In this calculation method, by selecting input image f'(i+jL fq(x+j)+control image f'(iIJL calculation parameters n, C and control parameter C'), conventional convolution calculation and n-th statistic calculation are performed. can be executed uniformly.

Here, the block configuration of the first image feature extraction section 6 in FIG. 1 will be explained. Reference numerals 20, 21, and 22 each indicate a path selector, which selects the input image memory or feature image memory as the input by switching between the two video buses. 24 is a selector, which selects the input image or feature image f″ selected by the path selector 20 when C=1, depending on the value of the calculation parameter C in the calculation formula.
Output.

When C=0, 10' is output and held in the latch circuit 25'. A subtracter 25 subtracts the contents of the latch circuit 25' from the input image or feature image f' selected by the path selector 21. The selector 31 operates according to the value of the calculation parameter n designated by the signal line 36.

If n=o, select 1 given from the signal line 26,
When N=1, the value of the subtraction result in the subtracter 25 is selected. When n=2, a value obtained by squaring the subtraction result is obtained using the square table 27, and this value is selected using the selector 31. n=3. When n=4, the cube and fourth power values are obtained from tables 28 and 29, respectively, and selected by the selector 31. Here, the selector 31 is.

If the pixel value f1 of the input image or feature image selected by the path selector 22 and the value C' of the constant register 34 are compared by the comparator 35, and the results are not equal, the signal line 3
'0' given by 0 is selected preferentially. By the above operation, the selector 31 sets F=(f'(i+, j
+Q) -C:f'(i, j)''(f'(i+, j+
outputs n)=C'(7)) or =O(f'(i+, j+4)≠C'(7)).

33 is the output value of the selector 31 and the mask function table 3
This is a product-sum calculator that calculates the product-sum W(k, Q)·F with two elements. The counter 37 accumulates the number of times when the output value of the comparator 35 is 1 (when the determination is a match). Selector 39
selects the output value of the counter 37 or the constant register 38 in which a constant is written in advance, and outputs the result S. 40 is a divider, which calculates (1/S)ΣΣw(k
, 42) Compute F and output the result to the feature image memory through the system bus or video bus.

Next, the feature extraction process of the second image feature extraction section 7 will be explained. The second image feature extraction unit 7 uses two images, an input feature image f' and a control feature image f', as input. 41 is a bus selector for the input feature image, and 42 is a path selector for the control feature image.By switching between two video buses, the feature image fP is generated.

f' is selected and read from either the input image memory or the feature image memory. At this time, the input feature image f' may be used as is, or may be input to a rising point detector 58 that performs detection processing of the rising point of the image signal (for example, the point where the pixel value changes from O to 1), and its output f ” may be used as the input feature image, and the plane input feature image f'
, f'' are selected by the selector 59. In the following description, no distinction will be made between the two.

In this calculation unit, the input feature image f′ is set to the point of interest P(IIJ)
By scanning radially from θ, the integral value from the point of interest to the stopping point for each scanning direction θ.

A feature value fg''(x+j) is calculated by a total of three types of calculations, including the distance and the feature value at the stop point.Then, the point of interest P(i, j) is moved from the upper left corner to the lower right corner in the image f'. As a result, an output feature image fS for the input feature image f' is calculated.

Examples of three types of calculations are shown in equations (2), (3), and (4).

fs'(Lj)=E w(k)f'(i+xa(k)
, j+ys(k)) (2) L'(iej)=J
Xs L +ya L (3) L
'(i, j)=f'(i +X#(L)? j+ye(
L)) (4) Here, x, (k), y,
(k), to=1,2. ....

L is the two-dimensional address of the radial scanning point in the θ direction, and L
indicates a stopping point. This two-dimensional address is obtained from the address/synchronization signal generation section 62 and supplied to the image memories 3 and 5 during radial scanning. Equation (2) gives the weighted integral value (arrival integral value) of the input feature value, w(kL k=1.2
．． -, -1L are weight sequences for scanning points. (3)
Equation (4) gives the reach distance to the stop point, and Equation (4) gives the feature value (reaching feature value) at the stop point. Here, =L at the stopping point is calculated by the following equation.

When k≧n, L=ne.

When k<n□, (5) L = ain (kl g(fq)≧TH, k = 1
．． 2 ,...v n eR) However, g(f')=f
'(i+x*(k)*j+y*(k))
(6) Or g(f')-Σf'b+X#(n)t
j+ys(12)) (7) 1 薯1 Here, naR is a calculation parameter that gives the size of the visual field, and is given when the size of the visual field is the radius R.
This is the number of scanning points when scanning in the direction. Here, R
and n, a+, R” J Xe, (nsJ”+y
There is a relationship of s(naR)2.

Scanning stop control is performed as follows by scanning a control feature image.

That is, even on the control feature image f', scanning is performed radially from the point of interest P(x, j) in parallel with the input feature image f'. Here, the scanning point position k is within nai1, and the function value g(f') of the control feature value f9 is a predetermined threshold T
When H is exceeded, that scanning point k is determined as the stopping point. Also, as the scanning point position k advances, 1g(f') becomes the threshold T
When it goes out of the field of view (k=n, ll) without exceeding H, the stopping point is set at k=n, . The value g(fq) of the stopping condition is usually the value of the control feature value f1 at the scanning point itself as shown in equation (6), and the threshold value TH is a control parameter for giving the stopping position. By setting the value of this threshold value TH sufficiently large, it is actually possible to shift the stop control to control only the visual field. On the other hand, as shown in equation (7), in addition to the scanning integral value, the value of the functional of f9 can generally be used as the stopping condition.

The calculation operation of the second feature extraction unit 7 in FIG. 1 will be specifically illustrated. First, scanning stop control is performed as follows. Direction θ of the control feature image selected by the bus selector 42
The feature value f'(i+xe(k), j+ya(
k)) becomes the input of the comparator 46 (in this case, g(f
"')=f') is compared with the threshold value TH of the threshold value storage register 45, and if it is equal to or greater than the threshold value TH, the comparator 4
6 to produce an output. On the other hand, the value of the visual field radius R is stored in the scanning radius register 48. Scanning point number table 49 storing the number of scanning points nag for direction θ
generates the number nil of scanning points by inputting the field of view radius R, and compares this with the value of the scanning address counter 47 that gives the position k of the scanning point in the comparator 50, and determines whether the scanning address k is equal to the number of scanning points ns++. If so, there will be an output on comparator 50. The logical OR operator 51 is the comparator 46
．． 50 as input, the scan stop command signal is sent to the product-sum calculator 4.
4. It is sent to latch circuit 44' and scanning distance table 54.

There are three types of calculations on the input feature image: reach integral value, reach distance, and reach feature value. In the arrival integral value calculation operation, the scanning feature value f'(x +
xe(k)+j+Ve(k)) is input to the 44 product-sum calculators, and the product-sum operation of this and the contents of the weight series table 43 storing the weight series w(k) is performed by logical ORing. The process continues until a stop command is received from the arithmetic unit 51.

On the other hand, when the reached feature value calculation operation is stopped by a stop command, the input feature value f'(1+xs(kLj+ya(k)) of =L is held in the latch circuit 44' as the reached feature value. In the selector 53.

When a stop command is issued, one of the two input values, the arrival integral value and the arrival feature value, is selected and outputted according to instructions from the processing control unit 8, and the result is written in the output value buffer 55 for each scanning direction. On the other hand, in the reach distance calculation calculation, the calculation is performed regardless of the value of the input feature image. When the scanning stop command is sent to the scanning address counter 47, the content k (=L) of the scanning address counter 47 at the time of stopping the scanning is read and supplied to the scanning distance table 54. The scanning distance table 54 stores the distance Jxe2(k)+ys(k) reached to each scanning step k in each scanning direction θ, and the distance J xe2(k)+ys(k) reached when stopping in response to a scanning stop command.
X6 (L)+ys (L) is read from the scanning distance table 54, and this read value is written into the output value buffer 55 for each scanning direction.

The contents of the output value buffer 55 for each scanning direction are determined by the selector 57.
Each image is written to a characteristic image memory set for each scanning direction. The scanning direction amount calculator 56 inputs the contents of the output value buffer 55 for each scanning direction, performs calculations between the feature values for each scanning direction, and outputs a feature value representing the relationship between the scanning directions. The output result is written into the feature image memory through the selector 57. An example of operation is shown below.

The output value for each scanning direction is fs' (lej), and the scanning force is 2π
2π 2π ``X2....9hair-U,..., , (D -1), the following calculation is executed.

f'(i, j)=Σfs'b+j)
(8) f″(i, j)=ΣIfs+♂byj) f
s'(i+jN (9) where α is a constant.

Equation (8) represents the total amount in all directions, Equation (9) represents the total amount of differences from directions separated by a certain interval α, and Equation (10) represents the ratio of feature values between orthogonal directions.

The above is a detailed explanation of the block configuration of the image feature extraction units 6 and 7, but the biggest feature of the single configuration is as follows.

The processing control unit 8 selects input feature images and control feature images for calculation by the image feature extraction units 6 and 7 from 1M input image memories and N feature image memories through a common system bus and video bus.

The point is that the calculation result is written back into the specified feature image memory. The output feature image of the first image feature extractor 6 is used as the input feature image or the control feature image of the second image feature extractor 7.

The output image of the second image feature extraction section 7 can be used as an input feature image or a control feature image of the first image feature extraction section 6.

Next, image features calculated by this configuration will be explained. In order of explanation, first, the first image feature extraction unit 6
Next, we will discuss examples of image features extracted mainly by the second image feature extractor 7, and then describe the image features generated by combining the characteristic points of the first feature extractor 6 and the second feature extractor 7. This paper describes a new image feature extraction unit. Furthermore, the advantages of combining with the histogram processing section will be discussed at the end.

First, the feature values calculated by the first image feature extraction section 6 will be described. In equation (1), the processing control unit 8 stores 1 for all pixels in one of the feature image memories, sets all the values of the control image f'(1y J ) to 1, and -f'(ir
j) Mit H==l, C='O,C'=1゜5=1
(Contents of constant register 38), it becomes a conventional convolution operation. In this calculation, the contents of the mask function table 32 and the mask function w(k, Q) are set to 2, for example, a mask function for edge detection (for example, an operator of 5 obel).
By using , first-order differential values in both the horizontal and vertical directions are obtained as the output of the product-sum calculator 33, and by performing inter-pixel addition processing on these output values in the inter-pixel calculation unit 14,
An edge strength feature image is calculated as the output feature image. moreover.

The first-order differential value feature images in both the horizontal and vertical directions are input to the data converter 15, and the data is calculated using an arctangent (tan-'') function table of two people and one output stored in the data converter 15.

An edge direction feature image can be calculated. Further, by inputting the edge strength feature image and performing binarization processing by setting an appropriate threshold value in the threshold processing unit 13, a boundary feature image (1 at the boundary and 0 outside the boundary) can be calculated. By inputting this boundary feature image and edge direction feature image to the inter-pixel calculation section 14, a boundary direction feature image can be calculated as a result of multiplication processing between pixels. Also,
A boundary point that does not depend on the brightness threshold, the so-called brightness 2
To obtain the zero crossing point where the order differential value becomes zero, first -f' CI P j) = 1 t n = 19
C=o rC'=1. S=1, mask function table 32
For example, the second-order differential function of the Gaussian function 1 k"+Q2k"+Q" W(k p fl )=--ko(1-]?-) e X
The product-sum output of the product-sum calculator 33 is obtained by setting p (]7−)π σ (where σ is the shelf standard deviation). Next, this product-sum output is read from the feature image memory and input to the data conversion unit 15, using a lookup table that requires one output per person.

Three-value (0, +1) quantization is performed: near-zero values, positive values, and negative values.

Furthermore, the output value is +1. -1. Perform well-known logical filtering using three values of ○, +1 → -1, -1 → +1
．． By detecting a point that changes from +1→O→−1 or -1→O→+1 as a zero-crossing point, a boundary feature image of 1 at the zero-crossing point and O at other points can be obtained.

Also, in equation (1), “(ir j) = 1
+n = 1, C": O, C' = 1, mask function W (k, Q) = 1, 5 = (2 + 1) (2
L+1), the calculation output f'(irj) gives the average value feature image μ(i, j) within the range (area S) of the mask function for the input feature image f'Cx+j).

Therefore.

Next, μ(i, j) is input to the feature image f′(l l j)
year.

f'(1+j)miO, n=2. C=1. C'=1゜W(
When k, Q)=1,5=(2+1)(2L+1), it is possible to calculate the variance value σ2(xtj) of f'(i, j). In general, n=2.3, 4. ..., it is possible to calculate higher-order statistical images of feature images for each order.

Furthermore, in equation (1).

F(1+J,+Q) only at the pixel (ir, j+Q) that satisfies f'(i+k,j+Q)=C'
) can contribute to the output feature image f''(xtj), so that only the three pixels targeted for calculation as the control image f'(1+j) have the value C', and the other pixels have values other than C'. By using , it is possible to select only specific pixels and perform calculations. For example, if f'' and f''' are both edge direction feature images and f' is a boundary feature image, n =
2, C=1, C'=1.

If W(kJ)El is used, a curvature characteristic image can be calculated as the magnitude of variation in the edge direction.

Here, the value of S is the sum total value (boundary frequency value) of the boundary feature images in the neighborhood Ω of the point of interest (i, j), is equal to the output of the counter 37, and is sent to the divider via the selector 39. 40
is input.

In this embodiment, since the configuration is such that two people can output one output, after setting the input feature value f''(l l j) in the subtractor 25, the input feature value f'(i + k, j + Q)
(k=±1.±2. . . , ±k) are sequentially read and input to the subtracter 25. Control feature value f'(i+
, j+k).

Further, this curvature feature image and the boundary feature image are input to the pixel-to-pixel calculation section 14, and by performing multiplication processing between pixels, a boundary curvature feature image representing the curvature component of the boundary is calculated.

In this way, the first image feature extraction unit 6 not only calculates basic image features that can be calculated, but also specifies calculation parameters, mainly using conventional convolution calculations such as edge strength, edge direction, and boundary. knee! J.
IinJ91-beg “2”1z Nuri snoring□! − It has the ability to calculate high-order statistics that could not be realized and important features that indicate the properties of boundaries, such as boundary curvature features.

Next, feature values calculated by the second image feature extraction section 7 will be described. First, in equation (2).

w(k)=1. Let f' be the rising point detection processed image f'' of the boundary feature image (output of the rising point detector 59), and the scanning stop condition is given by equation (5), g(f'')=f'=boundary feature image, TH By setting a sufficiently large value, the linear equation (2
') As shown in the formula, the complexity feature S6'(1+
J) can be calculated.

fs”(1+j)=Σw(k)f”(i+xa(k)+
j+ys(k)) (2') Here, f''(i+xs
(k)yj+ya(k))=1(fe'(i+xs(k
)tJ+ybak))=1. and f,'(i+x,(k-
1) When +j+ys(k-1))=O]=Oui 醇φ
] The above calculation counts the number of times the boundary is crossed.

In addition, in the above calculation, W(k)=-7 (α is a constant □1
By adopting (B) dog □ in I-7L-1-Key Qing White Monura Yukin 17, the closer the visual field to d'ts in the visual field within the radius R, the larger the weight is set. Complexity can be calculated.

Next, pick up a target object that has a pattern on its surface.

Assume that a boundary feature image is given as a feature representing a pattern, and a contour feature image (contour pixel value 1, other pixel values O) is given as a feature representing the outline of an object.

In this case, in equation (2'), f' is the boundary feature image w
(k)=1, the scan stop condition is g(f
') = f'' = border feature image, TH = 1, and by setting R sufficiently large, the complexity for each scanning direction from the first point of interest to the contour point is calculated. Here, fP is the boundary feature image. Instead, for example, by using a brightness (degree of brightness) feature image and dividing the output value of equation (2) by the number of scanning points, the brightness average feature can be calculated from the scanning direction from the point of interest to the contour point. is calculated.The number of scanning points is calculated by the OR operator 51.
This is obtained by reading the contents of the scanning address counter 47 into the output value buffer 55 for each scanning direction by outputting a stop command.

If R is set to a sufficiently large value, the distance traveled in each scanning direction from the point of interest to the contour point can be calculated. Here, when formula (10) is adopted as the scanning direction amount calculation, the amount of spread (fi, which corresponds to the area of the object region) indicating the extent to which a uniform area on the object surface spreads around the point of interest is calculated. Ru. If the field of view size R is set small, the amount of local expansion,
If you set it to a large value, the global spread of the entire object area will be calculated.If we focus on the reached feature value of equation (4), we can obtain the following:
f' is a feature image indicating the direction component of the contour (contour direction feature image L g (f') = f' = contour feature image, TH
= 1 and R is set sufficiently large, contour direction feature values for each scanning direction on contour points centered on the point of interest can be obtained.

By using this output value as an input to equation (9) and setting D=4°α=π, it is possible to calculate a value that is the sum of the differences in contour direction feature values between diagonal scanning directions. This output value is D
The value divided by 1 can be calculated as, for example, the likelihood of a square quadrilateral. Local features such as parallelism can also be calculated using the direction feature values. In this case, R is set to a value necessary and sufficient to express the local properties of the object, and D=8 or 16. By setting α=π and performing the calculation as shown in the following equation in the 56 scanning direction amount calculation units on the calculated contour direction feature value f a'cir j) for each scanning direction, parallelism F'(x
yj) is calculated.

h(lyJtθ)=lf, □'(IIJ) L'(1
+j) 1(fs”(ltJ)fs+z”(1+j)=
1) "0 (fa" (i, j) fa-z" (i
yj)=O) However, the value of f e'' (1+J) is a pre-calculated reached contour feature value for each scanning direction (a value of 1 or O).

N is the number of contour feature values for each scanning direction whose diagonal components are both 1 when scanning radially. Output value f s'(x t
The closer j) is to π, the higher the degree of parallelism.

In this way, the second image feature extraction unit 7 extracts regional features that determine the attributes of the region, such as complexity and extent, into global or local regions based on multidimensional quantities for each scanning direction and differences in the field of view R. various scalar quantities related to the attributes (in-field complexity, in-field spread).

Parallelism within the field of view, etc.) are calculated.

Next, a new image feature extraction example generated by combining the characteristic points of the first feature extraction section 6 and the second feature extraction section 7 will be described. As mentioned above, in the present invention, the image feature extraction units 6 and 7 are connected by a common video bus and system bus, and input feature images and output feature images can be easily exchanged via the feature image memory. What is the composition? I'm here. Here, a case will be considered in which the output feature image of the first image feature extraction section 6 is used as an input to the second image feature extraction section 7. The first image feature extraction unit 6 calculates important features such as higher-order statistics and boundary properties such as boundary curvature features. Now, the amount of variance for the brightness feature image is used as a higher-order statistic.

That is, it is assumed that a brightness distribution feature image is obtained as an output image. The amount of brightness dispersion is an important feature in understanding the non-uniformity of the shading pattern on the surface of an object. Here, this output image is set as the input feature image f' of the second image feature extraction unit 7, and (2
) by applying the reaching integral value calculation operation, w(k)E:1,
In equation (5), the scan stop condition is g(f')=f'=
Contour feature image, TH = 1, R set sufficiently large,
By dividing the output value of equation (2) by the number of scanning points, the multidimensional amount in the area related to the brightness variance from the point of interest to the contour point, that is, the brightness variance by scanning direction, is calculated as the output value. . This brightness dispersion amount by scanning direction is a multidimensional version of the non-uniformity of the shading pattern, and what is the complexity by scanning direction, which is a multidimensional version of the non-uniformity of the pattern of two figures (boundary feature images). in different amounts. Also, consider a case where the contour curvature feature that is the output image of the first image feature extraction section 6 is used as the input feature image f' of the second image feature extraction section 7. At this time.

Apply the reached feature value calculation calculation in equation (4), and set the scanning stop condition in equation (5), g(f″') = f' = axial feature image, TH = 1, and set R sufficiently large. By doing this, a contour curvature feature by scanning direction centered at the point of interest is calculated.The advantage of this new feature is that it is possible to express contour components of an object by using the contour curvature feature by scanning direction. In other words, when comparing the contour direction feature and the contour curvature feature that integrates the contour direction feature, the contour curvature feature has a wider range of information. It is stable against noise components due to the effect of the product-sum operation.

As shown in Figures 2 and 3, to express the same diamond, there are two cases in which feature values are taken for each scanning direction for contour direction features (Figure 2), and two cases in which feature values are taken for contour curvature features (Figure 2). When compared with the case where feature values for each direction are taken (FIG. 3), less scanning is required based on the contour curvature feature, and the feature value itself is a more reliable value. Generally, the first image feature extraction section 6 calculates feature values that incorporate a wider range of information, and by using this as input, the second image feature extraction section 7 extracts more compact and reliable object information. Obtainable.

Next, consider a case where the output feature image of the second image feature extractor 7 is used as an input to the first image feature extractor 6. Now, let us consider the case where the output feature image of the second image feature extractor 7 is the visual field complexity or the visual field expansion amount. The higher-order statistics calculation operation of the first image feature extraction unit 6 is applied to these output feature images. Conventionally, the calculation was limited to high-order statistics related to brightness features and directions, but the output values of the second image feature extraction unit 7 are high-order statistics for new dimension feature values such as visual field complexity and intra-field spread amount. More effective features are generated by calculating the next statistic. For example, by taking a second-order statistic (that is, variance) regarding the amount of spread, an amount related to the non-uniformity of the amount of spread in space is calculated. This feature value is a quantity that exhibits a higher value in areas where the pattern is unevenly distributed, and is an important quantity for pattern analysis.

Furthermore, when the image feature extractors 6 and 7 are combined, there is an advantage that the first image feature extractor 6 compensates for the drawbacks of the second image feature extractor 7. For example, the second image feature extraction unit 7
However, as shown in FIG. 4, if the outline of the object is interrupted or there is noise, there is a problem in that scanning does not stop on the contour points and stable feature extraction cannot be performed.

Here, by calculating and smoothing the first-order statistics (average value) of the first image feature extraction unit 6 for the edge strength feature that determines the contour feature, we can create an uninterrupted contour and a noise-free object. The region is obtained, and the second image feature extraction unit 7
A stable radial scan operation can be obtained in .

The advantages of combining the image feature extracting units 6 and 7 have been described above, but by also combining the histogram processing unit 16, more powerful feature extraction calculations can be executed. The histogram processing section 16 can perform maximum frequency value (mode value) calculation operations and median value (median) calculation operations. By using the in-field complexity feature value that gives the maximum frequency value in the pixel selection region as the output value, it is possible to obtain a feature image in which a relatively uniform region is formed compared to a variable complexity feature image. becomes. Furthermore, by similarly calculating the median value, a characteristic image containing noise can be smoothed without losing previous information. In this way, by applying the 16 histogram processing sections as pre-processing to the first image feature extraction section 6 and the second image feature extraction section 7, it becomes possible to perform more stable feature extraction.

Although the case in which the data compression section 4 is not included in the configuration of this embodiment has been described above, the following embodiment will be described with respect to the case in which the data compression section is incorporated.

The data compression section 4 is coupled to the input image memory of the input image storage section 3 and the characteristic image memory of the characteristic image storage section 5 via video buses 9, 10 and system buses 11, 12, respectively. The data compression unit 4 receives a command from the processing control unit 8, compresses the contents of the input image memory of the input image storage unit 3 and the feature image memory of the feature image storage unit 5, and writes the output to the feature image memory. . in particular,
The following operations are performed. A bus selector 17 selects an input image memory and a feature image memory to be input by switching between two video buses 9 and 10. 18
is a product-sum calculator, and performs the calculation shown in the following equation.

Here, m is a constant that determines the sampling interval, S is a constant, W (k, 12) is a mask function (two-dimensional data table) given from the mask function table of 18', f' (
1#j) is the content of the input image memory and feature image memory that are input, and f''(r, s) is the output image indicating the calculation result. Usually, W(k, Q)'El.

5 = (2 + 1) (2L + 1), and the average value is the output value and 1 k'', but W (k, Q) = -7eXp (-2a) 2
Smoothing processing may be performed using a general smoothing function such as πσ (where σ is the standard deviation) as a mask function.

Here, this smoothing processing operation is performed using the addresses (mr, m
s), and its output value is the video bus 1
1 to the feature image memory. As a result, a data compression effect of 1/m'' can be obtained.

Next, the advantages of adding the data compression section 4 to the configuration of the present invention will be described. Conventionally, in the first image feature extraction section 6, when the size nXn of the mask function is increased, the amount of calculation increases on the order of 0 (n2), and
The image feature extraction unit 7 also had a problem in that as the radial scanning distance increased, the amount of calculation increased even for the same order of magnitude. By compressing the data, the size of the input feature image becomes smaller,
The amount of calculation is on the order of O(n 2 /m 2 ) (m: sampling constant), and the calculation cost can be reduced. Furthermore, since the input feature image is smoothed, it is possible to reduce the effects of contour discontinuities and noise as shown in FIG. 4, and a synergistic effect can be expected in that stable feature extraction can be performed.

General Smoothing Function This results in a data compression effect of 1/m''.

Next, the advantages of adding the data compression section 4 to the configuration of the present invention will be described. Conventionally, in the first image feature extraction section, the size nXn of the mask function is increased (then,
There is a problem in that the amount of calculation increases on the order of O(n''), and also in the second image extraction section, as the radial scanning distance increases, the amount of calculation increases even on the same order. By compressing the data,
The size of the input feature image is small, and the amount of calculation is O
It is on the order of (n''/m'') (m: sampling constant), and the calculation cost can be reduced. Furthermore, since the input feature image is smoothed, it is possible to reduce the effects of contour discontinuities and noise as shown in FIG. 4, and a synergistic effect can be expected in that stable feature extraction can be performed.

(Effects of the Invention) As explained above, the present invention includes a feature extraction unit that extracts important features indicating the properties of a boundary such as a boundary curvature feature by a convolution operation and a higher-order statistical calculation, and , and a feature extraction unit that extracts various attributes of a region, such as the amount of spread, are connected via a common video bus and system bus, and input feature images and output feature images can be easily exchanged via a feature image memory. Because of this configuration, image features that are effective and important for object recognition, which could not be achieved by applying each feature extraction unit alone, can be stably extracted, resulting in highly versatile image processing and feature extraction. It has the advantage of being able to perform calculations.

[Brief explanation of drawings]

FIG. 1 is a detailed block configuration diagram for realizing an image feature extraction device according to the present invention, FIG. 2 is a diagram showing an example of conventional radial scanning type calculation, and FIG. 3 is a diagram showing a first image feature extraction device according to the present invention. FIG. 4 is a diagram showing the effect of combining the extracting section and the second feature extracting section, and FIG. 4 is a diagram showing the drawbacks of the second image extracting section. 1... Imaging device, 2... A/D converter, 3... Input image storage section, 4... Data compression section, 5... Characteristic image storage section, 6... First Image feature extraction section: 7... Second image feature extraction section. 8... Processing control unit, 9, 10.11... Video bus, 12... System bus, 13... Threshold processing unit. 14... Inter-pixel calculation unit, 15... Data conversion unit, 16... Histogram processing unit, 17, 20.21.22.41.42... Bus selector, 23, 26.30... Numerical input signal, 24, 31
．． 39.53.57.59...Selector, 25...
Subtractor, 25'... Latch circuit, 28... Cubic power table, 29... Fourth power table, 34, 38... Constant register, 35, 46.50... Comparator, 37... -Counter, 40...Divider, 43...Weight series table, 44'...Latch circuit. 45...Register for storing threshold value. 47... Scanning address counter, 48... Scanning radius register, 49... Scanning point number table, 51... OR operator, 54... Scanning distance table, 55... Output value by scanning direction Buffer, 56...Scanning direction amount calculation unit, 58...Rising point detector. Patent applicant: Japanese Igo Denwa Co., Ltd. Figure 3 Figure 4

Claims (2)

[Claims] (1) An input image storage unit that stores input image data, and stores a feature image obtained by performing feature extraction processing on the input image data or a feature image obtained by further performing feature extraction processing on the obtained feature image. A characteristic image storage section consisting of a plurality of characteristic image memories, two image storage areas A and B specified for the characteristic image storage section or the input image storage section; Specify two feature image memories, and perform a weighted sum-of-products operation of the power of the difference in pixel value between the pixel at the point of interest on image storage area B and the pixel in the corresponding neighboring area on image storage area A. a first image feature extraction unit that executes the extraction only when the value of a pixel in the feature image memory C corresponding to the pixel in the neighboring region is equal to a preset value, and stores the result in the feature image memory C; Two images among the input images or feature images stored in the image storage unit or the feature image storage unit are designated as the input feature image and the control feature image, respectively, and the size of the field of view that gives the range of radial scanning and the A scan stop condition defined for the feature value of the control feature image is set, and the above two feature images are moved from a point of interest at the same position to a stop point determined by either the field of view size or the scan stop condition. The scanning area is controlled by scanning radially, the feature extraction calculation is performed on the feature values on the scanning line of the input feature image in each scanning direction, and the position of the point of interest is sequentially moved on both feature images. generate a feature image composed of feature values at each point of interest,
a second image feature extraction section storing in a feature image memory designated by the feature image storage section; the input image storage section and the feature image storage section are commonly used by the first and second feature extraction sections; An image feature extraction device characterized by having a transparent configuration. (2) Perform smoothing processing on the input image stored in the input image storage unit or the feature image stored in the feature image storage unit, perform sampling processing on the result according to the set sampling interval, and An image feature extraction device according to claim (1), further comprising a data compression unit that stores the obtained image in a feature image storage unit.