JPH06348901A - Multiple resolution image feature extracting device - Google Patents

Abstract

PURPOSE:To provide the feature extracting device of multiple resolution which employs the convolution of templates capable of absorbing a position shift without causing an increase in device scale. CONSTITUTION:An image feature extracting device with 1st multiple resolution generates an enlarged or reduced image by an image formation part 101 according to a scale signal which is generated by a scale control part 106 and corresponds to an enlargement/-reduction rate. A template convolution part 102 convolutes plural templates for the image and an addition part 103 adds a generated convolution signal to generate an addition image. Further, a re- sampling part 104 performs a thinning-out process according to the scale signal and outputs image features. An image feature extracting device with 2nd multiple resolution after enlarging or reducing the templates by a template convolution part 102 according to a scale signal convolutes the output image of an image formation part 101.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image feature extraction device, and more particularly to a multi-resolution image feature extraction device.

[0002]

2. Description of the Related Art Conventionally, this type of image feature extraction apparatus extracts a graphic feature which is a key to recognition in image recognition, particularly character recognition, absorbs a hand shake or a graphic distortion due to noise superposition, and changes It is used for the purpose of extracting the features that are not affected by and passing the feature data to the recognition unit. For example, in "1978, Digital Image Processing, Modern Science Co., Ltd., pages 443 to 445", a character stroke or the like is used as a template, and by performing a convolution operation on the target image, the target image is convolved. A technique for detecting a portion similar to the shape of the template in the image is described.

FIG. 10 is a diagram showing an example of the operation of a conventional image feature extraction apparatus using convolution. The convolution unit 1003 of the template 1002 performs the calculation according to the equation 1 on the input image 1001.

[0004]

[Equation 1]

Here, the size of the input image 1001 is N × N pixels, and the value at coordinates (x, y) is h (x,
y), the value at the coordinates (a, b) of the template 1002 is f (a, b), and the value at the coordinates (x, y) of the image 1003 after convolution is H (f). This makes H
The value of (f) is obtained to the extent that the template 1002 exists at the coordinates (x, y) in the input image 1001. The larger this value is, the more local pattern similar to the template 1002 exists. The same process is performed on all the coordinates (x, y) of the input image 1001 to detect the degree of existence of the template 1002 at all the positions of the input image.

[0006]

In this conventional image feature extraction apparatus, a local feature is represented by a template, and a convolution operation is executed to detect the local feature in the input image. The detection of local features that are slightly misaligned can be performed by using the coordinates (x,
Since it is obtained by executing the convolution in y), the amount of calculation and the storage capacity for the realization increase, and the realization becomes difficult. Even if the local feature is expanded or reduced, this conventional image feature extraction device cannot detect it, and it is necessary to prepare a large number of types of the expanded or reduced local feature to be detected as a template. .

It is an object of the present invention to provide a multi-resolution image feature extraction apparatus capable of performing multi-resolution feature extraction by convoluting a template capable of absorbing positional deviation without increasing the scale of the apparatus.

[0008]

According to a first aspect of the present invention, there is provided an image feature extraction apparatus which includes a local feature of an image as a template and which convolves the template with an input image to extract the feature of the image, and which corresponds to a scaling ratio. A scale control unit that generates a scale signal, an image forming unit that generates an image according to the scale signal from the scale control unit, a template convolution unit that convolves a plurality of templates, and a convolution signal generated in the template convolution unit. It is characterized by comprising an adding section for adding and storing, and a resampling section for thinning out the convolutional signal output from the adding section according to the scale signal from the scale control section and outputting the characteristic.

A second invention is characterized in that, in the first invention, the template convolution unit convolves a plurality of templates after enlarging / reducing a plurality of templates according to a scale signal from the scale control unit.

[0010]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Next, the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing an embodiment of the first invention. Referring to FIG. 1, in the present exemplary embodiment, a scale control unit 106 that controls the size of a feature extracted from an image, an image forming unit 101, a template convolution unit 102, an addition unit 103, a resampling unit 104, and an output terminal 105. Composed of. The scale control unit 106 generates a scale signal according to the size of the extracted local feature. For example,
2 ^1/2 times, 2 times, 2 × 2 times the size of the template of the local feature used in the template convolution unit 102.
^When the presence / absence of six types of local features of ^1/2 times, 2 ^−1/2 times, and ^1/2 times is extracted as a feature, a scale signal s indicating the magnification is resampled with the image forming unit 101. Part 1
Send to 04. The operation of the image forming unit 101 will be described with reference to FIG. Note that FIG. 3 is an explanatory diagram of an image enlargement / reduction portion in the image forming unit. The target image for feature extraction is
It is input from the terminal 301 and stored in the image storage unit 302. The scale signal s sent from the scale control unit 106 is input from the terminal 305 and the scaling unit 304 is activated.
The scale signal s indicates that the image is magnified s times.
When the grayscale value of the image at the (x, y) coordinates of the input image stored in the image storage unit 302 is represented by h (x, y), the grayscale value h ′ (x ′, y ′) is calculated according to the equation shown in Equation 2.

## EQU00002 ## h '(x', y ') = (y' / s-F (y '/ s)) ((x' / s-F (x '/ s)) h (C (x' / s), C (y '/ s)) + (C (x' / s) -x '/ s) h (F (x' / s), C (y '/ s))) + (C (y '/ S) -y' / s) ((x '/ s-F (x' / s)) h (C (x '/ s), F (y' / s)) + (C (x '/ s) -x '/ s) h (F (x' / s), F (y '/ s))) where C (R) represents the smallest integer value equal to or greater than the real value R, and F ( R) represents the largest integer value smaller than the real value R. Further, if the size of the original image is 64 pixels × 64 pixels, the values of x and y take integer values between 1 and 64. In addition, the coordinate values x ′ and y ′ of the image h ′ multiplied by s are 1
To 64 s. The image h ′ (x ′, y ′) magnified s times as described above is stored in the image storage unit 302, and is input from the terminal 303 to the template convolution unit 10.
2 is sent.

Next, the template convolution unit 102 shown in FIG.
Will be described. The template convolution unit 102 is
In addition to having a plurality of templates representing the local features of the image generated in advance, a template f'corresponding to the case where the local features cause a slight positional deviation is also provided for the template f representing one local feature. An example of the template generation method will be described. For example, the template f (i, a, b) corresponding to the i-th local feature is represented by Equation 3.

F (i, a, b) = (1 / n) cos ((2π / L) (A (i) a + B (i) b)) × exp (−2 (a ² + b ² ) / L ² ) Here, i represents an integer value from 0 to 7, and n is 1, L
Although the value of is set to 8, there is no problem even if n and L take other values. As a result, the template is rotated by πi / 8 radians and forms a template representing a local feature that is composed of line segments that exist in parallel at every L pixel. Also, A (i), B
(I) takes a value depending on the i-th local feature, and can be given by the following equations 4 and 5, but this is not an essential problem, and other values can be employed.

## EQU00004 ## A (i) = cos (.pi.i / 8)

## EQU5 ## B (i) = sin (πi / 8) In this case, i
Is an integer value from 0 to 7, a template representing eight types of local features will be used. When the template defined by the equation 3 is used, the template f ′ (i, a, b) obtained by slightly shifting the template is given by the equation 6.

F ′ (i, a, b) = (1 / n) sin ((2π / L) (A (i) a + B (i) b)) × exp (−2 (a ² + b ² ) / L ² ) The template f ′ defined in this way has a position shift of about L / 4 pixels, that is, about 2 pixels, from the original template f. As described above, it is possible to generate a template representing a parallel line segment composed of eight types of directions and eight types of templates with a slight positional deviation.
The template generated in this way is stored in advance in the template convolution unit 102, and the template convolution unit 102 performs the same convolution operation on the i-th two templates shown in Formulas 3 and 6 as in Formula 1. Two types of result images H (i, f) (x, y) and H
(I, f ') (x, y) is generated and sent to the adder 103.

The adder 103 receives (x, y)
Two types of coordinate image data H (i, f) and H (i, f)
From f '), addition is performed according to the equation 7.

## EQU7 ## G (i, f) = (H (i, f) ² + H (i, f ') ² ) ^1/2 The value G at the (x, y) coordinates obtained by executing this calculation
(I, f) (x, y) is transferred to the resampling unit 104. Here, the description has been given assuming that the number of positional deviation templates is two, but the same applies to the case of three or more.

In the resampling unit 104, x direction, y
Sampling is performed every k pixels in both directions, and G (i, f) of the sampled pixels is sent out as a feature from the terminal 105, and the values G (i, f) of the other pixels are erased. Here, the parameter k is set to 1 for convenience, but other values are not an essential problem. Scale signal s
It is possible to extract the same feature corresponding to different resolutions from the same image by sequentially changing.

4 and 5 show another embodiment of the first invention. The input image h (x, y) that is the target of feature extraction is
It is assumed that each pixel of the two-dimensional laser array 401 is switched according to the scale signal s input from the input terminal 410 and transferred from the scale control unit 106.
The laser component shown in FIG. 5 is arranged in each pixel of the two-dimensional laser array. A case where the operation of the laser component arranged at the (x, y) coordinates is feature-extracted on three scales will be described with reference to FIG. Here, a case of 3 scales will be described, but the number of scales is not limited to 3.
From the pixel value input terminal 501, the value f (x, y) of the (x, y) coordinates of the input image is input. Scale control unit 10
The scale signal s transferred from No. 6 is input from the terminal 512, activates the selector 506, selects the signal line corresponding to the sth scale, and selects one of the three amplifiers 502.
Drive one. As a result, the three laser elements 503, 504, 50 that emit coherent light beams of different frequencies
Laser light is emitted from one of the five. The selected one coherent light is emitted from all the laser components at the coordinates (x, y) where f (x, y) of the two-dimensional laser array 401 has a value larger than a predetermined threshold value. Let P be the wavelength of the coherent light selected here.
And The light emitted from the two-dimensional laser array 401 is split into light in two directions by the beam splitter 402. One passes through the lens 404 and reaches the spatial modulator 405. The other is reflected by the mirror 403 and the lens 40
4 to reach the spatial modulator 406. Spatial modulator 40
5 can be realized by making an image obtained by Fourier transforming the template defined by the expression 3 dark and light and printing it on a transparent film. Further, the spatial modulator 406 can be realized by similarly printing the image obtained by Fourier transforming the template defined by the equation 6 on a transparent film by making the image dark and light. Two-dimensional laser array 401 to lens 40
4, the optical path length from the lens 404 to the spatial modulator 40
5. By matching the optical path length to the spatial modulator 406 with the focal length of the lens 404, the wavelength P of the incident coherent light is projected on the surfaces of the spatial modulator 405 and the spatial modulator 406.
An image corresponding to the Fourier transform image of the image input from the terminal 410 is projected in a size inversely proportional to. Therefore, the projection light is transmitted through the spatial modulator 405, and the result equivalent to the result obtained by performing the Fourier transform on the template obtained by performing the convolution operation on the input image according to the equation 1 is obtained. Similarly, the light transmitted through the spatial modulator 406 represents light equivalent to the light obtained by performing the Fourier transform on the template defined in Expression 6 and performing the convolution operation according to Expression 1. Further, the light transmitted through the spatial modulators 405 and 406 passes through the lens 409, the prism 407, and reaches the two-dimensional photoelectric conversion element array 408. Also in this case, the optical path length from the spatial modulators 405 and 406 to the lens 409 and the optical path length from the lens 409 to the photoelectric conversion element 408 are matched with the focal length of the lens 409. As a result, the spatial modulator 40 is provided on the photoelectric conversion element array 408.
The result obtained by superimposing the inverse Fourier transform images of the images transmitted through 5, 406, convolving the two templates defined by equations 3 and 6 on the input image, and adding the two convolution images according to equation 7. G (i, f) ² (x ′,
y ') is obtained. The result is resampled by the resampling unit 104.
, And thinning out every k pixels in accordance with the scale signal transferred from the scale control unit 106, and extracting the data from the terminal 105 as an image feature. Here, k is set to a value inversely proportional to the wavelength P, but this is not an essential problem. By changing the scale signal s sequentially,
The same feature corresponding to different resolutions can be extracted from the same image.

Still another embodiment of the first invention will be described with reference to FIG. The scale signal transferred from the scale control unit 106 is transferred to the laser irradiation unit 601. The laser irradiation unit 601 is one in which one or more laser components as shown in FIG. 5, for example, are arranged side by side, and the coherent light of the wavelength P according to the scale signal s is used as the target image 602.
To irradiate. The reflected light is reflected by the lens 404 and the spatial modulator 40.
5, 406, the lens 409, and the prism 407, and the photoelectric conversion element array 408 is irradiated. On this photoelectric conversion element array, by the same operation as that of the embodiment shown in FIG. 4, the two templates defined by the formulas 3 and 6 are convoluted to the input image, and the two convoluted images are calculated according to the formula 7. As a result of addition, G (i, f) ² (x ′, y ′) is obtained. The result is transferred to the resampling unit 104, thinned out every k pixels in accordance with the scale signal transferred from the scale control unit 106, and the data is output to the terminal 10.
5 is sent as an image feature. Here, k is the wavelength P
The value is inversely proportional to, but this is not an essential problem. By sequentially changing the scale signal s, the same feature corresponding to different resolutions can be extracted from the same image.

Next, an embodiment of the second invention will be described with reference to FIGS. The embodiment of the multi-resolution image feature extraction apparatus shown in FIG. 2 includes an image forming unit 101, a template convolution unit 201 capable of scaling a template, and an adding unit 10.
3, the resampling unit 104, the scale control unit 106,
The output terminal 105. Since the image forming unit 101, the scale control unit 106, and the addition unit 103 are the same as those in the embodiment of FIG. 1, the template convolution unit 201 will be described with reference to FIG. The template f (a, b) generated by the calculation according to Formula 3 or Formula 6 is input from the terminal 707 and stored in the template storage unit 704. Enlarging / reducing unit 70
5, the scale signal s transferred from the scale control unit 106 is input from the terminal 706, and the template f stored in the template storage unit 704 according to the value s.
(A, b) is subjected to calculation according to the following formula 8 to scale it to obtain a template f '(a', b ').

F ′ (a ′, b ′) = (a′s−F (b ′s)) ((a′s−F (a ′s)) f (C (a ′s), C ( b (s)) + (C (a's) -a's) f (F (a's), C (b's))) + (C (b's) -b's) (( a's-F (a's)) f (C (a's), F (b's)) + (C (a's) -a's) f (F (a's), F (B's))) The obtained template f '(a', b ') is stored again in the template storage unit 704. Convolution calculation unit 70
2 obtained according to Equations 3 and 6 for the image h (x, y) transferred from the image generation unit 101 through the terminal 701 in 2
The two convolutional images H are
(I, f ', x, y) is obtained and transferred to the adder 103 through the terminal 703. By the calculation according to the equation 7, the addition unit 1
The image G (i, f ′) (x, y) generated in 03 is transferred to the resampling control unit 104 and thinned out for every k pixels according to the scale signal s transferred from the scale control unit 106. It is output from the terminal 105 as an image feature. Here, k is defined as a value inversely proportional to s,
This is not an essential issue. In this embodiment, even when the image forming unit 101 does not perform image enlargement / reduction, the same effect as that obtained by performing s-fold enlargement can be obtained. Furthermore, when the image forming unit 101 enlarges the image by s times, the same effect as when the input image is enlarged by s ² times is obtained. By sequentially changing the scale signal s, the same feature corresponding to different resolutions can be extracted from the same image.

Next, another embodiment of the second invention will be described with reference to FIG. Input terminal 410, two-dimensional laser array 4
01, the beam splitter 402, the mirror 403, the lens 404, the lens 409, the prism 407, the photoelectric conversion element array 408, the resampling unit 104, and the terminal 105 are the same as those in the embodiment of FIG.
4, the enlargement / reduction unit 705 and the terminals 706 and 707 are the same as those in the embodiment of FIG. 7, and therefore the variable spatial modulator 805 will be described. In the embodiment shown in FIG. 4, the spatial modulators 405,
406 is a fixed template printed on a film, but here, the templates generated according to Equations 3 and 6 and magnified s times are used as spatial modulators 805 and 806, respectively, which are configured by liquid crystal display devices. To display. As a result, the two-dimensional laser array 401 can sequentially change the scale signal s without switching the wavelength of the laser light to extract the same feature corresponding to different resolutions from the same image. Further, when the wavelength of the laser light is switched by the two-dimensional laser array 401, it is possible to extract the characteristics of a wider variety of resolutions.

Next, still another embodiment of the second invention will be described with reference to FIG. In the example of FIG. 9, the target image 60
2, laser irradiation unit 601, lens 404, lens 40
9, the prism 407, the photoelectric conversion element array 408, the resampling unit 104, the output terminal 105, and the scale control unit 106 are the same as those in the embodiment of FIG.
5, template storage unit 704, terminals 706, 707
Is the same as FIG. 7 of the embodiment. The variable spatial modulators 805 and 806 are also the same as those described in FIG. The variable spatial modulator 8 uses the template magnified s times.
By displaying it on 05 and 806, it is possible to extract the characteristic of the scale of the scale signal s times. By sequentially changing the scale signal s and outputting the image features, it is possible to extract the same shape features of different sizes corresponding to different resolutions from the same image.

[0019]

The multi-resolution feature extraction apparatus described above is provided with a plurality of templates corresponding to positional deviation,
By adding the convoluted images using each of them, the feature represented by the template can be extracted without being affected by the intermediate positional deviation.

In addition, the template corresponding to one kind of local feature can be realized by providing the same effect as scaling the image itself without scaling the template in advance and scaling the template. Is 1
It is sufficient to provide only one, and a multiple feature extraction device can be easily realized, the number of templates corresponding to multiple features can be reduced, and templates corresponding to many types of local features can be efficiently provided.

[Brief description of drawings]

FIG. 1 is a configuration diagram showing an embodiment of a multi-resolution extraction device of the first invention.

FIG. 2 is a configuration diagram showing an embodiment of a multi-resolution extraction device of the second invention.

FIG. 3 is an explanatory diagram of an enlargement / reduction portion of an image in the image forming unit.

FIG. 4 is a configuration diagram showing an embodiment of a multi-resolution extraction device of the first invention.

FIG. 5 is a diagram showing a configuration of a laser component of a two-dimensional laser array.

FIG. 6 is a configuration diagram showing an embodiment of a multi-resolution extraction device of the first invention.

FIG. 7 is an illustration of an embodiment of a scaling portion of a template of a template convolution unit.

FIG. 8 is a configuration diagram showing an embodiment of a multi-resolution extraction device of the second invention.

FIG. 9 is a configuration diagram showing an embodiment of a multi-resolution extraction device of the second invention.

FIG. 10 is an explanatory diagram of feature extraction using conventional convolution.

[Explanation of symbols]

 101 image forming unit 102 template convolution unit 103 adder unit 104 resampling unit 105 output terminal 106 scale control unit 201 template convolution unit 301 image input terminal 302 image storage unit 303 image output terminal 304, 705 scaling unit 305 scale input terminal 401 2 Dimensional laser array 402 Beam splitter 403 Mirror 404,409 Lens 405,406 Spatial modulator 407 Prism 408 Photoelectric conversion element array 501 Pixel value input terminal 502 Amplifier 503,504,505 Laser element 506 Selector 512 Scale input terminal 601 Laser irradiation part 602 Target image 701, 703, 706, 707 Terminal 702 Convolution calculation unit 704 Template storage unit 805, 806 Variable spatial modulator 1001 Input image 1002 Template 1003 Convolutional image 1004 Convolutional processing

Claims (2)

[Claims] 1. An image feature extraction apparatus that includes local features of an image as a template, and extracts the features of the image by convolving the template with an input image, and a scale control unit that generates a scale signal corresponding to a scaling ratio. An image forming unit that generates an image according to the scale signal from the scale control unit, a template convolution unit that convolves a plurality of templates, an addition unit that adds and stores the convolution signals generated in the template convolution unit, and the addition And a resampling unit that thins out the convolutional signal output from the unit according to the scale signal from the scale control unit and outputs the feature. 2. The multi-resolution image feature extraction apparatus according to claim 1, wherein the template convolution unit performs convolution after enlarging / reducing a plurality of templates according to a scale signal from the scale control unit.