JPH08315141A - Image recognition device - Google Patents

Abstract

PURPOSE: To obtain an image recognition device which recognizes an image or confirms a frequency by feeding back and correcting the weight and threshold, and making multi-stage connections and processing and compressing two-dimensional input information at a time. CONSTITUTION: This device is equipped with neuron type threshold elements having weight coefficients for plural inputs respectively and a storage part which is constituted by hierarchically connecting the neuron type threshold elements in a pyramid shape so that the output of the top layer neuron is regarded as a final output and also variably sets the values of the weight coefficients and the threshold. In addition to the basic constitution, compressed information obtained in an intermediate layer of the hierachy is also utilized. In the basic constitution, plural inputs are regarded as two inputs and 2-input 1-output neuro type threshold elements 21 are used.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention mainly relates to image recognition, in which an image is subjected to image processing such as compression and frequency conversion by using a neuron-type element having a special structure and property, and the recognition rate and The present invention relates to an image recognition device with improved processing speed.

[0002]

2. Description of the Related Art When recognizing an image, it is desirable from the viewpoint of processing time if redundant information can be eliminated as much as possible and recognition can be performed using only the minimum necessary (compressed) image information.
Conventionally, various image compression methods have been proposed, but most of them have been compression techniques mainly used for communication.
For example, FIG. 18 is a circuit diagram of a conventional image compression apparatus shown in Japanese Patent Laid-Open No. 62-219766. As shown in the figure, the run-length coding method is improved in which similar information is counted and summarized on the scanning line. Specifically, the start point of the data to be compressed is marked, and as long as there is similar information, its length is incrementally added to the run length counter, and the end point is marked at the end to make a compressed code. Is. However, this method uses the information scanned in a certain direction,
It is effective for high-efficiency transmission through a one-dimensional route, but since two-dimensional information has different information, the entire two-dimensional image is converted with a constant resolution, that is, the same compression ratio,
It is a method and device that is rather cumbersome for the purpose of analysis / recognition.

As one method for solving such a problem, there is a compression method in which the resolution level is matched at all points of the image for recognition processing. For example, heretofore, there has been a general processing method in which an image is divided into several blocks and an average value is taken for each block to obtain compressed data. On the other hand, FIG. 19 shows another conventional image processing method shown in JP-A-5-61976. In the method shown in the figure, the accuracy of compression is improved by calculating and correcting the dispersion ratio of the luminance of the image, and thereby the recognition rate is improved. However, this method may be effective when compressing the image at a certain ratio, but if it is not, that is, compression with a halfway scale factor or using several images expressed in multiple resolutions. Whenever it is desired to perform integrated processing, each resolution expression must be derived starting from the block division method. This requires a fairly complicated procedure and time.

A technique for recognizing an image using a neural network is shown in FIG.
There are 300868 methods. This is not to directly input image data to a neural network, but to analyze the image processing contents in advance and construct and process an optimal neural network structure specialized for the processing. In the figure, logical operations ((A and B) and (C
(a) and (b) two types of neural networks that perform (and D)). The symbol and indicates exclusive OR, and A, B, C, D are "1" or "0".
The digital value of. For example, if A, B, C, D is "0
When "001" is input, only the third neuron from the top in the leftmost input layer of both networks outputs. "Xx01" written above each neuron
The number indicates that if the lower 2 bits are 01, an output can be output. Here, "x" may be 0 or 1. If you follow the network in the same way,
It can be seen that the two neural networks are two types of neural networks having different structures that can perform the above-described logical operation. (B) is a neural network that has been subjected to an optimization process using the method disclosed in this patent, and the number of neurons is the same as that in (a), but the number of links is 6 less, and only that much is calculated. The amount can be reduced. With this configuration, first, when the input and expected output are known, the optimum neural network structure is obtained. However, the method shown here is aimed at reducing the number of neurons, and its structure is not devised for recognition processing. Further, even for the output of the intermediate layer, it is not a system considering the use as "data for recognition", and it is unlikely that effective information will appear in the output of the intermediate layer for the subsequent processing.

[0005]

Since the conventional image recognition system is constructed as described above, a balanced and practical whole system cannot be obtained. As well as accurate understanding of the nature of the object, the characteristics of the analysis side and its effect on the object must be well known, which is difficult at this stage when the method of information integration is unknown. At present, a spatial filter with a certain size and a microprocessor that executes processing such as convolution at high speed have been introduced, but it is not possible to extract the target multipurpose features by such deterministic processing. Many.

The present invention has been made in order to solve the above-mentioned problems, and an optimum recognition system is provided by performing feedback correction at once in processing and compression frequency analysis of two-dimensional input information using a neutral network to recognition. It is intended to be realized.

[0007]

An image recognition apparatus according to the present invention comprises a neuron type threshold element having weighting coefficients for a plurality of inputs, and a pyramid type hierarchical connection of the neuron type threshold elements. The configuration is such that the output of the uppermost layer neuron is the final output, and a storage unit for variably setting the value of the weighting factor and the threshold value is provided.

Further, in addition to the basic structure, the output of the intermediate layer at any coordinate position of any intermediate layer of the hierarchy is switched and output as the final output.

Further, in the basic structure, the number of inputs is 2
A neuro-type threshold element having two inputs and one output is used as an input.

Furthermore, in the basic configuration, the value of the weighting factor and the threshold value are changed based on the recognition result of a part of the hierarchical connection of other neuron type threshold value elements.

Furthermore, the value of the weighting factor or the threshold value is changed based on the recognition result of the distance information of a part of the hierarchical connection of other neuron type threshold value elements.

[0012]

In the image recognition apparatus according to the present invention, the outputs determined by the weighting factor and the threshold value are hierarchically transmitted to a plurality of inputs in a pyramid type, and the final output is obtained by compressing the information of the lowermost layer. . The output of each neuron is obtained by emphasizing, smoothing, or filtering the input by the weight of the input and a threshold function. This action is repeated as many times as necessary by changing the weighting factor and the threshold value, and image recognition is performed by comparing with a standard pattern, for example.

Furthermore, the recognition result of the intermediate layer of an arbitrary hierarchy is used as the final output and compared with a small number of standard patterns for image recognition.

Furthermore, each neuron has two inputs, and each input is subjected to addition and difference calculation and output.

Furthermore, the value of the weighting factor and the threshold value are changed, and the image is obtained by being emphasized and filtered based on the image recognition result.

Furthermore, the value of the weighting factor or the threshold value is changed based on the recognition result of the distance information of a part of the hierarchical connection,
The frequency characteristic is changed.

[0017]

【Example】

Example 1. Before describing an embodiment of the present invention, first, an input portion for capturing a general image will be described. Usually, the image information contains a huge amount of information. For example, a grayscale image of 512 × 512 pixels and 8 bits for each pixel has approximately 2.1 million bits. Further, if each pixel R, G, and B is represented by 8 bits, the amount of information that is 64 times that is included in one image. Such a large amount of data is a bottleneck in order to analyze and recognize images with this enormous amount of information at high speed. However, it goes without saying that not all image information is necessary for recognition. Depending on the object, it is often possible to fully recognize it with only partial information. Further, it is convenient because the amount of data handled by that person is small and the processing time can be shortened. In other words, it can be said that it contains a lot of extra information that is not related to recognition. A lot of researches have been conducted on information reduction methods mainly in the field of image data transmission. However, this method pursues transmission efficiency. For example, the compression of one-dimensional data does not always give the information of two-dimensional data correctly, which makes it inconvenient for the recognition application to process and process the compressed image. There are many. Also divide the image into several blocks,
There is another image compression method that calculates the average value and variance etc. in the block and compresses it, but in the end, this is a system that strongly considers compression rather than image recognition, and it has flexibility and expandability. Lack.

The concept of the present invention will be described below. FIG. 1 is a diagram showing the configuration concept, and uses a special symmetric neural network. In the figure, 11 is an input target image, 1 is a neural network of the present invention, 2
Reference numeral 1 denotes a lowermost neuron, and 31 denotes a topmost neuron. FIG. 2 is a diagram showing an image recognition system for making a comprehensive judgment by performing pattern analysis using the output of the neural network of the present invention. In the figure, 2 is a synapse weight of each neuron, which will be described later, and 3 is a threshold value of the neuron. Reference numeral 4 is a compressed image information output from the neural network, and reference numeral 8 is a comprehensive determination unit including a processor 7, a feature extraction unit 5, and a pattern matching unit 6 therein. FIG. 3 is an operation flowchart showing the operation of the image recognition system shown in FIG. Since the present invention relates to a neural network, the operation of this comprehensive judgment unit will be briefly described below. That is, the comprehensive judgment unit 8 sets the values of the synapse weight 2 and the neuron threshold value 3 in step S3, and obtains the compressed image 4 in step S4. If the required frequency interval is obtained in step S5, the feature is extracted in step S6. Filtering is performed in step S7, comparison is made with a predetermined stored reference image in step S8, and a target is set in step S9. If the result is correct in step S11, the image recognition operation ends.

Now, returning to the neural network portion, description will be made. Considering the simplest 2-input 1-output neuron network in terms of compression, a binary tree structure as shown in FIG. 4 is obtained. It is assumed that the brightness value of an image that is one to several pixels apart in a certain direction on the input image is simply input to the neuron closest to or lower than the input image. Here, for the sake of clarity, the input is not a color image, but a grayscale image, which is also 0-255.
It will be described as gradation. Further, regarding the brightness level, 0 is the darkest and 255 is the brightest. Each neuron and its threshold value are specifically shown in FIG.
(C) and (d).

The details of the image processing using neurons will be described with reference to FIG. 6 showing the image enhancement processing. In the figure,
The neuron processes the image by weighting the input. There are synaptic connection weights at the ends of the two input lines, and in the figure, it is assumed that the upper side is α (0.5) and the lower side is β (2). In such a case, the neuron has a role of reducing the data on the upper side of the input line and expanding the data of the lower side.

Further, when the input is multiplied by the weight of positive / negative object as shown in FIG. 7C, if the threshold function has the shape of FIG. 7A, the pixel difference information is output as it is. (B)
When the threshold function of is used, the difference information is emphasized and output as in (d). In this case, when the images are viewed in order from the top, the place where the brightness changes from dark to bright and the place where the brightness changes from light to dark are emphasized and detected. Also, "blur" or smoothing, which is the opposite of emphasis, can be realized by making the slope of the threshold function gentle.
FIG. 8 is a diagram showing another circuit set by changing the weight of the input image of the neuron. The connection shown in FIG. 8C is a circuit for obtaining the average value of the input image, and the connection shown in FIG.
Is a circuit that emphasizes the average value of the input. By the way, making the threshold function non-linear as shown in FIG. 7B is also an important purpose to improve noise resistance in addition to enhancement.
In FIG. 4, 11 is an image, 21 is a neuron,
22 is a network connecting the neurons. Further, FIG. 5 is a diagram showing the input / output relationship and the threshold function of each neuron in the configuration shown in FIG. In the figure, 41 is an input line, 42 is a synapse, and 43 is an axon of a neuron, that is, an output line. 44 is a linear output function, 45 is an enhancement process, and 46 is an output function for smoothing. 47 is a non-linear output function, and in this case, it has a noise removing capability.

Next, the operation of the neural network having the above configuration will be described. FIG. 9 is a system explanatory diagram in the case where the neuron has a seven-layered structure in the basic structure of FIG. 4 and the input image has the gradation level shown in the drawing. In the figure, it is assumed that the neuron layer closest to the image is the "1" layer of the leftmost wing in the vertical direction, and the neuron of the farthest layer is the "7" layer. It is also assumed that the input to the neurons in the first layer corresponds to the brightness value of the image. In FIG. 9 (a), information of two pixels apart is detected in the third layer, and in FIG. 9 (b), image information of four pixels separated is detected in the fifth layer. Is. In the figure, patterns are added to the neurons relevant to the following description to distinguish them. Each neuron can also be expressed by using a diagonal coordinate system such as (7, 4) as shown in the figure.

First, it will be described that the neural network according to this configuration can perform frequency analysis by using the symbols of the coordinate system. For example, in FIG. 9A, in order to determine the output of the neuron (3, 3) from the data of the neuron (1, 1) and the data of the neuron (3, 1), first, the input of the neuron (2, 2) is input. It can be determined by setting the lower weight to zero and further setting the upper weight of the neuron (3, 2) to zero. Besides this, even if the output of the neuron (2, 1) is always zero, such data every other pixel can be accessed.
In this way, it is possible to access the necessary data by changing the network structure itself each time. Next is data processing. When a real number whose absolute value of positive / negative pair is 1 or more is given as the weight of two inputs of the neuron (3, 3), the difference between the data of the already accessed neurons (1, 1) and (3, 1) is calculated. Can be amplified. If each neuron of FIG. 9 takes a threshold function as shown in FIG. 10, if two points separated by a certain distance, or if the brightness of two lines is the same if the filter is two-dimensional, it is strongly output and the brightness is If it is different, the output will be smaller. That is, if the pixels separated by a certain distance have periodicity, the vicinity thereof is output strongly, and if there is no periodicity, it is output weakly. In this way, frequency decomposition of the image can be performed.

In the above description, the weight for each input of the intermediate neuron is set to 0 or 1, and the data is transmitted as it is to the neurons in the upper layer. However, by giving various values to this as well, a little more can be obtained. A detailed analysis of the grain is also possible. For example, a convolution integral with a complex function such as a wavelet transform basis or a Gabor function can also be calculated by changing the weight and threshold function of this simple binary tree structure network. 2 in the above
Although the example of the input / output device has been described, the same effect can be obtained in the basic configuration shown in FIG.

Example 2. In this embodiment, an image recognition device using multi-resolution will be described. As described in the apparatus using FIG. 7 and FIG. 8 in the first embodiment, by emphasizing the input image by selecting appropriate weights and threshold functions for each neuron,
Blurring, edge detection, etc. can be performed. A method of collecting two pixels into one pixel while taking such an average value or a value similar thereto will be described. With this method, the positional information of the target on the input image, which is in contrast to the target image, can be kept similar to the positional relationship in the hierarchy even for neurons in the hierarchy far from the input image. FIG. 11 is an output signal diagram of neurons of each layer showing this. As can be seen from the figure, the output of the neurons in the hierarchy far from the image is obtained as the compressed information itself of the image. Furthermore, the absolute brightness value of the image,
The accuracy can be improved by specifying the weighting coefficient or the slope of the non-linear portion of the threshold function. Next, a method of using the information obtained by using the above-mentioned neural network will be mentioned. For convenience of explanation, the following words are defined. The one closer to the image in the figure is "low" in the hierarchy of the neural network
Alternatively, it is assumed to be the “lower” layer. Conversely, the direction in which the network tapers is considered to be the “high” or “upper” layer. In this way, the output of the neuron in the layer far from the image can be the compressed information itself of the image. Of course, to be exact, the absolute brightness value of the image, the weighting coefficient for amplification,
Although it is necessary to specify the slope of the non-linear function in detail, this makes it possible to use the compressed image and efficiently perform the recognition process.

FIG. 12 is a diagram showing a recognition neural network and an image compression unit for explaining the concept of the resolution switching apparatus of this embodiment. This apparatus recognizes the input image in the following order. That is, first, the entire screen is viewed with a coarse resolution, and the situation in the screen, that is, the texture and the point of interest are extracted. Next, the details of the point of interest are recognized using the data with higher resolution, that is, the amount of information for the point of interest. As shown on the right side A of the figure, a rough estimation of the object is made on the side having a low resolution and a small amount of data. At this time, a threshold change request for fine adjustment and a synapse weight change request are fed back as indicated by C. This fine-tunes the resolution of the object and increases the accuracy.
It should be noted that when the position, direction, or size is greatly changed, it is necessary to shift the recognition neural network itself as shown by B in the figure.

FIG. 13 is a diagram for explaining this easily. FIG. 13B shows a one-dimensional equivalent representation of the planar structure of each layer in FIG. 13A. This image recognition is for detecting an identification target drawn in a certain size in the image. For example, the input layer has a three-layer structure including an input layer, an intermediate layer, and an output layer, and the input layer is 16 × 16
= 256 neurons. As many output layers as necessary to classify the input patterns are needed, and if the input patterns are classified into three now, and the firing of any one of the output neurons represents the selected pattern, the output neurons will be three. Become individual. If the output layer is fixed as in the conventional case, an extremely large number of standard patterns is required for comparison, including changes in direction and size. At the start of recognition, the size of the object in the entire image, or even where it is drawn, is unknown. Since it is difficult to determine both at once, it is assumed that the size is known for simplicity in this case. In such a case, if the position of the target in the image is misaligned, that is, if it is not known where it is drawn, while shifting the recognition network into the image of a certain resolution as shown in FIG. You can find out by examining them in order. That is, it means that the coordinate position of B in FIG. 12 is switched and scanning is performed in order.

Next, consider the case where the size is unknown. However, for simplicity, it is assumed that it is known where it is drawn in the image. In this case, shifting in the direction of the dotted arrow shown in FIG. 13D means switching the selection of the intermediate layer output in the direction A of FIG. That is, by observing from a distance, it is possible to easily find a small recognition target in the entire image. Thus, it is not necessary to prepare standard patterns of various sizes, and similarly, if the coordinate position of B in FIG. 12 is rotated, the rotated standard pattern can be omitted. Such multi-step processing can be easily performed only by changing the weight of the neuron, the threshold value, and the threshold function. Further, by extracting the local information of interest or the feature extraction, it is not necessary to process the entire image with high resolution, which leads to speeding up of the subsequent processes. In addition, when recognizing, the information obtained from the entire screen is fed back to the size and inclination of the spatial filter that performs local processing, and conversely, the local information is fed back to the entire background, A great effect can be expected in separating the points of interest.

Example 3. In the above embodiment, regarding the approach of the input screen, the example in which the neural networks for processing the pixels in the vertical direction are arranged in the horizontal direction as shown in FIG. 4 has been described. On the other hand, the input connection may be rotated by 90 degrees as shown in FIG. Further, as shown in FIG. 15, it is also possible to use a network configuration having the same configuration as the conventional method of averaging planar blocks. By appropriately changing the synaptic weights and output functions of these neural networks, it is possible to configure a network that is resistant to image rotation and many spatial filters such as edge detection.

Example 4. The rounding of the recognition learning result will be described. As a result of the difference information being collected, the higher the layer in the network becomes, the more the image information is compressed and held. This is also described in the description of FIG. 11 in the second embodiment. Further, by expanding and contracting the shape of the synapse weight in FIG. 10 to the left and right, the frequency information from the high band to the low band can be obtained with the network of the same configuration, but this one should be a higher layer. The higher the frequency, the lower frequency information will be obtained. Furthermore, by changing the shape of the synapse weight in FIG. 10 in the vertical direction, it is possible to realize a wavelet transform, which is a local frequency analysis. For more details,
For example, if there is a neuron with 8 synapses as shown in FIG. 14 (a), if the values of the synapses are taken as shown in FIG. 14 (b), values will be tentatively input to the 1st and 8th input lines. If this is the case, both inputs will be emphasized due to the shape of this synapse. In other words, it has the effect of emphasizing the input coming in at a certain interval during the input. Since this picks up image data having a pattern at such intervals (wavelengths), it eventually becomes a filter that picks up frequency information.

On the other hand, a process of picking up a similar frequency can be realized even in a 2-input neuron if the structure is dynamically changed. If the threshold value of the neuron and the synapse are tampered with as shown in FIG. Processing can be performed. In this case, selection of neurons, threshold values, and synapses are all changed. In the second embodiment, the luminance (intensity) information of the image is input and processed at various resolutions, but according to the configurations and methods of FIGS. 16 and 17, the threshold function is changed according to the distance. This allows processing at various frequency levels, using exactly the same structure as the processing of luminance information. Therefore, there is no structural change and only the threshold function needs to be changed. With this change, the shades of pixels with wider intervals are monitored above the hierarchy, and the wavelength is doubled as 2, 4, 8, and 16. It increases and the frequency changes by half. Information that could not be obtained by convolution integration using separate spatial frequency filters from small size to large size in the conventional method is automatically detected in the structure of the neural network from high frequency to low frequency components. It will be possible to extract, and it is expected that the calculation amount will be greatly reduced.

FIG. 12 also means the following: That is, even when it is used as a normal image compression, it is possible to first look over the entire screen at a coarse resolution and extract the situation in which the image is drawn, that is, the texture and the point of interest with a small amount of calculation. This processing is performed by inputting the low-resolution image information appearing in the intermediate layer to another neural network for recognition, and first making a rough estimate. If this recognition neural network is learned in advance to identify an object of a certain size, the same recognition networks are arranged from low resolution to high resolution, or the recognition network is changed from low resolution to high resolution. By performing pattern matching while shifting, it is possible to find an object with a changed size. The attention point is first extracted by the low resolution information, then the resolution is increased, the information of the attention point at the same relative position is taken out, and the information is input to the higher resolution neural network for recognition. By such an operation, stepwise processing such as gradually recognizing details can be performed, and the system can be an excellent system in terms of development of a recognition hardware system and calculation cost.

[0033]

As described above, according to the present invention, since the neuron type threshold value element having the weighting coefficient whose value can be changed is hierarchically connected, the elements having the same structure are connected to compress, emphasize and smooth the image. There is an effect that can be processed.

Furthermore, there is an effect that the recognition result of the intermediate layer can be effectively used.

Furthermore, since there are two inputs, the connection is simpler and there is the effect that the result is emphasized by the addition and difference of the immediately preceding inputs.

Furthermore, since the weighting factor and the threshold value are changed according to a part of the results, there is an effect that precision processing of the entire image can be omitted and the speed can be increased.

Furthermore, since the weighting coefficient or the threshold value is changed according to a part of the recognition result, there is an effect that the frequency characteristic of the image recognition can be changed.

[Brief description of drawings]

FIG. 1 is a general configuration diagram of an image compression unit of the present invention.

FIG. 2 is a configuration diagram of an image recognition system using a neuro network of the present invention.

FIG. 3 is an operation flowchart of the entire image recognition system.

FIG. 4 is a configuration diagram of a 2-input 1-output element of the image compression unit of the present invention.

FIG. 5 is a diagram showing an input / output and a threshold function of a neuron which is a basic constituent element in the present invention.

FIG. 6 is a diagram illustrating an emphasis process of directional difference information.

FIG. 7 is a diagram illustrating an emphasis process of direction difference information using symmetric weights and two types of threshold functions.

FIG. 8 is a diagram illustrating an emphasis process of an input image.

FIG. 9 is a diagram illustrating a process of emphasizing difference information having directions.

FIG. 10 is a diagram showing a frequency analysis threshold function.

FIG. 11 is a diagram showing an example of multi-resolution representation.

FIG. 12 is a diagram showing a combination of a recognition neural network and an image compression unit.

FIG. 13 is an explanatory diagram of using the intermediate layer output of the present invention for image recognition.

FIG. 14 is a block diagram of an image compression unit for horizontal compression of the present invention.

FIG. 15 is a configuration diagram of an image compression unit for plane compression of the present invention.

FIG. 16 is a diagram showing neurons and synapses for frequency analysis.

FIG. 17 is a diagram showing changes in the threshold for frequency analysis and changes in neuron output.

FIG. 18 is a diagram showing a first conventional image compression method.

FIG. 19 is a diagram showing a second conventional image processing method.

FIG. 20 is a diagram showing a third conventional image processing method.

[Explanation of symbols]

11 images, 21 neurons, 22 connections between neurons, 41 input lines, 42 synapses (weights), 43
Output line, input / output characteristics of 44 neurons f (x) =
Input / output characteristics of x, 45 neurons Gain> 1,46
Input / output characteristics of neuron Gain <1,47 Input / output characteristics of neuron f (x) is non-linear.

Claims (5)

[Claims] 1. A structure in which a neuron-type threshold value element having a weighting coefficient for each of a plurality of inputs and the neuron-type threshold value element are hierarchically connected in a pyramid type, and the output of the uppermost neuron is the final output, An image recognition apparatus including a storage unit that variably sets the value of the weighting factor and the threshold value. 2. The image recognition apparatus according to claim 1, wherein in image recognition, the final output is the output of the intermediate layer at an arbitrary coordinate position of an arbitrary intermediate layer of the hierarchy. 3. A neuro-type threshold element having two inputs and two inputs and one output is used as a plurality of inputs.
Alternatively, the image recognition device according to claim 2. 4. The image recognition apparatus according to claim 1, wherein the value of the weighting factor and the threshold value are changed based on the recognition result of a part of the hierarchical connection of other neuron type threshold value elements. 5. The image recognition apparatus according to claim 4, wherein the value of the weighting factor or the threshold value is changed based on the recognition result of the distance information of a part of the hierarchical connection of other neuron type threshold value elements.