JPH05165971A - Method and device for pattern recognition using neural network - Google Patents

Abstract

PURPOSE:To detect misrecognition and make a decision at a high speed by deciding a pattern according to the internal state of the neural network. CONSTITUTION:A pattern decision means 102 makes the decision according to the mutual relation among the output 101 of the neural network, the pattern of the internal state of the neural network, and the pattern of the internal state of the neural network which is learnt already. Thus, the misrecognition can be detected not only by deciding the pattern with the output 101 of the neural network for input data, but also by comparing even the output of an intermediate layer with the pattern of the output of the intermediate layer which is learnt already, and correct candidates can also be indicated unless the results of the both are equal to each other. Further, the pattern is decided with the internal state of the neural network to eliminate the subsequent calculation of the neural network, and the calculation is speeded up. Consequently, the misrecognition can be detected and the decision making is performed at a high speed.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a data processing device and method, and more particularly to a neural network suitable for pattern recognition.

[0002]

2. Description of the Related Art A method for recognizing a pattern using a neural network is described in Em. Eye. Tea Press (1986
Year) Parallel Distributed Processing, pages 318 to 362 (Parallel Distributed Proce
ssing, Vol.1.MIT Press, Cambridge MA, 1986 pp.318-
362).

A neural network for pattern recognition will be described here. The neural network is formed by connecting neurons in cascade in multiple layers, and FIG. 2 shows an example of a three-layer neural network. 1000,100 in the figure
1 is an input neuron, 1003 and 1004 are middle layer neurons, 1006 and 1007 are output neurons, 1002
1003 is a bias neuron. The input and bias neurons output the input as they are, but the intermediate layer and the output neuron have an input-output relationship with a saturation characteristic called a sigmoid function as shown in FIG. 3, where x is the input and z is the output.

[0004]

[Equation 1]

[0005] Here, T is a constant that determines the slope of the sigmoid function.

When the number of each layer is given from the input side, the input and output of the i-th layer and the j-th neuron are respectively x _j
(I) and z _j (i), the input / output relationship of the input neuron is as follows.

[0007]

[Equation 2]

[0008] Here, n (i) is the number of inputs to the i-th layer, and z _{n (i) +1} (1) is a bias term.

The outputs of the i-th neurons in the second and third layers are

[0010]

[Equation 3]

[0011] Here, z _{n (i) +1} (i) is a bias term.

Neurons between layers are completely connected via synapses, each synapse has a weight, and the output of the neuron is multiplied by the weight to be the input to the next-stage neuron. Therefore, the input to the j-th neuron in the second and third layers is given by the following equation.

[0013]

[Equation 4]

However,

[0015]

[Equation 5]

Is a weight vector w _jk (i-1) is (i-
1) Synaptic weight between the k-th neuron in the i-th layer and the j-th neuron in the i-th layer,

[0017]

[Equation 6]

Is the output vector of the (i-1) layer neuron, and t is the transpose of the matrix.

Input data is n by the above neural network.
(3) When separating into i classes, the i-th output neuron is made to correspond to the class i, and it is determined that the i-th output is 1 for one input and the other i is 0 for the class i. .. In order to separate the patterns in this way, the weight w _jk (i) (i, 2,3) needs to be appropriately determined.
Therefore, the weight is determined by learning using the input / output data as teacher data. Now the values of m teacher data

[0020]

[Equation 7]

Then,

[0022]

[Equation 8]

The weight w _jk (i) is determined so that However, z _jl (3) is the output of the output neuron corresponding to the teacher data input x _il (1). As a method for determining the weight, the back propagation method shown in the above document is widely used.
In this method, a set of teacher data inputs x _il (1), i =
The output z _jl (3) with respect to 1, ..., N (1) is s _jl , j = 1,
..., the weights are sequentially corrected from the output side to the input side so as to approach n (3), and when that is completed, the above procedure is repeated for the next teacher data.

[0024]

[Equation 9]

The correction is repeated until the condition is satisfied. Where ε
Is a small positive number for determining that it has converged.

After deciding the weight of the neural network in this way, pattern recognition becomes possible by examining the output of the network for unlearned data. In this method, a pattern identification net can be constructed by learning based on an input / output pattern, which is a great advantage that no identification algorithm is required.

In addition, as far as pattern recognition is concerned, the 41st National Convention Proceedings of the Information Processing Society of Japan (second half of 1990)
It is also clear how the network is created, as shown in Volume 2, pages 113-114.

[0028]

However, since the neural network itself is determined by learning, it is not possible to know whether or not the neural network is making a correct decision, and the network is multi-layered, which requires a processing time.

An object of the present invention is to solve the above problems and provide a data processing device and method using a neural network that detects erroneous recognition and makes a high-speed judgment.

[0030]

The present invention is characterized in that (a) erroneous recognition is detected, (b) pattern discrimination is performed at high speed, and the like, and not only these independent means and methods, The various combinations also have their own characteristics.

An example will be described below.

(B) Detecting erroneous recognition When pattern recognition is performed by a neural network, the output of the neural network for the teacher data is 1 or 0.
And the saturation characteristic of the sigmoid function has an important meaning. Since the inputs to the output neurons corresponding to the outputs 1 and 0 of the neural network correspond to + ∞ and −∞, assuming that 1 and 0 are 1, −ε and ε, respectively, and the learning is terminated by the equation 9, Equation 9 is when s _il = 1-ε

[0033]

[Equation 10]

When s _il = ε

[0035]

[Equation 11]

It becomes Considering this as an input to the output neuron, as shown in FIG. 4, when s _il = 1-ε

[0037]

[Equation 12]

When s _il = ε

[0039]

[Equation 13]

However, α = -Tlog (1 / (1-2ε) -1) = Tlog (1 / 2ε-1) That is, the finite section on the output side corresponds to the semi-infinite section on the input side. Thus, the minimization of Eq. 8 can be obtained by solving an inequality regarding the determination of the weight between the hidden layer and the output layer.

From this, the weight of the neural network can be interpreted as the coefficient of the equation describing the hyperplane. X in number 4
_{If j} (i) = 0

[0042]

[Equation 14]

Represents the hyperplane of the n (i-1) -dimensional space, and changing the value of the weight w _{j, n (i-1) +1} (i-1) of the bias term makes the hyperplane parallel. Corresponding to moving. The value of z _j (i) corresponding to x _j (i) satisfying Expression 12 from Expression 1 is 1/2. Where n (i-1) -dimensional points

[0044]

[Equation 15]

[0045]

[Equation 16]

When is on the positive side of the hyperplane,

[0047]

[Equation 17]

At the time of, it is assumed to be on the negative side of the hyperplane. A class is called separable by k hyperplanes when all the teacher data belonging to that class are on the same side of k hyperplanes and no other class data exists. .. A class is called separable when teacher data of a class is divided into a single separable set. These separable hyperplanes are hereinafter referred to as separable hyperplanes. At this time, the following can be shown.

Consider separating n (1) -dimensional data into n (3) classes. If n (1) -dimensional space has n (2) hyperplanes and all n (3) classes are separable by a subset of n (2) hyperplanes, the input n ( Pattern separation is possible with a three-layer neural network of 1), intermediate layer neurons n (2), and output n (3). A class can be divided into several subsets, each of which can be separated by a single hyperplane, and can be separated by a four-layer neural network.

From the above, if the weights of the neural network are learned according to the teacher data and the weights of the intermediate layer are set so as to be the separating hyperplanes, the intermediate values when the results obtained by inputting the unlearned data are correct are judged. If the output pattern of the layer and the actual output pattern of the intermediate layer are compared and they match, the judgment is correct. Further, when they do not match, it is possible to find a candidate that seems to be correct by examining the output corresponding to the actual intermediate layer pattern.

(B) Performing pattern discrimination at high speed When the separation plane is correctly learned, pattern separation can be performed by comparing the output of the intermediate layer neuron with the pattern of the intermediate layer neuron learned by learning.

[0052]

[Action]

(B) Detection of erroneous recognition Not only the pattern is discriminated by the output of the neural network for the input data, but the erroneous recognition can be detected by comparing the output of the intermediate layer with the learned pattern of the output of the intermediate layer. If the results do not match, correct candidates can also be shown. (B) Performing pattern determination at high speed By performing pattern determination based on the internal state of the neural network, the subsequent calculation of the neural network becomes unnecessary and the calculation can be speeded up.

[0053]

EXAMPLE FIG. 1 shows a specific example of the present invention. In the figure, 101 is a learned neural net, 102 is a pattern determining means, 103 is a display for displaying the determination result, and 104 is a learned net pattern stored in the memory. First, 104 will be described. After learning the neural network using the teacher data, it is possible to set the separation hyperplane corresponding to each pattern by observing the output of the intermediate layer. Table 1 shows learned patterns corresponding to the separated hyperplanes when the numbers 0 to 9 are learned in the three-layer neural network with the number of neurons in the intermediate layer being four.

[0054]

[Table 1]

This pattern is obtained by classifying the output of the intermediate layer neuron for each number of teaching data by setting 1 if the output of the intermediate layer of the learned neural network is larger than 0.5 and 0 if it is smaller. Here, it is assumed that x includes both 0 and 1. When the output of the middle layer of a certain number is not separated by some teacher data, the boundary point of 0 and 1 may be changed for each neuron instead of 0.5. Further, when the output is 0.5, the input is 0 according to the equation 1, so the input may be used for the determination. In this way, it is assumed that the learning pattern for each class for which pattern separation is desired is stored in 104. For a neural network having four or more layers, the intermediate layer learning the separating hyperplane may be taken. Next, the pattern determination means 102 outputs the output from 101 and the internal state and 10
Based on the learned pattern from 4, the class to which the input to the neural network belongs is determined. First, the maximum value is calculated from the output data. In normal judgment, the class is judged by the largest one. In the case of numeral recognition, it is assumed that this is the ninth neuron, that is, numeral 9. This judgment is the same when the sum of the inputs of the output neurons is seen. Next, 1 corresponding to the class determined by the maximum value of the output
The learned pattern of 04 is taken out. The number corresponding to the number 9 is 0000. Next, when the pattern of the input sum (or output) to the intermediate layer neuron corresponding to the learned pattern is converted to the output, if it is 0110, it does not match the learned pattern, so the output determination result may be incorrect. .. If the patterns match, the output judgment is correct. If they do not match, the output corresponding to the intermediate layer pattern is obtained and detected as a candidate for correct determination. In the case of 0110, it can be seen from Table 1 that there are 4 possibilities. Next, this result is displayed by 103.

When the correct separation hyperplane is completely learned by the neural network, pattern separation can be performed by the output (or the sum of inputs) of the intermediate layer neurons learning the separation hyperplane and the learned pattern. An example thereof is shown in FIG. In the figure, reference numeral 105 denotes a pattern determination means for recognizing a pattern from the learned pattern 104 and the internal state pattern of the neural network. 5 in FIG.
Although an example of layers is shown, the same configuration can be obtained even when there are four or more layers by taking an intermediate layer learning the separation plane. 105
Search for an internal state corresponding to the input data of the net, that is, a pattern of 104 that matches the pattern of the sum of the inputs to the intermediate layer neurons in this example, and use the class corresponding to the matched pattern as the determination result. In the above example of numeral recognition, the pattern of the intermediate layer corresponding to the input data is 000.
When set to 0, it is determined from Table 1 that the number is 9. Than this one
In 03, the judgment result is displayed as 9. The determination of 105 can be performed at a higher speed according to a two-branching method in which the state of the pattern is not matched but the two-branching method is used.

FIG. 6 shows an example in which the present invention is applied to a vehicle number recognition system. In the figure, the number plate of the car is
Captured on an image recognition device on an industrial television. The image recognition device cuts out a number plate, cuts out characters one by one, and extracts the characteristic amount of the character. The extracted feature amount is input to the neural network, and the identification result is output. Detection of false recognition and detection of correct answer candidates by the method of FIG. 1 or 5 shown above,
Alternatively, recognition can be speeded up.

[0058]

The present invention has the following effects.

(B) Detecting erroneous recognition By recognizing patterns in both the output of the neural network and the internal state, erroneous recognition can be detected and correct answer candidates can be indicated.

(B) Performing pattern judgment at high speed By judging only by the internal state of the neural network, the amount of calculation can be greatly reduced and the judgment can be speeded up.

[Brief description of drawings]

FIG. 1 is a diagram illustrating a specific example.

FIG. 2 is a configuration diagram of a neural network.

FIG. 3 is a diagram showing a sigmoid function.

FIG. 4 is a diagram showing convergence determination conditions.

FIG. 5 is a diagram illustrating another embodiment.

FIG. 6 is a diagram applied to vehicle number recognition.

[Explanation of symbols]

101 ... Learned neural network, 102 ... Pattern determination means, 103 ... Display.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Toshihiko Nakano 5-2-1, Omika-cho, Hitachi-shi, Ibaraki Hitachi Ltd. Omika factory

Claims (6)

[Claims] 1. A method of performing pattern recognition by using desired output data for input data as teacher data, and storing input / output relationships of some teacher data in a neural network in which several neurons are connected in multiple layers. Then, a pattern recognition method is characterized in that the pattern determination is performed according to the mutual relationship between the output of the neural network, the pattern of the internal state of the neural network, and the pattern of the internal state of the learned neural network. 2. The pattern recognition method according to claim 1, wherein when the pattern of the internal state of the learned neural network corresponding to the output of the neural network and the pattern of the current internal state are different, the reliability of the determination by the neural network is improved. A pattern recognition method characterized by displaying low. 3. The pattern recognition method according to claim 1, wherein when the determination from the internal state and the determination of the output are different, candidates for the determination estimated from the internal state are displayed. 4. A method for performing pattern recognition by using desired output data for input data as teacher data, and storing input / output relationships of some teacher data in a neural network in which several neurons are connected in multiple layers. Then, the final result is determined according to the internal state of the neural network. 5. A device for performing pattern recognition, wherein desired output data for input data is used as teacher data, and input / output relationships of some teacher data are stored in a neural network in which several neurons are connected in a multi-layer. A pattern recognition device comprising means for determining a pattern in accordance with a mutual relationship between a net output, a pattern of an internal state of a neural network, and a pattern of an internal state of a learned neural network. 6. A device for performing pattern recognition by using desired output data for input data as teacher data, and storing input / output relationships of some teacher data in a neural network in which several neurons are connected in multiple layers. And a pattern recognition device having means for determining a pattern based on the internal state of the neural network.