JPH02165285A - Pattern recognition device - Google Patents

Abstract

PURPOSE:To accurately recognize a pattern only by a pattern input by inputting 1 or 0 binary data to an input layer and using data from an output layer and the 1 or 0 binary data corresponding to the character pattern of a JIS code as educator data in a hierarchical network composed of three layers, which are the input layer, an intermediate layer, and the output layer. CONSTITUTION:A three-layer laminated neutral network having the input layer, the intermediate layer, and the output layer is composed of a unit 11 called as a neutron model, a synapse tie-branch 12 having a weight coefficient, an arithmetic part to multiply the weights of internal connections with each other and obtain the sum of the products, and a threshold value processing part to generate one output. Further, the pattern recognition device is composed of an interface part 21 to designate the number of units and the number of the times of learning for each layer, a network generating part 22, a pattern input part 23 to read in input data and the educator data, a learning execution part 24 to update the weight of each layer, a parameter setting part 25 to set the learning, a weight retaining part 26, a recognizing part 27, and a result display part 28.

Description

[Detailed Description of the Invention] [Summary] Regarding a pattern recognition device using a hierarchical neural network, an object of the present invention is to provide a pattern recognition device that reproduces an incomplete pattern into a complete pattern using a hierarchical neural network. A network generating means that generates a network with a three-layer structure of an input layer, a middle layer, and an output layer, and inputting each element of a two-dimensionally arranged pattern to each unit of a one-dimensionally arranged input layer, and passing it through the middle layer. learning calculation means for calculating the difference between the output data obtained from the output layer and the training data and determining the weight of the connection between the respective layers according to a backpropagation algorithm so that the difference becomes small; a storage means for storing connection weights between the networks; and a storage means for loading the connection weights from the weight storage means at the time of recognition, inputting a pattern to be recognized to each unit of the input layer, and outputting from the obtained output layer. The present invention is configured to perform pattern recognition using a hierarchical neural network having recognition means for outputting a pattern corresponding to teacher data whose difference is smaller than the difference between the data and the teacher data as a recognition pattern.

[Industrial Application Field] The present invention relates to a recognition device that reproduces a pattern with ambiguity into a complete pattern, and more particularly to a pattern recognition device using a hierarchical neural network.

Neurocomputers, which perform complex processing such as recognition and association by interconnecting neurons that perform simple processing, similar to the brains and nerves of living things, are attracting attention, and the basic circuit network to realize such neurocomputers is It's called a neural network. Neural networks perform computer and memory processing in massively parallel fashion, avoid the use of software, and instead automatically adjust the weights of the connections between neurons in the network, learn, and self-organize to adjust the weights. Decisions are made, and subsequent recognition and associations are made.

[Conventional technology]

In conventional pattern recognition, a Neumann computer is used to extract the features of a pattern, store a reference pattern created by the extraction, and match it with a given pattern.

[Problems to be solved by the invention] Conventional pattern recognition requires a huge amount of program and time to extract the features of a pattern, and also requires human intervention to create a dictionary for recognition, resulting in input errors etc. There is a problem in that there are errors in the dictionary data, and therefore, if the pattern does not match the dictionary, it may become unrecognizable. Furthermore, it is difficult to retrieve a complete pattern from a dictionary when an incomplete pattern is input, such as when a part of the pattern is damaged or there is noise in the pattern. This has caused the problem that processing cannot be performed.

The present invention aims to reproduce an incomplete pattern into a complete pattern using a hierarchical neural network.

[Means to solve the problem]

FIG. 1 is a diagram showing the principle configuration of the present invention.

The present invention uses a hierarchical neural network to recognize -patterns, and the network generation means 1 generates a network with a three-layer structure of an input layer, an intermediate layer, and an output layer, and the learning calculation means 2 Each element of the two-dimensionally arranged pattern is input to each unit of the one-dimensionally arranged input layer, and the difference between the output data obtained from the output layer via the intermediate layer and the training data is calculated and the difference is minimized. Then, the weights of the connections between the layers are determined according to a backpropagation algorithm. Further, the storage means 3 stores the weights of the connections between the networks as a result of the learning calculation, and loads the weights of the connections from the weight storage means 3 at the time of recognition,
A pattern to be recognized is input to each unit of the input layer, and the recognition means 4 outputs a pattern corresponding to the teacher data whose difference is smaller than the difference between the output data from the output layer obtained and the teacher data as a recognition pattern. do.

[For production]

In the present invention, in a hierarchical neural network having three layers: an input layer, a middle layer, and an output layer, binary data represented by , l or 0 is input to each unit, and output data is output from the output layer. I would like to learn by changing the weights of connections between units in each layer so that the error with the output data is small, using binary data of l or 0 corresponding to the character pattern of the JIS code as training data, and recognize it at the time of recognition. It is assumed that the input pattern is recognized when the error between the input data of the pattern and the output data outputted and the teacher data is small.

〔Example〕

FIG. 2 is a block diagram of a hierarchical neural network used in the present invention. In the figure, 11 is a unit called a formal neuron model, and 12 is a synalapse connection technique having weighting coefficients. A hierarchical neural network has a three-layer structure of an input layer, a middle layer, and an output layer, but there may be a plurality of middle layers. Each unit includes a product-sum operation section that multiplies each of the plurality of inputs by the weight of each internal connection and calculates the sum of these products, and a dark value processing section that further performs dark value processing and outputs one output. have.

FIG. 3 is a configuration diagram of a unit used in the present invention. In the unit shown in the figure, the calculation executed is shown by the following equation.

Xj,=ΣyWih−θ.

ypi=f h: i: p: θi: W, h: (Xpt) Unit number of layer h Unit number of layer i Cover turn number Threshold of the i-th unit of layer i h - Weight of internal connection between layer i xp , : sum of products of inputs from each unit of h layer to the 1st unit of i layer yph: output of h layer for P pattern input)'pi:
Output f of layer i in response to P pattern input: Dark value processing function Units in each layer execute the operation shown by the above formula. In the learning process, the weights on the net are determined so that the error between the input data and the output data with the teacher data is small. The following function, ie, the sigmoid function, is used as the darkness value function.

f (x)=1/ (1+exp (-x)) The learning process in this hierarchical neural network, ie, the backpropagation method, will be briefly explained. When the dark value processing function, that is, the sigmoid function is partially differentiated with respect to X, it becomes the following equation.

Error from the vector, that is, Ep = 1/2 Σ (ypJ - d9J) 2E = ΣEp However, EpThe error vector E for the SP-th pattern input:
The sum of error vectors for all pattern inputs dp'''In the learning algorithm using the back propagation method of training data to the j-th layer and the j-th unit for the P-th pattern input, the weight δWjl (1) is repeatedly updated using the steepest descent method. By doing so, E is converged to the minimum value. Therefore, the update amount of the weighting coefficient is expressed by the following equation.

The optimal weight search algorithm gives initial values of weights,
Teacher input vector and network output where ε and α are control parameters. That is, in the backpropagation method, the weight values are determined so that the error between the output data and the teacher data is minimized. In other words, in a hierarchical neural network, by repeatedly giving a set of input data and training data as a pattern,
By automatically adjusting and learning the weights of connections between units, and using the learned weights to provide human data, that is, incomplete information, it becomes possible to reproduce the corresponding output data, that is, complete information. Therefore, by incorporating this hierarchical neural network into a machine that reads patterns, it becomes possible to perform pattern recognition without programming.

FIG. 4(a) is a block diagram of the pattern recognition apparatus of the present invention, and FIG. 4(b) is a diagram showing the procedure of each part of pattern recognition. In the figure, 21 is an interface section for specifying the number of units in each layer and the number of learning times, 22 is a network generation section for generating a network, 23 is a pattern input section for reading human data and teacher data, and 24 is an intermediate layer section. a learning calculation unit that obtains data and output data and updates the weights of the intermediate layer and the output layer and the weights of the input layer and the intermediate layer based on the difference between the output data and the teacher data; 25 is a parameter setting unit that sets parameters related to learning; 26 is a weight storage unit that stores the weights of the input layer and the intermediate layer, and further stores the weights of the intermediate layer and the output layer; 27 is a recognition unit that receives input data and calculates and recognizes the intermediate layer data and output data; 28 is a result display section that displays output data. When a user inputs a command, the number of units in each layer of the hierarchical network, learning parameters, number of learning times, etc. are specified according to the command, and a network is formed. Next, a set of input data and corresponding training data is given, and learning begins using a backpropagation algorithm. When learning is completed a pre-specified number of times, the learning is terminated and the weights of connections between units are determined. Save this weight. During recognition, when input data of a pattern to be recognized is given to the network, the network displays output data using the weight of connections between each unit.

FIG. 5 is an embodiment of a pattern recognition device that recognizes alphabets drawn on 8×8 dots.

In the same figure, 31 is input data of 8×8 dots, 32
is an input layer consisting of 64 units, 33 is 1
The intermediate layer has a number of units of 3, and 34 is the output layer that has a number of units of 26. The teacher data is prepared for 26 characters, which is the same as the number of units in the output layer, and the answer to the input data is set to the value l, and the others are set to O. The 8×8 dot character shown in FIG. 5 is an alpha-headed character A, and the part corresponding to A is 1, and the other part is 0. The zero threshold pattern for each of the 64 dots is input to each of the 64 neurons in the input layer 32. The number of units in the intermediate layer 33 is generally smaller than the number of units in the input layer 32 or the number of units in the output layer 34, and is 13 here. Output layer 34
The number of units is 26, which is the alphabet from A to Z.
It corresponds to In this example, the pattern input during learning is the complete information of the letter A, and the output layer data for this input data is the output data for the initial weight value, so the alphabet A
Only the top unit of the output layer corresponding to is 1 and the others are O. A signal that is not the same as the teacher data is output, and this output data and the top unit are 1 and the others are O. The error with certain teacher data, that is, the minimum value error, is calculated, and the weighting coefficient between the input layer 32 and the intermediate layer 33 and the weighting coefficient between the intermediate layer and the output layer are calculated so that the error is minimized. Determined according to the backpropagation method. Only the weighting coefficients determined in this manner are stored in the storage unit. It is assumed that during recognition, the input data input to the input layer is not the complete information entered during learning, but is the character A, which is incomplete, that is, contains noise. Even in such a case, even if an input corresponding to a noisy dot is received, the corresponding weighting coefficient will be small and the noise will be suppressed and removed according to the weighting coefficient determined during learning. Therefore, incomplete information is converted into complete information and the result is output as output data, so that pattern matching is executed with the difference from the teacher data becoming very small.

FIG. 6 is a diagram showing an example of alphabetic character patterns and corresponding teacher data by the pattern recognition device of the present invention. In the figure, at the top is the input data of the letter A and the corresponding teacher data.The teacher data is 26-bit information, and the leftmost one represents 1, that is, the alphabet A. The next data is the input and teacher data for data B, so the teacher data with 1 in the second bit is shown. The third letter is the alphabet C, and the corresponding teacher data is data in which the third bit is 1. The next pattern is the letter D, and the teacher data corresponding to it is a signal in which the fourth position is 1. The fifth character is the letter E, and the corresponding teacher data is a pattern in which the fifth character is 1.

FIGS. 7(a) and 7(b) show recognition results using the recognition device of the present invention. In the figure (a), the input data is complete information of a character named pattern A. On the left is Ol
The pattern on the right is a clear representation of the corresponding binary data. 41 is output data, and in this case, the output has almost the same value as the teacher data. That is, although values between 0 and 1 are shown, the voltage value corresponding to the leftmost A is the largest at 0.901, and the other values are mostly close to 0. In this input data, A is the closest pattern, but the next ones are P, G, and N, and the output data corresponding to P is 0.0
The value corresponding to G at position 6 is 0.058, and the value corresponding to H is 0.053. Therefore, in descending order of similarity, the order is A, P, G, and H. This is the recognition result of the present invention corresponding to the complete pattern A.

In contrast, Fig. 7 et al.) is a pattern of the letter A with noise. The left side is input data 42. On the right is a clear pattern of this in black and white, with the letter A mixed with noise. The output result when such incomplete information is input to the pattern Ly2 recognition device of the present invention at the time of recognition is shown as a voltage value. In this case, since noise is included, the voltage value corresponding to A is 0.772, which is lower than the voltage value of 0.901 for the previous complete pattern. However, the voltage value is considerably higher than the output voltage values corresponding to other alphabets. Also, the output voltage value of the output line other than the output line corresponding to the alphabet A is relatively high at 0°236■ corresponding to M, and the next one is R
The voltage is 0.059 V, and the next voltage is 0°052■.

Therefore, the order of similarity is A, M, R, and L.

FIG. 8 is a human output relationship diagram of hiragana using the pattern recognition device of the present invention. In the figure, reference numeral 51 denotes input data in which hiragana is given as a 16×16 dot pattern, the number of units in the input Ji 52 is 256, the number of units in the intermediate layer 53 is IO, and the number of units in the output layer 54 is 16. In this case, the teacher data is expressed in JIS code, hexadecimal, and l or O for input data.

The character of the input data in Figure 8 is a pattern corresponding to the character "a", and the teacher data for it is a pattern shown in JIS code 2422, 001001000.
It is 0100010. Therefore, if we enter this input data, will the output data be the teacher data? ) The weighting coefficient is determined to be JIS code 2422. Then recognition will take place.

FIGS. 9(a), (b), and (1:) are examples of hiragana character patterns produced by the pattern recognition device of the present invention.

Figure (a) shows a pattern of a small “A” and a large “A”, and the corresponding training data is JIS.
A pattern corresponding to the chord "a" is shown. Figure (b) shows input data and teacher data for small "i" and large "i". The same figure (C) shows the input data pattern of small "U" and large "U" characters and the corresponding teacher data.If they are the same "U", even if it is "U", the pattern of the teacher data is the lower bit The sides are slightly different.

FIGS. 9(d) and 9(e) show the results of hiragana recognition by the pattern recognition device of the present invention. The figure (d) shows complete information, which is the input data of the character pattern "A" and its binary image, and shows the output voltage as a result of recognition. In this case, the same information as the teacher data is output. That is, the voltage value for the binary value 0010010000100010 is shown. In the same figure (e), the pattern “AJ” is given as incomplete information.
It contains noise. Even when such an incomplete pattern is input, the present invention reproduces the incomplete information into complete information, and the output voltage has almost the same voltage value as the pattern of teacher data corresponding to the complete information.

That is, the output voltage is 0010010000, 11.0
0001.00.1, and the output voltage is slightly different by 0.1■, but as a binary voltage, this almost corresponds to the O position pattern of the teacher data, so the pattern " It is recognized as the character ``A''.

FIG. 10(a) is a functional block diagram of the pattern recognition learning procedure of the present invention. When a command is input (Sl), a network is generated at 32.

At the time of network generation, the connection weights of each layer among the input layer, intermediate layer, and output layer are undetermined. In S3, the pattern to be stored and the teacher data are input. In S4, learning processing is performed according to the backpropagation method.

That is, the difference between the output data according to the input data and the teacher data is taken, and the weight of each layer is determined by back-propagating the difference so that the least squares error is minimized. Then, the weights are determined in S5. In S6, it is determined whether or not to save this weight, and if it is, the weight is saved in S7 and the learning process ends. If there is no need to save the weights, the learning process ends.

FIG. 10 is a functional block diagram for recognition processing of the pattern recognition device of the present invention. At the time of recognition, the weights determined by the learning process are loaded from the storage unit (S11). Assign the loaded weights to each corresponding branch of the network,
In step 312, the recognized character pattern is input. 31
In step 3, output data is calculated on the network, and if the output data to be outputted has almost the same pattern as the teacher data, the pattern corresponding to the teacher data is the recognized result, and in step S14, this output data is displayed, finish.

〔Effect of the invention〕

As described above, according to the present invention, a user can perform pattern recognition simply by inputting a pattern without creating a dictionary or program for extracting pattern features. Furthermore, it has a high tolerance for defects and noise in parts of characters, and can perform accurate pattern recognition.

In addition, in this hierarchical neural network, if the number of units in the intermediate layer is small, there will be fewer connections within the network (this will result in faster learning times and higher efficiency. Also, the required storage capacity will be smaller).

[Brief explanation of the drawing]

Figure 1 is a diagram of the principle of the present invention; Figure 2 is an explanatory diagram of the backpropagation method; Figure 3 is a diagram of the configuration of each unit in Figure 2; Figure 4(a) is the pattern recognition device of the present invention. 4, etc.) are diagrams showing the procedures of each part of the pattern recognition device, FIG. 5 is an example diagram of a pattern recognition device that recognizes alpha vents drawn on 8×8 dots, and FIG. 6 is a diagram showing the method of the present invention. FIGS. 7(a) and 7(b) are diagrams showing recognition results when the recognition device of the present invention is used. The figure is a hiragana input/output relationship diagram using the pattern recognition device of the present invention, Figures 9(a) to (C) are examples of hiragana character patterns using the pattern recognition device of the present invention, and Figure 9(d) ) and (e) are diagrams showing the results of hiragana recognition by the pattern recognition device of the present invention, and FIGS. 10(a) and (b) are flowcharts showing the learning procedure and recognition procedure, respectively. l...Network generation means, 2...Learning calculation means, - Storage means, - Recognition means.

Claims (1)

[Claims] 1) Network generation means (1) for generating a network with a three-layer structure of an input layer, an intermediate layer, and an output layer; and an input layer in which each element of a two-dimensionally arranged pattern is arranged one-dimensionally. Calculates the difference between the output data obtained from the output layer through the intermediate layer and the teacher data, and determines the weight of the connection between each layer according to the backpropagation algorithm so that the difference is small. a learning calculation means (2); a storage means (3) for storing connection weights between the networks as a result of the learning calculation; recognition means (4) for inputting a pattern into each unit of the input layer and outputting a pattern corresponding to the teacher data whose difference is smaller than the difference between the output data from the output layer obtained and the teacher data as a recognition pattern; A pattern recognition device characterized by performing pattern recognition using a hierarchical neural network. 2) The pattern recognition device according to claim 1, wherein the output data uses binary information corresponding to a standard code corresponding to a character pattern. 3) The pattern recognition device according to claim 1, wherein the output data is a code arranged one-dimensionally for a certain pattern according to the weight of the connection. 4) The pattern recognition device according to claim 1, wherein the output data has a value in only one output unit for a certain pattern according to the weight of the connection.