JPH05265991A - Pattern sorting device - Google Patents

Abstract

PURPOSE:To increase the storage capacity of a pattern without increasing learning time for every one pattern by connecting plural small-scale neural networks in parallel. CONSTITUTION:This device is provided with a judging neural network, a pattern sorting neural network A, and the pattern sorting neural network B. Each network is constituted of a common input layer 1, intermediate layers 2 to 4 arranged in parallel, and output layers 5 to 7. The judging network judges by which of the sorting network A or B an input pattern was learnt. The output layers 6, 7 of the sorting networks A, B output solution respectively. In this case, the output of the sorting network shown by the output of the output layer 5 of the judging network is the correct solution. Accordingly, the correct solution is outputted from a gate 8 with an output signal from the output layer 5.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a pattern classifier, and more particularly to a pattern classifier using a back propagation type neural network.

[0002]

2. Description of the Related Art Conventionally, research has been conducted to apply a neural network to pattern recognition. In back propagation, which is a neural net learning algorithm announced in 1976, a neural net can be automatically learned by giving an input and a desired output. That is, the input data is the input layer, the intermediate layer,
Each unit having a hierarchical structure of the output layer sequentially converts and transmits to the next unit, and an output is obtained from the output layer.
The output value is compared with the desired output value and the strength of the coupling is changed so as to reduce the difference. The calculation of the bond strength starts from the unit of the output layer and moves to the unit of the middle layer, but the unit of the middle layer cannot be calculated unless the strength of the connection of the preceding stage is determined. Therefore, the calculation can be performed only when the last input layer is traced back. In this way, learning proceeds in the opposite direction to the processing of input data, that is, backward. In learning by back propagation, learning data is first input and the result is output. Varying the bond strength backwards to reduce the resulting error. The learning data is input again and the result is output. This operation is repeated until it converges (for example, "Nikkei Electronics" 8-10, 1987.no.427, pp.115-126, "Use the neural net for pattern recognition, signal processing, knowledge processing". reference).

[0003]

As described above, in the conventional pattern classifying apparatus using the back-propagation type neural network, the output layer is different from the pattern input from the unit of the input layer. The output obtained in step 1 is compared with the output of the correct answer to be obtained, and the error is used to change the weight of the coupling with the layers before and after the unit of the intermediate layer by the error backpropagation learning algorithm. I was getting the desired output pattern. However, in such a pattern classification device, the capacity of the pattern that can be learned and stored is limited by the number of units in the intermediate layer. That is, since the number of units in the middle layer is proportional to the number of patterns that can be learned, if the number of units is doubled, the time for learning one pattern is increased by a factor of 2 to 3. Thus, in order to increase the capacity, when the middle tier with a large number of units is used, the learning time for each pattern increases and the network that has already been learned is newly added. There was a problem that the contents learned previously were destroyed when the unit was added to and the learning was continued. An object of the present invention is to solve such conventional problems and to provide a pattern classifying apparatus capable of increasing the memory capacity of a pattern without increasing the learning time for each pattern. To do.

[0004]

In order to achieve the above object, the pattern classifying apparatus of the present invention comprises a small scale in which a plurality of units are connected in parallel in the order of an input layer, an intermediate layer and an output layer. A back propagation type neural network and a judgment neural network which is connected in parallel to the neural network and determines which pattern is learned by which neural network. It is characterized in that the output of which output layer of the output layers of each neural network connected in parallel by the determination neural network is correct.

[0005]

In the present invention, a small-scale backpropagation
Option type neural network connected in parallel
A neural network for determining which pattern is being learned by which neural network is increased by increasing the capacity of the pattern that can be learned without increasing the learning time for each pattern. Are arranged in parallel with those networks, and it is clarified which output layer of the output layers of each neural network arranged in parallel shows the correct answer. This enables the neural network
Even if the number of patterns that can be learned and discriminated is doubled by increasing the number of intermediate layers of the pattern, the learning time for one pattern can be left as it is. Therefore, by adding the pattern classification network by the storage capacity of the judgment network, the pattern can be learned without increasing the learning time.
The number of units can be increased.

[0006]

Embodiments of the present invention will now be described in detail with reference to the drawings. FIG. 1 is a structural diagram of a pattern classification device showing an embodiment of the present invention. In FIG. 1, 1 is an input layer,
Reference numerals 2, 3, 4 are intermediate layers, 5, 6, 7 are output layers, and 8 is a gate. As shown in FIG. 1, the pattern classification device of the present invention is divided into upper, middle and lower parts by broken lines. The upper part is the judgment network and the central part is the pattern classification network. -A is a network, and the lower part is a network B for pattern classification. As described above, in the pattern classifying apparatus of the present invention, the input layer is provided with the unit 1 at the central position of any network, but the intermediate layer and the output layer each have three units.
Each unit is arranged in parallel. The units in the input layer 1 are connected in parallel to the units in the intermediate layers 2, 3 and 4, respectively, and the entire system has the input layer 1 → the intermediate layer 2 → the output layer 5 and the input layer 1 → the intermediate layer 3 -> Output layer 6, input layer 1-> middle layer 4-> output layer 7 independent three back propagation type neural networks
It is composed of When the input pattern is learned and stored, the input layer 1 and the intermediate layer 4 of the pattern classification network B are first disconnected, and the judgment network and the pattern classification network A are disconnected. Then, the input pattern is sequentially learned by the error backpropagation learning algorithm. At this time, the teacher signal input from the output layer 5 of the determination network is a correct answer for which the unit A indicating that it is always the input to the network A is desired.

Patterns up to the full capacity of network A
After learning the network, the input layer 1 of the network A and the intermediate layer 3 are disconnected, and the network for judgment and the network B are trained for the input pattern. In this case, the teacher signal input from the output layer 5 of the determination network is the correct answer for which the unit B indicating that it is the input to the network B is to be obtained. In this way, all the connections between the input layer 1 and the intermediate layers 2, 3 and 4 of the system that has been learned are connected and used as a pattern classifier. For the input pattern, the output layers 6 and 7 of the classification networks A and B respectively output solutions. In this case, the output of the classification network indicated by the unit obtained as the solution at the output of the output layer 5 of the determination network is the correct solution. Therefore, by arranging the gate 8 behind the output layers 6 and 7, it is determined which output is correct by the output signal from the output layer 5, and the correct output is obtained.

FIG. 2 is an explanatory diagram showing the operating principle of the present invention. In FIG. 2, in order to facilitate understanding, the operating principle of the present invention is simplified by setting the feature space as two-dimensional. The pattern classification by the back-propagation-type neural network used in the present invention is to divide the mapped feature space of these patterns into spaces to which each pattern belongs. I can say. Therefore, the storage capacity of a network is how many ways the network can divide the feature space. As shown in FIG. 2, in this embodiment, the classification network A and the classification network B are the input patterns learned by themselves.
The feature space is divided so that the input patterns learned by oneself and those learned by the other can be distinguished.
It may not be possible to distinguish That is, the network
The division of the feature space by the line A is divided in the vertical direction as indicated by the alternate long and short dash line. The ∘ mark is the characteristic amount of each pattern learned in the space into which the network A is divided. Further, the division of the feature space by the network B is divided in the horizontal direction as indicated by the broken line. The □ mark is the feature quantity of each pattern learned in the space into which the network B is divided. It becomes extremely difficult to classify both learned pattern feature quantities.

In this case, as shown by the solid line, it is distinguished which of the two classification networks the input network has learned as the input network. That is, it is possible to determine which one of the networks A and B is the input pattern learned by the determination network,
Since the feature space is roughly divided, it is possible to determine which of the classification networks outputs the correct answer by referring to the output. With the conventional method, the number of units in the middle layer and the pattern that can be learned and discriminated
The number of units is proportional, but if you double the number of units, the time required to learn one pattern is
It was generally said to take 2 to 3 times. In this embodiment, the number of patterns that can be learned and discriminated can be doubled while keeping the learning time for one pattern unchanged. That is, even when the number of patterns that can be learned and discriminated is increased by doubling the number of units in the intermediate layer, the learning and discrimination time does not change. Further, theoretically, the number of patterns that can be learned can be further increased by adding the pattern classification network by the storage capacity of the determination network.

[0010]

As described above, according to the present invention,
By connecting a plurality of small-scale neural networks in parallel, it is possible to increase the memory capacity of the pattern without increasing the learning time for each pattern.

[0011]

[Brief description of drawings]

FIG. 1 is a block diagram of a pattern classification device showing an embodiment of the present invention.

FIG. 2 is a diagram showing a principle of dividing a feature space according to the present invention.

[Explanation of symbols]

1 Input layer 2,3,4 Middle layer 5,6,7 Output layer 8 Gate 〇 Pattern feature network A's learned pattern features □ Pattern classification network B Features of patterns learned by

Claims (1)

[Claims] 1. A small-scale back-propagation type neural network in which a plurality of units are connected in parallel in the order of an input layer, an intermediate layer and an output layer, and in parallel with the neural network. Each unit of the input layer, the intermediate layer and the output layer is connected, and a judgment neural network for judging which pattern is learned by which neural network is provided. −
A pattern classifying device characterized in that the output of which output layer of the output layers of the respective neural networks connected in parallel by the laural network is shown as the correct answer.