JPH04175986A - Pattern identification processor - Google Patents

Abstract

PURPOSE:To automatically determine the structure of a multi-layered neural circuit by providing a decision means which sends an input pattern signal to a multi-layered neural circuit of a trailing stage and a storage means which adds information on the reason of decision making to a rejected input pattern signal. CONSTITUTION:Multistage structure wherein multi-layered neural circuits 1 and 3 and a decision part 2 are cascaded alternately is employed and when the decision part 2 decides that the maximum value of output layer elements of the multi-layered neural circuits 1 and 3 is larger than a predetermined threshold value and the difference between the maximum value and a following large value is larger than a predetermined threshold value, a category corresponding to the element which outputs the maximum value is regarded as an identification result. When the conditions are not satisfied, the identification result is rejected and the input pattern is sent out to the multi-layered neural circuit of the trailing stage. Consequently, the structure of the multi-layered neural circuits 1 and 3 which identifies the input pattern with a complicate distribution is automatically determined.

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention relates to a technique for identifying patterns using neural circuits, and in particular, relates to a technique for identifying patterns using neural circuits. It is related to circuits.

[Prior art]

As conventional learning methods using multilayer neural circuits, methods using perceptron, backpropagation, etc. are known. ' FIG. 5 is a schematic configuration diagram showing an example of a pattern recognition processing device that performs pack propagation learning using a conventional multilayer neural circuit.

In FIG. 5, 10 is a multilayer neural circuit, 101 is an input pattern, 11 is a load vector of an intermediate layer neural circuit element, 1
2 is the threshold value of the intermediate layer neural circuit element, 13 is the output of the intermediate layer neural circuit element, 14 is the weight vector of the output layer neural circuit element, 15 is the threshold value of the output layer neural circuit element, and 16 is the output layer Output of neural circuit element, 102 is output vector, 1o
3 is a teacher signal vector, and 17 is a neural circuit element.

The multilayer neural circuit 10 is as shown in FIG.

It is a neural circuit with a three-layer structure consisting of an input layer, a middle layer, and an output layer, and connections exist only from the neural circuit elements in the lower layer to the neural circuit elements in the next higher layer, and the neural circuit elements in the layer There are no connections between them.

FIG. 6 is a functional block diagram of one neural circuit element in FIG. 5.

Each neural circuit element 17 in FIG. 5 has the functions shown in FIG. 6.

In FIG. 6, 18 is a membrane potential calculation circuit, and 19 is a saturation type function calculation circuit. The membrane potential calculation circuit 18 multiplies each input signal input to each neural circuit element 17 by the load value of the corresponding load vector 14 (11) and calculates a total value. The saturation type function calculation circuit 19 outputs a value obtained by applying a saturation type function to a value obtained by subtracting the threshold value 15 (12) of the neural circuit element 17 from the previously obtained sum as an output 16 (13).

Backpropagation is a mechanism that adjusts the load of each neural circuit element 17 in order to minimize the squared error between the output vector 102 obtained from the input pattern 1°1 and the teacher signal vector 103, and thereby allows each input pattern to be identified. is naturally constructed inside the multilayer neural circuit 10.

However, in the conventional technique, when an input pattern has a complicated distribution, it is difficult to perform pattern identification using one multilayer neural circuit.

For example, the 71st! As shown in (diagram of principle for identifying category c2 and patterns belonging to category c2).

If the degree of separation between the distributions of patterns belonging to category C□ (represented by 0 marks) and patterns belonging to category C2 (represented by X marks) is good, they can be separated by a simple separation hyperplane Ω□. However, as shown in Figure 8, categories C,,C
, when the patterns belonging to , have a complicated distribution, it is required to discover a complicated separating hyperplane Ω2, but there is a method to find the number of ephemerides of the multilayer neural circuit and the number of elements in each layer necessary for this purpose. Furthermore, it is rare to deal with objects whose distributions are known, and in many cases the distributions are unknown.

As a method to improve the classification rate, it has been known to conduct a classification experiment by changing the number of neural circuits, the number of intermediate layer neural circuit elements, and the learning parameters, and then adopt the neural circuit that showed the highest classification rate. (For example, see IEICE technical research report PRU88-58.pp79-86).

However, the number of neural circuits to be generated by changing the number of neural circuits, the number of intermediate layer neural circuit elements, and learning parameters is enormous, and experiments require a large amount of time. Moreover,
There is no guarantee that an optimal solution will exist among them.

In addition, there is a method in which at the initial stage of learning, the pattern corresponding to the center of the distribution is mainly learned, and at the end of the learning stage, the pattern existing mainly in the vicinity of the separating hyperplane is learned (for example, the IEICE Technical research report PRU88-
94. (See pp. 23-30), as long as the structure of the multilayer neural circuit has been determined, this is merely an attempt to minimize the error rate8, and cannot be a method for actively relieving errors.

[Problem to be solved by the invention]

The present invention has been made to solve the above problems.

It is an object of the present invention to provide a technique that allows a pattern recognition processing device to automatically determine the structure of a multilayer neural circuit that allows input patterns with a complex distribution to be identified. can.

The above and other objects and novel features of the present invention will become apparent from the description of this specification and the accompanying drawings.

[Means to solve the problem]

In order to achieve the above object, the pattern recognition processing device of the present invention takes in input pattern signals from a plurality of neural circuit elements, multiplies each input pattern signal by a weight value corresponding to the total value, and saturates the sum of the signals. Apply type function 1
It consists of neural circuit elements that obtain individual outputs, and the layers constructed by arranging each neural circuit element are arranged from the lowest input layer to one or more ranked intermediate layers, and then to the highest output layer. A pattern identification processing device that identifies an input pattern using a multilayer neural circuit configured and connected from a lowest layer neural circuit element to an upper layer neural circuit element,
The maximum value of the output of the neural circuit of the output layer is a predetermined first value.
is above the threshold.

and the difference between the maximum value and the next maximum value is a predetermined second value.
When the two conditions that are equal to or greater than the threshold value are satisfied, the category corresponding to the neural circuit element that outputs the maximum value is set as the identification result, and when the two conditions are not satisfied, the identification result is rejected.

The method further comprises a determining means for sending the input pattern signal to a next-stage multilayer neural circuit, and a storing means for adding information about the reason for determination to input pattern signals rejected in the determining means and storing the resultant information. The most important feature.

Further, the first threshold value and the second threshold value in the determination means are determined using the maximum value and the histogram of the difference of the multilayer neural circuit with respect to the pattern signal used for learning, and are presented. The maximum value and the difference of the multilayer neural circuit with respect to the input pattern signal are superimposed on the histogram, and the method includes means for making it possible to change the settings of the first threshold value and the second threshold value using the histogram. It is characterized by:

[Effect]

According to the above-mentioned means, by adopting a multi-stage configuration in which the multilayer neural circuit and the determination unit are alternately connected in cascade, the maximum value of the output layer element of the multilayer neural circuit in the determination unit is set to the predetermined threshold value 01. If the above is the case, and the difference between the maximum value and the next maximum value satisfies the predetermined threshold value 02 or more, the category corresponding to the element that outputs the maximum value is set as the identification result, If the conditions are not satisfied, the identification result is rejected and the input pattern is sent to the next stage multilayer neural circuit, thereby determining the optimal number of calendars or the number of intermediate layer neural circuit elements for the input pattern with a complex distribution. Instead of dealing with this problem by structuring, relatively small-scale multilayer neural circuits are connected in cascade, and each multilayer neural circuit is responsible for only the distribution that can be identified, and other distributions are handled by other multilayer neural circuits. Since the system adopts the structure of allocating tasks, it is possible to automatically determine (automatically create) the structure of a multilayer neural circuit that can identify input patterns that have a complex distribution.

[Embodiments of the invention]

Hereinafter, one embodiment of the present invention will be specifically described using the drawings.

In addition, in all the times for explaining the example.

Components having the same function are given the same reference numerals, and repeated explanations thereof will be omitted.

FIG. 1 is a diagram for explaining a first embodiment of the present invention, which uses two stages of multilayer neural circuits.

In FIG. 1, 101 is an input pattern, 1 is a first multilayer neural circuit, 102 is an output vector, 103 is a teacher signal vector, 104 is a histogram memory section, 2 is a judgment section,
3 is the second multilayer neural circuit, 302 is the output vector, 304
4 is a histogram memory section, 4 is a determination section, 5 is an identification result memory section, 6 is an input pattern memory section, and 7 is a control section.

First, the operation during learning will be explained.

The first multilayer neural circuit 1 performs learning to recognize the input pattern 101 by changing the load of the first multilayer neural circuit 1 using the output vector 102 and the teacher signal vector 1o3 for the input pattern 101. conduct.

The determining unit 2 constitutes a main element of the present invention, and determines whether or not the input pattern 101 is appropriate for identification using the first multilayer neural circuit l at the point when learning has progressed considerably, If not, the input pattern 101 is stored in the input pattern memory section 6.

The details of the operation of the determining section 2 will be explained using FIG. 2.

In FIG. 2, 21 is a maximum value/next maximum value detection circuit, 22
23 is a first judgment circuit, 23 is a second judgment circuit, and 24 is a verification circuit.

The maximum value/next maximum value detection circuit 21 detects the maximum value m1 . and detect the next largest value m2.

The first judgment circuit 22 compares the predetermined threshold value θ and the maximum value m1, and if m<>01, the process proceeds to the second judgment circuit 23. If 0m1<θ1, the identification result is ejected, and the control unit 7 sends a signal to the input pattern 101 to be stored in the input pattern memory section 6 with information rejected by the first determination circuit.

The second determination circuit 23 compares a predetermined threshold value θ with the difference m□−m2 between the maximum value and the next largest value, and determines that ml−m2≧θ2
Then, it is determined whether the category corresponding to the neural circuit element that outputs m to the matching circuit 24 (first candidate category) and the category indicating the teacher signal vector 103 (correct category) match/mismatch.

If m<82, the identification result is determined to be rejected, and the control unit 7 adds information indicating that the input pattern 101 was rejected by the second determination circuit 23 and stores it in the input pattern memory unit 6.
Sends a signal to be stored in.

Furthermore, the contents of the histogram memory unit 104 corresponding to m□2m2 are updated depending on whether the first candidate category is the correct category and whether the conditions of m-k≧θ19mknee m2≧θ2 are satisfied.

The histogram memory unit 104 stores information equivalent to the contents shown in FIG.

In FIG. 3, the horizontal axis is the value of ml, the vertical axis is the frequency of the learned pattern, and the distribution of ml where the first candidate category is the correct category and satisfies m□≧θ□ is indicated by a. Also, if the first candidate category is the correct category and m, <
The distribution of m8 that satisfies θ is denoted by b, and the distribution that simply satisfies m, (θ) is denoted by C.

In FIG. 3, it is assumed that θ is set to satisfy the learning pattern α% (ratio of the hatched portion to the distribution of a).

Furthermore, when the number of learning patterns increases, the histogram corresponding to m is added by the process described above, and as shown in Figure 4, 1 distribution a changes to a', b to b', and C to C'. It shall be assumed that At this time, by changing θ1 to θ□', it becomes possible to keep α constant. θ3 (not shown) can also be changed in the same manner as θ1.

When the learning of the first multilayer neural circuit 1 progresses and a preset performance is achieved, the control section 7 presents the input pattern stored in the input pattern memory section 6 to the second multilayer neural circuit 113.
The second multilayer neural circuit 3 is trained in the same manner as the multilayer neural circuit 1 is trained. In this way, learning of a multilayer neural circuit with a multistage configuration is performed. In this embodiment, an example of a two-stage configuration has been described, but learning can also be performed in the case of three or more stages by repeating the above-described method.

In addition, in this embodiment, the second multilayer neural circuit 3 uses all nine input patterns stored in the two-person cover turn memory section 6.
Although we have explained an example of learning the input pattern memory unit 6
It is also possible to divide the patterns stored in, for example, into a set of patterns 1m□ x θ1 and a set of patterns 1m3 x θ2, and configure a multilayer neural circuit for each of these sets.

Furthermore, although back propagation is assumed as the learning method in this embodiment, any method may be used as long as the output vector 102 can be obtained.

Next, the operation at the time of II will be explained using FIG. 1.

The input pattern 101 is processed by the first multilayer neural circuit 1 and an output vector 102 is obtained.The zero determination unit 2 detects the maximum value m times the largest value m2 from the output vector 102, and sets the value to a predetermined threshold value θ1. The following determination is made using θ3.

(i) When m1≧θ and m2≧θ2, the identification result is
The result for each li is stored in the identification result memory section 5 as the first candidate category. ms + m z is m engineering ≧019m2
The histogram memory unit 104 is given the information of ≧02.
is superimposed on the distribution of

(...) When m<θ, or m2<θ2, the identification result is determined to be ejected, and the input pattern 101 is sent to the multilayer neural circuit 3 @ mz Hrn2 is m<θ□ or m2<
Information about θ2 is given and superimposed on the distribution in the histogram memory unit 104.

In the case of (ii) above, the output vector 302 obtained in the multilayer neural circuit 3 is processed by the determination unit 4.

The result is stored in the identification result memory section 5, and the identification process ends. In this process, m-process 2m2 extracted from the output vector 302 is superimposed on the distribution in the histogram memory unit 304 with information from the determination unit 4 added thereto.

The present invention has been specifically described above based on examples.
It goes without saying that the present invention is not limited to the embodiments described above, and can be modified in various ways without departing from the spirit thereof.

〔Effect of the invention〕

As described above, according to the second aspect of the present invention, by adopting a multi-stage configuration in which multi-layer neural circuits and determination units are alternately connected in cascade, sequential processing is performed while determining whether discrimination is possible in the multi-layer neural circuit at each stage. As the process progresses, it is possible to automatically create a multilayer neural circuit that can identify input patterns with complex distributions.

In addition, the determination unit superimposes the maximum value and the largest value of the elements of the output vector obtained each time an input pattern is processed on a histogram, and uses the histogram to change the threshold value used for 1 determination. Performance can be maintained constant.

[Brief explanation of the drawing]

FIG. 1 is a functional block diagram of a multilayer neural circuit for explaining one embodiment of the present invention. FIG. 2 is a functional block diagram showing one embodiment of the determination section of this embodiment. FIG. 3 is a diagram showing an example of the histogram of the first embodiment, and FIG. 4 is a diagram showing an example of the histogram when the number of learning patterns is increased in the first embodiment. FIG. 5 is a block diagram of one ordinary multilayer neural circuit, and FIG. 6 is a block diagram of one neural circuit element in FIG. FIG. 7 is a diagram showing an example of a distribution with a good one-separation degree, and FIG. 8 is a diagram showing an example of a distribution with a difficult one-separation degree. In the figure, 1... first multilayer neural circuit, 2... determination section, 3
. . . second multilayer neural circuit, 4. determination unit, 5. identification result memory unit, 6. input pattern memory unit, 7.
...control unit, 11...load vector of the middle layer neural circuit element, 12...threshold value of the middle layer neural circuit element, 13
... Output of the intermediate layer neural circuit element, 14... Load vector of the output layer neural circuit element, 15... Threshold value of the output layer neural circuit element, 16... Output of the output layer neural circuit element, 17... Neural circuit element, 18... Membrane potential calculation circuit, 19... Saturation type function calculation circuit, 21... Maximum value
Next largest value detection circuit, 22...first judgment circuit, 23...
Second judgment circuit, 24... Verification circuit, 101... Input pattern. 102... Output vector, 103... Teacher signal vector, 104... Histogram memory section, 302...
- Output vector, 304... histogram memory section.

Claims (2)

[Claims] (1) A neural circuit element that receives input pattern signals from multiple neural circuit elements, multiplies each input pattern signal by a corresponding weight value, and applies a saturation function to the summed value to obtain one output. It consists of a layer in which each neural circuit element is lined up, starting from the lowest input layer, continuing to one or more ranked intermediate layers, and continuing to the highest output layer. A pattern identification processing device that identifies an input pattern using a multilayer neural circuit connected from a neural circuit element to an upper layer neural circuit element, wherein the maximum value of the output of the output layer neural circuit is predetermined. output the maximum value when two conditions are met: the maximum value is equal to or greater than a first threshold value, and the difference between the maximum value and the next maximum value is equal to or greater than a predetermined second threshold value. determining means for determining a category corresponding to the neural circuit element that has been selected as an identification result, and rejecting the identification result if the two conditions are not satisfied, and sending the input pattern signal to a next-stage multilayer neural circuit;
A pattern identification processing apparatus characterized in that the determining means includes a storage means for storing rejected input pattern signals with information on the reason for the determination added thereto. (2) The first threshold value and the second threshold value in the determination means are determined using the maximum value and histogram of the difference of the multilayer neural circuit with respect to the pattern signal used for learning, and are presented. means for superimposing the maximum value and the difference of the multilayer neural circuit with respect to the input pattern signal on the histogram, and using the histogram to change the settings of the first threshold value and the second threshold value; The pattern identification processing device according to claim 1, further comprising: a pattern identification processing device;