JPH064676A - Recognition judgement device - Google Patents

Abstract

PURPOSE:To provide a recognition judgement device where recognition precision is improved and the addition of a recognition category is facilitated. CONSTITUTION:Plural unit recognition units (n) composed of quantizers n2 executing quantization in accordance with an input signal from signal input parts n1 inputting feature data being a recognition object, and path selection parts n3 having single or plural path input terminals n3a and single or plural path output terminals n3b and selecting a path in accordance with the output of the quantizers n2 are combined. Then, a circuit is constituted so that multiple hierarchies are given. The path of a unit recognition unit before an output layer is selected by a learning unit changing the strength of the single or plural path input terminals and a single or plural path output terminals, and the path input terminals and the path output terminals which the output value of the quantizer shows. Then, the number of the unit recognition units of the output layer is made larger than the number of the categories to be recognized.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a recognition judgment device for judging recognition of various objects.

[0002]

2. Description of the Related Art As a conventional learning type recognition / judgment device, for example, "Learning realignment by back propositioning error (Learning replay error"
presentations by back-propagating errors) ", Nature
It is shown in Vol.323 No.9 (1986). FIG. 7 shows a general configuration diagram of a conventional learning type recognition and determination device.
1 and 202 are input terminals, 217 is a learning circuit, 218, 219 and 220 are multi-input / one-output circuits, 221 is an output layer, and 222 is a hidden layer.
As shown in FIG. 7, a general learning type recognition / judgment device has a configuration in which multiple input / one output circuits are connected in a hierarchical manner.
The signal input from the input terminal is processed and output from the output terminal. In this way, among the multi-input / single-output circuits connected in a hierarchical manner, the layer consisting of the multi-input / single-output circuit that outputs the output signal is called an output layer, and the other multi-input / single-output circuits are formed. A layer is called a hidden layer. The hidden layer may be configured by a multi-input / one-output circuit forming one layer, or may be configured by a multi-input / one output circuit forming a plurality of layers.

FIG. 8 shows a concrete example of a conventional learning type recognition and judgment apparatus. In FIG. 8, 201 and 202 are input terminals, and 203 and 20.
4, 205, 206, 207, 208 are variable weight multipliers, 209, 210, 2
11 is an adder having a saturated input / output characteristic, 212 is an output terminal, 213
Is a teacher signal generation unit, 214 is an error calculation unit, 215 is the steepest descent direction determination unit, 216 is a weight change unit, 217 is a learning circuit, 218, 219,
220 is a multi-input / one-output circuit, 221 is an output layer, and 222 is a hidden layer. As shown in FIG. 7, the multi-input / single-output circuit 21
8, 219 and 220 consist of variable weight multipliers and adders with saturated input / output characteristics. That is, the j-th multi-input
The output signal of the one output circuit is

[0004]

[Equation 1]

It is represented by Here, y [i] is the output signal of the i-th multi-input / one-output circuit in the previous layer, and w [i,
j] is a weight to be applied when the output signal of the i-th multi-input / single-output circuit of the preceding layer is input to the j-th multi-input / single-output circuit. fnc () is a sigmoid function with a saturation characteristic

[0006]

[Equation 2]

And the like. FIG. 9 shows adders 209, 210 and 211 having saturated input / output characteristics, which are represented by the above fnc ().
The graph of the characteristic function of is shown.

The learning type recognition / judgment device has a structure in which such multi-input / one-output circuits are connected in a hierarchical manner, and in learning, a desired output signal (hereinafter, referred to as a teacher signal) with respect to an input signal is used. ) Is output, the weights multiplied by the variable weight multipliers 203, 204, 205, 206, 207 and 208 are changed. In order to obtain the weight change amount, first, the error shown in (Equation 3) is obtained from the teacher signal and the output signal of the output layer.

[0009]

[Equation 3]

Where y _p [j] is the output signal of the j-th multi-input one-output circuit of the output layer for the p-th input signal, and t _p [j] is the teacher signal for y _p [j] , ΣΣ are summations for all teacher signals p and all multi-input one-output circuits j in the output layer. The weight w is a vector having the weight w [i, j] as a component (hereinafter referred to as a weight vector). As shown in (Equation 3), the error E is represented by the table as the sum of squares of the difference between the teacher signal and the output signal of the output layer, and is a function of the weight vector w. In learning, the weight is changed to minimize the difference between the teacher signal and the actual output signal, that is, the error. The change amount of the weight is determined by (Equation 4).

[0011]

[Equation 4]

Here, ε is a positive constant called a learning parameter, α is a positive constant called momentum, and
∂E / ∂w is the weight w of the error E represented by (Equation 4).
A vector whose component is the derivative of [i, j] and is called the steepest descent direction. Δw ′ is a vector expression of the weight change amount in the previous learning.

FIG. 10 shows a block diagram of a learning circuit 217 of this conventional learning type recognition and judgment apparatus. In FIG. 10, 223 is an input terminal of output layer output, 224 is an input terminal of hidden layer output, 2
Reference numeral 25 is an input signal input terminal, 226 is an output layer weight output terminal, and 227 is a hidden layer weight output terminal. In the learning circuit 217 of the conventional learning-type recognition / judgment device, the teacher signal generator 213
Is the teacher signal for the input signal (desired output signal) t
Generate _p [j]. The error calculation unit 214 calculates the error E represented by the formula (3) from the teacher signal t _p [j] and the output signal y _p [j] of the output layer. The error calculation unit 214 calculates the difference signal t _p [j]-y _p [j] between the teacher signal and the output signal necessary for changing the weight.
Is output to the steepest descent direction determination unit 215. The steepest descent direction determining unit 215 determines the steepest descent direction of the error E in the weight space expressing the weights as a vector based on the weights of the difference signal, the output layer output signal, the hidden layer output signal, the input signal, and the output layer. Ask. The steepest descent direction is determined by the formula shown in (Equation 5).

[0014]

[Equation 5]

The right side of (Equation 5) is a vector expression of the differentiation by the weight of the error E. The steepest descent direction determination unit 215,
The steepest descent direction is multiplied by the learning parameter, and the weight changing unit 21
Output to 6. The weight changing unit 216 obtains the amount of weight change by (Equation 4), and each variable weight multiplier 203, 204, 205, 206,
Change the weight to be multiplied by 207 and 208.

As described above, the error is reduced by repeating the calculation of the weight change amount by the steepest descent method. When the error is sufficiently small, it is assumed that the output signal is sufficiently close to the desired value. To finish.

[0017]

In the conventional learning type recognition / judgment apparatus as described above, the number of neurons in the output layer of the circuit is the same as the number of categories to be recognized, and it is desired to actually recognize it. Even when an input is made that belongs to a category other than the category, any one of the categories to be recognized is output as the recognition result, so that erroneous recognition is often performed and the recognition accuracy is deteriorated. Further, it is impossible to add a new recognition category after the learning is completed, and in such a case, it is necessary to redesign a circuit and restart learning from the beginning, which lacks flexibility.

The present invention takes into consideration the problems of the conventional learning type recognition and judgment apparatus as described above, greatly improves the recognition accuracy, and can easily increase the number of recognition categories depending on the situation.

[0019]

According to the present invention, there is provided a signal input section, a quantizer for quantizing in accordance with an input signal from the signal input section, a single or a plurality of path input terminals, and a single path input terminal. Alternatively, the circuit is configured to have a multi-layered hierarchy by combining a plurality of unit recognition units each having a plurality of path output terminals and configured with a path selection unit that selects a path according to the output of the quantizer. A path selector of the unit recognition unit in the front layer of the output layer, a path input section having a single or a plurality of path input terminals, a path output section having a single or a plurality of path output terminals, and the path input. Category to be recognized by the learning device that changes the strength of the connection between the path input terminal of the section and the path output terminal of the path output section indicated by the output value of the quantizer, and the number of unit recognition units in the output layer More than number It is a recognition judgment apparatus.

[0020]

In the recognition / judgment device configured as described above, m unit recognition units are provided in the output layer, and learning is performed so as to recognize n categories. Here, the number m of unit recognition units in the output layer is larger than the number n of recognition categories, and the relationship of m> n is satisfied. When learning is performed by using a plurality of input patterns belonging to n categories and an unknown pattern is input after learning, if the pattern does not belong to any of the n recognition categories. The output of the unit recognition unit corresponding to other than the n recognition categories becomes large, the recognition result that the input pattern does not belong to the recognition category is output, and the recognition accuracy can be greatly improved.

Also, when it is desired to add a new recognition category depending on the situation, it is possible to easily increase the number of recognition categories by assigning a unit recognition unit corresponding to a category other than the recognition category to the newly added category. I can.

[0022]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 shows an embodiment of the recognition judging device according to the present invention. It recognizes and judges the five-dimensional signal input from the bottom layer, and a plurality of unit recognition units n are combined to form a circuit having a four-layer hierarchy. The fourth layer has 16 unit recognition units. Units p1 to p16 are provided. Also, the unit recognition units p1 to p16 of the fourth layer
And the output terminals of the third layer are mutually coupled. p1 ~
p14 is a category to be recognized, that is, recognition categories 1 to 14
, And p15 and p16 are labeled as something other than the recognition category, that is, outside the recognition category. 2
Is a teacher signal generator for giving a teacher signal to the unit recognition unit of the third layer. The first layer is an input layer composed of five terminals and is for inputting a five-dimensional signal. Figure 2
FIG. 4 shows an embodiment of a unit recognition unit used in the second layer of the recognition and judgment apparatus according to the present invention. Reference numeral n1 denotes a signal input unit, which inputs the series signal input via the signal input terminal n1a to the quantizer n2. The quantizer n2 quantizes the input sequence signal and inputs the quantized value to the route selection unit n3. Reference numeral n3a is a path input terminal, and n3b1, n3b2, and n3b3 are path output terminals, which connect these terminals to each other when a unit recognition unit is combined to form a circuit.
The route selection unit n3, based on the value input from the quantizer n2, the route input terminal n3a and the route output terminal n3b1, n3b2, n3b
It is configured to change the way of connection with 3. Figure 3
Shows an example of the route selection unit n3. The route selecting unit n3 includes one route input terminal n3a, three route output terminals n3b1, n3b2, n3b3, a load setter n3s and a load n3w.
1, n3w2 and n3w3. Load n3w1, n
3w2 and n3w3 are weights by which the signal input from the route input terminal n3a is multiplied, and the multiplied result is the route output terminal n3
Output from b1, n3b2 and n3b3. The load setter n3s loads the loads n3w1 to n3w3 so that the strength of the connection between the route output terminal and the route input terminal indicated by the output value of the quantizer n2 is maximized.
To set.

FIG. 4 shows a third embodiment of the recognition judgment device according to the present invention.
It is an example of a unit recognition unit used for a layer. n1t is a teacher signal input terminal to which the teacher signal generated by the teacher signal generator 2 is input.

In the learning process, the teacher signal indicates the number of the category to be learned, that is, the unit recognition unit number of the fourth layer. n3a is the route input terminal, n3b1
To n3b16 are the route output terminals, the loads n3w1 to n3w16 are the weights to be multiplied by the signals input from the route input terminal n3a,
The result of multiplication is route output terminal n3b1 of route output unit 3b.
To n3b16. Reference numeral n3c is a route learning device, which changes the strength of the connection between the route output terminal and the route input terminal indicated by the teacher input value in the learning process. In the recognition process, no signal is input to the teacher input terminal nt1 and the weight n
3w1 to n3w16 hold the weight state after learning, weight the route signal input from the route input terminal, and output the weighted route signals to the route output terminals n3b1 to n3b16. The route learner n3c is
Comparison judgment comparing the maximum output terminal detector n3c1 that detects the path output terminal with the maximum output, the number of the path output terminal detected by the maximum output terminal detector, and the number of the path output terminal pointed to by the teacher input N3c2, based on the comparison result of the comparison determiner, the strength of the connection between the path input terminal of the path input section and the path output terminal of the path output section indicated by the output value of the quantizer, that is, the load is increased. It is composed of a load increasing device n3c3. Comparison judge n3c2
Compares the number of the route output terminal detected by the maximum output terminal detector n3c1 with the number of the route output terminal indicated by the teacher input value. If both terminal numbers do not match, 0 is determined. When the output of the comparator n3c2 is 0, the connection between the path input terminal of the path input unit and the path output terminal indicated by the output value of the quantizer is The load increaser is configured to increase strength.

The unit recognition unit p1 forming the fourth layer
~ P16, for example, using the unit recognition unit shown in FIG. 5, adder n3as for adding the input signals from the plurality of path input terminals n3a, and thresholding the output signal of the adder n3as. , Threshold terminal n3b, which outputs to output terminal n3b
It is composed by s. A sigmoid function, a step function, or the like can be used as the function for performing the threshold processing.

FIG. 6 shows a detailed embodiment of the teacher signal generator 2. Reference numeral 11 is a load initializer, which generates a unit recognition unit number labeled outside the recognition category on the fourth layer of the circuit. A switch 12 switches whether to output a teacher number input from the outside or to output a unit recognition unit number which is generated by the load initializer 11 and is labeled outside the recognition category.

Next, the learning operation of the recognition / judgment device shown in FIG. 1 will be described. The learning operation is the third by the weight initializer 11.
It consists of a weight initialization process of the unit recognition unit of the layer and a supervised learning process by signal input.

In this load initialization process, the teacher signal generator
Switch the switch 12 of 2 to the side of the load initializer 11 so that the teacher signal generator 2 outputs the unit recognition unit number of the fourth layer labeled outside the recognition category, which is generated by the load initializer 11. And input to the teacher signal input terminal of the unit recognition unit of the third layer. By this operation, the load between the route input terminal and the route output terminal connected to the unit recognition unit labeled in the recognition category of the fourth layer is set to a predetermined value, for example, "0", and the load of the fourth layer is set. Set the load to the maximum value with the path output terminal connected to the unit recognition unit labeled outside the recognition category. By this operation, in the initial stage of the supervised learning process described below, the circuit outputs a recognition result indicating that it is out of the recognition category regardless of the input signal. After this load initialization process, the supervised learning process is executed.

Therefore, in the supervised learning process, the switch 12 of the teacher signal generator 2 is switched to the side of the teacher signal input from the outside, and the teacher signal from the outside is directly input to the teacher signal generator.
12 and outputs to the teacher signal input terminal of the unit recognition unit of the third layer. Next, 1 is first given as a route signal to the route input terminal n3a of the unit recognition unit n of the second layer. When a five-dimensional input signal is input to the five terminals of the first layer, these input signals are input to the signal input terminal n1a to the quantizer n2 of the unit recognition unit n of the second layer. The quantizer n2 of the unit recognition unit n of the second layer quantizes the input signal, and based on the quantized value,
The loads n3w1 to n3w3 for coupling with the unit recognition unit of the upper layer are set, and the route output signal is transmitted to the unit recognition unit of the third layer. At this time, the load n3 of the unit recognition unit of the third layer
The w1 to n3w16 weight the route signals input from the route input terminal n3a, and output the weighted route signals to the route output terminals n3b1 to n3b16. Maximum output terminal detector n3c
1 detects the route output terminal having the maximum output, and inputs the number of the detected route output terminal to the comparison / determination unit n3c2.
In addition, the teacher input terminal n1t, the number of the category to be learned, that is, a signal indicating which one of p1 to p16,
It is input via the teacher signal generator 2. Comparative judgment device n
3c2 compares the number of the route output terminal detected by the maximum output terminal detector n3c1 with the number of the route output terminal pointed to by the teacher input value, and if both terminal numbers do not match, 0 is matched. In this case, input 1 to the load increaser n3c3. When the output of the comparator n3c2 is 0, that is,
If the path output terminal that outputs the maximum output and the path output terminal indicated by the teacher input are different, the load increaser
n3c3 increases the strength of the connection between the route input terminal of the route input unit and the route output terminal of the route output unit indicated by the teacher input. Therefore, in the initial stage of the supervised learning process, the recognition result that any input signal is given is out of the recognition category is output by the learning by the input signal and the teacher signal. The load on the terminal and the path output terminal of the path output unit indicated by the teacher input increases, and the circuit finally outputs the recognition result that matches the teacher signal input.

Next, the recognition operation of the recognition determination device shown in FIG. 1 will be described.

Just as in the learning operation, as a route signal to the route input terminal n3a of the unit recognition unit of the second layer,
First give one. In addition, a five-dimensional input signal is added to the first layer of 5
The signal is input to the signal input terminal n1a to the quantizer n2 of the unit recognition unit in the second layer via these terminals. The quantizer n2 of the unit recognition unit of the second layer quantizes each input signal, and based on the quantized value, sets the weights n3w1 to n3w3 of the coupling with the unit recognition unit of the upper layer, The route signal is input to the route signal input terminal n3a of the unit recognition unit of the third layer. In the recognition operation, no teacher signal is input to the teacher signal input terminal n1t of the unit recognition unit of the third layer. Therefore, the load n3 of the unit recognition unit of the third layer
w1 to n3 w16 holds the weight state at the time of learning, weights the route signal input from the route input terminal n3a,
This weighted route signal is output from the route output terminals n3b1 to n3b1.
6, and the route signal is sent to the route input terminal n3a of the unit recognition unit of the fourth layer. The adder n3as in the path input unit of the unit of the fourth layer adds all the input path signals and sends them to the threshold value processor n3bs. Threshold processor n3
bs thresholds this signal and outputs it to the output terminal n3b. Therefore, if the value of the added signal is larger than a certain threshold value, output is performed, and each unit recognition processing unit p1 to p14 determines how much the input signal is in the recognition category corresponding to each unit. The degree of matching will be output.

Here, a case where an unknown signal that is very different from the signal presented by learning is input will be described. When such a signal is input, the route input terminal and the route output terminal of the unit recognition unit of the third layer connected to the route output terminal indicated by the output of the quantizer n2 of the unit recognition unit of the second layer The weight between and is less changed during the supervised learning process, and the
The maximum is with the path output terminal connected to the unit recognition unit of the layer, and as a result, the output of the unit recognition unit labeled outside the recognition category of the fourth layer is the maximum.
Therefore, the circuit can output the recognition result that the input signal is out of the recognition category.

Next, a case where a new recognition category is added after the learning of the recognition judgment apparatus shown in FIG. 1 is completed will be described. For example, when a recognition category N15 is added so that 15 categories can be recognized, the label adder 1 is instructed to recognize the fourth category unit recognition unit P15 which has been labeled outside the recognition category. A new category N
Label 15. And, for example, the added category
A recognition category can be easily added only by performing learning in the same manner as the supervised learning process described above using the learning data corresponding to N15.

Each means of the present invention may be realized by software using a computer, or may be realized by using a dedicated hardware circuit having each of these functions.

[0036]

According to the present invention, since the number of unit recognition units in the output layer is larger than the number of categories to be recognized for learning, the recognition accuracy can be greatly improved.

Also, when it is desired to add a new recognition category depending on the situation, it is possible to easily increase the number of recognition categories by assigning a unit recognition unit corresponding to a category other than the recognition category to the newly added category. I can.

[Brief description of drawings]

FIG. 1 is an embodiment of a recognition determination device according to the present invention.

FIG. 2 is an example of a unit recognition unit used in the second layer of the recognition determination device according to the present invention.

FIG. 3 is an example of a path selection unit used in the unit recognition unit according to the present invention.

FIG. 4 is an example of a unit recognition unit used in the third layer of the recognition determination device according to the present invention.

FIG. 5 is an example of a unit recognition unit used in the fourth layer of the recognition determination device according to the present invention.

FIG. 6 is an embodiment of a teacher signal generator of the recognition and judgment apparatus according to the present invention.

FIG. 7 is a configuration diagram of a conventional recognition determination device.

FIG. 8 is a configuration diagram of a conventional specific recognition determination device.

FIG. 9 is a graph of a function showing saturated input / output characteristics.

FIG. 10 is a diagram showing a configuration diagram of a learning circuit of a conventional learning type recognition and determination device.

[Explanation of symbols]

 1 Label adder (means) 2 Teacher signal generator (means) 11 Weight initializer (means) 12 Switch n, p Unit recognition unit n1 Signal input part n1a Signal input terminal n1t Teacher signal input terminal n2 Quantizer (means) ) N3 route selector n3a route input terminal n3b route output terminal n3s load setter (means) n3w load n3c route learner (means) n3c1 maximum output terminal detector (means) n3c2 comparison judge (means) n3c3 load increaser (means) means)

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Taiji Yoki, 1006 Kadoma, Kadoma City, Osaka Prefecture Matsushita Electric Industrial Co., Ltd. (72) Yoshihiro Kojima, 1006 Kadoma, Kadoma City, Osaka Matsushita Electric Industrial Co., Ltd. (72) Inventor Koji Yamamoto 1006 Kadoma, Kadoma City, Osaka Prefecture Matsushita Electric Industrial Co., Ltd.

Claims (4)

[Claims] 1. A signal input means for inputting feature data to be recognized, a quantizing means for quantizing in accordance with an input signal from the signal input means, a single or a plurality of path input terminals, and a single Alternatively, a plurality of unit recognition units having a plurality of route output terminals and configured by route selecting means for selecting a route according to the output of the quantizing means are combined to form a multi-layered layer, and an output layer. The route selection means of the unit recognition unit of the preceding layer has learning means for changing the strength of the connection between the route input terminal and the route output terminal indicated by the quantization means, and the unit recognition of the output layer A recognition and determination device characterized in that the number of units is larger than the number of categories to be recognized. 2. When the number of categories to be recognized is n, m (n <m) unit recognition units are provided in the output layer, and n of the unit recognition units in the output layer are desired to be recognized. The recognition determination apparatus according to claim 1, wherein the category is labeled, and mn of the unit recognition units of the output layer are labeled outside the recognition category. 3. The path output connected to the path input terminal of each unit recognition unit in the previous layer of the output layer and the n unit recognition units of the output layer labeled with the category to be recognized at the initial stage of learning. Set the strength of the connection with the terminal to a predetermined value,
The path input terminal of each unit recognition unit in the previous layer of the output layer, and m− of the output layer labeled outside the recognition category.
2. The recognition / judgment device according to claim 1, further comprising load initialization means for initializing the strength of the connection with the route output terminals connected to the n unit recognition units to the maximum value. 4. The recognition judgment apparatus according to claim 2, further comprising a label adding unit for relabeling the unit recognition unit of the output layer labeled outside the recognition category to a new recognition category. .