JPH04142658A - Unitary recognition unit and learning type recognition deciding apparatus - Google Patents

Abstract

PURPOSE:To automate the change of the configuration of an adaptive network, and construct and self-organize the adaptive network in conformity with input signal by providing a configuration storage means for storing a unitary unit, an internal condition storage means, for storing an internal condition of the unitary unit, and a copy means for preparing a copy of the unitary unit based on the stored internal condition. CONSTITUTION:The present unitary unit consists of a configuration storage means 4 for storing a configuration of the unitary unit, an internal condition storage means 5 for storing the configuration of the unitary unit, a copy means 6 for preparing a copy of the unitary recognition unit (n) based on the stored internal condition. Accordingly, the internal condition stored in the internal condition storage means 5 varies with input data. Further, when this internal condition reaches a certain condition, the copy means 6 prepares a copy of the unitary recognition unit n based on the configuration of a network stored in the internal condition storage means 5. With this, the configuration of an adaptive network can be automatically changed, constructed, and self-organized in conformity with input signal.

Description

DETAILED DESCRIPTION OF THE INVENTION Field of Industrial Application The present invention relates to a learning type recognition/determination device for recognizing and determining an object.

E. Rummel, Hart et al.
to 9i 7″o to °ke’-deink’ error’
ations by back-propagate
ing err.

rs)', Naichi+-(Nature) Vol.
, 323 No. 9 (1986). FIG. 9 shows a general configuration diagram of a conventional learning type recognition judgment device, in which 201 and 202 are input terminals, and 217 is a learning time! 218. 219, 220 is a large number of Kerr output circuits, and 221 is an output #222 is a hidden layer.As shown in FIG. The layer consisting of the multi-car output circuits connected in a hierarchical manner as shown above is called the output layer. A layer consisting of other large-scale Kerr output circuits is called a hidden layer. Good~ Figure 10 shows a configuration diagram of a conventional learning type recognition/judgment device.

In Fig. 10, 201.202 are input terminals, 203
．． 204°20mi 206. 207, 208 is a variable weight multiplier 20Q, 2IC% 211 is an adder 212 with saturation input/output characteristics, is an output terminal, 213 is a teacher signal generator 214 is an error calculation number 215 is a steepest descent direction determination a21
6 is the weight change eye 217 is the learning time! 218.219.
220 is a multi-car output circuit, 221 is an output layer, and 222 is a hidden layer.As shown in FIG. The output signal of the i-th multi-car output circuit is y[j] −fnc(Σ(w[i, j
y[i]))=(1) where &; y[i
] is the output signal of the i-th large-scale Kerr output circuit in the previous layer, and w[i, j] is the output signal of the i-th large-scale Kerr output circuit in the previous layer. It is a weight that is applied when input to the circuit. fncO is a function with saturation characteristics and can be expressed as a sigmoid function, etc. Figure 11.
Adder 209 with saturation input/output characteristics. 210 and 21
1 shows a graph of the characteristic function of No. 1. The learning recognition/judgment device has a structure in which a large number of Kerr output circuits like this are connected in a layered manner. Weight multiplier 203.204.205°206,2
In order to find the amount of change in the weights to be multiplied by 07 and 208, first, the teacher signal, the output signal of the output layer, and the convergence error -E (w) (2) are found. Here +: yp[jl is the output signal te[jl of the j-th multi-car output circuit of the output layer for the 9th input signal, te[jl is the teacher signal for y-[j:I, and Σ is the total private signal for all teacher signals. Σ is the total I for all the large output circuits in the output layer, w is the weight w[i.

jl is a vector (hereinafter W is called a weight vector), and as shown in formula a(2), the error E is expressed as the sum of squares of the difference between the teacher signal and the output signal of the output layer.Weight vector In learning, the weights are changed and the error between the teacher signal and the actual output signal is minimized.4 The amount of change in weights is determined by where ε is a positive constant α called the learning parameter. is a positive constant called momentum, and is a vector whose component is the differential of the error E expressed by equation (2) with respect to the weight wl jl, and is called the direction of steepest descent.
w'i! FIG. 12 shows a configuration diagram of the learning circuit 217 of this conventional learning type recognition judgment device. In FIG. 12, 223 is an input terminal for output layer output, 224 is an input terminal for hidden layer output,
225 is an input terminal for input signals, 226 is an output terminal for output layer weights, and 227 is an output terminal for hidden layer weights. Error calculation unit 21 generates a teacher signal (desired output signal) tp[jl for the input signal.
4c Teacher signal 1*[jl and output layer output signal y#[
The error E expressed by equation (2) such as jl is calculated.

The error calculation unit 214 calculates the difference signal te[j:l −ye[jl] between the teacher signal and the output signal, which is necessary for changing the weight, as
The steepest descent direction determining section 215 outputs the difference signal to the output layer output signal Ji! P, lI
Error E in the weight space where the weights are expressed as vectors based on the weights of the input signal and output layer
What is the steepest direction of descent? The right side of Equation (4) determined by j W is also a vector expression of the differential by the weight of the error E. Steepest descent direction determining unit 215+i
The direction of steepest descent is multiplied by the learning parameter and output to the weight change unit 216.The weight change unit 216 calculates the amount of weight change using equation (3).
4. 205. 206. Change the weights multiplied in steps 207 and 208. As described above, by repeatedly calculating the amount of weight change using the steepest descent method, the error becomes smaller. When the error becomes small enough, the output signal becomes sufficient to reach the desired value. In the conventional learning type recognition judgment device, which is similar to the problem to be solved by the present invention, the configuration of the network cannot be changed from the initially determined or designed fixed state. Therefore, the cognitive ability is also determined by the initial design, and it is not possible to deal with it adaptively depending on the input data (l
In addition, network design methods have not been established, and the actual design must rely on trial and error based on experience and intuition.The present invention takes this into account by performing learning on input data. The present invention aims to provide a unit recognition unit and a learning type recognition/judgment device that can automatically change the network structure adaptively according to input signals, and can self-organize.

Structure storage means for storing the configuration and function of the unit, and an internal state of the unit and a duplication means for creating a copy of the unit recognition unit, the unit recognition unit having a structure and function stored in the structure storage means of the unit recognition unit according to the stored internal state. A learning recognition judgment device is constructed by temporarily combining a plurality of the unit recognition units described above in a multi-layered hierarchical network.A unit recognition unit in each layer of the device configured as described above works. When you input various characteristics of the object into the signal input section of
The internal state stored in the internal state storage means changes according to the input data. When this internal state reaches a certain state, the replication means stores it in the internal state storage means based on the structure of the network. It is also possible to self-organize a network that automatically adapts to input data by creating copies of the units at appropriate locations. This is an example of Embodiment 1. , 1 is a signal input section, and when various data signals are input through the No. 4Z input terminals 11 to 14, the function processing section 2 (Table 1) processes the input data with a certain function, such as a threshold function such as a sigmoid function. Output via the output terminal 3a of the signal output section 3.Signal input terminals 11 to 14 and signal output terminal: f3a!: (-When combining unit recognition units to configure a network. The structure memory means 4 stores the structure of the signal input section and the number of output terminals of the signal output section.
The internal state storage means 5 stores information such as processing contents of the function processing section.The internal state storage means 5 stores the internal state of the unit recognition unit that changes moment by moment according to signal input. Copying means 6 (above) When the internal state stored in the internal state storage means reaches a certain value, a copy of the unit recognition unit is created according to the structure of the unit recognition unit stored in the structure storage means. Fig. 2 shows a second embodiment of the unit recognition unit according to the present invention.1 is a signal input section, and the signal input terminal 1
The feature data that is input through a and is the target of recognition is input to the quantizer 2. Z and 3a are route input terminals, 3bl and 3b2 are route output terminals, and these terminals are interconnected when a network is configured by combining unit recognition units. The structure storage means 4 is configured to change the way in which the path input terminal 3a is connected to the path output terminal 3bl or 3b2 based on the value input from the quantizer. Internal state storage means 5 for storing the range, the number of quantizations, the number of route input terminals and the number of route output terminals of the route selection section.
It is also possible to store the total number of inputs as the internal state of the unit recognition unit.The internal state stored in the internal state storage means reaches a certain value. According to the structure of the unit recognition unit stored in the structure storage means, a copy of the unit recognition unit is created.

Figure 3 (Multiple route input terminals (in this case 3a
4 shows a third embodiment of the unit recognition unit according to the present invention, which can perform the same operation as in FIG. 4 In this embodiment (Table 1), the route selection section 3 is composed of a route input section 3a having one route input terminal 3al, a route output section 3b having two route output terminals 3b1 and 3b2, and a switch 3C. l5 tamono de mo switch 3c
lt... Based on the value input from the quantizer 2, the way in which the route input terminal 3a of the route input section 3a is connected to the route output terminal 3bl or 3b2 of the route output section 3b is switched.

FIG. 5 shows a fourth embodiment of the unit recognition unit according to the present invention.
, a path output section 3b having two path output terminals 3bl and 3b2, and a path load section 3C. and 3b2, these weights are changed according to the value output from the loader 3cO1;L quantizer 2. Loads 3 cl and 3 c 2 ft, weight the route signal input from the route input section, route output section 3
bl, send this weighted route signal to the route output terminal 3b.
Figure 6 shows a fifth embodiment of the unit recognition unit according to the present invention.
a is composed of an adder 3aO that adds input signals from a plurality of route input terminals, and the route output section 3b is composed of a threshold processor 3bO that performs threshold processing on the route signal. Vessel 3a01; t,, 3
Route signals input from eight route input terminals from al to 3a8 are added and inputted to the route selector 3c. Route selector 3cLL The added route is added according to the value output from the quantizer 2. Determine how to output the signal to the route output terminal a Route selector 3cζ
As explained in FIG. 5, the fourth plate can be used as the route selector of this embodiment in any of the configurations shown in FIG. There is only one output terminal, so the route selector acl
& When using the configuration shown in Figure 3, you can decide whether to output or not, and when using the configuration shown in Figure 5, you can simply change the load for weighting the route signal, The operation of creating a copy of the unit recognition unit according to the present invention will be explained again using the embodiment shown in FIG. It is necessary to prepare an empty unit recognition unit that does not have a value of 1 to 10. The quantization range of the quantization unit is set to 1 to 10.The internal state storage means 5 calculates and stores the number of distributed inputs of the sequentially input data signals each time the data signal is input. , when the product of the number of inputs and the variance exceeds a certain value, the copy creation means stores the quantization range of the quantizer, the number of quantizations, and the route input of the route selection section. Refer to the number of terminals and the number of route output terminals.
Copy the information of this unit recognition unit to an empty, unused unit recognition unit, and copy the information of this unit recognition unit to another unit Lit [
Create an exact replica of yourself, including combining with units
At that time, divide the necessary quantization range from number Mi 1 to Mi 10, for example, divide the quantization range of the quantizer of the original unit recognition unit into 1 to 5, and divide the quantization range of the quantizer of the duplicate unit recognition unit to 1 to 5. The range is set to 6 to 1o.E copies are created so that the functions are shared between the L base unit and the copy unit.Figure M7 shows the first embodiment of the learning recognition judgment device according to the present invention. The unit recognition units according to the present invention are interconnected in a multi-layered manner to form a network. ”Yama~hi I- and nll~
n3J3 L friend Example cL The unit recognition unit shown in FIG. 3 is used, and as already explained, the route selection section 3 is composed of a route input section 3a having one route input terminal 3al, and two route outputs. It consists of a path output section 3b having terminals 3bl and 3b2, and a switch 3c. Also, the unit recognition unit pressure that constitutes the fourth layer ~
For example, using the unit recognition unit shown in FIG. 5, as already explained, the route input section 3a is
Adder 3 that adds input signals from multiple route input terminals
The learning type recognition judgment device shown in FIG.
Based on the three types of feature data consisting of individual units, the unit is classified into three types. First, 1 is given as the route signal to the route input terminals of the In the case of ζi, two pieces of first feature data are input to two unit recognition units, respectively.) These first feature data are quantized by a quantizer of - and mountains, and this quantized value is Based on the route pH and p12 power level selected by the switches shown in Figure 3, the route signal "1" is sent to the route input terminals of the unit recognition units 匝 and - in the second layer. The second series of features of the recognition object is input to the signal input terminal of the device.
(In the case of this figure, two pieces of second characteristic data are input into two unit recognition units, Hiro and Hiro.) These second characteristic data are input to the output and minus quantizers. Chemical L
Based on this quantized value, paths p21 and p22
(Selected by the switch shown in Figure 3, the path signal °1' is sent to the fourth layer coronation recognition unit mountain and the second path input terminal.The signal input terminal to the quantizer of these units is is a teacher input signal indicating which of the three categories the recognition target belongs to.Immediately h n41 to n4
(In the case of this figure, the teacher input signal is input to two unit recognition units IL.
Input this teacher input signal to aA and white.
The quantizer quantizes and based on this quantized value,
Paths p31 and p32 are selected by the switches shown in FIG.

When the unit recognition units of each layer up to the fourth layer create a copy, it is necessary to include all the unit recognition units connected to the lower layer of each unit.
This figure shows the state of replication including all the unit recognition units connected to the lower layer. Instructs to copy including all unit recognition units connected to the lower layer.For example, as shown in Figure 8, unit 71 (Y) includes unit recognition unit 1.1.712 connected to the lower layer. Because of the duplication, the unit recognition unit 7'2 of duplication and the unit recognition units 721 and 722 below it are created.

Therefore, it is possible not only to train the network based on input data, but also to automatically change the structure of the network adaptively according to the input signal and self-organize. (iii) The process of creating a copy of the learning recognition judgment device according to the present invention is as follows: The internal state of each unit recognition unit changes according to the input data, and the copy creation means operates to create a copy of each unit recognition unit. It is possible to automatically change the structure of the network adaptively in response to signals and to self-organize, and it also has excellent generalizability to new data input that has not been learned.
The learning process of the learning type recognition judgment device according to the present invention (Table 1) Various feature data of the object is input to the signal input section of each unit recognition unit constituting the multilayer hierarchical network, and the unit recognition unit It is only necessary to switch the connection path between the two and determine the selected path to the bottom layer by inputting a teacher signal in the layer before the bottom layer. Therefore, the learning process can be performed at a very high speed. I will explain about this.

In the recognition process, the unit L is stored in the internal state storage means! The internal state of the recognition unit does not change, and therefore the copying means does not operate (- The unit recognition units from the first layer to the second layer operate in exactly the same way as the learning operation. In addition, the feature data is quantized by the quantizer, and the switch is changed based on this to select paths 11111, p12, p21, and p22 sequentially.In the case of recognition operation, the unit recognition unit n3J of the third layer and the signal input of Therefore, the states of the switches at the time of learning are maintained whether the teacher input signal is input to the terminal or not, and paths p31 and p32 are selected according to the states of these switches.
Even if the route signal "1" is sent to the route input terminal of the unit recognition unit nU of the layer, the adder ζL in the route input section of this unit adds the route signals input through the routes p31 and p32. A signal “1” is input to the signal input terminal, and the quantizer quantizes it.
The route selector sets the route output to the possible state L (signal “0”)
If the signal is input (even if the route selector switches to a state where it does not output the route), the added route input signal is sent to the route output device. Output a Therefore, if the value of the added signal is greater than a certain threshold value, an output is made.In this way, the classification recognition judgment of the recognition object is made based on the input recognition object feature data. 4 Also, as a function for threshold processing (
A sigmoid function, a step function, etc. can be used, and as explained above, the recognition process of the learning type recognition judgment device according to the present invention (the signal input section of each unit recognition unit constituting a multi-layered hierarchical network & the object Input various feature data and switch the connection path between unit recognition units according to the output of the quantizer.Also, in the previous layer of the lowest layer, the connection path is set in the learning process to the bottom layer. Recognition results can be obtained simply by determining the selection path of and a unit recognition unit having internal state storage means for storing the internal state of the unit unit, and duplication means for creating a copy of the unit recognition unit according to the state of the stored internal state. The internal state of each unit recognition unit changes according to the input data, and the copy creation means operates to create a copy of each unit recognition unit. The structure of the network is automatically changed adaptively according to the input signal. It is possible to structure and self-organize, and has excellent generalizability to input new data that has not been learned. A configuration that only requires inputting data, switching the connection path between unit recognition units according to the output of the quantizer, and determining the selection path to the lowest layer by inputting a teacher signal in the layer before the lowest layer. Therefore, very high-speed learning processing is possible.In the recognition process, various characteristic data of the object are input to the signal input terminal of each unit recognition unit constituting the multilayer hierarchical network. The connection path between the unit recognition units is switched according to the output of the converter, and the selection path to the bottom layer is simply determined based on the connection path set in the tL learning process in the previous layer of the lowest layer. Therefore, it is possible to configure a system that can obtain recognition results, and therefore, based on the learning results, it is possible to perform very high-speed recognition processing.

[Brief explanation of drawings]

Figures 1 to 6 are blocks showing first to sixth embodiments of the unit recognition unit according to the present invention. Figures 9 and 10 show the duplication operation including the network of the learning type recognition and judgment device according to the present invention (Fig. 11 shows an example of a conventional learning type recognition and judgment device). Figure 12 is a block diagram showing the configuration of the learning circuit of a conventional learning type recognition judgment device.
- Idiomatic input j'J, 1a---1 signal input terminal, 2---
1 quantizer 3-11 route selection @ 3a---
1 signal input terminal, 3bl, 3b2---1 path output terminal, 4-111 structure memory hand & 5---1 internal state memory hand 6------Replication creation hand Dragon's agent's name Patent attorney Akira Okaji and 2 others

Claims (13)

[Claims] (1) Structure storage means that constitutes a unit including a signal input section, a function processing section, and a signal output section, and stores the configuration of the unit, and an internal state that stores the internal state of the unit. A unit recognition unit comprising a storage means and a duplication means for creating a copy of the unit based on the stored internal state. (2) a signal input section, a quantizer that performs quantization according to the input signal from the signal input section, a single or multiple path input terminal, at least one path output terminal, and the quantizer that performs quantization according to the input signal from the signal input section; a route selection unit that selects a route in accordance with the output of the converter, a structure storage unit that stores the structure of the unit, and a history of signals input from the signal input unit that stores the history of the signal as an internal state. 2. The unit recognition unit according to claim 1, comprising internal state storage means and duplication means for duplicating the entire structure of said unit. (3) A claim in which at least one of the quantization constant of the quantizer, the number of route input terminals of the route selection section, or the number of route output terminals of the route selection section is stored in the structure storage means as a unit structure. The unit recognition unit according to item 2. (4) The unit recognition unit according to claim 3, wherein the quantization range and the number of quantizations are stored in the structure storage means as quantization constants. (5) The unit recognition unit according to claim 2, wherein the average, variance, and number of input signals of the input signal are stored as internal states in the internal state storage means. (6) The quantization range stored in the structure storage means of the unit recognition unit that is the basis of duplication was divided into two, and one of the divided quantization ranges was set as the quantization range of the unit recognition unit. 5. The unit recognition unit according to claim 4, wherein the duplication means is configured to allocate the quantization range of the other unit recognition unit to the quantization range of the unit recognition unit to be duplicated. (7) A route input section having a single or multiple route input terminals, a route output section having a single or multiple route output terminals, and a route between the route input terminal of the route input section and the route output section. Claim 2, wherein the path selector is constituted by a switch that changes the connection with the output terminal according to the output of the quantizer.
Unit recognition unit described in. (8) A route input section having a single or multiple route input terminals, a route output section having a single or multiple route output terminals, and a route between the route input terminal of the route input section and the route output section. 3. The unit recognition unit according to claim 2, wherein the path selector includes a loader that changes the strength of connection with the output terminal in accordance with the output of the quantizer. (9) The path input section is configured using an adder that adds input signals from at least a plurality of path input terminals, and the path input section is configured using a threshold processor that performs threshold processing on at least the output signal of the adder. Claim 7 or 8 comprising an output terminal.
Unit recognition unit described in. (10) A learning type recognition determination device configured by combining a plurality of unit recognition units according to any one of claims 1 to 9 in a multilayered structure. (11) The learning type recognition judgment device according to claim 10, wherein the teacher signal is input to the signal input section of the unit recognition unit located at the lowest layer in the multi-layered hierarchical structure. (12) The learning type recognition determination device according to claim 10 or 11, wherein the unit recognition unit located at the lowest layer in the multi-layered hierarchical structure is constituted by the unit recognition unit according to claim 8. (13) A copy creation detection means for detecting a copy creation operation of a previous unit recognition unit in a multi-layered hierarchical structure is provided, and when a copy of the unit recognition unit is created, a lower unit connected to the unit recognition unit is provided. Claim 10: An instruction is issued to create a copy including all of the units.
The learning recognition judgment device described in .