JPH04291662A - Operation element constituted of hierarchical network - Google Patents

Abstract

PURPOSE:To execute average operation only by one neuron element in respect to an operation element constituted of a hierarchical network. CONSTITUTION:This operation element is provided with a multiplication processing part 4 for multiplying the weights (w1 to wn) of connections determined in each of plural input variables (x1 to xn), an accumulation processing part 4 for adding respective multiplied values obtained by the processing part 4 and finding out a sum and a threshold processing part 6 for applying a sigmoid function to the output of the processing part 5 and constituted so that the weight and the threshold theta are respectively set up in respective processing parts 4, 6 so as to attain average operation and the average operation can be executed only by one reference unit.

Description

[Detailed description of the invention]

0001

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a hierarchical network-configured arithmetic element, and more particularly to a hierarchical network-configured arithmetic element that performs an average calculation using one basic unit constituted by a hierarchical network.

[0002] In a data processing apparatus having a hierarchical network configuration, a set of input patterns of a target system and a desired output pattern corresponding to the input patterns are presented to a hierarchical neural network to learn and perform adaptive processing. In particular, a processing method called the backpropagation method is attracting attention because of its high practicality.

On the other hand, fuzzy theory, which is a new control theory that has been put into practical use in recent years, models the process of thinking and judgment based on ambiguity that humans make, such as whether the temperature is "high" or "low." It is a theory based mathematically on fuzzy set theory, which introduces membership functions to represent ambiguity. In fuzzy theory, an average calculation may be performed.

0004

[Prior Art] Conventional sequential processing computers (Neumann type computers) cannot adjust their data processing functions in response to changes in usage or environment. An adaptive data processing device that follows a parallel distributed processing method using a hierarchical network configuration has been proposed. In this data processing device with a hierarchical network configuration, an output signal (output pattern) from the network structure is output in response to the presentation of an input signal (input pattern) prepared for learning, without creating an explicit program. determines the weight values of internal connections in the hierarchical network structure according to a predetermined learning algorithm in order to match the teacher signal (teacher pattern). Once the weight values are determined through this learning process, a "soft" data processing function that outputs an appropriate output signal from this hierarchical network structure even if an unexpected input signal is input. is realized.

In a data processing device having a hierarchical network configuration, the hierarchical network is composed of a type of node called a basic unit (neuron element) and internal connections having weight values corresponding to internal state values. A data processing device with a hierarchical network configuration has the advantage that weight values for internal connections can be determined mechanically if a learning signal is obtained.

On the other hand, in recent years, fuzzy control has become popular as a new control processing method for control objects that are difficult to model. This fuzzy control expresses a control algorithm that includes ambiguity such as human judgment in an if-then format, and calculates the control operation amount from the detected control state quantity by executing this control algorithm according to fuzzy inference. This is to control the controlled object. In order to implement fuzzy control, it is necessary to generate fuzzy control rules that describe the fuzzy control algorithm.

Generally, the following procedure is taken to construct a fuzzy control system. In other words, vague control rules such as ``If the temperature is high, reduce the fire'' that skilled operators have are obtained. Next, the meanings of the words (propositions) therein are quantified in the form of membership functions, and the control rules are described using rule expressions of the "IF~THEN~" type. Next, the constructed fuzzy control system will be inspected through simulations and on-site tests. Then, the inspection results are evaluated and the membership functions and rules are improved accordingly. Through the above steps, a fuzzy control system suitable for the controlled object is obtained.

[0008]

[Problem to be Solved by the Invention] Fuzzy inference, which is realized by such a fuzzy controller, divides the relationship between input and output signals into cases and connects them vaguely according to attribute information called membership functions. , making it possible to establish an execution model for complex data processing functions. However, although fuzzy models established through fuzzy inference have the advantage of being relatively easy to establish, it is difficult to determine the exact function values of membership functions and the exact coupling relationships between membership functions. The disadvantage is that it is difficult to realize the desired data processing function, and it requires a great deal of effort and time to realize the desired data processing function.

[0009] Therefore, if the learning function of the data processing device with the above-mentioned hierarchical network configuration can be used,
Since membership functions etc. can be determined automatically, the above-mentioned drawbacks can be solved. In view of this background, the applicant of the present application has first made it possible to easily improve the accuracy of data processing functions by linking a hierarchical network configuration data processing device and a fuzzy model in a fusion manner. We filed an application for a data processing device with an easy-to-understand execution format (Japanese Patent Application No. 2-60257, ``Analysis Processing Method for Adaptive Data Processing Device'').

By the way, in such a fusion data processing device, if average calculation can be performed, more diverse data processing becomes possible. Therefore, it is expected that the range of application of the device will be greatly expanded.

[0011] Now, considering how to implement a data processing device with a hierarchical network structure that performs this average calculation, if it were attempted to be implemented using software, the burden would be too large and it would be impractical. On the other hand, if you try to implement it using hardware, the amount of hardware will become too large if you implement it with multiple basic units, such as in a data processing device with a normal hierarchical network configuration, and it will be necessary to perform various averaging operations in parallel. become unable to do so.

SUMMARY OF THE INVENTION An object of the present invention is to provide a hierarchical network configuration arithmetic element that performs an average calculation in one basic unit.

[0013]

[Means for Solving the Problems] FIG. 1 is a diagram showing the principle of the present invention, and shows a hierarchical network configuration arithmetic element according to the present invention. This hierarchical network configuration arithmetic element is an average arithmetic element 2 constructed using one basic unit 1, that is, one neuron element. The basic unit 1 has a plurality of input sections 3 and a weight value W1 of each internal connection for a plurality of input signals X1 to Xn inputted thereto.
A plurality of multiplication processing units 4 that multiply by Wn to Wn, one accumulation processing unit 5 that adds all the multiplication results, and nonlinear threshold (θ) processing is performed on all the addition results. and one threshold processing section 6 which outputs one output signal. Here, the weight values W1 to Wn are obtained as a result of learning so that one output signal corresponding to a plurality of input signals W1 to Wn substantially matches the teacher signal using the average calculation as a teacher signal. is the value given. The present invention provides an arithmetic element in which such values are set.

[0014]

[Operation] When a plurality of input signals X1 to Xn are input to the input section 3 of the average calculation element 2 in FIG.
is the weight value W1 for the input signals X1 to Xn
.about.Wn are respectively multiplied. The accumulation processing section 5 adds all of the multiplication results, and the threshold processing section 6 performs nonlinear threshold processing on the total addition results to obtain an output signal. Here, the weight values W1 to Wn are values given in advance, as described above.

Therefore, if average calculation is used as a teacher signal during learning, the element in FIG. 1 becomes average calculation element 2, which outputs the average of these as one output signal when multiple input signals are input. do. That is, the arithmetic element 2 that calculates the average of n input signals can be realized as a hierarchical network-configured arithmetic element using one basic unit 1, and the amount of hardware can be reduced to the necessary minimum. can.

[0016]

Embodiment FIG. 2 is a diagram illustrating the system configuration of a hierarchical network configuration data device. 1'-h are a plurality of input units forming the input layer of the average calculation element (hereinafter referred to as calculation element) 2, 1-i (same as basic unit 1 in FIG. 1)
is one basic unit constituting the intermediate layer of arithmetic element 2,
1'-j is one output unit constituting the output layer of the arithmetic element 2; 10 is a hierarchical network configuration data processing device;
Reference numeral 18 denotes a weight value storage unit that manages weight values Wi assigned to internal connections of the hierarchical network.

Reference numeral 21 is a learning signal storage unit that stores learning control data used for learning weight values, and 22 is a learning signal presentation unit that stores learning control data from the learning signal storage unit 21. 23 reads the control data, presents the control presentation data among them to the arithmetic element 2, and presents the other pair of control teacher data to the weight value changing unit 30 and the learning convergence determining unit 23. The learning convergence determination unit receives the control amount data output from the arithmetic element 2 and the control teacher data from the learning signal presentation unit 22, and determines whether the error of the data processing function of the arithmetic element 2 is within an allowable range. The learning signal presenting section 22 is notified of the determination result.

Reference numeral 30 denotes a weight value changing section corresponding to an internal state value learning processing device, which receives the control teacher data from the learning signal presentation section 22 and the output data of the arithmetic element 2, and changes the weight value according to the back propagation method. Learning is performed so that the weight values converge by calculating the update amount of the weight values and updating the weight values according to the update amounts.

In this embodiment, the system shown in FIG. 2 is used in the arithmetic element 2 to construct an average calculation function. First, the device configuration of the hierarchical network used to implement data processing will be explained in detail. In addition, in order to learn the weight values assigned to the internal connections of this hierarchical network, the back propagation method (D.E.
．． Rumelhart, G. E. Hinton, an
dR. J. Williams, “Learning
Internal Representations
by Error Propagation", P
ARALLEL DISTRIBUTED PROCE
SSING, Vol. 1, pp. 318-364,
The MIT Press, 1986).

The hierarchical network basically consists of a type of node called a basic unit 1 and internal connections having weight values. The basic configuration of the basic unit 1 is as shown in FIG. This basic unit 1 is a multi-input, one-output system, and multiple inputs are multiplied by the weight values of their respective internal connections (for example, W1
X1 , It consists of a processing section 6.

The accumulation processing section 5 executes the calculation of the following equation (1), and the threshold value processing section 4 executes the calculation of the following equation (2).

[0022]

[Math 4]

[0023]

[Math 5]

In these operations, the weight value Wi is (
3), and the threshold value θ is expressed as (4).

[0025]

[Math 6]

[0026]

[Math 7]

In this embodiment, in equation (4), s=
It is 3. n is the number of inputs. Furthermore, in equations (3) and (4), the input signal Xi is used after being processed as shown in equation (5) according to the rules (affirmation or negation) described later.

[0028]

[Math. 8]

In the hierarchical network 10 or the arithmetic element 2, one basic unit 1 having such a configuration is an input unit 1' which distributes and outputs the input signal value Xi as it is.
-h as an input layer, it performs a data processing function of converting an input pattern (input signal) into a corresponding output pattern (output signal).

In the back propagation method, the weight value Wi and threshold value θ of the hierarchical network are adaptively and automatically adjusted and learned by feedback of errors. (3)
, (4), it is necessary to adjust the weight value Wi and the threshold value θ at the same time. The weight value learning processing method using the back propagation method will be described below.

The weight value changing unit 30 uses the output pattern yp from the output layer that is output when the input pattern for learning is presented, and the teacher pattern dp (the input signal of the p-th pattern), which is the signal to be taken, When receiving the teacher signal (for the teacher signal for Calculate.

Next, the weight value changing unit 30 changes the calculated α
Using p, first calculate βp = yp (1-yp) αp, then ΔWi (T) = εΣβp yp + ζΔWi (T-
1) (where Σ is the total sum of p) Calculate the update amount ΔWi of the weight value according to the following. Here, ε is a learning constant, ζ is momentum, and T is the number of learning times. Furthermore, the reason for adding "ζΔWi (T-1)", which is related to the update amount of the weight value determined in the previous update cycle, is to speed up learning.

Next, the weight value changing unit 30 changes the weight value Wi for the next update cycle according to the calculated update amount.
By repeating the method of determining (T)=Wi (T-1)+ΔWi (T), the output pattern yp from the output layer that is output when the input pattern for learning is presented, A weight value Wi that matches the teacher pattern dp, which is the signal to be taken, is learned.

[0034] As a method of configuring a hierarchical network using hardware components, the method described in the patent application filed by the present applicant in 1983-
It is possible to use the one disclosed in No. 216865 (filed on August 31, 1988, "Network configuration data processing device").

That is, as shown in FIG. 3, the general basic unit 1 multiplies the output from the previous layer input via the input switch section 37 by the weight value held by the weight value holding section 38. a multiplier D/A converter 32; an analog adder 33a that adds the output value of the multiplier D/A converter 32 and the previous cumulative value to calculate a new cumulative value;
A sample hold circuit 33b that holds the addition result of the analog adder 33a, a nonlinear function generation circuit 34 that nonlinearly converts the data held in the sample hold circuit 33b when the accumulation process is completed, and outputs to the subsequent layer. It includes an output holding section 35 that holds the analog signal value of the nonlinear function generation circuit 34, an output switch section 36 that outputs the data held by the output holding section 35, and a control circuit 39 that controls each of these processing sections. It is realized by

Next, the hierarchical network configuration data processing device 10 including a plurality of arithmetic elements 2 will be explained according to FIG. In FIG. 4, L1 to L4 are arithmetic elements 2
1 (or 1-i in FIG. 2), each of which performs calculations according to average calculation rules L1 to L4. The rules are as follows. That is, L1:I
Fx(SS)MEANy(SS)THENz(SS)L
2:IF~x(LA)MEANy(SS)THENz(
SS)L3:IFx(SS)MEAN~y(LA)TH
ENz(LA)L4:IF~x(LA)MEAN~y(
LA) THENz(LA). Here, “x(SS
)” represents “x is small”. “~X(
"LA)" represents the negation of "x is large." Therefore, these are the rules for obtaining an average operation on two input signals. For example, rule L1 is the average of two inputs x (SS) and y (SS) (x (SS) + y (SS
))/2 and output it to z(SS). In other words, “IF
x is small mean y is
small THEN is small
” according to fuzzy control rules of the form.

x(SS), x(LA), y(SS), and y(LA) are input units (1'-h in FIG. 2) that are commonly used for a plurality of arithmetic elements 2 and constitute these. can be,
This is a unit in the previous stage of L1 to L4. For example, x(S
S) performs predetermined arithmetic processing and outputs “x
is small” to L1 and L3.

[0038] z (SS) and z (LA) are output units (1'-j in Fig. 2) that are commonly used by and constitute a plurality of arithmetic elements 2, and are units in the subsequent stages of L1 to L4. be. For example, z(SS) receives outputs from L1 and L2 and performs processing using a linear function.

[0039] L1 to L8 perform processing according to the above-mentioned equations (1) to (5), respectively. Since the number of inputs is two in each case, n=2. The input Xi is
It is given as x(SS), etc., and takes a value between "0" and "1". The threshold value θ is determined by equation (4). Weight value Wi
The weights associated with any input Xi are taken so that their absolute values S are equal, and the sign is set to be positive if the input value Xi is treated positively in the rule, and negative if it is treated negatively. The setting of this sign is based on equation (5).

In particular, the weight value Wi is the result of learning by the method shown in FIG. 2 described above, and its absolute value S is S=3 in all cases. Note that the value of S is made to converge in a predetermined case according to the evaluation criteria of the average calculation to be realized. That is, the value differs depending on how approximately the average calculation is realized by the calculation element 2. Then, when S was determined, the threshold value θ was also determined, and the value was 0 or ±3.

The above is summarized in FIG. 4. That is, L1
The threshold value θ is shown below L4, and the weight value Wi is shown for each input. The arithmetic element to which the present invention is applied corresponds to any one of L1 to L4, for example, L1. For example, in L1, the input values x (SS) and y
Find the sum of (SS) multiplied by the weight value Wi = 3, and substitute this and the threshold θ = 3 into equation (2) to find the average of the input values x (SS) and y (SS), This is z(S
Output to S). Since z(SS) performs linear function processing as described above, its threshold value is "0" and the weight of each input is also "1". Therefore, its output is, for example,
It becomes an algebraic sum.

FIG. 5 shows an example of average calculation. this is,
This shows the average y for two inputs X1 and X2, which is calculated mathematically ((X1+X2)/2). X1, X2 and y take values between "0" and "1", and the plane represented by the mesh is the average. In this example, the input
and X2 are both positive, and rule L in Figure 4
Corresponds to 4. Since this average calculation is ideal, it is used as a teacher signal during the learning shown in FIG. 2.

FIG. 6 shows the inputs x (SS) and y (SS) output from the arithmetic element 2 whose basic unit 1 is L1.
shows the average of Here, input x (SS) is positive, input y
(SS) is affirmative, the weight values thereof are both "3", and the threshold value +h is 3.0. Comparing this with the teacher signal of FIG. 5, it can be seen that FIG. 6 approximately realizes the average calculation that should be realized. Therefore, the arithmetic element 2 using one basic unit 1 realizes the average calculation of two inputs.

FIG. 7 shows the inputs x (LA) and y (SS) output from the arithmetic element 2 whose basic unit 1 is L2.
shows the average of Here, input x (LA) is negated, input y
(SS) is positive, and its weight values are “-3” and “3”.
”, and the threshold value +h is 0. Although not shown, the teacher signal in this case is equivalent to the average calculation in FIG. 5 shifted so that the direction of the rising slope matches that in FIG. 7. Therefore, it can be seen that FIG. 7 also approximately realizes the average calculation that should be realized.

FIG. 8 shows the inputs x (SS) and y (LA) output from the arithmetic element 2 whose basic unit 1 is L3.
shows the average of Here, input x (SS) is positive, input y
(LA) is negative, and its weight values are "3" and "-3"
”, and the threshold value +h is 0. The relationship between the teacher signal in this case and FIG. 8 is the same as the relationship between FIG. 7 and the teacher signal described above. Therefore, FIG. 8 also approximately realizes the average calculation that should be realized.

FIG. 9 shows the inputs x(LA) and y(LA) output from the arithmetic element 2 whose basic unit 1 is L4.
shows the average of Here, input x (LA) is negated, input y
(LA) is negative, both of its weight values are "-3", and the threshold +h is -3.0. The relationship between the teacher signal and FIG. 9 in this case is the same as that for FIG. 8 described above. Therefore, FIG. 9 also approximately realizes the average calculation that should be realized.

[0047] In the above calculation example, only the case of two inputs was shown;
) is easy, and even if there are multiple inputs as shown in FIG. 1, the calculation according to the present invention is similarly possible.

Returning to FIG. 4, this hierarchical network configuration data processing device 10 is capable of executing various average calculations in parallel by arranging a plurality of basic units 1, each of which performs an average calculation. Furthermore, each of the basic units 1 has a hardware configuration as shown in FIG.
As mentioned above, by selecting and applying the teacher signal, various average calculations can be realized. Therefore, for example, L1 can be easily changed to L2 without changing the hardware configuration.

The arithmetic element 2 is constructed using one basic unit 1, ie, a hardware configuration as shown in FIG. Therefore, it is possible to have a simpler configuration than the conventional one. Note that conventionally, for example, four basic units 1
If we use
The configuration was complex because it was necessary to extract the output and perform arithmetic processing on a time-sharing basis.

Furthermore, since each basic unit 1 is made to perform one average calculation, it is possible to know which basic unit (neuron element) 1 is performing what kind of calculation, which was not possible in the past. It has characteristics. In neurocomputing, there is no explicit software, so after learning, even if the processing performed by the basic unit 1 was not known individually, it was not considered a particular problem. However, even if the calculation results are correct, the fact that the actual situation is unknown is unsettling to the operator and puts a psychological burden on them. In this respect, in the present invention, since the calculations performed by each basic unit 1 are clear, the psychological burden on the operator can be removed.

[0051]

[Effect of the invention] As explained above, according to the present invention,
In a hierarchical network configuration arithmetic element, the arithmetic element that calculates the average of n input signals can be realized as one basic unit, that is, a hierarchical network configuration arithmetic element using neuron elements, so the configuration can be simplified. This makes it possible to reduce the amount of hardware to the necessary minimum, and to clarify the calculations performed by each neuron element.

[Brief explanation of the drawing]

FIG. 1 is a diagram showing the principle configuration of the present invention.

FIG. 2 is an explanatory diagram of learning.

FIG. 3 is a hardware configuration diagram.

FIG. 4 is a diagram showing a hierarchical network configuration data processing device.

FIG. 5 is a diagram showing an example of average.

FIG. 6 is a diagram showing an example of average calculation.

FIG. 7 is a diagram showing an example of average calculation.

FIG. 8 is a diagram showing an example of average calculation.

FIG. 9 is a diagram showing an example of average calculation.

[Explanation of symbols]

1 Basic unit 2 Average operation element 3 Input section 4 Multiplication processing section 5 Accumulation processing section 6 Threshold processing section 1'-h Input unit 1-i Basic unit 1'-j Output unit 10 Hierarchical network configuration data processing device 18
Weight value storage unit 20 Learning signal presentation device 21 Learning signal storage unit 22 Learning signal presentation unit 23 Learning convergence determination unit 30 Weight value change unit 32 Multiplying D/A converter 33 Accumulation circuit 33a Analog adder 33b Sample hold circuit 34 Linear function generation circuit 35 Output holding section 36 Output switch section 37 Input switch section 38 Weight value holding section 39 Control circuit

Claims (2)

[Claims] [Claim 1] A plurality of input variables (x1, x2,...
, xn) and set the connection weight (x1, x2,...,
A multiplication processing section (4) that multiplies the multiplication processing section (wn), an accumulation processing section (5) that adds up each multiplication value obtained by the multiplication processing section (4) to obtain a total sum, and an accumulation processing section (5) that calculates the total sum. ) is provided with a threshold processing unit (6) according to a sigmoid function for the output of the multiplication processing unit (4) and a threshold (θ) in the threshold processing unit (6) so as to realize an average calculation. A hierarchical network configuration arithmetic element, characterized in that the average calculation is realized by one basic unit. 2. The weight in the multiplication element processing section (4) and the threshold value in the threshold processing section (6) are set to values given by the following equations. A hierarchical network configuration arithmetic element according to claim 1, characterized in that: