JPH0464134A - Learning method for neural unit with fully coupled rule part - Google Patents

Abstract

PURPOSE:To improve the efficiency of learning by initializing a weight value and a threshold value at each of an antecedent part membership function realizing part, rule part, and consequent part membership function realizing part, and then simultaneously learning the antecedent part membership function realizing part and the rule part after learning the rule part. CONSTITUTION:The weight value and the threshold value based on knowledge stored preliminarily or based on random numbers are initialized at each of an antecedent part membership function realizing part 10, rule part 12, and consequent part membership function realizing part 14. Then, the weights of the antecedent membership function realizing part 10 and the rule part 12 are simultaneously learned after learning the weight of the rule part 12 using a learning data prepared preliminarily. Thus, the learning to operate a control suited to an object system can be realized efficiently.

Description

[Detailed Description of the Invention] [Summary] A control output corresponding to an input is generated according to a hierarchical network structure that realizes a fuzzy control rule consisting of an antecedent membership function realization part, a rule part, and a consequent part membership function realization part. Regarding the learning method of the generated rule part fully connected neuron, the aim is to efficiently learn to perform control suitable for the target system. After setting the initial values of weights and thresholds in each of the realization parts by introducing pre-existing knowledge, using random numbers, or a combination of both, the weights of the rule part are trained using system training data prepared in advance. After that, the weights of the rule section and the antecedent membership function realization section are configured to be learned at the same time.

[Industrial Field of Application] The present invention relates to a learning method for a fully connected rule part neuron that combines fuzzy control and a neural network.

In information processing using neurons, learning data consisting of a set of input patterns of the target system and desired output patterns for that input pattern is presented to a hierarchical neural network to learn and perform adaptive processing. Let them do it. In particular, a processing method called backpropagation method is attracting attention due to its high practicality in this learning process.

On the other hand, fuzzy theory was proposed by Zadeh in 1960, and is a theory that models the process of thinking and judgment based on ambiguity that humans make, such as whether the temperature is "low" or "high." This fuzzy theory is mathematically based on fuzzy set theory, which introduces membership functions to represent ambiguity.

As an application of fuzzy theory, in 1974 Maqmd
Various types of fuzzy control have been used since ani was first used to control steam engines.

Since the system that performs this fuzzy control handles uncertain things, it is necessary to have a configuration that can adaptively respond to adjustments and changes in the control algorithm.

[Prior Art] Generally, the following procedure is taken to construct a fuzzy control system.

■Obtain ambiguous control rules such as ``If the temperature is high, reduce the fire'' held by experienced operators.

■ Quantify the meaning of the words (propositions) in the control rules in the form of membership functions, and convert the control rules into rlF-THEN~
” Described using a type rule expression.

■Conduct inspections through simulations and on-site tests.

■Evaluate inspection results and improve membership functions and rules.

To explain in more detail, the fuzzy control rules are, for example, rif x, is big and X2 is s
"mall thenV+ is big".

Here, the IF part is called the antecedent part, and is the control state quantity such as temperature (
This is the part that describes the conditions for (control input). Further, the THEN section is called a consequent section, and is a section that describes conditions regarding the control operation amount (control output) of the operating end, etc.

Fuzzy control manages multiple fuzzy control rules prepared as the control logic of the target system, and also manages the meaning of ambiguous linguistic expressions such as "large" or "small" written in each fuzzy control rule. Quantify and manage it as a ship function.

The control procedure of the completed fuzzy control system is as follows.

■When the controlled Lq quantity such as temperature is input from the controlled object,
Calculates the truth value according to the membership function of the antecedent part that is set.

(2) Next, in accordance with the antecedent computation such as selecting the minimum value, a process is executed to determine the value to be applied to the consequent in each fuzzy control rule.

For example, the truth value of "r x, is big" = 0.8'
2 is small If the truth value of J = 0.5, then the antecedent operation that selects the minimum value, therefore, the truth value =
Processing is performed so that 0.5 is output as the applied value to the consequent of the fuzzy control rule.

(2) Next, a process is executed to determine an output application value from the input value (applied value) from the antecedent part to the membership function of the consequent part, which has the same amount of control operation.

For example, with two fuzzy control rules, r yl
Application value of is bigJ = 0.5' V+ is bi
If the common values of "g" are obtained, the applicable values -0, 6 are selected and determined according to the consequent operation that selects the maximum value.

■Next, the membership function of the control operation amount is reduced (or expanded) according to the determined applied value, and the center of gravity of the figure of the sum of the membership functions for the same control operation amount is calculated. The control operation amount, which is a fuzzy inference value, is calculated and output.

By the way, fuzzy control rules are planned to be generated according to the knowledge possessed by a skilled operator, and it is difficult to establish fuzzy control rules that can realize optimal control from the beginning. Therefore, it is necessary to evaluate the generated fuzzy control rules through simulations and on-site tests, and to improve them through trial and error.

Therefore, the inventors of the present application have proposed a hierarchical network structure that realizes fuzzy control rules by applying neurons, that is, an adaptive fuzzy control system using a rule section Briwire Neuro (Patent Application No. −
No. 60256 "Hierarchical network configuration data processing device and data processing system"). That is, the rule part prewire neuro has a hierarchical network configuration that realizes a fuzzy control rule consisting of an antecedent part membership function realization part, a rule part, and a consequent part membership function.

Considering the automatic extraction of fuzzy control rules by this rule section Briwire Neuro, the following procedure may be taken.

■Set the membership function of the rule section prewire neuro.

■The rule section Briwire Neuro is presented with an input pattern of the target system and a set of desirable output patterns for that input pattern, and is made to learn.

■Extract the optimal fuzzy control rule for the target system by analyzing the weights of connections in the rule section.

[Problem to be solved by the invention]

However, the learning that is performed to extract fuzzy control rules for the rule part fully connected neuron involves the weights of the antecedent membership function part, the rule part, and the consequent part menhashitube function realization part. The learning method for making the robot perform this action has not yet been determined, and there is a need to develop an efficient learning method that can maximize the performance of the rule-part fully connected neuron.

The present invention was made in view of the above situation, and an object of the present invention is to provide a learning method for a fully connected rule section neuron that can efficiently perform learning for controlling a target system. .

[Means to solve the problem]

FIG. 1 is a diagram explaining the principle of the present invention.

The present invention has a hierarchical network structure consisting of an antecedent membership function realization unit 10.1 or a rule unit 12 having multiple layers and a consequent membership function realization unit 14,
All units are internally connected between the internal layers of the rule part 12 and between the antecedent part Menhashinobu function realization part 10 of the preceding stage and the consequent part Menhashinobu function realization part 14 of the latter stage, and are inputted. Control state quantities (X+, Xz,
...

In the present invention, as a learning method for such a rule part fully connected neuron, <First process> Each of the antecedent part membership function realization part 10, the rule part 12, and the consequent part membership function realization part 14 is Set initial values of weights and thresholds.

<Second Process> After learning the weights of the rule part 12 using the learning data prepared in advance, the weights of the antecedent part Men/Nshi knob function realization part 10 and the rule part 12 are learned.

<Third Process> The weights of the rule unit 12 and the antecedent membership function realizing unit 10 are simultaneously trained using training data prepared in advance.

A learning method with a processing process is constructed.

Here, the method for setting the initial values of the weights and thresholds in the first process is based on prior knowledge (in the figure, "a priori knowledge" is used).
(denoted as j).

Another method for setting the initial values of the weights and thresholds in the first step is to set initial values created using random numbers.

Further, in setting the initial values of the weights and thresholds in the first process, those that can be set using knowledge held in advance are initialized using this knowledge, and other initial values may be set using random numbers.

Furthermore, at the start of learning in the second and third processes, a learning flag is set for each connection in the hierarchical network structure consisting of the antecedent membership function realization unit 10 and the rule unit 12, and the learning flag is set for each connection. The weight values are optimized through learning processing based on the set learning flag.

[For production]

According to the learning method according to the present invention for the rule section fully connected neuron having such a configuration and defined in FIG. 3, the following effects can be obtained.

First, we need to know in advance about the rule part fully connected neuron, which has a hierarchical network configuration consisting of an antecedent membership function realization part, a rule part, and a consequent membership function realization part, which realize the fuzzy control rules of the target system. Initial values of weights and thresholds are set based on random numbers, initial values of weights and thresholds are set based on random numbers, or both are used in combination.

Next, the learning process is started, the input pattern for learning is presented to the rule part fully connected neuron, and the output pattern from the rule part fully connected neuron is a learned output pattern (teacher signal ), learning is performed using, for example, a backpropagation method.

Furthermore, learning efficiency can be improved by initializing the weights and thresholds of the antecedent membership function realization section and the consequent membership function realization section into which knowledge held in advance is introduced.

Furthermore, the performance of the network can be maximized by initializing the weights and thresholds of the antecedent membership function implementation section, the rule section, and the consequent membership function implementation section using random numbers.

[Embodiment] FIG. 2 is a learning operation flowchart showing a processing procedure as a first embodiment of a learning method for a fully connected rule part neuron according to the present invention. The rule part fully connected neuron that is the object of the learning method according to the procedure shown in FIG.
2, and a consequent membership function implementation unit 14 (including a centroid calculation implementation unit 18), the system has a hierarchical network structure for realizing fuzzy control rules.

In the learning process in FIG. 2, the following steps S1 to S6
Performs processing consisting of.

<Step Sl> Weight Cab of the antecedent membership function realization unit 10.

Initial settings are made by introducing knowledge that is previously held so as to realize a membership function given Cbc.

Step 32> The weights Ccd and Cde of the rule section 12 are initialized using random numbers.

<Step S3> Preliminary knowledge is introduced and initialized so as to realize the membership function given the weight Cef of the consequent part membership function realization unit 14.

<Step 34> The weight Cfg of the center of gravity calculation realization unit 18 is initially set to realize center of gravity calculation. Incidentally, in FIG. 5, it is included in the consequent membership function realization unit 14.

Step S5> A group correspondence table for each phase in which the learning process is performed (a table that specifies the group of neurons to be learned, that is, a hierarchy) and a connection correspondence table for each group (a table that indicates the group of connections to be input to the group of units) )
Configure settings.

<Step 36> The learning scheduler is activated to perform learning processing using the pack propagation method.

Hereinafter, steps S1 to S6 shown in the operation flow of FIG.
will be explained in detail separately.

(Configuration of rule section fully connected neuron) The rule section fully connected neuron that is the object of the present invention has, for example, a hierarchical network configuration that can realize the fuzzy control rule shown in FIG.

In Fig. 5, the nodes shown in each part constitute one basic unit, and the basic unit consists of two units: a sigmoid function unit shown by a black circle, and a linear function unit shown by a white circle.
seeds are used.

FIG. 4 shows the configuration of the basic unit in the rule part fully connected neuron of FIG. 5.

In FIG. 4, the basic unit 20 constitutes a multi-power single output system, and includes a multiplication section 22 that multiplies a plurality of inputs by the weight value of each internal connection, and an addition section that adds the multiplication results of the multiplication section 22. and a threshold processing section 26 that performs threshold processing on the output of the addition section 24.

Here, if the first layer is the h layer and the second layer is the i layer, then the multiplier 2
2 and the addition unit 24 executes the calculation of equation (1) below, and the threshold processing unit 26 executes the calculation of equation (2) below.

Xpi−ΣyDhW1h (1
) However, h: Unit number of h layer i: Unit number of 1st layer p: Input signal pattern number θ1: Threshold value of the 1st unit of i layer Wih: h Weight value of internal connection between s layers x, i:
Sum of products 3'phi of inputs from each unit of the h layer to the 1st unit of the i layer. Output ypi from the 1st unit of the h layer for the input signal of the Pth pattern: Output from Unit 1 Here, the above equation (2) is known as a sigmoid function, and the threshold processing section 26 of the basic unit indicated by a black circle in the rule section fully connected neuron in Fig. 5 calculates the above equation (2). Performs threshold processing according to the sigmoid function.

On the other hand, the linear function units shown by white circles in FIG. 5 are set in the threshold value processing section 26 with a linear function such that 'pi=Xpi. However, the input section, which is the first unit of the antecedent membership function realization section 10, is excluded (the hardware constituting the basic unit in FIG. Device J can be used.

Referring again to FIG. 5, the configuration of each part of the rule part fully connected neuron will be explained.

First, the antecedent membership function realization unit 10 receives each input variable X, , X2 as input, and outputs the grade value of the membership function of the input variables X to X, . In the figure, each membership function is a one-variable function for simplicity, but it may be a multi-variable function.

In other words, with variables XI and X2 as input, X1 & Xz (SA) It may also be a membership function.

The rule part is a hierarchical neural network. The grade value of the membership function of input variables X, , X2 is input, and the grade value of the membership function of output variable Y is output.

The consequent membership function realization unit 14 (including the centroid determination unit) calculates the membership functions y(SA) to y(
Input the grade value of LA) and output the value of the output variable Y.

In addition, in Figure 5, each unit is explained as one neuron, but parts other than the rule part whose functions can be specified are not treated as neurons but as gate circuits, arithmetic units, etc. that realize the unit function. Good too.

In FIG. 5, a case is taken as an example in which two inputs, X, .

The input section 16 is composed of separation units 28-1 and 28-2, which divides the inputs Xr and Xz into two and supplies them to the antecedent membership function realization section 10. The antecedent membership function realization unit 10 includes four sigmoid function units 30-1 to 30.
-4 and each sigmoid number unit) 30-1 to 30-4 are input to the same four linear function units 32-1.
~32-3.

The rule section 12 includes sigmoid function units 34-1 to 34.
-5, each sigmoid function unit 34-1 to 34
The antecedent membership function realization unit 10 for -5 is connected to all units. The outputs of the sigmoid function units 34-1 to 34-5 are all connected to the linear function units 36-1 to 36-2 which perform consequent calculations.

The consequent membership function realization unit 14 is composed of nine linear function units) 38-1 to 38-9, and determines membership based on two types of output values (grade values) output from the rule unit 12. The function is reduced (or enlarged) and the function sum of the two reduced membership functions is calculated.

Furthermore, the center of gravity calculation realization unit 18 is configured to output device 40 for determining the center of gravity.
The weighting of the output from the consequent membership function realization unit 14 is performed by two linear function units l-4 provided in the center of gravity determining element output device 40.
0-1 and 40-2 respectively to calculate the two centroid calculation elements Y,,Y.

Finally, the center of gravity calculation device 42 calculates the center of gravity from the two center of gravity calculation elements YI and Yz to generate the control output Y.

Figure 6 shows the rule part fully connected neuron in Figure 5.
4 shows neuron groups and connection groups for implementing the learning method according to the present invention.

In Figure 6, the neuron group is Ga-Gg 7
However, among these, the input neuron group Ga
and the neuron group Gg of the center of gravity calculation implementation unit 18 are excluded from the learning target because they are simply separated and added and combined. Since the realization unit is to be trained, there are four neuron groups to be studied in the present invention: Gb, Gc, Gd, and Ge.

On the other hand, the connection group connecting each neuron group is Ca
t+-Cfg, and since learning is performed to correct the weight value of the input connection located in the previous stage of the neuron group to be learned by the pack propagation method, the connection group Cab-Cde is subject to the learning process according to the present invention. subject to.

Regarding the learning process according to the operation flow of FIG. 2 for the rule section fully connected neuron having the configuration shown in FIG. 5.6, the initial setting of weight values for each section will be explained first as follows.

(Initial setting of the antecedent membership function realization unit) The initial setting of the weight value for the antecedent membership function realization unit 10 is performed in the connection groups Cab and Cbc of the antecedent membership function realization unit shown in FIG. The weight of each connection is
Initialize to implement the given membership function.

FIG. 7 shows an extracted side of the control input X in the antecedent membership function realization unit 10 of FIG.

In FIG. 7, the sigmoid function unit 301.30
The output value y output from -2 is y-1/ (1+e-(
W"-") (3).

In the initial setting of the weight value for the antecedent membership function realization unit 10 of the present invention, based on the knowledge that is available in advance, for example, the weight value W is set to the input connection of the sigmoid function unit 30-1, such as W. 20 is set, and θ=-10 is set as the threshold value θ in the sigmoid function unit 30-1.

Further, the input connection of the other sigmoid function unit 30-2 is set as the weight value W, and at the same time, the threshold value θ is set as θ=10 in the sigmoid function unit 302. Note that the weight value of the input connection of the linear function unit 32-L32-2 is set to the initial weight value l.

In this way, in the sigmoid function unit 30-1, W〈0
When the weight value and threshold value are initially set to θ〈0, the eighth
Truth value calculation function of a membership function with a function shape such that the unit output value X, (SS) for the control input X1 shown in the characteristic curve 44 in the figure, that is, the unit output value X, (SS) decreases as the control input X increases. can be set.

Also, in the sigmoid function unit 30-2, W>0
By initially setting the weight value and threshold value such that θ〉0, the unit output value X, (LA) for the control input X shown in the characteristic curve 46 in FIG. It is possible to realize a function of calculating the truth value of a membership function having a function shape in which (LA) becomes large.

Sigmoid function unit 301.30 shown in FIG.
The initial setting of the weight value and threshold for -2 is such that the desired antecedent membership function is also realized for the remaining sigmoid interval number units 1-30-2 and 304 in the antecedent membership function realizing unit 10. Initial settings are made based on prior knowledge.

(Initial setting of the rule section) The initial setting of the weight value for the rule section 12 shown in FIG. 3 has in advance the weight value of each connection belonging to connection groups Ccd and Cde included in the rule section shown in FIG. Random numbers generated under certain conditions by introducing knowledge (for example, +
1 to -1).

Sigmoid function unit 34-1 in the rule section 12
The function of the facy logic operation based on the weight value initially set to the input connection of ~34-8 and the threshold value set to the unit itself will be explained as follows.

FIG. 9 shows one sigmoid function unit 34 provided in the rule section 12, in which the grade value X, (SS) to X2 (LA) is obtained from the antecedent membership function realization section 10 located in the previous stage. is input, and the grade value y (SS) of the output value y is calculated by product-sum calculation and threshold processing.
y (LA).

Above X+(33)=)[,,L (LA)=xzx
When z (SS)=X3 and Xz (LA)=X4, the output value y output from the sigmoid function unit 34 in FIG. 9 is as follows: S = WXI-X I+ Wxz・x2+WX,・X
3+Wx4・x3−θ (4)y=1
/ (1+e-") (5). The unit output value y given by this equation (5) can be calculated using an appropriate function by arbitrarily setting the input connection weight values WX, ~WX4 and the threshold value θ. shape can be realized.

Note that the linear function unit 3 provided at the final stage of the rule section
The weight value of the connection for 6-1.36-2 is initially set to 1, and the threshold processing is a linear function,
A threshold value θ-0 is set.

(Initial setting of consequent part membership function realization part) Initial setting of the weight value for the consequent part membership function realization part 14 in FIG. Initial settings are made to realize the given membership function based on the knowledge of the given membership function.

FIG. 10 is an explanatory diagram specifically showing the initial setting of weight values for the consequent membership function realization unit 14.

In FIG. 10, the consequent membership function realization unit 14
Nine linear function units 38-1 to 38- provided in
9, the linear function units 36-1 to 36-2 of the rule section 12 at the previous stage perform connection input as shown in the figure. These nine units 38-1 to 38-9 correspond to points in the range O to 1 on the control output Y training line at equal intervals.

The initial setting of the weight values for the input connections to the linear function units 38-1 to 38-9 is as follows: unit 3 according to the characteristic curve 46 of the graph where is the control output Y and the vertical axis is the unit output value.
Weight value from 8-1 side to unit 38-9 side, 1, 1. 1.0.75.0.5.0.25. O
,0,0 are initialized.

Conversely, for the connection from the linear function unit 1-36-2 of the rule section 12 that produces the unit output y (LA), the weight values 0, 0 are applied from units 38-1 to 38-9 according to the characteristic curve 48. ．． 0.0.25.0.5. 0.75. 1
．． 1. Initialize 1.

Furthermore, the linear function units 38-1 to 38-9 do not perform nonlinear threshold processing, and the threshold value θ is set to θ-0.

Due to the initial setting of the weight values in the consequent membership function realization section 14, the unit output value output from the linear function unit 36-1, 36-2 of the rule section)'
(SS) and y (LA) are reduced by multiplying by the initial weight value and then added, and the next center of gravity calculation implementation unit 18
is output to.

In FIG. 1O, a weight value of 1 or less is initially set so as to reduce the unit output, but it is also possible to set a weight value of 1 or more to enlarge it.

(Initial Setting of Center of Gravity Calculation Implementation Unit) The weight values for the center of gravity calculation implementation unit 18 shown in FIG. 5 are initialized so as to realize center of gravity calculation for the connection group Cfg shown in FIG. 6.

FIG. 11 shows a specific example of initial setting of weight values for realizing the center of gravity calculation performed on the center of gravity determining element output device 40 of the center of gravity calculation implementation unit 18.

For the two linear function units 40-1 and 40-2 provided in the center-of-gravity determining element output device 40, the nine linear function units 38- Each of 1 to 38-9 is connected. Then, as shown in the figure, weight values for calculating the two gravity center derived values Y, , Y2 used in the gravity center calculation are set for this input connection.

In fuzzy control, a method is often adopted in which the control output Y, which is a fuzzy inference value, is calculated by finding the center of gravity of the figure of the sum of functions of reduced membership functions for the same control output according to the following formula: .

Y -S grade (y)ydy/ S grade
e(y)dy (6) In order to realize this equation (6), linear function unit 3-1 provided in the consequent membership function realization unit 14 for the linear function unit 40-1
As the weight value for the connection from ~389, the control output Y
Starting from the minimum value Y=O, weight values are assigned that increase from O at equal intervals up to the maximum (!Y=1).

On the other hand, for the linear function unit) 40-2, from the maximum value Y=1 of the control output Y to the minimum value Y=0, it decreases from 0 to, for example, -1 at the same equal intervals as above. Assign weight values.

By weighting the input connections to the linear function units 40-1 and 40-2, if the unit output values of the linear function units 38-1 to 38-9 in the previous stage are C1 to C1, the linear function unit 401 becomes Y+ = I C, +○, 875C2+0.75C3+
0.625C, +0.5 C3 + 0.375C6 〇,
25C, +O, 125(, +OC, (7), and the linear function unit 40-2 outputs YZ = OC1-0
, 125C2-0, 25C30, 375C4-0, 5C
6-0,625C.

-0,75C, -0,875c8-zC, (8) is output.

The centroid derived values Y+ and Yz output from the linear function unit) 40-1.40-2 are given to the centroid calculation device 42, and are calculated as follows:
9) is used to calculate the center of gravity, that is, the control output Y, which is a fuzzy inference value.

By substituting the above equations (7) and (8) into the above equation (9), it can be seen that Y in the equation (9) matches Y in the equation (6).

With the above processing, the initial setting of the weight value for the rule section fully connected neuron shown at 5l-34 in FIG. 2 is completed, and the process proceeds to step S5.

(Setting of Learning Schedule) In step S5 of FIG. 2, following the completion of the initial settings, a learning plan, ie, a learning schedule, for causing the rule section fully connected neuron to perform a learning process is set. This learning schedule is set by setting the phase/group correspondence table shown in FIG. 12(a) and setting the group/connection correspondence table shown in FIG. 12(b).

First, the phase-group correspondence table specifies the neuron groups to be learned in each phase as the learning phase progresses. In the present invention, after setting the initial weight values, the weights of the rule section 12 are first learned, and then the weights of the antecedent membership function realization section 10 and the rule section 120 are learned simultaneously. As shown in FIG. 6, two neuron groups Gd and Ge shown in FIG. 6 are set as learning target groups in phase 1, and four neuron groups Gb, Gc, Cd, and Ge are further set in phase 2.

On the other hand, the group/connection correspondence table sets the correspondence between neuron groups Ga-Gg and input connection groups in the rule part fully connected neuron, and as shown in FIG. 12(b), The input connection groups Cab-Cgf are associated with each other (see FIG. 6).

This kind of phase group correspondence table and group
When the setting of the connection correspondence table is completed, proceed to step 86 in Figure 2 to start the learning scheduler, actually give learning data to the fully connected neuron in the rule section, and change the weight value of the input connection using the pack propagation method. Have students learn how to do things.

(Processing and configuration of learning scheduler) FIG. 13 shows a learning processing flow diagram of the rule part fully connected neuron of the present invention. This operation flow diagram can be realized, for example, by the device configuration shown in FIG.

First, to explain the device configuration for learning processing, the 14th
In the figure, a learning instruction unit 50 and a learning unit 70 are provided for the rule section fully connected neuron 1 (10).

The learning instruction unit 50 has a learning scheduler 52,
For the learning scheduler 52, there is a learning signal storage section 54 that stores learning signals consisting of pairs of control inputs and desired control outputs for the control inputs, and a phase group storage section 54 that stores the phase group correspondence table shown in FIG. A correspondence table 56 and a group/connection correspondence table 58 storing a group/connection correspondence table are connected. Further, a learning convergence determination section 60 is provided for the learning scheduler 52.

On the other hand, the learning unit 70 is provided with a learning instruction reading section 72, a learning flag setting section 74, and a weight changing section 76 that changes weight values using the hack propagation method.

Furthermore, a feature of the device configuration in FIG. 14 is that a learning regulator 80 is provided in the connection connecting each unit in the rule section fully connected neuron 1 (10).

The learning adjuster 80 has the configuration shown in FIG.

Note that the weight change unit 76 is shown together with the learning adjuster 80.

The learning adjuster 80 is provided with a flag storage section 82, a weight change information reading section 84, and a weight change amount adjustment section 86.

On the other hand, the weight change unit 76 includes a weight calculation unit 88 that performs weight calculation based on the pack propagation method, and a weight storage unit 9.
0 and a weight change amount storage section 92 are provided.

In this way, by providing the learning regulator 80 for each connection of the rule part fully connected neurons, the learning scheduler 52 of the learning instruction unit 50 can control the rule part fully connected neurons 1 (1
A learning flag can be set to determine whether or not to change the weight value of a connection relative to 0, and the weight can be changed, that is, learning can only be performed for connections for which the learning flag is valid, that is, the flag is on. becomes possible.

Further, the flag set for the learning coordinator 80 is set to set the antecedent member that can execute fuzzy rules for a general hierarchical child network that does not have a network structure that can implement fuzzy rules as shown in FIG. It is also used when specifying a hierarchical network consisting of the ship function realization section 10, the rule section 12, and the consequent membership function realization section 14.

Adjustment of weight values when actually learning by the back propagation method with a hierarchical network conforming to fuzzy rules configured is performed as follows.

In FIG. 15, the weight change information reading section 84 of the learning adjuster 80 monitors whether the weight change amount is written by the weight calculation section 88 in the weight change amount storage section 92 of the weight change section 76. . When the weight change information reading unit 84 detects that the weight has changed, it notifies the weight change amount adjustment unit 86 of the fact. The weight change amount adjustment unit 86 checks the learning flag in the flag storage unit 82, and does nothing if the learning flag is on.

On the other hand, if the learning flag is off, the weight change amount adjustment unit 8
6 sets zero in the weight change amount storage section 92 of the weight change section 76 to invalidate the weight change. The hardware constituting this learning scheduler is shown in Japanese Patent Application No. 63-227825 entitled "Learning Method for Network Configuration Data Processing Apparatus".

Next, with reference to the learning processing flow shown in FIG. 13, the learning processing of the rule section fully connected neuron according to the device configuration shown in FIG. 14 will be described.

First, when the learning scheduler 52 of the learning instruction unit 50 is activated, it first sets the phase counter i, which indicates the progress of the learning phase, to i=1 in Sl, and proceeds to S2, where the phase counter 52 is stored in the phase/group corresponding chifur 56. The neuron group number corresponding to learning phase i=1 is read out with reference to the correspondence table. In the present invention, in phase 1, the rule part fully connected neuron 1 (10 neuron groups Gd and Ge of the rule part 12), and in phase 2, the antecedent part membership function realization part 10 and the rule part Twelve neuron groups cb to Ge are set.

Next, the process advances to S3, and the connection number corresponding to the neuron group number is read out with reference to the group/connection correspondence table 58.

Next, the process proceeds to S4, where the connection group number is output and the learning flag of the learning adjuster 80 provided to the connection belonging to the connection group number is turned on.

Specifically, the learning scheduler 52 of the learning instruction unit 50 outputs a connection group number to the learning instruction reading section 72 of the learning unit 70, and the learning instruction reading section 72 reads the connection group number and sends it to the learning flag setting section 74. give. The learning flag setting unit 74 selects a learning adjuster 80 provided in the connection belonging to the connection group number that received the learning instruction.
The learning flag is set by instructing the flag storage unit 82 to turn on the flag.

Next, the process proceeds to S5, where the learning scheduler 52 issues a learning execution command to start the learning process. This learning process reads the control input X prepared in the learning signal storage section 54 of the learning instruction unit 50 and the teacher signal d that is a desired control output for the control input X, and inputs the control input X into the rule section fully connected neuron 1 ( 10, and the teacher signal d is given to the weight change unit 76 and the learning convergence determination unit 60.Then, the learning input X causes the control output Y output from the rule unit fully connected neuron 1 (10 The weighting unit 76 changes the weight value according to the hack propagation method, and the learning convergence determining unit 60 performs learning when the error of the control output Y with respect to the teacher signal d becomes less than a specified value. The end is determined and the learning process ends.

When the learning process ends based on the learning execution command in S5,
Increment the phase counter i by one in S6, and
Check the end of the learning phase at 7. Since the learning process of the present invention ends at phase 1, the series of processes ends as is.

(Principle of learning by backpropagation method) Next, the principle of learning by the bank propagation method of the present invention will be explained together with the hierarchical network.

FIG. 16 shows a general hierarchical neural network, which is composed of a type of note called a unit and internal connections with weights.

In Figure 16, the h layer is the input device, and the i layer is the intermediate layer.
layer, and the j layer as the output layer. Here, there is only one i-layer as the intermediate layer, but the same holds true even if there are a plurality of intermediate layers. As shown in FIG. 4, each unit includes a multiplier 22 that multiplies a plurality of inputs by the weight of each internal connection, and an adder 2 that adds all the multiplication results of the multiplier 22.
4, and a threshold processing section 26 that performs threshold processing on the addition result and outputs one output. The calculations performed in this unit are shown in (1) above regarding the basic unit 20.
The formula (2) is obtained. In the pack propagation method, the weight value Wih and threshold value θi of the hierarchical network are adaptively and automatically adjusted and learned using error feedback. Said (
1) As is clear from equation (2), it is necessary to adjust the weight value Wih and the threshold value θi at the same time, but this work is a difficult work that interferes with each other. Therefore, in addition to the normal unit, the applicant has provided a unit whose output value is always 1 in the h layer on the input side, and this output value is 1.
We have proposed a hack propagation method that can incorporate the threshold θi into the weight value W i h and treat the threshold θi as a weight value by assigning the threshold θ1 as a weight value to the output of the unit. Gansho 62-333
No. 484).

By doing this, the above equations (1) and (2) become XpJ
=Σypl, W, h(11) Vp;= 1/(1+exp(xpi))
(12), and the threshold value θi does not appear.

By expanding from equations (11) and (12) above, we get the following (1
3) Equation (14) is obtained.

Xpj””ΣypiWj□
(13) y pi = 1 / (1+exp(-x
pJ)) (14) However, j: unit number of the j layer W7H: weight value X of internal connection between the i-j layers, j: i
Sum of products ypj of the inputs from each unit of the layer to the 1st unit of the j layer: Output from the 5th unit of the j layer for the input signal of the I'th pattern. Cover turn
When receiving the teacher pattern dpj, which is the signal to be taken by the output cover turn 3'pj and the output cover turn ypj output from the output layer j when , the difference value (dpJ-yt+i) between the two is calculated, and δpj=)'p, N-yp,) (dpi-yp;)
(15) is calculated. Subsequently, ε: Learning constant ζ: Momentum t: Update amount of weight values between i layer and j layer according to the number of learning times △W, 1(t)
Calculate. Here, the reason why the update amount of the weight value determined in the previous update cycle is added in the second term on the right side of the equation (16) is to speed up the learning.

Next, the weight value changing unit 76 uses the dpJ calculated in (15) 2 to calculate δ9,=Vp;(I Vp,)Σδ,jWji(t−
1) Calculate (17), and update the weight values of h layer and i layer according to the following: △W=t, (t)
Calculate.

Next, the weight change unit 76 uses the above equation (16) and (18).
The weight value for the next update cycle is determined according to the update amount calculated using the formula Wji(t) = WJ, (t-1)+ΔWji(t)
(19) W = h (t)' = W = h
(t 1)+Δw i 1. (t).

The above learning process is repeated so that a weight value is obtained in which the output pattern ypj corresponding to the learning input pattern matches the teacher pattern d9.

On the other hand, when the units constituting the hierarchical network of FIG. 16 are linear units, the above equation (14) is expressed as y p == x pJ (20). The above (1
Equations 5) and (17) are δpJ=d pJ−y pj
(21) δ2, = Σδ,; in W (t-1)
(22).

(Weight Learning Process of the Present Invention) FIG. 17 is a processing flow diagram of weight learning performed by the pack propagation method on the rule part fully connected neuron of the present invention shown in FIG.

The parameters of each part in this learning process flow are the 18th
It is defined as shown in the figure.

Figure 18 schematically shows the hierarchical structure of the rule section fully connected neuron in Figure 5, with layers 1 to 6 as shown.
It has a six-layer structure, with the first and third layers made up of sigmoid function units, and the rest made up of linear function units. In addition, the learning target is from the first layer to the fourth layer, and the final stage is the centroid calculation realization unit 1.
The sixth layer on the day is excluded from weight learning.

Here, the fuzzy control value output by realizing the fuzzy rule by the fully connected neuron of the rule part, that is, the control output value, is y6, and the learning process is performed from the 6th layer to the 5th layer, the 4th layer, the 3rd layer, and Perform the second layer, first layer, and so on sequentially. The progress of learning for each layer is indicated by an i counter. That is, the i counter is set in the i layer 6 in the initial state.

To further simplify the explanation, Wi, i-1 and ΔWi+
i-1 indicates a matrix, and the size is (number of neurons in the i-th layer) x (number of neurons in the i-1th layer). Furthermore, δi and yi represent vectors whose size is the same as the number of neurons in the i-th layer.

The weight learning process flow shown in FIG. 17 will be explained based on the parameters of each part shown in FIG. 18 as follows.

First, in step Sl (hereinafter "step" will be omitted and simply described as 31), a counter i for setting a learning target layer is initialized. Specifically, since the device has a six-layer configuration as shown in FIG. 18, the counter i is initially set to 1-6. With this initial setting, the sixth layer is designated as the learning target.

Next, the process proceeds to S2, where it is determined whether the unit is a sigmoid function unit. In this case, since it is a linear function unit, 314
The process proceeds to step 315, where it is determined whether the layer is the final layer, and since the sixth layer is the final layer, the process proceeds to step 315, where the difference value δ6 is obtained using the teacher signal d and control output y6 obtained at that time.

Next, the process proceeds to step 36, where it is determined whether the unit is to learn weights. At this time, the 6th layer is excluded from the learning target (
Specifically, since the learning flag of the learning adjuster 80 provided in the connection is off), the process proceeds to S8, sets the weight value update amount △wbs to △W b s = O, and proceeds to S9.
In this case, the weight value W6S is not updated because it is ΔW6S−〇. Next, proceed to S12 and count the counter i=
1, that is, whether learning of all the layers has been completed, and since i=6 at this time, the process proceeds to 313, decrements the counter i by 1 to make i=5, and returns to S2. Proceed to learning the next 5th layer.

Since the fifth layer is also a linear function unit, 82
Proceeds to 314, but since it is not the final stage layer, it goes to 316
Proceed to step 6 and calculate the difference value δ6 obtained in the learning process of the previous 6 layers.
The difference value δ5 is determined from the weight value W65 and the weight value W65 according to a formula similar to the above formula (22).

Next, proceed to 86 to determine whether it is a learning target,
Since the fifth layer is not a learning target in the present invention, the weight value W6. (38 → S9 → S12 → 5
13).

Process all units in 4 layers S 16 → S 6 → S
7→S9→S12→513) When the processing of the 4th layer is completed, the process proceeds to the 3rd layer.

Proceeding to the third layer processing, since a sigmoid function unit is used in the third layer, the process proceeds from 32 to 33, and since it is not the final layer unit, in S5, the difference value δ3 unique to the sigmoid function unit is calculated as above. It is determined using an arithmetic expression similar to equation (17). Since this third layer is also a learning target, proceed from 36 to 87 and calculate the difference value δ in S5.
The weight update amount ΔW,2 is calculated using 3 and updated in S9.

Repeat the same process up to the final unit of these three layers, and
Proceed through the layers to learn about linear function units, and then learn about the first layer.
Proceeding to the layer, learning regarding the sigmoid function unit is performed, but here, only the rule part is learned, so learning processing for the second layer and the first layer is not performed. and,
Processing of the last unit of the first layer is completed and 310 to 81
When proceeding to step 2, since the counter i is now i=1, it is determined that learning of all layers has been completed, and the series of processing is ended.

Learning in phase 1 ends here, but in order to perform learning in phase 2 next, counter i is set to i=6 again and the above-described process is repeated.

In this phase 2, learning processing for the fourth layer to the first layer is performed.

(Initial Setting of Weights Using Random Numbers) Next, another embodiment of the learning method of the present invention will be described with reference to the operation flow diagram of FIG. 19.

In the learning process shown in the operation flow diagram of FIG.
1. S2. Random numbers were used as the initial setting of weight values (including thresholds) for the antecedent membership function realization unit 10, rule unit 12, and consequent membership function realization unit 14 of the rule unit fully connected neuro in S3. It is characterized by The other processes at 34, 35, and 36 are the same as in the first embodiment shown in FIG.

Furthermore, if there is some knowledge that is previously held, settings may be made based on this knowledge.

(Others) The above-mentioned specific example was an example of a rule section fully connected neuron that realizes a fuzzy rule targeting two control inputs X+ and Xz, as shown in FIG. However, it goes without saying that the number of control inputs may be one or three or more.

In addition, in the above embodiment, y= 1/ (1+e-'X") (23
) was taken as an example of the membership function, but
Two sigmoid function units 9 as shown in FIG.
The unit output of 11.94-2 may be applied to the subtraction unit 96 to realize the sigmoid function shown in FIG. The unit output y in this case is Y = 1/(1-e-”””)-1/(1-e-
12””) (24).

[Effects of the Invention] As explained above, according to the present invention, initial settings or random numbers based on knowledge held in advance in each of the antecedent membership function realization unit, the rule unit, and the consequent membership function realization unit are After making the initial settings using or a combination thereof, learning the rule part of the fully connected neuron, and then learning the antecedent membership function realization part and the rule part at the same time, efficient learning can be achieved. This makes it possible to realize more efficient connection weight analysis for realizing fuzzy rules used in any fuzzy control adaptive system.

[Brief explanation of the drawing]

Figure 1 is an explanatory diagram of the principle of the present invention. Figure 2 is a flowchart of the learning operation of the present invention. Figure 3 is an explanatory diagram of the definition of the rule part fully connected neuron. Figure 4 is an explanatory diagram of the basic unit. Figure 5 is a block diagram showing a specific embodiment of the rule section fully connected neuron of the present invention, FIG. 6 is an explanatory diagram of neuron groups and connection groups, and FIG. 7 is a diagram of the antecedent section membership function realization section of the present invention. An explanatory diagram of initial settings. FIG. 8 is an explanatory diagram of the membership function of the antecedent part. FIG. 9 is an explanatory diagram of the rule part unit of the present invention. FIG. 10 is an illustration of the initial settings of the membership function realization part of the consequent part of the present invention. Explanatory diagram, 11th
The figure is an explanatory diagram of the initial settings of the center of gravity calculation implementation unit of the present invention, No. 12
The figure is an explanatory diagram of the settings of the phase/group correspondence table and group/connection correspondence table of the present invention, Figure 13 is a flowchart of the learning process of the present invention, Figure 14 is a diagram of the device configuration of the learning process of the present invention, and Figure 15 16 is an explanatory diagram of a general hierarchical neural network, FIG. 17 is a flowchart of the weight learning process of the present invention, and FIG. Parameter explanatory diagram of each part, 1st
FIG. 9 is an operation flow diagram showing another embodiment of the present invention.
FIG. 0 is an explanatory diagram of another membership function used in the present invention. 10...Antecedent part membership function realization part, 12
...Rule part, 14...Consequent part Membership function realization part, 16...Input part, 18...
... Center of gravity calculation realization section, 20 ... Basic unit, 22 ... Multiplication section, 24 ... Addition section, 26 ... Threshold processing section, 28-128-2.3
2-1~31-4.38-1~389.40-1.40
-2...Linear function unit, 30-1 to 30-
2.3/11~34-5,941.94-2...
- Sigmoid function unit, 40... Center of gravity determining element output device, 42... Center of gravity calculation device, 50.
...Learning instruction unit, 52...Learning scheduler, 54...Learning signal storage unit, 56.
...Phase/group correspondence table, 58...
...Group phase correspondence table, 60...
...Learning convergence determination unit, 70...Learning unit, 72...Learning instruction reading unit, 74...
Learning flag setting unit, 76... Weight changing unit, 80
... Learning adjuster, 82 ... Flag storage section, 84 ... Weight change information reading section, 86 ...
... Weight change amount adjustment section, 88 ... Weight calculation section, 90 ... Weight storage section, 92 ... Weight change amount storage section, 96 ...・Subtraction unit, 1 (1
0...Rule part fully connected neuro.

Claims (5)

[Claims] 1. An antecedent membership function realization unit (10) configured by providing a plurality of basic units that calculate and output the membership function value of an input signal, and a calculation result value output from the preceding processing layer. One or more stages of processing layers each consisting of at least one basic unit that calculates and outputs a calculation result value according to the assigned operation by inputting the multiplication value of the calculation result value and the weight value of the internal connection associated with the calculation result value. between the basic units of the processing layers in the preceding and adjacent stages, the basic unit of the processing layer in the first stage, and the antecedent membership function realization unit (10).
The rule part (12) is configured by adopting a configuration in which the basic units are mutually internally coupled, and the calculation results output by the corresponding basic unit of the last stage processing layer of the rule part (12). a consequent membership function realization unit (14) constituted by at least one basic unit that receives a value as an input and calculates and outputs an output signal value derived according to the membership function value of the output signal. It has a hierarchical network, and the input control state quantity (X_1
~X_n) In the fully connected rule unit neuron that outputs the control operation amount (Y_1 to Y_m) corresponding to Realization part (14
), and a first process of setting weights and initial threshold values for each of the rule units (1) using pre-prepared learning data.
2); and a third process that simultaneously learns the weights of the antecedent membership function realization unit (10) and the rule unit (12). A learning method for fully connected neurons. 2. The initial values of the weights and thresholds of the antecedent membership function realization unit and the consequent membership function realization unit of the first process are set using initial values of weights and thresholds created based on knowledge held in advance. 2. The learning method of a fully connected rule part neuron according to claim 1, wherein the learning method is performed by setting the rule part, and the weight and threshold value of the rule part are set using random numbers. 3. 2. The method for learning a fully connected rule part neuron according to claim 1, wherein initial values of the weights and thresholds in the first step are set using random numbers. 4. In setting the initial values of the weights and thresholds in the first process, the weights and thresholds that can be set using prior knowledge are initialized using this knowledge, and the initial values of the other weights and thresholds are set using random numbers. 2. The method of learning rule part fully connected neurons according to claim 1. 5. At the time of starting learning in the second process, a learning flag is set for each connection of the hierarchical network of the antecedent membership function realization unit (10) and the rule unit (12), and is set on the connection. 2. The method for learning a fully connected rule part neuron according to claim 1, wherein the weight values are optimized through a learning process based on the learned flags.