JP3476211B2 - Network type information processing system and its learning method - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information processing system having a network structure and a learning system thereof useful in fields involving continuous value output such as control systems and signal processing systems.

[0002]

2. Description of the Related Art As one of conventional techniques for realizing advanced information processing functions, there is a neural network whose information processing structure is modeled on nerve cells of the human brain (for example, by Cohen and Feigenbaum). ,
"Artificial Intelligence Handbook" published by Kyoritsu Publisher, p487-
493, Amari et al., Special Feature "About Neural Networks", Journal of Japan Society for Artificial Intelligence, Vol. 4, pp118-
176 (1989)). In the conventional neural network technology, since the input / output accuracy is basically determined by the number of nodes in the middle layer, the data description amount may increase explosively when trying to ensure the accuracy. When the number of nodes in the middle layer is reduced, the output of the calculation unit at the final stage becomes a coarse stepwise expression, or the types that can be discriminated are reduced. Further, the number of nodes in the middle layer and the number of output nodes increase according to the number of results to be identified (equal to the number of output nodes). Therefore, it is difficult to physically configure the calculation unit, and at the same time, there are many problems in terms of calculation processing time. If the learning function is provided such that each time the inferred result is output, the quality of the result is judged and the threshold value and weight value are updated stochastically, the learning time becomes an even greater problem. Was there. Further, in the prior art, since the processing description inside the system to be learned is basically expressed in the form of link weight and has no conceptual concreteness,
There is a problem that the learning result can be determined only from the outside (that is, input / output data).

The patent applicant of the present application has previously filed a patent application for an invention for solving the above-mentioned problems of the prior art (Japanese Patent Application No. 3-310082 "Network type information processing system"). . This Japanese Patent Application No. 3-310082
In the network type information processing system according to the invention of No. 1, an input layer having a plurality of nodes and an output layer having a plurality of nodes are coupled via a directional link, and the directional link converts information passing therethrough. In the network-type information processing system having a function, and the node of the output layer has a function of performing a function operation on information input via a directional link, the information conversion function of the directional link has a bandwidth. It has a filter function operation unit for performing information conversion according to a filter function having a selective characteristic such as a pass type or a band stop type. According to this
The filter function of one directional link corresponds to the hyperplane realized by a plurality of intermediate layers of the conventional neural network, and the area specification created by the directional link connecting the input layer and the output layer is This corresponds to the area designation indicated by a plurality of hyperplanes each consisting of a combination link of the input layer, the intermediate layer, and the output layer and a node in the above-mentioned conventional technique. Therefore, according to the invention of the network type information processing system, the amount of description required to obtain the same accuracy is reduced as compared with the conventional neural network, the number of elements is reduced, the configuration is simplified, and the processing time is reduced. Can be shortened. The learning method in the network type information processing system is a means for obtaining a normal information processing result from the same input information while the network type information processing system processes information from input information through a directional link and a calculation unit. And the normal result as a teacher signal, the difference (error) between the teacher signal and the above information processing result (output of the calculation unit) is obtained by an evaluation function, and the magnitude of the difference and the vector value are calculated, It has a learning means for modifying the filter function of the directional link via the learning function. Here, as the means for obtaining a normal information processing result, for example, teacher information prepared by comparing input information with correct output information corresponding thereto is prepared in advance, and when input information is given, correct output information is obtained. It is configured to be taken out.

After that, the patent applicant of the present application filed Japanese Patent Application No. 3-
In order to improve the learning process in the invention of No. 310082, a patent application was further filed (Japanese Patent Application No. 4-175056 "Learning system for network type information processing device"). The outline is as follows. The network type information processing device has a plurality of input nodes, a plurality of output nodes, and a directional link connecting the input nodes and the output nodes. The directional link has a filter function storing means (FIG. 22) for storing a filter function which is a non-linear selection type function.
Pattern table 2213) and filter function calculation means (2212 in FIG. 22) for converting the information passing through the directional link by the filter function. Further, the output node is a means for performing a function operation on the information input via the directional link (adding unit 2214 in FIG. 22,
It has a threshold function computing unit 2215). The learning system of this network type information processing apparatus (221 in FIG. 22) is an input means (filter function calculation unit 22 of FIG. 22) for inputting learning data to the network type information processing apparatus.
2), a region determination unit (region determination unit 223 in FIG. 22) that determines whether the region indicated by the input learning data is included in the existing recognition region, and the region determination unit determines the existing region. Filter function updating means (filter function updating unit 224 in FIG. 22) for individually updating each filter function in the set of filter functions forming the existing recognition region when it is determined to be included in the recognition region; And a filter function generating unit (filter function generating unit 225 in FIG. 22) that generates a set of filter functions forming a new recognition region when the region determination unit determines that the region is not included in the existing recognition region. It has a basic configuration. The existing recognition area is an area defined by an existing set of filter functions.

In the above basic configuration, the area determining means determines whether the area (spatial coordinates in the multidimensional area) indicated by the input data belongs to the recognition area limited by the existing filter function set of the network type information processing apparatus. To judge. As a result of the determination, the filter function update means, when the filter function update means belongs to the recognition area, individually sets each filter function of the filter function set corresponding to the recognition area so as to meet the purpose of the network type information processing apparatus. To update. When it is determined that the recognition area does not belong to the recognition area, a new recognition area is set by the filter function generating means.
As a result, the recognition regions can be separated so that a set of filter functions in the same layer cuts a multidimensional region in the axial direction asymmetrically. Therefore, large and small recognition areas are naturally generated, and small rules are generated in the boundary area, so that the recognition rate is increased.

[0006]

The network type information processing system of the invention of the above-mentioned application basically describes the correspondence relationship of a multi-continuous value input to a specific output. Therefore, the output is discrete, and continuous value output cannot be obtained as it is. Therefore,
An object of the present invention is to extend the invention of the above-mentioned application to provide a network type information processing system capable of describing the relationship between multiple continuous value inputs and continuous value outputs and a learning method thereof. The present invention also operates at high speed,
It is an object of the present invention to provide a learning method for a network type information processing system which can converge learning from a small number of input / output pairs.

[0007]

According to the present invention, there are provided a plurality of first nodes (101 to 104 in FIG. 1), a plurality of second nodes (105 to 108 in FIG. 1), and the above-mentioned first node. 1
In a network-type information processing system including a directional link having a non-linear selective function characteristic connecting the second node and the second node, the second node calculated from the input value.
And the degree of matching, which is the output value of the node, for each second node.
A unique sequence defined according to the arithmetic or geometric sequence
Output combining means for obtaining a combined output value based on the output value is a network type information processing system in which a (109 to 112 in FIG. 1).

The unique output value defined in advance in each of the second nodes is a value to be used when combining and outputting the corresponding second node. In one aspect of the present invention, the value is a value according to a geometric progression or a geometric progression assigned to each second node.
For example, according to the arithmetic sequence, when the 10 second nodes are sequentially assigned, the output values thereof are 0, 0, 0.1, 0.2,
It is set as 0.3, ..., 0.7, 0.8, 0.9, 1.0. The combined output value is combined based on the output value (matching degree) of the selected second node and the defined unique output value corresponding to the node. For example, with the degree of matching of the selected second node as a weight, the defined unique output values are weighted and averaged.

In the learning for this network type information processing system, a continuous value is given as output teaching data, one or a plurality of second nodes are selected according to the teaching data, and the value of the teaching data is distributed in a predetermined manner. According to
Learning processing (that is, change of the filter function which is a selection type function, etc.) is performed for each second node using the distributed value as teaching data.

According to an aspect of the present invention, the allocation method allocates the discrete values of the second nodes numerically located on both sides of the input continuous value by linear interpolation. is there.

The network type information processing system learned as described above describes the relationship between multi-continuous value input and multi-continuous value output. In the present specification, the term “continuous value” of input or output refers to a case where a multivalued non-discrete continuous quantity, for example, an analog quantity such as speed or flow rate in a control system is detected by a sensor and converted into a digital value. It is possible to take multiple values that can be regarded as continuous quantities such as. Therefore, according to the present invention, a plurality of second nodes having unique output values (discrete values) are set, and the unique output values are combined on the basis of the degree of matching. It will be possible.

Further, according to the present invention, in the above-mentioned network type information processing system, at least one second network type information processing system (FIG. 17) for outputting an error of the output of the network type information processing system (NetB in FIG. 17). It is possible to adopt a mode in which Net B) of 17 is added. In this mode, a plurality of network type processing systems are used for one output, and the error between the output of one network type information system and the taught output is
It is possible to improve accuracy by learning as the output of the network type information system.

The present invention also includes a plurality of first nodes,
A network type information processing system including a plurality of second nodes and a directional link having a non-linear selective function characteristic that connects the first node and the second node (FIG. 18).
181) of the above, has an output value defined for each second node as a non-linear selection type function, and is represented by a continuous value reflecting the output value defined above and the output value of the second node. A second network type information processing system (182 in FIG. 18) having a function of obtaining one or more combined output values can be added. In the learning, the second network type processing system is learned, and the second node to be learned in the first network type processing system is selected according to the learning result. Then, for the selected second node, generation / change of the pattern set is performed. According to the specific example, when input / output data is input during learning, teaching is performed on the pattern table having the teaching output data as input information. If there is a pattern set showing a high degree of matching, a network type information processing system (18 in FIG. 18) paired with the pattern set is provided.
Select the pattern table of 1) and teach. In the aspect of the present invention, since the network type information processing system is used as the output synthesizing means, it is possible to output a plurality of data.
Further, improvement of output accuracy can be expected.

[0013]

[Embodiment] Learning of a network type information processing device basically has an unknown system M with multiple inputs and one output, and assuming that this input / output is observable, a characteristic equivalent to the system M from this input / output. To obtain system B having In the case where the unknown system is a multi-input-multi-output system, it can be dealt with by preparing a plurality of the systems B corresponding to multi-input-1 output. Embodiment 1 FIG. 1 is a diagram showing a schematic configuration of a network type information processing apparatus according to Embodiment 1 of the present invention. This shows the case of 4 inputs and 1 output. If the output becomes 2, two similar networks are configured in parallel. As shown in FIG. 1, this network type information processing apparatus includes a plurality of nodes 11, 12, 13, 14 of a first layer, a plurality of nodes 105, 106, 107, 108 of a second layer, and a plurality of nodes of a third layer. A plurality of nodes 109, 110, 111 are connected by directional links to form a network. An output combining unit 112 that combines the outputs of the third layer is connected to the output of the third layer. Each node in the first layer distributes multiple continuous inputs I _{0 to} I ₃ into directional links to each node in the second layer. Each input is a multi-continuous input. In the present specification, “multi-continuous” of input or output is a multi-valued non-discrete continuous quantity as described above, for example, an analog quantity such as speed or flow rate in a control system is detected by a sensor and converted into a digital value. It can be multi-valued as in the case. First
The directional links connecting the layers and the nodes of the second layer are composed of filter functions (membership functions) MF _{00 to} MF ₃₃ and weights w _{00 to} w ₃₃ .

The nodes 105 to 108 of the second layer respectively output the output values V _MF00 to _VMF00 of the filter function of each directional link connected thereto.
The weighted average or the weighted geometric average is calculated using the V _MF33 and the weights w ₀₀ to w ₃₃ corresponding thereto. For example, as for the output value V _PS0 of the node 105, the weighted average and the weighted geometric average are expressed by the following equations, respectively. In the case of the weighted _{averaging, V PS0 = {(V MF00} ) × w 00 + ··· + (V MF30) × w
_{30} / (w 00 + w} 01 + ·· + w 03) when the weighted geometric _{mean, V PS0 = {(V MF00} ) w00 × ··· × (V MF30) w30}
^{1 / (w00 + w01 + ... + w03)}

Each of the nodes 109 to 111 in the third layer performs a Max operation that leaves the maximum value. For example, the output value V _PT0 of the node 109 is expressed by the following equation. _{V PT0 = Max {V PS0,} V PS1}

The output synthesizing unit 112 synthesizes the outputs V _{PT0 to} V _PT2 from the nodes of the third layer to obtain continuous output values. Different set output values are assigned to the outputs from the layer 3 nodes in order to obtain continuous values. The method of allocating the set output value includes the following. a) When the output range of the setting system M based on the arithmetic progression is 0.0 to 1.0, 11 nodes of the third layer are set, and each set output value is set as follows. O ₀ 0.0 O ₁ 0.1 O ₂ 0.2 ··· · O ₅ 0.2 ··· · · O ₁₀ 1.0 b) Setting by geometric progression O ₀ 0.5 O ₁ 0.25 O ₂ 0.125 ··· · O ₅ (0.5) ⁶ ··· · O ₁₀ (0.5) ¹¹

FIG. 2 is a table representation of the filter function MF and weight w on the network link of the network type information processing apparatus shown in FIG. In the figure, trapezoidal waveforms 201 to 216 represent filter functions MF, and the heights of the trapezoidal waveforms are different because they are multiplied by the weight w.
The pattern sets PS _{0 to} PS ₃ respectively correspond to the nodes 105 to 108 of the second layer, and the pattern tables PT _{0 to} P T.
T ₂ corresponds to the nodes 109 to 111 of the output layer (third layer).

FIG. 3 is a block diagram showing an example of the configuration of the portion for performing the calculation shown in FIG. The node illustrated in FIG. 3 is a filter function calculation unit 31 to 3 that performs a calculation of a filter function that selectively passes input information in each directional link.
4, an adder unit 35 that performs addition and averaging processing by weighting the outputs of the filter function operation units 31 to 34, and the adder unit 35.
Of the threshold value calculation unit 36 for performing a threshold calculation on the output of the.

As shown in FIG. 4, the filter function is a member having a trapezoidal shape of unequal sides in a graph in which the horizontal axis represents the value (input value) of the input information given to the input node and the vertical axis represents the degree of matching. A ship function (fuzzy membership function) is used. The membership function is the center value c which is the center value of the input that obtains the maximum matching score, and the left variance value v that represents the allowable range of the input value that obtains the matching score v as the width from the center value c to the left and right. _{It is described by l} and the right variance value v _r , and the ambiguity a which shows the allowable range of the input value at which the maximum matching degree is obtained by the width from the central value c. The filter function calculation units 31 to 34 are for calculating the membership function, that is, converting the input value s into the matching degree v. An equation for converting the input value s into the matching degree v can be expressed as follows. v = 0.0 {s ≦ (c−v _l ) or (c + v _r ) ≦ s} v = 1.0 {(c−a) ≦ s and s ≦ (c + a)} v = ((v _l −a ) - (c-a-s )) / (v-a) {(c-v l) ≦ s and s ≦ (c-a)} v = ((v r -a) - (s-c-a )) / (v-a) {(c + a) ≦ s and _{s ≦ (c + v r)} }

The adder 35 includes filter function calculators 31 to 31.
An arithmetic mean calculation is performed in order to integrate the outputs calculated by 34. In the present embodiment obtains an overall matching degree by comprehensively a value obtained by multiplying the weight to the matching degree v _1j to v _ij of input information I ₀ ~I _i by each membership function. Although various methods of calculating the total degree of coincidence can be considered as described above, the weighted addition average is used in the present embodiment.

A learning operation by the network type information processing apparatus having the above-mentioned configuration will be described. When input / output data is input to the system, first, the output data is analyzed, a fixed number (1 to N) of output nodes are selected, and at the same time, the strength of teaching for each output node (= pattern table) is determined. . Next, in the pattern table,
The pattern changing process is executed using the input data and the teaching strength determined as described above as the teacher data. Figure 5
[Fig. 6] is a process flow diagram showing an outline of an operation at the time of learning. Using the network type information processing apparatus shown in FIGS. 1 and 2, output values O _PT0 , O _PT1 , and O _PT2 defined in the pattern tables PT ₀ , PT ₁ , and PT ₂ are respectively 0.0 , 1.0, and Assume 2.0.

(Step S51) Inputs I _{0 to} I ₃ are input to the nodes 101 to 104 of the first layer, respectively, and a teaching output value O _t (for example, 0.7) is input.

(Step S52) Which pattern table PT is selected as a teaching target is selected. For example, when 0.7 is input as the teaching output value O _t as described above, since 0.7 exists in the section (0.0, 1.0), the pattern tables PT ₀ and PT ₁ are set. select.

(Step S53) The teaching intensity is determined based on the output values O _PT0 and O _PT1 defined in the selected pattern table and the input teaching output value O _t . For example, the teaching strength for the pattern table PT _{0 is set} to 0.3 and the teaching strength for the pattern table PT _{1 is set} to 0.7 by linear distribution.

(Step S54) Using the input data I _{0 to} I ₃ , the output values (coincidence) of the selected pattern tables PT ₀ and PT ₁ are calculated. That is, the node 10 of FIG.
By the operation on the 5,106, their calculated output value V _PS0, V _PS1, further operation of the node 109 from their _{V PS0, V PS1 V PT0 =} Max {V PS0,
The output value V _PT0 is _obtained from V _PS1 }. Also, the node 10
Searching for the output value V _PS2 by the operation on 7, obtaining the output value V _PT1 by calculating V _PT1 = V _PS2 at node 110.

(Step S55) With respect to the output values of the pattern tables PT ₀ and PT ₁ calculated in step S54,
The following processing is performed one by one. In this example, processing is performed first for V _PT0 and then for V _PT1 .

[0027] To evaluate (step S56) the output value V _PT0, is compared with a threshold value V _ext for the output value V _PT0 and the pattern set extension.

(Step S57) When the output value V _PT0 is smaller than the threshold value Vext (Vext> V _PT0 ) as a result of the comparison in step S56, the pattern set PS having the same matching degree as the output value V _PT0.
(A plurality of pattern sets PS of the pattern table PT ₀
The learning process is performed on the pattern set PS which has the maximum matching degree among 0, PS1 ...

(Step S58) As a result of the comparison in step S56, the output value V _PT0 is larger than the threshold value V _ext (Vext
<VPT0) time, add the pattern set new patterns table PT _0, performing the learning process for the pattern set. In the case of this example, the pattern set PS ₄ is generated and added to the pattern table PT ₀ .

Next, for the output value V _PT1 , step S
The processing of 56 to S58 is performed. In this way, when all the processes of the selected teaching target pattern table are completed, the learning process is completed.

The learning process for the selected pattern set or the generated pattern set in step S57 or S58 will be described in more detail.
As an example of this learning processing, the method of changing the filter function disclosed in the specification of Japanese Patent Application No. 04-175056 “Learning system for network type information processing apparatus” previously filed by the present applicant is used. FIG. 22 is a functional block diagram of the learning system. As learning data for learning the selected or generated pattern set, FIG.
Input data and teacher data are input to the network-type information processing device 221 shown in FIG. 2 to change the filter function of each dimension forming the pattern set. FIG. 6 shows a processing flow for changing the filter function. The data input as the teaching signal is stored in the history buffer 2271 (step S61). The history buffer 2271 holds data of each dimension input to one pattern set, that is, a rule, and also has a capacity of storing a history of a predetermined M number of inputs. A history buffer is prepared corresponding to each rule, and a counter (not shown) for counting the data input to each history buffer is provided. The counter that counts the number of input data of the history buffer 2271 increments the value of the counter by 1 each time it is input (step S
62). Next, it is determined whether or not the value of the counter is 1 (step S63). If it is 1, the filter function updating unit 2
The parameters of the filter function are updated by 24 as follows (step S64). That is, the update method is switched according to the number of history data to perform the update. N: number of history data (number of sampling data) M: history buffer size X ₀ : initial input data X ₁ : minimum value of history data X ₂ : maximum value of history data R ₁ : minimum value of normal space R ₂ :
Normal space maximum value C: center value A: ambiguity V: variance value.

A) In the case of N = 1 C = X ₀ A = 0 V = Set as information in learning processing FIG. 7 shows an example of the set filter function.
The update method designating unit 228 determines whether the result of the counter value determination is
If it is 2 or more, it is determined whether or not the value of the counter exceeds a predetermined number M (step S65). If the result of the determination does not exceed M, the filter function is updated by the following method (step S66).

B) When 2 <N <M C = (X ₁ + X ₂ ) / 2 A = (X ₂ −X ₁ ) / 2 V = A + 2A / N FIG. 8 shows an example of the set membership function. It is shown. In the cases of a) and b) above, a small number of data that do not have statistical significance in the first place are targeted, and the learning method must rely on an intuitive non-intuitive method rather than statistically. I don't get it. This is an example of a method that can judge that data capture and evaluation are reliable by reflecting the characteristics of fuzzy.

If the result of determination in step S65 is that the value of the counter exceeds the number of history buffers, it is changed by the statistical processing as described below (step S67). As the method of setting by statistical processing, the method disclosed in the specification of Japanese Patent Application No. 3-310082 “Network type information processing system” filed by the present applicant can be used. c) Changes due to statistical processing During a certain observation period, the input data that matches the relevant pattern is aggregated and used as the population. FIG. 10 is an example of a graph showing an example of a data generation distribution in which the horizontal axis represents the quantization level Z of input data and the vertical axis represents the number G of data generations for each level. Note that G _c is a cutoff level for removing noise components and the like of the input data. When the number of elements included in the population reaches a certain number, the filter function updating unit 224 performs a changing operation of the membership function which is a filter function. A membership function is derived from the population according to the following procedure.

FIG. 9 is a flow chart of the membership function parameter extraction processing. It is determined whether or not the cutoff data G _c is set as the output data designation (step 91). If the cutoff level G _c is set, the distribution data is manipulated by obtaining a value obtained by subtracting the cutoff level for each quantization level Z _i (step 92). That is, Z _i −G _c is obtained and set as a new Z _i . FIG. 11 shows the result of processing the input data distribution of FIG. 10 at the cutoff level G _c . The arithmetic processing for truncating the data below the cutoff level is performed until all the quantization levels of Z are completed. for that reason,
Each time the calculation of each quantization level is completed, it is judged whether or not all the quantization levels are completed (step 9).
3). When all the quantization levels have been completed, the process proceeds to step 94. Also, if the cutoff level is not set by the determination in step 91, step 9
Go to 4.

Average value of distribution data Normalized coordinate value Z_mSeeking
(Step 94). Z_m= Σ (Z_i× G_i) / ΣG_i However, Σ represents the total sum from i = 0 to i = n
To do. From the averaged normalized coordinate value Zm, the minus side and the plus side
Standard deviation value S independently_l, S_r(Step 9
5). Average value normalized coordinate value Z obtained in step 94_mOh
And the standard deviation value S on the negative side obtained in step 95_l，
Positive standard deviation S_rNormalized coordinate values based on
Is calculated (step 96). That is, the average value normalized coordinate value
Z_mAs shown in Fig. 12, centering on the
Difference value S_lNormalized coordinate value Z which is 1 times (Note *)_L1,minus
Side standard deviation value S_lNormalized coordinate value Z of 3 times (*)_L2,
Positive standard deviation S_rNormalized coordinate value of 1 times (*)
Z_R1, Plus standard deviation S_rNormalization of 3 times (*)
Coordinate value Z_R2, (Note *: This value may change depending on conditions.
Each).

The normalized center value C _s , the normalized ambiguity V _as , and the normalized variance values V _ls , V _{rs as} shown in FIG. 13 are obtained by the following equations (step 97). Normalized center value C _s = (Z _L1 + Z _R1 ) / 2 Normalized ambiguity V _as = (Z _R1- Z _L1 ) / 2 Normalized left variance value V _ls = C _s- Z _L2 Normalized right variance value V _rs = Z _R2- C _s Next, the normalized center value C _s , the normalized ambiguity V _as , the normalized left variance value V _ls , and the normalized right variance value V _rs are denormalized to obtain the central value C, Ambiguity V _a , left variance V _l , right variance V _r
(Step 98). A new membership function (after one-time learning) is generated by using the original membership function before the change by learning and the membership function generated by the processing flow shown in FIG. 9 described above.

FIG. 14 shows the membership table in this embodiment.
It is a figure which shows the method (learning function) of changing the loop function. same
In the figure, the original (current) membership function is the point P
₁, P₂, P₃, P_FourIndicated by a straight line group (thick line) connecting
Obtained from sampling for a certain period of time calculated from FIG.
The membership function based on the data distribution is point P₁ ^'，
P₂ ^', P₃ ^', P_Four ^'Indicated by a group of straight lines (broken line) connecting
Newly generated methods based on these membership functions
The intersection function is point P₁ ^", P₂ ^", P₃ ^", P_Four ^"Straight tie
It is indicated by a group of lines (thin lines). P coordinates of each 4 points
(S, v), P^'(S^', V^'), P^"(S^", V^")
It v^"= V^'= V s^"= (1.0-g) xs + gxs^' However, 0.0 ≦ g ≦ 1.0 The gain value g at each of the four points can be set independently.

The result of changing the membership function varies depending on the same observation data depending on the setting of the gain value at each point. FIG. 15 shows how the change result changes depending on how to determine the gain. FIG. 15A shows
The current membership function (thin solid line) and the membership function based on the distribution obtained from sampling for a certain period (dashed line) are shown. FIG. 11B shows a membership function (thick solid line) generated when the gain value g at each point is set to 0.5. Further, FIG. 7C shows the case where the gain value g of the point P ₁ is g = 0 and the gain value of the other points is g = 0.5, and FIG. 7D is the ambiguity (P ₂ and P The distance between ₃ ) is not changed and the gain value g in the base expanding direction is 1.0, and the gain value in the base contracting direction is g = 0. The gain value can be set and the directionality of learning can be changed according to the characteristics of the sensor that is the input signal source, the intention of learning, and the like.

Here, regarding the selection of the teaching output node,
A description will be given including various modifications. a) Linear interpolation The linear interpolation method is an effective method in the case where the set output value is set by the arithmetic sequence (the first embodiment corresponds to this case). In the case of the form in which the set output value is set by the geometric progression, it cannot be used in principle. The linear interpolation method includes the following methods. It is assumed that a specified value, for example, 0.13 is input as an output signal in the case of setting by the arithmetic sequence. In this case, the teaching is carried out for _two output values O ₁ and O ₂ whose set output is near the specified value 0.13. Further, the teaching intensity for each output node is obtained from the position of the teacher signal in this section 0.1 to 0.2. In this example, 0.7 is used for O ₁ by using linear interpolation.
And a strength of 0.3 for O ₂ .

B) The teaching is performed by selecting a specific output node based on the teacher signal of the selective teaching output. Even when a unique output value is set by the arithmetic sequence, the teaching is performed by selecting the output node whose set output value is the closest to the teacher signal without using the linear interpolation of a) above. You can However, this is not very practical. This selective teaching method is effective when the output value is set by a geometric progression. For example, assume that 0.13 is input as an output signal. An output node smaller than the teacher signal and having the largest set output value is selected. In this case, the output node is O ₂ (0.12
5). Next, this difference is calculated (0.13-0.1
25 = 0.005), similarly, an output node smaller than this difference and having the largest set output value is selected. In this case, it is O ₇ (0.0039). The same processing is repeated until the output nodes smaller than the difference do not exist. The output node selected in this way is taught with a strength of 1.0.

C) Output distribution function As a generalization of the above-mentioned linear interpolation, consider a function that allocates data between output sections to output nodes. Out here
Data between force categories is, for example, 0.13
Output range, such as a value between two output values 0.1 and 0.2
Means the data in minutes. This distribution function may be defined for each output node, but normally a common one is used for each output node. This distribution function is given for each PS based on the absolute value of the difference between the teaching data value and the output node value .
Returns a distribution value that indicates the strength of the taught instruction . Due to the nature of the distribution function, the sum of distribution values is always 1 for one piece of data.
Designed to be. For functions of general, if wider function output domain, the allocation is that there is more than two nodes.

Next, the operation of the network type information processing apparatus after the learning is completed at the time of inference will be described. The input data is given to the input of the network type information processing apparatus, and the output value of the output node, that is, the degree of condition part matching is obtained. The unique set output value defined in each output node is weighted and averaged by the output value ( conditional part matching degree) of the output node and combined to obtain the output of the system B. FIG. 16 is a flowchart showing an operation example at the time of inference in this embodiment. (Step S161) Inputs I _{0 to} I ₃ are input to the nodes 101 to 10 of the first layer, respectively.
Enter in 4. (Step S162) The output values (coincidence) of PT ₀ , PT ₁ and PT ₂ are calculated using the input data. In this example, V _PT0 = Max
{V _PSO , V _PS1 } = 0.1, V _PT1 = V _PS2
= 0.6, V _PT2 = V _PS3 = 0.9. (Step S163) The section in which the output exists is determined. For example, the section is determined by selecting the combination having the highest total matching score of two adjacent pattern tables.

(Step S164) The output values are combined based on the matching degree. For example, each pattern table P
The output values defined in T are weighted and averaged using the degree of matching as a weight to synthesize the output values. In the case of the example, the output value O ₀ is O ₀ = 1.0 × 0.6 + 2.0 × 0.9 / (0.
6 + 0.9). (Step S165) The obtained output is output.

In the above description, as a method of output composition at the time of inference, a combination having a high total matching degree between two adjacent pattern tables PT is selected. Can be adopted. a) Method of synthesizing set output values of all output nodes In this case, the set output values are weighted averaged with the coincidence of the output nodes as a weight.

B) Method of selecting a specific output node and synthesizing the set output value. For this, for example, only output nodes having a certain degree of matching are selected. Output nodes having the highest degree of matching N are selected. and so on. Note that if the set output value is set by a geometric progression, logical inconsistency may occur, so
The principle method cannot be adopted. Further, there is the following method as a modification of the above method. Select the output node with the maximum degree of matching and the rule of 2n points in the vicinity (total 2n + 1) of rules. The set of nodes with the maximum sum of the degrees of matching of the nodes of the 2n points in the vicinity is used.

[0047] c) when Choosing inference output node for outputting the synthesis corresponding to the learning time allocation function combines the output value with an output node with the time of distribution. In this case, as the method of determining the output node to be used, it is natural to select the one having the maximum sum of the matching degrees of the output nodes to be used. Alternatively, it is also possible to simply pay attention to the output node having the highest degree of matching and adopt the distribution node having the output node as the center. The composite output value is calculated by weighted averaging the output values of the output nodes.

(Modification of First Embodiment) The first embodiment described above is
This is an example of the most compact embodiment as a processing system that realizes multi-input-continuous one-output. In order to further improve accuracy and the like, parallelization and hierarchization described below can be performed.

(Parallelization) As a form corresponding to multiple continuous value input-one continuous value output, a plurality of systems (hereinafter referred to as pattern groups) similar to that shown in FIG. 1 are prepared. (System B
₀ , ..., System B _N ) In each pattern group, learning characteristics (size of history buffer, learning gain value of filter function, gain value of weight change, etc.) are changed and learning processing is performed independently. To perform. The output as the system B is obtained by combining the outputs of the respective pattern groups.

(Hierarchization) As a form corresponding to multi-input-continuous one output, a plurality of systems (hereinafter referred to as pattern groups) similar to those shown in FIG. 1 are prepared. (System B0, ..., System BN) In system B0, the input and output of system M are used for learning as they are, as in the case shown in FIG. 1, but in systems other than system B0 (pattern groups),
The output information used for learning is not the output of the system M. The system B1 is configured to learn the output of the system B0 and the error of the system B. Similarly, the system B2 is the combined output of the systems B0 and B1 and the system M2.
Learn the error between and. As the learning stage, firstly, the system B0
If only the error with the system M is less than a certain value,
When the reduction of the error is stopped, the learning of the system B0 is stopped,
Save the characteristics of system B0. Next, the learning of the system B1 is started.
Similarly, the systems Bi are fixed one after another. In other words, "system
“B” was trained by giving input / output values to the unknown system “system M”
It is a known system.

FIG. 17 shows a network configuration example in the case of hierarchization, which is an example of a case of two inputs and one output.
The upper half network NetA has the same configuration and the same input / output as the network type information processing apparatus of FIG. The network NetB in the lower half has the same configuration as the network type information processing apparatus of FIG. 1, and the input data itself is the same as the network NetA, but the output is different. The network NetA teaches the output as it is, whereas the network NetB teaches the difference between the teaching output and the output of the network NetA. In short, the network NetB has a form in which the error between the input and NetA is learned. At the time of inference, the output O _{PG1 of the} network NetB is subtracted from the output O _PG0 of the network NetA. The basic operation mode during learning is as follows. First, learning is performed only by the network NetA. In the learning of the network NetA, when the error becomes small to some extent, the learning of the network NetA is stopped. The network NetB is learned by the error between the output of the network NetA and the teaching output.

Embodiment 2 FIG. 18 is a diagram showing the configuration of a network type information processing apparatus according to Embodiment 2 in the form of a pattern table. The second embodiment is composed of two networks 181 and 182, and the network 181 on the left side has the same configuration as the network of the first embodiment shown in FIGS. Example 1
The difference from is as follows. That is, in Example 1,
Output value OPT0 of each pattern table PT0-PT2
While OPT2 was fixed, in Example 2,
The difference is that it has already been described by another network 182 and is variable.

The operation at the time of learning of the second embodiment configured as described above will be described with reference to FIGS. 19 and 20. The output side network 182 is learned, and then the input side network 181 is learned. (Step S191) Inputs {I ₀ , ..., I ₃ } and teaching output values {O _t0 , O _t1 } are input.

(Step S192) Teaching output value {O _t0 ,
O _t1 }, the output value (matching degree) V of the pattern set PS′0, PS′1, PS′2 of the network 182 is used.
_PS'0 , _VPS'1 and _VPS'2 are calculated.

(Step S193) The pattern set PS having the highest matching score is selected from the calculated matching scores. In this example, assume that V _PS'0 was the maximum.

(Step S194) The matching degree V _PS'0 of the selected pattern set PS ' ₀ is evaluated. That is, the matching degree V _PS'0 is compared with a threshold value ((Vext) for expanding the selected pattern set PS, and Vext ≧ VPS '
If 0, the process proceeds to step S195, and if Vext <VPS'0, the process proceeds to step S197.

(Step S195) If Vext ≧ VPS′0 in the evaluation of step S194, the learning process is performed on the pattern set PS ′ ₀ having the maximum degree of matching. This learning process is the same as the process of changing the filter function of each dimension that forms the pattern set described in detail in the first embodiment with reference to FIGS. 6 to 15.

(Step S196) The teaching object pattern table PT of the input side network 181 is set to PT ₀ (the pattern table PT corresponding to the selected new pattern set PS ′ _{0 of the} output side network 182).
Proceed to step S199.

(Step S197) When Vext <VPS'0 in the evaluation of step S194,
A new pattern set PS for the output side network 182
To add. In this example generates a PS _'3. Then, the learning process is performed on the pattern set PS ′ ₃ . This learning process also performs the same process as the process of changing the filter function of each dimension forming the pattern set described in detail in the first embodiment with reference to FIGS. 6 to 15.

(Step S198) Set the teaching target pattern table PT of the network 181 on the input side to PT ₀ (P
And PT) which is connected to the S _'0. Proceed to step S199. (Step S199) The output side learning process is terminated, and the input side network 181 learning process is started.

(Step S200) Using the input data, the output value of the learning target PT ₀ determined in step S198 (the degree of matching is calculated).

(Step S201) The matching degree V _PT0 of the learning target pattern table is evaluated. That is, the matching degree V _PT0 is set to the threshold value (Ve) for expanding the pattern set PS.
xt). If Vext ≧ VPT0, go to step S202. If Vext <VPT0, go to step S20.
Go to 3.

(Step S202) If Vext ≧ VPS′0 in the evaluation of step S201, V
The learning process is performed on the pattern set PS ₀ having the same degree of coincidence as PT0 (PS having the maximum degree of coincidence). This learning process also performs the same process as the process of changing the filter function of each dimension forming the pattern set described in detail in the first embodiment with reference to FIGS. 6 to 15. Step S204
Go to.

(Step S203) When Vext <VPS'0 in the evaluation of step S194,
The pattern set PS is newly added to the pattern table PT0. In this case, the pattern set PS ₄ is generated and added to the pattern table PT ₀ . A learning process is performed on the pattern set PS ₄ . This learning process also performs the same process as the process of changing the filter function of each dimension forming the pattern set described in detail in the first embodiment with reference to FIGS. 6 to 15. (Step S204) The learning ends.

Next, the operation of the network type information processing apparatus after inference at the time of inference will be described. Figure 21
FIG. 7 is a flow chart showing an operation example at the time of inference in this embodiment. (Step S211) Inputs I _{0 to} I ₃ are input to the nodes of the first layer, respectively. (Step S212) PT ₀ , P using the input data
The output values (coincidence) of T ₁ and PT ₂ are calculated. In this example, V _PT0 = Max {V _PSO , V _PS1 } = 0.1, V _PT1 =
Assume that V _PS2 = 0.6 and V _PT2 = V _PS3 = 0.9. (Step S213) A pattern table is selected according to the calculated matching degrees. For example, 2 in descending order of match
Select from the two pattern tables. In this case,
The pattern tables selected are PT ₁ and PT ₂ .

(Step S214) The central value of the filter function of the pattern set connected to the selected pattern tables PT ₁ and PT ₂ is set as the temporary output value. For example, its value for the output O ₀ is {O _0PS'1 , _OPS'2 } = {1.0,
2.0}.

(Step 215) The output values are combined based on the matching degree. For example, each pattern table PT
The output values defined in 1 above are weighted and averaged using the degree of matching as a weight to synthesize the output values. In the case of the example, the output value O
₀ is O ₀ = 1.0 × 0.6 + 2.0 × 0.9 / (0.6
+0.9) = 1.6. (Step 216) The obtained output O ₀ is output. The processes of S214 to S216 are similarly performed for the output O ₁ .

[0068]

According to the present invention, it is possible to realize a network type information processing apparatus which describes the relationship between multiple continuous value inputs and continuous value outputs. In addition, it can operate at high speed and converge learning from a small number of input / output pairs.

[Brief description of drawings]

FIG. 1 is a diagram showing a schematic configuration of a network type information processing apparatus according to a first embodiment of the present invention.

FIG. 2 is a diagram showing a pattern table in which the network type information processing apparatus shown in FIG. 1 is represented in a tabular form by a filter function MF and a weight w on a network link.

FIG. 3 is a block diagram showing a configuration example of a portion for performing calculation in FIG.

FIG. 4 is a diagram for explaining a fuzzy membership function (filter function).

FIG. 5 is a processing flow chart showing an outline of operations during learning.

FIG. 6 is a flowchart showing a filter function changing process.

FIG. 7 is a diagram showing a filter function set when the number of history data is 1.

FIG. 8 is a diagram showing a filter function set when the number of history data is 2 or more and less than M.

FIG. 9 is a flow chart of parameter extraction processing of membership function.

FIG. 10 is a diagram showing an example of a graph showing an example of a data generation distribution in which the horizontal axis represents the quantization level Z of input data and the vertical axis represents the number of times G data is generated for each level.

FIG. 11 is a cutoff G of the data generation distribution of FIG.
The figure which shows the data distribution after cutting off by c

FIG. 12 is a diagram for explaining calculation of each normalized coordinate value.

13 is a diagram showing a membership function obtained from the data distribution of FIG.

FIG. 14 is a diagram for explaining a basic method (learning function) for changing a membership function.

FIG. 15 shows how the result of changing the membership function changes depending on how to determine the gain of the learning function. (A) shows the current membership function (thin solid line) and sampling for a certain period. Shows a membership function (dashed line) based on the distribution obtained from, (b) shows a membership function (thick solid line) generated when the gain value g at each point is 0.5, and (c) ) Shows the case where the gain value of the point P1 is g = 0 and the gain value of the other points is g = 0.5, and (d) is the base extension without changing the ambiguity (distance between P2 and P3). Direction gain value g = 1.0 and bottom side reduction direction gain value g = 0 are shown.

FIG. 16 is a diagram showing an example of an operation flow during inference according to the first embodiment.

FIG. 17 is a diagram showing an example of a network configuration in the case of hierarchization, which is an example of the case of two inputs and one output.

FIG. 18 is a diagram showing a configuration of a second embodiment expressed as a pattern table.

FIG. 19 is a processing flow chart showing an operation at the time of learning in the second embodiment.

FIG. 20 is a processing flow chart showing an operation at the time of learning in the second embodiment (continuation of FIG. 19).

FIG. 21 is a diagram showing an example of the flow of operations during inference according to the second embodiment.

FIG. 22 is a functional block diagram of a learning system of a network-type information processing system of Japanese Patent Application No. 4-175056.

[Explanation of symbols]

11, 12, 13, 14 ... First layer nodes (input nodes), 105, 106, 107, 108 ... Second layer nodes, 109, 110, 111 ... Third layer nodes, 112
... Output synthesizer.

─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Kazuhiko Hamaya, 1-15-8, Jinnan, Shibuya-ku, Tokyo Ken Naka Building, 4th floor, within the Add-In Research Laboratory (72) Inventor Masaaki Watanabe 1-15, Jinnan, Shibuya-ku, Tokyo No. 8 Kannaka Building, 4th floor, Add-in Research Institute, Inc. (72) Inventor Yoichi Ueishi 1-15-8, Jinnan, Shibuya-ku, Tokyo, 4th Floor, Kannaka Building, Add-in Research Institute, Inc. (56) References 3-271806 (JP, A) JP 2-100757 (JP, A) JP 2-292602 (JP, A) JP 4-1824 (JP, A) Nakano Kaoru, et al. Basics, Japan, Corona Co., Ltd., April 5, 1990, first edition, pp. 44-47, ISBN: 4-339-02276-4 Ken Nakamura et al., "Characteristic Vector Search by Fuzzy Arithmetic", Proc. Of the 5th Fuzzy System Symposium, Japan, International Fuzzy System Society of Japan, 1989. June 2, pp. 449-454 Ken Nakamura et al., "Knowledge Compile Pattern Pattern Inference System," Proceedings of the 1989 National Congress of Artificial Intelligence (3rd), Japan, Japan Society for Artificial Intelligence, 1989, pp ． 295-298 Asahi Kawamura et al., "Neuro-Fuzzy Fusion System", Information Processing Society of Japan, Research Report, Japan, Information Processing Society of Japan, May 10, 1991, Vol. 91, No. 36 (91-MIC-66), pp. 1-8 (58) Fields investigated (Int.Cl. ⁷ , DB name) G06N 1/00-7/08 G06G 7/60 G05B 13/00-13/04 JST file (JOIS) CSDB (Japan Patent Office )

Claims (7)

(57) [Claims] 1. A plurality of first nodes, square to the first node
A plurality of second nodes coupled via directional links,
A net having output combining means coupled to two nodes
A work-type information processing system, wherein the directional link filters a non-linear selective function.
A non-linear selection type function for input information.
A second filter function calculating unit for calculating a function value of the number of the directional links.
The output from the output function
Non-line of directional links connected to the second node of the
Matching input information for a set of shape-selective functions
An arithmetic unit for calculating a degree, the output combining unit outputs the plurality of second nodes if
1 or more according to a preset selection rule based on the degree of severity
Select the second node above and select the selected second node
Corresponds to each second node based on the match level output by
And combine the unique output values that are predefined discrete values
A network-type information processing system characterized in that a synthetic output that is a continuous value is obtained . 2. The network according to claim 1, wherein the predetermined unique output value is assigned to each of the second nodes according to a geometric progression or geometric progression. Type information processing system. 3. A plurality of first nodes, square to the first node
A plurality of second nodes coupled via directional links ,
A net having output combining means coupled to two nodes
A work-type information processing system, wherein the directional link filters a non-linear selective function.
A non-linear selection type function for input information.
A second filter function calculating unit for calculating a function value of the number of the directional links.
The output from the output function
Non-line of directional links connected to the second node of the
The degree of matching of input information to a set of shape-selective functions
The output synthesizing means has a calculation unit for outputting the output of the plurality of second nodes.
1 or more according to a preset selection rule based on the degree of severity
Select the second node above and select the selected second node
Corresponds to each second node based on the match level output by
Pre-defined according to a geometric progression or geometric progression
Synthesize continuous output by combining discrete output values
In the network type information processing system to obtain output, given continuous values as output instruction data, selecting one or more second node in response to the output teaching data, the value of the output teaching data, the selection Corresponding to the node
A network-type information processing system, comprising: learning means for allocating based on the unique output value, and performing learning processing for each second node having the allocated value as teaching data. 4. The distribution has two input continuous values.
Two separations of the second node such that the values are between the discrete values of
Select Chichi, network type information processing shea <br/> stearyl arm according to claim 3, characterized in that the distribution by linear interpolation. 5. A plurality of first nodes, square to the first node
A plurality of second nodes coupled via directional links ,
A net having output combining means coupled to two nodes
A work-type information processing system, wherein the directional link filters a non-linear selective function.
A non-linear selection type function for input information.
A second filter function calculating unit for calculating a function value of the number of the directional links.
The output from the output function
First directional links coupled to a second node of the
Input information for a set of nonlinear selective functions of
The output synthesizing means has a non-linear selection type output value defined for the second node.
The degree of matching that is output as a function from a plurality of second nodes
1 or more according to preset selection rules based on the size of
Select the second node and select the second node
Based on the output degree of match, a constant is set for each second node.
The values of the central values of the defined nonlinear selective functions are synthesized and combined.
A network-type information processing system characterized by obtaining a synthetic output which is a continuous value . 6. A plurality of first nodes and a method for the first nodes
A plurality of second nodes coupled via directional links,
It has an output synthesizing means connected to two nodes and a learning means.
A network-type information processing system for controlling the non-linear selective function
A non-linear selection type function for input information.
A second filter function calculating unit for calculating a function value of the number of the directional links.
The output from the filter function operation unit is combined and the second node
Nonlinear selection of a plurality of first directional links coupled to the
Calculate the degree of matching of input information to a set of alternative functions
The output synthesizing means has a third node connected to the second node,
The first with a non-linear selective function connected to the third node
Connected to a second directional link and a second directional link
It has a plurality of fourth nodes and, at the time of inference, a plurality of second nodes.
Preset selection based on the degree of matching
Select one or more second nodes according to the selection rule and select them
Based on the matching degree output by the generated second node,
Join to the third node corresponding to the selected second node
A second directional link coupled to the coupled second directional link.
The central values of the non-linear selective functions of
A learning output is obtained by using the input value and the teaching output value as a teaching signal during learning.
First, the teaching output value is given to the fourth node in the output synthesizing means.
Given, the second person tropism phosphorus which is connected to the third node
The learning process is applied to a set of
At the same time, select the second node to be learned, and then select the first node.
The input value which is the teaching signal is given to the node of
A plurality of first directional links coupled to the configured second node
Learn a set of non-linear selective functions of
Network type information processing system characterized by being
Stem. 7. The selection rule is one of the following:
The network type information processing system according to claim 1,
Stem. 1) Select the node whose output value of the second node is above a certain level
It 2) Select the upper N nodes whose output value is the second node.
Choose. 3) The second node that obtained the largest output value and its neighboring 2n
Select the dot second nodes. 4) The sum of the output values of 2n neighboring second nodes is the maximum
Select a set of nodes.