GB2259796A - Providing status output in a fuzzy logic system - Google Patents

Abstract

In the automatic tuning of a fuzzy logic computational network, the error between the logic result and the desired output value is back propagated, 1004, over the network; rates of change are calculated to indicate the degree of change in the output error with respect to the parameters contained in each network node, or the output values of each node; these calculations are performed, 1005-1007, on several teaching data input-output pairs; and a prescribed statistical quantity (e.g., average or standard deviation) related to the rate of change corresponding to that node is calculated, 1009, for each prescribed node and provided as external output, 1013. These rates of change indicate the degree by which changing the output of that node changes the output error, or rather a value which could be considered to be the sensitivity of the output error to that node's output; hence they can point to fuzzy rules requiring correction. <IMAGE>

Description

FUZZY LOGIC STATUS OUTPUT METHOD AND FUZZY LOGIC SYSTEM The present invention relates to a fuzzy logic status output, particularly used when tuning fuzzy logic rules and the membership functions contained in the fuzzy logic rules and the like such that the output of a fuzzy logic process becomes closer to prescribed teaching data, to support efficient implementation of such tuning.

An example of automatically tuning the shape of the membership function and other fuzzy logic parameters is described in a paper entitled "Self-Organizing Fuzzy Controller" on pages 50-56 of the Transactions of the Society of.Instrument and Control Engineers, Vol. 20, No. 8 (1984).

In addition, a proposal is made, Fuzzy Knowledge Model of Neural Network Type, on pages 55-58 of the Proceedings of the 1990 International Joint Conference on Neural Networks, Vol. 2, for implementing the fuzzy logic process in a computer using neural networks, thereby automatically tuning the logic parameters by means of the learning rules of the neural network. Another proposal for performing fuzzy logic on a neural network is made on pages 795-797 of the Journal of the Information Processing Society of Japan, Vol. 30, No. 6 (1989).

The prior art includes proposed methods of automatically tuning the shape of the membership function and other fuzzy logic parameters such that the output of a fuzzy logic is closer to the teaching data desired output within previously given fuzzy logic rules, particularly when given one or more pairs consisting of input and the corresponding desired output of teaching data.

The ajiti of the present invention is to analyze the prior art into problems and provide solutions to such problems.

The actual output resulting from a fuzzy logic typically does not coincide exactly with the desired output, so a certain amount of error often remains even after the learning process is complete. In the prior art, the internal status of the fuzzy logic process is handled as a black box, the contents of which are unknown externally. For this reason, it is difficult for the designer of a fuzzy knowledge base to identify and understand the reasons why error remains.

Accordingly, to reduce the error between the actual output of a fuzzy logic and the desired output even further, the designer must repeatedly change the initial state of the parameters at the start of tuning, add and/or delete fuzzy rules in a trial-and-error fashion, or perform other tasks.

This has prevented efficient tuning.

Particularly in the prior art, only the membership function and specific other fuzzy logic parameters are automatically tuned by learning; fuzzy rules are not automatically tuned. In addition, one cannot determine which rule is inappropriate to what degree. Accordingly, addition and/or deletion of fuzzy rules to reduce error has no logical basis, making the task an inefficient trial-and-error operation.

It wd be desirable, in such an analysis, to be able to .rove efficiency when teaching data is used to tune fuzzy logic parameters and teach input/output relationships by allowing the designer or other person to easily identify the status of fuzzy logics for efficient addition, deletion and modification of fuzzy logic rules.

The present invention: expresses a fuzzy logic process with a computational network such that, in a fuzzy logic system in which fuzzy logics are performed as a forward propagation of signals on this network, the error between the actual output of a fuzzy logic and the desired output is back propagated through the network; and determines a rate of change which indicates the degree of change in the output error with respect to parameters contained in each node of the network, and a rate of change which indicates the degree of change in the output error with respect to the output of each node in the network. These rates of change are determined for several teaching data pairs consisting of an input and the corresponding desired output.

Statistical quantities related to these rates of change for each of a prescribed number of nodes are calculated, displayed, printed or otherwise provided as external output.

First, by expressing a fuzzy logic process with a computational network, it is possible to clearly express the dependency relationships between the resulting output of the logic process and various logic parameters contained in the fuzzy logic. Here, the fuzzy logic process is executed as a forward propagation of signals on this network.

The input/output relationships at each node in the computational network.and the links which represent the connection relationships between nodes are defined according to fuzzy logic processes to be executed on that computational -network. #In other.words, the.logic parameters (e.g., the shape of the membership function and specific other fuzzy logic parameters) are set by establishing prescribed input/output relationships at prescribed nodes in the computational network. In the same manner, the rules of fuzzy logic are set by establishing prescribed input/output relationships at prescribed nodes and links in the computational network.In this manner, the actual input/output/connection relationships of each part of the computational network are defined depending on the fuzzy logic process to be executed. Accordingly, when logic parameters are varied slightly, values (e.g., the derivative coefficient or difference) can be found to indicate the degree by which the output of each node varies. In the same manner, one may find out how the output of each node changes when its input is changed by a small amount.

The object of the automatic tuning of logic parameters during learning is to minimize the error between the value actually generated as the logic output and the desired output.

When automatically tuning logic parameters, since the logic output is the output of the final node in the computational network, if that final node contains a parameter to be tuned, a derivative coefficient (or difference) related to that parameter is found first and then derivative coefficients are found for the inputs to the final node. Since the inputs to the final node are the outputs of the other nodes located one rank before the final node, derivative coefficients (of the logic output) are found for the outputs of nodes located one rank before the final node. Accordingly, derivative coefficients related to the logic parameters and inputs contained in the nodes located one rank ahead of the final node are found by the same procedure.

By repeating the operations above, namely by back propagating the error between the actual logic process output and the desired teaching output through the computational network, a rate of change which indicates the degree of change in the output error with respect to the desired output is obtained for each node in the network. In addition, it is possible to obtain rates of change which indicate the degree of change of the logic output with respect to the logic parameters contained in each node. Each node in the computational network corresponds to a membership function contained in the IF part or THEN part of a fuzzy rule, for example. Accordingly, derivative coefficients of the logic output with respect to the membership function values contained in the IF parts and THEN parts of fuzzy logic rules are obtained.

In the automatic tuning of fuzzy logic parameters during learning using teaching data,. a derivative coefficient or difference is found-as above with respect to data pairs consisting of a given input and the desired output for that input. These calculations are performed on several sets of teaching data pairs, and the logic parameters are automatically tuned based on the average value of the results.

This tuning modifies the logic parameters such that the final logic result coincides with the desired output value.

Accordingly, when the learning process is complete and the logic parameters have settled down to a state in which they vary no further, this average value should ideally become nearly zero.

However, there are cases in which the desired output and the actual output do not coincide completely even after the learning process is complete, resulting in poor performance of the fuzzy logic. In these cases, with only the method of automatically tuning parameters, the average values of said derivatives coefficient or difference are only used for internal processing, so the external human operator (e.g., the fuzzy logic knowledge base designer) cannot know any more than this. Accordingly, as described above in analyzing the prior art, the designer can only change the initial state of the parameters at the start of tuning, or add, delete and/or modify fuzzy rules in trial-and-error fashion.

In contrast, by means of the present invention, the error between the logic result and the desired output value is back propagated over the computational network, rates of change are calculated to indicate the degree of change in the output error with respect to the parameters contained in each node, or the output values of each node, these calculations are performed on several teaching data pairs, a prescribed statistical quantity (e.g., average or standard deviation) related to the rate of change corresponding to that node is calculated for each prescribed node and provided as external output.Accordingly, if the node generating that statistical quantity is a node corresponding to a membership function contained in the IF part or the THEN part of a fuzzy rule, the average, standard deviation, or other statistical quantity of the rate of change (derivative or difference) related to that membership function is found. In addition, if that node is a node corresponding to the fuzzy grade of the IF part of a fuzzy rule, the average, standard deviation, or other statistical quantity of that grade is found.

These rates of change indicate the degree by which changing the output of that node changes the ultimate output error, or rather a value which could be considered to be the sensitivity of the output error to that node1s output.

Accordingly, the aforesaid average, standard deviation, or other statistical.quan.tity.9,f. the rate of change is information which represents what effect the output of that node has on the ultimate output error. For example, for a node corresponding to a certain membership function, assume that one finds the derivative coefficient of the logic results with respect to the output of that node, and calculates its standard deviation. If the value of this standard deviation is large, this means that there is a large amount of dispersion in the change in the output error when that membership function value is changed by a small amount, so there is strong evidence to suggest that this membership function and the rule which uses this membership function is inappropriate, allowing one to surmise that it is a cause of error in the logic results.

In this manner, the present invention calculates new information in the form of prescribed statistical quantities related to rates of change, thereby facilitating the discovery of membership functions or fuzzy rules that cause error in fuzzy logics. In particular, the membership functions and fuzzy rules can be sorted by their average, standard deviation or. other statistical quantities in descending order or ascending order, and provided as an external output which is convenient and visually very easy to read. The external output may be provided in displayed or printed form.

M#oreover, in this external output, by arranging the membership functions (in a display which emphasizes membership functions) or rules (in a display which emphasizes rules) in a 2-dimensional matrix, the membership function or rule mostly causing the error can be discovered very easily by visual inspection. Accordingly, a fuzzy knowledge base designer can easily identify the rule or membership function to be revised based on this information.

An embodiment of the present invention will now be described, by way of example, with reference to the accompanying drawings, in which: Figure 1 is a flowchart of the typical work of building a fuzzy knowledge base; Figure 2 is a flow chart of graphs of membership functions to explain the fuzzy logic process; Figure 3 is a diagram of the fuzzy logic process implemented on a computational network; Figure 4 is a diagram of the typical shape of a membership function; Figures 5A and 5B are together a diagram showing examples of fuzzy rules; Figure 6A and Figure 6B are together showing a display screen of membership correction quantities of the present invention; Figure 7A and Figure 7B are together showing the display of Figure 6 sorted according to the standard deviation of the membership correction quantity in descending order;; Figures 8A and 8B are together a diagram showing a display screen of rule correction quantities of the present invention; Figures 9A and 9B are together a diagram showing the display of Figure 8 sorted according to the standard deviation of the correction quantity in descending order; Figure 10 is an overall flowchart of the learning and fuzzy logic status display processes implemented in the present invention; Figure 11 is a diagram of a computer system used to implement the present invention; Figure 12 is an example of a fuzzy logic process; Figure 13 is an example of an output of the present invention for the process of Figure 12; and Figure 14 shows the results of a comparison of the present invention and the prior art applied to an actual case.

In this specification a statistical quantity is a result of a statistical process of: assembling or collecting plural data of back propagated expected teaching output and output from fuzzy logic network processing'teaching input; and mathematically analyzing, interpreting, tabulating or classifying the plural data to produce one data quantity as the result.

An enbodim#t of the present invention is described in reference to Figures 1 through 14.

Figure 1 shows the typical work-flow of building a fuzzy knowledge base according to the present invention.

First, fuzzy rules are defined in a rule definition process step 101 by a user, for example. Next, the functions within the fuzzy rules are defined in a membership function definition process step 102 by a user, for example. Then in a learning process step 103, an appropriate learning algorithm is used to perform learning. This learning process step 103 is a process in which several sets of data pairs consisting of a teaching input and the corresponding desired teaching output (teaching data) are provided, and then the shape of the membership function and other logic parameters contained in the fuzzy logic are automatically tuned such that the result of the fuzzy logic becomes closer to the desired/expected teaching output value. After the learning process is complete, according to the present invention, the status of the fuzzy logic after learning is displayed in a logic status display process step 104. At this time, prescribed statistical quantities are also displayed. In an evaluation/analysis process 105, based on the information displayed, the system builder, for example user, evaluates and analyzes the fuzzy logic system. If the performance is satisfactory, the system builder may end the system construction, but if not, the system may be rebuilt starting from the rule definition process step 101 or the membership function definition process step 102.

In this scheme, process steps 101, 102, and 105 are tasks performed by the system builder/user while processes steps 103 and 104 are executed by software running on a computer system, for example.

The following is a description of the fuzzy logic method used in the fuzzy logic system of this example.

There are several different definitions of the fuzzy logic method, but here we will describe the most typical method by making an example of the method described on pages 942-947 of the Journal of the Information Processing Society of Japan, Vol. 30, No. 8 (1989). Naturally, the scope of application of the present invention is not limited thereby, but rather the present invent-ion may also be applied.

to other fuzzy logic methods.

First, the fuzzy rules are written in the following format: if x1 is All and x2 is A21 and ... xn is Anl then y=B1 if x1 is A12 and x2 is A22 and ... Xn is An2 then y=B2 if x1 is Alm and x2 is A2m and ... and xn is Anm then Y=Bm (1) where, n is the number of input variables, m is the number of fuzzy rules related to output variable y, Aij is the membership function for the input variable xi in the jth rule, and Bj is the membership function for the output variable y in the jth rule. The domains of Aij or Bj are the entire domains of their respective independent variables, and their ranges are each (0,1 ] .

In equation (1), each input variable appears in every rule, but this need not be the case. For example, if the ith input variable Xi is unrelated to the jth rule, then the membership function Aij may be considered to be a function which always takes a value of 1. In addition, the same membership function may appear in multiple rules. In this case, the prescribed membership functions Aij and Akj may be considered to represent the same membership function. In addition, since groups of rules for difference output variables may be considered to be independent, the following description will be limited to rules in the format of equation (1).

The logic process when the input variables for equation (1) take values of xi=Xj is as follows (herein lower case letters denote variable names and upper case letters denote specific variable values). First, the grade Wj of the predicate of the IF part of each rule is calculated as follows: Wj = min(Aij(X1), A2j(X2), ..., Anj(Xn)) (2) The membership function B(y) of the output variables is calculated as follows: B(y) = max(W1*B1, W2*B2, ..., Wm*Bm) (3) where, * indicates multiplication, having the same meaning as X.The ultimate output value (logic result) Y is found as ths average value Y = jB(y)y (Y) Y dy \ it tB(y) dy (4) when function B is considered to be a probability distribution function.

Figure 2 is a graphic representation of this logic process using graphs of the membership functions. For simplicity, consider the case in which a number of rules m is 2 and the number of input variables n is 2. In this figure, 201 is a graph of the membership function A11 of variable x1 in the first rule. In the same manner, 202 is a graph of membership function A21 of variable x2, 203 is a graph of membership function A12 of variable x1, and 204 is a graph of membership function A22 of variable x2. From equation (2) above, 205 indicates the process of calculating the minimum value of Ail(Xi) where i=1,2. In process 205, the grade W1 of the first rule is found. Graph 206 represents the membership function B1, and the shaded graph 207 represents the membership function of W1*B1.

In the same manner, the grade W2 of rule 2 is found by process 208. Then, W2*B2 is found from the membership function B2 represented by graph 209. Graph 210 represents the membership function of W2*B2.

From equation (3) above, 212 indicates the process of finding the membership function B(y) of the output variables.

In this process 212, the maximum value of the membership function of output variable y in each rule is calculated. The ultimate membership function B(y) found by process 212 is given in graph 211. Then, the position of the center of gravity of the shaded.area enclosed by graph 211 is found according to equation (4), and this result is generated as the logic computational output value Y which is the ultimate result of- the logic using an input of X1, X2.

The following is a description of a fuzzy logic method in which the max operation of equation (3) is replaced by algebraic summation.

In this case, equation (3) becomes B(y) = W1*B1 + W2*B2 + ... + Wm*Bm (5) Substituting equation (5) into equation (4) and simplifying gives

Figure 3 shows this fuzzy logic process implemented on a computational network.

Node 301 corresponds to input variable x1. In an actual fuzzy logic, the node would take the concrete input value x1.

In the same manner, node 302 corresponds to input variable x2, with the remaining input nodes (vertically aligned in the drawing) similarly corresponding to each of n input variables.

Node 303 corresponds to the membership- function All of variable x1 in the first rule. Node 303 generates the value A11(X1) when.provided.-wi.th i#nput of. the value X.. Nodes are' provided corresponding to all of the membership functions Aij appearing in equation (1).

Node 304 finds the grade W1 of the first rule. Namely, from equation (2), this node accepts input of A11(X1), A21(X2, ..., An1(xn) and generates output W1 consisting of its smallest input. In the same manner, nodes are provided to find the grade Wj for each rule.

Node 305 calculates the numerator zY1*Wl of equation (6).

Namely, it multiplies the grade Wi of each rule calculated in nodes 304 and the like by Yi from equation (8) as a weight, and generates output of the sum of the products. Similarly, node 306 calculates the denominator zSi*Wi of equation (6).

Here, Yi is considered to be a parameter of node 305, and Si a parameter of node 306.

Node 307 divides the output zYi*Wi of node 305 by the output zSi*Wi of node 306, to calculate the ultimate output value Y.

In this manner, the computational network of Figure 3 executes the fuzzy logic in the forward propagation direction FP, which is the logic direction, expressed by equations (1) through (8). The following is a description of the learning process step 103 of Figure 1.

Here, we shall assume that the objection of learning by tuning of parameters is to minimize the sum of squared error between the desired or expected output Yt and the actual computational output (logic result) Y. To achieve this purpose, a target function of E = (Yt-Y)**2(where **2 means raise to the second power) is used to perform parameter tuning by means of the steepest descent method with ta a -DE/805 (where ais an arbitrary parameter). Accordingly, in the learning process, the partial derivative coefficient of the target function E is first found for all parameters contained in the fuzzy logic process described above.

First, one prepares a desired or expected output Yt for each actual computational output value Y. Node 308 of the computational network of Figure 3 is the node which calculates the difference Yt-Y. This value Yt-Y becomes the error signal 6307 for node 307. Thereafter, by tracing along the computational procedure in reverse order or back propagation BP in the parameter tuning direction, the partial derivative coefficient of the target function E with respect to the output of each node in Figure 3 can be found. This is described hereafter.

The error signal 6307 that is input to the output node 307 is proportional to the value of the partial derivative of the target function with respect to the output Y of, node 307.

The'other inputs to node 307 are the outputs of nodes 305 and 306. Accordingly, the partial derivative coefficient of the target function with respect to the output z305 of node 305m from the chain rule for composite functions, can be found by

Similarly, the partial derivative coefficient of the target function with respect to the output value z306 of node 306 can be found by

Next, the partial derivative coefficient with respect to the output value Z304 of node 304 becomes

which can be readily calculated from the equations

derived from the.definition given above.

Thereafter, the partial derivative coefficient of the target function E can be found sequentially for each node in the'same manner. Here, even if there may be a partially nondifferentiable function such as in node 304 which calculates a minimum value, the computational procedure above can be executed without modification. Namely, since the output of node 304 may be considered to be equal to one of the inputs Ail(Xi) the value of the partial derivative may be set to 1 for the equal input and to 0 for all others.

Thus, the procedure described above is used to find the partial derivative coefficient of the target function E for each node in the network. The logic parameters to be tuned are contained in the input/output relationships of each node.

Namely, as the output z of any single node is a function of the inputs ## ,2 ,k #k to that node and the parameters al, ## ## ..., aqfor that node, one may write z = f(#1, #2, ~~~, #k# al, a2t ..., #q) (13) The partial derivative coefficient #E/#z of the target function E with respect to the output z for an arbitrary node can be found by the procedure above, and the partial derivative coefficient with respect to a parameter ai can be found by aE/8ai = AE/8z ~ #Z/### (14) For example, the partial derivative coefficient with respect to the parameter Yi contained in node 305 of Figure 3 can be found by

If parameter Yi is tuned by an amount AYi given as AYi oc -#E/#Yi (16) then the steepest descent method can be used to tune the parameters contained in each node. The proportional coefficient in equation (16), generally called the learning coefficient, is one factor in determining the rate of convergence.

Next, we will describe the method of tuning parameters related to nodes such as node 303 which represent membership functions.

The input/output relationship of node 303 can be written as z303 = A11(X1) (17) where Z303 is the output. Membership functions do not typically take arbitrary shapes, but rather most are specified by a relatively small number of parameters. Figure 4 shows one example of the shape of a membership function. In Figure 4, 401 indicates a membership function which has a trapezoidal shape. In this case, the shape of the membership function can be specified with the coordinate values a1, a2, a3 and a4 of the four vertices. Accordingly, equation# (17) can be written as z303 = A11(X1; a1, a2, a;3, a4) (18) when these parameters are written explicitly.Since the value of the partial derivative Az303/dai of output Z303 with respect to each parameter can be easily found, parameter tuning can be performed by the steepest descent method as in equation (16).

By finding the partial derivative coefficient AE/@cti using the method described in this example, one may greatly reduce the amount of calculation in comparison to the method of changing one parameter at a time by a small amount, performing a fuzzy logic and then finding a partial derivative coefficient.

The method of selecting the parameters which specify the shape of a membership function is arbitrary. For example, one may select for parameter tuning only the left-right horizontal motion of the trapezoid shown in Figure 4. In addition, the same can be done for membership functions of shapes other than trapezoidal.

In the network of Figure 3, the membership functions for input variables are all different. In contrast, when there are constraints that one must use the same membership functions and the membership functions must have the same shape after parameter tuning, parameter tuning can still be performed while satisfying these constraints by limiting the number'of corresponding nodes to one.

This completes our explanation of an example of the learning process. In this learning process, by giving pairs consisting of an input and the corresponding desired output, or teaching data (learning data), the parameters of a fuzzy logic can be automatically tuned such that the logic result which is the actual computational output becomes closer to the desired output.

In addition to the above explanation of the learning procedure, the present example will use 3a = aE/8a (where a is an arbitrary parameter, or the value of an intermediate output in the fuzzy logic) calculated above to display the logic status. In the conventional error back propagation method, the value of 5a is used to automatically tune parameters.

Accordingly, once the learning process is complete, ideally the sum of 8a for the input/output pairs (namely, the sum of 6a calculated when given several sets of teaching data) should be nearly zero. However, these are cases in which the desired performance is not obtained even after the learning process is complete. For example, a fuzzy rule defined at the start may be. inappropriate, or the learning may be trapped at a local optimal value. In these cases, one may not be able to determine the cause of why the desired performance is not obtained, and with the prior art, the system cannot give guidance regarding what changes should bemade. In the . . .

present example, the value of 5a calculated for the pairs of teaching data by the above method is retained, suitably manipulated and displayed. Concretely, the average at each node for plural teaching data pairs, standard deviation and other statistical quantities of the value of a are calculated and displayed externally as useful information.

Next, we will describe such an logic status display process step 104 in Figure 1. The quantity ba = aE/Da (where a is an arbitrary parameter or output of an arbitrary node) can be called the rate of change in the output error E which indicates the degree by which the output error changes when a is varied, but in the following discussion, Sa will typically be called the 'correction quantity'.

Figure 5 shows an example of the fuzzy rules used in this explanation of the logic status display process. In this figure, there are ten input variables: 'moving average 1 (past)', 'moving average 1 (present)', 'separation rate 1 (past)', 'separation rate 1 (present)', 'moving average 2', 'stock price change 1 (present), 'stock price change 1 (past)', 'separation rate 2 (past)', 'stock price change 2 (present)', 'separation rate 3 (present)', and one output variable 'prediction'.There are eight rules, and rule 0, for the example of Figure 5 says: Rule O; IF 'moving average 1 (past)' is 'decreasing' AND 'moving average 1 (present)' is 'unchanged-increasing' AND 'separation rate 1 (past)' is 'negative' AND 'separation rate 1 (present)' is 'positive' THEN 'prediction' is 'certain increase' The other rules are defined in the same manner as shown in Figure 5.

Figure 6 is an example of a screen displaying the logic status for example 1, in which the display emphasizes the correction quantities for membership functions. Here, the predictions corresponding to 16 membership functions are displayed in numerical order according to the prediction number. Note that the fuzzy rules of Figure 5 would give a total of 19 predictions, but for simplicity of explanation, only 16 are shown. One horizontal row of Figure 6 corresponds to one membership function. Column 601 contains the average values of the membership correction quantities (Mbshp. Avg).

The values in this column 601 are each calculated by ####/N.

In the calculation, the correction quantity ##i in question is the correction quantity for the output of nodes such as node 303 of Figure 3, whi?-h.rep.re#.s...ent. membership functions, and the sum'is taken per teaching data pair. Namely, when N pairs of teaching data are given, a correction quantity ##i is calculated for each and their sum is calculated. N is the number of teaching data pairs for which the correction quantity is actually back propagated through the membership function (node) in question. - Column 602 contains the standard deviation of the membership correction quantities azi (Std.

dev.) which are calculated in the same manner.

Columns 603 show, in 2-dimensional matrix form, which individual membership functions are referenced by which rules.

According to Figure 6, the membership functions numbered 0 through 3 are referenced by rule 0, and membership function 4 is referenced by rules 1, 2 and 7, for example. This sort of matrix display allows the operator to determine the reference relationships between membership function and rule.

Column 604 gives the linguistic representation of the membership functions (predicates or predictions). This column expresses the concrete meaning of the individual membership functions.

Figure 7 shows the rows of Figure 6 sorted according to the standard deviation of the membership correction quantity (column 602) in descending order. The prediction as the fuzzy output is set forth in Figure 7.

By means of example 1 shown .in Figures 6 and 7, the membership functions which have large standard deviations and the rules containing those membership functions can be readily discovered. As described above, membership functions which have large standard deviation values result in a large amount of dispersion in the rate of change of the logic result with respect to the membership function output. Accordingly, membership functions which have large standard deviations and the rules containing those membership functions are likely causes for the error in the fuzzy logic result. In this manner, the membership functions (predicates) or rules which cause the error can be readily identified, and thus this system can support the fuzzy knowledge base designer in the work of tuning rules or membership functions.

Figure 8 is an example of a screen displaying the logic status for example 2, in which the display emphasizes the correction quantities (CORR QTY) of rules. Here, the eight rules (0-7) are displayed in numerical order. One horizontal row corresponds to one rule. Column 801 contains the averages (standard deviation, STD DEV) of the correction quantities for the grade of the IF part of each rule (corresponding to the partial derivative coefficient of the output of node 304 of Figure 3), while column 802 contains the standard deviation values (STD DEV) of the correction quantities for the grade of the same IF...part..... .The.methqd of calculating these values is exactly the same as the calculation method used in example 1.

Column 803 contains the averages of the correction quantities for output, while column 804 similarly contains the standard deviation values. The correction quantities for output are the partial derivative coefficients for yi which is the parameter for node 305 of Figure 3.

Column 805 shows the membership functions (predicates) used in the IF parts of each rule, while column 806 shows the membership functions used in the THEN parts, in matrix form.

Figure 9 shows the rows of Figure 8 sorted according to the standard deviation of the output correction quantity in descending order.

By means of example 2 shown in Figures 8 and 9, the membership functions which have large standard deviations and the rules containing those membership functions can be readily discovered. Accordingly, the membership functions or rules which cause error in the fuzzy logic results can be readily identified, and thus this system can support the fuzzy knowledge base designer in the work of tuning rules or membership functions.

The display may emphasize individual membership functions as in example 1, emphasize individual rules as in example 2, or both may be used when suitably combined, to make a more effective.#logic -st#tus display.

Figure 10 is a flowchart (in problem analysis diagram (PAD) format) which visualizes examples 1 and 2. The flowchart of Figure 10 represents steps 103 and 104 of Figure 1 in greater detail. Hereafter, the output of each computational node in Figure 3 is represented by Zit and the derivative coefficients of the target function E with respect to output zi is represented as bi.

In step 1001, repeat steps 1002 through 1012 unconditionally. In step 1002, read the teaching data used in learning, namely the values of input variables and corresponding desired output values Yt. In step 1003, perform an actual fuzzy logic computation based on the value of the input variables read in step 1002. In this process, output values zi are found for all nodes of the computational network of Figure 3.

In step 1004, trace all nodes of the computational network in reverse order from output to input. Since the computational network does not contain any closed loops, this sequence can be definitely determined. Assume that the number of. the node selected in step 1004 is i. In step 1005, determine whether or not the node selected in step 1004 is the output node at the end of the computational network; if it is.

the output node at the end, execute step 1006; if not, execute step .1.007..;.:. In step 1006, calculate for the fin#al..output.

node as Si = Yt-Y. In step 1007, calculate bi for a node other than the final output node from the equation

which is a generalized form of equation (11). The summation here is taken for all nodes j which have the output Zi of the selected node as input.

So far, the derivative coefficients were found for the target function E with respect to the output of every node in the computational network. These values are to be used in the logic status display process 104 of Figure 1, so they are stored in an appropriate storage area.

In step 1008, execute steps 1009 and 1010 for each parameter a to be tuned. In step 109, calculate dzilaa for each node i containing parameter a. Since a is a parameter which determines the input/output relationship in node i, the value of its derivative is easily found. In step 1010, use the value azilaa found in step 1009 to find the change in a, symbolized by Aa, by Aa = -# . #i# azi/aa and change the value of parameter a such that a t a + ta.

In step 1011, determine whether or not a predetermined end condition has been reached. For example, the end condition may be that the above steps have been executed a predetermined number of times, or that. the difference between the actual output and the desired output has been reduced to below a predetermined value as a result of the parameter tuning, or the like. Only when the end condition has been reached, execute step 1012. If the end condition has not been reached, continue the learning process starting with step 1002. In step 1012, assume that the parameter tuning is finished and break out of the loop of step 1001.

In step 1013, in order to display the logic status after the learning process is complete, take the values of bi calculated in step 1007, manipulate them into a form such as that shown in Figures 6 through 9, and display them.

Figure 11 is an example of a computer system configuration required to implement the examples of the present invention described above. A CPU 1101 executes the process described in Figure 10 according to a predetermined procedure contained in a program. The program, data and the like are stored in memory 1102 which is read or written to by the CPU 1101 as necessary. The CPU 1101, following the program stored in memory 1102, performs fuzzy logic processing on the computational network, calculating error when given teaching data, implementing the learning process by back propagating the error along the network, calculating rates of change and storing them in the memory 1102, calculating statistical quantities related to the rates of change, and providing external output of statistical quantities and the like.In this system, a keyboard, mouse or other input device 1103 is provided, so that a human operator may enter commands and the like. Thereby the execution of the system may be controlled. The results of the logic state display process 1013 of Figure 10 and the like are output to a display device 1104. The system of Figure 11 may be concretely implemented in a personal computer, workstation, (general purpose) mainframe computer or other computer system.

Note that the examples are described above such that the logic status is displayed on the display device 1104 such as a CRT or liquid crystal display, or a printer (or several displays may be used together). In addition, in the above examples, the logic status is displayed or provided as output by another method such that it may be referred to by the user or designer or system builder, who can specify that the system add or change fuzzy logic rules or logic parameters. However, this externally displayed information could also be used to automatically add or change fuzzy logic rules or logic parameters.

In the present application, statistical quantities refer to the result of any statistical analysis or calculation involving the rates of change that are obtained for all of the teaching pairs of data. For example, a simple statistical: quantity would be an average for one node of all of the rates of change at that node determined during back propagation of the error obtained with all of the teaching data pairs. For example, if 700 teaching data input pairs were successively employed, at each node there would be 700 grades of change obtained by back propagating the 700 errors in the fuzzy logic process to that node. Another example of a statistical quantity would be the taking of a root means square of those 700 grades of change at the node to get the degree of change at that node as a statistical quantity.A more accurate statistical quantity would be the average of the squared error, that is < (#/ & )2 > , which quantity is the average of the squared error, as another example of a statistical processing to get a statistical quantity. Of even greater accuracy would be the statistical quantity that would be the divergence of the back propagated error, which would be in general the average of the squared error minus the squared average, or specifically would equal < (#E/ & )2 > -( < DE/a0' > )2.

In Figure 12, a fuzzy logic system is schematically shown, which involves five rules. The membership functions were self tuned using a computational network. However, the automatic or self tuning in accordance with the prior art would not identify the problem relating to rule 2, wherein the case. of Xl being positive and X.2 being negative would produce or should produce a positive result instead of the negative result provided by rule 2. That is, self tuning or automatic tuning according to the prior art would neither correct such a rule error nor identify the location of such an error.

Next, the present invention was applied to the example set forth in Figure 12, with the results as schematically shown in Figure 13. In Figure 13, the first column, starting from the left relates to the input variables X1 and X2. The second column, from the left, is a visual display of the membership function node statistical quantities obtained by the present invention respectively for (starting from the top) negative, zero and positive values for X1, negative, zero and positive values for X2, and negative zero and positive values for generated back propagation errors. The fourth column from the left and the third column from the right contains the statistical quantities obtained with the present invention for the rule nodes.The statistical quantities as represented in Figure 13 are the divergence of the back propagated error and the size of the circles representing the statistical quantities are proportional to the divergence of the back propagation error, with the largest size being related to the largest divergence. From this, it is easy to see that the main problem relates to. rule 2 (having the largest. divergence representing circle) insofar as it deals with a negative value for the variable X2. With the present invention outputting this information, for example in a display, the programmer or knowledge base builder could easily identify that the problem is related to rule 2 particularly insofar as rule 2 handles a negative value for the input variable X2. As a result, the user would recognize that rule 2 should probably be split into two separate rules.The output represented by Figure 13 is merely for convenience in understanding the present invention, since it is output in network form. For the actual user, it would be more appropriate to output the results of the present invention in the form illustrated with respect to Figure 5, which would be easier to understand for the user.

As a further example of the present invention, reference is made to Figure 14. Figure 14 sets forth the results of a comparison of the present invention to the prior art automatic tuning. In 1988, a ventilation control system for a highway tunnel was built around a fuzzy knowledge base. The fuzzy knowledge base was tuned in accordance with the automatic tuning of the prior art as described under the background of the present invention. The purpose of the ventilation control system was to predict pollution levels five minutes in the future, so that various fans could be turned on, adjusted and the like along with the turning on and adjustment of dust collectors. The more precise such predictions, the more precisely the ventilation control system can control the fans and dust collectors to save money, energy and maintenance.

When constructing the fuzzy knowledge base for the actual system in 1988, the development time was 24 weeks. The predicted error, as determined by the root mean square system RMS, was 9.71us after the automatic tuning with respect to 700 cases of teaching data. The prediction error with 700 cases of actual test data was 7.43%. By comparison, the present invention was employed to rebuild the knowledge data base in accordance with the improvement features of the present invention with the same teaching data, which resulted in a reduction of the prediction error to 6.26% for the teaching data and a reduction in the prediction error to 5.81% for the test data. The development time was reduced by the present invention from 24 weeks to 4 weeks.

As described above, the present invention is able to visually display the degree to which membership functions or fuzzy rules contribute to logic error, allowing the fuzzy knowledge base designer to easily determine which membership function or rule should be tuned, thereby improving the efficiency of work.

While a preferred embodiment has been set forth along with modifications and variations to show specific advantageous details of the present invention, further embodiments, modifications and variations are contemplated within the broader aspects of the present invention.

Claims (25)

1. An apparatus for conducting fuzzy logic and having a computational fuzzy logic network having a plurality of nodes, the nodes having node parameters, rules and membership functions, and the network further having links connecting between the nodes, an input and an output, the apparatus having means for executing the fuzzy logic with forward propagation through the network of each input of a plurality of teaching value pairs, each pair consisting of a teaching input and a corresponding desired teaching output, to provide corresponding computational outputs; means for determining an error between the computational output and the corresponding desired output for each pair; means for back propagating through the network each of the errors and determining influences on the errors of at least some of the nodes;; means for determining a statistical quantity for each of said some of the nodes, each statistical quantity related to the influences corresponding to said each of said some nodes, as a fuzzy logic status output to be used; and means for outputting the statistical quantities to show the effect of each of said some of said node upon the error relative to the statistical quantities of the others of said some of said nodes as an indication of the relative need to tune or chånge-the nodes;

2. Apparatus according to claim 1, wherein said means for outputting provides a visual representation of the fuzzy logic status output.

3. Apparatus according to claim 1, wherein each of said influences is a derivative coefficient of the computational output with respect to one of the parameter or the node output of each node.

4. Apparatus according to claim 1, wherein the means for outputting correlates the statistical quantities and membership functions in the fuzzy logic status output.

5. Apparatus according to claim 4, wherein said means for outputting provides the statistical quantities and corresponding membership functions in the form of a 2dimensional matrix.

6. Apparatus according to claim 5, wherein said means for outputting provides the statistical quantities correlated with information corresponding to the rules.

7. Apparatus according to claim 1, wherein said means for'outputting provides the statistical quantities correlated with information corresponding to the rules.

8. Apparatus according to claim 7, wherein some of the nodes have a grade of the membership function used in an IF part of a rule or in the THEN part of å rule.

9. Apparatus according to claim 7, wherein said means for outputting provides the statistical quantities and their corresponding rules in the form of a 2-dimensional matrix.

10. Apparatus according to claim 9, wherein said means for outputting provides information which identifies a membership function used by a rule and/or a predicate corresponding to that membership function correlated to said statistical quantities.

11. Apparatus according to claim 10, wherein said means for outputting provides the statistical quantities and their corresponding membership functions or rules sorted in one of ascending order and descending order of the statistical quantities.

12. Apparatus according to claim 1, wherein said means for outputting provides the statistical quantities as one of an average and standard deviationof the influences.

13. Apparatus for conducting fuzzy logic processing, comprising: a computational network having a plurality of nodes having fuzzy logic rules with membership function and links connecting between the nodes, means for accepting input to the network, and means for providing a computational output from forward propagating the input through the network; and means for #determ#ining; # tat ls# tidal# quanìtiee' including the computational output in a learning mode with teaching data input so that the statistical quantities are unique to some of the nodes, and means for explicitly indicating correspondence of the statistical quantities with at least one of the membership functions and the fuzzy logic rules of the corresponding nodes.

14. Apparatus for performing fuzzy logic processing, comprising: a computational network having a plurality of nodes having node parameters, rules and membership functions, and links connecting between the nodes; means for accepting input to the network and providing computational output from the forward propagation of the input through the computational network; means for determining an error as the difference between the computational output of teaching data input and desired teaching output associated with the teaching data input by operating the computational network in forward prop#agation of the teaching data input; means for determining a contribution of a specific parameter, rule or membership function to the error by operating the computational network in back propagation of the error;; means for determining statistical- quantit-ies for'a plurality of the contributions corresponding to a plurality of different pairs of teaching data inputs and desired teaching outputs; and means for outputting the statistical quantities in association with their corresponding parameters, rules or membership functions.

15. An apparatus according to claim 14, wherein said means for outputting provides the statistical quantities, and the parameter, membership functions or rules in the form of a 2-dimensional matrix.

16. An apparatus according to claim 14, wherein said means for determining statistical quantities provides average and standard deviation of the rate of change of the back propagated error at the nodes.

17. An apparatus for analyzing a fuzzy logic processing using a computational network having a plurality of nodes having node parameters, rules and membership functions, and links connecting between the nodes, which network has means which accept input and provides processing output from forward propagating the input, comprising:: means for executing the fuzzy logic processing with forward propagation for each of a plurality of teaching value pairs to provide a corresponding processing output; each pair consisting of an input and a corresponding desired output; means for determining an error between the processing output and the corresponding desired output for each pair; means for back propagating each of the errors through the fuzzy logic network and using said back propagating for determining an influence of the node upon the back propagated errors for each of a plurality of the nodes; ; means at each node for determining statistical quantities of the influences for each node as a fuzzy logic status output during operation with a plurality of the pairs to be used for analyzing the effect of each node upon error relative to the effect of the other nodes upon error as an indication of the relative need for a user to tune or change the parameters, rules and membership functions at the nodes; and means for providing output of the statistical quantities correlated to information about the respective nodes.

18. An apparatus according to claim 17, wherein said means for providing output provides external output of information which identifies the statistical quantities and their corresponding parameters, membership functions and/or rules in the form,,of a 2-dimensional matrix..

19. An apparatus according to claim 18, wherein said means for providing output sorts the external output of information in ascending order or descending order of said statistical quantities.

20. An apparatus according to claim 19, wherein said means for determining statistical quantities provides the fuzzy logic status output as one of an average and a standard deviation of the influences.

21. A fuzzy logic system, comprising: a fuzzy logic computational network having a plurality of system nodes; means for automatically tuning by back propagating an error of the system output produced by a teaching data input relative to an expected teaching output; and means for producing a statistical quantity of a plurality of back propagated errors of different teaching data inputs and corresponding expected teaching outputs at each of a plurality of the system nodes.

22. The apparatus of claim 21, wherein said statistical quantities are averages of the back propagated errors at the nodes.

23. The apparatus of claim 21, wherein said statistical quantities include at least the average of the squared back propagated errors at the nodes.

24. The apparatus of claim 21, wherein the statistical quantities are the divergence of the errors during back propagation, specifically the average of squared error minus the squared average.

25. An apparatus for performing fuzzy logic processing substabtially as herein described with reference to the accarpanying drawings.