JPH05342189A - Learning system for network type information processor - Google Patents

Abstract

PURPOSE:To provide the system of network inference for which configuration is simplified, processing speed is (higher and learning efficiency is improved. CONSTITUTION:This system is provided with a data input part 12 for learning to input data for learning to a network type information processor, area judge part 13 to judge whether an area shown by the inputted data for learning is included in an existent recognizing area or not, filter function update part 14 to individually update respective filter functions in the set of filter functions forming the existent recognizing area when the area judge part 13 judges that the area is included in the existent recognizing area, and filter function generation part 15 to generate the set of filter functions for forming the new recognizing area when the area judge part 13 judges that the area is not included in the existent recognizing area.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information processing system having a network structure using a computer such as a pattern recognition device, a control system, a diagnostic system, a decision support system, and more particularly to a learning system thereof.

[0002]

2. Description of the Related Art As one of the conventional techniques for realizing advanced information processing functions, there is a neural network whose information processing structure is modeled on human brain nerve cells (for example, by Cohen and Feigenbaum). ,
"Artificial Intelligence Handbook" published by Kyoritsu Publisher, p487-
493, Amari, Special Feature "About Neural Networks", Journal of Japan Society for Artificial Intelligence, Vol. 4, pp118-
176 (1989)). A nerve cell is a multi-input / single-output information processing element. The signal is x = (x ₁ , x ₂ , x ₃ ...
X _n ) is represented by an n-dimensional vector, and the output signal is v. If the input-output relationship that determines the output v with respect to the input x of the nerve cell is known, the characteristics of the nerve cell are known. If a model that elucidates the characteristics of nerve cells and performs similar information processing is configured on a computer, information processing similar to that of human nerve cells becomes possible. further,
By providing multiple models (calculation units) that resemble nerve cells so that they can perform the same function as the information transmission function of synapses of nerve cells between them, a neural circuit similar to the neural network can be created. There is a possibility to build. In nerve cells, the signals x _i and v are considered to be expressed in the form of the frequency of nerve cell excitation pulses transmitted through nerve fibers (synapses), that is, the frequency of nerve cell firing. Therefore, in such a case, the signals x _i and v have continuous analog values. In the example of the conventional neural network model, analog values are often treated as digital values approximately. That is,
Signals x _i and v are often handled by associating a state in which nerve cells are not excited and a state in which they are excited with binary values of 0 and 1.

An example of the neural network model will be described in more detail below. The influence when the input signal x _i is transmitted to the node (computation unit) through the directional link (edge) is the weight value w provided in the directional link.
_It shall be determined by _i . Output v with respect to the input signal x is a constant input in a unit time is _{v j = f (u j)} (1) u j = Σw ij · v i -θ j (2) v i is located in one pre-stage The output of the calculation unit, the first being the input signal x. v _j is the output of calculation unit _j ,
w _ij is a weight (for example, a real number) of the connection from the calculation unit i to _j , and θ _j is a threshold value of the calculation unit j. u _j is called the sum of the inputs to calculation unit j. f is called the input function or characteristic function of the calculation unit. The function f (u) in the above equation (1) is usually a sigmoid {S-shaped monotonically increasing function 1 / (1 + e ^−x )} having a saturation characteristic because of the knowledge of neurophysiology and mathematical ease of handling. Be seen. .. Although Equation (2) is linear, Equation (1) has non-linearity, and thus such a calculation unit is called a quasi-linear unit.

Now, in order to perform desired information processing by the above quasi-linear model, it is necessary to simultaneously talk about the capability of the calculation unit and the structure of the network. First, an explanation will be given of a case where the network structure has only one layer, that is, an input layer and an output layer only. When one calculation unit is provided as an output for a plurality of n pieces of input information, that is, a combination of n variables, the input / output characteristics describe a function of n variables. For example, as the input / output function f of the calculation unit, a threshold function having a threshold value (value 0 if the total sum of inputs is negative,
If the value is 1 if it is positive), the n-dimensional space is cut by a hyperplane with a certain slope and the input pattern (n-dimensional)
Is output on the one side, and 0 is output on the other side. If there are three inputs, a tilted plane is placed in the three-dimensional space and one side becomes 0. The deviation (distance) from the origin is determined by the threshold value of the threshold function f. In the inference through the network having such a structure of only one hierarchy, the calculation unit obtains the sum based on the linear information on the input value, so that the information processing capable of the n-dimensional hyperplane is eventually linear. It is only a separation characteristic.

In order to perform non-linear separation, a network structure requires a plurality of layers. If one n-input calculation unit configures one n-dimensional hyperplane and m calculation units that share the same input are provided, then n hyperspaces having different inclinations will be generated in the n-dimensional space. There will be. Non-linear separation characteristics can be obtained by deciding which side of each of the plurality of hyperplanes the event to be separated is on and by setting the local deviation of the space. By arranging m-input calculation units so that all outputs of the m calculation units become inputs of one calculation unit, information processing in two stages becomes possible. A network that can perform information processing in multiple stages is called a hierarchical network. The first node is called an input node and basically does not perform any information processing, and transmits information to the input of the node at the next stage (hierarchy). The node at the final stage is called an output node, and the node existing between the input node and the output node is called an intermediate layer or hidden layer node. The middle layer node and the output node are calculation units.

In the prior art, the accuracy of input / output is
Since it is basically determined by the number of nodes in the middle tier, there were cases where the amount of data description increased explosively when trying to ensure accuracy. When the number of nodes in the middle layer is reduced, the output of the calculation unit at the final stage becomes a coarse stepwise expression, or the types that can be discriminated are reduced.
However, there is no established theory as to how much the number of intermediate layers should be increased.

Further, in the prior art, there is no effective means for describing the initial state of the system which is the basis of learning,
Basically, there were many cases where learning was started from an equilibrium state or a random state. Whether it is a physical problem or a problem that simulates the same ability as a human sense, the relationship between some input signal and the information to be output is often regarded as a domain. By the way, the role of one node in the middle layer of the neural network is to mean that one side of the hyperplane placed in the n-dimensional space is significant. In other words, when it is attempted to set a region in an n-dimensional superspace sufficiently densely, hyperplanes, that is, intermediate nodes, of about 2 n to 3 n power are prepared, and each of them surrounds the region. Must be placed. The placement or movement of the hyperplane is performed by simultaneously changing the weights of the n input edges (directional positive links) of the node and the threshold value of the transformation formula of the calculation unit. In other words, assuming a network in which the number of intermediate layers is one and only one is output, first, the number of intermediate nodes having a threshold value N2 = k ⁿ has weights between the input and the intermediate layer nodes. Number of edges e (1-2) = n × k ⁿ Next, between the intermediate layer node and the output layer node, the number of edges with weight e (2-3) = k ⁿ threshold Output node N3 = 1 is required. From the above, the total number of edges e = e (1-2) + e (2-3) = (n +
1) × k ⁿ total number of nodes having threshold value N = N2 + N3 = k ⁿ +1
Total number of adjustment points with weight value and threshold value = (n + 2)
× k ⁿ +1. k is a coefficient representing the fineness, and 2 <k <3 is a standard. Now, if k is 2, input signals are 10 and the number of outputs is 1, then the total number of adjustment points is (10 + 2) × 2 ¹⁰ = 12,888, which is a number that can be initialized even in rough settings. Not be. Therefore, it is impossible to specify the initial state having directionality, and it is difficult to adjust the directionality of learning and increase the convergence speed of learning. It is considered that the factor that makes this initial setting difficult is due to the structure of the neural network itself. As described above, the n-dimensional hyperplane can be formed by the intermediate layer node, but a plurality of n-dimensional hyperplanes that are initially randomly arranged have input signals according to the learning rule of the neural network as described later. Each time, the inclination is changed in the plane so that the output signal approximates the desired pattern, and the distance from the origin is changed to move. However, there is no guarantee that the hyperplanes will be arranged such that each hyperplane uniformly wraps a specific region (if it is spherical, it becomes an n-dimensional hypersphere). This is because hyperplanes cannot be described by describing their mutual relationships.

As described above, if there are a plurality of results to be identified (equal to the number of output nodes) due to the problem of setting a region with a certain degree of precision, the number of nodes in the intermediate layer and the number of output nodes increase. Do it. The premise so far has been explained assuming a framework for deriving one output based on n inputs. The value of the coefficient k is determined depending on the degree of precision when performing identification or diagnosis, but when m outputs are required, the number of intermediate layers increases by about m times. With the same calculation as above, the total number of edges and the total number of nodes are: e = n × m × k ⁿ + m × k ⁿ = (n + 1) × m × k ⁿ N = m × k ⁿ + m = m × (k ⁿ +1 ) If the number of inputs is n = 10, the number of outputs is m = 10, and k = 2, then e = (10 + 2) × 10 × 2 ¹⁰ = 122,880 N = 10 × (2 ¹⁰ +1) = 10,250 Actual problem As described above, it is difficult to physically configure the calculation unit to provide 10,000 or more calculation units as described above, calculate the sum and the threshold function, and calculate the weight of up to 120,000. At the same time, it often becomes a problem in terms of calculation processing time. As for the calculation processing time, if there is a learning function such as judging the quality of the result every time the inferred result is output and updating the threshold value and the weight value stochastically, the learning time Becomes an even bigger problem.

Parameters such as the weight of the directional link and the threshold value of the calculation unit of the hierarchical neural network in which the calculation units are hierarchically arranged like the input layer, the intermediate layer, and the output layer are initially determined from the problem analysis. Setting is difficult for the above reasons. Therefore, the neural network is often equipped with a learning method. A mechanism is required to give an example to a neural network, evaluate the inference error in the inference process through the example, and change the above parameters so that the inference of the network approaches the desired behavior. In this way, inductively determining various parameters is called learning. There are roughly two types of learning methods for the hierarchical neural network: learning without a teacher signal (or simply called a teacher) and learning with a teacher signal. The network type inference method according to the present invention can also perform these two learnings, as will be described later.

A number of studies have proposed a rule governing the learning process, a so-called learning rule,
Here, as a typical example, D.I. E. Error backpropagation proposed by Rumelhart (error back propagation, error back propagation)
on) Learning rule (eg, Rumelhart Out 2, "L
Earning Representations b
y Back-propagating error
s ", Nature, Vol. 323-9, pp533
-536) will be described. Formula (1), Formula (2)
The learning of a neural network having three layers will be described by using the above assumption. Neural networks with only input / output layers have many restrictions because they can only perform simple linear separation, but hierarchical neural networks with three or more layers have 1-
In the feature space linearly separated by two layers, the features are further combined and linearly separated by the next combination of layers 2-3. From this, it is said that any non-linear relationship between input and output can be described by sufficiently preparing the calculation units (nodes) in the middle layer (here, the second layer). Even if the description ability of the multi-layered neural network improved, it became available only after the discovery of an effective learning rule, the back-propagation learning rule.

The h-th calculation unit of the input layer (first layer)
To U_1hInput signal to x_h, I-th calculation of layer 2
Unit U_2i, Sum of its inputs u_i, Calculation unit
U_1hAnd calculation unit U_2iEdge between (directional link)
The conversion efficiency of w _hi, Calculation unit U_2iOut of
Force v_i, The j-th calculation unit U of the output layer (third layer)
_3jWeight w between_ij, U_2jThe sum of the input of u_j, Output
v_jThen, the relationship between them should be expressed as
You can However, in order to simplify the following explanation, the formula
The threshold value θ in (2) is omitted. v_j= F (u_j) = F (Σw_ij・ V_i) (3) v_i= F (u_i) = F (Σw_hi・ X_h) (4) The pattern of the input vector X of the network for learning now
A set of output patterns in the initial state of the network and the network
{(X_p, V_p)},_p= 1, 2, ..., N are prepared
There is. Input an appropriate input pattern X and
Network output v_jJ-th correct pattern Y corresponding to
Minutes to y_jThen, the total error E is positive with the network.
It can be represented by the square of the absolute value of the error from the solution. E = | e |²= Σ (y_p-V_p)² (5) <The bottom of Σ is p> Neural network finally obtained by learning
Is the training set {(X_p, V_p)}
So that the total sum E of the errors in the local network is minimized
If so, you have achieved your learning goals. Minimize E
In order to do so, E of half the error component of each node of E_jTo
Minimize. E_j= (Y_j-V_j)²/ 2 (6) dE_j/ Dv_j=-(Y_j-V_j) (7)

Now, the degree of influence dv _{j on} the output v _j when the weight w _ij, which is the strength of the coupling, changes slightly by dα.
From equation (3), / dα _j is dv _j / dα _j = v _i · f ′ (Σw _ij · v _i ) = f ′ (u _j ) · v _i (8). Therefore, the change dα _{j in} the weight of the connection is
DE _j / dα _j is given by: dE _j / dα _j = (dE _j / dv _j ) × (dv _j / dα _j ) = − (y _j −v _j ) · f ′ (u _j ) · v _{i = -δ j · v i (9} ) _{However, δ j = f '(u} j) · (y j -v j) (10) as described above, to the error at the time of changing the weights of the neural network Since the influence can be obtained by calculation, when a certain error is observed, the weight can be changed to reduce the error.

Here, the learning by the backpropagation (backpropagation) learning rule that is generally performed, when there is a certain input, the node component of the error generated in the final output stage in the equations (3) and (4). Based on the value, the weight w _ij of the preceding stage is slightly modified in the direction approaching the correct answer, and a new weight is set.
Reverse the calculation of equation (10) from the output error to the input,
It is possible to calculate the change amount Δα of the weight w _ij that reduces the error E _j . _{That, Δα j = η · δ j} · v i (11) w ij = w ij + Δα j = w ij + η · δ j · v i (12) δ j = (y j -v j) · f '(u _j) (10) formula in the same _{δ i = f '(u i} ) Σw ij · δ j (13) < bottom of Σ is j> formula (12), generalized error [delta] _j computing unit U _j
Is a modification equation for modifying the connection weight w _ij to the calculation unit U _j according to the product of the signal and the signal v _i transmitted by the connection. The constant η is generally a small positive number. Formula (12)
Acts to reduce the weight of the coupling from the unit that was signaling positive to the computing unit U _j if the error is positive, ie the computing unit U _j lacks activity. If the error is negative, the modification of the connection weight is reversed. Expression (1
Δ _j of 0) is called a generalized error and is obtained by multiplying the error of the output unit U _j in the final output stage by f ′ corresponding to the output sensitivity. On the other hand, in equation (13), δ _i
Is exactly the input of the error δ _j in the output layer, and is propagated in the reverse direction from the output layer to the input layer while being multiplied by f ′ (u). For this reason, it is called the back propagation learning rule. Expressions (10) to (13) can be applied even when there are more layers than the three-layer neural network shown above.

Improvements have also been made for effective learning. For example, in the method called the simulated annealing method, the degree of weight change at the beginning is large, and the degree of weight change is decreased when learning progresses and the error becomes small. is there. However, substituting equation (12) in the direction of the input layer, equation (1
3) also, w _hi = w _hi + Δα _i = w _hi + η · δ _i · v _h = w _hi + η · f ′ (u _i ) · Σ w _ij · δ _j · v _h (14) After all, once When a signal for training is input, any of the equations (10) and (12) or (14) must be calculated for all edges. Moreover, since a non-linear function is mainly used for the threshold function f (u) of the calculation unit, the calculation load becomes larger than it seems. The above calculation amount increases in proportion to the number of edges. On the other hand, as the number of edges increases, the amount of signal sets for training, that is, the number of times of learning must be increased. The time required for learning increases exponentially as the problem is more difficult to target.

Further, in the prior art, it is difficult to set the direction of learning convergence because it is difficult to set the initial state. Basically, learning is performed when the internal state is changed by backpropagation for each sampling, and there is a bias in how input data is given, or when the giving method is inappropriate for the learning method adopted. Occasionally, there was a failure. Therefore, there may be a problem in the learning speed and the validity of the learning result.

In the prior art, the processing description inside the system to be learned is basically expressed in the form of link weights and has no conceptual concreteness. It can be judged only from input / output data). That is, in the conventional system, the explanation of the internal process description was not good. Therefore, it is difficult to directly evaluate the learning result by the internal state (description).

In the prior art, the processing description inside the system to be learned is basically expressed in the form of link weights, and basically the links between nodes are operated in a state close to full connection. .. For this reason, one system in which learning results are partially extracted and used, two learning results are combined to form one system, and learning results are applied to systems having different data description formats (such as production systems) It was difficult to modify, expand, or shrink from the system to other systems. Also,
For the same reason, it was difficult to add / delete the input / output node.

The patent applicant of the present application has previously filed a patent application for an invention for solving the above-mentioned problems of the prior art (Japanese Patent Application No. 3-310082 "Network type information processing system"). .. This Japanese Patent Application No. 3-310082
In the network type information processing system of the invention (hereinafter, referred to as “previous invention”), an input layer having a plurality of nodes and an output layer having a plurality of nodes are coupled via a directional link, and the directional link is In the network type information processing system, which has an information conversion function of converting information passing therethrough, and the node of the output layer has a function of performing a function operation on information input via a directional link, The information conversion function of the sex link has a filter function operation unit that performs information conversion according to a filter function having a selective characteristic such as a band pass type or a band stop type. According to this, the filter function of one directional link corresponds to the hyperplane realized by the plurality of intermediate layers of the conventional neural network, and the directional link connecting the input layer and the output layer is The area specification to create is
This corresponds to the area designation indicated by a plurality of hyperplanes each including a combination link of the input layer, the intermediate layer, and the output layer and a node in the above-mentioned conventional technique. Therefore, the invention of the network type information processing system described above reduces the amount of description required to obtain the same accuracy as compared with the conventional neural network, reduces the number of elements, simplifies the configuration, and reduces the processing time. Can be shortened.

The learning method in the network type information processing system of the above-mentioned invention is such that while the network type information processing system processes information from the input information through the directional link and the calculation unit, the same input information is normally processed. Means for obtaining various information processing results, and using the normal result as the teacher signal, the difference (error) between the teacher signal and the above information processing result (output of the calculation unit) is obtained by the evaluation function, and the difference is large or small. And a vector value are calculated, and a learning means for correcting the filter function of the directional link is provided through the learning function. Here, as the means for obtaining a normal information processing result, for example, teacher information prepared by comparing input information and correct output information corresponding thereto is prepared in advance, and when the input information is given, correct output information is obtained. It is configured to be taken out. The learning function is a learning rule such as a gain value that changes the shape of the membership function described later. The learning means statistically processes the results / evaluations of a plurality of reaction cycles in the network type information processing system, not the error backpropagation learning for each reaction cycle, and based on the result of this statistical processing, It has a function modifying means for modifying the parameters of the filter function. In addition, the reaction cycle is one cycle of a process of “inputting and outputting”.

[0020]

SUMMARY OF THE INVENTION It is an object of the present invention to further improve the network type information processing system of the prior invention which solves the problems of the conventional neural network.

1) In the rule generation method in the network type information processing system of the previous invention, there is one distribution function for each dimension. Therefore, there is a possibility that the recognition rate in the boundary area may be low. In addition, in the network type information processing system of the previous invention, in order to increase the recognition rate, it is possible to divide an effective area in the target space into a plurality of squares and set a distribution function for each dimension for each square. , Then there is a risk of explosion of the number of divisions.
It is an object of the present invention to provide a learning system that improves the recognition rate in the boundary area of a network type information processing system.

2) In the network type information processing system of the previous invention, the evaluation data is used only for checking the end of learning, and is not incorporated in the learning system. The present invention effectively uses the evaluation data, surely creates a rule in the area where the rule is not generated,
It is an object of the present invention to provide a learning system that can improve learning efficiency.

3) In a processing system that performs inference, recognition, etc., in order to perform processing efficiently,
In many cases, preprocessing such as integration of multiple data and differentiation is performed. Conventionally, the contents of this preprocessing are created based on the analysis results obtained by statistical analysis methods such as principal component analysis and least squares method, empirical rules for the target system, physical characteristics of the target system, etc. The method of artificially creating it was adopted. In the statistical processing, particularly when performing non-linear analysis, the number of procedures in the processing device required for the processing in which the type and amount of data to be handled increases, and the processing time becomes long. Also, when artificially creating,
Workers have to research and describe empirical rules, characteristics of the target system, etc., which takes a lot of work time. Further, since the processing is described only depending on the knowledge of the worker and the content of the investigation, there is a risk of lack of objectivity and completeness. It is an object of the present invention to provide a learning system for a network-type information processing device that can construct a preprocessing system for performing automatic nonlinear analysis and reduce the preparation time and workload of the preprocessing system. This object is achieved by using a network type information processing apparatus to perform learning processing without teacher data and performing non-linear analysis with a plurality of input data.

[0024]

The present invention is a learning system for a network type information processing apparatus, and the network type information processing apparatus is used for the learning. The network type information processing device has a plurality of input nodes (2 in FIG.
1), a plurality of output nodes (FIG. 2; 24), and a directional link connecting the input node and the output node. The directional link is a non-linear selective function (m in FIG. 2).
_{11 to} m ₃₂ ) and a filter function storage means (113 in FIG. 1) for storing the filter function and a filter function calculation means (112 in FIG. 11) for converting the information passing through the directional link by the filter function. Have. Further, the output node has means (114, 115 in FIG. 1) for performing a functional operation on the information input via the directional link. The learning system of this network-type information processing device (11 in FIG. 1) includes an input unit (12 in FIG. 1) for inputting learning data to the network-type information processing device, and an area indicated by the input learning data. Area determination means (13 in FIG. 1) for determining whether or not the area is included in the existing recognition area
And a filter function updating means for individually updating each filter function in the set of filter functions forming the existing recognition area when it is judged by the area judging means to be included in the existing recognition area (see FIG. 1). 14) and a filter function generating means for generating a set of filter functions forming a new recognition area when the area determining means determines that the area is not included in the existing recognition area (FIG. 1).
15) and with a basic configuration. An existing recognition area is an area limited by an existing set of filter functions.

In the above basic configuration, the area determining means determines whether the area (spatial coordinates in the multidimensional area) indicated by the input data belongs to the recognition area limited by the existing filter function set of the network type information processing apparatus. To judge. As a result of the determination, the filter function updating means individually assigns each filter function of the filter function set corresponding to the recognition area so as to meet the purpose of the network type information processing device. To update. When it is determined that the recognition area does not belong to the recognition area, a new recognition area is set by the filter function generating means.
As a result, the recognition regions can be separated so that a set of filter functions in the same hierarchy cuts a multidimensional region axially asymmetrically. Therefore, large and small recognition areas are naturally generated and small rules are generated in the boundary area, so that the recognition rate is increased. In the conventional RCE method, which is a learning method for a neural network, virtual elements having energy (coulomb force) radii similar to charges are successively arranged in a new multidimensional region, and the radius is close to the data input. The learning is executed by expanding the radius if there is a certain element, and arranging a new element if there is no element, but the present invention is different from this. That is, the present invention is not a space division method such as the RCE method in which a sphere or a hypersphere is set as a new area, but a distribution such that an axis can be set independently of an area such as an ellipsoid or a cone of one-sided It can be formed (the area is not limited to a sphere or a hypersphere like the RCE method), and during the learning period, the central value of the area of the present invention is theoretically free and can move in space (the RCE method). It is not the same as the central value in principle).

In the present invention, the data used for learning is composed of input data and a teacher signal for the data except for the case of unsupervised learning described later. Generally, the teacher signal is "what is the result inferred from the input data?"
Although it has a positive meaning, in the present invention,
It is possible to use the one that has the negative meaning of "there is no result inferred from the input data". In systems with complex and unclear objects, it may be advantageous to use a negative output teacher signal. In order to deal with such a case, the present invention uses a negative teacher signal and generates a recognition area representing the negative. In the configuration of the invention in this case, the input means for inputting the learning data including the input data and the corresponding positive or negative teacher signal to the network type information processing device, and the area indicated by the input data are the same as the existing positive data. Area determination means for determining whether or not the recognition area is included in the recognition area that represents the negative or the recognition area that represents the negative, and the area determination means determines that the area is included in the existing recognition area that represents the affirmation or recognition area that represents the negative. In this case, a filter function updating means for individually updating each filter function in the set of filter functions forming the existing recognition area, and a recognition area representing the existing affirmative or negative recognition area by the area determining means. If it is determined that the teacher signal does not include any of the above, and if the teacher signal indicates affirmation, the fiducial area that forms a new affirmation region is formed. Generates a set of data functions, also includes a filter function generating means for generating a set of filter functions that form the recognition region representing a new negative when the teacher signal are representative of a negation. Further, for post-processing, it is convenient to unify the outputs of the recognition areas to be positive. Therefore, it is preferable to provide means for converting the negative output node into the positive output node by referring to the positive output node having a region adjacent to the negative output node.

In another aspect of the present invention, the filter function is generated and updated based on the evaluation with respect to the target value. That is, the configuration is such that, in a learning system of a network type information processing device, input data for inputting a teacher signal corresponding to the input data to the network type information processing device, and a teacher signal as a target value,
An evaluation unit that determines an error from the value of the result of network information processing of the input data and determines whether to update the existing region based on the error, and the existing recognition region should be updated by the evaluation unit. Filter function updating means for individually updating each filter function in the set of filter functions forming the recognition region when it is determined that there is,
And a filter function generating means for generating a set of filter functions forming a new recognition area when the evaluation means determines that the update is not possible.
According to this aspect, since the evaluation is appropriately performed and the filter function is generated and updated, the learning process is performed more efficiently.

According to one aspect of the present invention, it is possible to learn statistical analysis functions such as feature analysis and principal component analysis by performing learning without using a teacher signal. The configuration is such that, in a learning system of a network type information processing apparatus, input means for inputting learning data not including a teacher signal to the network type information processing apparatus, and input data by the network type information processing apparatus Determining means for determining an output node having an output with the maximum value of the result, and filter function updating means for individually updating each filter function in the set of filter functions connected to the output node determined by the determining means. And. The initial variation method of unsupervised learning is maximum and minimum of feature axis,
It suffices if the middle part is as uniform as possible. If all combinations are to be created, a rule of 2 ^an is required (n-dimensional, a is 1 if the maximum and minimum are the same), but this may be set appropriately.

In the basic configuration of the present invention, in a specific mode, the area determination means compares the degree of agreement obtained by the operation of the filter function operation means of the network type information processing device with a predetermined threshold value, According to the result, it can be configured to determine whether or not it is included in the existing recognition area.

In the learning system having the above basic structure, the filter function can be updated by selecting an update method according to the learning history. That is, in this case, based on the history storage means (171 in FIG. 1) for storing the learning history information for each recognition area, and the learning history information stored in the history storage means, a filter function is provided for each recognition area. As an update method of the update method, an update method instructing means (FIG.
18) and the filter function update unit has a plurality of update methods, and each filter function in the set of filter functions that forms the existing recognition region is individually updated by the update method according to the instruction of the update method instruction unit. Is configured to update. For example, when the number of history data is 1, as the first updating method, the central value C of the membership function is set to the initial input data, the ambiguity A is set to 0, and the history data number is 2 or more. When it is less than M (predetermined value), as a second updating method, the central value C of the membership function is set to an intermediate value between the maximum value and the minimum value of the history data, and the ambiguity A of the history data is set. Set based on the difference between the maximum and minimum values. When the number of pieces of history data is larger than a predetermined value M, a statistical method is used as the third updating method.

Further, according to still another aspect of the present invention, there is provided a configuration for using a statistical method as the updating method. That is, the frequency distribution measuring unit 17 for obtaining the frequency distribution from the learning history information is provided, and the filter function updating unit changes the filter function based on the frequency distribution obtained by the frequency distribution measuring unit.

The learning history information is collected for each filter function corresponding to the directional link, and only the input data determined by the region determining means to enter the region defines the region. Store as history data corresponding to a set of functions.

According to another aspect, after the learning for a certain period of time, whether the set of filter functions having extremely few teacher signals is noise based on the contents of the set of filter functions having adjacent regions. It is advisable to provide a noise removing means for determining whether or not there is noise and removing the rule when it is determined to be noise.

[0034]

【Example】

(Network-type information processing device to be learned) FIG. 2 is a diagram showing a schematic configuration of an embodiment of a network-type information processing system (device) to which the learning method of the present invention is applied.
In this network type information processing system, as shown in FIG. 2, a plurality of nodes 21, 22, 23 in the input layer and a plurality of nodes 24, 25 in the output layer are connected by directional links to form a network. FIG. 2 is formally similar to a conventional neural network, but is fundamentally different from the conventional one in that each node is connected through a directional link having a filter function.

FIG. 3 is a block diagram showing an example of the configuration of the portion for performing the calculation in FIG. The nodes shown in FIG. 3 are filter function operation units 31 to 34 that perform operation of filter functions that selectively pass input information in each directional link.
An adder unit 35 that performs addition and averaging processing by weighting the outputs of the filter function calculation units 31 to 34, and a threshold function calculation unit 36 that performs a threshold calculation on the output of the addition unit 35. The relationship between the input and the situation (output) can be represented by a set of rules (matrix) as follows. Input 1 Input 2 Input 3 Situation 1 f ₁₁ <g ₁₁ > f ₂₁ <g ₂₁ > f ₃₁ <g ₃₁ > ... Situation 2 f ₁₂ <g ₁₂ > f ₂₂ <g ₂₂ > f ₃₂ <g ₃₂ > ... Situation 3 _{f 13 <g 13> f 23} <g 23> f 33 <g 33> ... · · · · · · · · However, f ₁₁ ~f ₃₃ is a membership function, respectively. In addition, g ₁₁ ~g ₃₃ is the "weight" of each of the membership function. Note that setting the weight is not essential.

In this embodiment, as shown in FIG. 4, the filter function has a trapezoidal trapezoid of an unequal side in a graph in which the horizontal axis represents the value (input value) of the input information given to the input node and the vertical axis represents the degree of matching. A membership function with a shape (fuzzy membership function) is used. The membership function is the center value c which is the center value of the input that obtains the maximum matching score, and the left variance value v that represents the allowable range of the input value that obtains the matching score v as the width from the center value c to the left and right. _{It is described by l} and the right variance value v _r , and the ambiguity a that indicates the allowable range of the input value that gives the maximum matching degree with the width from the central value c. The filter function calculation units 31 to 34 are for calculating the membership function, that is, converting the input value s into the matching degree v. An equation for converting the input value s into the matching degree v can be expressed as follows. v = 0.0 {s ≦ (c−v _l ) or (c + v _r ) ≦ s} v = 1.0 {(c−a) ≦ s and s ≦ (c + a)} v = ((v _l −a ) - (c-a-s )) / (v-a) {(c-v l) ≦ s and s ≦ (c-a)} v = ((v r -a) - (s-c-a )) / (V−a) {(c + a) ≦ s and s ≦ (c + v _r )} (14)

The adder 35 includes filter function calculators 31 to 31.
An arithmetic mean calculation is performed in order to combine the outputs calculated by S34. In the present embodiment obtains the final match degree by comprehensively a value obtained by multiplying the weight to the matching degree v _1j to v _ij of input information I ₁ ~I _i by each membership function. Although various methods of calculating the final match degree are conceivable, in the present embodiment, the weighted addition average as shown in Expression (15) is used. V _j = Σv _ij · g _ij / Σg _ij (15) where V _j : the degree of matching of the pattern (or rule) j v _ij : the degree of matching by each membership function (= f _ij (I
_i ) I _i is an input value) g _ij : Weight of each membership function N _s : Number of input data Note that Σ represents the sum total from i = 1 to N _s .

(Embodiment of Learning System) Next, an embodiment of the present invention for learning the above-described network type information processing apparatus will be described. In the learning, a set of input data and a teacher signal representing a correct output signal for the input data is prepared in advance as learning data in the network information processing apparatus of the learning target shown in FIG. Type information processing apparatus 11 and outputs an error to the teacher signal to detect an error by an evaluation function. The error is detected as an input of a learning function (rule), and a backpropagation learning method is used to detect a directional link filter function or weight. By changing. FIG. 1 is a block diagram showing the function of performing learning in this embodiment. The network-type information processing apparatus 11 to be learned, the learning data input section 12 for inputting learning data, and the output of the filter function calculation section 112 of the network-type information processing apparatus 11 are specified by the teacher signal. An area determination unit 13 that determines whether or not input data enters an existing rule (area) corresponding to an output node, and if the area determination unit 13 determines that the input data is included in the existing rule, the existing rule A filter function updating unit 14 for individually updating each filter function in the set of filter functions forming a rule
And a filter function generation unit 15 that generates a set of filter functions forming a new rule when the area determination unit 13 determines that the rule is not included in the existing rules,
It has the evaluation part 16 which evaluates learning. Further, an update method designating unit 18 for switching the filter function to an appropriate update method according to the number of input data, and a frequency distribution measuring unit 17 for obtaining frequency distribution data for updating the filter function by a statistical method. Is equipped with. The frequency distribution data is obtained by analyzing the contents of the history buffer 171.

(Automatic Rule Generation Based on Teacher Data) FIG. 5 is a diagram showing a schematic flow of automatic rule generation with teacher data. Data consisting of a fixed number of input data for learning and teacher data corresponding thereto is input from the data input unit 12 to the network type information processing device 11 (step S51). Automatic rule generation processing is performed based on the input data (step S52). The procedure of automatic rule generation processing will be described later. At the stage where learning has progressed, noise removal processing is performed if necessary (step S53). Then, an evaluation process described later is performed (step S54). It is determined whether the average error has become smaller than the threshold value (step S55), and if it has become smaller, the process ends. If the average error is larger than the threshold value, the evaluation data for reuse is input as learning data (step S56), and the rule is automatically generated (step S57).

Step S51 in the processing flow of FIG.
FIG. 6 shows details of the automatic rule generation processing in S57 and S57. As shown in the figure, input data and teacher data are input to the network type information processing device 11 (step S61). A pattern table that stores a set of rules (patterns) that the network-type information processing device has is searched, and all the rules that have the same output as the teacher data are selected (step S62). The pattern table holds rule data in a format as shown in FIG. That is, the pattern table stores the rule name, the normal recognition result (area), the parameter of the filter function of each axis of the feature dimension corresponding to the rule, and the like. FIG. 8 shows an example of rules and regions in the feature space. And
The input learning input data is collated with one of the selected rules (step S63). That is,
For each dimensional filter function of the selected one rule (pattern), the matching degree of the dimensional data corresponding to the input data is obtained. The area determination unit 13 determines whether or not the obtained matching degree is larger than a predetermined threshold value (step S64). When the degree of matching is larger than the threshold value, the filter function of each dimension on the rule is changed (step S65). That is, as shown in FIG. 9, the rule in which the degree of agreement is equal to or higher than the threshold and the average of the degree of agreement is highest in all dimensions is changed to be expanded. The procedure for changing the filter function will be described later with reference to the flowchart of FIG. When the degree of matching is smaller than the threshold value, the filter function is not changed. It is determined whether any of the rules selected in step S62 has not been matched against the input data (step S6).
6). If there are still rules, step S63-
Repeat S66. For all selected rules,
If all the matching has been completed, it is next checked whether or not there is a rule that exceeds the threshold value (step S67). If no rule exceeds the threshold value, a new set of filter functions is created and registered as a new rule in the pattern table (step S68). That is, if there is at least one dimension for which no matching degree equal to or greater than the threshold value is obtained for any rule, the filter function generating unit 15 adds a new rule. FIG. 10 is a diagram showing the generation of the new rule. Then, the filter function of the generated rule is changed as described later (step S69), and the process ends. When it is determined in step S67 that there is a rule that the threshold values are exceeded in all dimensions, the process ends.

(Generation and Change of Filter Function) Steps S65 and S69 in the process of FIG. 6 change the filter function (represented by a membership function) of each dimension constituting the rule as shown in the flowchart of FIG. .. The data input to the system of FIG. 2 is stored in the history buffer (step S111). The history buffer holds the data of each dimension input to one rule, and also has the capacity to store the history of a predetermined M number of inputs. A history buffer is prepared for each rule,
A counter is provided for counting the input data for each history buffer. The counter that counts the number of input data in the history buffer increments the value of the counter by 1 each time it is input (step S112). Next, it is determined whether or not the value of the counter is 1 (step S113), and if 1
If so, the filter function updating unit 14 updates the parameters of the filter function as follows (step S11).
4). That is, the update method is switched according to the number of history data to perform the update. Note that N: number of history data (number of sampling data) M: history buffer size X ₀ : initial input data X ₁ : minimum value of history data X ₂ : maximum value of history data R ₁ : minimum value of regular space R ₂ : normal Spatial maximum value C: central value A: ambiguity V: dispersion value

A) In the case of N = 1 C = X ₀ A = 0 V = Set as information in learning processing FIG. 12 shows an example of the set filter function.

When the value of the counter is determined to be 2 or more, it is determined whether or not the value of the counter exceeds a predetermined number M (step S115). The result of the judgment M
If not, the filter function is updated by the following method (step S116).

B) In the case of 2 <N <M C = (X ₁ + X ₂ ) / 2 A = (X ₂ −X ₁ ) / 2 V = A + 2A / N FIG. 13 shows an example of the set membership function. It is shown. The cases a) and b) above are cases where a small amount of data that does not have statistical significance is targeted in the first place, and the learning method must rely on an intuitive non-intuitive method rather than statistically. I don't get it. This is an example of a method that can judge that data capture and evaluation are reliable by reflecting the characteristics of fuzzy.

If the result of determination in step S115 is that the counter value exceeds the number of history buffers, it is changed by statistical processing as described below (step S11).
7). As a method of setting by statistical processing, the method disclosed in the above-mentioned Japanese Patent Application No. 3-310082 "Network type information processing system" can be used. c) Changes by statistical processing During a certain observation period, the input data that matches the applicable pattern is tabulated and used as the population. FIG. 15 is an example of a graph in which the horizontal axis represents the quantization level Z of the input data and the vertical axis represents the number of data generations G for each level, showing an example of the data generation distribution. Note that G _c is a cutoff level for removing noise components and the like of the input data. When the number of elements included in the population reaches a certain number, the membership function, which is a filter function, is changed. A membership function is derived from the population according to the following procedure.

FIG. 14 is a flow chart of a membership function parameter extraction process. It is determined whether or not the cutoff data G _c is set as the output data designation (step 141). If the cutoff level G _c is set, the distribution data is manipulated by obtaining a value obtained by subtracting the cutoff level for each quantization level Z _i (step 142). That is, Z _i −G _c is obtained and set as a new Z _i . FIG. 16 shows the result of processing the input data distribution of FIG. 15 at the cutoff level G _c . The arithmetic processing for truncating the data below the cutoff level is performed until all the quantization levels of Z are completed. For this reason, it is determined whether or not all the quantization levels have been calculated each time the calculation of each quantization level is completed (step 143). When it is completed for all the quantization levels, the routine proceeds to step 144. Further, if it is determined in step 141 that the cutoff level is not set, the process also proceeds to step 144.

Average value of distribution data Normalized coordinate value Z_mSeeking
(Step 144). Z_m= Σ (Z_i× G_i) / ΣG_i  However, Σ represents the total sum from i = 0 to i = n
To do. Average value Normalized coordinate value Zm minus side, plus
Standard deviation value S independently for each side_l, S_rAsk (step
145). Average value normalized coordinates obtained in step 144
Value Z_mAnd the negative standard deviation obtained in step 145
Difference value S_l, Plus standard deviation S_rBased on
The converted coordinate value is obtained (step 146). That is, the average value is positive
Normalized coordinate value Z_mMainly, as shown in FIG.
Standard deviation S_lNormalized coordinate value of 1 times (*)
Z_L1, Negative standard deviation S_l3 times (Note *) regular
Coordinate value Z_L2, Plus side standard deviation value S_r1 time (Note *)
Normalized coordinate value Z_R1, Plus side standard deviation value S_r3 times
(*) Normalized coordinate value Z_R2, (Note *: This value changes depending on the conditions).

The normalized center value C _s , the normalized ambiguity V _as , and the normalized variance values V _ls , V _{rs as} shown in FIG. 18 are obtained by the following equations (step 147). Normalized center value C _s = (Z _L1 + Z _R1 ) / 2 Normalized ambiguity V _as = (Z _R1- Z _L1 ) / 2 Normalized left variance value V _ls = C _s- Z _L2 Normalized right variance value V _rs = Z _R2- C _s Then, the normalized center value C _s , the normalized ambiguity V _as , the normalized left variance value V _ls , and the normalized right variance value V _rs are denormalized to obtain the central value C, Ambiguity V _a , left variance V _l , right variance V _r
(Step 148). Using the original membership function before the change by learning and the membership function generated by the processing flow shown in FIG.
Generate a membership function (after learning).

FIG. 19 shows the member table in this embodiment.
It is a figure which shows the method (learning function) of changing the loop function. same
In the figure, the original (current) membership function is the point P
₁, P₂, P₃, P_FourIndicated by a group of straight lines (thick line) connecting
Obtained by sampling for a certain period of time, which was obtained from Fig. 14.
The membership function based on the data distribution₁ ^', P
₂ ^', P₃ ^', P_Four ^'Indicated by a group of straight lines (broken line) connecting
Newly generated members based on these membership functions
Barship function is point P₁ ^", P₂ ^", P₃ ^", P_Four ^"Straight line connecting
Shown by groups (thin lines). P for each four points
(S, v), P^'(S^', V^'), P^"(S^", V^")
It v^"= V^'= Vs^"= (1.0-g) xs + gxs^'  However, 0.0 ≦ g ≦ 1.0, the gain value g at each of the four points can be set independently.

The result of changing the membership function differs depending on the same observation data depending on the setting of the gain value at each point. FIG. 20 shows how the change result changes depending on how to determine the gain. FIG. 20A shows
The current membership function (thin solid line) and the membership function based on the distribution obtained from sampling for a certain period (dashed line) are shown. FIG. 11B shows a membership function (thick solid line) generated when the gain value g at each point is set to 0.5. Further, FIG. 6C shows a case where the gain value g of the point P ₁ is g = 0 and the gain value g of the other points is g = 0.5, and the same figure (d) shows the ambiguity (P ₂ and P The distance between ₃ ) is not changed, and the gain value in the bottom expanding direction is set to g = 1.0, and the gain value in the bottom contracting direction is set to g = 0. The gain value can be set and the directionality of learning can be changed depending on the characteristics of the sensor that is the input signal source, the intention of learning, and the like.

As described above, when the teacher advances, noise removal processing is performed as necessary, which is performed as follows. a) Check N (number of historical data) of membership function of each rule. This can be done by looking at the value of the counter provided for each history buffer. b) In the case of N <the number of boundary data, if one or more rules in the same recognition area are adjacent to each other, the rule weight is not changed. That is, the rule weight does not change. Note that the rules are adjacent to each other means that the bases (dispersion values) of membership functions of a certain dimension overlap with each other. If no rules in the same recognition area are adjacent, the weight of the rule is lowered. That is, the weight of the membership function of each dimension is reduced. FIG. 21 is for explaining an example of noise removal. In FIG. 21, points are teacher data, and areas surrounded by circles are rules. These rules form an area for recognition. It The rule 1 and the rule 2 in the figure are rules with extremely few teacher signals. Regarding such a rule, the rule weight is not changed for the rule 1 adjacent to the rule in the same recognition region, and the rule weight is the same. For rule 2, which has no adjacent rules in the recognition region, the rule weight is lowered.

(Negative Teacher Signal Learning) It may be convenient to carry out learning using a negative teacher signal. For example, in a plant system, it is not possible to give affirmative information such as the occurrence of a failure of XX to the situation represented by a certain sensor information group, but at least it is not an event of fire occurrence. It may be possible to give side information. In such a case, it is useful to teach using the information on the negative side. a) Input data (sensor information) and a teacher signal representing a negative recognition result (not, for example, notA),
When input to the network type information processing device for learning, rules are generated in the same manner as a positive (that is, not negative) teacher signal. b) In the learning process using the teacher data, the normal recognition result (for example, notA →
When one or more rules of B or Cor ... Are adjacent, the negative rule is also determined to be the rule of the same recognition area as the rule of the adjacent normal recognition result. FIG. 22 is a diagram showing a processing flow for converting a rule generated by a negative teacher signal into a positive output rule.
It is checked whether or not the rule based on the normal teacher signal is adjacent to the rule based on the negative teacher signal (step S221). If the rule based on the normal signal is adjacent to the rule, the negative rule is converted into the same area as the adjacent normal rule (step S222). FIG. 23 shows an example in which a rule based on a negative teacher signal is converted into a positive rule that is not negative, and a region B (or C ...) Other than A around notA.
If even one rule with is adjacent, notA is B
Change to (orC ...). If there is another negative rule, steps S221 to S223 are repeated for the negative rule. When there is no negative rule, it ends. (Example) Recognition result X axis Y axis Z axis Rule 1: notA MF1 MF2 MF3 ↓ Rule 1: B MF1 MF2 MF3

(Example of Reuse Learning of Evaluation Data)
After the completion of a certain number of learning, in order to evaluate the degree of learning,
Give evaluation data. Like the learning data, the evaluation data also includes sensor information (input data) and a normal result. The evaluation unit 16 obtains the difference (error) between the information processing result (inference result) of each input data and the normal result given for evaluation, and evaluates it by the evaluation function to judge pass / fail.
These pass / fail are integrated for each rule, and the distribution of integration is evaluated when all the information processing of the input data for evaluation is completed. This makes it possible to judge the appropriateness of the area division. The learning ends when it is judged to be appropriate. According to this evaluation method, the evaluation results are accumulated in the feature space and finally divided into regions, so there is no risk of falling into the local minimum minimum solution. Therefore, the evaluation function does not need to be linear and can be used freely. As a peculiar case, in the case where it does not match any rule of the pattern table (applied rule = none, matching degree = 0),
Reuse evaluation data as teacher data. Figure 24 shows
It shows a processing flow of evaluation processing and reuse of evaluation data. a) The rule of the pattern table is applied to each evaluation data (step S241). b) With respect to the evaluation data that should be a normal recognition result (matching degree = 1.0), there is no application rule = matching degree = 0.
0, error = 1.0) is obtained (steps S242 to S245). c) Using the evaluation data corresponding to b) as teacher data,
Teacher learning is performed (step S246). d) After the completion of c), the learning with the teacher data is restarted. FIG. 25 is a diagram for explaining reuse of evaluation data. When there is no rule corresponding to the evaluation data at the position of the X mark in the area A, learning is performed by using the evaluation data as teacher data, You can make up for the missing parts.

(Example in which learning is performed without teacher data) In this example, the basic configuration is the same as that shown in FIG. FIG. 26 is a block diagram showing its function, which is different from FIG. 1 in that a maximum output node determination unit 261 is provided instead of the area determination unit. Note that in FIG.
The same reference numerals are used for parts having the same functions as. Modification (learning) of the internal state is performed by the following procedure.
FIG. 27 shows a processing flow. a) A membership function (= band-pass function) appropriately distributed on the link between the input / output nodes.
Is defined. (Depending on the matching degree synthesis method, the weight on the link is also set.) B) The input data string is input to the network type information processing device 11 as in the case of normal learning (step S27).
1). However, unlike the case of normal learning, no teacher data is input. c) Arithmetic processing (inference) is performed to obtain an output (step S272). d) Among the output nodes, the node with the largest output is selected (step S273). e) As shown in FIG. 28, the output value of the output node is positive (for example, 1.0), and the output values of the other output nodes are false (for example, 0.0) to perform the teacher (step S27).
4). f) The filter function is changed by the predetermined learning method as described in the embodiment of FIG. 1 (step S275). Note that the weight on the link is also changed depending on the matching degree combining method.

According to this embodiment, since the modification of the internal state progresses in the direction of emphasizing the output value for a certain specific input, each characteristic describes some characteristic component of the input information data. This is equivalent to the non-linear analysis in a broad sense. In this network itself, nonlinear analysis is performed on the input by learning, and the analysis result (=
Network) to convert the input data into features. It is possible to replace this with a preprocessing system that has been conventionally performed by statistical analysis or the like, and obtain a system as shown in FIGS. An example of one operation form will be described below. First, a network for learning without teacher data is created to allow learning without teachers. When a certain amount of learning (for example, inputting a certain number of data, using the network as a constant convergence index) is completed, the network is fixed (learning is stopped). It is connected to a network that performs actual output processing, and supervised learning is performed. The connection between the learning network without the teacher data and the normal network with the teacher is as shown in FIG. 29 in which the learning network without the teacher data coexists, or the learning network without the teacher data is used as a complete preprocessing system in FIG. It can be implemented in various forms such as the above form.

[0056]

【The invention's effect】

1) According to the present invention, a small rule is generated in the boundary region and the recognition rate is increased by the learning process of generating and updating the filter function. 2) According to the present invention, large and small rules are naturally generated, so that the recognition rate can be increased without dividing the grid as in the conventional case. 3) According to the present invention, when the data in the area where the rule is not generated in the evaluation processing is used again as the teacher signal, it is possible to surely create the rule in the area where the rule is not generated. .. The evaluation data can be reused and the learning speed can be increased. 4) According to the present invention, by performing the learning process without the teacher data by using the network type information processing system, it is possible to perform the nonlinear analysis with a plurality of input data. It is possible to construct a pretreatment system and reduce the preparation time and workload of the pretreatment system.

[Brief description of drawings]

FIG. 1 is a functional block diagram of a learning system of a network type information processing system according to the present invention.

FIG. 2 is a diagram showing a configuration of a network type information processing system.

FIG. 3 is a block diagram showing a configuration example of a portion for performing calculation in FIG.

FIG. 4 is a diagram for explaining a fuzzy membership function.

FIG. 5 is a flowchart showing an outline of learning processing by a teacher signal.

FIG. 6 is a flow chart showing details of automatic rule generation processing in the flow of FIG.

FIG. 7 is a diagram for explaining a rule format.

FIG. 8 is a diagram for explaining a recognition area in the present embodiment.

FIG. 9 is a diagram for explaining expansion of rules.

FIG. 10 is a diagram for explaining rule generation.

FIG. 11 is a flowchart showing a filter function changing process.

FIG. 12 is a diagram showing a filter function set when the number of history data is 1.

FIG. 13 is a diagram showing a filter function set when the number of history data is 2 or more and less than M.

FIG. 14 is a flowchart of parameter extraction processing of membership function.

FIG. 15 is a diagram showing an example of a graph showing an example of a data generation distribution in which the horizontal axis represents the quantization level Z of input data and the vertical axis represents the number of times data is generated for each level.

FIG. 16: Cutoff G of the generation distribution of the data in FIG.
The figure which shows the data distribution after cutting off by c

FIG. 17 is a diagram for explaining calculation of each normalized coordinate value.

FIG. 18 is a diagram showing a membership function obtained from the data distribution of FIG.

FIG. 19 is a diagram for explaining a basic method (learning function) for changing the membership function in the present embodiment.

FIG. 20 shows how the result of changing the membership function changes depending on how to determine the gain of the learning function. (A) shows the current membership function (thin solid line) and sampling for a certain period. Shows a membership function (broken line) based on the distribution obtained from, (b) shows a membership function (thick solid line) generated when the gain value g at each point is 0.5, and (c) ) Shows the case where the gain value of the point P1 is g = 0 and the gain value of the other point is g = 0.5, and (d) is the base expansion without changing the ambiguity (distance between P2 and P3). Direction gain value g = 1.0 and bottom side reduction direction gain value g = 0 are shown.

FIG. 21 is a diagram for explaining noise removal.

FIG. 22 is a flowchart of a process of converting a rule of negative teaching to a rule of positive teaching.

FIG. 23 is a diagram for explaining conversion of a rule of negative teaching into a rule of positive teaching.

FIG. 24 is a flowchart of evaluation processing

FIG. 25 is a diagram for explaining reuse of evaluation data.

FIG. 26 is a functional block diagram of an embodiment for performing learning without teacher data.

FIG. 27 is a processing flowchart of learning without teacher data.

FIG. 28 is a diagram for explaining learning without teacher data.

FIG. 29 is a diagram showing an example of a network-type information processing device to which learning without teacher data is applied.

FIG. 30 is a diagram showing another example of a network-type information processing device to which learning without teacher data is applied.

[Explanation of symbols]

11 ... Network-type information processing device, 12 ... Learning data input unit, 13 ... Region determination unit, 14 ... Filter function updating unit, 15 ... Filter function generating unit, 16 ... Evaluation unit, 17 ...
Frequency distribution measuring unit, 171 ... History buffer, 18 ... Update method designating unit.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Yoichi Ueishi Inventor Yoichi Ueishi 1-15-8, Jinnan, Shibuya-ku, Tokyo, 4th floor, Ken Naka Building, Add-in Research Institute Co., Ltd. (72) Toshihide Fujimaki Kita, Chuo-ku, Sapporo, Hokkaido Article 2 Nishi 2-chome 1 Hattori Building 3rd floor Inside Add-in Research Institute Sapporo Laboratory

Claims (10)

[Claims] 1. A plurality of input nodes, a plurality of output nodes, and a directional link connecting the input node and the output node; the directional link storing a filter function which is a non-linear selective function. Filter function storage means for
Network type information having filter function operation means for converting information passing through a directional link by the filter function; the output node having means for performing function operation on information input via the directional link In a learning system of a processing device, an input means for inputting data and a teacher signal for the data to the network type information processing device, and a region for determining whether or not an area indicated by the input data is included in an existing recognition area Determining means, and a filter function updating means for individually updating each filter function in a set of filter functions forming the existing recognition area when it is determined by the area determining means to be included in the existing recognition area, , A new recognition area is formed when it is judged by the area judgment means that the recognition area is not included in the existing recognition area. And a filter function generating means for generating a set of filter functions. 2. A plurality of input nodes, a plurality of output nodes, and a directional link connecting the input node and the output node; the directional link storing a filter function which is a non-linear selective function. Filter function storage means for
Network type information having filter function operation means for converting information passing through a directional link by the filter function; the output node having means for performing function operation on information input via the directional link In the learning system of the processing device, an input means for inputting learning data including input data and a corresponding positive or negative teacher signal to the network type information processing device, and an area indicated by the input learning data is An area determination unit that determines whether or not the area is included in an existing recognition area that represents affirmation or a recognition area that represents a negative, and the area determination unit includes an existing recognition area that represents affirmation or a recognition area that represents a negative recognition area. If so, each filter function in the set of filter functions that forms the existing recognition region is individually updated. Filter function updating means and the area determining means determines that the teacher signal does not belong to any of the existing recognition area indicating affirmation or the existing recognition area indicating negation, and if the teacher signal indicates affirmative, a new A filter function generating means for generating a set of filter functions forming a recognition region expressing affirmation, and generating a set of filter functions forming a recognition region expressing a new negation when the teacher signal represents negation; A learning system for a network type information processing device, comprising: 3. A network type information processing apparatus according to claim 2, further comprising means for converting a negative output node into a positive output node by referring to a positive output node having a region adjacent to the negative output node. Learning system. 4. A plurality of input nodes, a plurality of output nodes, and a directional link connecting the input node and the output node; the directional link storing a filter function which is a non-linear selective function. Filter function storage means for
Network type information having filter function operation means for converting information passing through a directional link by the filter function; the output node having means for performing function operation on information input via the directional link In a learning system of a processing device, an input means for inputting a teacher signal corresponding to the input data to the network type information processing device, a teacher signal as a target value, and an error between the value obtained as a result of the network information processing of the input data are calculated. An evaluation unit that determines whether or not to update the existing region based on the error, and forms the recognition region when the existing recognition region is determined to be updated by the evaluation unit Filter function updating means for individually updating each filter function in the set of filter functions; A learning system for a network-type information processing device, comprising: a filter function generating means for generating a set of filter functions that form a new recognition region when determined. 5. A plurality of input nodes, a plurality of output nodes, and a directional link connecting the input node and the output node; the directional link storing a filter function which is a nonlinear selective function. Filter function storage means for
Network type information having filter function operation means for converting information passing through a directional link by the filter function; the output node having means for performing function operation on information input via the directional link In a learning system of a processing device, an input unit for inputting learning data not including a teacher signal to the network type information processing device, and a maximum value of a result obtained by performing network information processing on the input data by the network type information processing device. And a filter function updating unit that individually updates each filter function in the set of filter functions connected to the output node determined by the determining unit. Learning system for network type information processing equipment. 6. The area judging means compares the degree of matching obtained by the calculation of the filter function calculating means of the network type information processing device with a predetermined threshold value, and judges the recognition area based on the result. A learning system for a network type information processing apparatus according to claim 1 or 5. 7. A history storage means for storing learning history information for each recognition area, and based on the learning history information stored in the history storage means when it is determined to update an existing recognition area. , Update method instructing means for selecting and instructing an appropriate update method from a plurality of types of update methods as update methods of the filter function for each recognition area, and the filter function updating means determines the update method selecting means. 6. The network according to claim 1, wherein each of the filter functions in the set of filter functions forming the existing recognition region is individually updated by using the update method described above. System for learning type information processing equipment. 8. A frequency distribution measuring means for obtaining a frequency distribution from history information stored in the history storage means is provided, and the filter function updating means uses a statistical method using the frequency distribution obtained by the frequency distribution measuring means. The learning system for a network type information processing apparatus according to claim 1, wherein the filter function is updated. 9. After completion of learning for a certain period, it is determined whether or not there is noise in a set of filter functions having extremely few teacher signals, based on the contents of the set of filter functions having adjacent regions, and noise is detected. 9. The learning system for a network type information processing apparatus according to claim 1, further comprising a noise removing unit that removes the rule when it is determined to be true. 10. An area in which a set of filter functions is not generated, comprising means for deriving an error using the evaluation data and checking the end of learning, the evaluation data is not used only for deriving the error. 10. The learning system for a network-type information processing device according to claim 1, further comprising an evaluation data reuse unit that is also used as learning data for.