KR100361168B1 - Neural network model with node-expansion learning - Google Patents

Abstract

The present invention relates to a node extended learning neural network model formation and learning method used for pattern recognition.

The present invention utilizes a set of sample vectors belonging to pattern classes with complex cluster shapes in a multidimensional hyperplanar feature space, and does not fix the number of hidden layer nodes, that is, inner node nodes, which are necessary to form an appropriate crystal boundary. By creating and destroying nodes accordingly, we construct neural networks of appropriate scale.

To this end, each inner end node forms a representative vector representing clusters in the pattern space. If there are many sample vectors similar to the generated nodes, the learning strength of the nodes is enhanced. On the other hand, if there are many sample vectors belonging to other classes in the adjacent position, the learning strength of the node is weakened. If the learning strength drops below a proper level, the node is destroyed.

According to the present invention, it is possible to appropriately control the number of hidden layer nodes required according to a set of sample vectors during learning, and to dynamically control the learning rate of each node .

Description

Node extension learning neural network model formation and learning method {Neural network model with node-expansion learning}

The present invention relates to an artificial neural network (hereinafter referred to as a neural network) model and a learning method thereof, and more particularly, to a method for forming and learning a node extended learning neural network model. Neural networks attempt to overcome the difficulties of intelligent processing experienced by existing methods by modeling the cognitive processes of living organisms. Neural network models are used in various fields such as pattern classification and recognition, signal processing, and coding.

In particular, the neural network model for classifying and recognizing patterns of a predetermined category includes BP (back propagation), which is a learning model of multilayer perceptron, and Kohonen's Self-Organizing Feature Map, RCE model, which is a competitive learning model.

BP repeatedly applies delta rules to minimize the mean square error of the output of the multi-layer feedforward perceptron and the desired output. Multilayered perceptrons learned by BP have different types of crystal regions that can be distinguished according to the number of layers. In the case of a two-layer structure, a typical convex region can be studied. . The BP algorithm has been recognized for its successful learning in various fields such as signal processing and pattern recognition, but it is difficult to determine the number of internal nodes required when the number of classes and the complexity of decision domains are complicated. There is a problem that it is difficult to converge values. In addition, if a new pattern is added to the already learned neural network, the previously learned content may be lost, so there is a problem that all the patterns must be re-learned at once.

The Self-Organizing Feature determines the representative vector of each node to minimize the total quantization error to approximate the input signal. However, the pattern recognizer classifies the input signal into a finite number of classes, and it is important for the representative vectors to represent each class well. In addition, it is required to conduct supervised learning in which class each sample signal corresponds to, namely, the desired output. Kohonen's suggestion for this is a teaching-learning vector quantizer (LVQ).

In LVQ, a predetermined number of representative vectors representing a set of learning vectors are trained. The process of learning the representative vectors for each learning vector x is as follows. First find the representative vector m _c that most closely matches x. If m _c is the same class as the learning vector x, adjust it by (Equation 1).

On the other hand, if m _c is a different class from the learning vector x, this is adjusted by (2).

Here, t is the time axis, and a (t) is a value that adjusts the amount of shifting the representative vector in the learning vector direction or the opposite direction. 0 <a (t) <1 and decrease monotonically with t. However, the number of representative vectors required to train with a given learning data depends on the type of sample cluster. If the number of classes is large and the class shape in the feature space is not simple, it is difficult to estimate the number of representative vectors needed to form an appropriate decision boundary. Therefore, a lot of trial and error are required to construct an efficient neural network. Accordingly, a model having a fast learning time and a representative vector can be actively formed by adaptively coping with the form of the class.

Therefore, an object of the present invention is to provide a method for forming and learning neural network model that can be usefully used for pattern classification and recognition. The model formed according to the present invention is a neural network model having adaptability to various types of pattern classes and has the following characteristics.

① Convergence time required for learning is short, and convergence must be guaranteed.

② Since the number of inner-end nodes needed to represent the cluster type formed by each class in the feature space cannot be known in advance, the inner-end nodes should be self-organized as needed according to the training data. That is, the patterns of the actual application field are extremely rare in the distribution form in the feature space, and since the patterns are difficult to predict, the ability to form inner node according to the sample pattern is required.

③ The decision boundary to classify should be formed more accurately.

1: Neural network structure diagram of the present invention

2 is a conceptual diagram showing a process of adjusting learning intensity

3 is a conceptual diagram of a process of obtaining an adjustment amount of learning intensity

4: Conceptual diagram of the process of learning the representative vector by the sample vector

The neural network model proposed in the present invention is based on a learning vector quantizer (LVQ) model. Unlike the LVQ model, however, it is possible to form an appropriate number of representative vectors according to the type of cluster to be trained, and to adjust the adaptive learning rate according to how sure the position of the generated representative vectors is.

The structure of the neural network proposed in the present invention is shown in FIG. 1. Each inner end node is connected to all input terminals to form a representative vector. One representative vector may be referred to as a connection weight between one inner end node and input nodes. Each inner-end node has an active region, and operates only when the input learning vector belongs to its active region. Whether the learning vector belongs to the active region of the inner node is determined based on the calculation of the distance between the learning vector and the representative vector represented by the inner node. The distance D (x, m) between the learning vector x and the representative vector m is calculated using the Euclidean distance of (3).

During learning, if a learning vector belonging to the same class as the inner node is input to the active region of any inner node, the inner node is reinforced by positive stimulation. At this time, the degree of reinforcement is larger as the learning vector is closer to the internal node. On the contrary, when a learning vector belonging to a class different from that of the internal node is input to the active region of the internal node, the internal node is weakened by receiving a negative stimulus. In FIG. 2, when a sample vector x _A belonging to class A is input, a positive stimulus is applied to node A, and a negative stimulus is applied to nodes B and C. FIG. The strength of the internal node thus changed is defined as learning strength in the present invention. The greater the learning intensity, the more obvious the position of the representative vector represented by the node. Therefore, the less the amount of movement by the learning vector during the learning process is. On the other hand, as the learning intensity is smaller, the internal node may be regarded as uncertain, and thus the moving distance increases when the representative vector needs to be modified by the learning vector.

The learning intensity Ci of the internal node ni is adjusted by using Equations 4 and 5. (Equation 4) is when the internal node is positively stimulated, and (Eq. 5) is negatively stimulated. α is the radius of the active area. FIG. 3 is a diagram showing the learning intensity changed by the sample vector. One sample vector has an intensity of 1, and as the distance from the center point decreases, the intensity decreases with a slope of 1 / α.

Internal nodes are divided into two classes, active memory and standby memory, depending on the learning intensity. The nodes of the active memory are nodes that are stimulated with sufficient amount of learning, that is, the learning intensity is above a certain threshold. On the other hand, the nodes in the standby memory are nodes whose learning intensity is lowered below the threshold due to a lot of negative stimuli in the learning process. Nodes in the standby memory participate in the learning process but do not work when they are actually aware, and thus do not affect the formation of the decision boundary.

After a learning vector is presented and the adjustment of the learning intensity is completed, the position learning of the representative vectors is performed. Learning consists of supervised learning, where the representative vector is adjusted as follows according to the position of the learning vector x.

① If no internal node is active: Create a new node. The initial value of the learning intensity is 1.

② When only the inner node ni belonging to the same class as the learning vector is activated: The representative vector of the minimum distance inner node is adjusted by (Equation 6). Β i in Equation 6 is a learning rate indicating the degree to which the representative vector is adjusted, as defined by Equation 7. m i is the representative vector of ni . This process is shown in FIG.

③ When only the inner node nj belonging to a class different from the learning vector is activated: A new inner node ni corresponding to the class of the learning vector is created. The representative vector of the new internal node is the same as the learning vector, and the learning intensity Ci is as shown in (8). Cj is the learning intensity of nj .

④ can be activated while the internal nodes belonging to the same class as the learning vector, if it is activated during an internal node belongs to a different class: when the same class and the other classes the minimum distance inside the node in La Each ni and nj, ni and If the distance between the learning vectors is smaller than the distance between nj and the learning vector, the node is learned according to (6). Otherwise, if the dot product obtained as in (9) is less than 0, create a new node as in (3), otherwise ni is adjusted according to (6), and nj is according to (10). Adjust

As described above, the node-expanded learning neural network model proposed in the present invention forms internal nodes autonomously for pattern classes having various types of distributions in a feature space. In addition, each internal node manages its own learning intensity so that it operates as a normal internal node only when it is above a predetermined threshold, so that a more accurate crystal boundary can be formed even when noise is included or the crystal boundary is ambiguous. . This feature can be usefully used in various pattern recognition fields.

Claims (4)

(Correction) a) connecting all input terminals to each of the plurality of nodes of the inner end to which the active region is assigned to form a representative vector for each node; b) determining whether the learning vector belongs to an active region of the node based on the distance between the learning vectors input through the input terminal and the representative vector of one node; c) when the learning vector belongs to the active region of the node, increases the learning intensity of the node when the learning vector belongs to the same class as the node, and when the learning vector does not belong to the same class as the node. Reducing the learning intensity of the node to adjust the learning intensity of each node; And d) if the learning intensity is greater than the set value, increase the learning rate for modifying the representative vectors by the learning vector; if the learning intensity is less than the set value, reduce the learning rate, and perform position learning of each representative vector of the node. Node extended learning neural network model formation and learning method comprising a. (Newly made up) according to claim 1, In step c), When the learning vector belongs to the same class as the node , the learning intensity C _i of the node n _i is If the learning vector does not belong to the same class as the node Node extension learning neural network model formation and learning method that is variable according to the conditions of. Where α is the radius of the active area x: learning vector m: representative vector D (x, m): distance between the learning vector and the representative vector (Newly made up) according to claim 1, Step d) d-1) creating a new node if there is no active node; d-2) When only the node ni belonging to the same class as the input learning vector is activated, the representative vector of the minimum distance internal node is (here, silver Is, Is a representative vector of ni ); And d-3) a node belonging to a different class from the inputted learning vector nj If only active, a new node corresponding to the class of the learning vector ni Steps to generate Including; In step d-3), the representative vector of the newly generated node ni is the same as the learning vector, and the learning intensity Is Node extended learning neural network model formation and learning method, characterized in that to satisfy. (Newly set forth in paragraph 3), Step d), When there are activated internal nodes belonging to the same class as the learning vector, and there are activated internal nodes belonging to another class, when the minimum distance internal nodes of the same class and different classes are called ni and nj , If the distance between ni and the learning vector is less than the distance between nj and the learning vector Learning according to the node according to the; If the distance between ni and the learning vector is greater than the distance between nj and the learning vector, Creating a new node if the value of is less than 0; And remind If the value of is greater than 0, According to the learning process for that node Node extended learning neural network model formation and learning method comprising a.