JP2005284128A - Quick learning method for self-organization network - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To conduct a quick learning speed by appropriately providing learning data to a self-organization network. <P>SOLUTION: A situation frequently occurs that the winning node of the learning data in the early sequence is denied learning by the learning data in the late sequence. A range of values that learning data (n) satisfy so that learning (n-1) is not canceled by the learning (n) is derived to make a learning speed fast. Further, conditions are newly provided to impart learning data in increasing or decreasing order, and the learning speed can be made fast. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention is a kind of artificial neural network, and relates to a learning method for increasing the learning speed of a self-organizing network capable of unsupervised learning.

  The self-organizing network and its learning method are introduced in many documents such as “Introduction to Neural Computing” (ISBN4-303-72640-0). The outline is shown below. The self-organizing network has a two-layer structure having a number of input nodes equal to the number of dimensions of data, and in many cases having a number of output nodes in the form of a two-dimensional map. A collection of output nodes is called a feature map. Since there is a positional relationship between the output nodes, it is possible to select a certain output node and an output node in the vicinity by specifying the size of the neighborhood in advance. There is a connection between the input node and the output node, and a connection weight is given to all the connections. In other words, data having the same number of dimensions as the learning data is given to each output node as a collection of connection weights. This collection of connection weights is called a connection weight vector.

  In general, the initial value of the connection weight gives a random value.

  At the time of learning, each dimension value of the learning data is given to the corresponding input node, the distance between the learning data and the connection weight vector given to the output node is calculated for all the output nodes, Choose the node that is closest. The selected output node will be called the victory node.

  In general, the Euclidean distance is used as the distance.

  The value of the connection weight vector is adjusted so that the connection weight vector is close to the learning data for the winning node and the output node in the vicinity thereof. How much the value is adjusted is adjusted by a variable called learning rate. The learning rate is a function whose value is determined by the positional relationship within the neighborhood. One learning is completed by sequentially performing this operation for all the learning data. The network can be self-organized by repeating learning while gradually decreasing the size of the neighborhood and the learning rate.

  Next, determination of the end of learning will be described. There are methods for determining the end of learning, such as 1) predetermining the number of times of learning in advance, 2) determining according to the progress of learning, such as the farthest distance between the learning data and the winning node being equal to or less than a threshold. . In the case of 1), since the number of times of learning is constant, the number of times of learning is constant regardless of the progress of learning, and the degree of progress of learning at the end of learning differs for each learning. In the case of 2), the progress of learning at the end of learning is the same for each learning, and the earlier the learning progress, the faster the learning ends and the faster the learning speed.

  The progress of learning can be determined by the distance between the learning data and the winning node.

  Next, how to use the network after learning is described. The value of the connection weight vector is fixed at the end of learning. When the network is used, by selecting an output node having the closest distance between the input data and the coupling weight vector, it is possible to know which of the learning data is closest to, and to cluster the input data.

  Many of the conventional methods for increasing the learning speed of self-organizing networks are mainly optimizing parameters such as neighborhood size and learning rate. Improvement of the learning algorithm, initial value of the joint weight vector, and provision of learning data There are few studies on optimization.

  In Patent Document 1 listed below, the input order of learning data is adjusted for the purpose of increasing the learning speed. The difference between Patent Document 1 and the present invention will be clarified in the disclosure of the invention.

In Patent Document 2 listed below, a part of the initial value of the connection weight is adjusted, but this is for controlling the formation of the feature map and is not aimed at increasing the learning speed.
Japanese Patent Application 2003-86533 Japanese Patent Application 2003-114379

  Increasing the learning speed is a problem common to the method of automatically learning. An object of the present invention is to improve this problem and increase the learning speed by optimally providing learning data to a self-organizing network.

  Consider changes in connection weights during learning of self-organizing networks. A change in the connection weight of an output node is expressed by a recurrence formula of Formula 101 in FIG. When the neighborhood is large and all the output nodes are within the neighborhood for all the learning data, the range of n is the entire learning data in all the output nodes. When the neighborhood is small and only some of the output nodes are within the neighborhood, the change in the coupling weight of each output node is similarly expressed by Equation 101 if the range of n is considered as learning data that affects each output node. it can. By transforming the recurrence formula of Formula 101, Formula 103 showing the change in the connection weight by the initial value and the learning data is obtained. From Equation 103, it can be seen that when g = 0, there is no change in the connection weight, and when g = 1, the connection weight is always the same value as the last learning data. While the order of the learning data advances, the size of the neighborhood and the value of g are not changed. As the number of learning progresses, the size of the neighborhood and the value of g are reduced.

  When the learning is advanced, a situation occurs in which the winning node of the learning data in the early order enters the vicinity of the learning data in the later order, and the learning in the earlier order is canceled by the learning in the later order.

  In Patent Document 1, when the condition shown in Expression 202 in FIG. 2 is satisfied between learning data, the learning of n−1 is not canceled by the learning of n, and the state of Expression 203 is obtained. It is pointed out from the second term that it can be seen that the greater the distance between the target learning data, the greater the degree of cancellation, and the smaller the distance, the smaller the degree of cancellation. It is clearer when the equation 204 obtained by modifying the equation 201 is seen.

  In Patent Document 1, it is pointed out that the condition that the learning is not canceled is the case of Expression 203, but more precisely, it is the case of Expression 301 of FIG. Therefore, the condition of equation 202 requires a more precise discussion.

  Moreover, as pointed out in Patent Document 1, it is possible to infer the degree of learning cancellation by the distance between the learning data from the second term on the right side of Equation 201, but this is also a more precise discussion. is required.

  A condition for satisfying the expression 301 in FIG. 3 is obtained by modifying the expression. Equation 301 is transformed to obtain Equation 303. When the absolute value of the right side of Expression 303 is positive or 0, Expression 404 is obtained by performing the transformation shown in FIG. When the absolute value of the right side of Expression 303 is negative, Expression 504 is obtained by performing the transformation shown in FIG. Therefore, the condition for satisfying Expression 301 is Expression 601 in FIG. Expression 202 is included in expression 601. FIG. 6 shows that if the learning data for n is within the range of the equation 601, the learning for n−1 is not canceled by the learning for n.

  Claim 1 is a high-speed learning method in which the input order of learning data is always determined so as to satisfy Equation 601.

  Since the expression 601 is established for any n, it can be expected that the learning speeds up even when a part of the input order of the learning data satisfies the expression 601. Claim 11 is a high-speed learning method for determining a part of the input order of the learning data to satisfy Expression 601.

  Looking at Equation 103, the learning rate g and (1-g) are multiplied many times in the learning data in the early order, and therefore, when the learning is completed once, the learning data in the earlier order is compared with the learning data in the earlier order. It can be seen that the learning data in the later order has a stronger influence on the learning result. Therefore, when a part of the input order of learning data satisfies Expression 601, it can be expected that the effect of speeding up learning is large when the rear part satisfies Expression 601. Claim 12 is a high-speed learning method in which the rear part of the learning data input order is determined so as to satisfy Expression 601.

  The relationship between the initial value of the connection weight of the self-organizing network and the learning data is clarified by transforming the formula. When the absolute value on the right side of Expression 303 is positive or 0, Expression 704 is obtained from Expression 101 and Expression 401. When the absolute value of the right side of Expression 303 is negative, Expression 707 is obtained from Expression 101 and Expression 501. Equations 704 and 707 indicate that the initial value of the connection weight is either greater than or equal to the maximum value of the learning data or less than the minimum value so that n-1 learning is not canceled by n learning. ing.

  The fast learning method according to claim 2, wherein the initial value of the connection weight of the self-organizing network is determined so as to satisfy all the expressions 704 or 707. Claim 5 is a fast learning method in which the initial values of the connection weights are all determined so as to satisfy Expression 704. Claim 6 is a fast learning method in which the initial values of the connection weights are all determined so as to satisfy Expression 707.

  Since the connection weight is assigned to each output node and each dimension, it can be expected that the learning speeds up even when some of the initial values satisfy the condition. Claim 13 is a fast learning method for determining such that a part of the initial value of the connection weight satisfies Expression 704 or Expression 707.

  Since many calculations are required to determine the input order of the learning data so as to satisfy Expression 601, a method for determining the input order relatively easily will be considered although the conditions become severe. From Expression 801, it can be seen that the learning data of n−1 is within the range to be satisfied by the learning data of n shown in Expression 601.

  Therefore, a case was considered in which, after satisfying Expression 601, a condition that a certain magnitude relationship is maintained between the n-1 learning data and the n learning data is newly added. As shown in Expression 803, the condition of Expression 301 is also satisfied when the learning data is arranged in ascending or descending order.

  Claims 3 and 4 are high-speed learning methods for determining the input order of the learning data so that the expression 803 is always satisfied.

  The present invention is a high-speed learning method for a self-organizing network that performs high-speed processing based on the above-described method.

  According to the present invention, the learning speed can be increased without changing the learning algorithm of the self-organizing network. It can be used at the same time as the learning algorithm is improved, so it is expected to have a greater effect.

  By processing a part of the learning data, the processing amount can be reduced, and even in that case, the speed can be increased.

  The effect of speeding up was able to be obtained by a simple process of making the learning data ascending or descending.

  The effect of speeding up was able to be obtained by a simple process of adjusting the initial value of the connection weight.

  A self-organizing network program was created using a computer and experiments were performed to verify the effects of the present invention. The conceptual diagram of the self-organizing network used in the experiment is shown in FIGS.

  In the experiment, a self-organizing network in which 100 output nodes are arranged in two dimensions of 10 × 10, the number of input nodes is five, which is the same as the number of data dimensions, and all connection weights are initialized by random numbers. Learned. As the learning data, five center points are given in a five-dimensional space, and a total of 50 normal distributions around each center point are synthesized by random numbers. A) Learning in the case of the initial value of the neighborhood size 7 × 7 and the initial value of the learning rate 0.7, B) Learning in the case of the initial value of the neighborhood size 5 × 5 and the initial value of the learning rate 0.5 1) random order, 2) the order in which the short distances are arranged with respect to 5 which is 10% of the total number of data, the order in which they are arranged after the remaining random order data, and 3) all the data The distance was calculated, and the average number of learning was measured by experimenting 10 times each in the order in which the distances closer to each other were arranged in the slow order. The end determination is made when the distance of the winning node having the longest distance from the learning data is equal to or less than the threshold.

  As a result of the experiment, A1 was 1027 times, A2 was 944 times, A3 was 951 times, B1 was 603 times, B2 was 528 times, and B3 was 571 times. These results show that the present invention speeds up the learning time from about 6% to about 14%.

It is a formula which shows the change of joint weight. It is a formula which shows the proximity of the existing learning result and learning data. It is a formula which shows the nearness of the new learning result and the data for learning. It is a modification of the equation indicating the new proximity. It is a modification of the equation indicating the new proximity. It is the result of the formula which shows new proximity. It is a formula which shows the relation between the data for learning and the initial value of connection weight. It is a formula which shows the relation at the time of making learning data into ascending order or descending order. It is a conceptual diagram of the relationship between the output node and input node of the self-organizing network used for experiment. FIG. 10 is a conceptual diagram of a 10 × 10 two-dimensional output map, victory node, and 5 × 5 neighborhood used in the experiment.

Claims (13)

A high-speed learning method for a self-organizing network that increases the speed by adjusting the input order of learning data based on the value of the learning data. A high-speed learning method for a self-organizing network that increases the speed by adjusting the initial value of the connection weight based on the learning data. 2. The high-speed learning method for a self-organizing network according to claim 1, wherein the speed is increased by adjusting the input order of the learning data so that the values of each dimension of the learning data are in ascending order. 2. The high-speed learning method for a self-organizing network according to claim 1, wherein the speed is increased by adjusting the input order of the learning data so that the values of each dimension of the learning data are in descending order. The high-speed learning method for a self-organizing network according to claim 2, wherein the speed is increased by setting the initial value of the connection weight to be equal to or greater than the maximum value of the learning data. 3. The high-speed learning method for a self-organizing network according to claim 2, wherein the speed is increased by setting the initial value of the connection weight to be equal to or less than the minimum value of the learning data. A high-speed learning method for a self-organizing network comprising the features of claims 1 and 2. The self-organizing network fast learning method according to claim 7 comprising the features of claims 3 and 5. 8. A method for fast learning of a self-organizing network according to claim 7 comprising the features of claims 4 and 6. A high-speed learning method for a self-organizing network having the features of claim 1, claim 3, claim 4, claim 7, claim 8, or claim 9 for each dimension. A high-speed learning method for a self-organizing network in which a part of learning data comprises the features of claim 1, claim 3, claim 4, claim 7, claim 8, claim 9, or claim 10. The high-speed learning method of the self-organizing network according to claim 11, wherein the rear part of the learning data comprises the features of claim 1, claim 3, claim 4, claim 7, claim 8, claim 9, or claim 10. . A part of the initial value of the joint weight is self-organized with the features of claim 2 or claim 5 or claim 6 or claim 7 or claim 8 or claim 9 or claim 10 or claim 11 or claim 12. Learning method for network.

Images