JPH09147045A - Neural network - Google Patents

Abstract

PROBLEM TO BE SOLVED: To correctly identify the categories by deleting the nodes corresponding to plural categories of an adaptive resonance theory(ART1) network execution means, using only the patterns resonant with the deleted nodes and performing the re-learning by means of a monitoring parameter of larger value. SOLUTION: The neural network 9 deletes the nodes corresponding to plural categories without changing the monitoring parameter which is used for the pattern identification processing. Then the network 9 takes out only the patterns that are resonant with the deleted nodes and performs the re-learning by means of a monitoring parameter of larger value. That is, a re-learning control means 7 sets the value of the monitoring parameter ρ at ρ+Δ (Δ>0). Then, the means 7 deletes the nodes corresponding to plural categories out of the output layer of an ART1 network execution means 2 and takes the patterns resonant with the deleted nodes out of a learning pattern storage means 1. These patterns are given to the means 2 for execution of the re-learning.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to adaptive resonance theory (Adaptive Resonance Theory), which is a type of neural network.
ory) of the binary input method (hereinafter abbreviated as ART1) of pattern recognition, which relates to a re-learning method for improving the recognition rate.

[0002]

2. Description of the Related Art Generally, ART1 is used as a reference.
Carpenter, Gail A. and Grossberg, Stephen: "A Mass
collectively Parallel Architecture for a Self-Organizing
Neural Pattern Recognition Machine ", Computer Visi
on, Graphics and ImageProcessing, vol.37, pp.54-11
5, 1987), it has the excellent property of autonomously extracting and storing the structure inherent in the learning pattern, regardless of the teacher signal. The network has a bit pattern as an input and consists of two layers, an input layer and an output layer,
The feature is that the coupling between the two layers is bidirectional. The logical operation of the ART 1 when asymptotic learning is simplified is shown as follows. (1) Compare one of the unknown pattern and the memory pattern,
Find the similarity. (2) The degree of similarity is the logical product of the unknown pattern and the memory pattern, and the number of 1's in the logical product is divided by the number of 1's in the unknown pattern. (3) Similarity is a monitoring parameter (ρ: vigilance parame
ter) or more, the memory pattern number (node number of the output layer) is output (resonance). (4) If the similarity is less than the monitoring parameter, it is compared with the next storage pattern. (5) If it is not similar even if compared with all the stored patterns, it is stored as a new pattern. (6) The order in which the unknown pattern and the memory pattern are compared is determined in the order that is likely to be similar. (7) This possibility is obtained by the product sum operation of the network.

[0003]

However, since the learning method of ART1 is the unsupervised learning, the category classification as expected may not be generated when the pattern categories are identified. If there are categories similar to each other, there arises a problem that patterns belonging to a plurality of categories resonate with one node in the output layer. FIG. 4 is a conceptual diagram showing the state of resonance with a plurality of categories. In FIG. 4, each node of the output layer in the input space when a pattern belonging to a plurality of categories resonates with one node of the output layer. Shows the range that resonates with. Since it is a conceptual diagram, the shape of the resonating range and the shape of the boundary line have no meaning. Only the relationship as a set is meaningful. ● indicates a learning pattern belonging to category A, and ○ indicates a learning pattern belonging to category B.
Further, A1 to A3 are resonance ranges corresponding to the nodes in which the learning patterns belonging to the category A resonate, and B1 to B2 are resonance ranges corresponding to the nodes in which the learning pattern belonging to the category B resonate. Further, C is a resonance range of a node in which both the learning pattern belonging to category A and the learning pattern belonging to category B resonate. Regions that do not fall within any resonance range are regions that cannot be identified by this network. Further, D is the boundary between category A and category B. If the monitoring parameters are increased and the learning is performed again, the ability of category identification is improved, but this time another problem occurs that the generalization ability of the network is impaired.

FIG. 5 is a conceptual diagram showing a case after re-learning by increasing the monitoring parameter ρ, and using the same learning pattern as in FIG. 4 and re-learning by increasing the monitoring parameter. In FIG. However, since the resonance range of one node becomes small, the area that cannot be identified increases. In addition, A1 to A6 are resonance ranges corresponding to the nodes in which the learning patterns belonging to category A resonate, B1 to B5 are resonance ranges corresponding to the nodes in which the learning patterns belonging to category B resonate, and the region corresponding to C is It is divided into two, A5 and B3, and there is no category identification error. However, the individual areas have become smaller and the generalization ability has decreased. The present invention has been made in view of the above-mentioned points, and an object thereof is to solve these drawbacks, and to provide a node corresponding to a plurality of categories in ART1 network execution means. Delete it and use only the pattern that resonates with that node,
Relearn with a larger value of the monitoring parameter,
The purpose is to provide a neural network that enables correct category identification.

[0005]

[Means for Solving the Problems] That is, means for achieving the object are as follows: a) learning pattern storage means for storing a set of a learning pattern and a category to which the learning pattern belongs; Adaptive resonance theory (ART1) network execution means for outputting the node number of the resonated output layer, c) Monitoring parameter control means for controlling monitoring parameters of the adaptive resonance theory network execution means, d) The adaptive resonance theory Execution result storage means for storing a combination of a node number and a learning pattern of the output layer resonated by the network execution means, e) output from the adaptive resonance theory network execution means from the learning pattern storage means and the execution result storage means The node number of the resonated output layer and the pattern category are obtained, and then the Category correspondence means for determining the category of the pattern from the node number of the resonated output layer output from the resonance theory network execution means, f) Based on a certain monitoring parameter value ρ, the adaptive resonance theory network execution means performs the learning. Learning control means for giving learning by giving a learning pattern stored in the pattern storage means, g) As a result of the learning, when there is a node associated with a plurality of categories, the node is deleted and the monitoring parameter is set. The control means temporarily sets the value of the monitoring parameter to a large value ρ + Δ (where Δ> 0) so that more detailed identification can be performed, and the learning pattern resonating with the node is searched from the execution result storage means. ,
Re-learning control means for performing re-learning only with these learning patterns, h) Learning with the monitoring parameter value ρ by controlling the above means, and when there are nodes associated with multiple categories The control means is provided for re-learning with the monitoring parameter value ρ + Δ, and for performing the identification process using the original monitoring parameter value ρ after the re-learning.

[0006]

In the neural network according to the present invention, the node corresponding to a plurality of categories is deleted without changing the value of the monitoring parameter when performing pattern identification processing, and only the pattern resonating with this node is extracted. ,
Re-learn using a monitoring parameter with a larger value.
Re-learning is performed by using the monitoring parameter having a larger value, so that the category identification of these learning patterns can be correctly performed. The ART1 network has the property that the nodes in which the pattern resonates do not change even if the value of the monitoring parameter ρ is changed to a smaller value. Therefore, the category identification ability by the monitoring parameter used during re-learning is the same as the original monitoring parameter. It is included in the category identification ability by. That is, if it is correctly identified at the time of re-learning, it is guaranteed to be correctly identified when the original monitoring parameters are used in the same network. Also,
Since only learning patterns that resonate with nodes corresponding to a plurality of categories are used during re-learning, the time required for re-learning is short.

[0007]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram of an embodiment of the present invention. In FIG. 1, 1 is a learning pattern storage means, 2 is an ART1 network execution means showing the concept when the conventional node deletion is not performed, and 3 is a monitor. Parameter control means, 4 is execution result storage means, 5 is category correspondence means, 6 is learning control means, 7 is relearning control means, 8 is control means, and 9 is the entire neural network. The solid arrow indicates when learning
A data flow at the time of re-learning is shown, and a dotted arrow shows a data flow when identifying unknown data. The learning pattern storage means 1 is a means for storing a set of a learning pattern and a category to which the learning pattern belongs, and the ART1 network execution means 2 receives the learning pattern from the learning pattern storage means 1 as an input and outputs the node number of the resonated output layer. The means for outputting and the monitoring parameter control means 3 are means for controlling the monitoring parameters of the ART1 network execution means 2.

The execution result storage means 4 is a means for storing a set of the node number of the resonated output layer and the learning pattern output from the ART1 network execution means 2, and the category correspondence means 5 is the learning pattern storage means 1 and the execution result storage. From the means 4, the correspondence between the node number of the resonated output layer output from the ART1 network execution means 2 and the category of the pattern is obtained, and thereafter the category of the pattern is derived from the node number of the resonated output layer output from the ART1 network execution means 2. Learning control means 6 is a means for giving learning patterns stored in the learning pattern storage means 1 to the ART1 network execution means 2 for learning under a certain monitoring parameter ρ, and re-learning control. Means 7
As a result of learning by the learning control unit 6, when there is a node associated with a plurality of categories, the node is deleted, and the monitoring parameter control unit 3 temporarily increases the value of the monitoring parameter to make more detailed identification. In addition, the control means 8 controls each of the above means by retrieving a learning pattern that resonates with the node from the execution result storage means 4 and performing relearning only with these learning patterns. Then, learning with the monitoring parameter value ρ, re-learning with the monitoring parameter value ρ + Δ when there are nodes associated with multiple categories, and using the original monitoring parameter value ρ after re-learning Is a means for performing the identification processing of.

Next, the operation of the neural network having such a configuration will be described with reference to FIG.
FIG. 3 is a flowchart showing an embodiment of the present invention.
The learning control means 6 takes out the learning pattern from the learning pattern storage means 1 and gives it to the ART1 network execution means 2. ART1 network execution means 2 performs learning,
The output is stored in the result storage means 4. The learning control means 6 controls the category correspondence means 5, compares the learning pattern storage means 1 with the execution result storage means 4, and creates the correspondence between the node number of the output layer output from the ART1 network execution means 2 and the category ( (See 10 in FIG. 3). This part is the normal operation of ART1. If the category correspondence means 5 is checked and relearning is unnecessary, the network learning process for this learning pattern ends. If re-learning is necessary, the following processing is performed (see 11 in FIG. 3). Whether or not re-learning is necessary can be determined by whether or not there are nodes corresponding to a plurality of categories. The re-learning control means 7 controls the monitoring parameter control means 3 to change the value of the monitoring parameter ρ to ρ.
Set to + Δ (where Δ> 0). However, if the same node appears in this step again without being divided, it is necessary to use Δ larger than the previous Δ (see 12 in FIG. 3). The re-learning control means 7 deletes the nodes corresponding to a plurality of categories from the output layer of the ART1 network execution means 2 (see 13 in FIG. 3). The re-learning control means 7 takes out the pattern resonating with the deleted node from the learning pattern storage means 1 and gives it to the ART1 network execution means 2 to re-learn (see 14 in FIG. 3). The re-learning control means 7 controls the monitoring parameter control means 3 to return the value of the monitoring parameter to the original value ρ (FIG. 3).
15). Here, the process returns to the step of determining the necessity of re-learning.

FIG. 2 shows the effect of re-learning according to the present invention,
It is a conceptual diagram which shows the range which resonates with each node of the output layer in an input space when re-learning is performed using the learning pattern of FIG. 4, and the range of C of FIG. 4 corresponds to category A. It is divided into CA and CB corresponding to category B. CA and CB during re-learning are A5 and B in FIG.
3 is a range in which each node resonates with the same node, but it is wider because the monitoring parameter is returned to the original value. The boundary line between CA and CB includes the boundary line between A5 and B3 in FIG. 5, and they match in that range.

[0011]

As described above, according to the present invention, A
It is possible to improve the category identification ability by re-learning without impairing the unsupervised learning ability and generalization ability of the RT1 network. Also, less time is required for re-learning.

[Brief description of the drawings]

FIG. 1 is a configuration diagram showing one embodiment of the present invention.

FIG. 2 is a conceptual diagram showing the effect of re-learning according to the present invention.

FIG. 3 is a flowchart showing an embodiment of the present invention.

FIG. 4 is a conceptual diagram showing a range of resonance in each node of the output layer in the input space when a pattern belonging to a plurality of categories resonates in one node of the output layer.

FIG. 5 is a conceptual diagram showing a case where a monitoring parameter is increased and learning is performed again.

[Explanation of symbols]

 1 Learning Pattern Storage Means 2 ART1 Network Execution Means 3 Monitoring Parameter Control Means 4 Execution Result Storage Means 5 Category Corresponding Means 6 Learning Control Means 7 Relearning Control Means 8 Control Means 9 Overall Neural Network

Claims (1)

[Claims] 1. A) learning pattern storage means for storing a set of a learning pattern and a category to which the learning pattern belongs, b) an adaptive resonance theory for outputting a node number of a resonated output layer with the learning pattern as an input ( ART 1) network execution means, c) monitoring parameter control means for controlling monitoring parameters of the adaptive resonance theory network execution means, d) node number and learning pattern of the resonated output layer output from the adaptive resonance theory network execution means Execution result storage means for storing a set of e), e) Obtaining the correspondence between the node number of the resonated output layer output from the adaptive resonance theory network execution means and the category of the pattern from the learning pattern storage means and the execution result storage means. Then, the resonance of the output layer output from the adaptive resonance theory network execution means is output. Category corresponding means for determining the category of the pattern from the code number, f) the learning pattern stored in the learning pattern storage means is given to the adaptive resonance theory network execution means under a certain monitoring parameter value ρ. Learning control means for learning by means of: g) As a result of the learning, if there is a node associated with a plurality of categories, the node is deleted and the value of the monitoring parameter is temporarily increased by the monitoring parameter control means. The value ρ + Δ (where Δ> 0) is set to enable more detailed identification, and the learning pattern resonating with the node is searched from the execution result storage means,
Re-learning control means for performing re-learning only with these learning patterns, h) Learning with the monitoring parameter value ρ by controlling the above means, and when there are nodes associated with multiple categories The re-learning with the monitoring parameter value ρ + Δ, and the control means for performing the identification processing using the original monitoring parameter value ρ after the completion of the re-learning.