JPH0934862A - Pattern learning method and device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To learn lots of patterns efficiently by eliminating an undesired dictionary or setting a penalty through the use of a recognition result. SOLUTION: The device is made up of an input control section 101, a display control section 102, a clustering pre-processing control section 105, a clustering control signal 106, a clustering post-processing control section 107, a pattern recognition control section 108, a dictionary (clustering result) 111, and a learning pattern 112. Then learning patterns with very high similarity are collected before the learning to reduce the number of learning patterns in advance. Furthermore, a radial pattern space is divided around a representative point of the clusters and only a pattern remotest from the representative point of the cluster in the region is stored as a pattern in the cluster. Furthermore, a limit for discontinuously merging clusters with high similarity is provided. Then an undesired dictionary is eliminated or a penalty is set by using the result of recognition.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and apparatus for learning a pattern by clustering (classifying) it to create a dictionary (learned cluster) for pattern recognition in the field of pattern recognition, and more particularly to , A learning method and apparatus capable of efficiently learning a large number of patterns.

[0002]

2. Description of the Related Art Conventionally, as a method of learning a pattern, a pattern or a set of patterns is regarded as one cluster (group), the similarity between the clusters is digitized based on a certain scale, and the maximum similarity, Or, more generally, a method of classifying patterns by merging clusters having similar values or more to form one cluster (“recognition engineering-pattern recognition and its application” (Junichiro Toriwaki, Corona, 1993), page 85, "Hierarchical clustering").

[0003]

When the above-mentioned conventional "hierarchical clustering" is applied to the learning of a huge amount of patterns, all the patterns are collectively used for learning, so that the repetitive processing is extremely difficult. It took a lot of time and took a huge amount of processing time. Therefore, efficient reduction of learning patterns is an issue.

When there are three or more highly similar clusters, the order of merging those clusters has not been specifically indicated.

When repeating merging of highly similar clusters, a defect generally called a chain effect appears. When the learning patterns are enormous, the patterns are often clustered and this chain effect becomes noticeable, which is a problem.

When a large number of patterns are learned by clustering, a large number of clusters (dictionaries) consisting of a small number of patterns are generated. Among these, a large number of unnecessary dictionaries that are unrelated to the recognition accuracy. Exists in. The problem is to create a highly efficient dictionary by excluding these.

[0007]

The method or apparatus of the present invention is a process or means for preliminarily reducing the number of learning patterns by, for example, collecting learning patterns having extremely high similarity before learning, A process or means for dividing a pattern space into radial areas centered on a representative point, and storing only the pattern with the longest distance from the representative point of the cluster as an intra-cluster pattern in the area, a cluster with high similarity The process or means for limiting the merging of the two is discontinuous, the process or means for removing the unnecessary dictionary by using the recognition result, or the process for setting the penalty.

[0008]

[Function] The learning time can be shortened by reducing the number of learning patterns. By limiting the merging of the clusters, the clusters at the initial stage of learning can be dispersed, so that the chain effect is unlikely to occur and a huge amount of patterns can be learned. At the same time, a concrete merging method is also shown. Further, use of the recognition result facilitates determination of unnecessary dictionaries.

[0009]

[Examples] Clustering will be described. When a finite number of pattern samples are given to the pattern space, the locations where the pattern distributions are dense in this space are called clusters (pattern sets) (see FIG. 2). FIG. 2 shows a case where there are two types of characteristic axes. The operation of finding this cluster is called clustering. A typical method of clustering is a process of repeatedly merging patterns having similarities (generally called the nearest neighbor method).

Distance, similarity and the like are used as a measure of the similarity between patterns. The distance includes so-called city distance, Euclidean distance, Minkowski distance, Mahalanobis distance, and the like, and the similarity includes so-called simple similarity, directional similarity, complex similarity, weighted similarity, and the like. When the similarity is high, the similarity is high, and when the similarity is low, the similarity is low. On the contrary, when the distance is used, the similarity is high when the distance is short, and the similarity is low when the distance is long.

For pattern or cluster merge processing,
There are two typical methods: (1) a method of not saving the position of the pattern to be merged in the pattern space, and (2) a method of saving the position of the pattern to be merged in the pattern space.

In the above (1), a pattern representing a cluster is obtained by accumulating feature values for each of the feature axes of the pattern space, taking the average, taking the center of gravity, or normalizing with a certain threshold. Is created, and in the subsequent merge processing,
This is a method of comparing the similarity with the representative pattern and using it for merging. By creating the representative pattern, the position of the merged pattern in the pattern space disappears.

On the other hand, the above (2) is a method of comparing the similarities with individual patterns in the cluster and repeating the merging process without creating a representative pattern. Therefore
The method (2) is not suitable for learning a large number of patterns because it is necessary to save the positions of the patterns to be merged in the pattern space.

In the case of merging processing thereafter, the above method (1) is used. However, due to the development of computers, it is not impossible to adopt the above method (2).

"Hierarchical clustering" which is a conventional method
Will be described. The set of n patterns to be clustered is P = {p0, p1, ..., pn-1}. The distance or similarity between the patterns pi and pj is the i-th row j
The matrixes that form the columns will be referred to as a similarity matrix and a distance matrix, respectively. This matrix may be realized in advance in the form of a table (list) on a computer, but when learning a large number of patterns, the size of the table will be huge, so it will be calculated sequentially as needed. May be. The “distance” used hereafter will include the above-mentioned similarity in addition to these distances, and will be used as a word to express a scale representing the similarity between patterns.

FIG. 5 is a diagram for explaining the conventional "hierarchical clustering". It is assumed that the initial cluster is each learning pattern (process 601). The threshold T in the process 602 is a distance threshold for determining cluster merging. The djk in the process 604 is the cluster Cj
And the cluster Ck. In process 604,
If the distance djk is less than or equal to the threshold T, the cluster Ck is merged with the cluster Cj (process 605). K in processing 606
Is incremented (incremented by 1), and if the number of clusters is not exceeded, this processing is repeated (processing 607). Next, in process 608, k is set to zero, j is incremented, and if the number of clusters is not exceeded, these processes are repeated (process 609). The threshold value T is increased by a preset increase amount (process 610), and if the final value is not exceeded, this process is repeated (process 61).
1). By repeating such processing, small clusters gradually grow into large clusters, and learning is executed.

The method (2) described above is not suitable for learning a large number of patterns because it is necessary to store the positions of the patterns to be merged in the pattern space, but the following method is used. It becomes possible to learn by using.
Since this method enables high-performance clustering processing by using the so-called nearest neighbor method, farthest neighbor method, Ward method, and center of gravity method, it is an essential method for learning a large number of patterns.

Taking the case where the pattern space is two-dimensional as an example,
This will be described with reference to FIG. FIG. 3 shows only the periphery of the cluster representative point (center point, average point, or center of gravity point) 301 in an enlarged manner. The point on the pattern space of the learning pattern is the point 30.
As indicated by 2 and point 303, it is indicated by an X mark. When the learning patterns are enormous, the patterns may be concentrated around the cluster representative point in the order of 1,000, 10,000, or more. The storage capacity for storing all the points in the pattern space of such a pattern becomes enormous, and in some cases, the capacity of the computer is exceeded, so that the learning process may not be realized.

This method is a method that enables the same clustering as the conventional method without storing all positions. In FIG. 3, a line such as 306 is drawn around the representative point of the cluster, the region is divided at an angle (the direction of the vector with respect to the cluster center point), and in each region, from the cluster representative point. This is a method of storing only the pattern at the farthest position. That is, for example, if the division is performed once, it will be divided into 360 equal parts, but at most 3
All that is required is to remember the positions of the 60 patterns in the pattern space.

In FIG. 3, the pattern position 302 is stored because the distance 305 is the farthest in this region, but the distance 3
04 is the pattern position 3 closer to the cluster representative point 301
It is not necessary to remember 03. The patterns that do not need to be stored may be simply deleted and not learned, or may be merged with the representative points of the cluster. According to this method, even if the number of patterns of thousands, tens of thousands, or more per cluster is concentrated and learning cannot be performed, learning can be executed by processing hundreds of patterns.

For example, handwritten characters and omni fonts (various types of printed characters) are extremely deformed, so it is necessary to prepare and learn a huge amount of learning patterns.
If you try to learn all patterns as before, it will take a huge amount of processing time. Therefore, before learning
It is necessary to reduce the learning pattern by some processing.

Of course, it is not necessary to simply reduce the pattern, and it is not possible to simply exclude the pattern to be learned. By reducing the number of learning patterns, the number of repetitive processes decreases as a power of the number of patterns. In FIG. 5, the number of loops is triple, and therefore the number decreases with the cube.

To reduce the number of learning patterns, (1) divide into categories (2) simply divide the learning pattern (3) collect patterns that have very close distances in advance (4) make the distance extremely Among the adjacent patterns, only one pattern or several patterns are left as a representative pattern,
Exclude the rest (5) There are methods such as dividing using the structural analysis results. Here, the category refers to, for example, “0” to “9” in the case of numbers and “a” to “n” in the case of hiragana.

By dividing the learning pattern as shown in the above (1), (2) and (5), the learning process can be distributed to a plurality of computers and processed in parallel to shorten the learning time. There is also. If the clustering results obtained by dividing and clustering in advance are collected and re-clustered, a clustering result equivalent to the case of not dividing can be obtained in a short time.

In the above (3), an ordinary clustering method may be used. In this case, the processing is interrupted in the very beginning. (4) is based on the idea that, when there are enormous learning patterns, many patterns with almost the same shape are stochastically present. In other words, it is wasteful to use many patterns having almost the same shape for learning, and it is necessary to use only one pattern. As for the method of selecting this one pattern, it is sufficient to select a pattern of the center, the average, and the center of gravity, or randomly.

When learning a large number of patterns, the patterns are densely arranged in the pattern space, and the so-called chain effect shown in FIG. 4 appears. FIG. 4 shows a state in which the center point (or representative point of a cluster) 501 (marked by a cluster) 501 of a cluster in the pattern space and another cluster existing in the vicinity are merged, and the cluster center moves to 502. Shows. Similarly, the cluster center is 50
It has moved to 3. Reference numeral 504 indicates the locus of movement of the cluster center. In the conventional “hierarchical clustering”, this chain effect is represented by reference numerals 604, 60 in FIG.
It is generated by a series of repeated processes indicated by reference numerals 5,606,607. The chain effect is a phenomenon that should not occur in that the center of a cluster that should be originally moved moves to cause a merger with a cluster that should not be originally merged. A clustering method that does not cause this chain effect is necessary.

In order to prevent the chain effect from occurring,
For example, as shown in FIG. 6, it is possible to solve the problem by limiting the merge process so that the merge process is not continuously performed. The processes of FIGS. 5 and 6 substantially correspond to each other.
Only after 05 and processing 705 are different. That is, when the cluster merging process is performed in the process 705, the merging process with this cluster is temporarily stopped and the merging process with another cluster is performed. By not merging continuously in this way, especially in the initial state of learning, minimal clusters will be scattered, and clusters will gradually grow with this cluster as a seed (center). become. As a result, it is possible to create a dictionary that absorbs pattern deformations abundantly, has many variations, and is effective for recognition accuracy.

When a large number of patterns are collected, the number of patterns including various deformations inevitably increases, but the absolute number decreases depending on the degree of deformation. That is, there are many patterns having a standard shape with little deformation, and the number of patterns with large deformation is small. Therefore, when a large number of patterns are clustered, a large number of clusters (dictionaries) composed of a small number of patterns are generated. Such dictionaries require a certain number in order to improve the recognition accuracy, but there are a large number of unnecessary dictionaries among them. It is important to create a highly efficient dictionary that has a small absolute number of dictionaries and can be recognized with high accuracy by accurately determining and excluding this unnecessary dictionary.

To this end, (1) delete a dictionary with few correct answers and many incorrect answers (FIG. 7) (2) provide a penalty for a dictionary with few correct answers and many incorrect answers (FIG. 8) (3) shape It is effective to delete the dictionary that is judged unnecessary because of the similarity of.

In the above (1), as shown in FIG. 7, learning (any method including the method of the present invention) processing 9 is performed.
After 01, recognition processing 902 is performed. Although any method may be used for the recognition processing, normally, the similarity or distance between the separately prepared test pattern and all learned dictionaries is calculated, and the dictionary showing the maximum similarity in the recognition judgment 905 is used as the recognition result. And The recognition result may be a correct answer or an incorrect answer. For example, when recognizing the number 0, the recognition result must be 0 (zero),
If the recognition result is 6, it is an incorrect answer.
The correct answer and the incorrect answer are counted individually for each dictionary, and the dictionary with few correct answers and many incorrect answers is deleted (90
4) The method is to repeat this process.

Specifically, the dictionary having zero correct answers and one or more incorrect answers is deleted and recognition is performed again, and the dictionary having zero correct answers and one or more incorrect answers is deleted. If there is no dictionary to be deleted, a dictionary with one correct answer or less and one incorrect answer or more is deleted. This process is repeated, and the number of correct answers is increased by one, for example, until the correct answer is 10 or less and the incorrect answer is 1 or more. This is a very polite method. For example, the process of deleting a dictionary having 10 or less correct answers and one or more incorrect answers may be repeated immediately.

When the recognition accuracy limit is reached, this repetitive processing is terminated (903). Since the recognition accuracy can be calculated each time the recognition processing 902 is performed, the limit point of the recognition accuracy is
For example, when the increase in recognition accuracy slows down, when the recognition accuracy decreases, or when the degree of increase in the recognition accuracy falls below a certain reference value.

The above (2) is a method in which the dictionary itself is not erased, but a dictionary that should be the correct answer appears as a recognition result by adding a penalty. A penalty is set for each dictionary.

The penalties can be added by methods such as (a) similarity subtraction type and (b) similarity integration type.

In (a) above, the initial value of the penalty is zero. After the recognition process, a penalty is added based on the number of correct and incorrect answers counted for each dictionary. For example, the number of incorrect answers / the number of correct answers / k is added. In the recognition process performed again, this penalty is subtracted in the recognition determination 1005 from the actually calculated similarity. Since the similarity usually takes a decimal value between 0 and 1, k is a constant value such as one hundred or one thousand. k does not necessarily have to be a constant value and may be changed according to the progress of learning.

In the above (b), the initial value of the penalty is 1. After the recognition processing, the penalties are accumulated based on the number of correct and incorrect answers counted for each dictionary. For example,
The number of correct answers / (the number of correct answers + the number of incorrect answers) × k is added to the penalty. k is a constant or a changing value as described above. The larger the number of incorrect answers, the smaller the penalty becomes, and in the recognition processing to be performed again, if this penalty is added to the actually calculated similarity, the similarity of the unnecessary dictionary can be relatively lowered. it can. A dictionary with a sufficiently large penalty may be removed.

In the determination in (3) above, for a dictionary determined to have a high degree of similarity by calculating the degree of similarity or distance, if there are one or several dictionaries among them. There can be good cases. As shown in (3), the other dictionaries may be simply deleted, or the recognition process is repeated every deletion one by one, and only the dictionaries having no adverse effect are selected while checking the effect of the deletion. May be erased.

[0038]

As described above, the present invention has an effect that it is possible to efficiently learn a large number of patterns.

[Brief description of drawings]

FIG. 1 is a block diagram showing an example of a system configuration that realizes the present invention.

FIG. 2 is a conceptual explanatory diagram of clustering.

FIG. 3 is an explanatory diagram of a learning pattern reduction method.

FIG. 4 is an explanatory diagram of chain effect.

FIG. 5 is a flow chart of conventional “hierarchical clustering”.

FIG. 6 is a flow chart of an example of a clustering method for eliminating chain effect.

FIG. 7 is a flowchart of a method for removing an unnecessary dictionary.

FIG. 8 is a flowchart of a method of setting a penalty in an unnecessary dictionary.

[Explanation of symbols]

101 ... Input control unit, 102 ... Display control unit, 105 ... Clustering pre-processing control unit, 106 ... Clustering control unit, 107 ... Clustering post-processing control unit, 108 ...
Pattern recognition control unit, 111 ... Dictionary (clustering result), 112 ... Learning pattern, 302, 303 ... Position of learning pattern on pattern space, 304, 305 ... Distance from cluster representative point, 501, 502, 503 ... Position of the learning pattern in the pattern space, 504 ... Trajectories for explaining chain effect.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Yoshihiro Shima Shima 1-280, Higashi Koikeku, Kokubunji, Tokyo Inside Central Research Laboratory, Hitachi, Ltd. (72) Kazumi Suzuki, 2880, Kozu, Odawara, Kanagawa Prefecture Hitachi, Ltd. Storage System Division

Claims (12)

[Claims] 1. A device for learning by clustering patterns, comprising means for inputting a pattern for learning, means for storing the pattern, means for storing a dictionary which is a clustering result, and pre-clustering. A pre-processing unit for clustering that adjusts the number of patterns, a clustering unit that clusters the patterns, a post-processing unit that clusters the dictionary that is the clustering result created by the clustering unit, and the dictionary. And a pattern recognition means for recognizing a pattern. 2. The preprocessing device for clustering according to claim 1, wherein the learning pattern is divided into categories for each learning pattern. 3. The clustering preprocessing device according to claim 1, wherein the learning pattern is simply divided. 4. The clustering preprocessing device according to claim 1, wherein the clustering preprocessing means divides the learning pattern based on a structural analysis result of the pattern. 5. A preprocessing device for clustering according to claim 1, wherein the preprocessing means for clustering collects learning patterns in advance with highly similar patterns to reduce the number of learning patterns. 6. The clustering preprocessing device according to claim 1, wherein the number of learning patterns is reduced by selecting a representative pattern from among patterns having high similarity as a learning pattern. . 7. The clustering preprocessing device according to claim 2, claim 3 or claim 4, when the clustering results learned by dividing in advance are combined and not divided by re-clustering. A pattern learning device which obtains a clustering result equivalent to. 8. The pattern learning apparatus according to claim 1, further comprising means for dispersing cluster merging. 9. The clustering means according to claim 1, wherein the pattern space is divided into radial regions centered on the representative point of the cluster, and only the pattern whose distance from the representative point of the cluster is the longest in the region. A pattern learning device characterized by storing as a pattern in a cluster. 10. The post-processing for clustering according to claim 1, wherein the result of the pattern recognition means according to claim 1 is used to delete a dictionary determined to be unnecessary. apparatus. 11. The clustering post-processing means according to claim 1 uses the result of the pattern recognition means according to claim 1 to set a penalty in a dictionary judged to be unnecessary. Aftertreatment device. 12. A process of inputting a pattern for learning, a process of storing the pattern, a process of storing a dictionary that is a clustering result, and a pre-processing of clustering that adjusts the number of patterns before clustering. A pattern learning method, comprising: a process of clustering the pattern, a post-processing of clustering for adjusting a dictionary that is a created clustering result, and a pattern recognition process of recognizing a pattern using the dictionary. .