JPH07225836A - Method for generating pattern recognition dictionary - Google Patents

Abstract

PURPOSE:To provide the method which efficiently generates a pattern recognition dictionary which reflects the distribution condition or reference vectors and has a high recognition precision. CONSTITUTION:A set of plural reference vectors corresponding to categories is prepared as an initial dictionary (S21), and this initial dictionary is used to perform the recognition processing of learning vectors whose categories are known (S23 and S24), and reference vectors are moved in the vector space based on results, and this dictionary optimization processing is repeated a finite number of times. In this case, the distance having a scale factor which reflects the distribution condition of reference vectors is used, and this scale factor is changed at the time of moving each reference vector.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for creating a pattern recognition dictionary used for pattern recognition of characters, voices, etc., and more particularly to a method for optimizing vector positions in the dictionary using a learning vector quantization method. .

[0002]

2. Description of the Related Art Generally, in character recognition, voice recognition, etc., a character or voice as a pattern recognition target is divided into a plurality of categories to generate respective pattern vectors, and this pattern vector and a pattern recognition dictionary are stored. A reference pattern vector (hereinafter,
The similarity with the reference vector) is determined by using some evaluation function, for example, Euclidean square distance, and the category with the highest similarity (that is, the distance between the two vectors is short) is output as the recognition result. In this case, for a character, for example, 1 character unit such as "A", "I", "U", etc.
There are one category, and for speech, the phonemes and phonemes that make up the speech are one category.

Usually, the reference vector is given as an average value of a plurality of sample vectors existing for each category. However, characters such as numbers and hiragana are likely to have various fonts depending on the writer, and there are individual differences in utterance even in voice recognition.Therefore, in order to deal with these, multiple reference vectors are set for each category. It is necessary to create a pattern recognition dictionary for this.

Learning vector quantization method (LVQ, T. Kohone
n, G.Barna, and R.Chirisley, "Statistical pattern wit
h neural networks: Benchmarking studies. ", Proceedin
gs of the IEEE International Conference on Neural
Networks, pp. I-61-68, 1988) is a technique for optimizing the vector space by generating multiple reference vectors and performing learning while correcting errors, and consists of the following algorithms.

(1) First, a plurality of reference vectors are generated by performing intra-category clustering processing on all pattern vectors (learning vectors) used for learning.
The set of these reference vectors is used as the initial dictionary. (2) The reference vector closest to one learning vector whose category is known is detected. (3) If the detected reference vector belongs to a category different from the learning vector, that is, if the recognition vector is erroneous, the reference vector is moved away from the learning vector. On the other hand, when the detected reference vector belongs to the same category as the learning vector, it is brought closer to the learning vector. The above process is repeated until erroneous recognition is eliminated or a predetermined number of repetitions is reached, thereby optimizing the vector space in the initial dictionary.

As described above, the LVQ has a simple algorithm and can be expected to be effective in many cases. Therefore, the LVQ is often used for creating a pattern recognition dictionary of this kind.

[0007]

However, the conventional LVQ simply moves the reference vector when optimizing the vector position, as described above.
The problem remains from the viewpoint of creating a pattern recognition dictionary with high recognition accuracy.

[0008] That is, in order to improve the recognition accuracy, it is preferable to perform the optimization process after reflecting the entire distribution state so that the reference vectors are grouped to some extent for each category. It was difficult to reflect the distribution of vectors.

Particularly, when considering a method for learning a distance function used in ordinary pattern recognition, the Euclidean distance is simply learned, and the weighted Euclidean distance weight and the covariance matrix in the Mahalanobis distance are used. It was not possible to learn about scale factors (variables) that reflect the shape of the distribution such as the inverse matrix of.

An object of the present invention is to solve the above problems and provide a method for efficiently creating a pattern recognition dictionary with high recognition accuracy that reflects the distribution of reference vectors.

[0011]

A method of the present invention for achieving the above object is a method of optimizing a vector position in a pattern recognition dictionary by using a learning vector quantization method, and all patterns used for learning. , A first step of creating a plurality of reference vectors respectively representing the belonging categories, a second step of selecting one learning vector of which the belonging category is already known, and a distance from the selected learning vector is minimized. The third step of detecting the reference vector, and if the detected reference vector belongs to a category different from the learning vector, move it in a direction away from the learning vector, and on the other hand, belong to the same category as the learning vector. A fourth step of moving the reference vector in a direction closer to the learning vector, A scale factor that reflects the distribution of a plurality of reference vectors is attached to the distance function that is represented, the scale factor is changed when each reference vector moves, and the second to fourth steps are performed until a predetermined evaluation criterion is reached. It is characterized by repeating a finite number of times.

In the above method, the movement of each reference pattern vector is performed, for example, by taking the sum of the reference vector and the scalar multiple of the difference vector between the learning vector and the reference vector. In this case, it is preferable to reduce the scalar magnification of the difference vector each time the first to fourth steps are repeated.

[0013]

In general, if reference vectors belonging to different categories are adjacent to each other, misrecognition is likely to occur. Therefore, when moving the reference vectors, it is preferable to move the reference vectors having different belonging categories away from each other and move the reference vectors to form a small group for each belonging category in the vector space. However, when a distance that does not consider the scale factor is used, the distribution state of the reference vectors is not reflected, and as a result of moving the reference vectors having different categories, the small group as described above may be destroyed.

In the pattern recognition dictionary creating method of the present invention,
Vector quantization is performed while reflecting the distribution of the reference vector by changing the scale factor when moving each reference vector by using a distance function accompanied by a scale factor that reflects the distribution state of the reference vector. Therefore, in addition to optimizing the position of the reference vector in the feature space, the probability that the reference vector forms the above-mentioned small group for each category to which it belongs increases, and it is possible to avoid collapse of such a small group. The subsequent reference vector has a high probability of being used for recognizing a pattern having a feature vector close to this learning vector. Further, a pattern having a feature vector close to the learning vector has a high probability of belonging to the same category as this learning vector, and therefore the recognition rate is also increased.

If a scalar multiple of the difference vector between the learning vector and the reference vector is added to the original reference vector,
The scalar multiple of the difference vector when the reference vector is moved is the smallest of the vectors that move the original reference vector within a certain distance from the learning vector. Generally, as the movement distance of the reference vector increases, the distance that the reference vector deviates from the ideal point to converge tends to increase. Therefore, if the vector with the smallest movement distance is used as in the method of the present invention, the deviation from the ideal point can be suppressed as small as possible.

Furthermore, the fourth step, that is, each time the vector position is optimized, the scalar magnification of the difference vector is reduced, so that the deviation from the ideal point becomes smaller as the number of iterations increases. In particular, when the reference vector is moved regardless of the number of times of learning, the movement of the reference vector may get stuck in a loop and may not be set to a constant value (infinite loop). Since the moving distance of the reference vector becomes smaller by making it smaller successively, an infinite loop can be avoided.

[0017]

Embodiments of the present invention will now be described in detail with reference to the drawings. FIG. 1 is a block diagram of a pattern recognition dictionary creating apparatus according to an embodiment of the present invention, in which 1 is a learning pattern input device, 2 is a learning pattern storage unit, and 3 is a central processing unit (C).
PU), 4 is a dictionary storage memory.

The learning pattern input section 1 is for inputting a pattern to be learned and uses, for example, a scanner, a microphone or the like. The learning pattern storage unit 2 is a disk that vectorizes the input learning target pattern and stores it for each category.

The CPU 3 also uses the learning pattern storage unit 2
A dictionary creating / updating unit 31 that creates and registers an initial dictionary by a method such as intra-category clustering for all patterns stored in, and one pattern (learning vector) whose belonging category is already known is selected. From the learning pattern selecting unit 32, the recognizing unit 33 that performs the learning pattern recognizing process, the dictionary processing unit 34 that optimizes the vector position in the dictionary, and the learning control unit 35 that controls the above units. Become.

The dictionary storage memory 4 includes a dictionary creating / updating unit 3
The dictionary (set of reference vectors) created in 1 is stored, and this is used as the recognition dictionary in the recognition unit 33. This dictionary is sequentially updated each time the processing of the dictionary processing unit 34 ends, and is used as a recognition dictionary in the next cycle.

Next, an operation example of the pattern recognition dictionary creating apparatus according to the present embodiment will be described with reference to FIG. 2 showing a processing procedure in the CPU 3. In addition, S represents a processing step.
In this example, first, the initial dictionary creating unit 31 creates an initial dictionary (S21). Specifically, the learning vector input from the learning pattern storage unit 2 is subjected to clustering processing for each category using a clustering method such as the Ward method or the k-means method to obtain a plurality of reference vectors. In addition,
The category to which each reference vector belongs is the same as the category to which the original learning pattern belongs. This initial dictionary is registered in the dictionary storage memory 4 (S22).

Next, in the learning vector selection unit 32,
An arbitrary learning vector whose belonging category is known is selected (S23). The recognition unit 33 uses the initial dictionary created and registered in S21 and S22 to recognize the category to which the selected learning vector belongs (S24). Specifically, it determines that the belonging category of the reference vector that is most similar to the selected learning vector, that is, has a short distance, is the belonging category of the learning vector to be recognized. Then, it is determined whether or not the belonging category is correctly recognized (S2
5).

When the category to which the learning vector belongs is not correctly recognized, the dictionary processing unit 34 moves the reference vector and the scale factor (S2).
6). For example, assuming that the position of the learning vector in the vector space exists near the discrimination boundary with the reference vector of the category to be recognized, this process is specifically performed as follows.

Of the reference vectors in the same category as the selected learning vector, the reference vector closest to this learning vector is detected, and this reference vector is brought closer to the learning vector. At that time, the scale factor that reflects the category distribution is changed. On the other hand, all other reference vectors whose distance from the learning vector is closer than the reference vector of the same category are moved away from the learning vector, and at the same time, the scale factor is changed. Naturally, the category to which these other reference vectors belong is
It is different from the category to which the learning vector belongs.

Then, it is determined whether or not the end condition is satisfied (S27), and if not satisfied, the processes of S22 and thereafter are repeated. When the termination condition is satisfied, for example, when all recognition vectors are erroneously recognized or the number of times of learning reaches a predetermined number, the learning is ended and the recognition dictionary at that time is set as the pattern recognition dictionary. .

Next, the distance function of interest in S24 to S27 will be described in detail. As the definition of the distance between the vectors, there are Euclidean distance, Mahalanobis distance, Bhattacharya distance, etc., but in the present embodiment, the evaluation for recognizing the Mahalanobis distance in consideration of the scale factor reflecting the distribution state of the reference vector. Used as a function.

Below, this Mahalanobis distance is used, and a vector obtained by scalar-multiplying the difference vector between the learning vector and the reference vector before movement is taken as the correction vector, and this correction vector and the original reference vector are An example in which the reference vector is moved by taking the sum will be given. The specific movement of the reference vector in this example is represented by the following equation (1).

[0028]

## EQU1 ## μ _i (t) = μ _i (t-1) ± α (t) {x _i −μ _i (t-1)} (1) where t time (number of learning times) x learning vector α ( t) Decrease function (0 <α (t) <1) i Index representing the number of dimensions

The sign of α (t) is positive when the reference vector is brought close to the learning vector, and is negative when it is moved away from the learning vector. In this equation, {x _i −μ
_The vector represented by _i (t-1)} is the correction vector.
Here, when α (t) is 1 and the sign is positive, the reference vector after movement matches the learning vector.

Further, when the reference vector is moved regardless of the number of times of learning, the movement of the reference vector may get stuck in a loop and may not be set to a constant value. Therefore, in this example,
By making α (t) a decreasing function, the moving distance of the reference vector (the size of the correction vector) becomes smaller as the number of learning increases, and such an infinite loop is avoided as much as possible. The same applies to the optimization of the scale factor described later. This moving formula is a relaxation descent method that minimizes the evaluation function when the evaluation function as shown in the following formula (2) is set in the entire recognition dictionary.

[0031]

[Mathematical formula-see original document] φ _L = ΣC∫ {Σi (x _i −μ _i ) ² } dx−ΣF ∫ {Σ _i (x _i −μ _i ) ² } dx (2) However, the φ _L evaluation function is correct according to the ΣC dictionary. Sum of patterns for which belonging category is obtained Σi Sum of 1 to d (number of dimensions of feature space) regarding i ΣF Reference vector used for category judgment of sum μ _i x _i regarding patterns for which correct belonging category cannot be obtained from dictionary

Mahalanobis distance D considering the scale factor ω _ij reflecting the distribution state of the reference vector
² is represented by the following formula (3), for example.

[Equation 3] D² = ΣiΣj {ω_ij(x_i-Μ_i) (x_j-Μ_j)}… (3)

In the recognition system using the Mahalanobis distance D ² , the same evaluation function as in the above example is expressed by the following equation (4).

[Equation 4]  φ_L= ΣC∫ [ΣiΣj {ω_ij(x_i-μ_i) (x_j-μ_j)}] dx -ΣF∫ [ΣiΣj {ω_ij(x_i-μ_i) (x_j-μ_j)}] dx… (4)

The relaxation descent method for minimizing this evaluation function is expressed by the following equations (5) and (6).

[Expression 5] μ _i (t) = μ _i (t-1) ± α (t) ω _ij {x _j −μ _j (t-1)} (5) ω _ij (t) = ω _ij (t −1) ± α (t) {x _i −μ _i (t−1)} {x _j −μ _j (t−1)} (6) where ω (t) learning number t (after movement) Reference vector ω (t-1) Reference number x of learning number t-1 (before movement) x Learning vector α (t) Decreasing function (0 <α (t) <1) i Index μ _i , μ _j representing the number of dimensions the reference vector closest to x _i , x _j

As in the equation (1), the sign of α (t) is positive when the reference vector is close to the learning vector, and is negative when the reference vector is far from the learning vector.

In the recognition system as in this example, the reference vector can be moved by the equation (1), and the scale factor can be changed by the equation (5). Although the above description is an example of the evaluation function based on the Mahalanobis distance, the present invention is not limited to the above distance function,
It goes without saying that it can be implemented by other distance functions.

[0037]

As is apparent from the above description, according to the pattern recognition dictionary creating method of the present invention, when the learning is repeated, the reference vector is moved while reflecting the distribution state of the reference vector. Therefore, not only the optimization of the position of the reference vector in the feature space but also the reference vectors can be grouped to some extent for each category, and a pattern recognition dictionary with high recognition accuracy can be efficiently created.

Further, by adding a scalar multiple of the difference vector between the learning vector and the reference vector to be moved to the original reference vector, the deviation from the ideal point to be converged can be suppressed as small as possible, and the pattern recognition dictionary created. It has the effect of further increasing the recognition rate.

Further, each time the vector position is optimized,
By reducing the scalar magnification of the difference vector, the deviation from the ideal point becomes smaller as the number of repetitions increases, and the reference vector can be efficiently moved.

[Brief description of drawings]

FIG. 1 is a configuration diagram of a pattern recognition dictionary creating apparatus according to an embodiment of the present invention.

FIG. 2 is a flowchart showing an example of a processing procedure of the pattern recognition dictionary creation device.

[Explanation of symbols]

 1 Learning Pattern Input Device 2 Learning Pattern Storage Unit 3 Central Processing Unit (CPU) 31 Dictionary Creation / Update Unit 32 Learning Pattern Selection Unit 33 Recognition Unit 34 Dictionary Processing Unit 35 Learning Control Unit 4 Dictionary Storage Memory

Claims (3)

[Claims] 1. A method for optimizing a vector position in a pattern recognition dictionary using a learning vector quantization method, wherein a plurality of reference pattern vectors representing respective belonging categories are created for all patterns used for learning. A first step of selecting, a second step of selecting one learning vector whose belonging category is already known, a third step of detecting the reference pattern vector having a minimum distance from the selected learning vector, If the reference pattern vector belongs to a category different from the learning vector, it is moved in a direction away from the learning vector. On the other hand, if the reference pattern vector belongs to the same category as the learning vector, the reference pattern vector is changed by the learning vector. A fourth step of moving in the approaching direction, and a distance function representing the distance A scale factor reflecting the distribution of a plurality of reference pattern vectors is attached, the scale factor is changed when each reference pattern vector is moved, and the second to fourth steps are performed a finite number of times until a predetermined evaluation criterion is reached. A method for creating a pattern recognition dictionary characterized by repeating. 2. The pattern recognition dictionary creating method according to claim 1, wherein the movement of each reference pattern vector is a sum of a scalar multiple of a difference vector between the learning vector and the reference pattern vector and the reference pattern vector. A method for creating a pattern recognition dictionary, which is characterized by being performed. 3. The pattern recognition dictionary creating method according to claim 2, wherein the scalar magnification of the difference vector is reduced each time the first to fourth steps are repeated.