JPH05181972A - Method and device for pattern classification - Google Patents

Abstract

PURPOSE:To enable even the addition or deletion of a category after learning without relearning the whole. CONSTITUTION:When a question for classification into categories that a user considers as classification destination is an entire question and a question for classification into limited classification destinations is a partial question, the duplication of the limited classification destinations is allowed and partial question as many as combinations are set to generate an identification function for the partial questions. When a new category is added, all partial questions having the category as a limited classification destination are set and an identification function is generated for the partial questions and added. When the category is deleted, all identification functions having the category as a limited classification destination are not used.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a pattern classification method and device.

[0002]

2. Description of the Related Art In a conventional pattern classification method, there is a method of inputting a feature extracted from an object to a multi-layered neural network trained with sample data and determining the classification destination by the output of the network. FIG. 4 shows a neural network model in the case of three layers. Each node 20
Has a plurality of inputs, and when the weighted sum of input data exceeds a threshold value, the output is determined based on a normalization function or a sigmoid function. Determining this weight and threshold value from sample data is called learning. As a learning method, there is a learning method by error reverse transfer proposed by Rumelhart. This learning method is
The difference between the output data of each layer and the target value is calculated, and the weights are sequentially adjusted so that the total sum of squares of the difference is minimized. This is repeated for the learning data, and the learning ends when the weight converges [see Tokikaihei 2-143384].

In the neural network method,
In the learned network, the nodes in the output layer are non-linear functions of the input features. These functions are configured by the number of categories, and are considered to be effective classification methods even for linearly inseparable problems [see Neural Network Information Processing Sangyo Tosho Co., Ltd. pp39].

In the discriminant analysis method, there is a discriminant method involving dimension reduction when the number of categories is three or more. The axis that most clearly represents the mutual position of each category is obtained, and the axis is determined. Classification is performed by the Euclidean distance in the stretched space. This method requires solving the eigenvalue problem,
It takes a lot of calculations. As a method of avoiding solving the plurality of eigenvalue problems, a linear discriminant function that classifies two between arbitrary two categories is created, and the most frequent discriminant destination of the discriminant function group is defined as the discriminant destination. There is a method to do so [see Basics of Image Recognition II, Ohm Co., pp202].

[0005]

In the method using a neural network, when a network is once constructed and learned, and then a category is added, it is necessary to re-learn by using all learning data again, and it is easy to perform the category. Cannot be added. Conversely, if you reduce the categories, you can classify without learning again, but the effect of unnecessary categories on the identification rate remains.
It is considered that the discriminating ability will not be higher than that of the network configured from the beginning without the category. For example, assume that there are now categories A, B, and C.
When only the data of category A and category B is input, the network created by only A and B and A and B
In the network constructed by C and C, the latter identification rate does not exceed the former identification rate. This is because, in the latter case, the information about the category C is not useful for distinguishing between the category A and the category B. On the contrary, since the category C is also configured to be identified at the same time, the identification rate for the two categories A and B may decrease. Therefore, if it is desired to improve the identification rate, it is necessary to retrain the network. Further, as one of the models of the neural network, which can add a category, there is an RCE model.
15866 publication]. The RCE model is a three-layer network, and the nodes in the middle layer are constructed as follows:
The output layer is the output or of the intermediate layer. z = 1 [λ-lx-wl] (1) Here, z is the output of the node, x is the input vector to the node, w is the weight vector of the node, and λ is the radius of super-salvation that the node represents. 1 [] is a function that outputs 1 when positive and 0 when negative. This model tries to represent a category by a set of super rescues, and each node in the middle layer represents super rescue in the input feature space. Therefore, by adding super rescue, it is possible to add or remove categories. However, the super rescue gap is a region that does not belong to any category, and the generalization ability (estimation ability by interpolation / extrapolation from learning data) depends on the size of the gap. In addition, this model has a tendency that the radius of super rescue becomes smaller as the learning progresses with respect to noisy inputs, and the number of intermediate layers tends to increase to almost the same number as the training data or more. The effectiveness for the problem of pattern recognition, which includes statistical fluctuations in the data, has not been established.

The discriminant function created between arbitrary two categories in the discriminant analysis which is the second conventional technique is a linear function. Therefore, if the first given problem has multi-category classification targets and even one set of them cannot be linearly separated, the method described in the prior art will not always output the correct classification targets. Absent. Further, which function is used has not been considered in the related art, and it does not correspond to the addition / removal of categories.

In addition, the method using a neural network is said to be able to construct a discriminant function even when it has a plurality of categories and cannot be linearly separated.
When the number of categories increases, it tends to be difficult to determine the function with high accuracy, that is, the convergence of learning becomes slow, and the output value has a bias. As a method of solving this, there is a method of constructing a large-scale neural network by combining small-scale neural networks. In these, the categories to be in charge are assigned to the small-scale neural networks in advance, learning is performed, and the outputs of the small-scale neural networks are compared to determine the classification destination. However, it is not possible to predict what will happen to the output when the data of the unlearned category is input, and for example, there is a possibility that a high output value will be output for the data of the category not in charge. Therefore, if the classification destination is determined by the size of the output value of each net, the correct selection may not be made.

The present invention is intended to solve the above problems, and an object thereof is to discriminate a classification function with a limited number of classification destinations in pattern classification by making the number of combinations by overlapping the classification destinations. By making it possible to specify a category that can be classified as a whole without doing so, it is possible to make changes such as adding or deleting categories and specify the classification destination, and even if there are many categories to be classified An object of the present invention is to provide a classification method and a device that do not reduce the identification rate.

[0009]

A first invention is a pattern classification method for determining a category to which an object belongs from a plurality of features relating to the object, and combining partial problems with limited classification destinations while allowing overlapping of the classification destinations. By setting the number of points, and by configuring a discriminant function that inputs multiple features related to the target in the set subproblem and outputs the classification destination, by specifying which discriminant function is used to select a category that can be classified. It is characterized in that the classification destination is determined by comprehensively judging the outputs of the designated discriminant functions.

According to a second aspect of the invention, in a pattern classifying apparatus which inputs a plurality of features related to an object and outputs a category to which the object belongs, category management means for outputting category information indicating initial value designation and addition and deletion of the category. When,
It receives the category information from the category management means, sets the number of combinations of sub-problems with limited classification destinations, sets the number of combinations, and outputs the setting information. Receive configuration information and enter multiple features about the target in that configuration,
Only when there is an input from the feature distribution unit, the function configuring unit that configures the identification function that outputs the classification destination and outputs the setting information and the function information, and the setting information and the function information output from the function configuring unit The function storing means for storing the classified destination as a functional block for outputting, the pattern characteristic storing means for storing the characteristic of the data to be classified, and the pattern characteristic read out from the pattern characteristic storing means from the category managing means. Feature distribution means for selectively outputting to the functional blocks of the function storage means based on the received category information, and the output is received from only the functional blocks of the function storage means that have outputs, and the integrated judgment is made. It is characterized by comprising an integrated determination means for outputting a classification destination.

[0011]

The principle of the present invention in which the entire re-learning required when adding or deleting a category in the conventional method can be processed by the addition and selection of subproblems will be described below.

[0012] Here, the problem of classifying a category that the user is currently considering as a classification destination is called an overall problem, and the problem of performing classification with a limited classification destination is called a partial problem.

In the classification method of the present invention, the place where the category of the classification destination is given by the user (this is the classification destination of the general problem) and the limited classification destination (this is the partial classification destination) are all the general problems. We need a set of discriminant functions for subproblems that are included in the classification destination of.

Each discriminant function is learned only for subproblems. If there are N classification destinations for the overall problem, and p classification targets for the partial problem, then _N C _P
The number becomes independent and does not depend on the number of learning data. Further, since each discriminant function constitutes a discriminant plane that divides the partial classification destinations, there is no gap that the RCE model has.

In order to add a category, a sub-problem having the category as a sub-classification destination is newly set, a discriminant function is constructed for the sub-problem, the discriminant function is added to the function group, and the discriminant function newly added is added. Just register to use. This provides all the discriminant functions needed for the whole problem. Figure 2 shows a conceptual diagram of the learned state.
FIG. 3 shows a conceptual diagram when a category is added. In FIG. 2, 10 to 14 indicate categories, and the straight line drawn between the categories represents the discriminant function. In FIG. 3, 15
Is the added category, and 101 to 104 are the newly added discriminant functions. The states other than 15, 101 to 104 are the same as those in FIG. That is, it is possible to cope with the addition of categories by adding a new discriminant function without re-learning the discriminant function group learned so far. The number of sub-problems to be set is as many as the number of already existing categories, but since the classification destination of each sub-problem is limited, it is possible to make the learning easier than the whole problem. Since each sub-problem is exactly the same, it has a high degree of parallelism, and if a highly parallel machine is used, it is possible to perform operations faster.

Further, in order to remove a category, it is sufficient to register so that the discriminant function for the partial problem including the category as the partial classification destination is not used. As a result, only necessary and sufficient discriminant functions are used for the overall problem after removal, and a classification system for the overall problem can be constructed without being affected by the removed category. That is, in order to construct a classification system without being affected by the removed categories, it is possible to remove categories by selecting an existing discriminant function without retraining the overall problem after removal, which was necessary in the conventional method. Can handle.

Further, since a functional block is constructed for a subproblem in which the classification destinations are limited and the outputs thereof are integrated to determine the classification destinations, even if there are many classification destinations as a whole, the classification rate will drop due to that. Absent. Also, when integrating, not only the size of the output value is used to determine the classification destination, but also the number of functional blocks that output the classification destination is taken into consideration. Therefore, each functional block is not a partial classification destination. No matter what output is output when category data is input, as long as the functional block that contains the category to which the input data belongs as the partial classification destination outputs the correct output, the classification destination of the integrated result is based on the majority decision logic. Output the correct classification destination.

[0018]

1 is a block diagram showing the configuration of an embodiment of a pattern classification device according to the present invention. This pattern classifier
Category management means 1, partial problem setting means 2, sample data storage means 3, function configuration means 4, function storage means 5, pattern feature storage means 6, feature distribution means 7, integrated determination means 8
have. Hereinafter, the numerical image data stored on the array as illustrated in FIG. 5 will be considered as a target. As categories, 10 categories corresponding to 0 to 9 {C ₀ , C ₁ ,
,, C ₉ } and proceed. The feature extracted from the image data is, for example, a grayscale feature f resampled after being blurred by a Gaussian filter.

First, the operation for learning a category will be described. The category management means 1 has, for example, 10 symbols, C ₀ , C ₁ , ...
・, C ₉ is memorized. Then, the ten symbols indicating the category and the signal indicating the initial state are sent to the partial problem setting means 2.
Only the signal indicating the initial state is output to the feature distribution means 7.

The partial problem setting means 2 receives the signal indicating the initial state and the ten category names indicating the categories from the category managing means 1 and stores them. Then, the number of combinations of sub-problems in which the classification destination is limited to two categories of C ₀ and C ₁ , for example, is (C ₀ , C ₁ ), (C ₀ , C ₂ ), ... ., (C ₈ , C
And ₁₀ C ₂ pieces set consisting of _9), set C _ij = (C _i of the two categories Name is a partial grouping destination as the setting information, C _j) sequentially, and outputs to the function structuring unit 4.

The function constructing means 4 receives a set C _ij of two category names as setting information of the classification problem set from the sub-problem setting means 2 and receives from the sample data storage means 3 two categories C _i included in the set. , C _j , sample data is read, a discriminant function for classifying the categories C _i and C _j is constructed, and the function information is output to the function storage means 5 together with the setting information C _ij for the partial problem. The discriminant function is, for example, a method of learning the error inverse transfer method with two categories as the classification destination in a neural network that is a self-organizing method [see Neural Network Information Processing Sangyo Tosho Co., Ltd. pp39], or a polynomial based on the GMDH algorithm Method [Sovie
t Automatic Control, Vol. 1
3, No. 5, 1968, pp31-41 Soviet Soviet Automata Control]]. In the case of a neural network, the output function information becomes the network structure and the weight of each connection.
When the H algorithm is used, it becomes a coefficient that describes a polynomial function. The sample data storage unit 3 stores, as learning data, for example, a sample data group in which a number on a paper or the like belonging to each category is input as an image with a scanner or the like.

The function storage means 5 receives the function information and the partial problem setting information C _ij output from the function forming means 4,
It is stored as a functional block. The functional blocks are stored by the number of subproblems. Each functional block is a feature distribution means 7
From the two categories C _i and C _j which are the partial classification destinations stored as the setting information according to the output value of the stored discrimination function when the feature amount is input as a signal from The name of the category to which the input signal is determined to belong is output to the integrated determination means 8.

When the feature distribution means 7 receives a signal indicating the initial state from the category management means 1, the function storage means 5
All the flags for the functional blocks stored in are turned on. The flag may be represented by ON / OFF of a bit on a normal memory. This ends the learning.

Next, the operation of recognizing the input data will be described. The pattern feature storage unit 6 stores f as the feature amount extracted from the image. The feature distribution means 7 reads the feature quantity f from the pattern feature storage means 6 and outputs it only to the functional blocks in the function storage means 5 whose flag is turned on. When each functional block receives the input, it outputs the category name as described above.

The integrated judgment means 8 receives the category name only from the functional block of the function storage means 5 which has output, and integrates it to determine the classification destination. As a method of integration, an array V that has a one-to-one correspondence with the classification destination of the overall problem
Prepare [i] (corresponding to C _i and V [i]),
When C _i is output as a category name from each functional block, the value of the array V [i] is incremented by 1. This is a method of determining C having the largest value in the array V [i] as a classification destination when the addition of the array V is completed for all the outputs from the functional blocks to which the features are input.

The operation of adding a category will be described. After learning 10 numbers 0 to 9 now, suppose that a category C _A corresponding to the alphabet _A is added. First, the user stores the learning data of the category C _{A in} the sample data storage means 3. Then, the user externally instructs the category management means 1 to add C _A. Then, the category management means 1 outputs a signal indicating addition and the category name C _A to the partial problem setting means 2 and the feature distribution means 7. The partial problem setting means 1 receives the signal indicating addition and the category name C _A , and does nothing if the received category name is already stored. If it does not exist, C _A is newly stored, and between the already stored categories, that is, (C ₀ , C _A ), (C ₁ ,
C _A ), ..., (C ₉ , C _A ), 10 sub-problems are set, and a set of two category names C _ij is output to the sequential function forming means 4 as setting information. The function constructing means 4 receives a set C _ij of two category names as the setting information of the classification problem set from the partial problem setting means 2, and the two categories C _i , C included in the set from the sample data storage means 3. The sample data of _j is read and the category C _i
And a classification function for classifying the categories C _j and C, and outputs the function information to the function storage unit 5 together with the setting information C _ij of the partial problem. The function storage means 5 stores the received function information and setting information C _ij as a functional block.
When the feature distribution means 6 receives the signal indicating addition and the category name C _A , it searches for a functional block having the category name as a partial classification destination. In this case, C _OA , C _1A , ...
_.. There are 10 functional blocks with setting information of C _9A. If there is a flag indicating whether to output to that functional block, turn on the flag. If not, create the flag and turn on. .. This allows you to store ₁₀
C ₂ functional blocks do not need to be changed, category C
_A can be added.

Next, the operation for removing categories will be described. Category C ₀ corresponding to the number 0 now
Suppose you want to remove from the classification destination. In this case, first, the user externally instructs the category management means 1 to remove the category C ₀ . Then, the category managing means 1 outputs a signal indicating removal and the category name C ₀ to the feature distributing means 6.
In the feature distribution means 6, a signal indicating removal and a category name C ₀
, A functional block having the category name as a partial classification destination is searched, and in this case, there are 10 functional blocks having setting information of C ₀₁ , C ₀₁ , ..., C _0A . Turns off the flag that indicates whether to output. As a result, the category C ₀ can be removed without changing the function storage means 5.

Although the present invention has been described in detail with reference to the embodiment, the present invention is not limited to this embodiment.

[0029]

According to the present invention, even if the categories are added after the learning, it is not necessary to re-learn all, and the partial problem including the newly added category is set to construct the discriminant function. It can be dealt with by adding. Also, even if you delete a category after learning,
By removing the discriminant function having the category as the classification destination from the discriminant function used, it is possible to deal with it without re-learning. As a result, it becomes possible to dynamically configure the classification system by designating the category according to the wishes of the user.

Further, since the subproblems whose categories are limited are overlapped and set by the number of combinations and the discrimination function is constructed for each subproblem, the discrimination rate is not lowered due to the large number of categories. Classification can be done. Further, since the classification destination is determined in consideration of the output value of each discriminant function and the number thereof, it is possible to eliminate the instability when the classification destination is determined only by the output value.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an embodiment of a pattern classification device of a second invention.

FIG. 2 is a conceptual diagram showing the principle of the first invention.

FIG. 3 is a conceptual diagram showing the principle of the first invention.

FIG. 4 is a diagram showing a conventional technique.

FIG. 5 is a diagram showing an example of data used for input.

[Explanation of symbols]

 1 category management means 2 partial problem setting means 3 sample data storage means 4 function construction means 5 function storage means 6 pattern feature storage means 7 feature distribution means 8 integrated determination means 10-15 symbols representing categories 101 to 104 symbols representing identification functions

Claims (2)

[Claims] 1. A pattern classification method for determining a category to which an object belongs from a plurality of features related to the object. A partial problem with a limited classification destination is set by the number of combinations by allowing overlapping of the classification destinations, and the set portion is set. In the problem, we construct a discriminant function that inputs multiple features related to the subject and output it as a classification destination, select categories that can be classified by specifying which discriminant function to use, and output those discriminant functions. A pattern classification method characterized in that the classification destination is determined by comprehensively judging. 2. A pattern classification device for inputting a plurality of features relating to an object and outputting a category to which the object belongs, a category management means for outputting category information indicating initial value designation and addition / deletion of the category, and category management. The sub-problem setting means that receives the category information from the means, sets the number of combinations of sub-problems with limited classification destinations, and sets the number of combinations, and the setting information output from the sub-problem setting means Received,
In the setting, a plurality of features relating to the object are input, a discriminant function that outputs the classification destination is configured, and a function configuring unit that outputs the setting information and the function information, and the setting information and the function information output from the function configuring unit, A function storage means for storing a limited classification destination only when there is an input from the feature distribution means, a function storage means for storing a feature of data to be classified, and a pattern feature storage means From the feature distribution means that selectively outputs the pattern features read from the function storage means to the functional blocks of the function storage means based on the category information received from the category management means, and only the output of each function block of the function storage means A pattern classification apparatus comprising: an integrated determination unit that receives the output, makes an integrated determination, and outputs a classification destination.