JP2018147107A - Classifier learning device, and device, method and program for class determination - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce computational complexity for determining a class of input data.SOLUTION: A calculation unit calculates a score of each of 2m bits from a score function and data inputted from an input device, as shown by (B). A generation unit generates a ranking of respective scores of 2m bits calculated by the calculation unit, as shown by (C). A giving unit 206 gives 0 or 1 to each of 2m bits on the basis of the ranking of respective scores of 2m bits calculated by the generation unit, as shown by (C). A determination unit determines a class corresponding to the input data on the basis of contents of 0 or 1 given to each of 2m bits by the giving unit 206 and contents of 0 or 1 of respective bits in a plurality of classes, as shown by (D).SELECTED DRAWING: Figure 6

Description

  The present invention relates to a classifier learning device, a class determination device, a method, and a program.

  Conventionally, a classifier that is a technology belonging to the field of information processing / artificial intelligence / machine learning and automatically classifies an input into several predefined classes has been proposed. The process of classifying an input into several predefined classes is one of the basic operations of information processing by a computer.

  FIG. 8 shows an example of a classifier that classifies inputs into predefined labels (classes). As shown in FIG. 8, for example, when a certain document is input, the classifier classifies the document into classes such as politics, economy, and sports. When an image is input, the image is classified into classes such as elephant, car, and tree. Alternatively, actions such as standby, forward movement, and right rotation are determined in a situation where the robot is automatically operated by a computer. Or, it can be classified into classes such as recognizing human instructions and sending mail, connecting to the Internet, or displaying a clock.

  The process of classifying such inputs into corresponding classes is executed when the process is autonomously performed by a computer as part of various information processing systems.

  Here, in order to classify a certain input into a corresponding class as described above, it is necessary to create a classification rule in advance.

  The simplest method can be realized if a human describes a classification rule for each input.

  However, it is difficult for humans to create classification rules in terms of determining the level of abstraction of the classes to be classified, and from the viewpoint of completeness of all possible inputs, which increases costs in terms of both time and cost. Is a problem. A human creates a classification rule and inputs are correctly classified into classes only when the input contents are simple and the number of inputs and the number of output classes are small.

  In addition, classification rules are also created by machine learning from learning data.

  Here, a problem is assumed that the number of handled inputs and the number of output classes are various (see FIG. 8). It is usually difficult for a human to create a classification rule for performing such classification with high accuracy. However, if there is learning data, classification rules can be created using machine learning technology. Thereby, the classifier can classify the input with high accuracy.

  Conventionally, the classifier uses the following model.

Here, x is input data.

Θ is a parameter.
y is a class.

Is a prediction result class.

Y is a set of all classes determined in advance.

S (x, y, Θ) is a function that calculates a score at which class y is selected at input x. The classifier determines the class having the highest score calculated by the function for each class as the class of the input x.

  The function for calculating the score is as follows when a linear model and a neural network are used.

(Linear model)

It is.

w is a parameter vector (weight vector).
f (x, y) is a feature vector.

  In general, an extraction rule is manually determined for each element value of a feature vector.

  Note that Θ = w.

(neural network)

It is.

w is a parameter vector (weight vector).

Is a feature vector.

  The value of each element of the feature vector is determined by a neural network.

It is.

Mohamed, Aly, `` Survey on multiclass classification methods '', 2005, Technical Report, Caltech. Christopher Bishop, "Pattern Recognition and Machine Learning" Springer; 1st ed. 2006

  In the prior art (acquisition of classification rules by machine learning), as the number of classes increases, the calculation cost increases, and the following problems occur in terms of execution speed and required memory.

  The problem of execution speed will be described. For example, when used in robot autonomous behavior, the calculation speed greatly affects performance. In other words, if it takes too much time to determine an action, the action to be selected may not be appropriate.

  The problem of the required memory amount will be described. For example, in an environment where there are limited calculation resources such as a mobile terminal or a sensor node, a method that cannot be calculated without a large amount of memory cannot be put into practical use.

  As described above, conventionally, in order to deal with various situations, it is desired to classify data in detail. Therefore, we want to increase the number of classes to be classified.

  However, when the number of classes to be classified increases, the amount of calculation for class determination increases, and it becomes difficult to use from the viewpoint of execution speed and required memory amount as described above.

  The present invention has been made in view of the above problems, and provides a classifier learning device, a class determination device, a method, and a class determination program capable of reducing the amount of calculation for classifying input data. The purpose is to do.

  The classifier learning device according to the present invention is a plurality of bits each determined by 0 and 1, and any one of a plurality of classes represented by a plurality of bits in which the number of bits to be 1 is predetermined. Learning that learns, as a classifier, a score function for calculating a score at which the bit is 1 for each of the plurality of bits based on learning data that is a plurality of input data assigned Part.

  In the classifier learning method, the learning unit is a plurality of bits each determined by 0 and 1, and any one of a plurality of classes represented by a plurality of bits in which the number of bits to be 1 is predetermined. Based on learning data that is a plurality of input data that has been assigned, a score function for calculating a score of 1 for the input data for each of the plurality of bits is learned as a classifier.

  The class determination device according to the present invention includes a plurality of bits, each of which has a plurality of bits each determined by 0 and 1, and the number of bits to be 1 is represented by a plurality of bits determined in advance. A class determination device for classifying into any of the plurality of bits, using a score function that is pre-learned for calculating a score of 1 for the input data for each of the plurality of bits, Each of the plurality of classes represents a calculation unit that calculates a score in which each of the plurality of bits is 1, a content of 0 or 1 based on the score of each of the plurality of bits calculated by the calculation unit, and each of the plurality of classes A class determining unit that determines a class corresponding to the input data based on the plurality of bits.

  In the class determination method according to the present invention, input data is a plurality of bits each defined by 0 and 1, and a plurality of classes in which the number of bits to be 1 is represented by a plurality of bits determined in advance. A class determination method in a class determination device for classifying into any one of the above, wherein a calculation unit calculates a score function learned in advance for calculating a score at which the bit is 1 for input data for each of the plurality of bits. And calculating a score for each of the plurality of bits to be 1 for the input data, and the class determination unit is a 0 or 1 based on the score of each of the plurality of bits calculated by the calculation unit. Determining a class corresponding to the input data based on content and the plurality of bits representing each of the plurality of classes.

  The program which concerns on this invention is a program for functioning as each part of said classifier learning apparatus or said class determination apparatus.

  As described above, according to the classifier learning device, the class determining device, the method, and the program of the present invention, it is possible to reduce the amount of calculation for determining the class of input data. It is done.

It is a figure which shows the class determination system of embodiment of this invention. 3 is a functional block diagram of a CPU 20 of a class determination device 10. FIG. It is a figure which shows a class determination program. It is a figure which shows the code sequence which each represented K class (20 pieces) by 0 and 1. FIG. It is a figure which shows the upper limit of the number of classes for every value of m. It is a figure explaining a class determination process. (A) shows the calculation amount for determining the conventional class, and (B) is a diagram showing the calculation amount for determining the class of the present embodiment. It is a figure which shows the example of the classifier which classify | categorizes an input into the label (class) defined beforehand.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 shows a class determination system according to an embodiment of the present invention. As shown in FIG. 1, the class determination system includes a class determination device 10, an input device 12, and a display device 14.

  The class determination device 10 is configured by a computer including a CPU 20, a ROM 22, a RAM 24, an input / output port (I / O) 26, and a bus 28. The CPU 20, ROM 22, RAM 24, and input / output port (I / O) 26 are connected to each other via a bus 28.

  The ROM 22 stores a class determination program for the CPU 20 to execute class determination processing (see FIG. 3). The class determination program is read from the ROM 22, expanded in the RAM 24, and executed by the CPU 20.

  FIG. 2 shows a functional block diagram of the CPU 20 of the class determination device 10. When the CPU 20 executes a class determination process (see FIG. 3) described later, the CPU 20 functions as the following units 202 to 208.

  As shown in FIG. 2, the class determination device 10 includes a calculation unit 202 that calculates each score of 2 m bits from a score function described later and data input from the input device 12.

As will be described in detail later, the score function is a correspondence among a plurality of classes in which data input from the input device 12 is represented by 2m bits (where m is a natural number of 1 or more) where 0 and 1 are the same number. It is a score function for calculating the score of each bit according to the class to be performed. More specifically, when the score function assigns 0 or 1 to each bit based on the ranking of the score of each bit, 2m bits to which 0 or 1 is assigned become the contents of the bits of the class corresponding to the data. It is a score function for calculating a score.
Based on learning data that is a plurality of input data to which any of a plurality of classes represented by 2m bits is given by the learning method described later, the class determination device 10 A score function for calculating a score with 1 bit is learned in advance.

  The class determination device 10 receives data based on 0 or 1 content based on each score of 2 m bits calculated by the calculation unit 202 and 0 or 1 content of each bit of a plurality of classes. Is provided with a class determining unit that determines a class corresponding to. The class determination unit includes a creation unit 204, a grant unit 206, and a determination unit 208.

  The class determination unit creation unit 204 creates a ranking of each score of 2 m bits calculated by the calculation unit 202. The assigning unit 206 assigns 0 or 1 to each of the 2m bits based on the ranking of each score of 2m bits created by the creating unit 204. The determination unit 208 determines the class corresponding to the input data based on the content of 0 or 1 given to each of the 2m bits by the grant unit 206 and the content of 0 or 1 of each bit of the plurality of classes. To decide.

Next, the operation of the present embodiment will be described.
First, based on learning data that is a plurality of input data to which any of a plurality of classes represented by 2m bits is assigned, the class determination device 10 sets the bit to 1 for each of a plurality of bits. A score function for calculating a score is learned in advance.
And the class determination apparatus 10 performs a class determination process with respect to the input data using the learned score function.
FIG. 3 shows a flowchart showing an example of the class determination process. In this embodiment, it is determined to which class the input data corresponds.

  FIG. 4 shows a code string in which each of K (20) classes is represented by 0 and 1. FIG. 5 shows the upper limit of the number of classes according to the value of m. FIG. 6 is a diagram for explaining the class determination processing according to the present embodiment.

  As shown in FIG. 4, each class A to T is represented by 2m (m is a natural number of 1 or more) bits where 0 and 1 are the same number. The bit strings of the classes A to T are different from each other. In the above example, m = 3, so each class is represented by 6 bits. Note that the number of 0s and the number of 1s in each class of bit strings are both the same number of three. In each class, 0 and 1 do not have to be the same number. However, in this embodiment, 0 and 1 are the same in each class. The reason why each class is expressed by the same number of 2m bits (where m is a natural number of 1 or more) will be described below.

First, the upper limit of the number of classes that can be expressed by a 2m-bit code (0, 1) will be described.
As described above, if each class is represented by 2m bits and 0 and 1 are the same number, the upper limit of the number of classes is calculated as _2m C _m .
When m = 1, the number of classes that can be expressed is one.
In the case of m = 3, the number of classes that can be expressed is 20.
In the case of m = 16, the number of classes that can be expressed is 600 million or more.
The relationship regarding the upper limit of the number of classes that can be expressed by m and m + 1 is as follows.

  Thus, it can be understood that the relationship regarding the upper limit of the number of classes that can be expressed by m and m + 1 asymptotically approaches four times as m increases.

  Consider that the upper limit of the number of classes represented by 2m bits is maximized under the condition that the number of bits of each class is fixed and each bit is a constant of 0 or 1.

  Of the 2m bits, for example, if the number of 1 is m−1 (the number of 0 is m + 1), the number of classes that can be expressed is as follows.

  In addition, for example, if the number of 1s in 2m bits is m (the number of 0s is also m), the number of classes that can be expressed is as follows.

  Of 2m bits, for example, if the number of 1 is m + 1 (the number of 0 is m-1), the number of classes that can be expressed is as follows.

That means

The relationship holds.

_2m C _m is the maximum number of classes that can be represented. As described above, each class is represented by 2m bits, and 0 and 1 are the same number.

For example,

  In step 302 of FIG. 3, data x is input from the input device 12 as shown in FIG.

  In step 304, as shown in FIG. 6B, the score function (S (x, ε.Θ)) for calculating the score of each bit ε and the input data x are used to calculate each bit ε. Calculate the score. For example, as shown in FIG. 6B, the higher bits 1 to 6 are calculated in order as 0.3, 1.3, 0.5, 0.8, −0.1, 2.5. .

  In step 306, the ranking of each bit is created from the score of each bit. In the above example, as shown in FIG. 6C, the order is 6 bits, 2 bits, 4 bits, 3 bits, 1 bit, and 5 bits.

  In step 308, 1 is assigned to the upper m bits and 0 is assigned to the lower m bits. In the above example, 1 is assigned to 6 bits, 2 bits, and 4 bits, and 0 is assigned to 3 bits, 1 bit, and 5 bits.

  In step 310, from the 2m-bit content to which 0 or 1 is assigned and the 0 or 1 content of a plurality of classes (20 in the above example (see FIG. 4)) to the input data x Determine the corresponding class. As shown in FIG. 6D, in the above example, 010101 is obtained, and from FIG. 4, the class F corresponding to the input data x is determined.

  In step 312, the class (F) is displayed on the display device 14.

  As described above, in the present embodiment, the number of bits that become 1 (or 0) is fixed to m. Thus, when the number of bits that become 1 (or 0) is fixed to m, the determination of the class corresponding to the data x input as described above becomes a problem of ranking of each bit. On the other hand, if the number of bits that become 1 (or 0) is not fixed as in the prior art, class determination becomes a problem of multi-label classification.

  When class determination becomes a problem of ranking of each bit, there are the following advantages.

First, learning (modeling) for determining the score function is easy. Easy reverse class calculation.
Second, no threshold is required. In the case of the problem of multi-label classification, for example, a threshold for an agreement such as “0 or more is set to 1” is required.
Third, in the case of a ranking problem, a relative score between classes can be used.

  The ranking problem only needs to be relatively higher than the score of the bit for which the correct answer is 1 and the score for the 0 is 0. Therefore, the relative score can be used for modeling (if necessary), and the expressive power of the model can be increased.

  Next, a learning method for the score function will be described. The learning of the score function may be performed by the class determination device 10 or may be performed by a classifier learning device different from the class determination device 10. In the present embodiment, the score function is determined by machine learning using learning data. For comparison, a conventional determination method will be described first.

(Conventional)
Learning data

And

x ⁽ⁿ⁾ is the nth input in the learning data.
y ⁽ⁿ⁾ is the correct class for the nth input in the learning data.
N is the total number of learning data.

  In the conventional learning, the parameter Θ is adjusted so that the score in the correct class F is higher than that in other classes. That is, the parameter Θ is adjusted so that the score F> {A, B, C, D, E, G, H,..., T}.

Specifically, the following objective function Ψ is minimized.

However,

S (x, y ′, Θ) is a score function for calculating a score for which the classification class is y ′.
S (x, y, Θ) is a score function for calculating a score for which the classification class is y.
E (y, y ′) is the distance between the classification class y ′ other than the correct answer and the correct classification class y.

  As understood from the specific contents of the objective function, in order to adjust the parameter Θ, a value S (x, y, Θ) obtained from the inputted data x and the correct class y is inputted. It is necessary to calculate the difference between the data S and the value S (x, y ′, Θ) obtained from each class y ′ other than exact.

(This embodiment)
On the other hand, in the present embodiment, it can be formulated as a ranking problem. That is, what is necessary is just to exceed the one where the score of the correct answer class is 1 but the score is 0.

  In the above example (when the correct answer class is F), since the code is 010101, the difference between the score of the second, fourth, and sixth bits from the upper bit and the score of the first, third, and fifth bits occurs. To learn. More specifically, the score of the 2nd, 4th, and 6th bits from the upper bits only needs to be larger than the score of the 1st, 3rd, and 5th bits.

In particular,

It is. The parameter Θ of the score function of each bit is learned so that these are satisfied.

However,

Is learning data.

x ⁽ⁿ⁾ is the nth input data in the learning data.
g ⁽ⁿ⁾ is a set of bit numbers that becomes 1 with the correct class code for the nth input data in the learning data.
ε is a set of integers from 1 to the number of bits of a fixed-length code (2m bits).
N is the total number of learning data.

  As can be understood from the specific contents of the objective function, in order to learn the parameter Θ, the input data x and the bits that become 1 (2, 4 and 6 bits in the above example) e are obtained. A value S (x, e ', Θ) obtained from the value S (x, e, Θ) and the input data x and the bits that become 0 (bits 1, 3, 5 in the above example) e' It is sufficient to calculate the difference between

  In the example described above, the score function is determined by machine learning using learning data, but is not necessarily limited to machine learning, and may be determined using, for example, a neural network or a support vector machine.

Next, the effect of this embodiment will be described.
FIG. 7A shows the calculation amount for determining the conventional class, and FIG. 7B shows the calculation amount for determining the class of the present embodiment.

  Let H be the feature vector of the input data x. This is the same in the conventional and the present embodiment.

  On the other hand, in order to determine the class, it is conventionally necessary to calculate the number of times H × K corresponding to the total number K of classes, whereas in this embodiment, H × B (total number of bits of class (2 m ))).

  The ratio of the conventional calculation amount to the calculation amount of the present embodiment is K / B. The larger K / B is, the more effective the amount of calculation is reduced.

  For example, when K = 250 and B = 10 (m = 5), the amount of calculation can be reduced by (K / B) = 25.2 times.

  When K = 600,000,000 and B = 32 (m = 16), the amount of calculation can be reduced by (K / B) = 18,750,000 times.

  Therefore, in this embodiment, it is possible to solve the super multi-class classification problem at high speed even in a low computing resource environment. This makes it possible to build a setting that allows you to select more actions and detailed actions in an environment such as automatic piloting by machines, which can expand the range of autonomous actions of machines in the real world. It becomes.

  It is to be noted that a class determination process is performed by recording a program for executing the class determination process of the class determination apparatus 10 on a computer-readable recording medium, causing the computer system to read and execute the program recorded on the recording medium. You may perform the various process which concerns on the apparatus 10 mentioned above. Here, the “computer system” may include an OS and hardware such as peripheral devices. Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used. The “computer-readable recording medium” means a flexible disk, a magneto-optical disk, a ROM, a writable nonvolatile memory such as a flash memory, a portable medium such as a CD-ROM, a hard disk built in a computer system, etc. This is a storage device.

  Further, the “computer-readable recording medium” means a volatile memory (for example, DRAM (Dynamic DRAM) in a computer system that becomes a server or a client when a program is transmitted through a network such as the Internet or a communication line such as a telephone line. Random Access Memory)), etc., which hold programs for a certain period of time. The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

10 class determination device 12 input device 14 display device 20 CPU
202 Calculation unit 204 Creation unit 206 Assignment unit 208 Determination unit

Claims (8)

Learning that is a plurality of input data to which any of a plurality of classes, each of which is a plurality of bits each determined by 0 and 1, and in which the number of bits to be 1 is represented by a plurality of predetermined bits. A classifier learning device that learns, as a classifier, a score function for calculating a score at which the bit is 1 for input data for each of the plurality of bits based on data.   The classifier learning according to claim 1, wherein the score function is learned so that a difference between a score of a bit that is 1 and a score of a bit that is 0 is generated in a plurality of bits representing a class assigned to the input data. apparatus. A class determination device that classifies input data into any one of a plurality of classes, each of which is a plurality of bits each determined by 0 and 1, and the number of bits to be 1 is represented by a plurality of predetermined bits. There,
For each of the plurality of bits, a score that is 1 for each of the plurality of bits is calculated for the input data using a pre-learned score function for calculating the score for the bit of 1 for the input data. A calculation unit for calculating,
The class corresponding to the input data is determined based on the content of 0 or 1 based on the score of each of the plurality of bits calculated by the calculation unit and the plurality of bits representing each of the plurality of classes. A class decision unit to
A class decision device comprising: The class determination unit
A creation unit for creating a ranking of each score of the plurality of bits calculated by the calculation unit;
An assigning unit that assigns 0 or 1 to each of the plurality of bits based on the ranking of the scores of the plurality of bits created by the creating unit;
A class corresponding to the input data is determined based on the contents of 0 or 1 assigned to each of the plurality of bits by the assigning unit and the contents of 0 or 1 of each bit of the plurality of classes. A decision unit;
The class determination apparatus according to claim 3.   5. The class determination device according to claim 3, wherein each of the plurality of classes is represented by 2m (m is a natural number of 1 or more) bits in which 0 and 1 are the same number. A classifier learning device is a plurality of bits each having a plurality of bits determined by 0 and 1, and one of a plurality of classes in which the number of bits to be 1 is represented by a plurality of bits determined in advance. A classifier learning method that learns, as a classifier, a score function for calculating a score at which the bit is 1 for the input data for each of the plurality of bits based on learning data that is input data. In a class determination device that classifies input data into any one of a plurality of classes, each of which is a plurality of bits each determined by 0 and 1, and the number of bits to be 1 is represented by a plurality of bits determined in advance. A class determination method,
For each of the plurality of bits, for each of the plurality of bits, each of the plurality of bits is set to 1 for the input data by using a score function that is pre-learned to calculate a score for which the bit is 1 for the input data. To calculate the score
A class determining unit configured to input data based on a content of 0 or 1 based on a score of each of the plurality of bits calculated by the calculating unit and the plurality of bits representing each of the plurality of classes; Determine the corresponding class,
Class determination method including that.   The program for functioning a computer as each part of the classifier learning apparatus of Claim 1 or 2, or the class determination apparatus of any one of Claims 3-5.