JP6831221B2 - Learning device and learning method - Google Patents

Description

The present invention relates to a learning device and a learning method for learning based on learning data and correctly recognizing unlearned data.

In recent years, a learning device that performs learning based on a plurality of learning data and makes it possible to predict a correct output for unlearned data has been disclosed (see Patent Document 1 and the like). The learning device is well known for machine learning that iteratively analyzes the learning data to determine the tendency and predicts the correct output for the unlearned data.

Japanese Unexamined Patent Publication No. 2015-185149

By the way, in this kind of learning device, it is conceivable to select a feature amount used for learning in order to improve learning efficiency. As this selection method, for example, there is a method of selecting a feature amount from the correct answer rate of a support vector machine, but in this case, there is a problem that the calculation cost becomes high. Further, for example, a method of selecting a feature amount from principal component analysis can be considered. In this case, although the calculation cost can be reduced for the feature amount selection, there is a problem that the calculation cost in the actual machine becomes high.

An object of the present invention is to provide a learning device and a learning method capable of keeping a calculation cost low in both feature quantity selection and learning.

The learning device that solves the above-mentioned problems extracts the feature amount from the information data of the learning target and performs learning using the feature amount as the learning data, so that the unlearned data can be identified. A feature amount selection unit for selecting a feature amount to be used for the learning is provided through a plurality of processes having different methods.

According to this configuration, a plurality of processes capable of selecting the feature amount are provided, and the feature amount is appropriately narrowed down by utilizing the features of each process. Therefore, the features are gradually characterized in a manner that does not require a calculation cost. It is possible to reduce the amount. Therefore, it is possible to keep the calculation cost required for feature quantity selection low. In addition, when learning is performed using the selected features, it is possible to perform learning with a smaller number of features than usual, so that the calculation cost of learning can be kept low. In this way, it is possible to keep the calculation cost low both during feature quantity selection and learning.

In the learning device, the feature amount selection unit has a first selection process of selecting the feature amount from the correlation between the feature amounts and a second selection process of selecting the feature amount through the ease of classifying the discriminant analysis class. Of the third selection process of selecting the feature amount based on the correct answer rate of the learning result of the feature amount, it is preferable to narrow down the feature amount through at least two or more processes. According to this configuration, the first selection process of evaluating the correlation and selecting the feature amount, the second selection process of evaluating the ease of class division of the discriminant analysis and selecting the feature amount, and the correct answer rate of the learning result. Since the feature amount is narrowed down through the third selection process of evaluating and selecting the feature amount, it is advantageous to accurately select the feature amount required for learning.

In the learning device, among the feature amounts excluded in the second selection process, the feature amount considered to be effective in the third selection process is left and added to the third selection process. It is preferable to provide a part. According to this configuration, it is possible to perform the third selection process while leaving the feature amount considered to be effective in the third selection process even if it is not determined to be effective in the second selection process. Therefore, it is advantageous to improve the accuracy of discrimination in learning.

In the learning device, out of all the feature quantities before being selected, those considered to be effective in the third selection process are added to the third selection process by skipping the first selection process and the second selection process. It is preferable to include an effective feature amount selection unit. According to this configuration, it is possible to perform the third selection process while leaving the feature amount considered to be effective in the third selection process even if it is not determined to be effective in the first selection process and the second selection process. Therefore, it is advantageous to improve the accuracy of discrimination in learning.

The learning method for solving the above-mentioned problem is a method of extracting a feature amount from information data to be learned and learning using the feature amount as learning data to make unlearned data identifiable. The learning device is based on a step of selecting a feature amount to be used for the learning through a plurality of processes in which the feature amount selection method is different from each other by the amount selection unit and the feature amount selected by the feature amount selection unit. It is equipped with a step to execute learning.

According to the present invention, the calculation cost can be kept low in both feature quantity selection and learning.

The block diagram of the learning apparatus of 1st Embodiment. (A) is a schematic diagram when registering linear data in the terminal, and (b) is a schematic diagram when the terminal is motion-operated. A flowchart showing the operation of feature selection. Explanatory drawing of the first selection process. Explanatory drawing of the 2nd selection process and the 3rd selection process. The block diagram of the learning apparatus of 2nd Embodiment. A flowchart showing the operation of feature selection. The flowchart which shows the operation of the feature amount selection of 3rd Embodiment.

(First Embodiment)
Hereinafter, the first embodiment of the learning device and the learning method will be described with reference to FIGS. 1 to 5.
As shown in FIG. 1, the learning device 1 includes a data input unit 2 for inputting information data Da to be learned, and a feature amount extraction unit 3 for extracting feature amount x as learning data from a group of collected information data Da. And a learning unit 4 that executes learning (machine learning) based on the feature amount x extracted by the feature amount extraction unit 3. The learning device 1 of this example calculates the discriminant Y (boundary of class discrimination) capable of discriminating the movement given to the terminal 6 (see FIG. 2) such as the electronic key of the vehicle by learning.

The information data Da is preferably an acceleration signal output from, for example, an acceleration sensor (G sensor). The feature amount x is composed of various parameters related to the waveform of the acceleration signal, for example. The feature amount x is constructed from, for example, a primary feature amount x1 and a secondary feature amount x2. The primary feature amount x1 is composed of parameters such as "mean value", "maximum value", "minimum value", "time width", and "slope" related to the waveform of the acceleration signal. The secondary feature amount x2 is a parameter derived from the primary feature amount. Secondary feature quantity x2, for example square feature quantity x ² and which is obtained by the primary feature quantity x1 square and are constructed from such different types of value obtained by multiplying the primary feature quantity x1 together.

The learning unit 4 (learning method of the learning device 1) executes pattern recognition by a support vector machine (SVM) using the feature amount x as learning data. One of the features of the support vector machine is margin maximization, and pattern identification is performed through the discriminant Y obtained by margin maximization. The support vector machine of this example performs two classes of pattern recognition by a linear discriminant Y (called a linear form). The learning unit 4 obtains the discriminant Y from the plurality of features x extracted from the information data Da by the support vector machine, and makes it possible to classify the unlearned data by the discriminant Y which is the boundary of the class discrimination. ..

As shown in FIG. 2A, the learning result discriminant Y is written and registered in a terminal 6 (memory 9 or the like) such as an electronic key of the vehicle. Then, as shown in FIG. 2B, when the terminal 6 is subjected to a motion such as a motion operation by the user, the acceleration signal output from the acceleration sensor mounted on the terminal 6 is used as a base. The movement is class-discriminated from the discriminant Y, and what kind of operation the motion operation is is determined.

Returning to FIG. 1, the learning device 1 includes a feature amount selection unit 12 for selecting the feature amount x used for learning. The feature amount selection unit 12 is provided in the learning unit 4. The feature amount selection unit 12 of this example selects the feature amount x through a plurality of processes in which the feature amount selection methods are different from each other. Specifically, the feature amount selection unit 12 selects the feature amount x by using the first selection process of selecting the feature amount x from the correlation between the feature amounts and the criteria of the ease of class division of the discriminant analysis. Of the two-selection process and the third selection process of selecting the feature amount x based on the correct answer rate of the learning evaluation of the feature amount x, the feature amount x is narrowed down through at least two or more processes.

In the case of this example, the feature amount selection unit 12 narrows down the feature amount x by three-step feature amount selection. In this case, the feature amount selection unit 12 includes a first selection unit 13 that performs the first selection process, a second selection unit 14 that performs the second selection process, and a third selection unit 15 that performs the third selection process. The first selection unit 13 first selects a feature amount, and then selects a necessary feature amount from the normalized feature amount x. The second selection unit 14 further narrows down the feature amount x selected by the first selection unit 13. The third selection unit 15 further narrows down the feature amount x selected by the second selection unit 14.

The learning unit 4 includes a learning processing unit 18 that performs learning based on the feature amount x selected by the feature amount selection unit 12. The learning processing unit 18 performs learning based on the feature amount x input from the feature amount selection unit 12 (third selection unit 15), and calculates a discriminant Y capable of pattern recognition.

Next, the operation and effect of the learning device 1 will be described with reference to FIGS. 3 to 5.
As shown in FIG. 3, the feature amount extraction unit 3 extracts a plurality of (hundreds to thousands) of feature amounts x from the information data Da acquired in advance (step 101). The characteristic amount x, for example, there is such a primary feature quantity x1,2 primary feature quantity x2,2 square feature quantity ^{x 2.}

The first selection unit 13 selects the feature amount x to be used for learning in the feature amount x extracted by the feature amount extraction unit 3 by the first selection process (step 102: first selection process). The first selection process is a process in which the evaluation value of the feature amount x is used as the correlation coefficient and the selection method is set as the threshold value determination. The correlation coefficient is an index showing the relationship between the features x of each other, and it can be said that the higher the value, the higher the relationship. The reason for narrowing down the feature amount x by the correlation coefficient is that if the number of feature amount x is large, in the feature amount selection by the Scatter Matrix (F value) or the support vector machine performed in the feature amount selection in the next and subsequent stages. This is because overfitting may occur.

FIG. 4 illustrates a specific example of feature quantity selection based on the correlation coefficient. In the example of the figure, the absolute value of the correlation coefficient of the feature amount 1 to the feature amount 5 is shown, and an example of narrowing down these is shown. For example, when the threshold value of the correlation coefficient is "0.9", the number of combinations exceeding the threshold value is "3" for the feature amount 1, "3" for the feature amount 2, "2" for the feature amount 3, and "2" for the feature amount 4. “2” and the feature amount 5 are “0”. Of these, the one with the largest number of combinations (the one with the smaller number of features when the number of combinations is the same) is deleted, and the same process is repeated until there are no combinations having a correlation coefficient equal to or higher than the threshold value. In the example of the figure, the feature amounts 3, 4, and 5 are finally selected. The first selection unit 13 outputs the feature amount x selected by the correlation coefficient to the second selection unit 14.

Returning to FIG. 3, the second selection unit 14 selects the feature amount x to be used for learning in the feature amount x narrowed down by the first selection unit 13 by the second selection process (step 103: second selection process). .. The second selection process is a process in which the evaluation value of the feature amount x is set as the F value of the discriminant analysis, and the evaluation method is, for example, forward, backward, SFFS (Sequential Floating Forward Selection) or the like. In the case of this example, the feature amount x is reduced to the number that can be selected by the support vector machine by using the cotter matrix using the operation distribution, the non-operation distribution, and the distance between each distribution as an index.

FIG. 5 illustrates a specific example of feature quantity selection based on the F value of discriminant analysis. In the example of the figure, an example of narrowing down the feature amounts 1 to 5 by the F value of the discriminant analysis is shown. First, when the selection method is "forward selection", the feature amount 1 to 5 having the best evaluation value is added to the combination. Specifically, in the first process, the F value of the feature amount 3 is the highest, so this is added to the combination. In the second processing, the F values of the remaining feature quantities 1, 2, 4, and 5 are obtained for the added feature quantity 3, and the one with the highest F value among these (feature quantity in the example of the figure). 4) is added to the combination. In the third process, the F value with the remaining feature quantities 1, 2 and 5 is obtained for the combination of the added feature quantities 3 and 4, and the one with the highest F value among these (example in the figure). Then, the feature amount 1) is added to the combination. After that, the same process is repeated.

When the selection method is "backward selection", the value that gives the best evaluation when deleted is deleted from all the features 1 to 5. The table in the figure shows what kind of F value the combination of other features takes when the marked features are removed. In the first process, since the F value when the feature amount 3 is removed is the highest, the feature amount 3 is deleted. In the second process, since the F value when the feature amount 4 is removed is the highest, the feature amount 4 is deleted. After that, the same process is repeated.

Further, when the selection method is "SFFS", the feature amount having the best evaluation value is added through the forward selection process, but when the evaluation value is reduced and the evaluation value is improved, the backward selection process is performed to perform the feature amount x. To reduce. In the second selection process of this example, any method of forward selection, backward selection, and SFFS may be used.

The second selection unit 14 selects the optimum combination of feature quantities x through the above-mentioned "forward selection", "backward selection", "SFFS" and the like. When the selected feature quantity x reaches a predetermined number (for example, 30), the second selection unit 14 outputs the combination having the best evaluation value to the third selection unit 15. Alternatively, the second selection unit 14 executes the feature amount selection to the end (all features in the case of forward selection and SFFS, the last one in the case of backward selection), and selects the combination with the best evaluation value as the third selection. Output to unit 15.

Returning to FIG. 3, the third selection unit 15 selects the feature amount x to be used for learning in the feature amount x narrowed down by the second selection unit 14 by the third selection process (step 104: third selection process). .. The third selection process is a process in which the evaluation value of the feature amount x is set as the support vector machine correct answer rate, and the evaluation method is, for example, forward, backward, SFFS, or the like. The third selection unit 15 makes final adjustments for narrowing down the feature amount x by selecting the feature amount by a support vector machine having high expected performance. Here, the feature amount x that reduces the discrimination rate of the support vector machine is reduced.

The feature quantity selection based on the support vector machine correct answer rate is almost the same as the process in which the "F value" shown in FIG. 5 is replaced with the "support vector machine correct answer rate". Therefore, the description of feature quantity selection based on the support vector machine correct answer rate will be omitted here. The third selection unit 15 outputs the feature amount x selected by the support vector machine correct answer rate to the learning processing unit 18.

The learning processing unit 18 executes learning based on the feature amount x selected by the feature amount selection unit 12 (first selection unit 13 to third selection unit 15) (step 105). The learning processing unit 18 of this example executes learning by, for example, a support vector machine, and calculates a discriminant Y capable of pattern-recognizing unlearned data. Then, this discriminant Y is registered in the terminal 6, and the motion operation determination in the terminal 6 becomes possible.

By the way, in the case of this example, a plurality of processes (first selection process, second selection process, third selection process) capable of selecting the feature amount x are provided, and the features of each process are utilized to be suitable. Since the feature amount x is narrowed down, it is possible to gradually reduce the feature amount x in a manner that does not require a calculation cost. Therefore, the calculation cost required for feature quantity selection can be kept low. In particular, in the case of this example, since the feature amount x is narrowed down in the first selection process and the second selection process, the calculation cost in learning the support vector machine correct answer rate performed in the third selection process can be suppressed low.

Further, when learning is performed by the learning processing unit 18 using the selected feature amount x, it is possible to perform learning with a smaller number of feature amounts x than usual, so that the learning calculation cost should be kept low. Can be done. That is, since the feature amount x used for learning is reduced, the calculation cost when mounted on a computer or the like can be reduced. As described above, the calculation cost can be kept low in both feature quantity selection and learning.

The feature amount selection unit 12 selects the feature amount x by using the first selection process (evaluation by the correlation coefficient) for selecting the feature amount x from the correlation between the feature amounts and the criteria of the ease of class division of the discriminant analysis. At least two or more of the second selection process (evaluation based on the F value) and the third selection process (evaluation based on the SVM correct answer rate) for selecting the feature amount x based on the correct answer rate of the learning result of the feature amount x. The feature amount x is narrowed down through the processing (all three in this example). Therefore, since the feature amount x is narrowed down through a plurality of processes that go through these three stages, it is advantageous to accurately select the feature amount x required for learning.

(Second Embodiment)
Next, the second embodiment will be described with reference to FIGS. 6 and 7. This example is an example in which the selection method of the feature amount is changed with respect to the first embodiment. Therefore, the same parts as those in the first embodiment are designated by the same reference numerals, detailed description thereof will be omitted, and only different parts will be described in detail.

As shown in FIG. 6, the learning device 1 includes an effective feature amount selection unit 21 that selects a feature amount x effective for learning processing from the feature amounts x deviating from the main determination loop of feature amount selection. The effective feature amount selection unit 21 is provided in the learning unit 4. The effective feature amount selection unit 21 of this example leaves the feature amount x considered to be effective in the third selection process among the feature amounts x excluded in the second selection process, and adds this to the third selection process. ..

As shown in FIG. 7, the effective feature amount selection unit 21 acquires the feature amount x0 that was not selected in the second selection process (evaluation by the F value of the discriminant analysis), and among these feature amount x0, the third selection. The feature amount x'that is considered to be effective in the processing (evaluation based on the support vector machine correct answer rate) is selected (step 201). The feature amount selection in the same step is preferably an evaluation using, for example, a support vector machine. When the effective feature amount selection unit 21 selects the feature amount x'that is considered to be effective in the third selection process, the effective feature amount selection unit 21 outputs these feature amounts x'to the third selection unit 15.

The third selection unit 15 uses the feature amount x selected in step 103 and the feature amount x'selected in step 201 to select the feature amount x to be used for learning (step 104). Therefore, even if it is not judged to be effective in the second selection process, the third selection process can be performed while leaving the feature amount x'that is considered to be effective in the third selection process, so that the accuracy of discrimination is improved. It will be advantageous to do.

(Third Embodiment)
Next, the third embodiment will be described with reference to FIG. This example will also detail only the parts that differ from the first and second embodiments.

As shown in FIG. 8, the effective feature amount selection unit 21 of this example performs the first selection process and the second selection process on all the feature amounts before being selected, which are considered to be effective in the third selection process. It is skipped and added to the third selection process (step 301). The feature amount selection in the same step is preferably an evaluation using, for example, a support vector machine. When the effective feature amount selection unit 21 selects the feature amount x'that is considered to be effective in the third selection process, the effective feature amount selection unit 21 outputs these feature amounts x'to the third selection unit 15.

The third selection unit 15 uses the feature amount x selected in step 103 and the feature amount x'selected in step 301 to select the feature amount x to be used for learning (step 104). Therefore, even if it is not determined to be effective in the first selection process and the second selection process, the third selection process can be performed while leaving the feature amount x'that is considered to be effective in the third selection process. This is advantageous for improving the accuracy of discrimination.

The embodiment is not limited to the configuration described so far, and may be changed to the following aspects.
-In each embodiment, various methods such as a sketter matrix can be adopted for the evaluation analysis based on the F value of the discriminant analysis.

-In the second and third embodiments, the determination of whether or not the feature amount x is effective in the third selection process is not limited to the process using the support vector machine, and may be changed to another process.
-In each embodiment, the information data Da is not limited to the acceleration data, and may be changed to various data.

-In each embodiment, the discriminant Y for performing the motion determination is not limited to the calculation, and the determination target is not particularly limited.
-In each embodiment, the method of specifying the discriminant Y may be a method other than the examples.

-The discriminant Y, the discriminant function, the determination function, the linear form, etc. described in each embodiment have the same meaning.
-In each embodiment, the support vector machine is not limited to one that adopts a linear pattern recognition method, and may be non-linear, for example.

-In each embodiment, the first selection process is not limited to the evaluation process using the correlation coefficient, and may be any process that can narrow down the feature amount x.
-In each embodiment, the second selection process is not limited to the process using the cluster analysis (F value), and may be any process that can narrow down the feature amount x.

-In each embodiment, the third selection process is not limited to the process using the correct answer rate of the learning evaluation, and may be any process that can narrow down the feature amount x.
-In each embodiment, the number of stages for selecting the feature amount is not limited to three stages, and may be, for example, two stages or four or more stages.

-In each embodiment, the terminal 6 is not limited to the electronic key, and can be changed to another device or device.
-In each embodiment, learning is not limited to the support vector machine, and other methods such as neural networks and boosting may be adopted. Further, the learning may be, for example, perceptron, linear multiple regression analysis, or the like.

Next, the technical idea that can be grasped from the above embodiment and another example will be added below.
(B) In a program that makes it possible to identify unlearned data by extracting feature quantities from information data to be learned and learning using the feature quantities as learning data, the feature quantity selection unit of the learning device features each other. A step of selecting a feature amount to be used for the learning through a plurality of processes having different quantity selection methods, and a step of causing the learning device to execute learning based on the feature amount selected by the feature amount selection unit. A program characterized by having a computer execute.

1 ... Learning device, 2 ... Data input unit, 3 ... Feature amount extraction unit, 4 ... Learning unit, 12 ... Feature amount selection unit, 13 ... First selection unit, 14 ... Second selection unit, 15 ... Third selection unit , 18 ... learning section, 21 ... effective characteristic amount selecting unit, Da ... information ^{data, x (x1, x2, x} 2) ... feature amount.

Claims (2)

In a learning device that makes it possible to identify unlearned data by extracting features from information data to be learned and learning using the features as learning data.
Based on the first selection process that selects the feature amount from the correlation between the feature amounts, the second selection process that selects the feature amount through the ease of class division of the discriminant analysis, and the correct answer rate of the learning result of the feature amount. Of the third selection process for selecting the feature amount, the feature amount is narrowed down through at least two or more processes, and the feature amount used for the learning is obtained through a plurality of processes in which the feature amount selection method is different from each other. Feature selection section to be selected and
An effective feature amount selection unit that skips the first selection process and the second selection process and adds the total feature amount before selection that is considered to be effective in the third selection process to the third selection process. A learning device characterized by being equipped with. In a learning method that makes it possible to identify unlearned data by extracting features from information data to be learned and learning using the features as learning data.
The first selection process of selecting the feature amount from the correlation between the feature amounts by the feature amount selection unit of the learning device, the second selection process of selecting the feature amount through the ease of classifying the discriminant analysis class, and the feature. Of the third selection process of selecting the feature amount based on the correct answer rate of the amount learning result, the feature amount is narrowed down through at least two or more processes, and through a plurality of processes in which the feature amount selection method is different from each other. , The step of selecting the feature amount to be used for the learning, and
The third selection process skips the first selection process and the second selection process for all the feature amounts before being selected by the effective feature amount selection unit of the learning device, which are considered to be effective in the third selection process. Steps to add to the selection process and
A learning method including a step of causing the learning device to execute learning based on the feature amount selected by the feature amount selection unit.

Images