JP7044164B2 - Information processing equipment, information processing method, program - Google Patents

Description

The present invention relates to an information processing device for classifying input data, an information processing method, and a program.

With the development of information and communication technology, the accuracy of recognizing the outside world has improved dramatically. In certain fields, the accuracy of classifying objects into known categories from moving images has surpassed that of humans. In particular, the development of deep learning is remarkable, and high classification accuracy is realized by learning Neural Network (NN) in advance using learning data. Such recognition technology is being put to practical use in various fields such as high-speed determination of product defects in factories and detection of pedestrians even in the dark of automatic driving technology.

However, there are still many points that are not as good as humans. Conventional recognition techniques require a large amount of learning data, but a person can learn from even one sample. In other words, it is possible to classify even those that have never been seen. Even the first look can be distinguished from the known categories, and the next time you look at the same kind, you can judge that it is the same as the one you saw before. For example, even if you don't know a giraffe, once you encounter it, the next time it appears, you can tell it's the same as the long-necked animal you've seen before.

Here, some methods for classifying from an unknown sample are known. In Non-Patent Document 1, an image is input to NN, and the output is taken out as a feature vector for classification (see FIG. 1). Specifically, first, the training data is used to train the NN so that the feature vector approaches the average vector of the category. The closeness between vectors is defined by, for example, the Euclidean distance between feature vectors. When the test data is input to the NN learned in this way, it is converted into a feature vector close to the average vector of the category to which the test data belongs. Therefore, by calculating the distances from all the average vectors, the category of the closest average vector can be used as the classification result.

For example, when an unknown image 1 that does not belong to any category is input, it is converted into a feature vector that is far from any average vector learned from the training data. Next, when another unknown image 2 belonging to the same category as the unknown image 1 is input, a feature vector separated from the known average vector and at a position close to the feature vector of the unknown image 1 is output. Therefore, it can be inferred that the unknown image 2 does not belong to the known category and will belong to the same category as the unknown image 1.

The technique described in Non-Patent Document 2 is also a technique for classifying the feature vector of the output of NN trained using the training data (see FIG. 2). Specifically, in the method of Non-Patent Document 2, NNs are learned so that the distance between the feature vectors is close when the two images belong to the same category with respect to the feature vectors for the two images, and when they are different, they are learned from each other. The NN is learned so that the feature vector of is separated by a constant value (for example, a distance of 1). Both methods can be said to be methods for constructing a feature vector space that can form and classify new categories from one sample.

J. Snell, K. Swersky, and R. S. Zemel, “Prototypical networks for few-shot learning,” in NIPS 2017. G. Koch, R. Zemel, and R. Salakhutdinov. “Siamese neural networks for one-shot image recognition,” in ICML Workshop 2015.

However, the above-mentioned method uses only the knowledge of whether or not the feature vectors belong to the same category, and the relationship between the categories is not included in the optimization constraint. Therefore, even if the average vector of the category is obtained after training with the same training data, the position on the feature vector space will be different each time training is performed depending on the initial value. Then, although it is possible to classify the learned categories, it is not always possible to classify unknown categories accurately, and the classification accuracy changes depending on the position of the categories obtained by learning. become. That is, the above-mentioned method has a problem that the accuracy of classifying the input data into categories cannot be improved.

Therefore, an object of the present invention is to provide an information processing device, an information processing method, and a program capable of solving the above-mentioned problem that the accuracy of classifying input data into categories cannot be improved. It is in.

The information processing device, which is one embodiment of the present invention, is
A feature extractor that converts different input data into feature vectors,
A distance calculation unit that calculates a vector distance that represents the distance between the feature vectors of different input data,
A loss calculation unit that calculates error information based on the vector distance is provided.
The loss calculation unit calculates the error information from the loss function based on the relationship information representing the relationship between the categories to which the different input data belong, and based on the error information, the input data by the feature extraction unit is obtained. Update the method of converting to the feature vector,
It takes the composition.

Further, the program which is one form of the present invention is
For information processing equipment
A feature extractor that converts different input data into feature vectors,
A distance calculation unit that calculates a vector distance that represents the distance between the feature vectors of different input data,
In addition to realizing a loss calculation unit that calculates error information based on the vector distance,
The loss calculation unit calculates the error information from the loss function based on the relationship information representing the relationship between the categories to which the different input data belong, and based on the error information, the input data by the feature extraction unit is obtained. Update the method of converting to the feature vector,
To make that happen
It takes the composition.

The information processing method, which is one embodiment of the present invention, is
Convert different input data into feature vectors,
Calculate a vector distance that represents the distance between the feature vectors of different input data.
Error information is calculated based on the vector distance,
Further, the method of calculating the error information from the loss function based on the relationship information representing the relationship between the categories to which the different input data belongs and converting the input data into the feature vector based on the error information is updated. do,
It takes the composition.

The present invention can improve the accuracy of classifying the input data into categories by being configured as described above.

It is a figure for demonstrating the classification method in non-patent document 1. FIG. It is a figure for demonstrating the classification method in Non-Patent Document 2. It is a block diagram which shows the structure of the computer in Embodiment 1 of this invention. It is a functional block diagram which shows the structure of the processor disclosed in FIG. It is a figure for demonstrating an example of the relationship information used for the processing by the loss calculation unit disclosed in FIG. It is a figure for demonstrating an example of the relationship information used for the processing by the loss calculation unit disclosed in FIG. It is a flowchart which shows the processing operation by the processor disclosed in FIG. It is a block diagram which shows the structure of the processor in Embodiment 2 of this invention. It is a flowchart which shows the processing operation by the processor disclosed in FIG. It is a block diagram which shows the structure of the information processing apparatus in Embodiment 3 of this invention.

<Embodiment 1>
The first embodiment of the present invention will be described with reference to FIGS. 3 to 7. 3 to 6 are diagrams for explaining the configuration of the information processing apparatus, and FIG. 7 is a diagram for explaining the operation of the information processing apparatus.

[Constitution]
The present invention comprises a computer 1 as shown in FIG. The computer 1 is an information processing device including hardware such as a processor 10, a CPU (Central Processing Unit), a main memory 30, and an interface 20 used for data input / output. Hereinafter, each configuration will be described in detail.

The interface 20 is used for exchanging data by data input / output means. For example, a network device, a file device, and a sensor device are connected to the interface 20. Then, various information is input / output to / from the processor 10 via the interface 20.

The main memory 30 stores programs and data executed by the processor 10. In the stored program, an instruction group for causing the processor 10 to process in order to execute the processing of each part described later is described.

The processor 10 constructs an input unit 1, a feature extraction unit 2, a distance calculation unit 3, and a loss calculation unit 4 as shown in FIG. 4 by executing a program stored in the main memory 30. As described above, each part 1 to 4 included in the information processing apparatus of the present invention is realized by incorporating software into hardware such as the processor 10. However, each of these parts 1 to 4 may be constructed by hardware such as an IC (Integrated Circuit) circuit.

The above program can be stored in various types of non-transitory computer readable medium and supplied to a computer. Non-temporary computer-readable media include various types of tangible storage media. Examples of non-temporary computer-readable media include magnetic recording media (eg, flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (eg, magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs. Includes CD-R / W, semiconductor memory (eg, mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (Random Access Memory)). The program may also be supplied to the computer by various types of transient computer readable medium. Examples of temporary computer readable media include electrical, optical, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

At the time of learning, the input unit 1 extracts one or a plurality of signal information and the teacher label corresponding to the signal information from the learning data, and outputs the signal information to the feature extraction unit 2. At the time of the test, the signal information is taken out from the test data and output to the feature extraction unit 2. The learning data is composed of a pair of signal information such as images and sounds to be used for learning of the feature extraction unit 2 and a teacher label indicating which category the signal information belongs to. The test data is composed of signal information such as images and sounds to be classified.

The feature extraction unit 2 converts the input signal information into a feature vector by a neural network (NN). For example, a network model consisting of an N-layer convolutional NN and an M-layer fully connected layer N + M layer is used, and the output of the final layer is used as a feature vector. The network model is optimized based on the error information obtained from the loss calculation unit as described later. Existing deep learning methods can be used for optimization.

The distance calculation unit 3 calculates a vector distance representing a distance between feature vectors converted from two signal information to be compared. As the vector distance, Euclidean distance, Mahalanobis distance, cosine distance and the like can be used. At the time of the test, the distance between the feature vector converted from the input data and the feature vector of the representative value of each category is calculated, and the category of the representative value with the closest value is used as the classification result. The representative value of the category is, for example, a vector obtained by averaging the feature vectors belonging to the category, and is calculated using the learning data.

The loss calculation unit 4 obtains a loss function with reference to the given relationship information, and calculates error information from the loss function. Here, the relationship information gives the relationship between the categories, and the distance D between the categories can be obtained by referring to the relationship information. The loss function is defined so that, for example, when the input data to be compared belongs to the same category, the distance between the feature vectors converted from the input data becomes the minimum as the distance approaches 0, and the input data to be compared is different. If it belongs to a category, it is defined so that the distance between the feature vectors becomes the minimum as it approaches the distance D between the categories.

As a specific example, the feature vectors x1 and x2 are input, the Euclidean distance, which is the vector distance between the feature vectors, is U (x1, x2), and the distance between the categories to which each feature vector belongs is D (x1, x2). In this case, the loss function L (x1, x2) is defined as follows.
L (x1, x2) = δ (x1, x2) ・ U (x1, x2) + (1-δ (x1, x2)) ・ (D (x1, x2) -U (x1, x2))
Here, δ (x1, x2) is a function that becomes 1 when the feature vectors x1 and x2 belong to the same class, and 0 when they belong to different classes. The loss function can also be relaxed and set to the following.
L'(x1, x2) = δ (x1, x2) ・ U (x1, x2) + (1-δ (x1, x2)) ・ max (0, (D (x1, x2) -U (x1, x2) )))
Here, max (a, b) is a function that returns the larger value of a and b.

Then, the loss calculation unit 4 uses the result calculated by using the loss function as error information, and changes the method of converting the result into a feature vector by the neural network in the feature extraction unit 2 based on the error information. That is, the loss calculation unit 4 outputs the error information to the feature extraction unit 2, and the feature extraction unit 2 updates the weight of the neural network and the like based on the error information. As a result, the feature extraction unit 2 learns the training data using the updated neural network.

The above-mentioned relationship information may be, for example, a value based on a major category classification, a category system diagram, or the like as described below, and is calculated by an evaluation index specialized for a specific field. It may indicate a relationship based on the similarity between the categories. As an example, as shown in FIG. 5, when a major classification that further includes categories is set, the distance between data belonging to the same category is 0, and the distance between categories that are included in the same major classification is 0. 1. The distance between categories included in different major categories can be 2, and so on. That is, it can be defined that the distance between the categories included in different major categories is larger than the distance between the categories included in the same major category. As a specific example, in the case of two character inputs, it is conceivable that the same character has a distance of 0, different characters have a distance of 1, and different languages have a distance of 2.

Further, as shown in FIG. 6, when a category is represented by a system diagram, that is, a bifurcation diagram having a plurality of branch points, it is possible to define a distance according to the proximity of the system, that is, the number of branches between categories. can. For example, it can be defined that the value of the relationship information becomes smaller as the number of branches decreases. As a specific example, in a phylogenetic diagram showing an animal lineage, it can be defined that the category distance increases as the lineage increases. By referring to the relationships between categories in this way, in the feature vector space, the closer the relationships are, the closer they are placed, and the farther the relationships are, the farther they are placed.

[motion]
Next, the operation of the computer 1, that is, the processor 10 described above will be described with reference to the flowchart of FIG. Here, the operation of updating the neural network using the training data will be mainly described.

First, the input unit 1 randomly acquires two input data from the training data (step S1). When performing K batch processing, input 2 × K data. Subsequently, the feature extraction unit 2 converts 2 data (or 2 × K data) into a feature vector by NN (step S2). At this time, all the data is converted by the same NN.

Subsequently, the distance calculation unit 3 calculates the distance between the two feature vectors (step S3). The vector distance is calculated, for example, by the Euclidean distance. For 2 × K data, calculate the vector distance for the K pair. Then, the loss calculation unit 4 refers to the relationship information, obtains the loss function, calculates the error information, and outputs the error information to the feature extraction unit 2 (step S4). The feature extraction unit 2 updates the NN by updating the weight of the NN based on the error information (step S6), returns to the data input, and repeats the learning as described above. When calculating the error information, the error information is compared with the error information calculated so far, and the process ends when the value does not decrease (Yes in step S5).

As described above, according to the present invention, it is possible to optimize the neural network by updating the weight of the neural network in consideration of the distance based on the relationship between the categories to which the training data belongs, for example, the similarity of the categories. can. As a result, the feature vector is transformed from the input data so as to be arranged in the feature vector space according to the relationship between the categories, for example, the similarity of the categories, so that the feature vector is continuous according to the relationship of the categories. It is arranged in the feature vector space. For example, similar input data will be continuously arranged in the category space. As a result, unknown categories are also arranged according to their similarity, and are separated from known categories according to their similarity. Therefore, even unknown categories can be classified accurately, and the classification accuracy should be improved. Can be done.

<Embodiment 2>
Next, a second embodiment of the present invention will be described with reference to FIGS. 8 to 9. FIG. 8 is a diagram for explaining the configuration of the processor, and FIG. 9 is a diagram for explaining the operation of the processor.

[Constitution]
The processor 1 in the present embodiment includes a relationship calculation unit 5 constructed by executing a program in addition to the configuration described in the first embodiment. Hereinafter, a configuration different from that of the first embodiment will be mainly described.

The relationship calculation unit 5 calculates the relationship information between categories from the learning data. That is, in the first embodiment, the case where the given relationship information is used is illustrated, but in the present embodiment, the loss is calculated using the calculated relationship information.

Specifically, the relationship calculation unit 5 calculates the relationship information using the variance or covariance of the feature vector extracted from the input data. At this time, the relationship calculation unit 5 can also calculate the relationship information between categories after converting all the training data into feature vectors, and each time the training data is input, the converted feature vector is used. It is also possible to calculate the relationship information.

When the relationship calculation unit 5 calculates the relationship information using the covariance, for example, the relationship calculation unit calculates the covariance matrix for each category to which the input data belongs, and the relationship is based on the distance between the covariance matrices. Calculate the information. The distance of the matrix is calculated by the matrix norm, the eigenvalues, and the size of the eigenvectors. For example, the matrix norm (second-order norm) is defined as follows for the matrix A with the element a_ij.
|| A || = √ (Σ_ij (a_ij) ^ 2)

When calculating the relationship using the variance, for example, the relationship information is calculated based on the combination of elements having a small variance. The variance of each element of the feature vector calculated for each category is different. It is also possible to calculate the relationship by the Euclidean distance of the variance vector having the variance as an element. It is also possible to set a certain threshold value, generate a vector in which the element of the variance vector is 1 when the element is equal to or less than the threshold value, and 0 in other cases, and calculate the relationship by the Hamming distance or the binary distance between the vectors. ..

[motion]
Next, the operation of the processor 10 described above will be described with reference to the flowchart of FIG. Here, the operation of updating the neural network using the learning data will be described as in the first embodiment.

First, similarly to steps S1 to S3 of the first embodiment, learning data is input, a feature vector is extracted, and a vector distance is calculated (steps S11, S12, S13). Subsequently, the relationship calculation unit 5 calculates the relationship information using the converted feature vector (step S14). At this time, the variance vector for each category and the covariance matrix between the categories are obtained using the feature vectors converted so far. A vector is generated in which 1 is set when the element of the variance vector is equal to or less than the threshold value and 0 is set in other cases, and the distance between categories is obtained by the Hamming distance between the vectors. Alternatively, calculate the matrix norm between the covariance matrices and find the distance between the categories.

After that, the loss calculation unit 4 refers to the relationship information (distance between categories) calculated by the relationship calculation unit 5, obtains a loss function, generates error information, and outputs it to the feature extraction unit 2 (step). S15). The feature extraction unit 2 updates the NN by updating the weight of the NN based on the error information (step S17), returns to the data input, and repeats the learning as described above. When calculating the error information, the error information is compared with the error information calculated so far, and the process ends when the value does not decrease (Yes in step SS16).

<Embodiment 3>
Next, a third embodiment of the present invention will be described with reference to FIG. FIG. 10 is a block diagram showing the configuration of the information processing apparatus according to the third embodiment. In this embodiment, the outline of the configuration of the computer 1 described in the first embodiment is shown.

As shown in FIG. 10, the information processing apparatus 200 in this embodiment is
A feature extraction unit 210 that converts different input data into feature vectors, and
A distance calculation unit 220 for calculating a vector distance representing a distance between the feature vectors of different input data, and
A loss calculation unit 230 that calculates error information based on the vector distance is provided.
Then, the loss calculation unit 230 calculates the error information from the loss function based on the relationship information representing the relationship between the categories to which the different input data belong, and the feature extraction unit uses the error information to calculate the error information. Update the method of converting the input data to the feature vector,
It takes the composition.

The feature extraction unit 210, the distance calculation unit 220, and the loss calculation unit 230 are realized by the information processing apparatus executing a program.

The information processing device 200 having the above configuration is
Convert different input data into feature vectors,
Calculate a vector distance that represents the distance between the feature vectors of different input data.
Error information is calculated based on the vector distance,
Further, the method of calculating the error information from the loss function based on the relationship information representing the relationship between the categories to which the different input data belongs and converting the input data into the feature vector based on the error information is updated. do,
It works to execute the process.

According to the above invention, the neural network can be optimized by updating the weight of the neural network and the like in consideration of the distance based on the relationship between the categories to which the learning data belongs. This makes it possible to improve the classification accuracy.

<Additional Notes>
Part or all of the above embodiments may also be described as in the appendix below. Hereinafter, the outline of the configuration of the information processing device, the information processing method, and the program in the present invention will be described. However, the present invention is not limited to the following configuration.

(Appendix 1)
A feature extractor that converts different input data into feature vectors,
A distance calculation unit that calculates a vector distance that represents the distance between the feature vectors of different input data,
A loss calculation unit that calculates error information based on the vector distance is provided.
The loss calculation unit calculates the error information from the loss function based on the relationship information representing the relationship between the categories to which the different input data belong, and based on the error information, the input data by the feature extraction unit is obtained. Update the method of converting to the feature vector,
Information processing equipment.

(Appendix 2)
The information processing apparatus described in Appendix 1
The relationship information is information in which the distance between the categories to which the different input data belong is defined.
The loss function is defined so that when different input data belong to different categories, the error information of the value based on the distance between the categories and the vector distance is calculated.
Information processing equipment.

(Appendix 3)
The information processing device described in Appendix 2
The loss function reduces the value of the error information as the vector distance approaches when different input data belong to the same category, and to the distance between the categories when different input data belong to different categories. It is defined that the value of the error information decreases as the vector distance approaches.
Information processing equipment.

(Appendix 4)
The information processing device according to Appendix 2 or 3,
When the category is further configured to be classified into a major category that includes the category, the relationship information has a value when different input data belong to the category classified into the different major categories. , The different input data is defined to be larger than the value when it belongs to the category classified into the same major category.
Information processing equipment.

(Appendix 5)
The information processing device according to Appendix 2 or 3,
When the category is set based on a bifurcation diagram, the relationship information is defined so that the smaller the number of branches between the categories to which the different input data belongs, the smaller the value.
Information processing equipment.

(Appendix 6)
The information processing apparatus according to any one of Supplementary note 1 to 3.
A relationship calculation unit that calculates the relationship information representing the relationship between the categories to which the input data belongs based on the feature vector converted from the input data and the category to which the input data belongs. Prepared,
Information processing equipment.

(Appendix 7)
The information processing apparatus described in Appendix 6
The relationship calculation unit calculates the relationship information based on the variance of the feature vector for each category or the covariance of the feature vector between the categories.
Information processing equipment.

(Appendix 8)
For information processing equipment
A feature extractor that converts different input data into feature vectors,
A distance calculation unit that calculates a vector distance that represents the distance between the feature vectors of different input data,
In addition to realizing a loss calculation unit that calculates error information based on the vector distance,
The loss calculation unit calculates the error information from the loss function based on the relationship information representing the relationship between the categories to which the different input data belong, and based on the error information, the input data by the feature extraction unit is obtained. Update the method of converting to the feature vector,
A program to make that happen.

(Appendix 9)
The program described in Appendix 8
In addition to the information processing device,
A relationship calculation unit that calculates the relationship information representing the relationship between the categories to which the input data belongs, based on the feature vector converted from the input data and the category to which the input data belongs.
A program to realize.

(Appendix 10)
Convert different input data into feature vectors,
Calculate a vector distance that represents the distance between the feature vectors of different input data.
Error information is calculated based on the vector distance,
Further, the method of calculating the error information from the loss function based on the relationship information representing the relationship between the categories to which the different input data belongs and converting the input data into the feature vector based on the error information is updated. do,
Information processing method.

(Appendix 11)
The information processing method described in Appendix 10
Based on the feature vector converted from the input data and the category to which the input data belongs, the relationship information representing the relationship between the categories to which the input data belongs is calculated.
Information processing method.

Although the invention of the present application has been described above with reference to the above-described embodiments and the like, the invention of the present application is not limited to the above-described embodiments. Various changes that can be understood by those skilled in the art can be made to the structure and details of the present invention within the scope of the present invention.

1 Input unit 2 Feature extraction unit 3 Distance calculation unit 4 Loss calculation unit 5 Relationship calculation unit 10 Processor 20 Interface 30 Main memory 100 Computer 200 Information processing device 210 Feature extraction unit 220 Distance calculation unit 230 Loss calculation unit

Claims (10)

A feature extractor that converts different input data into feature vectors,
A distance calculation unit that calculates a vector distance that represents the distance between the feature vectors of different input data,
A loss calculation unit that calculates error information based on the vector distance is provided.
The loss calculation unit calculates the error information from the loss function based on the relationship information representing the relationship between the categories to which the different input data belong, and based on the error information, the input data by the feature extraction unit is obtained. Update the method of converting to the feature vector,
Information processing equipment. The information processing apparatus according to claim 1.
The relationship information is information in which the distance between the categories to which the different input data belong is defined.
The loss function is defined so that when different input data belong to different categories, the error information of the value based on the distance between the categories and the vector distance is calculated.
Information processing equipment. The information processing apparatus according to claim 2.
The loss function reduces the value of the error information as the vector distance approaches when different input data belong to the same category, and to the distance between the categories when different input data belong to different categories. It is defined that the value of the error information decreases as the vector distance approaches.
Information processing equipment. The information processing apparatus according to claim 2 or 3.
When the category is further configured to be classified into a major category that includes the category, the relationship information has a value when different input data belong to the category classified into the different major categories. , The different input data is defined to be larger than the value when it belongs to the category classified into the same major category.
Information processing equipment. The information processing apparatus according to claim 2 or 3.
When the category is set based on a bifurcation diagram, the relationship information is defined so that the smaller the number of branches between the categories to which the different input data belongs, the smaller the value.
Information processing equipment. The information processing apparatus according to any one of claims 1 to 3.
A relationship calculation unit that calculates the relationship information representing the relationship between the categories to which the input data belongs based on the feature vector converted from the input data and the category to which the input data belongs. Prepared,
Information processing equipment. The information processing apparatus according to claim 6.
The relationship calculation unit calculates the relationship information based on the variance of the feature vector for each category or the covariance of the feature vector between the categories.
Information processing equipment. For information processing equipment
A feature extractor that converts different input data into feature vectors,
A distance calculation unit that calculates a vector distance that represents the distance between the feature vectors of different input data,
In addition to realizing a loss calculation unit that calculates error information based on the vector distance,
The loss calculation unit calculates the error information from the loss function based on the relationship information representing the relationship between the categories to which the different input data belong, and based on the error information, the input data by the feature extraction unit is obtained. Update the method of converting to the feature vector,
A program to make that happen. Convert different input data into feature vectors,
Calculate a vector distance that represents the distance between the feature vectors of different input data.
Error information is calculated based on the vector distance,
Further, the method of calculating the error information from the loss function based on the relationship information representing the relationship between the categories to which the different input data belongs and converting the input data into the feature vector based on the error information is updated. do,
Information processing method. The information processing method according to claim 9 .
Based on the feature vector converted from the input data and the category to which the input data belongs, the relationship information representing the relationship between the categories to which the input data belongs is calculated.
Information processing method.

Images