WO2022199214A1

WO2022199214A1 - Sample expansion method, training method and system, and sample learning system

Info

Publication number: WO2022199214A1
Application number: PCT/CN2022/070604
Authority: WO
Inventors: 詹忆冰; 韩梦雅
Original assignee: 北京沃东天骏信息技术有限公司
Priority date: 2021-03-26
Filing date: 2022-01-07
Publication date: 2022-09-29
Also published as: CN113033665A

Abstract

The present disclosure provides a sample expansion method, a training method and system, and a sample learning system, and relates to the technical field of machine learning. The sample expansion method of the present disclosure comprises: extracting the features of samples via a feature extraction network; determining, on the basis of the features of samples having category labels, a value parameter of each sample having a category label; acquiring the weighted sum of the features of the samples pertinent to a same category in the samples having the category labels to serve as a feature corresponding to the category; determining, on the basis of the similarity between the feature of a sample of which the category is to be determined and the feature of each category, the category of said sample, thus labeling said corresponding sample to acquire an expanded sample.

Description

Sample expansion method, training method and system, and sample learning system

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on the CN application number 202110325808.7 and the filing date is March 26, 2021, and claims its priority. The disclosure of the CN application is hereby incorporated into this application as a whole.

technical field

The present disclosure relates to the technical field of machine learning, and in particular, to a sample expansion method, a training method and system, and a sample learning system.

Background technique

In real life, it is often necessary to classify the categories of objects, such as using a unified representation for the same object to facilitate subsequent object classification, retrieval and other operations. The current classification of items is often obtained through deep learning algorithms. Due to the complexity of the deep model, a large amount of labeled data (item sample data) is often required for training in order to obtain a more robust and highly reliable deep classification model.

In actual scenarios, it is time-consuming and labor-intensive to obtain a large amount of labeled sample data, and in some scenarios, only a small number of labeled samples can be obtained. Therefore, how to learn robust and reliable new category knowledge from limited labeled samples, namely small-sample learning, has important practical value. We call the modeling of the problem of classifying object categories based on a small number of labeled samples in real life as a few-shot learning problem.

The main solution in few-shot learning is Meta-Learning. The core of meta-learning is to hope that the model acquires the ability to "learn to learn", so that it can quickly learn new tasks on the basis of acquiring existing "knowledge". The current meta-learning methods for small sample problems can be roughly divided into two categories:

1. The meta-learning method based on optimization, that is, through a small amount of data, the parameter initialization of the learning model, the learning rate of the model, the gradient update strategy of the model, etc.;

Second, the metric-based meta-learning method, that is, learning the representation of the sample and the class representation belonging to the same class, so as to directly measure the distance between the new sample representation and the class representation, and predict the class of the new sample.

SUMMARY OF THE INVENTION

One object of the present disclosure is to improve the accuracy of sample expansion, and to improve the efficiency and accuracy of sample collection.

According to an aspect of some embodiments of the present disclosure, a method for sample expansion is proposed, including: extracting features of samples through a feature extraction network, where the samples include samples with category annotations and samples with categories to be determined; The feature determines the value parameter of each sample with category annotation; obtains the weighted sum of the features of the samples belonging to the same category in the sample with category annotation, as the feature of the corresponding category, wherein the weight of the feature of the sample is the value parameter; Determine the similarity between the characteristics of the samples of the category and the characteristics of each category, and determine the category of the samples of the category to be determined; label the samples of the corresponding category to be determined with the determined category to obtain extended samples.

In some embodiments, when the sample is an image, extracting the features of the sample through a feature learning extraction network includes: obtaining initial features of the sample through CNN (Convolutional Neural Networks, convolutional neural networks); performing second-order pooling on the initial features processing to obtain the characteristics of the sample.

In some embodiments, in the case that the sample is text information, extracting the features of the samples through the feature learning extraction network includes: extracting the features of the samples through LSTM (Long Short-Term Memory, long short-term memory artificial neural network).

In some embodiments, determining the value parameter of each class-labeled sample according to the feature of the class-labeled sample includes: obtaining the scalar feature of the sample through a layer of nonlinear transformation according to the feature of each class-labeled sample ; Connect each scalar feature in series to obtain the sample set feature vector; according to the sample set feature vector, obtain the value parameter vector through the sample value estimation network, where the elements in the value parameter vector are the value parameters of the sample, and the value parameter vector in the value parameter vector The order of the samples corresponding to the elements matches the order of the samples corresponding to the elements in the sample set feature vector.

In some embodiments, determining the category of the sample of the category to be determined according to the similarity between the feature of the sample of the category to be determined and the feature of each category includes: obtaining the cosine of the feature of the sample of the category to be determined and the feature of each category Similarity: determine the category to which the feature with the largest cosine similarity to the feature of the sample of the category to be determined belongs, as the category of the sample of the category to be determined.

According to an aspect of some embodiments of the present disclosure, a sample extension training method is proposed, including: extracting features of samples through a feature extraction network to be trained, where the samples include training samples with category labels and training samples to be extended; The characteristics of the training samples with category annotations are obtained through the sample value estimation network to be trained to obtain the value parameters of each training sample with category annotations; the weighted sum of the features of the samples belonging to the same category in the training samples with category annotations is obtained as the corresponding The characteristics of the category, wherein the weight of the characteristics of the sample is a value parameter; according to the similarity between the characteristics of the training samples of the category to be determined and the characteristics of each category, the category of the training samples of the category to be determined is determined; The parameters of the trained feature extraction network and the sample value estimation network to be trained increase the similarity between the features of the training samples of the same category to be determined and the features of the corresponding category until the training is completed.

In some embodiments, the sample extension training method further includes: adjusting parameters of the feature extraction network to be trained and the sample value estimation network to be trained based on the second objective function, so as to reduce the similarity of features of different categories.

In some embodiments, adjusting the parameters of the feature extraction network to be trained and the sample value estimation network to be trained based on the first objective function includes: determining by the first objective function according to the characteristics of the category and the characteristics of the training samples of the category to be determined The first target parameter; adjusting the parameters of the feature extraction network to be trained and the sample value estimation network to be trained based on the second target function includes: according to the features of the training samples with categories and the features of the categories, determine the second target function through the second target function. Target parameter; according to the weighted sum of the first target parameter and the second target parameter, adjust the parameters of the feature extraction network to be trained and the sample value estimation network to be trained, so that the weighted sum of the first target parameter and the second target parameter is reduced. Small.

In some embodiments, adjusting the parameters of the feature extraction network to be trained and the sample value estimation network to be trained based on the first objective function includes:

The first target parameter L _cls is determined according to the following formula,

in,

c is the feature of the category, q identifies the feature of the training sample of the category to be determined,

Identifies that the class corresponding to the training sample with class annotation is i,

Identify the training samples to be determined as j,

Identifies the category determined by the training sample j to be determined as iN is the number of categories of training samples with category annotations, M is the number of training samples of the category to be determined; e is a natural constant, c _i is the feature of category i, q _j is the characteristic of the training sample j of the category to be determined;

The probability normalized value of the cosine similarity between the characteristics of the training samples to be determined and the characteristics of each category;

L _cls is reduced by adjusting the parameters of the feature extraction network to be trained and the sample value estimation network to be trained.

In some embodiments, adjusting the parameters of the feature extraction network to be trained and the sample value estimation network to be trained based on the second objective function includes:

The second target parameter L _str is determined according to the following formula,

in,

Identifies that the training sample x with class annotation belongs to class i;

Identifies that the category corresponding to the training sample with category annotation is i, c is the feature of the category, x _ih is the training sample with category annotation belonging to category i, h is a positive integer less than or equal to the number of training samples with category annotation NUM, c _i and c _v are the features of categories i and v respectively; i and v are positive integers less than N, and N is the number of categories of training samples with category labels;

Identify the similarity between the features of the training samples with category labels and the features of the corresponding categories; ∑ _i≠j c _i ^T c _j identifies the differences between the features of the categories;

L _str is reduced by adjusting the parameters of the feature extraction network to be trained and the sample value estimation network to be trained.

According to an aspect of some embodiments of the present disclosure, a sample expansion system is proposed, including: a feature extraction network configured to extract features of samples through the feature extraction network, the samples including samples with category labels and samples of to-be-determined categories; The sample value determination unit is configured to determine the value parameter of each sample with category annotations according to the features of the samples with category annotations; the category feature determination unit is configured to obtain the characteristics of samples belonging to the same category in the samples with category annotations The weighted sum of , as the feature of the corresponding category, wherein the weight of the feature of the sample is the value parameter; the category determination unit is configured to determine the category to be determined according to the similarity between the feature of the sample of the category to be determined and the feature of each category The category of the sample; the sample labeling unit is configured to label the sample of the corresponding to-be-determined category with the determined category, and obtain the extended sample.

In some embodiments, the sample value determination unit includes: a preprocessing subunit, configured to obtain scalar features of the samples through a layer of nonlinear transformation according to the features of each sample with a class label; serialize the scalar features Connect to obtain the feature vector of the sample set; the sample value estimation network is configured to obtain the value parameter vector according to the feature vector of the sample set, wherein the elements in the value parameter vector are the value parameters of the sample, and the elements in the value parameter vector correspond to the values of the samples. The order matches the order of the samples corresponding to elements in the sample set feature vector.

According to an aspect of some embodiments of the present disclosure, a sample extension training system is proposed, including: a feature extraction unit configured to extract features of samples through a feature extraction network to be trained, where the samples include training samples with category labels and samples to be trained The extended training sample; the value determination unit, configured as a second neural network, is configured to obtain the value parameter of each training sample with category annotation through the sample value estimation network to be trained according to the characteristics of the training sample with category annotation ; The category feature determination unit is configured to obtain the weighted sum of the features of the samples belonging to the same category in the training samples with category annotations, as the feature of the corresponding category, wherein the weight of the feature of the sample is the value parameter; is configured to determine the category of the training sample of the category to be determined according to the similarity between the characteristics of the training sample of the category to be determined and the characteristics of each category; the objective function unit is configured to adjust the feature extraction network to be trained based on the first objective function. The value of the samples to be trained estimates the parameters of the network, and the similarity between the characteristics of the training samples of the same category to be determined and the characteristics of the corresponding category increases until the training is completed.

In some embodiments, the objective function unit is further configured to adjust the parameters of the feature extraction network to be trained and the sample value estimation network to be trained based on the second objective function, so that the similarity of different categories of features is reduced.

In some embodiments, the objective function unit is configured to: determine the first objective parameter through the first objective function according to the characteristics of the category and the characteristics of the training samples of the category to be determined; feature, determine the second target parameter through the second target function; according to the weighted sum of the first target parameter and the second target parameter, adjust the parameters of the feature extraction network to be trained and the sample value estimation network to be trained, so that the first target A weighted sum reduction of the parameter and the second target parameter.

According to an aspect of some embodiments of the present disclosure, there is provided a sample learning system, comprising: a memory; and a processor coupled to the memory, the processor being configured to perform any one of the above methods based on instructions stored in the memory.

According to an aspect of some embodiments of the present disclosure, there is provided a computer-readable storage medium having computer program instructions stored thereon, the instructions implementing the steps of any one of the above methods when executed by a processor.

Description of drawings

The accompanying drawings described herein are used to provide further understanding of the present disclosure and constitute a part of the present disclosure. The exemplary embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation of the present disclosure. In the attached image:

FIG. 1 is a flowchart of some embodiments of the sample expansion method of the present disclosure.

FIG. 2 is a flowchart of other embodiments of the sample expansion method of the present disclosure.

FIG. 3 is a flowchart of some embodiments of the sample extension training method of the present disclosure.

FIG. 4 is a flowchart of other embodiments of training based on an objective function in the sample extension training method of the present disclosure.

5 is a schematic diagram of some embodiments of the sample expansion system of the present disclosure.

FIG. 6 is a schematic diagram of some embodiments of a sample value determination unit in the sample expansion system of the present disclosure.

7 is a schematic diagram of some embodiments of the sample extension training system of the present disclosure.

FIG. 8 is a schematic diagram of some embodiments of the sample learning system of the present disclosure.

FIG. 9 is a schematic diagram of other embodiments of the sample learning system of the present disclosure.

Detailed ways

The technical solutions of the present disclosure will be further described in detail below through the accompanying drawings and embodiments.

In the meta-learning method based on optimization, in the inference test phase, the network needs to be fine-tuned on new tasks according to the learned strategy. This process is time-consuming and labor-intensive, and methods for learning optimization objectives have limited utility. The metric-based meta-learning method is to use the trained model to learn the sample category representation on a small number of labeled samples of the new task, so as to directly determine the sample class by the distance between the samples that need to be consulted and classified and the samples that have been classified. category.

The inventor found that, in the related art, when learning a sample category representation through a small number of labeled samples, all samples have the same importance by default. In fact, the amount of effective information contained in different samples and the proportion of effective information in the sample information are different. For example, the same image samples, an image with an annotation target is better than an image full of background in learning sample category representation. For another example, when an image has multiple annotated category information, it often has a negative impact on the final desired category representation, so the actual value of such an image is lower. Defaulting the samples to have the same importance can easily lead to the deviation of the determined features, which is greatly interfered by invalid information, which affects the robustness and reliability of small-sample learning and sample expansion.

A flowchart of some embodiments of the sample expansion method of the present disclosure is shown in FIG. 1 .

In step 101, the features of the samples are extracted through a feature extraction network, and the samples include samples with category labels and samples with categories to be determined. In some embodiments, the feature extraction network can be generated for training based on a neural network.

In some embodiments, if the samples are image samples, the feature extraction network may be based on CNN; if the samples are text samples, the feature extraction network may be based on LSTM. In this way, an appropriate neural network model can be selected, the processing capability of samples can be improved, and the efficiency and accuracy of sample expansion can be improved.

In step 102, according to the characteristics of the samples with category annotations, the value parameter of each sample with category annotations is determined. In some embodiments, the value parameter of the sample may be determined by key feature matching and value parameter assignment based on the key feature. In some embodiments, a neural network can be built, a sample value estimation network can be generated by using training samples for training, and the network can be used to determine the value parameters of the features of different samples with category annotations.

In step 103, the weighted sum of the features of the samples belonging to the same category in the samples with category annotations is obtained as the features of the corresponding category, wherein the weights of the features of the samples are value parameters. The process of sample weighting has higher interpretability, and the category features obtained by weighting have better robustness.

In step 104, the category of the sample of the category to be determined is determined according to the similarity between the feature of the sample of the category to be determined and the feature of each category. In some embodiments, the cosine distance between the features of the samples of the category to be determined and the features of each category may be calculated, the similarity between the features of the samples of the category to be determined and the features of each category may be determined, and the features and the features to be determined may be filtered out. The category with the highest feature similarity of the samples of the category is the category of the samples of the category to be determined.

In step 105, the samples of the corresponding to-be-determined category are marked with the determined category to obtain extended samples.

Through this method, it is possible to take into account the different amount of valid information contained in different small samples, obtain the importance of different small samples, reduce the influence of invalid information on the reliability and accuracy of sample learning, and improve the confidence in the determination of sample types. Therefore, it is beneficial to obtain more robust and accurate samples, improve the reliability and accuracy of sample expansion, reduce the burden of sample collection, and improve the confidence, efficiency and accuracy of sample collection.

A flowchart of other embodiments of the sample expansion method of the present disclosure is shown in FIG. 2 .

In step 201, the features of the samples with class annotations are extracted through a feature extraction network.

In some embodiments, when the sample is an image, the initial features of the sample can be obtained first through CNN. The structure of the convolutional neural network can be 4 residual blocks, each residual block contains 3 3*3 convolutional structures, each of which uses the Relu activation function and uses Batch normalization. There is a 2*2 max pooling between each residual block for downsampling. Its process is defined as:

x _i =CNN(I _i ),i=1,2,...K

where x _i ∈ R ^N×1 .

Further, the second-order pooling is used to obtain fine-grained image representations, that is, the characteristics of samples, and the process is defined as:

in

will eventually

The features are pulled into a column vector namely:

in

In step 202, a scalar feature of the sample is obtained through a layer of nonlinear transformation according to the feature of each class-labeled sample.

Due to the small sample size

The dimension is large, and if all the representations of small samples are directly input into the network, it will have great computational complexity. Therefore, firstly, the feature _zi of all samples with category annotations is transformed through a layer of nonlinear transformation to generate a scalar, which can reduce the computational complexity and improve the processing efficiency. The scalar feature of the sample a _i =Wz _i +b, where a _i records part of the information of the current sample zi _i , W and b are constants that can be specified and adjusted.

In step 203, each scalar feature is serially connected to obtain a sample set feature vector, that is

y=Concat(a _i ), i=1,...K

Among them, y is the feature vector of the sample set, and Concat() is a function of serially concatenating scalar features; the number of samples with category labels is K.

In step 204, a value parameter vector is obtained through the sample value estimation network according to the sample set feature vector, wherein the elements in the value parameter vector are the value parameters of the sample, and the order of the samples corresponding to the elements in the value parameter vector is the same as the sample set feature. The elements in the vector match the order of the corresponding samples, i.e.

e=Sigmoid(Wy+b)

Among them, the i-th element of the sample set feature vector e is the value parameter of the i-th sample with category annotation, and Sigmoid() is a sigmoid function.

In step 205, the weighted sum of the features of the samples belonging to the same category in the samples with category annotation is obtained as the feature of the corresponding category, wherein the weight of the feature of the sample is the value parameter, that is, the feature c of the category is:

Number of samples in the current category

The process of sample weighting has high interpretability, and the category features obtained by weighting have better robustness.

In step 206, the features of the samples of the category to be determined are extracted through the feature extraction network. In some embodiments, step 206 may be performed at any point in time before step 207 , including being performed synchronously with step 201 . In some embodiments, the feature extraction network that extracts the features of the samples of the category to be determined may be the same as the feature extraction network in step 201, or the same feature extraction network.

In step 207, the cosine similarity between the features of the samples of the category to be determined and the features of each category is obtained.

In some embodiments, according to the formula:

d _i =COS(q, _ci )=q ^T c _i /||q||·||c _i ||

Determine the cosine distance between the feature q of the sample of the class to be determined and the feature ci of the class _i .

In step 208, the category to which the feature with the largest cosine similarity to the feature of the sample of the category to be determined belongs is determined as the category of the sample of the category to be determined, that is, the category i of the largest d _i is determined, which is the category to be determined the category of the sample.

In step 209, the samples of the corresponding to-be-determined category are marked with the determined category to obtain extended samples.

Through such a method, the complexity is reduced in the operation process, and the processing efficiency is improved; the samples of the to-be-determined category are classified through the two steps of calculating the features of the category to be determined and the distance between the features, so as to determine the samples of the category to be determined. In the process of determining the category features, the value parameters of each sample are determined through the sample value estimation network, and they are applied by weights, reducing the number of samples in the features. The influence of invalid information on the characteristics of the category improves the reliability and accuracy of the expanded samples.

In order to obtain the neural network used in the sample expansion method, training samples can be collected, some of which are training samples with category labels, and the other are training samples of the category to be determined. A neural network model is constructed and an objective function is designed. Using the training samples Perform training operations.

A flowchart of some embodiments of the sample extension training method of the present disclosure is shown in FIG. 3 .

In step 301, the features of the samples are extracted through the feature extraction network to be trained, and the samples include training samples with category labels and training samples to be expanded. In some embodiments, the features of the training samples with category labels may be obtained first, and the features of the training samples to be expanded are extracted at any time before step 304 .

In some embodiments, a neural network model, such as a CNN or LSTM based network, can be constructed to extract features of the samples.

In step 302, according to the characteristics of the training samples with category annotations, the value parameter of each training sample with category annotations is obtained through the sample value estimation network to be trained.

In some embodiments, the features of the training samples with category annotations may be preprocessed first, and the preprocessing process may be as shown in

steps

202 and 203 .

In some embodiments, a neural network model can be constructed as the sample value estimation network to be trained.

In step 303, the weighted sum of the features of the samples belonging to the same category in the training samples with category annotations is obtained as the features of the corresponding category, wherein the weights of the features of the samples are value parameters.

In step 304, the category of the training sample of the category to be determined is determined according to the similarity between the feature of the training sample of the category to be determined and the features of each category. In some embodiments, the category of the training sample of the category to be determined may be determined by a method similar to that in the above steps 207 and 208 .

In step 305, the parameters of the feature extraction network to be trained and the sample value estimation network to be trained are adjusted based on the first objective function, so as to increase the similarity between the features of the training samples of the same category to be determined and the features of the corresponding category , until the training is completed.

In some embodiments, a first objective function whose operation result is the first objective parameter may be constructed, and the first objective parameter may be reduced by adjusting the parameters of the feature extraction network to be trained and the sample value estimation network to be trained.

In some embodiments, the training is completed when the use of the training samples is completed, or the number of training rounds reaches a predetermined number of times.

Through this method, the feature extraction network and the sample value estimation network can be obtained by training the neural network based on the training samples, so that the importance of different small samples can be obtained in the process of small sample learning, and invalid information can be reduced for sample learning. The influence of reliability and accuracy, improve the reliability and accuracy of sample type determination, and improve the confidence and accuracy of sample expansion.

In some embodiments, the sample extension training method may further include step 306: adjusting the parameters of the feature extraction network to be trained and the sample value estimation network to be trained based on the second objective function, so as to reduce the similarity of different categories of features, Improve the discriminativeness of training for different categories of features.

In some embodiments, a second objective function whose operation result is the second objective parameter can be constructed, and the second objective parameter is reduced by adjusting the parameters of the feature extraction network to be trained and the sample value estimation network to be trained.

Through such a method, the distinction between different categories of features can be regarded as one of the goals in the training process, the degree of differentiation of different categories of features can be increased, the convergence efficiency can be improved, and the operation of the feature extraction network and the sample value estimation network after training can be further improved. effect, improving the confidence and accuracy of the expanded samples.

The flowchart of other embodiments of training based on the objective function in the sample extension training method of the present disclosure is shown in FIG. 4 .

In step 401, according to the characteristics of the category and the characteristics of the training samples of the category to be determined, the first objective parameter is determined by the first objective function.

In some embodiments, according to the formula

Determine the first target parameter L _cls ,

in,

Identifies the class of the training sample j to be classified,

Identify the category determined by the training sample j to be determined as i, N is the number of categories of training samples with category annotations, M is the number of training samples of the category to be determined; e is a natural constant, c _i is the feature of category i , q _j is the feature of the training sample j of the category to be determined;

A probability-normalized value that identifies the cosine similarity between the features of the training samples of the class to be determined and the features of each class.

It can be seen that the smaller the L _cls , the higher the similarity between the features of the training samples belonging to the same category in the training samples of the category to be determined and the features of the category determined for them, that is, the more stable the category features are determined. High, which means that the higher the stability of the sample category determination, the better the convergence effect.

In step 402, a second objective parameter is determined by a second objective function according to the characteristics of the training samples having the category and the characteristics of the category.

In some embodiments, according to the formula

determine the second target parameter L _str ;

in,

Identifies that the training sample x with class annotation belongs to class i;

Identify the similarity between the features of the training samples with category labels and the features of the corresponding categories; ∑ _i≠j c _i ^T c _j identifies the differences between the features of different categories.

It can be seen that the smaller the L _str , the greater the feature difference between categories.

In step 403, according to the weighted sum of the first target parameter and the second target parameter, adjust the parameters of the feature extraction network to be trained and the sample value estimation network to be trained, so that the weighted value of the first target parameter and the second target parameter is and decrease. The smaller the weighted sum of the first target parameter and the second target parameter, the higher the stability of the determination of the sample category, the greater the difference between the features, and the better the convergence effect.

Through this method, the neural network in the system can be expanded by synchronizing training samples from the perspectives of the stability of category features and the differences of different categories of features, so as to improve the training efficiency, and also improve the robustness and confidence of sample expansion. .

A schematic diagram of some embodiments of the sample expansion system of the present disclosure is shown in FIG. 5 .

The feature extraction network 501 can extract the features of the samples extracted by the network, and the samples include samples with category labels and samples with categories to be determined. In some embodiments, the feature extraction network can be generated for training based on a neural network.

The sample value determination unit 502 can determine the value parameter of each class-annotated sample according to the characteristics of the class-annotated sample. In some embodiments, the value parameter of the sample may be determined by key feature matching and value parameter assignment based on the key feature. In some embodiments, a neural network can be built, and a sample value estimation network can be generated by using training samples for training, and the network can be used to determine the value parameters of the features of different samples with category annotations.

The category feature determining unit 503 can obtain the weighted sum of the features of the samples belonging to the same category in the samples with category annotations as the feature of the corresponding category, wherein the weight of the feature of the sample is the value parameter. The process of sample weighting has high interpretability, and the category features obtained by weighting have better robustness.

The category determination unit 504 can determine the category of the samples of the category to be determined according to the similarity between the characteristics of the samples of the category to be determined and the characteristics of each category. In some embodiments, the cosine distance between the features of the samples of the category to be determined and the features of each category may be calculated, the similarity between the features of the samples of the category to be determined and the features of each category may be determined, and the features and the features to be determined may be filtered out. The category with the highest feature similarity of the samples of the category is the category of the determined samples of the category to be determined.

The sample labeling unit 505 can label the samples of the corresponding to-be-determined class with the determined class to obtain extended samples.

Such a sample expansion system can take into account the different amount of valid information contained in different small samples, obtain the importance of different small samples, reduce the influence of invalid information on the reliability and accuracy of sample learning, and improve the robustness of sample type determination. Therefore, it is beneficial to obtain more accurate and high-confidence samples, improve the reliability and accuracy of sample expansion, reduce the burden of sample collection, and improve the reliability, efficiency and accuracy of sample collection.

A schematic diagram of some embodiments of the sample value determination unit in the sample expansion system of the present disclosure is shown in FIG. 6 .

The preprocessing subunit 601 can obtain the scalar feature of the sample in the manner as in steps 201 to 203 .

The sample value estimation network 602 can obtain the value parameter vector according to the feature vector of the sample set, wherein the elements in the value parameter vector are the value parameters of the sample, and the order of the samples corresponding to the elements in the value parameter vector corresponds to the element in the sample set feature vector. match the order of the samples.

Such a system can reduce the computational complexity of sample feature processing, reduce processing pressure, and improve processing robustness and efficiency.

A schematic diagram of some embodiments of the sample extension training system of the present disclosure is shown in FIG. 7 .

The feature extraction unit 701 can extract features of the samples through the feature extraction network to be trained, and the samples include training samples with category labels and training samples to be expanded. In some embodiments, the features of the training samples with category labels may be obtained first, and the features of the training samples to be expanded are extracted at any time before step 304 .

The value determination unit 702 can obtain the value parameter of each training sample with category annotation through the sample value estimation network to be trained according to the characteristics of the training sample with category annotation.

The category feature determining unit 703 can obtain the weighted sum of the features of the samples belonging to the same category in the training samples with category annotations as the features of the corresponding category, wherein the weights of the features of the samples are value parameters.

The category determination unit 704 can determine the category of the training sample of the category to be determined according to the similarity between the characteristics of the training sample of the category to be determined and the characteristics of each category. In some embodiments, the category of the training sample of the category to be determined may be determined by a method similar to that in steps 207 and 208 above.

The objective function unit 705 can adjust the parameters of the feature extraction network to be trained and the sample value estimation network to be trained based on the first objective function, so as to increase the similarity between the features of the training samples of the same category to be determined and the features of the corresponding category. , until the training is completed. In some embodiments, a first objective function whose operation result is the first objective parameter may be constructed, and the first objective parameter may be reduced by adjusting the parameters of the feature extraction network to be trained and the sample value estimation network to be trained.

Such a system can obtain the feature extraction network and the sample value estimation network by training the neural network based on the training samples, so that the importance of different small samples can be obtained during the small sample learning process, and the invalid information can be reduced for the sample learning. Improve the robustness and accuracy of sample type determination, and improve the confidence and accuracy of sample expansion.

In some embodiments, the objective function unit 705 can also adjust the parameters of the feature extraction network to be trained and the sample value estimation network to be trained based on the second objective function, so as to reduce the similarity of different categories of features and improve the model accuracy. Discrimination of different categories of features until the training is completed. In some embodiments, a second objective function whose operation result is the second objective parameter may be constructed, and the second objective parameter may be reduced by adjusting the parameters of the feature extraction network to be trained and the sample value estimation network to be trained.

Such a system can take the distinction of different types of features as one of the goals in the training process, increase the discrimination of different types of features, improve the convergence efficiency, and further improve the operation effect of the feature extraction network and sample value estimation network after training. Improve the robustness and accuracy of the expanded samples.

In some embodiments, the objective function unit 706 can adjust the parameters of the feature extraction network to be trained and the sample value estimation network to be trained according to the weighted sum of the first objective parameter and the second objective parameter, so that the first objective parameter and the A weighted sum reduction of the second target parameter. The smaller the weighted sum of the first target parameter and the second target parameter, the higher the stability of the determination of the sample category, the greater the difference between the features, and the better the convergence effect.

Such a system can expand the neural network in the system by synchronizing training samples from the perspectives of the stability of category features and the differences of different categories of features, improving training efficiency and improving the robustness of sample expansion.

A schematic structural diagram of an embodiment of the sample learning system of the present disclosure is shown in FIG. 8 . The sample learning system includes a memory 801 and a processor 802 . Wherein: the memory 801 may be a magnetic disk, a flash memory or any other non-volatile storage medium. The memory is used to store the instructions in the corresponding embodiments of the sample extension method or the sample extension training method above. The processor 802 is coupled to the memory 801 and may be implemented as one or more integrated circuits, such as a microprocessor or microcontroller. The processor 802 is configured to execute the instructions stored in the memory, which can improve the accuracy of sample type determination and the accuracy of sample expansion.

In one embodiment, as shown in FIG. 9 , the sample learning system 900 includes a memory 901 and a processor 902. Processor 902 is coupled to memory 901 through BUS 903 . The sample learning system 900 can also be connected to an external storage device 905 through a storage interface 904 for recalling external data, and can also be connected to a network or another computer system (not shown) through a network interface 906 . It will not be described in detail here.

In this embodiment, the data instructions are stored in the memory and the above instructions are processed by the processor, so that the accuracy of sample type determination and the accuracy of sample expansion can be improved.

In another embodiment, a computer-readable storage medium stores computer program instructions thereon, and when the instructions are executed by a processor, implements the steps of the method in the corresponding embodiment of the sample extension method or the sample extension training method. As will be appreciated by one skilled in the art, embodiments of the present disclosure may be provided as a method, apparatus, or computer program product. Accordingly, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present disclosure may take the form of a computer program product embodied on one or more computer-usable non-transitory storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein .

The present disclosure is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

So far, the present disclosure has been described in detail. Some details that are well known in the art are not described in order to avoid obscuring the concept of the present disclosure. Those skilled in the art can fully understand how to implement the technical solutions disclosed herein according to the above description.

The methods and apparatus of the present disclosure may be implemented in many ways. For example, the methods and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above unless specifically stated otherwise. Furthermore, in some embodiments, the present disclosure can also be implemented as programs recorded in a recording medium, the programs including machine-readable instructions for implementing methods according to the present disclosure. Thus, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure and not to limit them; although the present disclosure has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand: The disclosed specific embodiments are modified or some technical features are equivalently replaced; without departing from the spirit of the technical solutions of the present disclosure, all of them should be included in the scope of the technical solutions claimed in the present disclosure.

Claims

A sample extension method including:

Extract features of samples through a feature extraction network, and the samples include samples with category labels and samples of to-be-determined categories;

Determine the value parameter of each class-labeled sample according to the features of the class-labeled samples;

Obtaining the weighted sum of the features of the samples belonging to the same category in the samples with category annotations as the features of the corresponding category, wherein the weight of the features of the samples is the value parameter;

Determine the category of the sample of the category to be determined according to the similarity between the feature of the sample of the category to be determined and the feature of each category;

Label the samples of the to-be-determined category corresponding to the determined category to obtain extended samples.
The sample expansion method according to claim 1, wherein, in the case that the sample is an image, the feature of extracting the sample through a feature learning extraction network comprises:

Obtain the initial features of the sample through the convolutional neural network CNN;

The initial features are subjected to a second-order pooling process to obtain the features of the samples.
The sample expansion method according to claim 1, wherein, in the case that the sample is text information, the feature of extracting the sample through a feature learning extraction network comprises:

The features of the samples are extracted by a long short-term memory artificial neural network LSTM.
The sample expansion method according to claim 1, wherein said determining the value parameter of each sample with category annotation according to the characteristics of the sample with category annotation comprises:

According to the features of each class-labeled sample, the scalar features of the samples are obtained through a layer of nonlinear transformation;

serially connect each of the scalar features to obtain a sample set feature vector;

According to the feature vector of the sample set, a value parameter vector is obtained through a sample value estimation network, wherein the elements in the value parameter vector are the value parameters of the sample, and the order of the samples corresponding to the elements in the value parameter vector is the same as that in the value parameter vector. The order of the samples corresponding to the elements in the sample set feature vector matches.
The square sample expansion method according to claim 1, wherein the determining the category of the samples of the category to be determined according to the similarity between the characteristics of the samples of the category to be determined and the characteristics of each category includes:

Obtain the cosine similarity between the features of the samples of the category to be determined and the features of each category;

The category to which the feature with the largest cosine similarity to the feature of the sample of the category to be determined belongs is determined as the category of the sample of the category to be determined.
A sample expansion training method, including:

Extract features of samples through the feature extraction network to be trained, the samples include training samples with category labels and training samples to be expanded;

According to the characteristics of the training samples with category annotations, the value parameters of each training sample with category annotations are obtained through the sample value estimation network to be trained;

Obtain the weighted sum of the features of the samples belonging to the same category in the training samples with category annotations as the features of the corresponding category, wherein the weight of the features of the samples is the value parameter;

According to the similarity between the characteristics of the training samples of the category to be determined and the characteristics of each category, determine the category of the training samples of the category to be determined;

The parameters of the feature extraction network to be trained and the sample value estimation network to be trained are adjusted based on the first objective function, so that the features of the training samples of the to-be-determined category of the same category are similar to the features of the corresponding category increase until training is complete.
The sample extension training method according to claim 6, further comprising:

The parameters of the feature extraction network to be trained and the sample value estimation network to be trained are adjusted based on the second objective function, so as to reduce the similarity of features of different categories.
The sample extension training method according to claim 7, wherein,

The adjusting the parameters of the feature extraction network to be trained and the sample value estimation network to be trained based on the first objective function includes: according to the characteristics of the category and the characteristics of the training samples of the category to be determined, through the first an objective function to determine the first objective parameter;

The adjusting the parameters of the feature extraction network to be trained and the sample value estimation network to be trained based on the second objective function includes: according to the features of the training samples with categories and the features of the categories, through the second The objective function determines the second objective parameter;

It also includes: adjusting the parameters of the feature extraction network to be trained and the sample value estimation network to be trained according to the weighted sum of the first target parameter and the second target parameter, so that the first target A weighted sum reduction of the parameter and the second target parameter.
The sample expansion training method according to claim 6, wherein the adjusting the parameters of the feature extraction network to be trained and the sample value estimation network to be trained based on the first objective function comprises:

The first target parameter L cls is determined according to the following formula,

in,
c is the feature of the category, q identifies the feature of the training sample of the category to be determined,
Identifies that the class corresponding to the training sample with class annotation is i,
Identify the training samples to be determined as j,
The category determined by the training sample j that identifies the category to be determined is i; N is the number of categories of training samples with category annotations, M is the number of training samples of the category to be determined; e is a natural constant, and c i is the feature of category i , q j is the feature of the training sample j of the category to be determined;
The probability normalized value of the cosine similarity between the characteristics of the training samples of the category to be determined and the characteristics of each category;

L cls is reduced by adjusting the parameters of the feature extraction network to be trained and the sample value estimation network to be trained.
The sample extension training method according to claim 7, wherein the adjusting the parameters of the feature extraction network to be trained and the sample value estimation network to be trained based on the second objective function comprises:

The second target parameter L str is determined according to the following formula,

in,
Identifying that the training sample x with category label belongs to category i;
Identifies that the category corresponding to the training sample with category annotation is i, c is the feature of the category, x ih is the training sample with category annotation belonging to category i, h is a positive integer less than or equal to the number of training samples with category annotation NUM, c i and c v are the features of categories i and v respectively; i and v are positive integers less than N, and N is the number of categories of training samples with category labels;
Identify the similarity between the features of the training samples with category labels and the features of the corresponding categories; ∑ i≠j c i T c j identifies the differences between the features of the categories;

L str is reduced by adjusting the parameters of the feature extraction network to be trained and the sample value estimation network to be trained.
A sample expansion system including:

a feature extraction network, configured to extract features of samples through the feature extraction network, the samples including samples with category labels and samples of to-be-determined categories;

a sample value determination unit, configured to determine the value parameter of each class-labeled sample according to the characteristics of the class-labeled samples;

a category feature determination unit, configured to obtain a weighted sum of features of samples belonging to the same category in the samples with category annotations as a feature of the corresponding category, wherein the weight of the feature of the sample is the value parameter;

a category determination unit, configured to determine the category of the samples of the category to be determined according to the similarity between the characteristics of the samples of the category to be determined and the characteristics of each of the categories;

The sample labeling unit is configured to label the corresponding samples of the to-be-determined class with the determined class to obtain extended samples.
The sample expansion system according to claim 11, wherein the sample value determination unit comprises:

The preprocessing subunit is configured to obtain the scalar feature of the sample through a layer of nonlinear transformation according to the feature of each sample with category labeling; serially connect each of the scalar features to obtain the feature vector of the sample set;

The sample value estimation network is configured to obtain a value parameter vector according to the feature vector of the sample set, wherein the elements in the value parameter vector are the value parameters of the sample, and the order of the samples corresponding to the elements in the value parameter vector is the same as that of the sample. The order of the samples corresponding to the elements in the feature vector of the sample set matches.
A sample extension training system, including:

a feature extraction unit, configured to extract features of samples through a feature extraction network to be trained, the samples include training samples with category labels and training samples to be extended;

The value determination unit is configured to obtain the value parameter of each training sample with category annotation through the sample value estimation network to be trained according to the characteristics of the training sample with category annotation;

A category feature determination unit, configured to obtain a weighted sum of features of samples belonging to the same category in the training samples with category annotations, as a feature of the corresponding category, wherein the weight of the features of the samples is the value parameter;

a category determination unit, configured to determine the category of the training samples of the category to be determined according to the similarity between the characteristics of the training samples of the category to be determined and the characteristics of each of the categories;

The objective function unit is configured to adjust the parameters of the feature extraction network to be trained and the sample value estimation network to be trained based on the first objective function, and the characteristics of the training samples of the same category to be determined correspond to The similarity of the features of the categories increases until training is completed.
The sample extension training system according to claim 13, wherein the objective function unit is further configured to adjust parameters of the feature extraction network to be trained and the sample value estimation network to be trained based on a second objective function , in order to reduce the similarity of features of different categories.
The sample expansion training system according to claim 14, wherein, the objective function unit is configured as:

According to the characteristics of the category and the characteristics of the training samples of the category to be determined, the first objective parameter is determined by the first objective function;

According to the characteristics of the training samples with category labels and the characteristics of the categories, the second objective parameter is determined by the second objective function;

Adjust the parameters of the feature extraction network to be trained and the sample value estimation network to be trained according to the weighted sum of the first target parameter and the second target parameter, so that the first target parameter and the The weighted sum reduction of the second target parameter.
A sample learning system including:

memory; and

A processor coupled to the memory, the processor configured to perform the method of any one of claims 1 to 10 based on instructions stored in the memory.
A non-transitory computer-readable storage medium having computer program instructions stored thereon, the instructions, when executed by a processor, implement the steps of the method of any one of claims 1 to 10.