WO2022116479A1

WO2022116479A1 - End-to-end multi-instance learning method based on automatic instance selection

Info

Publication number: WO2022116479A1
Application number: PCT/CN2021/094142
Authority: WO
Inventors: 詹德川; 王魏; 李新春
Original assignee: 南京智谷人工智能研究院有限公司
Priority date: 2020-12-01
Filing date: 2021-05-17
Publication date: 2022-06-09
Also published as: CN112183752B; CN112183752A

Abstract

An end-to-end multi-instance learning method based on automatic instance selection, comprising the following specific steps: (I) acquiring multi-instance data, and dividing the data into several multi-instance data packets, the multi-instance data packets comprising several instances, and the multi-instance data packets being set as a group of instance sets consisting of the several instances, the multi-instance data packets having labels, and the instances being set as a multi-dimensional vector; (II) building a deep multi-instance network, the deep multi-instance network comprising an instance processing layer, an instance selection layer and a classification layer; and (III) processing each of the multi-instance data packets by means of the deep multi-instance network, and performing training by means of forward or reverse propagation, the training comprising deep multi-instance network training and deep multi-instance network testing. The described method can automatically select important instances by means of an instance selection layer, so that the optimization process of the entire deep network can be trained in an end-to-end manner.

Description

An end-to-end multi-instance learning method based on automatic example selection

technical field

The invention relates to the technical field of machine learning for processing multi-instance data by using a deep network, in particular to an end-to-end multi-instance learning method based on automatic example selection.

Background technique

Traditional machine learning techniques often assume that there is a one-to-one correspondence between samples and labels. For example, in document classification tasks, a document corresponds to a specific category; in image recognition tasks, each image corresponds to a label; in fan fault detection tasks , the sampled signals of a fan over a period of time have the same label. However, in practical tasks, the document contains many sentences, and there are many phrases in the sentences. Different sentences may describe things that involve multiple aspects. Only the things described by some core sentences determine the category of the document. ; Each image can contain multiple objects, only the main objects will be marked; the failure mode of the fan will only appear in a certain time domain or frequency domain.

Consider a document, an image, and a time-series signal as a set of examples, that is, a multi-instance package. The domain range is used as an example, and the problem of labeling granularity can be effectively solved by using multi-instance learning techniques. Multi-example learning assumes that there are some important examples in the multi-example package that determine the category of the example package, so how to automatically select important examples is a very critical technology.

How to select important examples end-to-end in deep networks is a difficult technique to implement, mainly because the process of "example selection" is non-derivative, and the training of deep networks is mainly optimized by gradient propagation. Therefore, the present invention endows the deep multi-instance network with the ability to "auto-select examples" so that the entire optimization process can be performed end-to-end.

SUMMARY OF THE INVENTION

In order to solve the deficiencies of the prior art, the purpose of the present invention is to provide an end-to-end multiplexer based on automatic example selection, which can not only process a scenario where a group of examples corresponds to a single label, but also can effectively realize the automatic selection of examples in a deep network. Learning by Example.

In order to achieve above-mentioned goal, the present invention adopts following technical scheme:

An end-to-end multi-instance learning method based on automatic example selection, comprising the following specific steps: (1) collecting multi-instance data, and dividing the data into several multi-instance data packets, wherein the multi-instance data packets include several and the multi-instance data package is set as a set of examples composed of several examples, the multi-instance data package has a label, and the examples are set as a multi-dimensional vector; (2), build a deep multi-instance network , the deep multi-instance network includes an example processing layer, an example selection layer and a classification layer; (3), each multi-instance data packet is processed through a deep multi-instance network, and is trained by forward or backward propagation. Including deep multi-instance network training and deep multi-instance network testing.

Preferably, in the aforementioned step (1), the multi-instance data collection includes the following specific steps:

100. Determine the target that the example and multi-example data packets refer to in specific tasks;

101. Construct the specific data in the task into multiple example data packets;

102. Assign a label to the multi-example data packet;

103. Organize the data into the form of multiple groups of "(multiple instance data packets, tags)".

Further preferably, in the aforementioned step (2), the construction of a deep multi-instance network includes the following specific steps:

200. Build an example processing layer module;

201. Build an example selection layer module;

202. Build a multi-example data packet classification layer module.

More preferably, in the aforementioned step (3), the training of the deep multi-instance network includes the following specific steps:

300. Prepare a set of "(multi-example data packets, labels)" as training data;

301. Process each instance in the multiple instance data packets through the instance processing layer;

302. Select several examples from all the processed examples through the example selection layer;

303. Aggregate several selected examples;

304. Classify the result obtained by the aggregation through the multi-instance data packet classification layer;

305. Calculate the loss value according to the classification loss function;

306. Optimize all parameters in the network through the gradient optimization method;

307. Repeat 300-306 until the network converges.

Further preferably, in the aforementioned step (3), the deep multi-instance network test includes the following specific steps:

400. Organize the data to be tested into "(multiple example data packets, labels)";

401. Process each instance in the multi-instance data packet through an instance processing layer;

402. Select several examples from all the processed examples through the example selection layer;

403. Aggregate several selected examples;

404. Classify the result obtained by the aggregation through the multi-instance data packet classification layer;

405. Output the prediction result.

The advantages of the present invention are: the present invention can automatically select important examples through the example selection layer, on the one hand, the optimization process of the entire deep network can be trained end-to-end, and on the other hand, it can assist in mining a multi-example package. An important example to enhance the interpretability of the model; the present invention is applicable to a multi-instance data scenario where a group of examples corresponds to a single label, and uses deep learning technology for training and prediction.

Description of drawings

FIG. 1 is a flow chart of multi-example data collection according to an embodiment of the present invention;

FIG. 2 is a flowchart of building a multi-example deep network according to an embodiment of the present invention;

3 is a flowchart of a multi-example deep network training according to an embodiment of the present invention;

FIG. 4 is a flowchart of multi-example deep network prediction according to an embodiment of the present invention.

Detailed ways

The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

With reference to Fig. 1, the multi-instance data collection includes the following specific steps in order: determining the meaning of the example and the multi-instance package in the fan fault diagnosis task (step 100), and the example refers to the fan fault signal in a certain frequency domain range. Each example is represented as a vector of length D, the frequency domain range can be divided into K frequency bands, and the collected fan fault signals can be organized into a set of K D-dimensional vectors, that is, a multi-example package, denoted as {V1,V2,...,VK} (step 101); if the collected fan signal is from a faulty fan, the label is marked as 1, otherwise it is marked as 0 (step 102); all collected data are represented as ({V1, V2,...,VK}, y), y is 0 or 1 (step 103).

Referring to Figure 2, the construction of a deep multi-instance network includes the following specific steps: building an example processing layer module (step 201): the example processing layer can be modeled as a fully connected network, denoted as hi=F(Vi, W_ins), Vi can be is any example, the dimension is D, W_ins is the parameter of the example processing layer, the final output representation is hi, and the dimension is d; build an example selection layer module (step 202): mainly includes a scoring module, and scores each example si =S(hi,W_sel), where si is the score corresponding to the example hi, and W_sei is the relevant parameter; build a multi-example bag classification layer module (step 203): specifically include g=C(h_agg, W_clf), where h_agg is the pair of selection The result of example aggregation, W_clf is the classification parameter, g is the probability distribution of the final prediction, indicating the probability that the example package is faulty.

Referring to Fig. 3, the deep multi-instance network training includes the following specific steps: sampling training data ({V1, V2, ..., VK}, y) (step 300); the example processing layer processes each example data, hi=F(Vi ,W_ins),i=1,2,...,K (step 301); select important examples according to the example selection layer (step 302), first score the examples, si=S(hi, W_sel), and then go through ui=Softmax (log(si+gi)/lambda), gi～Gumbel(0,1) distribution, and then select the Top-jK largest ui, whose subscripts are j1, j2,...,jK in turn; aggregate the selected examples (step 303), for example, take the average h_agg=(Vj1+Vj2+...+VjK)/jK as the aggregated example bag representation; classify g=C(h_agg, W_clf) through the classification layer (step 304); calculate the loss function (step 305), such as calculation through cross-entropy loss; optimize all parameters through gradient backpropagation (step 306), the optimized parameters include W_ins, W_sel, W_clf; finally iterate steps 300-306 until the model converges (step 307).

Referring to Figure 4, the deep multi-instance network testing includes the following specific steps in order: express the collected time series signals in the form of (multi-instance package, ) (step 400); pass the example processing layer, the example selection layer, the aggregation operation and the final classification The layer performs prediction (steps 401, 402, 403, 404); and outputs the fault classification result (405).

The foregoing has shown and described the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the above-mentioned embodiments do not limit the present invention in any form, and all technical solutions obtained by means of equivalent replacement or equivalent transformation fall within the protection scope of the present invention.

Claims

An end-to-end multi-instance learning method based on automatic example selection, which is characterized by comprising the following specific steps: (1), collecting multi-instance data, and dividing the data into several multi-instance data packets, the multi-instance The data package includes several examples, and the multi-instance data package is set as a set of examples composed of several examples, the multi-instance data package has a label, and the examples are set as a multi-dimensional vector; (2), build A deep multi-instance network, the deep multi-instance network includes an example processing layer, an example selection layer, and a classification layer; (3), each multi-instance data packet is processed through a deep multi-instance network, and trained by forward or backpropagation , the training includes deep multi-instance network training and deep multi-instance network testing.
An end-to-end multi-instance learning method based on automatic example selection according to claim 1, wherein in the step (1), multi-instance data collection comprises the following specific steps:

100. Determine the target that the example and multi-example data packets refer to in specific tasks;

101. Construct the specific data in the task into multiple example data packets;

102. Assign a label to the multi-example data packet;

103. Organize the data into the form of multiple groups of "(multiple instance data packets, tags)".
An end-to-end multi-instance learning method based on automatic example selection according to claim 1, wherein in the step (2), the construction of a deep multi-instance network includes the following specific steps:

200. Build an example processing layer module;

201. Build an example selection layer module;

202. Build a multi-example data packet classification layer module.
An end-to-end multi-instance learning method based on automatic example selection according to claim 1, wherein in the step (3), the deep multi-instance network training comprises the following specific steps:

300. Prepare a set of "(multi-example data packets, labels)" as training data;

301. Process each instance in the multiple instance data packets through the instance processing layer;

302. Select several examples from all the processed examples through the example selection layer;

303. Aggregate several selected examples;

304. Classify the result obtained by the aggregation through the multi-instance data packet classification layer;

305. Calculate the loss value according to the classification loss function;

306. Optimize all parameters in the network through the gradient optimization method;

307. Repeat 300-306 until the network converges.
An end-to-end multi-instance learning method based on automatic example selection according to claim 1, wherein in the step (3), the deep multi-instance network test comprises the following specific steps:

400. Organize the data to be tested into "(multiple example data packets, labels)";

401. Process each instance in the multi-instance data packet through an instance processing layer;

402. Select several examples from all the processed examples through the example selection layer;

403. Aggregate several selected examples;

404. Classify the result obtained by the aggregation through the multi-instance data packet classification layer;

405. Output the prediction result.