WO2023132061A1

WO2023132061A1 - Training method, information processing device, and training program

Info

Publication number: WO2023132061A1
Application number: PCT/JP2022/000378
Authority: WO
Inventors: 俊也清水; 郁也森川
Original assignee: 富士通株式会社
Priority date: 2022-01-07
Filing date: 2022-01-07
Publication date: 2023-07-13

Abstract

According to the present invention: a determination result is output in which, by using a determination model, each of a plurality of pieces of data, including first data included in training data (X) and second data generated according to the first data and prescribed parameters (w), is determined as the first data or the second data; the prescribed parameters (w) are updated so that the determination result will fail to determine the second data; and the determination model is trained by using new second data generated according to the updated prescribed parameters (w).

Description

Training method, information processing device and training program

The present invention relates to a training method, an information processing device, and a training program.

In recent years, the development and use of systems and services using machine learning have progressed rapidly. On the other hand, various machine learning-specific security problems have also been found. In particular, research is also progressing on pollution attacks that mix data that pollutes machine learning models with training data.

A poisoning attack is an attack that intentionally alters a machine learning model by mixing "unusual data (in other words, contaminated data)" into the training data. Small contaminant data contamination can significantly reduce the accuracy of machine learning models.

JP 2018-92613 A

However, there are poisoning attacks that are difficult to detect with normal anomaly detection technology, and there is a risk that it may not be possible to accurately or quickly determine whether certain data in the training data is intended to be a poisoning attack.

One aspect aims to accurately detect contaminated data and shorten the time required to detect contaminated data.

In one aspect, the training method is such that each of a plurality of data including first data included in training data and second data generated according to the first data and predetermined parameters is A judgment result obtained by judging whether the data is the first data or the second data using the judgment model is output, and a predetermined parameter is updated so that the judgment result fails the judgment of the second data. , the computer executes a process of training the decision model using the new second data generated according to the updated predetermined parameters.

In one aspect, it is possible to accurately detect contaminated data and reduce the time required to detect contaminated data.

It is a figure explaining machine learning in a related example. 1 is a block diagram schematically showing a hardware configuration example of an information processing apparatus as an embodiment; FIG. 3 is a block diagram schematically showing a software configuration example of the information processing apparatus shown in FIG. 2; FIG. FIG. 4 is a flow chart illustrating a tainted data learning phase in a machine learning model as an embodiment; FIG. FIG. 5 is a flow chart illustrating details of a pollution data generation algorithm shown in FIG. 4; FIG. FIG. 4 is a flow chart illustrating a tainted data detection phase in a machine learning model as an embodiment; FIG.

[A] Related Example FIG. 1 is a diagram illustrating machine learning in a related example.

In the training phase indicated by symbol A1, training data (see symbol A11) in which input data x and correct output data y are associated is input. Training is performed based on training data and an empty model (see A12) and a trained model (see A13). A trained model is composed of an empty model and model parameters, as indicated by A14.

In the inference phase indicated by symbol A2, inference (y=f(x)) based on the trained model (see symbol A22) is performed from the query data x (see symbol A21) to obtain the output data y (see symbol A23). ) is output.

For example, the query data x may be an email text, and the output data y may be the spam determination result. Also, for example, the query data x may be an image, and the output data y may be animal species.

All input and output data are expressed as numeric strings. A trained model can be thought of as a function of the form y=f(x), where x, y are vectors. Training is the task of determining a function f suitable for a large number of pairs of x and y.

Machine learning for determining poisoning attacks can be divided into methods that actually train (in other words, a method that measures the impact of contaminated data on a machine learning model) and methods that do not train during operation (in other words, data method for detecting anomalies) and

The method of actually training has the advantage that the accuracy of the machine learning model is higher than the method of not training during operation. On the other hand, the method of actually training has the disadvantage that it is not easy to quickly find contaminated data because the training takes time, and it is difficult to apply to online learning, etc., in which learning is performed sequentially.

In addition, the method that does not require training during operation has the advantage of being able to detect contaminated data quickly with a short execution time, but has the disadvantage of relatively low detection accuracy of contaminated data.

In the method without training during operation, when "normal" data is known to some extent, sanitization is performed to detect contaminated data using normal data distribution. For example, data that is more than a certain distance away from the center point of normal data is regarded as contaminated data and detected.

A method that does not perform training during operation requires a certain amount of information in advance, such as the distribution of "normal data." If we don't know whether the training data is contaminated or not, we can't use it as a reference because we don't know what the normal data is. Then, an adaptive attack (in other words, an attack that knows its defense criteria) may evade detection of tainted data.

[B] Embodiment An embodiment will be described below with reference to the drawings. However, the embodiments shown below are merely examples, and are not intended to exclude the application of various modifications and techniques not explicitly described in the embodiments. In other words, the present embodiment can be modified in various ways without departing from the spirit of the embodiment. Also, each drawing does not mean that it has only the constituent elements shown in the drawing, but can include other functions and the like.

[B-1] Configuration Example FIG. 2 is a block diagram schematically showing a hardware configuration example of the information processing apparatus 1 according to the embodiment.

As shown in FIG. 1, the information processing apparatus 1 includes a CPU 11, a memory section 12, a display control section 13, a storage device 14, an input interface (IF) 15, an external recording medium processing section 16 and a communication IF 17.

The memory unit 12 is an example of a storage unit, and is exemplified by Read Only Memory (ROM) and Random Access Memory (RAM). A program such as a Basic Input/Output System (BIOS) may be written in the ROM of the memory unit 12 . The software programs in the memory unit 12 may be appropriately read into the CPU 11 and executed. Also, the RAM of the memory unit 12 may be used as a temporary recording memory or a working memory.

The display control unit 13 is connected to the display device 131 and controls the display device 131 . A display device 131 is a liquid crystal display, an organic light-emitting diode (OLED) display, a cathode ray tube (CRT), an electronic paper display, or the like, and displays various information for an operator or the like. The display device 131 may be combined with an input device, such as a touch panel.

The storage device 14 is a storage device with high IO performance, and may be, for example, Dynamic Random Access Memory (DRAM), SSD, Storage Class Memory (SCM), or HDD.

The input IF 15 may be connected to input devices such as the mouse 151 and keyboard 152 and may control the input devices such as the mouse 151 and keyboard 152 . The mouse 151 and keyboard 152 are examples of input devices, and the operator performs various input operations via these input devices.

The external recording medium processing unit 16 is configured so that the recording medium 160 can be attached. The external recording medium processing unit 16 is configured to be able to read information recorded on the recording medium 160 when the recording medium 160 is attached. In this example, the recording medium 160 has portability. For example, the recording medium 160 is a flexible disk, optical disk, magnetic disk, magneto-optical disk, or semiconductor memory.

The communication IF 17 is an interface for enabling communication with external devices.

The CPU 11 is an example of a processor, and is a processing device that performs various controls and calculations. The CPU 11 implements various functions by executing an operating system (OS) and programs read into the memory unit 12 . Note that the CPU 11 may be a multiprocessor including a plurality of CPUs, a multicore processor having a plurality of CPU cores, or a configuration having a plurality of multicore processors.

A device for controlling the operation of the entire information processing device 1 is not limited to the CPU 11, and may be, for example, any one of MPU, DSP, ASIC, PLD, and FPGA. Also, the device for controlling the operation of the entire information processing device 1 may be a combination of two or more of CPU, MPU, DSP, ASIC, PLD and FPGA. Note that MPU is an abbreviation for Micro Processing Unit, DSP is an abbreviation for Digital Signal Processor, and ASIC is an abbreviation for Application Specific Integrated Circuit. PLD is an abbreviation for Programmable Logic Device, and FPGA is an abbreviation for Field Programmable Gate Array.

FIG. 3 is a block diagram schematically showing a software configuration example of the information processing device 1 shown in FIG.

The CPU 11 of the information processing apparatus 1 shown in FIG. 2 functions as a feature extraction unit 111, a classifier block 110, and a contamination candidate data generator 114 (contamination candidate data generator h_w). The classifier block 110 has functions as a classifier 112 (classifier D) and a loss calculator 113 .

In the contamination data learning phase indicated by symbol B1 in FIG. 3, a large amount of contamination candidate data is generated in advance, and features and determination methods are learned from the large amount of contamination candidate data. Created before the tainted data detection phase shown.

In the contaminated data learning phase indicated by symbol B1, a data set X for training, testing, etc. is accepted as an input. A dataset may be represented by X=[(x_1, y_1), (x_2, y_2), . . . , (x_n, y_n)]. Here, x_i and y_i are data (in other words, first data) and labels, respectively.

The contamination candidate data generator 114 creates a database h_w(X) of contamination candidate data (in other words, second data) based on the data set X and the parameter w. Parameter w is a parameter of the contamination candidate data generation algorithm.

In the contamination candidate data generation algorithm, the degree of contamination of the label to be contaminated and the machine learning model are parameterized and used as the parameter w. Contamination candidate data generation algorithm includes BGD (Back-gradient Descent; Luis Munoz-Gonzalez, Battista Biggio, Ambra Demontis, Andrea Paudice, Vasin Wongrassamee, Emil C. Lupu, Fabio Roli, “ Towards Poisoning of Deep Learning Algorithms with Back-gradient Optimization”, Aug. 29, 2017) may be used, or other algorithms may be used.

In BGD, an appropriate machine learning model M for generating contamination candidate data is prepared, and a label pair (y_i, y_j) is defined that indicates which label should be tainted so that it is misidentified as which label. A function that is controlled by a parameter ε representing a tradeoff between BGD accuracy and efficiency and generates contamination candidate data from initial value data x is described as bgd(M, y_i, y_j, ε, x). At this time, the contamination candidate data generator h_w and the contamination candidate data can be created by the following procedures (1) to (3).

(1) Train one machine learning model M using dataset X.

(2) Let the parameters be w=(y_1, y_2, ε) and the contamination candidate data database be h_w(X)=bgd(M, w, X).

(3) With the machine learning model M fixed, only the parameter w is updated to repeatedly generate contamination candidate data.

It should be noted that training of the machine learning model M is not necessarily required to create the contamination candidate data generator h_w. For example, by using a method that generates contamination candidate data only from the architecture of the assumed model to be trained (Pang Wei Koh, Percy Liang, “Understanding Black-box Predictions via Influence Functions”, December 29, 2020) The contamination candidate data generator h_w may be created without the machine learning model M already in place.

The feature quantity extraction unit 111 extracts feature quantities from the data set X and the contamination candidate data database h_w(X).

The discriminator 112 discriminates between contamination candidate data and normal data using a machine learning model (in other words, a judgment model). The discriminator 112 is trained to discriminate between contamination candidate data and normal data.

In addition, when distinguishing between contamination candidate data and normal data, the data set X itself may be used, or a data set that has been characterized using a feature extraction means such as Principal Component Analysis (PCA). may be used.

The loss calculation unit 113 updates the parameter w so that the discrimination between the contamination candidate data and the normal data by the discriminator 112 fails. The loss calculator 113 returns, for example, an evaluation result using a loss function representing the inability to discriminate between the data set X and the contamination candidate data database h_w(X) to the contamination candidate data generator 114 as a parameter w.

In the contaminated data detection phase indicated by reference symbol B2 in FIG. 3, when the detection target data set X' is given, the feature amount extraction unit 111 extracts the feature amount of the detection target data set X'. The discriminator 112 detects tainted data by computing features of the data using the discriminator 112 or intermediate layers of the discriminator 112 .

[B-2] Operation Example The contaminated data learning phase in the machine learning model as an embodiment will be described according to the flowchart (steps S1 to S5) shown in FIG.

The feature quantity extraction unit 111 and the contamination candidate data generator 114 receive input of the data set X (step S1).

The contamination candidate data generator 114 executes the contamination data generation algorithm h_w (step S2). Details of the tainted data generation algorithm h_w will be described later with reference to FIG.

The discriminator 112 uses a machine learning model to discriminate between contaminated data and normal data (step S3). Note that the contaminated data and the normal data used for discrimination by the discriminator 112 may be data from which the feature quantity is extracted by the feature quantity extraction unit 111 .

The discriminator 112 is trained to successfully discriminate between contaminated data and normal data (step S4).

The loss calculator 113 updates the parameter w so that the discrimination by the discriminator 112 fails, and returns the updated parameter w to the contamination candidate data generator 114 (step S5). The processing in steps S2 to S5 may be repeatedly executed until a predetermined termination condition (for example, completion of a certain number of loops or no update of parameter w) is satisfied. Then the tainted data learning phase in the machine learning model ends.

Next, the details of the tainted data generation algorithm (step S2) shown in FIG. 4 will be described using the flowchart (steps S21 to S23) shown in FIG.

The contamination candidate data generator 114 uses the dataset X to train one machine learning model M (step S21).

The contamination candidate data generator 114 sets the parameters to w=(y_1, y_2, ε) and the contamination candidate data database to h_w(x)=bgd(M, w, x) (step S22).

The contamination candidate data generator 114 updates the parameter w while keeping the machine learning model M fixed (step S23). Then, the process returns to step S21.

Next, the contaminated data detection phase in the machine learning model as an embodiment will be described according to the flowchart (steps S11 to S13) shown in FIG.

The feature quantity extraction unit 111 receives input of the detection target data set X' (step S11).

The feature amount extraction unit 111 extracts feature amounts from the data set X' (step S12).

The discriminator 112 uses the machine learning model M to discriminate between contaminated data and normal data (step S13). Then the tainted data detection phase in the machine learning model ends.

[C] Effects According to the training method, the information processing device 1, and the training program in the above-described embodiments, the following effects can be obtained, for example.

The discriminator 112 discriminates each of a plurality of data including first data included in the training data and second data generated according to the first data and a predetermined parameter from the first data. A judgment result obtained by judging whether it is the second data or the second data using the judgment model is output. The loss calculation unit 113 updates a predetermined parameter such that the determination result fails the determination of the second data. The discriminator 112 trains the decision model using new second data generated according to the updated predetermined parameters.

As a result, it is possible to accurately detect contaminated data and reduce the time required to detect contaminated data. In addition, detection of contaminated data can be performed without training in the contaminated data detection phase. Furthermore, by precomputing features of various types of contamination data, a wider range of contamination data can be detected than defenses based on normal data.

The discriminator 112 extracts a feature amount from each of the first data and the second data, and performs determination using a determination model on the extracted feature amount. This makes it possible to efficiently determine the first data and the second data.

The tainted candidate data generator 114 generates tainted data for poisoning attacks on the decision model as second data. This makes it possible to detect poisoning attacks in the dirty data detection phase. In addition, the contamination candidate data generator 114 can be trained adversarially by receiving the results of the discriminator block 110 and updating learning. In addition, the classifier block 110 can also be trained adversarially by receiving and learning the input of the data set X and the database h_w(X) of contamination candidate data.

[D] Others The technology disclosed herein is not limited to the above-described embodiments, and various modifications can be made without departing from the spirit of the embodiments. Each configuration and each process of this embodiment can be selected or discarded as necessary, or may be combined as appropriate.

1: information processing device 11: CPU
110: Discriminator block 111: Feature extraction unit 112: Discriminator 113: Loss calculation unit 114: Contamination candidate data generator 12: Memory unit 13: Display control unit 131: Display device 14: Storage device 15: Input IF
151: mouse 152: keyboard 16: external recording medium processing unit 160: recording medium 17: communication IF

Claims

Each of a plurality of data including first data included in training data and second data generated according to the first data and a predetermined parameter is the first data and the second data. Output the judgment result judged using the judgment model, which is the data of
updating the predetermined parameter such that the determination result fails the determination of the second data;
training the decision model using the new second data generated according to the predetermined parameter after updating;
A training method in which the processing is performed by a computer.
extracting a feature amount from each of the first data and the second data;
performing determination using the determination model on the extracted feature amount;
2. The training method of claim 1, wherein the processing is performed by the computer.
generating pollution data for a poisoning attack on the decision model as the second data;
3. The training method according to claim 1 or 2, wherein said computer executes the processing.
Each of a plurality of data including first data included in training data and second data generated according to the first data and a predetermined parameter is the first data and the second data. Output the judgment result judged using the judgment model, which is the data of
updating the predetermined parameter such that the determination result fails the determination of the second data;
training the decision model using the new second data generated according to the predetermined parameter after updating;
An information processing device comprising a processor.
The processor
extracting a feature amount from each of the first data and the second data;
performing determination using the determination model on the extracted feature amount;
The information processing apparatus according to claim 4.
The processor
generating pollution data for a poisoning attack on the decision model as the second data;
The information processing apparatus according to claim 4 or 5.
Each of a plurality of data including first data included in training data and second data generated according to the first data and a predetermined parameter is the first data and the second data. Output the judgment result judged using the judgment model, which is the data of
updating the predetermined parameter such that the determination result fails the determination of the second data;
training the decision model using the new second data generated according to the predetermined parameter after updating;
A training program that makes a computer perform processing.
extracting a feature amount from each of the first data and the second data;
performing determination using the determination model on the extracted feature amount;
8. The training program according to claim 7, causing said computer to execute processing.
generating pollution data for a poisoning attack on the decision model as the second data;
9. The training program according to claim 7 or 8, causing the computer to execute processing.