WO2023188241A1 - Generation method, generation program, and information processing device - Google Patents

Generation method, generation program, and information processing device Download PDF

Info

Publication number
WO2023188241A1
WO2023188241A1 PCT/JP2022/016443 JP2022016443W WO2023188241A1 WO 2023188241 A1 WO2023188241 A1 WO 2023188241A1 JP 2022016443 W JP2022016443 W JP 2022016443W WO 2023188241 A1 WO2023188241 A1 WO 2023188241A1
Authority
WO
WIPO (PCT)
Prior art keywords
candidate data
candidate
training
adversarial
data
Prior art date
Application number
PCT/JP2022/016443
Other languages
French (fr)
Japanese (ja)
Inventor
海斗 岸
郁也 森川
俊也 清水
Original Assignee
富士通株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 富士通株式会社 filed Critical 富士通株式会社
Priority to PCT/JP2022/016443 priority Critical patent/WO2023188241A1/en
Publication of WO2023188241A1 publication Critical patent/WO2023188241A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning

Definitions

  • the present invention relates to a generation method, a generation program, and an information processing device.
  • an adversarial sample attack is known that causes misclassification in a machine learning model (classification model) that classifies data.
  • Performing machine learning for a classification model can be said to find boundaries (decision boundaries) in the feature space that can separate data points to be classified into different classes.
  • boundaries decision boundaries
  • data points that slightly exceed the decision boundary are intentionally created to induce misclassification of the model.
  • attack data points created in the adversarial sample attack may also be referred to as adversarial samples.
  • FIG. 7 is a diagram illustrating an adversarial sample.
  • symbol A is image data representing the number 7
  • symbols B and C represent adversarial samples generated based on the image data indicated by symbol A.
  • FIG. 8 is a diagram for explaining adversarial samples.
  • Adversarial samples are generated based on training data, that is, classified data points (hereinafter referred to as original points).
  • the perturbation size which is the distance between the original point and the adversarial sample, is denoted by the symbol ⁇ .
  • adversarial training for classification models is known as a defense method against adversarial sample attacks.
  • adversarial samples are added to the training data of the classification model, and the decision boundaries corresponding to the adversarial samples are updated.
  • candidate points for adversarial samples are generated based on the original points.
  • the target class is a class whose classification destination is to be changed from the original class using an adversarial sample.
  • the distance between the candidate point and the original point is calculated.
  • the adversarial sample candidates are updated so that the sum (a+b) of the confidence (a) of the candidate point and the distance (b) between the candidate point and the original point is minimized, and the candidate closest to the original point is selected. , is determined as the adversarial sample to generate.
  • the sum (a + b) of the confidence of the candidate point (a) and the distance (b) between the candidate point and the original point is minimized.
  • the adversarial sample candidates are updated as follows. As a result, the value of the distance (b) between the candidate point and the original point is also minimized, and the perturbation size changes. Therefore, it is difficult to generate adversarial samples with a constant perturbation size, and it is difficult to use them for adversarial training using a constant perturbation size.
  • the present invention aims to enable efficient generation of adversarial samples having a specific perturbation size.
  • this generation method is a generation method for generating adversarial samples used for training a class classification model, and includes a process for generating a plurality of candidate data based on training data used for training the class classification model. , the confidence that each of the plurality of candidate data is classified into the class associated with the training data, and a sphere whose radius is the target perturbation size centered on the training data from each of the plurality of candidate data. and determining a hostile sample from among the plurality of candidate data based on the distance to the surface.
  • adversarial samples with a specific perturbation size can be efficiently generated.
  • FIG. 1 is a diagram schematically showing the configuration of an information processing device as an example of an embodiment.
  • FIG. 2 is a diagram for explaining processing by a hostile sample candidate updating unit of an information processing device as an example of an embodiment.
  • 7 is a flowchart for explaining processing of a hostile sample generation unit of an information processing device as an example of an embodiment.
  • FIG. 3 is a diagram illustrating the generation results of hostile samples by the hostile sample generation unit of the information processing device as an example of the embodiment in comparison with a conventional method.
  • FIG. 3 is a diagram illustrating the generation results of hostile samples by the hostile sample generation unit of the information processing device as an example of the embodiment in comparison with a conventional method.
  • 1 is a diagram illustrating a hardware configuration of an information processing device as an example of an embodiment.
  • FIG. 3 is a diagram illustrating an adversarial sample.
  • FIG. 2 is a diagram for explaining an adversarial sample.
  • FIG. 1 is a diagram schematically showing the configuration of an information processing device 1 as an example of an embodiment.
  • the information processing device 1 has a function as a training data generation unit 100 that generates training data used in adversarial training.
  • the training data generation unit 100 has a function as an adversarial sample generation unit 101 that generates adversarial samples.
  • the adversarial sample generation unit 101 has the functions of a candidate point generation unit 102, a confidence calculation unit 103, a distance calculation unit 104, and an adversarial sample candidate update unit 105.
  • the candidate point generation unit 102 generates candidate data that is a candidate for an adversarial sample based on training data used for training a class classification model (machine learning model).
  • the training data has already been classified. Such classified training data may be referred to as original points.
  • candidate data that is a candidate for a hostile sample may be referred to as a candidate point.
  • candidate data that is a candidate for a hostile sample may be referred to as hostile sample candidate data or hostile sample candidate points.
  • a class classification model may simply be called a classification model.
  • the candidate point generation unit 102 may generate candidate points, for example, by copying the original point. That is, in the feature space, a point having the same coordinates as the original point may be used as a candidate point. Further, the candidate point generation unit 102 may generate candidate points by randomly moving the original point within a predetermined range in the feature space. That is, the candidate point generation unit 102 may generate candidate points by randomly moving the original point a little.
  • Information on candidate points generated by the candidate point generation unit 102 is stored in a predetermined storage area such as the memory 12 or the storage device 13 (see FIG. 6).
  • the certainty calculation unit 103 calculates the certainty that the candidate point is classified into a class other than the target class.
  • the confidence level corresponds to the confidence level of classification into the class associated with the training data.
  • the target class is a class whose classification destination is to be changed from the original class using an adversarial sample. For example, in FIG. 8, the class indicated by an x corresponds to the target class for the original point indicated by a circle.
  • Confidence is a predicted value of the degree of certainty of classification.
  • the confidence that a candidate point is classified into a class other than the target class can simply be called the confidence of the candidate point. Further, the confidence level of a candidate point may be expressed by the symbol s.
  • the confidence calculation unit 103 stores the calculated confidence s of the candidate point in a predetermined storage area such as the memory 12 or the storage device 13.
  • the distance calculation unit 104 calculates the distance (shortest distance) between the candidate point and the spherical surface whose center is the original point and whose radius is the perturbation size ⁇ 0 .
  • the perturbation size ⁇ 0 is the perturbation size of the adversarial sample generated by the adversarial sample generation unit 101, and may be set to an arbitrary value by the user or the like.
  • the perturbation size ⁇ 0 may be referred to as the target perturbation size ⁇ 0 .
  • a spherical surface whose center is the original point and whose radius is the target perturbation size ⁇ 0 may be referred to as the target spherical surface.
  • the distance calculation unit 104 calculates, for example, the distance between the candidate point and the target spherical surface.
  • the distance between the candidate point and the target sphere surface may be, for example, a Euclidean distance.
  • the distance between the candidate point and the target sphere surface may be expressed by the symbol d.
  • the distance calculation unit 104 stores the calculated distance d between the candidate point and the target sphere surface in a predetermined storage area such as the memory 12 or the storage device 13.
  • the adversarial sample candidate updating unit 105 updates the candidate point ( multiple candidate points are generated by updating the adversarial sample candidate points).
  • the updating of a candidate point by the adversarial sample candidate updating unit 105 may also be referred to as moving the candidate point.
  • the adversarial sample candidate updating unit 105 generates a plurality of candidate data (candidate points) based on training data (original points).
  • FIG. 2 is a diagram for explaining processing by the hostile sample candidate updating unit 105 of the information processing device 1 as an example of the embodiment.
  • the adversarial sample candidate updating unit 105 updates the candidate points so that the value of (cs + d) is minimized.
  • c is a coefficient indicating which of the confidence level s of the candidate point and the distance d between the candidate point and the target sphere surface is given higher priority.
  • the gradient vector of the confidence level s 1 of the candidate point P1 is represented by the symbol S 1 .
  • the gradient vector of the distance d 1 between the candidate point P 1 and the surface of the target sphere is represented by the symbol D 1 .
  • the candidate point P1 is generated by the candidate point generation unit 102 copying the original point P0.
  • the original point P0 and the candidate point P1 have the same coordinates in the feature amount space.
  • the gradient vector with the confidence level s 1 of the candidate point P1 is represented by the symbol S 1 .
  • the gradient vector of the distance d 1 between the candidate point P 1 and the surface of the target sphere is represented by the symbol D 1 .
  • the adversarial sample candidate updating unit 105 generates a candidate point P2 by updating the candidate point P1 so that the value of (cs 1 + d 1 ) is minimized. That is, the hostile sample candidate updating unit 105 updates the candidate point P1 to generate a candidate point P2.
  • the vector corresponding to the confidence level s2 of the candidate point P2 is represented by the symbol S2 .
  • a vector corresponding to the distance d 2 between the candidate point P2 and the surface of the target sphere is represented by the symbol D 2 .
  • the adversarial sample candidate updating unit 105 generates a candidate point P3 by updating the candidate point P2 so that the value of (cs 2 + d 2 ) is minimized. That is, the hostile sample candidate updating unit 105 updates the candidate point P2 to generate a candidate point P3.
  • the gradient vector with the confidence level s3 of the candidate point P3 is represented by the symbol S3 .
  • the gradient vector of the distance d 3 between the candidate point P3 and the surface of the target sphere is represented by the symbol D 3 .
  • the hostile sample candidate updating unit 105 generates new candidate points by updating (moving) the previously generated candidate points.
  • the adversarial sample candidate update unit 105 stores information about each candidate point generated by the update in a predetermined storage area such as the memory 12 or the storage device 13.
  • the adversarial sample generation unit 101 determines, as an adversarial sample, the candidate point whose perturbation size is closest to the target perturbation size ⁇ 0 from among the plurality of candidate points generated by the adversarial sample candidate update unit 105 .
  • the adversarial sample generation unit 101 generates a spherical surface whose radius is ⁇ 0 and the target perturbation size centered on the original point (training data) from each of the plurality of candidate data and the confidence s of each of the plurality of candidate data.
  • An adversarial sample is determined from among a plurality of candidate points (candidate data) based on the distance d to.
  • step S1 the candidate point generation unit 102 updates the coefficient c by, for example, binary search.
  • the certainty calculation unit 103 calculates the certainty s that the candidate point to be processed is classified into a class other than the target class.
  • the candidate point to be processed corresponds to candidate data extracted from a plurality of candidate data (candidate points), and the candidate point generated by the candidate point generation unit 102 based on the original point and the adversarial sample generation unit 101 are combined. , a plurality of candidate points (adversarial sample candidate points) generated based on the original point, and a plurality of candidate points (candidate point data).
  • step S5 the candidate point generation unit 102 checks whether the processes of steps S2 to S4 have been executed a specified number of times, that is, whether the first termination condition is satisfied. As a result of the confirmation, if the first termination condition is not satisfied, that is, if the number of processing steps S2 to S4 has not reached the specified number of times (see No route in step S5), the process returns to step S2.
  • step S6 if the first termination condition is satisfied, that is, if the number of times steps S2 to S4 are processed reaches the specified number (see the Yes route in step S5), the process moves to step S6.
  • step S6 it is checked whether the coefficient c has been updated a specified number of times, that is, whether the second termination condition is satisfied. As a result of the confirmation, if the second termination condition is not satisfied, that is, if the number of updates of the coefficient c has not reached the specified number of times (see No route in step S6), the process returns to step S1. By repeating this No route in step S6, a plurality of different candidate points are generated depending on the value of each coefficient c.
  • step S6 if the second termination condition is satisfied, that is, if the number of updates of the coefficient c reaches the specified number (see the Yes route in step S6), the process moves to step S7.
  • step S7 the adversarial sample generation unit 101 determines, as an adversarial sample, the candidate point whose perturbation size is closest to the target perturbation size ⁇ 0 from among the plurality of candidate points generated by the adversarial sample candidate update unit 105. .
  • the generated adversarial sample is attached with a label (correct label) of the original point used to generate the adversarial sample, and is used as training data for a classification model (machine learning model).
  • the adversarial sample candidate updating unit 105 updates (cs + Update the candidate points so that the value of d) is minimized. Then, the adversarial sample generation unit 101 determines, as an adversarial sample, the candidate point whose perturbation size is closest to the target perturbation size ⁇ 0 from among the plurality of candidate points generated by the adversarial sample candidate update unit 105.
  • the adversarial sample candidate updating unit 105 determines a candidate point whose perturbation size is closest to the target perturbation size ⁇ 0 as an adversarial sample. As a result, training data with a constant perturbation size ⁇ 0 can be obtained. Therefore, it is possible to easily generate a plurality of adversarial samples with a constant effective perturbation size in adversarial training for generating a classification model that is difficult to fool.
  • FIGS. 4 and 5 are diagrams illustrating the generation results of hostile samples by the hostile sample generation unit 101 of the information processing device 1 as an example of the embodiment in comparison with the conventional method.
  • the adversarial sample generation unit 101 of the information processing apparatus can generate many adversarial samples with a set specific perturbation size ⁇ .
  • FIG. 6 is a diagram illustrating the hardware configuration of the information processing device 1 as an example of the embodiment.
  • the information processing device 1 includes, for example, a processor 11, a memory 12, a storage device 13, a graphic processing device 14, an input interface 15, an optical drive device 16, a device connection interface 17, and a network interface 18 as components. These components 11 to 18 are configured to be able to communicate with each other via a bus 19.
  • a processor (control unit) 11 controls the entire information processing device 1 .
  • Processor 11 may be a multiprocessor.
  • the processor 11 includes, for example, a CPU, an MPU (Micro Processing Unit), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), an FPGA (Field Programmable Gate Array), and a GPU (Graphics Processing Unit). It may be any one of the following. Further, the processor 11 may be a combination of two or more types of elements among CPU, MPU, DSP, ASIC, PLD, FPGA, and GPU.
  • the information processing device 1 realizes the function as the training data generation unit 100 by executing a program (generation program, OS program) recorded on a computer-readable non-temporary recording medium, for example.
  • a program that describes the processing content to be executed by the information processing device 1 can be recorded on various recording media.
  • a program to be executed by the information processing device 1 can be stored in the storage device 13.
  • the processor 11 loads at least a portion of the program in the storage device 13 into the memory 12 and executes the loaded program.
  • the memory 12 is a storage memory including ROM (Read Only Memory) and RAM (Random Access Memory).
  • the RAM of the memory 12 is used as a main storage device of the information processing device 1. At least a part of the program to be executed by the processor 11 is temporarily stored in the RAM.
  • the memory 12 also stores various data necessary for processing by the processor 11.
  • the storage device 13 may store various data generated when the training data generation unit 100 described above executes each process.
  • a monitor 14a is connected to the graphic processing device 14.
  • the graphics processing device 14 displays images on the screen of the monitor 14a according to instructions from the processor 11.
  • Examples of the monitor 14a include a display device using a CRT (Cathode Ray Tube), a liquid crystal display device, and the like.
  • the optical drive device 16 uses laser light or the like to read data recorded on the optical disc 16a.
  • the optical disc 16a is a portable, non-temporary recording medium on which data is readably recorded by light reflection. Examples of the optical disc 16a include a DVD (Digital Versatile Disc), a DVD-RAM, a CD-ROM (Compact Disc Read Only Memory), and a CD-R (Recordable)/RW (ReWritable).
  • the device connection interface 17 is a communication interface for connecting peripheral devices to the information processing device 1.
  • a memory device 17a or a memory reader/writer 17b can be connected to the device connection interface 17.
  • the memory device 17a is a non-temporary recording medium equipped with a communication function with the device connection interface 17, such as a USB (Universal Serial Bus) memory.
  • the memory reader/writer 17b writes data to or reads data from the memory card 17c.
  • the memory card 17c is a card-type non-temporary recording medium.
  • the network interface 18 is connected to a network.
  • the network interface 18 sends and receives data via the network.
  • Other information processing devices, communication devices, etc. may be connected to the network.
  • the function of the candidate point generation unit 102 may be performed in another information processing device connected to the information processing device 1 via a network or the like.
  • Information on candidate points generated by another information processing device may be received via a network or the like, and used by the adversarial sample generation unit 101 to generate an adversarial sample.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The present invention can efficiently generate an adversarial sample with a specific perturbation size by comprising: a process of generating a plurality of pieces of candidate data on the basis of training data to be used for training a classification model; and a process of determining an adversarial sample from among the plurality of pieces of candidate data on the basis of the level of confidence that each of the plurality of pieces of candidate data will be classified into a class associated with the training data and a distance between each of the plurality of pieces of candidate data and the surface of a sphere having a radius constituted by a target perturbation size with the training data serving as the center thereof.

Description

生成方法，生成プログラムおよび情報処理装置Generation method, generation program, and information processing device
　本発明は、生成方法，生成プログラムおよび情報処理装置に関する。 The present invention relates to a generation method, a generation program, and an information processing device.
　近年、機械学習の利用が急速に広がっている。その一方で、機械学習を対象としたセキュリティの問題が指摘されている。 In recent years, the use of machine learning has been rapidly expanding. On the other hand, security issues related to machine learning have been pointed out.
　例えば、データをクラス分類する機械学習モデル（分類モデル）に誤分類を生じさせる敵対的サンプル攻撃が知られている。 For example, an adversarial sample attack is known that causes misclassification in a machine learning model (classification model) that classifies data.
　分類モデルの機械学習を行なうことは、特徴量空間において、異なるクラスに分類されるべきデータ点を分けられるような境界線（決定境界）を見つけることといえる。敵対的サンプル攻撃においては、決定境界をわずかに超えるようなデータ点を意図的に作成し、モデルの誤分類を誘導する。 Performing machine learning for a classification model can be said to find boundaries (decision boundaries) in the feature space that can separate data points to be classified into different classes. In adversarial sample attacks, data points that slightly exceed the decision boundary are intentionally created to induce misclassification of the model.
　なお、敵対的サンプル攻撃において作成された攻撃用のデータ点を敵対的サンプルといってもよい。 Note that the attack data points created in the adversarial sample attack may also be referred to as adversarial samples.
　図７は敵対的サンプルを例示する図である。 FIG. 7 is a diagram illustrating an adversarial sample.
　この図７において、符号Ａは数字の7を表す画像データであり、符号Ｂ，Ｃは符号Ａに示す画像データに基づいて生成した敵対的サンプルを示す。 In this FIG. 7, symbol A is image data representing the number 7, and symbols B and C represent adversarial samples generated based on the image data indicated by symbol A.
　符号Ｂに示す敵対的サンプルは、ＡＩ（Artificial Intelligence）が3と誤認識するように生成された敵対的サンプルであり、摂動サイズε=0.1である。符号Ｃに示す敵対的サンプルは、ＡＩが2と誤認識するように生成された敵対的サンプルであり、摂動サイズε=0.2である。 The adversarial sample indicated by code B is an adversarial sample generated so that AI (Artificial Intelligence) incorrectly recognizes it as 3, and the perturbation size ε=0.1. The adversarial sample shown by code C is an adversarial sample generated so that the AI misrecognizes it as 2, and the perturbation size ε=0.2.
　図８は敵対的サンプルを説明するための図である。 FIG. 8 is a diagram for explaining adversarial samples.
　敵対的サンプルは、訓練データ、すなわち、クラス分類済みのデータ点（以下、オリジナル点という）に基づいて生成される。オリジナル点と敵対的サンプルの間の距離である摂動サイズを符号εで表す。 Adversarial samples are generated based on training data, that is, classified data points (hereinafter referred to as original points). The perturbation size, which is the distance between the original point and the adversarial sample, is denoted by the symbol ε.
　また、敵対的サンプル攻撃に対する防衛手法として、分類モデルに対する敵対的訓練が知られている。敵対的訓練においては、敵対的サンプルを分類モデルの訓練データに加え、敵対的サンプルに対応した決定境界に更新する。 Additionally, adversarial training for classification models is known as a defense method against adversarial sample attacks. In adversarial training, adversarial samples are added to the training data of the classification model, and the decision boundaries corresponding to the adversarial samples are updated.
　ここで、敵対的訓練においては、分類モデルをより騙せる強い敵対的サンプルで分類モデルの訓練を行なうことで、分類モデルを騙されにくくすることができる。そのため、訓練に用いる敵対的サンプルが強力である場合に、騙されにくい分類モデルを生成できると期待される。 Here, in adversarial training, by training the classification model with strong adversarial samples that can more easily fool the classification model, it is possible to make the classification model less likely to be fooled. Therefore, it is expected that if the adversarial samples used for training are strong, it will be possible to generate a classification model that is difficult to fool.
　また、敵対的訓練において摂動サイズを少しずつ調整することで分類モデルを騙されにくくすることができる。例えば、敵対的訓練において、先に小さな摂動サイズの訓練データで訓練を行なった後に、大きな摂動サイズで訓練を行なうことで、大きな摂動でも間違えにくい分類モデルを生成することができる。そこで、訓練に用いる敵対的サンプルの摂動サイズを一定に揃えることで、騙されづらい分類モデルを効率的に生成することができる。 Also, by adjusting the perturbation size little by little during adversarial training, it is possible to make the classification model less susceptible to deception. For example, in adversarial training, by first training with training data with a small perturbation size and then training with a large perturbation size, it is possible to generate a classification model that is difficult to make mistakes even with large perturbations. Therefore, by making the perturbation sizes of adversarial samples used for training constant, it is possible to efficiently generate a classification model that is difficult to fool.
　従来においては、例えば、正則化ベースの手法を用いて強力な敵対的サンプルを生成することが知られている。 In the past, it has been known to generate strong adversarial samples using, for example, a regularization-based method.
　この手法においては、オリジナル点に基づいて敵対的サンプルの候補点を生成する。生成した候補点がターゲットクラス以外に分類される確信度を算出する。ターゲットクラスは、敵対的サンプルにより元のクラスから分類先を変更させようとしている対象のクラスである。また、候補点とオリジナル点との距離を算出する。そして、候補点の確信度（ａ）と、候補点とオリジナル点との距離（ｂ）との和（ａ＋ｂ）が最小化するように敵対的サンプル候補を更新し、最もオリジナル点に近い候補を、生成する敵対的サンプルとして決定する。 In this method, candidate points for adversarial samples are generated based on the original points. Calculate the confidence that the generated candidate point is classified into a class other than the target class. The target class is a class whose classification destination is to be changed from the original class using an adversarial sample. Also, the distance between the candidate point and the original point is calculated. Then, the adversarial sample candidates are updated so that the sum (a+b) of the confidence (a) of the candidate point and the distance (b) between the candidate point and the original point is minimized, and the candidate closest to the original point is selected. , is determined as the adversarial sample to generate.
特開２０１７－４９９９６号公報JP 2017-49996 Publication 特開２０２０－１１２９６７号公報JP2020-112967A 国際公開第２０２０／２３０６９９号International Publication No. 2020/230699 米国特許第１０２９６８１３号明細書US Patent No. 10296813 米国特許出願公開第２０２０／００６５６６４号明細書US Patent Application Publication No. 2020/0065664
　敵対的訓練を効率的に実施するために、一定の値に限定された摂動サイズをもつ強力な敵対的サンプルを訓練データとして用いたい。 In order to efficiently implement adversarial training, we would like to use strong adversarial samples with perturbation sizes limited to a certain value as training data.
　しかしながら、上述した従来の正則化ベースの手法を用いる敵対的サンプルの生成手法では、候補点の確信度（ａ）と候補点とオリジナル点との距離（ｂ）との和（ａ＋ｂ）が最小化するように敵対的サンプル候補を更新する。これにより、候補点とオリジナル点との距離（ｂ）の値も最小化され、摂動サイズが変化してしまう。そのため、摂動サイズが一定の敵対的サンプルを生成することが困難であり、一定の摂動サイズを用いる敵対的訓練に利用することが難しい。 However, in the adversarial sample generation method using the conventional regularization-based method described above, the sum (a + b) of the confidence of the candidate point (a) and the distance (b) between the candidate point and the original point is minimized. The adversarial sample candidates are updated as follows. As a result, the value of the distance (b) between the candidate point and the original point is also minimized, and the perturbation size changes. Therefore, it is difficult to generate adversarial samples with a constant perturbation size, and it is difficult to use them for adversarial training using a constant perturbation size.
　なお、上述した従来の正則化ベースの手法を用いる敵対的サンプルの生成手法で生成された敵対的サンプルのうち、特定の摂動サイズの敵対的サンプルのみを抽出することで、特定の摂動サイズの敵対的サンプルを取得することができる。しかしながら、この手法では、莫大な量のオリジナル点（オリジナルの訓練データ）が必要となり、非現実的である。 Note that by extracting only the adversarial samples with a specific perturbation size from among the adversarial samples generated by the adversarial sample generation method using the conventional regularization-based method described above, target samples can be obtained. However, this method requires a huge amount of original points (original training data) and is unrealistic.
　１つの側面では、本発明は、特定の摂動サイズを有する敵対的サンプルを効率良く生成できるようにすることを目的とする。 In one aspect, the present invention aims to enable efficient generation of adversarial samples having a specific perturbation size.
　このため、この生成方法は、クラス分類モデルの訓練に用いる敵対的サンプルを生成する生成方法であって、前記クラス分類モデルの訓練に用いられる訓練データに基づいて複数の候補データを生成する処理と、前記複数の候補データのそれぞれが、前記訓練データに対応付けられたクラスに分類される確信度と、前記複数の候補データのそれぞれから前記訓練データを中心とするターゲット摂動サイズを半径とする球表面までの距離とに基づき、前記複数の候補データの中から敵対的サンプルを決定する処理とを備える。 Therefore, this generation method is a generation method for generating adversarial samples used for training a class classification model, and includes a process for generating a plurality of candidate data based on training data used for training the class classification model. , the confidence that each of the plurality of candidate data is classified into the class associated with the training data, and a sphere whose radius is the target perturbation size centered on the training data from each of the plurality of candidate data. and determining a hostile sample from among the plurality of candidate data based on the distance to the surface.
　一実施形態によれば、特定の摂動サイズを有する敵対的サンプルを効率良く生成できる。 According to one embodiment, adversarial samples with a specific perturbation size can be efficiently generated.
実施形態の一例としての情報処理装置の構成を模式的に示す図である。1 is a diagram schematically showing the configuration of an information processing device as an example of an embodiment. 実施形態の一例としての情報処理装置の敵対的サンプル候補更新部による処理を説明するための図である。FIG. 2 is a diagram for explaining processing by a hostile sample candidate updating unit of an information processing device as an example of an embodiment. 実施形態の一例としての情報処理装置の敵対的サンプル生成部の処理を説明するためのフローチャートである。7 is a flowchart for explaining processing of a hostile sample generation unit of an information processing device as an example of an embodiment. 実施形態の一例としての情報処理装置の敵対的サンプル生成部による敵対的サンプルの生成結果を従来手法と比較して示す図である。FIG. 3 is a diagram illustrating the generation results of hostile samples by the hostile sample generation unit of the information processing device as an example of the embodiment in comparison with a conventional method. 実施形態の一例としての情報処理装置の敵対的サンプル生成部による敵対的サンプルの生成結果を従来手法と比較して示す図である。FIG. 3 is a diagram illustrating the generation results of hostile samples by the hostile sample generation unit of the information processing device as an example of the embodiment in comparison with a conventional method. 実施形態の一例としての情報処理装置のハードウェア構成を例示する図である。1 is a diagram illustrating a hardware configuration of an information processing device as an example of an embodiment. 敵対的サンプルを例示する図である。FIG. 3 is a diagram illustrating an adversarial sample. 敵対的サンプルを説明するための図である。FIG. 2 is a diagram for explaining an adversarial sample.
　以下、図面を参照して本生成方法，生成プログラムおよび情報処理装置にかかる実施の形態を説明する。ただし、以下に示す実施形態はあくまでも例示に過ぎず、実施形態で明示しない種々の変形例や技術の適用を排除する意図はない。すなわち、本実施形態を、その趣旨を逸脱しない範囲で種々変形して実施することができる。また、各図は、図中に示す構成要素のみを備えるという趣旨ではなく、他の機能等を含むことができる。 Hereinafter, embodiments of the present generation method, generation program, and information processing device will be described with reference to the drawings. However, the embodiments shown below are merely illustrative, and there is no intention to exclude the application of various modifications and techniques not specified in the embodiments. That is, this embodiment can be modified and implemented in various ways without departing from the spirit thereof. Furthermore, each figure is not intended to include only the constituent elements shown in the figure, but may include other functions.
　（Ａ）構成
　図１は実施形態の一例としての情報処理装置１の構成を模式的に示す図である。 (A) Configuration FIG. 1 is a diagram schematically showing the configuration of an information processing device 1 as an example of an embodiment.
　本情報処理装置１は、敵対的訓練に用いられる訓練データを生成する訓練用データ生成部１００としての機能を備える。 The information processing device 1 has a function as a training data generation unit 100 that generates training data used in adversarial training.
　また、訓練用データ生成部１００は、敵対的サンプルを生成する敵対的サンプル生成部１０１としての機能を備える。 Additionally, the training data generation unit 100 has a function as an adversarial sample generation unit 101 that generates adversarial samples.
　敵対的サンプル生成部１０１は、図１に示すように、候補点生成部１０２，確信度算出部１０３，距離算出部１０４および敵対的サンプル候補更新部１０５としての機能を有する。 As shown in FIG. 1, the adversarial sample generation unit 101 has the functions of a candidate point generation unit 102, a confidence calculation unit 103, a distance calculation unit 104, and an adversarial sample candidate update unit 105.
　候補点生成部１０２は、クラス分類モデル（機械学習モデル）の訓練に用いられる訓練データに基づいて敵対的サンプルの候補となる候補データを生成する。訓練データはクラス分類済みである。このようなクラス分類済みの訓練データをオリジナル点といってもよい。また、敵対的サンプルの候補となる候補データを候補点といってもよい。さらに、敵対的サンプルの候補となる候補データを敵対的サンプル候補データもしくは敵対的サンプル候補点といってもよい。クラス分類モデルを単に分類モデルといってもよい。 The candidate point generation unit 102 generates candidate data that is a candidate for an adversarial sample based on training data used for training a class classification model (machine learning model). The training data has already been classified. Such classified training data may be referred to as original points. Further, candidate data that is a candidate for a hostile sample may be referred to as a candidate point. Furthermore, candidate data that is a candidate for a hostile sample may be referred to as hostile sample candidate data or hostile sample candidate points. A class classification model may simply be called a classification model.
　候補点生成部１０２は、例えば、オリジナル点をコピーすることで、候補点を生成してもよい。すなわち、特徴量空間において、オリジナル点と同一座標の点を候補点としてもよい。また、候補点生成部１０２は、特徴量空間において、オリジナル点を所定範囲内においてランダム移動させることで候補点を生成してもよい。すなわち、候補点生成部１０２は、オリジナル点をランダムに少しだけ移動させることで候補点を生成してもよい。 The candidate point generation unit 102 may generate candidate points, for example, by copying the original point. That is, in the feature space, a point having the same coordinates as the original point may be used as a candidate point. Further, the candidate point generation unit 102 may generate candidate points by randomly moving the original point within a predetermined range in the feature space. That is, the candidate point generation unit 102 may generate candidate points by randomly moving the original point a little.
　候補点生成部１０２が生成した候補点の情報は、メモリ１２や記憶装置１３（図６参照）等の所定の記憶領域に記憶される。 Information on candidate points generated by the candidate point generation unit 102 is stored in a predetermined storage area such as the memory 12 or the storage device 13 (see FIG. 6).
　確信度算出部１０３は、候補点がターゲットクラス以外に分類される確信度を算出する。確信度は、訓練データに対応付けられたクラスに分類される確信度に相当するものである。ターゲットクラスは、敵対的サンプルにより元のクラスから分類先を変更させようとしている対象のクラスである。例えば、図８において、丸で示すオリジナル点に対して、×で示すクラスがターゲットクラスに相当する。 The certainty calculation unit 103 calculates the certainty that the candidate point is classified into a class other than the target class. The confidence level corresponds to the confidence level of classification into the class associated with the training data. The target class is a class whose classification destination is to be changed from the original class using an adversarial sample. For example, in FIG. 8, the class indicated by an x corresponds to the target class for the original point indicated by a circle.
　確信度は分類の確からしさの度合いの予測値である。候補点がターゲットクラス以外に分類される確信度を、単に、候補点の確信度といってよい。また、候補点の確信度を符号sで表す場合がある。 Confidence is a predicted value of the degree of certainty of classification. The confidence that a candidate point is classified into a class other than the target class can simply be called the confidence of the candidate point. Further, the confidence level of a candidate point may be expressed by the symbol s.
　例えば、分類モデルの出力する各クラスの確信度ziに対してターゲットクラスをtとすると、候補点がターゲットクラス以外に分類される確信度sは以下の式で算出することができる。
　s = max(zi:i≠t)-zt
　なお、候補点の確信度は、例えば、ソフトマックス関数の値やニューラルネットワークモデルの出力層のニューロンの出力値であるロジットの値に基づいて求めてもよい。 For example, if the target class is t for the confidence level z i of each class output by the classification model, the confidence level s that a candidate point is classified into a class other than the target class can be calculated using the following formula.
s = max(z i :i≠t)-z t
Note that the reliability of the candidate point may be determined based on, for example, the value of a softmax function or the value of logit, which is the output value of a neuron in the output layer of a neural network model.
　確信度算出部１０３は、算出した候補点の確信度sをメモリ１２や記憶装置１３等の所定の記憶領域に記憶させる。 The confidence calculation unit 103 stores the calculated confidence s of the candidate point in a predetermined storage area such as the memory 12 or the storage device 13.
　距離算出部１０４は、候補点と、オリジナル点を中心とし摂動サイズε0を半径とする球表面との距離（最短距離）を算出する。摂動サイズε0は、敵対的サンプル生成部１０１が生成する敵対的サンプルの摂動サイズであり、ユーザ等が任意の値を設定してよい。摂動サイズε0をターゲット摂動サイズε0といってもよい。また、オリジナル点を中心とし半径がターゲット摂動サイズε0の球表面をターゲット球表面といってもよい。 The distance calculation unit 104 calculates the distance (shortest distance) between the candidate point and the spherical surface whose center is the original point and whose radius is the perturbation size ε 0 . The perturbation size ε 0 is the perturbation size of the adversarial sample generated by the adversarial sample generation unit 101, and may be set to an arbitrary value by the user or the like. The perturbation size ε 0 may be referred to as the target perturbation size ε 0 . Furthermore, a spherical surface whose center is the original point and whose radius is the target perturbation size ε 0 may be referred to as the target spherical surface.
　距離算出部１０４は、例えば、候補点とターゲット球表面との距離を算出する。候補点とターゲット球表面との距離は、例えば、ユークリッド距離であってもよい。候補点とターゲット球表面との距離を符号dで表す場合がある。 The distance calculation unit 104 calculates, for example, the distance between the candidate point and the target spherical surface. The distance between the candidate point and the target sphere surface may be, for example, a Euclidean distance. The distance between the candidate point and the target sphere surface may be expressed by the symbol d.
　距離算出部１０４は、算出した候補点とターゲット球表面との距離dをメモリ１２や記憶装置１３等の所定の記憶領域に記憶させる。 The distance calculation unit 104 stores the calculated distance d between the candidate point and the target sphere surface in a predetermined storage area such as the memory 12 or the storage device 13.
　敵対的サンプル候補更新部１０５は、確信度算出部１０３が算出した候補点の確信度sと、距離算出部１０４が算出した候補点とターゲット球表面との距離dとに基づいて、候補点（敵対的サンプル候補点）を更新することで複数の候補点を生成する。敵対的サンプル候補更新部１０５が候補点を更新することを候補点を移動させるといってもよい。敵対的サンプル候補更新部１０５は、訓練データ（オリジナル点）に基づいて複数の候補データ（候補点）を生成する。 The adversarial sample candidate updating unit 105 updates the candidate point ( multiple candidate points are generated by updating the adversarial sample candidate points). The updating of a candidate point by the adversarial sample candidate updating unit 105 may also be referred to as moving the candidate point. The adversarial sample candidate updating unit 105 generates a plurality of candidate data (candidate points) based on training data (original points).
　図２は実施形態の一例としての情報処理装置１の敵対的サンプル候補更新部１０５による処理を説明するための図である。 FIG. 2 is a diagram for explaining processing by the hostile sample candidate updating unit 105 of the information processing device 1 as an example of the embodiment.
　敵対的サンプル候補更新部１０５は、（cs + d）の値が最小化するように候補点を更新する。なお、cは、候補点の確信度sと候補点とターゲット球表面との距離dとのどちらをより優先するかを表す係数である。 The adversarial sample candidate updating unit 105 updates the candidate points so that the value of (cs + d) is minimized. Note that c is a coefficient indicating which of the confidence level s of the candidate point and the distance d between the candidate point and the target sphere surface is given higher priority.
　図２に示す例においては、候補点Ｐ１の確信度s1の勾配ベクトルを符号S1で表す。また、候補点Ｐ１とターゲット球表面との距離d1の勾配ベクトルを符号D1で表す。候補点Ｐ１は、候補点生成部１０２がオリジナル点Ｐ０をコピーすることで生成されたものである。図２においては、オリジナル点Ｐ０と候補点Ｐ１とは、特徴量空間において同一座標である。 In the example shown in FIG. 2, the gradient vector of the confidence level s 1 of the candidate point P1 is represented by the symbol S 1 . Further, the gradient vector of the distance d 1 between the candidate point P 1 and the surface of the target sphere is represented by the symbol D 1 . The candidate point P1 is generated by the candidate point generation unit 102 copying the original point P0. In FIG. 2, the original point P0 and the candidate point P1 have the same coordinates in the feature amount space.
　また、図２に示す例においては、候補点Ｐ１の確信度s1の勾配ベクトルを符号S1で表す。また、候補点Ｐ１とターゲット球表面との距離d1の勾配ベクトルを符号D1で表す。 Furthermore, in the example shown in FIG. 2, the gradient vector with the confidence level s 1 of the candidate point P1 is represented by the symbol S 1 . Further, the gradient vector of the distance d 1 between the candidate point P 1 and the surface of the target sphere is represented by the symbol D 1 .
　敵対的サンプル候補更新部１０５は、候補点Ｐ１を（cs1 + d1）の値が最小化するように更新することで候補点Ｐ２を生成する。すなわち、敵対的サンプル候補更新部１０５は、候補点Ｐ１を更新して候補点Ｐ２を生成する。 The adversarial sample candidate updating unit 105 generates a candidate point P2 by updating the candidate point P1 so that the value of (cs 1 + d 1 ) is minimized. That is, the hostile sample candidate updating unit 105 updates the candidate point P1 to generate a candidate point P2.
　また、図２に示す例においては、候補点Ｐ２の確信度s2に対応するベクトルを符号S2で表す。また、候補点Ｐ２とターゲット球表面との距離d2に対応するベクトルを符号D2で表す。 In the example shown in FIG. 2, the vector corresponding to the confidence level s2 of the candidate point P2 is represented by the symbol S2 . Further, a vector corresponding to the distance d 2 between the candidate point P2 and the surface of the target sphere is represented by the symbol D 2 .
　敵対的サンプル候補更新部１０５は、候補点Ｐ２を（cs2 + d2）の値が最小化するように更新することで候補点Ｐ３を生成する。すなわち、敵対的サンプル候補更新部１０５は、候補点Ｐ２を更新して候補点Ｐ３を生成する。 The adversarial sample candidate updating unit 105 generates a candidate point P3 by updating the candidate point P2 so that the value of (cs 2 + d 2 ) is minimized. That is, the hostile sample candidate updating unit 105 updates the candidate point P2 to generate a candidate point P3.
　また、図２に示す例においては、候補点Ｐ３の確信度s3の勾配ベクトルを符号S3で表す。また、候補点Ｐ３とターゲット球表面との距離d3の勾配ベクトルを符号D3で表す。 Furthermore, in the example shown in FIG. 2, the gradient vector with the confidence level s3 of the candidate point P3 is represented by the symbol S3 . Further, the gradient vector of the distance d 3 between the candidate point P3 and the surface of the target sphere is represented by the symbol D 3 .
　敵対的サンプル候補更新部１０５は、候補点Ｐ３を（cs3 + d3）の値が最小化するように更新することで候補点Ｐ４を生成する。すなわち、敵対的サンプル候補更新部１０５は、候補点Ｐ３を更新して候補点Ｐ４を生成する。 The hostile sample candidate updating unit 105 generates a candidate point P4 by updating the candidate point P3 so that the value of (cs 3 + d 3 ) is minimized. That is, the hostile sample candidate updating unit 105 updates the candidate point P3 to generate a candidate point P4.
　なお、候補点の確信度s0～s3を特に区別しない場合には、確信度sと表記する。また、候補点とターゲット球表面との距離d0～d3を特に区別しない場合には、距離dと表記する。 Note that when the confidence levels s 0 to s 3 of candidate points are not particularly distinguished, they are expressed as confidence levels s. Furthermore, when the distances d 0 to d 3 between the candidate point and the target sphere surface are not particularly distinguished, they are expressed as distances d.
　敵対的サンプル候補更新部１０５は、先に生成した候補点を更新する（移動させる）ことで新たな候補点を生成する。敵対的サンプル候補更新部１０５は、更新により生成した各候補点の情報を、メモリ１２や記憶装置１３等の所定の記憶領域に記憶させる。 The hostile sample candidate updating unit 105 generates new candidate points by updating (moving) the previously generated candidate points. The adversarial sample candidate update unit 105 stores information about each candidate point generated by the update in a predetermined storage area such as the memory 12 or the storage device 13.
　敵対的サンプル生成部１０１は、確信度算出部１０３による候補点の確信度sの算出と、距離算出部１０４による候補点とターゲット球表面との距離dの算出と、敵対的サンプル候補更新部１０５による候補点の更新とを、予め規定した回数（第１終了条件）に到達するまで繰り返して実行させる。 The adversarial sample generation unit 101 calculates the confidence s of the candidate point by the certainty calculation unit 103, the distance d between the candidate point and the target sphere surface by the distance calculation unit 104, and the adversarial sample candidate update unit 105. The updating of candidate points is repeatedly executed until a predetermined number of times (first termination condition) is reached.
　また、敵対的サンプル生成部１０１は、係数cの値を、例えば二分探索等の手法を用いて更新しながら最適化する。敵対的サンプル生成部１０１は、係数cの更新を、予め規定した回数（第２終了条件）に到達するまで繰り返して実行させる。これにより、敵対的サンプル生成部１０１は、オリジナル点に基づいて複数の候補点（敵対的サンプル候補点）を生成する。 Additionally, the adversarial sample generation unit 101 optimizes the value of the coefficient c while updating it using a method such as binary search. The adversarial sample generation unit 101 repeatedly updates the coefficient c until a predetermined number of times (second termination condition) is reached. Thereby, the adversarial sample generation unit 101 generates a plurality of candidate points (adversarial sample candidate points) based on the original point.
　敵対的サンプル生成部１０１は、敵対的サンプル候補更新部１０５が生成した複数の候補点の中から、摂動サイズがターゲット摂動サイズε0に最も近い候補点を敵対的サンプルとして決定する。 The adversarial sample generation unit 101 determines, as an adversarial sample, the candidate point whose perturbation size is closest to the target perturbation size ε 0 from among the plurality of candidate points generated by the adversarial sample candidate update unit 105 .
　すなわち、敵対的サンプル生成部１０１は、複数の候補データのそれぞれの確信度sと、複数の候補データのそれぞれからオリジナル点（訓練データ）を中心とするターゲット摂動サイズを半径ε0とする球表面までの距離dとに基づき、複数の候補点（候補データ）の中から敵対的サンプルを決定する。 That is, the adversarial sample generation unit 101 generates a spherical surface whose radius is ε 0 and the target perturbation size centered on the original point (training data) from each of the plurality of candidate data and the confidence s of each of the plurality of candidate data. An adversarial sample is determined from among a plurality of candidate points (candidate data) based on the distance d to.
　訓練用データ生成部１００は、生成した敵対的サンプルに対して、当該敵対的サンプルの生成に用いたオリジナル点のラベル（正解ラベル）を付して分類モデル（機械学習モデル）の訓練データとする。 The training data generation unit 100 attaches a label (correct label) of the original point used to generate the adversarial sample to the generated adversarial sample and uses it as training data for a classification model (machine learning model). .
　（Ｂ）動作
　上述の如く構成された実施形態の一例としての情報処理装置１の敵対的サンプル生成部１０１の処理を、図３に示すフローチャート（ステップＳ１～Ｓ７）に従って説明する。 (B) Operation The processing of the hostile sample generation unit 101 of the information processing device 1 as an example of the embodiment configured as described above will be described according to the flowchart (steps S1 to S7) shown in FIG.
　本処理に先立って、候補点生成部１０２が、オリジナル点に基づいて候補点を生成する。また、候補点生成部１０２は、係数cの初期化を行なう。係数cの初期化においては、係数cに任意の値を設定してもよい。 Prior to this process, the candidate point generation unit 102 generates candidate points based on the original points. The candidate point generation unit 102 also initializes the coefficient c. In initializing the coefficient c, an arbitrary value may be set for the coefficient c.
　ステップＳ１において、候補点生成部１０２が、係数cを例えば、二分探索により更新する。 In step S1, the candidate point generation unit 102 updates the coefficient c by, for example, binary search.
　ステップＳ２において、確信度算出部１０３が、処理対象の候補点がターゲットクラス以外に分類される確信度sを算出する。処理対象の候補点は、複数の候補データ（候補点）のうちから抽出された候補データに相当し、候補点生成部１０２がオリジナル点に基づいて生成した候補点と敵対的サンプル生成部１０１が、オリジナル点に基づいて生成した複数の候補点（敵対的サンプル候補点）とを含む複数の候補点（候補点データ）から抽出されたものであってよい。 In step S2, the certainty calculation unit 103 calculates the certainty s that the candidate point to be processed is classified into a class other than the target class. The candidate point to be processed corresponds to candidate data extracted from a plurality of candidate data (candidate points), and the candidate point generated by the candidate point generation unit 102 based on the original point and the adversarial sample generation unit 101 are combined. , a plurality of candidate points (adversarial sample candidate points) generated based on the original point, and a plurality of candidate points (candidate point data).
　ステップＳ３において、距離算出部１０４が、候補点とターゲット球表面（オリジナル点を中心とし摂動サイズε0を半径とする球表面）との距離dを算出する。 In step S3, the distance calculation unit 104 calculates the distance d between the candidate point and the target spherical surface (a spherical surface whose center is the original point and whose radius is the perturbation size ε 0 ).
　ステップＳ４において、敵対的サンプル候補更新部１０５が、確信度算出部１０３が算出した候補点の確信度sと、距離算出部１０４が算出した候補点とターゲット球表面との距離dとに基づいて、敵対的サンプルを更新する。敵対的サンプル候補更新部１０５は、（cs + d）の値が最小化するように候補点を更新する。 In step S4, the adversarial sample candidate updating unit 105 updates the candidate point based on the confidence s of the candidate point calculated by the certainty calculation unit 103 and the distance d between the candidate point and the target sphere surface calculated by the distance calculation unit 104. , update the adversarial sample. The adversarial sample candidate updating unit 105 updates the candidate points so that the value of (cs + d) is minimized.
　ステップＳ５において、候補点生成部１０２は、ステップＳ２～Ｓ４の処理を規定回数実行したか、すなわち、第１終了条件を満たすかを確認する。確認の結果、第１終了条件が満たされていない場合、すなわち、ステップＳ２～Ｓ４の処理回数が規定回数に到達していない場合には（ステップＳ５のＮｏルート参照）、ステップＳ２に戻る。 In step S5, the candidate point generation unit 102 checks whether the processes of steps S2 to S4 have been executed a specified number of times, that is, whether the first termination condition is satisfied. As a result of the confirmation, if the first termination condition is not satisfied, that is, if the number of processing steps S2 to S4 has not reached the specified number of times (see No route in step S5), the process returns to step S2.
　一方、第１終了条件が満たされている場合、すなわち、ステップＳ２～Ｓ４の処理回数が規定回数に到達した場合には（ステップＳ５のＹｅｓルート参照）、ステップＳ６に移行する。 On the other hand, if the first termination condition is satisfied, that is, if the number of times steps S2 to S4 are processed reaches the specified number (see the Yes route in step S5), the process moves to step S6.
　ステップＳ６において、係数cの更新を規定回数実行したか、すなわち、第２終了条件を満たすかを確認する。確認の結果、第２終了条件が満たされていない場合、すなわち、係数cの更新回数が規定回数に到達していない場合には（ステップＳ６のＮｏルート参照）、ステップＳ１に戻る。このステップＳ６のＮｏルートによる繰り返しが行なわれることで、各係数cの値により異なる複数の候補点が生成されることになる。 In step S6, it is checked whether the coefficient c has been updated a specified number of times, that is, whether the second termination condition is satisfied. As a result of the confirmation, if the second termination condition is not satisfied, that is, if the number of updates of the coefficient c has not reached the specified number of times (see No route in step S6), the process returns to step S1. By repeating this No route in step S6, a plurality of different candidate points are generated depending on the value of each coefficient c.
　一方、第２終了条件が満たされている場合、すなわち、係数cの更新回数が規定回数に到達した場合には（ステップＳ６のＹｅｓルート参照）、ステップＳ７に移行する。 On the other hand, if the second termination condition is satisfied, that is, if the number of updates of the coefficient c reaches the specified number (see the Yes route in step S6), the process moves to step S7.
　ステップＳ７において、敵対的サンプル生成部１０１は、敵対的サンプル候補更新部１０５が生成した複数の候補点の中から、摂動サイズがターゲット摂動サイズε0に最も近い候補点を敵対的サンプルとして決定する。 In step S7, the adversarial sample generation unit 101 determines, as an adversarial sample, the candidate point whose perturbation size is closest to the target perturbation size ε 0 from among the plurality of candidate points generated by the adversarial sample candidate update unit 105. .
　生成した敵対的サンプルには、当該敵対的サンプルの生成に用いたオリジナル点のラベル（正解ラベル）が付されて分類モデル（機械学習モデル）の訓練データとして用いられる。 The generated adversarial sample is attached with a label (correct label) of the original point used to generate the adversarial sample, and is used as training data for a classification model (machine learning model).
　（Ｃ）効果
　実施形態の一例としての情報処理装置１によれば、敵対的サンプル候補更新部１０５が、候補点の確信度sと候補点とターゲット球表面との距離dとに基づき、（cs + d）の値が最小化するように候補点を更新する。そして、敵対的サンプル生成部１０１が、敵対的サンプル候補更新部１０５が生成した複数の候補点の中から、摂動サイズがターゲット摂動サイズε0に最も近い候補点を敵対的サンプルとして決定する。 (C) Effects According to the information processing device 1 as an example of the embodiment, the adversarial sample candidate updating unit 105 updates (cs + Update the candidate points so that the value of d) is minimized. Then, the adversarial sample generation unit 101 determines, as an adversarial sample, the candidate point whose perturbation size is closest to the target perturbation size ε 0 from among the plurality of candidate points generated by the adversarial sample candidate update unit 105.
　これにより、決定境界をギリギリのところで越え、且つ、オリジナル点（元データ）からのズレが最適な敵対的サンプルを生成することができる。すなわち、騙されにくい分類モデルを生成するための敵対的訓練において有効な、強力な敵対的サンプルを容易に生成することができる。 As a result, it is possible to generate an adversarial sample that barely crosses the decision boundary and has an optimal deviation from the original point (original data). That is, it is possible to easily generate strong adversarial samples that are effective in adversarial training to generate a classification model that is difficult to fool.
　また、敵対的サンプル候補更新部１０５が、摂動サイズがターゲット摂動サイズε0に最も近い候補点を敵対的サンプルとして決定する。これにより、一定の摂動サイズε0に揃った訓練データを得ることができる。従って、騙されにくい分類モデルを生成するための敵対的訓練において有効な摂動サイズが一定な複数の敵対的サンプルを容易に生成することができる。 Further, the adversarial sample candidate updating unit 105 determines a candidate point whose perturbation size is closest to the target perturbation size ε 0 as an adversarial sample. As a result, training data with a constant perturbation size ε 0 can be obtained. Therefore, it is possible to easily generate a plurality of adversarial samples with a constant effective perturbation size in adversarial training for generating a classification model that is difficult to fool.
　図４および図５は、それぞれ実施形態の一例としての情報処理装置１の敵対的サンプル生成部１０１による敵対的サンプルの生成結果を従来手法と比較して示す図である。 FIGS. 4 and 5 are diagrams illustrating the generation results of hostile samples by the hostile sample generation unit 101 of the information processing device 1 as an example of the embodiment in comparison with the conventional method.
　これらの図４および図５においては、正則化ベースの手法である(C&W) [N. Carlini and D. Wagner, 2016]を用いて、敵対的サンプルを生成した例を示す。 4 and 5 show examples in which adversarial samples were generated using the regularization-based method (C&W) [N. Carlini and D. Wagner, 2016].
　図４および図５においては、画像分類 ＡＩ（Artificial Intelligence）に対して、従来手法による C&W により生成した敵対的サンプルと、本情報処理装置の敵対的サンプル生成部１０１によるC&W により生成した敵対的サンプルとを摂動サイズεでヒストグラムとして表示している。 4 and 5, for image classification AI (Artificial Intelligence), adversarial samples generated by C&W using a conventional method, and adversarial samples generated by C&W by the adversarial sample generation unit 101 of this information processing device. is displayed as a histogram with perturbation size ε.
　図４はε=1に固定した場合に生成された敵対的サンプルの分布を示し、図５はε=2に固定した場合に生成された敵対的サンプルの分布を示す。 FIG. 4 shows the distribution of adversarial samples generated when ε=1 is fixed, and FIG. 5 shows the distribution of adversarial samples generated when ε=2 is fixed.
　これらの図４および図５に示すように、本情報処理装置の敵対的サンプル生成部１０１は、設定した特定の摂動サイズεの敵対的サンプルを多く発生させることができる。 As shown in FIGS. 4 and 5, the adversarial sample generation unit 101 of the information processing apparatus can generate many adversarial samples with a set specific perturbation size ε.
　敵対的訓練において、先に小さな摂動サイズ（例えば、ε=0.1）の訓練データで訓練を行なった後に、大きな摂動サイズ（例えば、ε=0.2）で訓練を行なうことで、大きな摂動でも間違えにくい分類モデルを生成することができる。 In adversarial training, by first training with training data with a small perturbation size (e.g. ε=0.1) and then training with a large perturbation size (e.g. ε=0.2), classification is difficult even with large perturbations. A model can be generated.
　（Ｄ）その他
　図６は実施形態の一例としての情報処理装置１のハードウェア構成を例示する図である。 (D) Others FIG. 6 is a diagram illustrating the hardware configuration of the information processing device 1 as an example of the embodiment.
　情報処理装置１は、例えば、プロセッサ１１，メモリ１２，記憶装置１３，グラフィック処理装置１４，入力インタフェース１５，光学ドライブ装置１６，機器接続インタフェース１７およびネットワークインタフェース１８を構成要素として有する。これらの構成要素１１～１８は、バス１９を介して相互に通信可能に構成される。 The information processing device 1 includes, for example, a processor 11, a memory 12, a storage device 13, a graphic processing device 14, an input interface 15, an optical drive device 16, a device connection interface 17, and a network interface 18 as components. These components 11 to 18 are configured to be able to communicate with each other via a bus 19.
　プロセッサ（制御部）１１は、情報処理装置１全体を制御する。プロセッサ１１は、マルチプロセッサであってもよい。プロセッサ１１は、例えばＣＰＵ，ＭＰＵ（Micro Processing Unit），ＤＳＰ（Digital Signal Processor），ＡＳＩＣ（Application Specific Integrated Circuit），ＰＬＤ（Programmable Logic Device），ＦＰＧＡ（Field Programmable Gate Array），ＧＰＵ（Graphics Processing Unit）のいずれか一つであってもよい。また、プロセッサ１１は、ＣＰＵ，ＭＰＵ，ＤＳＰ，ＡＳＩＣ，ＰＬＤ，ＦＰＧＡ，ＧＰＵのうちの２種類以上の要素の組み合わせであってもよい。 A processor (control unit) 11 controls the entire information processing device 1 . Processor 11 may be a multiprocessor. The processor 11 includes, for example, a CPU, an MPU (Micro Processing Unit), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), an FPGA (Field Programmable Gate Array), and a GPU (Graphics Processing Unit). It may be any one of the following. Further, the processor 11 may be a combination of two or more types of elements among CPU, MPU, DSP, ASIC, PLD, FPGA, and GPU.
　そして、プロセッサ１１が情報処理装置１用の制御プログラム（生成プログラム：図示省略）を実行することにより、図１に例示する、訓練用データ生成部１００としての機能が実現される。 Then, when the processor 11 executes a control program (generation program: not shown) for the information processing device 1, the function as the training data generation section 100 illustrated in FIG. 1 is realized.
　なお、情報処理装置１は、例えばコンピュータ読み取り可能な非一時的な記録媒体に記録されたプログラム（生成プログラム，ＯＳプログラム）を実行することにより、訓練用データ生成部１００としての機能を実現する。 Note that the information processing device 1 realizes the function as the training data generation unit 100 by executing a program (generation program, OS program) recorded on a computer-readable non-temporary recording medium, for example.
　情報処理装置１に実行させる処理内容を記述したプログラムは、様々な記録媒体に記録しておくことができる。例えば、情報処理装置１に実行させるプログラムを記憶装置１３に格納しておくことができる。プロセッサ１１は、記憶装置１３内のプログラムの少なくとも一部をメモリ１２にロードし、ロードしたプログラムを実行する。 A program that describes the processing content to be executed by the information processing device 1 can be recorded on various recording media. For example, a program to be executed by the information processing device 1 can be stored in the storage device 13. The processor 11 loads at least a portion of the program in the storage device 13 into the memory 12 and executes the loaded program.
　また、情報処理装置１（プロセッサ１１）に実行させるプログラムを、光ディスク１６ａ，メモリ装置１７ａ，メモリカード１７ｃ等の非一時的な可搬型記録媒体に記録しておくこともできる。可搬型記録媒体に格納されたプログラムは、例えばプロセッサ１１からの制御により、記憶装置１３にインストールされた後、実行可能になる。また、プロセッサ１１が、可搬型記録媒体から直接プログラムを読み出して実行することもできる。 Furthermore, the program to be executed by the information processing device 1 (processor 11) may be recorded on a non-temporary portable recording medium such as the optical disk 16a, the memory device 17a, or the memory card 17c. The program stored in the portable recording medium becomes executable after being installed in the storage device 13 under the control of the processor 11, for example. Furthermore, the processor 11 can also directly read and execute a program from a portable recording medium.
　メモリ１２は、ＲＯＭ（Read Only Memory）およびＲＡＭ（Random Access Memory）を含む記憶メモリである。メモリ１２のＲＡＭは情報処理装置１の主記憶装置として使用される。ＲＡＭには、プロセッサ１１に実行させるプログラムの少なくとも一部が一時的に格納される。また、メモリ１２には、プロセッサ１１による処理に必要な各種データが格納される。 The memory 12 is a storage memory including ROM (Read Only Memory) and RAM (Random Access Memory). The RAM of the memory 12 is used as a main storage device of the information processing device 1. At least a part of the program to be executed by the processor 11 is temporarily stored in the RAM. The memory 12 also stores various data necessary for processing by the processor 11.
　記憶装置１３は、ハードディスクドライブ（Hard Disk Drive：ＨＤＤ）、ＳＳＤ（Solid State Drive）、ストレージクラスメモリ（Storage Class Memory：ＳＣＭ）等の記憶装置であって、種々のデータを格納するものである。記憶装置１３は、情報処理装置１の補助記憶装置として使用される。記憶装置１３には、ＯＳプログラム，制御プログラムおよび各種データが格納される。制御プログラムには生成プログラムが含まれる。 The storage device 13 is a storage device such as a hard disk drive (HDD), SSD (Solid State Drive), or storage class memory (SCM), and stores various data. The storage device 13 is used as an auxiliary storage device of the information processing device 1. The storage device 13 stores an OS program, a control program, and various data. The control program includes a generation program.
　なお、補助記憶装置としては、ＳＣＭやフラッシュメモリ等の半導体記憶装置を使用することもできる。また、複数の記憶装置１３を用いてＲＡＩＤ（Redundant Arrays of Inexpensive Disks）を構成してもよい。 Note that a semiconductor storage device such as an SCM or a flash memory can also be used as the auxiliary storage device. Further, a plurality of storage devices 13 may be used to configure RAID (Redundant Arrays of Inexpensive Disks).
　また、記憶装置１３には、上述した訓練用データ生成部１００が各処理を実行する際に生成される各種データを格納してもよい。 Furthermore, the storage device 13 may store various data generated when the training data generation unit 100 described above executes each process.
　グラフィック処理装置１４には、モニタ１４ａが接続されている。グラフィック処理装置１４は、プロセッサ１１からの命令に従って、画像をモニタ１４ａの画面に表示させる。モニタ１４ａとしては、ＣＲＴ（Cathode Ray Tube）を用いた表示装置や液晶表示装置等が挙げられる。 A monitor 14a is connected to the graphic processing device 14. The graphics processing device 14 displays images on the screen of the monitor 14a according to instructions from the processor 11. Examples of the monitor 14a include a display device using a CRT (Cathode Ray Tube), a liquid crystal display device, and the like.
　入力インタフェース１５には、キーボード１５ａおよびマウス１５ｂが接続されている。入力インタフェース１５は、キーボード１５ａやマウス１５ｂから送られてくる信号をプロセッサ１１に送信する。なお、マウス１５ｂは、ポインティングデバイスの一例であり、他のポインティングデバイスを使用することもできる。他のポインティングデバイスとしては、タッチパネル，タブレット，タッチパッド，トラックボール等が挙げられる。 A keyboard 15a and a mouse 15b are connected to the input interface 15. The input interface 15 transmits signals sent from the keyboard 15a and mouse 15b to the processor 11. Note that the mouse 15b is an example of a pointing device, and other pointing devices can also be used. Other pointing devices include touch panels, tablets, touch pads, trackballs, and the like.
　光学ドライブ装置１６は、レーザ光等を利用して、光ディスク１６ａに記録されたデータの読み取りを行なう。光ディスク１６ａは、光の反射によって読み取り可能にデータを記録された可搬型の非一時的な記録媒体である。光ディスク１６ａには、ＤＶＤ（Digital Versatile Disc），ＤＶＤ－ＲＡＭ，ＣＤ－ＲＯＭ（Compact Disc Read Only Memory），ＣＤ－Ｒ（Recordable）／ＲＷ（ReWritable）等が挙げられる。 The optical drive device 16 uses laser light or the like to read data recorded on the optical disc 16a. The optical disc 16a is a portable, non-temporary recording medium on which data is readably recorded by light reflection. Examples of the optical disc 16a include a DVD (Digital Versatile Disc), a DVD-RAM, a CD-ROM (Compact Disc Read Only Memory), and a CD-R (Recordable)/RW (ReWritable).
　機器接続インタフェース１７は、情報処理装置１に周辺機器を接続するための通信インタフェースである。例えば、機器接続インタフェース１７には、メモリ装置１７ａやメモリリーダライタ１７ｂを接続することができる。メモリ装置１７ａは、機器接続インタフェース１７との通信機能を搭載した非一時的な記録媒体、例えばＵＳＢ（Universal Serial Bus）メモリである。メモリリーダライタ１７ｂは、メモリカード１７ｃへのデータの書き込み、またはメモリカード１７ｃからのデータの読み出しを行なう。メモリカード１７ｃは、カード型の非一時的な記録媒体である。 The device connection interface 17 is a communication interface for connecting peripheral devices to the information processing device 1. For example, a memory device 17a or a memory reader/writer 17b can be connected to the device connection interface 17. The memory device 17a is a non-temporary recording medium equipped with a communication function with the device connection interface 17, such as a USB (Universal Serial Bus) memory. The memory reader/writer 17b writes data to or reads data from the memory card 17c. The memory card 17c is a card-type non-temporary recording medium.
　ネットワークインタフェース１８は、ネットワークに接続される。ネットワークインタフェース１８は、ネットワークを介してデータの送受信を行なう。ネットワークには他の情報処理装置や通信機器等が接続されてもよい。 The network interface 18 is connected to a network. The network interface 18 sends and receives data via the network. Other information processing devices, communication devices, etc. may be connected to the network.
　そして、開示の技術は上述した実施形態に限定されるものではなく、本実施形態の趣旨を逸脱しない範囲で種々変形して実施することができる。本実施形態の各構成および各処理は、必要に応じて取捨選択することができ、あるいは適宜組み合わせてもよい。 The disclosed technology is not limited to the embodiments described above, and can be implemented with various modifications without departing from the spirit of the present embodiments. Each configuration and each process of this embodiment can be selected or selected as necessary, or may be combined as appropriate.
　例えば、上述した情報処理装置１においては、訓練用データ生成部１００（敵対的サンプル生成部１０１）が候補点生成部１０２としての機能を有しているが、これに限定されるものではない。 For example, in the information processing device 1 described above, the training data generation unit 100 (adversarial sample generation unit 101) has a function as the candidate point generation unit 102, but the present invention is not limited to this.
　例えば、候補点生成部１０２としての機能を、本情報処理装置１とネットワーク等を介して接続された他の情報処理装置において実行してもよい。この他の情報処理装置によって生成された候補点の情報をネットワーク等を介して受信し、敵対的サンプル生成部１０１が敵対的サンプルの生成に用いてもよい。 For example, the function of the candidate point generation unit 102 may be performed in another information processing device connected to the information processing device 1 via a network or the like. Information on candidate points generated by another information processing device may be received via a network or the like, and used by the adversarial sample generation unit 101 to generate an adversarial sample.
　上述した実施形態においては、例えば、図３のフローチャートに示したように、１つの候補点から１つの敵対的サンプルを生成する例を示しているが、これに限定されるものではない。敵対的サンプル生成部１０１は、敵対的サンプル候補更新部１０５が生成した複数の候補点の中から、摂動サイズがターゲット摂動サイズε0に対して所定範囲内にある複数の候補点を敵対的サンプルとして決定してもよい。 In the embodiment described above, for example, as shown in the flowchart of FIG. 3, an example is shown in which one adversarial sample is generated from one candidate point, but the present invention is not limited to this. The adversarial sample generation unit 101 selects a plurality of candidate points whose perturbation size is within a predetermined range with respect to the target perturbation size ε 0 from among the plurality of candidate points generated by the adversarial sample candidate update unit 105 as adversarial samples. It may be determined as
　すなわち、敵対的サンプル候補更新部１０５は、複数の候補点（候補データ）のうち、摂動サイズとターゲット摂動サイズε0との差分が基準よりも小さい候補データを敵対的サンプルとして決定してよい。 That is, the adversarial sample candidate updating unit 105 may determine, as an adversarial sample, candidate data for which the difference between the perturbation size and the target perturbation size ε 0 is smaller than the reference among the plurality of candidate points (candidate data).
　また、上述した開示により本実施形態を当業者によって実施・製造することが可能である。 Furthermore, the present embodiment can be implemented and manufactured by those skilled in the art based on the above-mentioned disclosure.
　１　　情報処理装置
　１１　　プロセッサ（制御部）
　１２　　メモリ
　１３　　記憶装置
　１４　　グラフィック処理装置
　１４ａ　　モニタ
　１５　　入力インタフェース
　１５ａ　　キーボード
　１５ｂ　　マウス
　１６　　光学ドライブ装置
　１６ａ　　光ディスク
　１７　　機器接続インタフェース
　１７ａ　　メモリ装置
　１７ｂ　　メモリリーダライタ
　１７ｃ　　メモリカード
　１８　　ネットワークインタフェース
　１９　　バス
　１００　　訓練用データ生成部
　１０１　　敵対的サンプル生成部
　１０２　　候補点生成部
　１０３　　確信度算出部
　１０４　　距離算出部
　１０５　　敵対的サンプル更新部 1 Information processing device 11 Processor (control unit)
12 Memory 13 Storage device 14 Graphic processing device 14a Monitor 15 Input interface 15a Keyboard 15b Mouse 16 Optical drive device 16a Optical disk 17 Device connection interface 17a Memory device 17b Memory reader/writer 17c Memory card 18 Network interface 19 Bus 100 Training data generation section 10 1 Adversarial sample generation unit 102 Candidate point generation unit 103 Confidence calculation unit 104 Distance calculation unit 105 Adversarial sample update unit

Claims (9)

  1. 　クラス分類モデルの訓練に用いる敵対的サンプルを生成する生成方法であって、
    　前記クラス分類モデルの訓練に用いられる訓練データに基づいて複数の候補データを生成する処理と、
    　前記複数の候補データのそれぞれが、前記訓練データに対応付けられたクラスに分類される確信度と、前記複数の候補データのそれぞれから前記訓練データを中心とするターゲット摂動サイズを半径とする球表面までの距離とに基づき、前記複数の候補データの中から敵対的サンプルを決定する処理と
    をコンピュータが実行することを特徴とする生成方法。     A generation method for generating adversarial samples used for training a class classification model, the method comprising:
    A process of generating a plurality of candidate data based on training data used for training the class classification model;
    The confidence that each of the plurality of candidate data is classified into a class associated with the training data, and a spherical surface whose radius is the target perturbation size centered on the training data from each of the plurality of candidate data. A generation method characterized in that a computer executes a process of determining a hostile sample from among the plurality of candidate data based on the distance to.
  2. 　前記複数の候補データを生成する処理は、
    　前記複数の候補データのうちから抽出した候補データに基づいて、当該抽出した候補データの確信度と、当該抽出した候補データ前記球表面との距離とに基づく値が最小化するように更新することで、新たな候補データを生成する処理を含む
    ことを特徴とする請求項１に記載の生成方法。     The process of generating the plurality of candidate data includes:
    Based on the candidate data extracted from among the plurality of candidate data, updating is performed so that a value based on the confidence of the extracted candidate data and the distance between the extracted candidate data and the spherical surface is minimized. 2. The generation method according to claim 1, further comprising a process of generating new candidate data.
  3. 　前記複数の候補データの中から敵対的サンプルを決定する処理は、
    　前記複数の候補データのうち、摂動サイズと前記ターゲット摂動サイズとの差分が基準よりも小さい候補データを前記敵対的サンプルとして決定する処理を含む
    ことを特徴とする請求項１または２に記載の生成方法。     The process of determining a hostile sample from among the plurality of candidate data includes:
    3. Generation according to claim 1, further comprising a process of determining candidate data, among the plurality of candidate data, for which a difference between a perturbation size and the target perturbation size is smaller than a reference, as the adversarial sample. Method.
  4. 　クラス分類モデルの訓練に用いられる訓練データに基づいて複数の候補データを生成し、
    　前記複数の候補データのそれぞれが、前記訓練データに対応付けられたクラスに分類される確信度と、前記複数の候補データのそれぞれから前記訓練データを中心とするターゲット摂動サイズを半径とする球表面までの距離とに基づき、前記複数の候補データの中から前記クラス分類モデルの訓練に用いる敵対的サンプルを決定する
    処理をコンピュータに実行させることを特徴とする生成プログラム。     Generate multiple candidate data based on the training data used for training the class classification model,
    The confidence that each of the plurality of candidate data is classified into a class associated with the training data, and a spherical surface whose radius is the target perturbation size centered on the training data from each of the plurality of candidate data. A generation program that causes a computer to execute a process of determining an adversarial sample to be used for training the classification model from among the plurality of candidate data based on the distance to the object.
  5. 　前記複数の候補データを生成する処理は、
    　前記複数の候補データのうちから抽出した候補データに基づいて、当該抽出した候補データの確信度と、当該抽出した候補データ前記球表面との距離とに基づく値が最小化するように更新することで、新たな候補データを生成する処理を含む
    ことを特徴とする請求項４に記載の生成プログラム。     The process of generating the plurality of candidate data includes:
    Based on the candidate data extracted from among the plurality of candidate data, updating is performed so that a value based on the confidence of the extracted candidate data and the distance between the extracted candidate data and the spherical surface is minimized. 5. The generation program according to claim 4, further comprising a process of generating new candidate data.
  6. 　前記複数の候補データの中から敵対的サンプルを決定する処理は、
    　前記複数の候補データのうち、摂動サイズと前記ターゲット摂動サイズとの差分が基準よりも小さい候補データを前記敵対的サンプルとして決定する処理を含む
    ことを特徴とする請求項４または５に記載の生成プログラム。     The process of determining a hostile sample from among the plurality of candidate data includes:
    6. Generation according to claim 4, further comprising a process of determining candidate data, among the plurality of candidate data, for which a difference between a perturbation size and the target perturbation size is smaller than a reference, as the adversarial sample. program.
  7. 　クラス分類モデルの訓練に用いられる訓練データに基づいて複数の候補データを生成し、
    　前記複数の候補データのそれぞれが、前記訓練データに対応付けられたクラスに分類される確信度と、前記複数の候補データのそれぞれから前記訓練データを中心とするターゲット摂動サイズを半径とする球表面までの距離とに基づき、前記複数の候補データの中から前記クラス分類モデルの訓練に用いる敵対的サンプルを決定する
    処理部を備えることを特徴とする情報処理装置。     Generate multiple candidate data based on the training data used for training the class classification model,
    The confidence that each of the plurality of candidate data is classified into a class associated with the training data, and a spherical surface whose radius is the target perturbation size centered on the training data from each of the plurality of candidate data. An information processing apparatus comprising: a processing unit that determines an adversarial sample to be used for training the classification model from among the plurality of candidate data based on the distance to the object.
  8. 　前記複数の候補データを生成する処理は、
    　前記複数の候補データのうちから抽出した候補データに基づいて、当該抽出した候補データの確信度と、当該抽出した候補データ前記球表面との距離とに基づく値が最小化するように更新することで、新たな候補データを生成する処理を含む
    ことを特徴とする請求項７に記載の情報処理装置。     The process of generating the plurality of candidate data includes:
    Based on the candidate data extracted from among the plurality of candidate data, updating is performed so that a value based on the confidence of the extracted candidate data and the distance between the extracted candidate data and the spherical surface is minimized. 8. The information processing apparatus according to claim 7, further comprising a process of generating new candidate data.
  9. 　前記複数の候補データの中から敵対的サンプルを決定する処理は、
    　前記複数の候補データのうち、摂動サイズと前記ターゲット摂動サイズとの差分が基準よりも小さい候補データを前記敵対的サンプルとして決定する処理を含む
    ことを特徴とする請求項７または８に記載の情報処理装置。     The process of determining a hostile sample from among the plurality of candidate data includes:
    The information according to claim 7 or 8, further comprising a process of determining, as the adversarial sample, candidate data in which a difference between a perturbation size and the target perturbation size is smaller than a reference among the plurality of candidate data. Processing equipment.
PCT/JP2022/016443 2022-03-31 2022-03-31 Generation method, generation program, and information processing device WO2023188241A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/JP2022/016443 WO2023188241A1 (en) 2022-03-31 2022-03-31 Generation method, generation program, and information processing device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2022/016443 WO2023188241A1 (en) 2022-03-31 2022-03-31 Generation method, generation program, and information processing device

Publications (1)

Publication Number Publication Date
WO2023188241A1 true WO2023188241A1 (en) 2023-10-05

Family

ID=88199866

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/016443 WO2023188241A1 (en) 2022-03-31 2022-03-31 Generation method, generation program, and information processing device

Country Status (1)

Country Link
WO (1) WO2023188241A1 (en)

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
JORN-HENRIK JACOBSEN; JENS BEHRMANNN; NICHOLAS CARLINI; FLORIAN TRAMER; NICOLAS PAPERNOT: "Exploiting Excessive Invariance caused by Norm-Bounded Adversarial Robustness", ARXIV.ORG, 25 March 2019 (2019-03-25), XP081157787 *
LIU FANGCHENG, ZHANG CHAO, ZHANG HONGYANG: "Towards Transferable Adversarial Perturbations with Minimum Norm", ICML 2021 WORKSHOP AML POSTER, 21 June 2021 (2021-06-21), XP093095372 *

Similar Documents

Publication Publication Date Title
RU2373575C2 (en) System and method for recognition of objects handwritten in ink
JP4757116B2 (en) Parameter learning method and apparatus, pattern identification method and apparatus, and program
US20210201181A1 (en) Inferencing and learning based on sensorimotor input data
CN114730398A (en) Data tag validation
JP7465048B2 (en) Formal safe symbolic reinforcement learning for visual input
CN114330588A (en) Picture classification method, picture classification model training method and related device
US11651276B2 (en) Artificial intelligence transparency
WO2023188241A1 (en) Generation method, generation program, and information processing device
WO2022185432A1 (en) Image recognition learning system, image recognition learning method, image recognition learning program, image recognition machine learning unit, and image recognition system
JP2023541450A (en) Apparatus and method for classifying images and accessing robustness of classification
JP6880494B2 (en) Classification device manufacturing method, image classification method, image classification device, semiconductor inspection device and classification standard module
WO2021005898A1 (en) Object detection device, object detection method, and program
JPH1131226A (en) Method and device for processing information
CN112070093A (en) Method for generating image classification model, image classification method, device and equipment
WO2023127062A1 (en) Data generation method, machine learning method, information processing device, data generation program, and machine learning program
JP7487469B2 (en) Image generating device, image generating method, and program
WO2023188354A1 (en) Model training method, model training program, and information processing device
JP2023183079A (en) Training data generation program, training data generation method, and information processing apparatus
WO2020026395A1 (en) Model creation device, model creation method, and recording medium in which model creation program is recorded
JP2021184148A (en) Optimization device, optimization method, and optimization program
JP2023087266A (en) Machine learning program, machine learning method, and machine learning device
JP2023144562A (en) Machine learning program, data processing program, information processing device, machine learning method and data processing method
WO2022254626A1 (en) Machine learning program, machine learning method, and machine learning device
WO2024047758A1 (en) Training data distribution estimation program, device, and method
JP2024010795A (en) Training data generation program, training data generation method, and training data generation apparatus

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22935386

Country of ref document: EP

Kind code of ref document: A1