WO2024079802A1

WO2024079802A1 - Evaluation device, evaluation method, and evaluation program

Info

Publication number: WO2024079802A1
Application number: PCT/JP2022/037938
Authority: WO
Inventors: 俊樹芝原; 尭之三浦; 真昇紀伊; 敦謙市川
Original assignee: 日本電信電話株式会社
Priority date: 2022-10-11
Filing date: 2022-10-11
Publication date: 2024-04-18

Abstract

This evaluation device calculates a privacy risk of each piece of data included in a data set used for training a machine-learning model. In addition, the evaluation device calculates a gain that a user has obtained by providing the data to the data set. For example, the evaluation device calculates how much the accuracy of the machine-learning model is improved by using the data for training. In addition, the evaluation device calculates, as an inequality degree, the difference between the gain estimated from the privacy risk of the data and a user gain obtained by providing the data, and evaluates the inequality degree of the machine-learning model by using the calculated inequality of each of the users.

Description

EVALUATION APPARATUS, EVALUATION METHOD, AND EVALUATION PROGRAM

The present invention relates to an evaluation device, an evaluation method, and an evaluation program for evaluating fairness regarding privacy risks.

Machine learning technologies such as Deep Neural Networks (DNNs) have been pointed out as posing privacy risks due to their tendency to memorize training data. Specifically, it has been shown that it is possible to infer from the output of a trained model whether or not specific data was included in the training data. Therefore, consideration must be given to privacy risks when handling data that users do not want others to know, such as medical data or web browsing history.

In addition, when it comes to data used as training data in machine learning, it is necessary to consider the balance between the user's privacy risk and the benefit the user receives from providing data (e.g., how useful the provided data is in improving the accuracy of the trained model). For example, if a user's privacy risk is balanced against the benefit the user receives from providing data, it can be considered fair.

　In the field of machine learning, there are conventional techniques for evaluating inter-personal fairness in classification problems (see Non-Patent Documents 1 and 2), but there are no techniques for evaluating fairness with respect to privacy risks as described above. Therefore, the objective of the present invention is to evaluate fairness with respect to privacy risks.

In order to solve the above-mentioned problems, the present invention is characterized by comprising a privacy risk calculation unit that calculates the privacy risk of each piece of data included in a dataset used to train a machine learning model, a gain calculation unit that calculates the gain that a user who provides data to the dataset will gain from providing the data, an unfairness calculation unit that calculates the difference between the privacy risk of the data and the gain that the user will gain from providing the data as the unfairness of the user, and evaluates the unfairness of the machine learning model based on the calculated unfairness of each of the users, and an output processing unit that outputs the evaluation result of the unfairness of the machine learning model.

The present invention makes it possible to evaluate fairness regarding privacy risks.

FIG. 1 is a diagram for explaining an overview of the evaluation device. FIG. 2 is a diagram illustrating an example of the configuration of the evaluation device. FIG. 3 is a flowchart illustrating an example of a processing procedure executed by the evaluation device. FIG. 4 is a flowchart for explaining an application example of the evaluation device. FIG. 5 is a diagram illustrating a computer that executes an evaluation program.

Below, a form (embodiment) for carrying out the present invention will be described with reference to the drawings. The present invention is not limited to this embodiment.

[overview]
The evaluation device of this embodiment evaluates whether users who provide data to a dataset used to build a machine learning model have obtained benefits that are commensurate with the privacy risk (fairness).

For example, as shown in FIG. 1, the evaluation device extracts data from a dataset used to build a machine learning model and calculates the privacy risk of the data. The evaluation device also calculates the gain that a user can obtain by providing the data to the dataset.

Then, the evaluation device calculates the unfairness of each user based on the difference between the gain expected from the user's privacy risk and the gain that the user actually obtains by providing data. Here, the evaluation device calculates a higher unfairness the greater the difference between the gain expected from the user's privacy risk and the gain that the user actually obtains by providing data. In this way, the evaluation device calculates whether each user who provided data to the dataset has obtained a gain commensurate with the privacy risk (unfairness). The evaluation device then evaluates the unfairness of the machine learning model based on the calculated unfairness of each user.

[Configuration example]
Next, a configuration example of the evaluation device 10 will be described with reference to Fig. 2. The evaluation device 10 includes, for example, an input/output unit 11, a storage unit 12, and a control unit 13.

The input/output unit 11 is an interface that handles the input and output of various data. For example, the input/output unit 11 accepts input of a dataset used to build a machine learning model. The input dataset is stored in the storage unit 12.

The storage unit 12 stores data, programs, etc. that are referenced when the control unit 13 executes various processes. The storage unit 12 is realized by a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. For example, the storage unit 12 stores a data set, etc., received by the input/output unit 11. Also, for example, the storage unit 12 may store information indicating which user provided each data of the data set.

The control unit 13 is responsible for controlling the entire evaluation device 10. The functions of the control unit 13 are realized, for example, by a CPU (Central Processing Unit) executing a program stored in the storage unit 12. The control unit 13 includes, for example, a privacy risk calculation unit 131, a gain calculation unit 132, an unfairness calculation unit 133, and an output processing unit 134.

The privacy risk calculation unit 131 calculates the privacy risk of each piece of data included in the dataset. For example, the privacy risk calculation unit 131 calculates the privacy risk by calculating the lower bound (LB) of the differential privacy parameter ε based on the following formula (1).

The upper bound of the false positive rate (FPR _UB ) and the upper bound of the false negative rate (FNR _UB ) in formula (1) are calculated using the false positive rate (FPR) and the false negative rate (FNR) when a game of guessing whether the data to be evaluated was used for model training is repeated many times (for example, about 1000 times) (see formula (2)). The upper bounds of the false positive rate (FPR) and the false negative rate (FNR) can be calculated using the Clopper-Pearson method.

The privacy risk calculation unit 131 may also calculate the privacy risk based on, for example, the success rate when membership estimation is performed multiple times.

The gain calculation unit 132 calculates the gain that a user will gain by providing data to the dataset. The gain that a user will gain by providing data to the dataset is, for example, the degree to which the accuracy of a machine learning model improves when the data is used to train the machine learning model.

For example, the gain calculation unit 132 calculates the degree to which the accuracy of the machine learning model improves when the data provided by the user is used for learning as follows. First, the gain calculation unit 132 constructs n shadow models (first shadow models) that use the data provided by the user for learning, and n shadow models (second shadow models) that do not use the data for learning.

Next, the gain calculation unit 132 determines the number of first shadow models that output correct answer data among the n first shadow models as c _in . Then, the value obtained by dividing c _in by n (c _in /n) is the accuracy of the first shadow models. The gain calculation unit 132 also determines the number of second shadow models that output correct answer data among the n second shadow models as c _out . Then, the value obtained by dividing c _out by n (c _out /n) is the accuracy of the second shadow models.

Then, the gain calculation unit 132 calculates the difference between the accuracy (c _in /n) of the first shadow model and the accuracy (c _out /n) of the second shadow model as the gain (g) of the user's data (see equation (3)).

In addition to the above method, the gain calculation unit 132 may calculate the gain based on the extent to which other data held by the user improves the accuracy of the shadow model. The user's gain may also be a service or monetary reward provided in exchange for the data provided by the user.

The unfairness calculation unit 133 calculates the unfairness of each user who provided data to the dataset. Then, the unfairness calculation unit 133 evaluates the unfairness of the machine learning model based on the unfairness of each user calculated. For example, the unfairness calculation unit 133 calculates the difference between the user's gain expected from the privacy risk of the data calculated by the privacy risk calculation unit 131 and the user's gain obtained by providing the data calculated by the gain calculation unit 132 as the unfairness of the user. Then, the unfairness calculation unit 133 determines the maximum value of the unfairness of each user calculated as the unfairness of the machine learning model.

For example, let all users be U = {u _i } _{i =} ⁿ _1. Let the risk of all users be R = {r _i } _{i =} ⁿ ₁ , and the payoff be G = {g _i } _{i =} ⁿ _1. Let the risk normalized to mean = 0 and variance = 1 be R', and the payoff be G'. Then, the unfairness calculation unit 133 calculates the difference between the normalized risk r' and the normalized payoff g' for each user, and the maximum value is the unfairness (δ^) of the machine learning model (see formula (4)).

In addition, if the unfairness of each user includes an outlier, the unfairness calculation unit 133 may determine the maximum value among the unfairnesses excluding the outlier as the unfairness of the machine learning model.

The output processing unit 134 outputs the processing result by the control unit 13. For example, the output processing unit 134 outputs the evaluation result of the unfairness of the machine learning model by the unfairness calculation unit 133.

Using such an evaluation device 10, it is possible to evaluate the unfairness of a machine learning model.

[Example of processing procedure]
Next, an example of a processing procedure executed by the evaluation device 10 will be described with reference to Fig. 3. First, the privacy risk calculation unit 131 of the evaluation device 10 calculates a privacy risk of each piece of data included in the data set (S1).

Next, the gain calculation unit 132 calculates the gain obtained by the user by providing data to the dataset (S2). For example, the gain calculation unit 132 calculates the extent to which the accuracy of the shadow model will improve if the data provided by the user is used for learning.

Next, the unfairness calculation unit 133 evaluates the unfairness of the machine learning model (S3). For example, the unfairness calculation unit 133 calculates the difference between the gain expected from the privacy risk of the data calculated in S1 and the actual gain of the user obtained by providing the data calculated in S3 as the unfairness of the user. The unfairness calculation unit 133 then determines the maximum value of the unfairness of each user calculated as the unfairness of the machine learning model. The output processing unit 134 then outputs the evaluation result of the unfairness of the machine learning model obtained in S3 (S4).

By performing the above process, the evaluation device 10 can evaluate the unfairness of the machine learning model.

[Application example]
Next, an application example of the evaluation device 10 will be described with reference to Fig. 4. For example, the administrator of the evaluation device 10 designs a neural network (NN) for calculating privacy risks (S11). For example, training of a NN that satisfies Differential Privacy is performed by DP-SGD (Differentially Private Stochastic Gradient Descent).

Next, the administrator selects the dataset to be used in evaluating the unfairness of users and the users to be evaluated for unfairness (S12). For example, the administrator selects about 100 users, taking into consideration the diversity of users.

Then, the evaluation device 10 uses the dataset selected in S12 to calculate the unfairness of each user selected in S12, and evaluates the unfairness of the machine learning model (NN) based on the unfairness of each user calculated (S13). For example, the evaluation device 10 calculates the privacy risk of each data of the dataset for the NN designed in S11. The evaluation device 10 also calculates the gain of the user selected in S12 (the user who provided data to the dataset). Then, the evaluation device 10 calculates the unfairness of each user based on the calculated privacy risk of each data and the gain of each user who provided the data. Then, the evaluation device 10 determines the maximum value of the unfairness of each user calculated as the unfairness of the machine learning model (NN).

[System configuration, etc.]
In addition, each component of each part shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or a part of it can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc. Furthermore, each processing function performed by each device can be realized in whole or in any part by a CPU and a program executed by the CPU, or can be realized as hardware using wired logic.

Furthermore, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using known methods. In addition, the information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified.

[program]
The evaluation device 10 can be implemented by installing a program (evaluation program) as package software or online software on a desired computer. For example, the program can be executed by an information processing device, causing the information processing device to function as the evaluation device 10. The information processing device referred to here includes mobile communication terminals such as smartphones, mobile phones, and PHS (Personal Handyphone Systems), as well as terminals such as PDAs (Personal Digital Assistants).

FIG. 5 is a diagram showing an example of a computer that executes an evaluation program. The computer 1000 has, for example, a memory 1010 and a CPU 1020. The computer 1000 also has a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. Each of these components is connected by a bus 1080.

The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM (Random Access Memory) 1012. The ROM 1011 stores a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to a display 1130, for example.

The hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, the programs that define each process executed by the evaluation device 10 described above are implemented as program modules 1093 in which computer-executable code is written. The program modules 1093 are stored, for example, in the hard disk drive 1090. For example, a program module 1093 for executing processes similar to the functional configuration of the evaluation device 10 is stored in the hard disk drive 1090. The hard disk drive 1090 may be replaced by an SSD (Solid State Drive).

The data used in the processing of the above-described embodiment is stored as program data 1094, for example, in memory 1010 or hard disk drive 1090. Then, the CPU 1020 reads the program module 1093 or program data 1094 stored in memory 1010 or hard disk drive 1090 into RAM 1012 as necessary and executes it.

The program module 1093 and program data 1094 are not limited to being stored in the hard disk drive 1090, but may be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (such as a LAN (Local Area Network), WAN (Wide Area Network)). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070.

REFERENCE SIGNS LIST 10 Evaluation device 11 Input/output unit 12 Storage unit 13 Control unit 131 Privacy risk calculation unit 132 Gain calculation unit 133 Unfairness calculation unit 134 Output processing unit

Claims

A privacy risk calculation unit that calculates the privacy risk of each piece of data included in a dataset used to train a machine learning model;
a profit calculation unit that calculates a profit that a user who provides data to the data set will obtain by providing the data;
an unfairness calculation unit that calculates a difference between the privacy risk of the data and the user's gain obtained by providing the data as an unfairness of the user, and evaluates the unfairness of the machine learning model based on the calculated unfairness of each of the users;
and an output processing unit that outputs an evaluation result of the unfairness of the machine learning model.
The unfairness calculation unit,
The evaluation device according to claim 1 , further comprising: normalizing the privacy risk of the data and the payoff of the user, and determining a difference between the normalized privacy risk of the data and the payoff of the user as the unfairness of the user.
The unfairness calculation unit,
The evaluation device according to claim 1 , further comprising: a maximum value of the unfairness degrees calculated for each of the users as an evaluation result of the machine learning model.
The benefits of providing the data are as follows:
The evaluation device according to claim 1 , wherein the evaluation is a degree to which the accuracy of a machine learning model is improved when the data is used for training the machine learning model.
The gain calculation unit
The evaluation device according to claim 4, further comprising: constructing a shadow model using the data for training and a shadow model not using the data for training; and calculating a difference in accuracy between the shadow model using the data for training and the shadow model not using the data for training, thereby calculating a degree to which accuracy of the machine learning model is improved when the data is used for training.
The benefits of providing the data include:
The evaluation device according to claim 1 , wherein the evaluation is a service or a monetary reward provided to the user in exchange for providing the data.
An evaluation method performed by an evaluation device, comprising:
Calculating the privacy risk of each piece of data included in the dataset used to train the machine learning model;
A step of calculating a profit that a user who has contributed data to the data set will gain from providing the data;
A step of calculating a difference between the privacy risk of the data and the user's gain obtained by providing the data as an unfairness of the user, and evaluating the unfairness of the machine learning model based on the calculated unfairness of each of the users;
and outputting an evaluation result of the unfairness of the machine learning model.
Calculating the privacy risk of each piece of data included in the dataset used to train the machine learning model;
A step of calculating a profit that a user who has contributed data to the data set will gain from providing the data;
A step of calculating a difference between the privacy risk of the data and the user's gain obtained by providing the data as an unfairness of the user, and evaluating the unfairness of the machine learning model based on the calculated unfairness of each of the users;
and outputting an evaluation result of the machine learning model.