WO2024047735A1 - 公平性評価プログラム、公平性評価方法、及び、情報処理装置 - Google Patents

公平性評価プログラム、公平性評価方法、及び、情報処理装置 Download PDF

Info

Publication number
WO2024047735A1
WO2024047735A1 PCT/JP2022/032546 JP2022032546W WO2024047735A1 WO 2024047735 A1 WO2024047735 A1 WO 2024047735A1 JP 2022032546 W JP2022032546 W JP 2022032546W WO 2024047735 A1 WO2024047735 A1 WO 2024047735A1
Authority
WO
WIPO (PCT)
Prior art keywords
attribute
value
data
ratio
proportion
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/032546
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
貴三郎 福田
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fujitsu Ltd
Original Assignee
Fujitsu Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fujitsu Ltd filed Critical Fujitsu Ltd
Priority to EP22957338.1A priority Critical patent/EP4583014A4/en
Priority to JP2024543641A priority patent/JP7768409B2/ja
Priority to PCT/JP2022/032546 priority patent/WO2024047735A1/ja
Publication of WO2024047735A1 publication Critical patent/WO2024047735A1/ja
Priority to US19/049,211 priority patent/US20250190876A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/10—Machine learning using kernel methods, e.g. support vector machines [SVM]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/01—Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N7/00—Computing arrangements based on specific mathematical models
    • G06N7/01—Probabilistic graphical models, e.g. probabilistic networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N99/00—Subject matter not provided for in other groups of this subclass
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/10—Office automation; Time management

Definitions

  • the present invention relates to a fairness evaluation program, a fairness evaluation method, and an information processing device.
  • Machine learning models trained using biased data may output unfair inference results, such as inference results that cause discrimination.
  • Bias is, for example, bias in protected attributes such as gender.
  • Protected attributes include gender, race, skin color, nationality, and age.
  • fairness may be evaluated in subgroups included in the data.
  • Subgroups are defined by a combination of protected attributes and other attributes.
  • the number of subgroups increases as the number of attributes to be combined increases.
  • the computational load increases as the number of subgroups increases.
  • one of the objects of the present invention is to reduce the computational processing load for evaluating the fairness of data used for training in machine learning.
  • the fairness evaluation program may cause the computer to perform the following processing.
  • the process may include a process of acquiring a plurality of pieces of data.
  • the process may include a process of specifying, among the plurality of data, a first proportion of data in which a first attribute of a plurality of attributes of the plurality of data has a first value.
  • the processing includes determining a second ratio of data in which the first attribute is the first value among the first group in which the second attribute of the plurality of attributes is the second value;
  • the method may include a process of identifying a third proportion of data in which the first attribute is the first value among the second group in which the attribute is the third value.
  • the process includes the processing in which the second attribute is the second value and the third attribute of the plurality of attributes is the fourth value, and the first attribute is the first attribute.
  • performing a fairness evaluation with respect to the third group if a fourth proportion of data that is a value of satisfies criteria based on the first proportion, the second proportion, and the third proportion; May include processing.
  • the computational processing load for evaluating the fairness of data used for training in machine learning can be reduced.
  • FIG. 2 is a diagram for explaining the fairness of inference results of a machine learning model.
  • FIG. 3 is a diagram illustrating an example of data to be processed by a fairness evaluation device.
  • FIG. 3 is a diagram illustrating an example of criteria for determining whether or not to perform a fairness evaluation by a fairness evaluation device.
  • FIG. 7 is a diagram showing another example of data to be processed by the fairness evaluation device.
  • 5 is a diagram showing an example of a plurality of subgroups in the data shown in FIG. 4.
  • FIG. 1 is a block diagram showing an example of a functional configuration of a fairness evaluation device according to an embodiment.
  • FIG. 3 is a flowchart illustrating an example of the operation of the fairness evaluation device according to an embodiment.
  • 1 is a block diagram illustrating an example of a hardware (HW) configuration of a computer that implements the functions of a fairness evaluation device according to an embodiment.
  • HW hardware
  • FIG. 1 is a diagram for explaining the fairness of the inference results of a machine learning model.
  • the trained machine learning model may output unfair inference results, such as inference results that cause unfair discrimination. be.
  • the training data is an example of a plurality of data to be processed by the fairness evaluation device 1.
  • Discrimination includes discrimination with respect to protected attributes.
  • a protected attribute is sometimes called a sensitive attribute.
  • protected attributes include at least one attribute defined by the U.S. Fair Housing Act (FHA), such as race, skin color, national origin, religion, gender, household status, and the presence or absence of a disability.
  • FHA U.S. Fair Housing Act
  • Other examples of protected attributes include at least one attribute such as whether the user is married or unmarried as defined by the Credit Card Elimination of Discrimination Act (ECIA) in the United States, whether or not the user receives public subsidies, and age.
  • ECIA Credit Card Elimination of Discrimination Act
  • the fairness evaluation device 1 performs a fairness evaluation on whether the training data contains bias regarding a specific protected attribute such as gender. Execute.
  • the fairness evaluation device 1 may generate training data for machine learning from which unfair bias has been removed by processing a portion of the plurality of training data based on the fairness evaluation results. . Thereby, the original training data that includes bias can be modified, for example, expanded, to appropriate training data that does not include bias. Note that the fairness evaluation device 1 may execute training of a machine learning model using the training data generated in this manner.
  • FIG. 2 is a diagram showing an example of data 111a to be processed by the fairness evaluation device 1.
  • the fairness evaluation device 1 acquires data 111a and protection attribute information from a device (not shown) that provides the data.
  • the data 111a is an example of a plurality of data (data set) acquired by the fairness evaluation device 1.
  • the data 111a may be processed to generate training data used for training a machine learning model.
  • Data 111a includes multiple attributes. Each of the plurality of attributes may be a protected attribute or an unprotected attribute.
  • the protected attribute information is information indicating which attribute is a protected attribute among multiple attributes.
  • the protected attribute information may be determined in advance based on external knowledge such as laws.
  • An unprotected attribute means an attribute other than a protected attribute among a plurality of attributes.
  • the fairness evaluation device 1 determines the first attribute (first attribute) among the plurality of attributes included in the data 111a, and also determines the second attribute (second attribute) based on the protected attribute information.
  • the first attribute is, for example, a target variable.
  • the first attribute is an attribute that indicates the success or failure (pass or fail) of an examinee at a certain university.
  • the second attribute is, for example, any protected attribute. In FIG. 2, the second attribute is "gender". Note that attributes other than the objective variable, such as the second attribute, may be called explanatory variables.
  • the value indicating "pass” may be positive, that is, a positive example, and is an example of the first value.
  • the fairness evaluation device 1 may specify the ratio of the first value to the plurality of values of the first attribute included in the data 111a (first ratio, positive example ratio). Note that the ratio of the first value may be included in the data 111a. As shown by reference numeral A1 in FIG. 2, the first ratio is "pass rate": 30% (0.3).
  • the value shown is an example of the third value.
  • the data 111a may include a first group 31a whose second attribute is a second value, and a second group 31b whose second attribute is a third value.
  • the first group 31a is an example of a first group
  • the second group 31b is an example of a second group.
  • the fairness evaluation device 1 specifies the proportion (positive example proportion) of the first group 31a in which the first attribute has the first value ("pass" in FIG. 2) as the second proportion. Furthermore, the fairness evaluation device 1 specifies the proportion of the second group 31b in which the first attribute is the first value as the third proportion. In the example of FIG. 2, the second percentage is 36% (0.36) and the third percentage is 24% (0.24).
  • Reference numeral A3 in FIG. 2 indicates subgroups 32 and 33 obtained by grouping the first group 31a and the second group 31b by a third attribute (for example, "faculty").
  • the value of the third attribute for example, "Faculty”
  • the value of the third attribute for example, "Faculty”
  • the value of the third attribute is "Computer Science”.
  • the subgroup 32 may include a subgroup 32a whose second attribute value is "male” and a subgroup 32b whose second attribute value is “female.” Furthermore, the subgroup 33 may include a subgroup 33a whose second attribute value is "male” and a subgroup 33b whose second attribute value is "female.”
  • Each of the subgroups 32a and 33a is a third group in which the second attribute (gender) has a second value (for example, male) and the third attribute (for example, "faculty") has a fourth value.
  • the second value of the second attribute (“gender”) may be "female” and the third value may be "male.”
  • the second attribute (gender) has a second value (for example, female)
  • the third attribute (for example, "faculty" has a fourth value. This is an example of the third group.
  • the fairness evaluation device 1 may identify the proportion of the plurality of third groups in which the first attribute is the first value ("pass" in FIG. 2) as the fourth proportion.
  • the fourth percentage in subgroup 32a is 20% (0.2) and the fourth percentage in subgroup 33a is 40% (0.4).
  • the fourth percentage in subgroup 32b is 20% (0.2), and the fourth percentage in subgroup 33b is 40% (0.4).
  • the fairness evaluation device 1 is configured such that the fourth proportion of data whose first attribute is a positive example among the plurality of third groups is the first proportion, the second proportion, and the fourth proportion of data whose first attribute is a positive example. Based on whether or not the criterion based on the ratio of 3 is satisfied, it is determined whether or not to perform the fairness evaluation of the third group.
  • the fairness evaluation device 1 performs fairness evaluation on a subgroup for which a fourth proportion satisfies the criterion among the plurality of subgroups.
  • the fairness evaluation device 1 suppresses the fairness evaluation for a subgroup for which the fourth ratio does not meet the criteria among the plurality of subgroups, and omits the evaluation as an example (executes the evaluation). do not).
  • FIG. 3 is a diagram illustrating an example of criteria for determining whether or not to perform fairness evaluation by the fairness evaluation device 1.
  • the vertical axis in FIG. 3 corresponds to the positive example ratio.
  • parameters C 0 , C 1 , and C 2 may be calculated based on the following equations (1) to (3).
  • C 0 P 0 (1)
  • C 1 P 0 + (
  • C 2 P 0 -(
  • A a1)
  • A a2) shows.
  • Y 1
  • Y 1
  • the fairness evaluation device 1 calculates a numerical range C based on the above formula (4), and sets the numerical value range C to include the parameter C 0 of the above formula (1) (as an example, the median value).
  • the upper limit value C 1 and the lower limit value C 2 of the numerical range C may be calculated.
  • the fairness evaluation device 1 performs the following operations when the fourth ratio> C1 or the fourth ratio ⁇ C2 , in other words, when the fourth ratio is outside the numerical range C, It may be determined that the fourth ratio satisfies the criterion.
  • the fourth proportion (positive example proportion), which is the proportion of data whose first attribute is the first value, is "Fourth proportion>C 1 or the fourth ratio ⁇ C 2 ”, fairness evaluation is performed for the subgroups that meet the criteria.
  • the group whose fourth proportion (positive case proportion) is "C 1 ⁇ fourth proportion ⁇ C 2 ", fairness evaluation is performed. is suppressed.
  • the estimated distribution of the proportion of positive cases in the subgroup is a normal distribution
  • the proportion of positive cases in each subgroup will be densely distributed near the center of the normal distribution, while the proportion of positive cases in the subgroup containing unfair bias will be distributed far from the center of the normal distribution. distribution.
  • the subgroups do not contain any unfair bias, and therefore, it is permissible to omit execution of the fairness evaluation. Therefore, by suppressing execution of fairness evaluation for such subgroups, it is possible to reduce the calculation processing load for evaluating fairness for data.
  • the second ratio P 1 and the third ratio P 2 used to calculate the numerical range C are based on the value of the protected attribute of interest. This is an example of an index indicating the statistics of bias occurring between men and women (for example, between men and women).
  • the larger the difference between the second ratio P 1 (Y 1
  • A a2), the larger the numerical range C becomes.
  • the number of subgroups executed is reduced. In this way, the larger the difference between the second ratio P 1 and the third ratio P 2 is, the more the calculation processing load can be reduced.
  • the second attribute has a second value (in one example, gender is male), and the other attribute (third attribute, for example, the department attended) has a certain value (fourth value, For example, an example will be shown in which a group (Faculty of Pharmacy) is set as the third group.
  • the other attribute (third attribute) is not limited to a single attribute, and may be a combination of multiple attributes.
  • FIG. 4 is a diagram showing another example of the data 111b to be processed by the fairness evaluation device 1.
  • Data 111b is another example of a plurality of data (data sets) to be acquired.
  • the data 111b is an example of training data used for training a machine learning model.
  • the data 111b illustrated in FIG. 4 is an attribute that indicates, as a first attribute, whether or not an applicant is hired by a certain company (hired or not hired). Furthermore, the data 111b includes a gender attribute as a second attribute (protected attribute). Further, the data 111b may include the marital status (single, married) and the applied employment type (full-time, part-time) as attributes (explanatory variables).
  • FIG. 5 is a diagram showing an example of multiple subgroups in the data 111b shown in FIG. 4.
  • the data 41 (all data 111b) indicating the first attribute (recruitment success/failure) is the first group whose second attribute (for example, "gender") has the second value (male).
  • 41a and a second group 41b whose second attribute is a third value (female).
  • the “acceptance rate” is specified as the first proportion P 0 (positive example proportion) of the first value (for example, “acceptance”) of the first attribute (acceptance/failure).
  • the first proportion P 0 is 36% (0.36).
  • DI Dispose Impact
  • DI is a fairness evaluation index and is an example of a fairness index.
  • DI is the ratio of positive example proportions in the second group, which is the second value, and the third group, which is the third value, in the second attribute.
  • the fairness evaluation device 1 may perform the fairness evaluation using DI.
  • the DI in the second group (for example, men) may be calculated by the proportion of positive cases in the second group/the proportion of positive cases in the third group
  • the DI in the third group (for example, women) may be calculated by the proportion of positive cases in the second group/the proportion of positive cases in the third group.
  • the ratio of positive examples in the second group to the ratio of positive examples in the second group may be calculated.
  • the second proportion P 1 (adoption rate) of which the first attribute is the first value (adoption) among the first group 41a is 51% (0.51)
  • the third proportion P2 in which the first attribute has the first value is 22% (0.22).
  • reference numeral B2 in FIG. 5 indicates subgroups 42 to 45, which are obtained by dividing the first group 41a and the second group 41b by a third attribute (for example, "employment type” or "marital status").
  • the value of the third attribute for example, "employment type”
  • the value of the third attribute for example, "employment type”
  • the value of the third attribute is “part-time” (indicated as “part-time” in FIG. 5).
  • “full time” indicated as "regular” in FIG. 5.
  • the value of the third attribute for example, "marital status
  • the value of the third attribute is "single”
  • the value of the third attribute for example, "marital status
  • reference numeral B3 in FIG. 5 indicates subgroups 46 to 51, which are divided from subgroups 42 to 45 based on a "composite attribute" that is a combination of multiple attributes of "employment type” and “marital status.”
  • the value of the "complex attribute” is "part-time and single”
  • the value of the "complex attribute” is "part-time and married.”
  • the value of the “compound attribute” is “single and regular”
  • the value of the “compound attribute” is "married and regular.”
  • "and” is indicated by a multiplication symbol "x”.
  • the subgroups 42 to 51 may include a subgroup whose second attribute value is "male” and a subgroup whose second attribute value is "female.”
  • subgroups 42 to 51 subgroups whose second attribute value is the second value (for example, "male") are referred to as subgroups 42a to 51a
  • subgroups whose second attribute value is the third value for example, "male”
  • subgroups 42a to 51a subgroups that are "women”
  • Subgroups 42a to 51a are examples of the third group.
  • the fairness evaluation device 1 determines, for each subgroup, whether the positive example ratio (adoption rate) of the subgroup>C 1 (0.51) or the positive example ratio (adoption rate) ⁇ C 2 (0.22). Determine whether In the example of FIG. 5, the fairness evaluation device 1 evaluates the subgroups 42 to 46, 47a, 48, 49, 50a, and 51 whose positive example proportions are outside the range of C 1 to C 2 (numerical range C) to Identify the target for evaluation. On the other hand, the fairness evaluation device 1 determines to suppress execution of the fairness evaluation for the subgroups 47b and 50b whose positive example ratio falls within the range of C 1 to C 2 (numerical range C).
  • the difference in DI between the first group 41a and the second group 41b is large.
  • the DI values are similar between men and women.
  • the difference in the first attribute for example, the difference in the number of successful applicants
  • the second attribute in the subgroups 42 to 45 is evaluated to be explainable discrimination (distinction) using conventional methods, for example. there is a possibility. If it is evaluated that it is an explainable discrimination (distinction), there is a possibility that the fairness evaluation will not be performed for the lower subgroups 46 to 51 (see reference numeral B3).
  • the fairness evaluation device 1 determines that the subgroups 46, 47a, 48, 49, 50a, and 51 that meet the criteria for performing the evaluation are to be subjected to the fairness evaluation. Can be done.
  • FIG. 6 is a block diagram showing a functional configuration example of the fairness evaluation device 1 according to an embodiment.
  • the fairness evaluation device 1 is an example of an information processing device or a computer that executes a fairness evaluation process to evaluate the fairness of data.
  • the fairness evaluation device 1 may extend the training data used for training the machine learning model by executing the fairness evaluation process and processing the training data based on the execution result. This makes it possible to suppress output of unfair inference results by the machine learning model.
  • the fairness evaluation device 1 includes, for example, a memory unit 11, an acquisition unit 12, a protection attribute determination unit 13, a bias calculation unit 14, a subgroup distribution estimation unit 15, an execution standard calculation unit 16, It may include a subgroup search section 17, a positive example ratio calculation section 18, a determination section 19, a fairness index calculation section 20, and a data processing section 21. Further, the fairness evaluation device 1 may include a machine learning section 22 and may further include an inference processing section 23. These blocks 12 to 23 are examples of the control section 24.
  • the memory unit 11 is an example of a storage area, and stores various data used by the fairness evaluation device 1.
  • the memory unit 11 may be realized, for example, by a storage area included in one or both of the memory 10b and the storage unit 10c shown in FIG. 8, which will be described later.
  • the memory unit 11 may be able to store data 111, protected attribute information 112, and training data 113 (processed data), for example. Further, when the fairness evaluation device 1 includes the machine learning section 22, the memory section 11 may be able to store the machine learning model 114. Furthermore, when the fairness evaluation device 1 includes the inference processing section 23, the memory section 11 may be able to store the inference result 115.
  • the information stored in the memory unit 11 may be in a table format or in other formats. In one example, at least one of the information stored in the memory unit 11 may be in various formats such as a DB or an array.
  • the acquisition unit 12 acquires various information used by the fairness evaluation device 1.
  • the acquisition unit 12 may acquire the data 111 and the protected attribute information 112 from a data providing device (not shown) and store them in the memory unit 11.
  • the data 111 is data that includes multiple attributes and is an example of training data.
  • Each of the plurality of attributes may be a protected attribute or an unprotected attribute.
  • the data 111 may be data 111a shown in FIG. 2 or data 111b shown in FIG. 4.
  • the protected attribute information 112 is information for specifying (for example, specifying) a protected attribute among the multiple attributes included in the data 111.
  • the protected attribute determining unit 13 determines a protected attribute from among the multiple attributes included in the data 111 based on the protected attribute information 112.
  • the bias calculation unit 14 calculates the bias in the proportion of positive cases among a plurality of values in the determined protected attribute (for example, between men and women).
  • A a2), and calculates the second ratio P 1 and Based on the third ratio P2 , the bias occurring in the attribute of interest (in one example, the protected attribute) is calculated.
  • the bias is an example of the numerical range C calculated based on the above formula (4).
  • the bias calculation unit 14 calculates a statistical parity difference.
  • the statistical equilibrium difference is expressed as a difference in the proportion of positive cases between groups having different values in the attribute of interest.
  • Statistical equilibrium difference is an example of a fairness indicator.
  • the subgroup distribution estimating unit 15 may identify the overall positive example ratio by estimating the distribution of the positive example ratio for each of the plurality of subgroups included in the data 111. In one example, the subgroup distribution estimating unit 15 may estimate the positive example ratio that indicates the peak of the distribution of positive example ratios for each of the plurality of subgroups.
  • the execution standard calculation unit 16 calculates a standard for selecting a subgroup to perform the fairness evaluation based on the first ratio P 0 , the second ratio P 1 , and the third ratio P 2 .
  • the execution standard calculation unit 16 calculates the parameters C 0 , C 1 , and C 2 based on the above equations (1) to (3) or the above equations ( 1 ) and (4) as criteria for executing the fairness evaluation. You can calculate it.
  • the subgroup search unit 17 searches for multiple subgroups of the data 111.
  • the subgroup searching unit 17 may combine a plurality of attributes (explanatory variables) included in the data 111 to identify a plurality of subgroups, each of which includes at least one protected attribute.
  • the upper limit of the combination length which is the number of attributes to be combined, may be determined in advance based on the expected amount of calculation, depending on the content of the data 111, for example.
  • the subgroup search method may be a width search or a depth search.
  • the positive example ratio calculation unit 18 obtains the positive example ratio of each of the plurality of searched subgroups. For example, the positive example ratio calculation unit 18 may calculate the positive example ratio for each protection attribute value for each of the plurality of subgroups.
  • the positive example ratio in each subgroup is an example of the fourth ratio of data whose first attribute is the first value in the third group.
  • the determining unit 19 determines whether the positive example ratio (fourth ratio) calculated for each of the plurality of subgroups satisfies the criterion. In one example, the determination unit 19 determines that the criterion is satisfied when the calculated positive example ratio is positive example ratio>C1 or positive example ratio ⁇ C2.
  • the fairness index calculation unit 20 performs fairness evaluation on the subgroups determined by the determination unit 19 to satisfy the criteria.
  • the subgroup determined to perform the fairness evaluation is an example of a third group in which the second attribute has the second value and the third attribute of the plurality of attributes has the fourth value.
  • the fairness index calculation unit 20 calculates whether the second attribute is the third value (in one example, the gender is female) and the third attribute of the plurality of attributes is the fourth value. A fifth proportion of data in which the first attribute has a first value may be identified. In other words, the fairness index calculation unit 20 determines that the fourth group is related to the third group for which fairness evaluation is to be performed, and the value of the second attribute, which is the protected attribute, is the same as that of the third group. may identify a fifth proportion for a different fourth group.
  • the fairness index calculation unit 20 executes a fairness evaluation based on the fourth ratio (in one example, the ratio of positive cases among men) and the fifth ratio (in one example, the ratio of positive cases among women). You may.
  • the fairness index may be a statistical parity difference or a DI (Disparate Impact).
  • the statistical parity difference may be based on the difference between the fourth proportion and the fifth proportion.
  • DI (Disparate Impact) may be based on the ratio between the fourth ratio and the fifth ratio.
  • the fairness index is not limited to these cases, and various indexes may be used.
  • the data processing unit 21 generates training data 113 for machine learning by processing a part of the plurality of data 111 based on the fairness evaluation result.
  • the processing method may be any method that suppresses unfairness (discrimination) in the data 111 based on the fairness evaluation result, and may be realized by various known methods.
  • the values of non-protected attributes other than the protected attributes may be rewritten from the data 111 to reduce the correlation between the protected attributes and the non-protected attributes.
  • the fairness evaluation device 1 may include the machine learning section 22 and may further include the inference processing section 23.
  • the machine learning unit 22 executes machine learning processing to train the machine learning model 114 using the processed data processed by the data processing unit 21 to alleviate discrimination as training data 113.
  • the machine learning model 114 may be a neural network (NN) model that includes trained parameters.
  • Machine learning processing may be realized by various known techniques.
  • the inference processing unit 23 performs inference processing using the machine learning model 114 trained based on the processed training data 113. For example, the inference processing unit 23 inputs target data for inference processing (not shown) to the machine learning model 114 and stores the inference result 115 output from the machine learning model 114 in the memory unit 11.
  • FIG. 7 is a flowchart illustrating an example of the operation of the fairness evaluation device 1 according to an embodiment.
  • the acquisition unit 12 of the fairness evaluation device 1 acquires the data 111 and the protected attribute information 112 (step S1), and stores them in the memory unit 11.
  • the first ratio may be a positive example ratio, which is a ratio where the first attribute is positive.
  • A a1) (step S3).
  • the second ratio P 1 (Y 1
  • A a2) (step S4).
  • the third ratio P 2 (Y 1
  • steps S1 to S3 are not limited to the case shown in FIG.
  • the processes of steps S1 to S3 may be executed in parallel.
  • A a2). ), the criteria C 0 , C 1 , and C 2 for selecting a subgroup to perform fairness evaluation are calculated (step S5).
  • C 0 , C 1 , and C 2 may be adjusted according to the estimation results.
  • C 0 may be adjusted depending on the peak position of the distribution of positive example proportions of multiple subgroups, and as the half-width of the distribution of positive example proportions of multiple subgroups becomes wider, C 1 and C C 1 and C 2 may be adjusted so that the numerical range between C 1 and C 2 becomes wide.
  • the subgroup search unit 17 searches for one subgroup included in the first group (step S6). In other words, the subgroup search unit 17 searches for a third group in which the second attribute is the second value (included in the first group) and the third attribute of the plurality of attributes is the fourth value. Explore.
  • the positive example ratio calculation unit 18 specifies the fourth ratio (step S7).
  • the determination unit 19 determines whether the fourth ratio specified in step S7 satisfies the criterion calculated in step S5 (step S8). If the fourth ratio does not meet the criteria (NO in step S8), the process proceeds to step S11.
  • the fairness index calculation unit 20 performs a fairness evaluation for the subgroup determined by the determination unit 19 to satisfy the criterion (step S9).
  • the data processing unit 21 generates training data 113 for machine learning by processing a part of the plurality of data 111 based on the fairness evaluation result (step S10).
  • the processing method may be any method that suppresses unfairness (discrimination) in the data 111 based on the fairness evaluation result, and may be realized by various known methods.
  • the subgroup search unit 17 determines whether the subgroup search has ended (step S11). Note that the subgroup search unit 17 may determine whether the search has ended based on the upper limit of the combination length and the searched subgroups.
  • step S11 If the search for the subgroup is not completed (NO in step S11), the processes from step S6 to step S11 are repeated.
  • step S11 If the search is completed (YES in step S11), the machine learning unit 22 trains the machine learning model 114 using the generated training data 113 (step S12), and the process ends.
  • A a2).
  • the fairness evaluation device 1 selects the first value from among the third group in which the second attribute is the second value and the third attribute of the plurality of attributes is the fourth value. If a fourth proportion of data whose attribute has the first value satisfies the criterion, then a fairness evaluation is performed for the third group.
  • A a2). Based on.
  • the objective variable (Y) is whether the user felt cold or not.
  • the explanatory variables (attributes) are five attributes including "gender" which is a protected attribute.
  • the number of dimensions of open data after binary conversion is 20.
  • the fairness index is calculated for all combinations of the open data described above, the fairness index will be calculated for 21,699 subgroups.
  • the fairness index is calculated for 6301 subgroups. Therefore, in the above example, the calculation of the fairness index for 15,398 subgroups can be omitted, and the calculation processing load can be reduced accordingly.
  • the process of performing the fairness evaluation has a numerical range obtained based on the difference between the second ratio P 1 and the third ratio P 2 and ranges the first ratio P 0 . This includes processing to calculate the numerical range C included within the range C. If the fourth ratio is outside the numerical range C, the computer 10 determines that the criterion is met.
  • the process of performing the fairness evaluation is performed when the second attribute is a third value and the third attribute of the plurality of attributes is a fourth value.
  • a fifth proportion of data whose attribute has the first value is specified, and the fairness evaluation is performed based on the fourth proportion and the fifth proportion.
  • the fairness evaluation device 1 further generates training data for machine learning by processing a portion of the plurality of data based on the fairness evaluation results.
  • the fairness evaluation device 1 may be a virtual server (VM) or a physical server. Further, the functions of the fairness evaluation device 1 may be realized by one computer, or may be realized by two or more computers. Furthermore, at least some of the functions of the fairness evaluation device 1 may be realized using HW (Hardware) resources and NW (Network) resources provided by a cloud environment.
  • HW Hardware
  • NW Network
  • FIG. 8 is a block diagram showing an example of the hardware (HW) configuration of the computer 10 that implements the functions of the fairness evaluation device 1 according to an embodiment.
  • HW hardware
  • the computer 10 includes, as an example, a processor 10a, a memory 10b, a storage section 10c, an IF (Interface) section 10d, an IO (Input/Output) section 10e, and a reading section 10f as an HW configuration. You can prepare.
  • the processor 10a is an example of an arithmetic processing device that performs various controls and calculations.
  • the processor 10a may be communicably connected to each block within the computer 10 via a bus 10i.
  • the processor 10a may be a multiprocessor including a plurality of processors, a multicore processor having a plurality of processor cores, or a configuration including a plurality of multicore processors.
  • Examples of the processor 10a include integrated circuits (ICs) such as a CPU, MPU, GPU, APU, DSP, ASIC, and FPGA. Note that a combination of two or more of these integrated circuits may be used as the processor 10a.
  • ICs integrated circuits
  • CPU is an abbreviation for Central Processing Unit
  • MPU is an abbreviation for Micro Processing Unit
  • GPU is an abbreviation for Graphics Processing Unit
  • APU is an abbreviation for Accelerated Processing Unit.
  • DSP is an abbreviation for Digital Signal Processor
  • ASIC is an abbreviation for Application Specific IC
  • FPGA is an abbreviation for Field-Programmable Gate Array.
  • the processor 10a includes a processing device such as a CPU that executes the fairness evaluation process, It may be combined with an accelerator that executes machine learning processing or inference processing.
  • the accelerator include the above-mentioned GPU, APU, DSP, ASIC, or FPGA.
  • the memory 10b is an example of HW that stores information such as various data and programs.
  • Examples of the memory 10b include one or both of a volatile memory such as a DRAM (Dynamic Random Access Memory), and a non-volatile memory such as a PM (Persistent Memory).
  • a volatile memory such as a DRAM (Dynamic Random Access Memory)
  • a non-volatile memory such as a PM (Persistent Memory).
  • the storage unit 10c is an example of HW that stores information such as various data and programs.
  • Examples of the storage unit 10c include various storage devices such as magnetic disk devices such as HDDs (Hard Disk Drives), semiconductor drive devices such as SSDs (Solid State Drives), and nonvolatile memories.
  • Examples of nonvolatile memory include flash memory, SCM (Storage Class Memory), and ROM (Read Only Memory).
  • the storage unit 10c may store a program 10g (fairness evaluation program) that implements all or part of various functions of the computer 10.
  • a program 10g fairness evaluation program
  • the processor 10a of the fairness evaluation device 1 expands the program 10g stored in the storage unit 10c into the memory 10b and executes it, thereby functioning as the fairness evaluation device 1 (the control unit 24 illustrated in FIG. 6). function can be realized.
  • the IF unit 10d is an example of a communication IF that controls connections and communications between various networks, including a network between the fairness evaluation device 1 and devices not shown.
  • Examples of the device include a computer such as a user terminal or a server that provides data to the fairness evaluation device 1, and a computer such as a server that performs machine learning processing based on data output from the fairness evaluation device 1.
  • the IF unit 10d may include an adapter compliant with LAN (Local Area Network) such as Ethernet (registered trademark), optical communication such as FC (Fibre Channel), etc.
  • the adapter may be compatible with one or both of wireless and wired communication systems.
  • program 10g may be downloaded from the network to the computer 10 via the communication IF and stored in the storage unit 10c.
  • the IO unit 10e may include one or both of an input device and an output device.
  • Examples of the input device include a keyboard, mouse, touch panel, and the like.
  • Examples of the output device include a monitor, a projector, and a printer.
  • the IO unit 10e may include a touch panel or the like that is an integrated input device and display device.
  • the reading unit 10f is an example of a reader that reads data and program information recorded on the recording medium 10h.
  • the reading unit 10f may include a connection terminal or device to which the recording medium 10h can be connected or inserted.
  • Examples of the reading unit 10f include an adapter compliant with USB (Universal Serial Bus), a drive device that accesses a recording disk, a card reader that accesses a flash memory such as an SD card, and the like.
  • the program 10g may be stored in the recording medium 10h, or the reading unit 10f may read the program 10g from the recording medium 10h and store it in the storage unit 10c.
  • Examples of the recording medium 10h include non-temporary computer-readable recording media such as magnetic/optical disks and flash memories.
  • Examples of magnetic/optical discs include flexible discs, CDs (Compact Discs), DVDs (Digital Versatile Discs), Blu-ray discs, and HVDs (Holographic Versatile Discs).
  • Examples of flash memory include semiconductor memories such as USB memory and SD cards.
  • the HW configuration of the computer 10 described above is an example. Therefore, the number of HWs within the computer 10 may be increased or decreased (eg, adding or deleting arbitrary blocks), dividing, integrating in any combination, adding or deleting buses, etc., as appropriate.
  • the blocks 12 to 21 included in the fairness evaluation device 1 shown in FIG. 6 may be combined in any combination, or may be divided.
  • the fairness evaluation device 1 shown in FIG. 6 may have a configuration (system) in which a plurality of devices cooperate with each other via a network to realize each processing function.
  • the memory unit 11 may be a DB server
  • the acquisition unit 12 may be a web server or an application server
  • the blocks 13 to 23 may be an application server.
  • each processing function of the fairness evaluation device 1 may be realized by the DB server, application server, and web server cooperating with each other via a network.
  • Fairness evaluation device 10 Computer 10a Processor 10b Memory 10c Storage unit 10d IF unit 10e IO unit 10f Reading unit 10h Storage medium 10g Program 11 Memory unit 12 Acquisition unit 13 Protection attribute determination unit 14 Bias calculation unit 15 Subgroup distribution estimation unit 16 10 Hisao criterion calculation unit 17 Subgroup search unit 18 Positive example ratio calculation unit 19 Judgment unit 20 Fairness index calculation unit 21 Data processing unit 22 Machine learning unit 23 Inference processing unit 24 Control unit

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Business, Economics & Management (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Human Resources & Organizations (AREA)
  • Strategic Management (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Probability & Statistics with Applications (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • General Health & Medical Sciences (AREA)
  • Pure & Applied Mathematics (AREA)
  • Algebra (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Economics (AREA)
  • Marketing (AREA)
  • Operations Research (AREA)
  • Quality & Reliability (AREA)
  • Tourism & Hospitality (AREA)
  • General Business, Economics & Management (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
PCT/JP2022/032546 2022-08-30 2022-08-30 公平性評価プログラム、公平性評価方法、及び、情報処理装置 Ceased WO2024047735A1 (ja)

Priority Applications (4)

Application Number Priority Date Filing Date Title
EP22957338.1A EP4583014A4 (en) 2022-08-30 2022-08-30 FAIRNESS ASSESSMENT PROGRAM, FAIRNESS ASSESSMENT METHOD AND INFORMATION PROCESSING DEVICE
JP2024543641A JP7768409B2 (ja) 2022-08-30 2022-08-30 公平性評価プログラム、公平性評価方法、及び、情報処理装置
PCT/JP2022/032546 WO2024047735A1 (ja) 2022-08-30 2022-08-30 公平性評価プログラム、公平性評価方法、及び、情報処理装置
US19/049,211 US20250190876A1 (en) 2022-08-30 2025-02-10 Computer-readable recording medium having stored therein fairness evaluation program, fairness evaluation method, and information processing apparatus

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2022/032546 WO2024047735A1 (ja) 2022-08-30 2022-08-30 公平性評価プログラム、公平性評価方法、及び、情報処理装置

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US19/049,211 Continuation US20250190876A1 (en) 2022-08-30 2025-02-10 Computer-readable recording medium having stored therein fairness evaluation program, fairness evaluation method, and information processing apparatus

Publications (1)

Publication Number Publication Date
WO2024047735A1 true WO2024047735A1 (ja) 2024-03-07

Family

ID=90098935

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/032546 Ceased WO2024047735A1 (ja) 2022-08-30 2022-08-30 公平性評価プログラム、公平性評価方法、及び、情報処理装置

Country Status (4)

Country Link
US (1) US20250190876A1 (https=)
EP (1) EP4583014A4 (https=)
JP (1) JP7768409B2 (https=)
WO (1) WO2024047735A1 (https=)

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022044064A1 (ja) * 2020-08-24 2022-03-03 富士通株式会社 機械学習データ生成プログラム、機械学習データ生成方法、機械学習データ生成装置、分類データ生成プログラム、分類データ生成方法および分類データ生成装置

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022044064A1 (ja) * 2020-08-24 2022-03-03 富士通株式会社 機械学習データ生成プログラム、機械学習データ生成方法、機械学習データ生成装置、分類データ生成プログラム、分類データ生成方法および分類データ生成装置

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
FAISAL KAMIRAN; TOON CALDERS: "Data preprocessing techniques for classification without discrimination", KNOWLEDGE AND INFORMATION SYSTEMS, SPRINGER-VERLAG, LO, vol. 33, no. 1, 3 December 2011 (2011-12-03), Lo , pages 1 - 33, XP035120934, ISSN: 0219-3116, DOI: 10.1007/s10115-011-0463-8 *
KAMI RAN , FAISALZLIOBAITE, INDRECALDERS, TOON: "Quantifying explainable discrimination and removing illegal discrimination in automated decision making", KNOWLEDGE AND INFORMATION SYSTEMS, 2013
See also references of EP4583014A4
YANG MINGZHE, ARAI HIROMI, BABA YUKINO, RIKEN : " Mitigating Unconscious Biases by Machine Teaching and Fairness-aware Machine Learning", 2C3-OS-9A-03. THE 35TH ANNUAL CONFERENCE OF THE JAPANESE SOCIETY FOR ARTIFICIAL INTELLIGENCE., 1 June 2021 (2021-06-01) - 11 June 2021 (2021-06-11), pages 1 - 4, XP093145516, DOI: 10.11517/pjsai.JSAI2021.0_2C3OS9a03 *

Also Published As

Publication number Publication date
US20250190876A1 (en) 2025-06-12
EP4583014A1 (en) 2025-07-09
JP7768409B2 (ja) 2025-11-12
EP4583014A4 (en) 2025-10-15
JPWO2024047735A1 (https=) 2024-03-07

Similar Documents

Publication Publication Date Title
JP6771751B2 (ja) リスク評価方法およびシステム
CN117521117A (zh) 一种医疗数据应用安全与隐私保护方法及系统
CN114270391B (zh) 量化隐私影响
US11620406B2 (en) Information processing device, information processing method, and recording medium
Hou et al. Multi-feature SEIR model for epidemic analysis and vaccine prioritization
Liu Application of K-means supported by clustered systems in big data association rule mining
WO2014034557A1 (ja) テキストマイニング装置、テキストマイニング方法及びコンピュータ読み取り可能な記録媒体
Luo et al. Fairness without demographics through learning graph of gradients
CN114490302B (zh) 一种基于大数据分析的威胁行为分析方法及服务器
JP6445415B2 (ja) 匿名化装置、匿名化方法、プログラム
JP7768409B2 (ja) 公平性評価プログラム、公平性評価方法、及び、情報処理装置
Lou et al. A new and unified method for regression analysis of interval‐censored failure time data under semiparametric transformation models with missing covariates
JP2022111841A (ja) 表示プログラム、情報処理装置および表示方法
JP7143599B2 (ja) メタデータ評価装置、メタデータ評価方法、およびメタデータ評価プログラム
JP6556681B2 (ja) 匿名化テーブル生成装置、匿名化テーブル生成方法、プログラム
JP2022007311A (ja) 学習モデルからの情報漏洩リスクを評価するシステム
JP6549076B2 (ja) 匿名化テーブル生成装置、匿名化テーブル生成方法、プログラム
JP2023039832A (ja) 抽出プログラム、抽出方法および情報処理装置
WO2019019711A1 (zh) 行为模式数据的发布方法、装置、终端设备及介质
KR102875886B1 (ko) 밀도 기반 군집화 결과를 차분 프라이버시를 만족하도록 변환하는 장치 및 방법
EP4092585A1 (en) Control method, control program, and information processing device
JP7639946B2 (ja) 生成方法、生成プログラム、及び、情報処理装置
JP7634800B2 (ja) 情報処理装置、情報処理方法、及び情報処理プログラム
KR102932663B1 (ko) 개인정보를 이용하여 타겟 이벤트의 발생을 예측하기 위한 단말, 개인정보 관리 서버 및 이들을 포함하는 시스템
KR102751506B1 (ko) 개인정보 관리 서버 및 이의 작동 방법

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22957338

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2024543641

Country of ref document: JP

WWE Wipo information: entry into national phase

Ref document number: 2022957338

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2022957338

Country of ref document: EP

Effective date: 20250331

WWP Wipo information: published in national office

Ref document number: 2022957338

Country of ref document: EP