WO2022121032A1 - 一种在联邦学习场景下的数据集划分方法及系统 - Google Patents
一种在联邦学习场景下的数据集划分方法及系统 Download PDFInfo
- Publication number
- WO2022121032A1 WO2022121032A1 PCT/CN2020/140882 CN2020140882W WO2022121032A1 WO 2022121032 A1 WO2022121032 A1 WO 2022121032A1 CN 2020140882 W CN2020140882 W CN 2020140882W WO 2022121032 A1 WO2022121032 A1 WO 2022121032A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- data
- model
- federated
- federated learning
- training
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 27
- 238000012549 training Methods 0.000 claims abstract description 83
- 238000012360 testing method Methods 0.000 claims abstract description 79
- 238000013145 classification model Methods 0.000 claims abstract description 60
- 238000009826 distribution Methods 0.000 claims abstract description 59
- 238000012795 verification Methods 0.000 claims abstract description 20
- 238000010200 validation analysis Methods 0.000 claims description 38
- 238000011156 evaluation Methods 0.000 claims description 33
- 230000002776 aggregation Effects 0.000 claims description 12
- 238000004220 aggregation Methods 0.000 claims description 12
- 238000004364 calculation method Methods 0.000 claims description 11
- 238000011056 performance test Methods 0.000 claims description 6
- 238000005457 optimization Methods 0.000 claims description 2
- 238000000638 solvent extraction Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 7
- 238000009825 accumulation Methods 0.000 description 3
- 238000010801 machine learning Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000013136 deep learning model Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Definitions
- the invention relates to a data division method, in particular to a data set division method and system in a federated learning scenario.
- Federated machine learning also known as federated learning, federated learning, and federated learning
- federated learning is a machine learning framework that can effectively help multiple institutions conduct data usage and machine learning while meeting the requirements of user privacy protection, data security, and government regulations modeling.
- the data distribution provided by various institutions is often uneven, and it is common that the data do not meet the same data distribution conditions. If the data provided by each institution is directly used for federated learning without any processing, the accuracy of the learned model is usually not high. Therefore, in federated learning, it is particularly necessary to ensure the consistency of the data distribution of the data provided by all parties.
- the dataset is pre-divided into a training set for model training and a validation set for validating model performance.
- the evaluation of the actual performance of the model depends on the test data set.
- the data distributions of the validation dataset and the test dataset are significantly different, different results will be obtained by verifying the model performance of the same model through the validation dataset and the test dataset, and the model performance cannot be accurately evaluated. Therefore, in model training, how to divide the data set so that the divided validation data set is as consistent as possible with the data distribution of the test data set becomes the key to ensuring the effect of model training.
- the invention aims to find the data samples most similar to the data distribution of the test data set in the data provided by each participant of the federated learning as the verification set for model training, and provides a data set division method and system in the federated learning scenario .
- Step S1 judging whether the data distribution of the original data provided by each federated learning participant is consistent
- Step S2 using the original data provided by each of the federated learning participants with consistent data distribution, as well as model test data training, and optimization using the validation set to obtain the optimal federated classification model M1;
- Step S3 inputting the raw data belonging to each federated learning participant into the federated classification model M1, and the federated classification model M1 outputs the probability that the model input data is the model test data;
- Step S4 according to the prediction probability from high to low, select a specified number of the model input data and divide it into a verification set provided by the federated learning participant to which the data belongs to verify the performance of the model, and the remaining model input data is divided into: The training set for training the model provided by the federated learning participant to which the data belongs.
- the method for judging whether the data distribution of the raw data provided by each of the federated learning participants is consistent specifically includes:
- Step S11 dividing the raw data provided by the federated learning participant into a training set, a verification set and a test set that are consistent with the data distribution of the raw data;
- Step S12 assigning corresponding data labels to the divided training sets and verification sets belonging to each of the federated learning participants
- Step S13 using the training set with data labels belonging to each of the federated learning participants to train, and using the verification set to optimize to obtain the optimal federated classification model M2;
- Step S14 inputting the test set belonging to each of the federated learning participants into the federated classification model M2, to obtain several local performance evaluation indicators for the federated classification model M2 to distinguish the input data of each belonging party;
- Step S15 perform aggregation calculation on the values of each of the local performance evaluation indicators obtained by the federated classification model M2 to distinguish the attribution of the input data to obtain a global evaluation indicator value, and determine each Whether the data distribution of the original data provided by the federated learning participants is consistent.
- the present invention also provides a data set division system in a federated learning scenario, where the data set division system includes:
- the data distribution consistency judgment module is used to judge whether the data distribution of the original data provided by each federated learning participant is consistent
- a data label assignment module used for assigning and storing corresponding data labels to the original data provided by the federated learning participants with consistent data distribution, and assigning and storing corresponding data labels to model test data;
- a data acquisition module connected to the data label assignment module, for acquiring the original data after label assignment as a model training sample, and acquiring the model test data as a model verification sample;
- the M1 federated classification model training module is connected to the data acquisition module, and is used for training using the obtained raw data and the model test data provided by the federated learning participants, and optimizing the validation set to obtain the optimal federated classification model M1;
- the M1 model performance testing module is connected to the data acquisition module and the M1 federated classification model training module respectively, and is used to input the acquired raw data belonging to each federated learning participant into the federated classification model M1 , the probability that the federated classification model M1 outputs the model input data as the model test data;
- the verification set selection module is connected to the M1 model performance test module and the data acquisition module, and is used to select a specified amount of the model input data from high to low according to the predicted probability as the federated learning participant of the data attribution to provide.
- the validation set used to verify the performance of the model, and the remaining input data of the model is used as a training set provided by the federated learning participant to which the data belongs for training the model.
- the data distribution consistency judgment module specifically includes:
- a data dividing unit configured to divide the raw data provided by each federated learning participant into a training set, a verification set and a test set that are consistent with the data distribution of the raw data
- the data label assignment unit is connected to the data division unit, and is used to assign corresponding data labels to the divided training sets and verification sets belonging to each of the federated learning participants, and assign corresponding data labels to the model test data.
- data label ;
- the M2 federated classification model training unit is connected to the data label assignment unit, and used for training using the training set with data labels belonging to each federated learning participant, and optimizing the validation set to obtain the optimal federated classification model M2;
- the M2 model performance testing unit is respectively connected to the data division unit and the M2 federated classification model training unit, and is used for inputting the test set belonging to each federated learning participant into the federated classification model M2, obtaining several local performance evaluation indexes for distinguishing the input data of each attributable party by the federated classification model M2;
- the numerical aggregation calculation unit is connected to the M2 model performance testing unit, and is used for performing aggregation calculation on the values of each of the local performance evaluation indicators obtained by the federated classification model M2 for distinguishing the attribution of the input data, to obtain a global evaluation index value ;
- the data distribution consistency judgment unit is connected to the numerical aggregation calculation unit, and is used for judging whether the data distribution of the original data provided by each of the federated learning participants is consistent according to the global evaluation index value.
- the original data provided by each federated learning participant can be reasonably divided into a training set and a validation set.
- the divided validation set and the test set have the same or similar data distribution, which is beneficial to improve the model performance of the federated learning model.
- FIG. 1 is a step diagram of a data set division method in a federated learning scenario provided by an embodiment of the present invention
- FIG. 2 is a schematic diagram of dividing a data set by a data set dividing method in a federated learning scenario provided by an embodiment of the present invention
- Fig. 3 is a method step diagram of the present invention for judging whether the data distribution of the original data provided by each federated learning participant is consistent;
- FIG. 4 is a schematic diagram of the present invention for judging whether the data distribution of the original data provided by each federated learning participant is consistent;
- FIG. 5 is a schematic diagram of the system structure of a data set partitioning system in a federated learning scenario provided by an embodiment of the present invention
- FIG. 6 is a schematic diagram of the internal structure of the data distribution consistency judgment module in the data set division system.
- connection or the like appears to indicate a connection relationship between components, the term should be understood in a broad sense, for example, it may be a fixed connection or a detachable connection It can be connected or integrated; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be an internal connection between two components or an interaction relationship between the two components.
- connection or the like appears to indicate a connection relationship between components, the term should be understood in a broad sense, for example, it may be a fixed connection or a detachable connection It can be connected or integrated; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be an internal connection between two components or an interaction relationship between the two components.
- a method for dividing a dataset in a federated learning scenario provided by an embodiment of the present invention, as shown in FIG. 1 includes the following steps:
- Step S1 judging whether the data distribution of the original data provided by each federated learning participant is consistent
- Step S2 use the original data with consistent data distribution provided by each federated learning participant and the model test data to train, and use the validation set to optimize to obtain the optimal federated classification model M1;
- the present invention firstly assigns corresponding data labels to the original data provided by each federated learning participant, and assigns corresponding data labels to the model test data.
- the raw data provided by the federated learning participant A is recorded as A-data
- the raw data provided by the federated learning participant B is recorded as B-data.
- the same data label 0 can be assigned to A-data and B-data, which are respectively recorded as A-data
- the model test data is recorded as test
- the label 1 can be assigned to the test data, recorded as test
- Step S3 input the original data belonging to each federated learning participant into the federated classification model M1, and the federated classification model M1 outputs the probability that the model input data is the model test data; for example, the raw data A- provided by the federated learning participant A- data is input into the model M1, and the model M1 will output the probability that A-data is the model test data;
- Step S4 according to the prediction probability from high to low, select a specified number of model input data and divide it into a validation set provided by the federated learning participant to which the data belongs to verify the performance of the model, and the remaining model input data is divided into the federated learning participation of the data attribution.
- the training set provided by the party for training the model. For example, when the data input into the model M1 in step S3 is A-data, according to the predicted probability from high to low, a specified amount of data is selected from the original data A-data provided by the federated learning participant A and divided into the federated learning participant A.
- the validation set used to verify the performance of the model (for example, 20% of the A-data is selected as the validation set according to the predicted probability from high to low), and the remaining 80% of the model input data is divided into federated learning participant A for training.
- the training set of the model is selected as 20% of the A-data.
- the objects participating in the federated learning are two federated learning participants A and B as an example, and with reference to FIG. 2 , the data set division method provided by the present invention is described in detail:
- the raw data provided by the federated learning participant A is recorded as A-data
- the raw data provided by the federated learning participant B is recorded as B-data.
- Add data label 0 to the data denoted as A-data
- add label 1 to the model test data denoted as test-1
- test-1 add label 1 to the model test data
- test-1 uses A-data
- Perform federated learning train and optimize to form an optimal federated learning classification model M1.
- the model M1 is used to distinguish whether the input data is model test data or whether it can be used as model test data.
- the model M1 After the training of the model M1 is completed, input the data A-data and B-data into the model M1 for scoring prediction, and output the probability that the input data belongs to or can be used as the model test data, and the probability value is between 0 and 1.
- the input data is sorted according to the probability value output by the model from high to low. The higher the score, the closer the data distribution of the input data and the model test data is.
- select an appropriate number of data samples from the input data as the validation set for model training for example, select 20% of the input data according to the probability value from high to low as the validation set provided by the federated learning participants to which the data belongs.
- the remaining 80% is used as the training set provided by the federated learning participant to which the data belongs, so that the original data A-data provided by the federated learning participant A is split into a training set A-train and a validation set A-valid.
- the original data B-data provided by the federated learning participant B is split into a training set A-train and a validation set B-valid.
- step S1 the method for judging whether the data distribution of the original data provided by each federated learning participant is consistent is shown in Figure 3, which specifically includes:
- step S11 the original data provided by each federated learning participant is divided into a training set, a validation set and a test set that are consistent with the data distribution of the original data; for example, the raw data provided by the federated learning participant A is recorded as A-data, and the original The data A-data is divided into a training set A-train, a validation set A-valid and a test set A-test that are consistent with the data distribution of the original data.
- the original data provided by the federated learning participant B is recorded as B-data, and the original data B-data is divided into training set B-train, validation set B-valid and test set B-test;
- Step S12 assigning corresponding data labels to the divided training sets belonging to each federated learning participant, for example, adding a data label 0 to the training set A-train provided by the federated learning participant A, denoted as A-train
- Step S13 use the training sets with data labels belonging to each federated learning participant, for example, when the federated learning participants only have A and B, use the training sets A-train
- Step S14 input the test set A-test and B-test into the model M2, and obtain several local performance evaluation indicators for the model M2 to distinguish the input data of each belonging party;
- Step S15 perform aggregation calculation on the values of each local performance evaluation index obtained by the federated classification model M2 to distinguish the attribution of the input data to obtain a global evaluation index value, and according to the global evaluation index value, determine the respective federated learning participants respectively. Whether the data distribution of the provided raw data is consistent.
- the data distribution of the original data provided by the existing federated learning participants A and B, A and B is inconsistent.
- the original data provided by Party A is divided into training set, validation set and test set which are consistent with the original data distribution, which are recorded as A-train, A-valid and A-test respectively.
- the original data provided by Party B is divided into training set, validation set and test set that are consistent with the original data, which are recorded as B-train, B-valid and B-test respectively, and A-train, A-valid Add data label 0, denoted as A-train
- Model M2 is used to distinguish the input data belonging to Party A. Or the B side.
- A-test and B-test are input into model M2, and the evaluation index AUC (AUC (Area Under Cruve)) for evaluating whether A-test belongs to party A is defined as the coordinate axis under the ROC curve
- the area formed by the enclosure is a common local performance evaluation index used to evaluate the prediction performance of the model.
- the AUC evaluation index is used to evaluate the probability that the input data is provided by the corresponding owner), and to obtain the evaluation B-test Whether it belongs to the evaluation index AUC of Party B.
- the indicator values of the two AUC indicators are aggregated, and according to the aggregation results, for example, through a threshold judgment method, it is judged whether the data distribution of the original data provided by the A and B parties is consistent.
- the threshold judgment method is as follows:
- the difference between the numerical accumulation result of each performance evaluation index AUC and 0.5 is greater than or equal to the threshold ⁇ , it means that the data distribution of the original data provided by each federated learning participant is inconsistent.
- the difference between the numerical accumulation result of each performance evaluation index AUC and 0.5 is less than or equal to - ⁇ , it indicates that the model performance of the federated classification model M2 is unqualified, and the model M2 needs to be retrained.
- the threshold ⁇ is based on experience. When there are only two federated learning participants participating in the federated learning, the threshold ⁇ is preferably equal to 0.2.
- the present invention also provides a data set division system in a federated learning scenario, which can implement the above-mentioned data set division method.
- the data set division system includes:
- the data distribution consistency judgment module is used to judge whether the data distribution of the original data provided by each federated learning participant is consistent
- the data label assignment module is used to assign the corresponding data labels to the original data provided by the federated learning participants with the same data distribution, and store them.
- the test data is assigned a corresponding data label and stored, for example, a data label 1 is assigned to the model test data test;
- the data acquisition module is connected to the data label assignment module, which is used to obtain the original data after the label assignment as a model training sample, and obtain the model test data as a model verification sample;
- the M1 federated classification model training module is connected to the data acquisition module, and is used for training with the obtained original data and model test data provided by each federated learning participant, and using the validation set to optimize to obtain the optimal federated classification model M1;
- the M1 model performance test module is connected to the data acquisition module and the M1 federated classification model training module respectively, and is used to input the acquired raw data belonging to each federated learning participant into the federated classification model M1, and the federated classification model M1 outputs the model input data the probability that data is or can be used as model test data;
- the validation set selection module is connected to the M1 model performance test module, which is used to select a specified number of model input data from high to low according to the predicted probability as the validation set provided by the federated learning participants to verify the model performance.
- the model input data is used as a training set for training the model provided by the federated learning participant to which the data belongs. The methods for selecting the validation set and the training set are described in detail in the above-mentioned data set division method, and will not be repeated here.
- the data distribution consistency judgment module specifically includes:
- the data division unit is used to divide the original data provided by each federated learning participant into a training set, a validation set and a test set that are consistent with the data distribution of the original data;
- the data label assignment unit is connected to the data division unit, which is used to assign corresponding data labels to the divided training sets and validation sets belonging to each federated learning participant, and assign corresponding data labels to the model test data;
- the method is specifically described in the above-mentioned data set division method, and will not be repeated here;
- the M2 federated classification model training unit is connected to the data label assignment unit, which is used to train the training set with data labels belonging to each federated learning participant, and use the validation set to optimize to obtain the optimal federated classification model M2;
- the M2 model performance test unit is connected to the data division unit and the M2 federated classification model training unit respectively, and is used to input the test set belonging to each federated learning participant into the federated classification model M2, and obtain the federated classification model M2 to distinguish the input of each attributable party.
- the numerical aggregation calculation unit is connected to the M2 model performance test unit, and is used to aggregate and calculate the index values of each local performance evaluation index obtained by the federated classification model M2 to distinguish the attribution of the input data, and obtain a global evaluation index value;
- the data distribution consistency judgment unit is connected to the numerical aggregation calculation unit, and is used to judge whether the data distribution of the original data provided by each federated learning participant is consistent according to the global evaluation index value.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
本发明公开了一种在联邦学习场景下的数据集划分方法及系统,其中方法包括:判断各联邦学习参与方提供的原始数据的数据分布是否一致;使用各联邦学习参与方提供的原始数据和模型测试数据训练形成联邦分类模型;将归属于各联邦学习参与方的原始数据输入到联邦分类模型中,联邦分类模型输出模型输入数据为模型测试数据的概率;按照预测概率由高到低选取指定数量的模型输入数据作为数据归属的联邦学习参与方提供的用于验证模型性能的验证集,剩余的模型输入数据作为数据归属的联邦学习参与方提供的用于训练模型的训练集。本发明能够在联邦学习各参与方提供的数据中找到与测试数据集数据分布最相似的数据样本作为模型训练的验证集。
Description
本发明涉及一种数据划分方法,具体涉及一种在联邦学习场景下的数据集划分方法及系统。
联邦机器学习,又名联邦学习、联合学习、联盟学习,联邦学习是一个机器学习框架,能有效帮助多个机构在满足用户隐私保护、数据安全和政府法规的要求下,进行数据使用和机器学习建模。在联邦学习场景中,各个机构提供的数据分布往往不均衡,互相不满足数据同分布条件的情况很常见。如果对各个机构提供的数据不作任何处理直接使用进行联邦学习,学习而得的模型精度通常不高。所以在联邦学习中,确保各方提供的数据的数据分布的一致性显得尤为必要。
训练深度学习模型时,会预先将数据集划分为用于模型训练的训练集和用于验证模型性能的验证集。但对于模型实际性能好坏的评价依赖于测试数据集。理想状态下,我们希望验证数据集和测试数据集的数据分布一致,这样在模型训练过程中利用验证数据集即可较好地评估模型性能,而且评估结果近似于使用测试数据集对模型实际性能的评估结果。但如果验证数据集和测试数据集的数据分布明显不同,则通过验证数据集和测试数据集验证同个模型的模型性能就会得到不同的结果,模型性能无法获得准确评估。所以在模型训练中,如何划分数据集,以使得划分的验证数据集尽可能与测试数据集的数据分布一致成为确保模型训练效果的关键。
发明内容
本发明以在联邦学习各参与方提供的数据中找到与测试数据集数据分布最相似的数据样本作为模型训练的验证集为目的,提供了一种在联邦学习场景下的数据集划分方法及系统。
为达此目的,本发明采用以下技术方案:
提供一种在联邦学习场景下的数据集划分方法,包括如下步骤:
步骤S1,判断各联邦学习参与方提供的原始数据的数据分布是否一致;
步骤S2,使用各所述联邦学习参与方提供的数据分布一致的所述原始数据,以及模型测试数据训练、使用验证集优化得到最优的联邦分类模型M1;
步骤S3,将归属于各所述联邦学习参与方的所述原始数据输入到所述联邦分类模型M1中,所述联邦分类模型M1输出模型输入数据为所述模型测试数据的概率;
步骤S4,按照预测概率由高到低选取指定数量的所述模型输入数据划分为数据归属的所述联邦学习参与方提供的用于验证模型性能的验证集,剩余的所述模型输入数据划分为数据归属的所述联邦学习参与方提供的用于训练模型的训练集。
优选地,所述步骤S1中,判断各所述联邦学习参与方提供的所述原始数据的数据分布是否一致的方法具体包括:
步骤S11,将所述联邦学习参与方提供的所述原始数据划分为与所述原始数据的数据分布一致的训练集、验证集和测试集;
步骤S12,为划分的分属于各所述联邦学习参与方的训练集、验证集赋予相对应的数据标签;
步骤S13,使用带有数据标签的分属于各所述联邦学习参与方的所述训练集训练、使用验证集优化得到最优的联邦分类模型M2;
步骤S14,将分属于各所述联邦学习参与方的所述测试集输入到所述联邦分类模型M2中,得到所述联邦分类模型M2区分各归属方输入数据的若干个本地性能评价指标;
步骤S15,对所述联邦分类模型M2为区分输入数据归属方得到的各所述本地性能评价指标的值进行聚合计算,得到一全局评价指标值,并根据所述全局评价指标值判断出各所述联邦学习参与方分别提供的所述原始数据的数据分布是否一致。
本发明还提供了一种在联邦学习场景下的数据集划分系统,所述数据集划分系统包括:
数据分布一致性判断模块,用于判断各联邦学习参与方提供的原始数据的数据分布是否一致;
数据标签赋予模块,用于为数据分布一致的各所述联邦学习参与方提供的所述原始数据赋予相对应的数据标签并存储,并为模型测试数据赋予相对应的数据标签并存储;
数据获取模块,连接所述数据标签赋予模块,用于获取经标签赋予后的的所述原始数据作为模型训练样本,获取所述模型测试数据作为模型验证样本;
M1联邦分类模型训练模块,连接所述数据获取模块,用于使用获取的各所述联邦学习参与方提供的所述原始数据和所述模型测试数据训练、使用验证集优化得到最优的联邦分类模型M1;
M1模型性能测试模块,分别连接所述数据获取模块和所述M1联邦分类模型训练模块,用于将获取的归属于各所述联邦学习参与方的所述原始数据输入 到所述联邦分类模型M1中,所述联邦分类模型M1输出模型输入数据为所述模型测试数据的概率;
验证集选定模块,连接所述M1模型性能测试模块和所述数据获取模块,用于按照预测概率由高到低选取指定数量的所述模型输入数据作为数据归属的所述联邦学习参与方提供的用于验证模型性能的验证集,剩余的所述模型输入数据作为数据归属的所述联邦学习参与方提供的用于训练模型的训练集。
优选地,所述数据分布一致性判断模块中具体包括:
数据划分单元,用于将各所述联邦学习参与方提供的所述原始数据划分为与所述原始数据的数据分布一致的训练集、验证集和测试集;
数据标签赋予单元,连接所述数据划分单元,用于为划分的分属于各所述联邦学习参与方的训练集、验证集赋予相对应的数据标签,并为所述模型测试数据赋予相对应的数据标签;
M2联邦分类模型训练单元,连接所述数据标签赋予单元,用于使用带有数据标签的分属于各所述联邦学习参与方的所述训练集训练、使用验证集优化得到最优的联邦分类模型M2;
M2模型性能测试单元,分别连接所述数据划分单元和所述M2联邦分类模型训练单元,用于将分属于各所述联邦学习参与方的所述测试集输入到所述联邦分类模型M2中,得到所述联邦分类模型M2区分各归属方输入数据的若干个本地性能评价指标;
数值聚合计算单元,连接所述M2模型性能测试单元,用于对所述联邦分类模型M2为区分输入数据归属方得到的各所述本地性能评价指标的值进行聚合计算,得到一全局评价指标值;
数据分布一致性判断单元,连接所述数值聚合计算单元,用于根据所述全 局评价指标值,判断出各所述联邦学习参与方分别提供的所述原始数据的数据分布是否一致。
本发明的有益效果是:
1、实现了对各联邦学习参与方提供的原始数据的数据分布是否一致的有效判断;
2、能够将各联邦学习参与方提供的原始数据合理划分为训练集和验证集,所划分的验证集与测试集的数据分布相同或近似,有利于提高联邦学习模型的模型性能。
为了更清楚地说明本发明实施例的技术方案,下面将对本发明实施例中所需要使用的附图作简单地介绍。显而易见地,下面所描述的附图仅仅是本发明的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。
图1是本发明一实施例提供的在联邦学习场景下的数据集划分方法的步骤图;
图2是本发明一实施例提供的在联邦学习场景下的数据集划分方法划分数据集的原理图;
图3是本发明判断各联邦学习参与方提供的原始数据的数据分布是否一致的方法步骤图;
图4是本发明判断各联邦学习参与方提供的原始数据的数据分布是否一致的原理图;
图5是本发明一实施例提供的在联邦学习场景下的数据集划分系统的系统 结构示意图;
图6是所述数据集划分系统中的所述数据分布一致性判断模块的内部结构示意图。
下面结合附图并通过具体实施方式来进一步说明本发明的技术方案。
其中,附图仅用于示例性说明,表示的仅是示意图,而非实物图,不能理解为对本专利的限制;为了更好地说明本发明的实施例,附图某些部件会有省略、放大或缩小,并不代表实际产品的尺寸;对本领域技术人员来说,附图中某些公知结构及其说明可能省略是可以理解的。
本发明实施例的附图中相同或相似的标号对应相同或相似的部件;在本发明的描述中,需要理解的是,若出现术语“上”、“下”、“左”、“右”、“内”、“外”等指示的方位或位置关系为基于附图所示的方位或位置关系,仅是为了便于描述本发明和简化描述,而不是指示或暗示所指的装置或元件必须具有特定的方位、以特定的方位构造和操作,因此附图中描述位置关系的用语仅用于示例性说明,不能理解为对本专利的限制,对于本领域的普通技术人员而言,可以根据具体情况理解上述术语的具体含义。
在本发明的描述中,除非另有明确的规定和限定,若出现术语“连接”等指示部件之间的连接关系,该术语应做广义理解,例如,可以是固定连接,也可以是可拆卸连接,或成一体;可以是机械连接,也可以是电连接;可以是直接相连,也可以通过中间媒介间接相连,可以是两个部件内部的连通或两个部件的相互作用关系。对于本领域的普通技术人员而言,可以具体情况理解上述术语在本发明中的具体含义。
本发明一实施例提供的在联邦学习场景下的数据集划分方法,如图1所示,包括如下步骤:
步骤S1,判断各联邦学习参与方提供的原始数据的数据分布是否一致;
步骤S2,使用各联邦学习参与方提供的数据分布一致的原始数据以及模型测试数据训练、使用验证集优化得到最优的联邦分类模型M1;为了对原始数据的归属方以及对原始数据和模型测试数据进行区分,本发明在训练联邦分类模型M1之前,首先为各联邦学习参与方提供的原始数据赋予相对应的数据标签,并为模型测试数据赋予相对应的数据标签。比如联邦学习参与方A提供的原始数据记为A-data,联邦学习参与方B提供的原始数据记为B-data,可为A-data和B-data赋予相同的数据标签0,分别记为A-data|0和B-data|0;模型测试数据记为test,可为test数据赋予标签1,记为test|1。
步骤S3,将归属于各联邦学习参与方的原始数据输入到联邦分类模型M1中,联邦分类模型M1输出模型输入数据为模型测试数据的概率;比如将联邦学习参与方A提供的原始数据A-data输入到模型M1中,模型M1将输出A-data为模型测试数据的概率;
步骤S4,按照预测概率由高到低选取指定数量的模型输入数据划分为数据归属的联邦学习参与方提供的用于验证模型性能的验证集,剩余的模型输入数据划分为数据归属的联邦学习参与方提供的用于训练模型的训练集。比如步骤S3输入到模型M1中的数据为A-data时,按照预测概率由高到低从联邦学习参与方A提供的原始数据A-data中选取指定数量的数据划分为联邦学习参与方A提供的用于验证模型性能的验证集(比如根据预测概率由高到低选取A-data中的20%作为验证集),剩余80%的模型输入数据划分为联邦学习参与方A提供的用于训练模型的训练集。
以下以参与联邦学习的对象为联邦学习参与方A和B两个为例,并结合图2,对本发明提供的数据集划分方法进行具体阐述:
如图2所示,将联邦学习参与方A提供的原始数据记为A-data,将联邦学习参与方B提供的原始数据记为B-data,首先为联邦学习参与方A和B提供的原始数据增加数据标签0,记为A-data|0和B-data|0,为模型测试数据增加标签1,记为test-1,然后使用A-data|0、B-data|0和test|1进行联邦学习,训练、优化形成最优的联邦学习分类模型M1,模型M1用于区分输入数据是否为模型测试数据或是否可作为模型测试数据。
模型M1训练完成后,将数据A-data、B-data输入到模型M1中进行打分预测,输出输入数据属于或可作为模型测试数据的概率,概率值在0到1之间。将输入数据按照模型输出的概率值由高到低进行排序,分值越高,说明该输入数据与模型测试数据的数据分布越接近。最后按照模型训练具体需求,在输入数据中选取合适数量的数据样本作为模型训练的验证集,比如按照概率值从高到低选取输入数据的20%作为数据归属的联邦学习参与方提供的验证集,剩余的80%作为数据归属的该联邦学习参与方提供的训练集,这样便将联邦学习参与方A提供的原始数据A-data拆分为训练集A-train和验证集A-valid,将联邦学习参与方B提供的原始数据B-data拆分为训练集A-train和验证集B-valid。
步骤S1中,判断各联邦学习参与方提供的原始数据的数据分布是否一致的方法如图3所示,具体包括:
步骤S11,将各联邦学习参与方提供的原始数据划分为与原始数据的数据分布一致的训练集、验证集和测试集;比如联邦学习参与方A提供的原始数据记为A-data,将原始数据A-data划分为与原始数据的数据分布一致的训练集A-train、验证集A-valid和测试集A-test,联邦学习参与方B提供的原始数据记 为B-data,将原始数据B-data划分为训练集B-train、验证集B-valid和测试集B-test;
步骤S12,为划分的分属于各联邦学习参与方的训练集赋予相对应的数据标签,比如为联邦学习参与方A提供的训练集A-train增加数据标签0,记为A-train|0,为联邦学习参与方B提供的训练集B-train增加数据标签1,记为B-train|1;
步骤S13,使用带有数据标签的分属于各联邦学习参与方的训练集,比如当联邦学习参与方只有A和B时,使用训练集A-train|0和B-train|1训练形成联邦分类模型M2;然后将分属于各联邦学习参与方的验证集,比如由联邦学习参与方A和B分别提供的测试集A-valid、B-valid输入到模型M2中,以评估模型M2区分输入数据归属方的模型性能,得到最优的联邦分类模型M2。
步骤S14,将测试集A-test,B-test输入到模型M2中,得到模型M2区分各归属方输入数据的若干个本地性能评价指标;
步骤S15,对联邦分类模型M2为区分输入数据归属方得到的各本地性能评价指标的值进行聚合计算,得到一全局评价指标值,并根据该全局评价指标值,判断出各联邦学习参与方分别提供的原始数据的数据分布是否一致。
以下还是以参与联邦学习的对象仅有联邦学习参与方A和B两个为例,并结合图4,对判断各联邦学习参与方提供的原始数据的数据分布是否一致的方法进行具体说明:
现有联邦学习参与方A和B,A、B两方提供的原始数据的数据分布不一致。首先将A方提供的原始数据划分为与原始数据分布一致的训练集、验证集和测试集,分别记为A-train、A-valid和A-test。同样地,将B方提供的原始数据划分为与原始数据一致的训练集、验证集和测试集,分别记为B-train、B-valid和 B-test,并为A-train、A-valid增加数据标签0,记为A-train|0、A-valid|0,为B-train、B-valid增加数据标签1,记为B-train|1、B-valid|1,然后以A-train|0和B-train|1为训练样本训练、以A-valid|0、B-valid|1为验证集优化得到最优的联邦分类模型M2,模型M2用于区分输入数据归属于A方还是B方。
模型M2训练完成后,将A-test、B-test输入到模型M2中,得到评价A-test是否归属于A方的评价指标AUC(AUC(Area Under Cruve)被定义为ROC曲线下与坐标轴围合形成的面积,是用于评价模型预测性能的一种常用的本地性能评价指标。本发明中,AUC评价指标用于评价输入数据为对应归属方提供的概率),以及得到评价B-test是否归属于B方的评价指标AUC。然后对两个AUC指标的指标值进行聚合,并根据聚合结果比如通过阈值判断方法判断出A方和B方提供的原始数据的数据分布是否一致。该阈值判断方法具体为:
判断各性能评价指标AUC的数值累加结果的值与0.5的差值是否小于阈值δ并大于-δ,
若是,则判定各联邦学习参与方提供的原始数据的数据分布一致;
若否,则判定各联邦学习参与方提供的原始数据的数据分布不一致。
更加具体地,当各性能评价指标AUC的数值累加结果的值与0.5的差值大于或等于阈值δ时,则表示各联邦学习参与方提供的原始数据的数据分布不一致。当各性能评价指标AUC的数值累加结果的值与0.5的差值小于或等于-δ时,表示联邦分类模型M2的模型性能不合格,需要对模型M2进行重新训练。
阈值δ根据经验总结而得,当参与联邦学习的联邦学习参与方只有两个时,阈值δ优选等于0.2。
本发明还提供了一种在联邦学习场景下的数据集划分系统,可实现上述的数据集划分方法,如图5所示,该数据集划分系统包括:
数据分布一致性判断模块,用于判断各联邦学习参与方提供的原始数据的数据分布是否一致;
数据标签赋予模块,用于为数据分布一致的各联邦学习参与方提供原始数据赋予相对应的数据标签并存储,比如为联邦学习参与方A提供的原始数据A-data赋予标签0,并为模型测试数据赋予相对应的数据标签并存储,比如为模型测试数据test赋予数据标签1;
数据获取模块,连接数据标签赋予模块,用于获取经标签赋予后的原始数据作为模型训练样本,获取模型测试数据作为模型验证样本;
M1联邦分类模型训练模块,连接数据获取模块,用于使用获取的各联邦学习参与方提供的原始数据和模型测试数据训练、使用验证集优化得到最优的形成联邦分类模型M1;
M1模型性能测试模块,分别连接数据获取模块和M1联邦分类模型训练模块,用于将获取的归属于各联邦学习参与方的原始数据输入到联邦分类模型M1中,联邦分类模型M1输出模型输入数据为模型测试数据或可作为模型测试数据的概率;
验证集选定模块,连接M1模型性能测试模块,用于按照预测概率由高到低选取指定数量的模型输入数据作为数据归属的联邦学习参与方提供的用于验证模型性能的验证集,剩余的模型输入数据作为数据归属的该联邦学习参与方提供的用于训练模型的训练集。验证集和训练集选定方法在上述的数据集划分方法中作了具体阐述,在此不再赘述。
如图6所示,数据分布一致性判断模块中具体包括:
数据划分单元,用于将各联邦学习参与方提供的原始数据划分为与原始数据的数据分布一致的训练集、验证集和测试集;
数据标签赋予单元,连接数据划分单元,用于为划分的分属于各联邦学习参与方的训练集、验证集赋予相对应的数据标签,并为模型测试数据赋予相对应的数据标签;数据标签赋予方法在上述的数据集划分方法中作为具体阐述,在此不再赘述;
M2联邦分类模型训练单元,连接数据标签赋予单元,用于使用带有数据标签的分属于各联邦学习参与方的训练集训练、使用验证集优化得到最优的联邦分类模型M2;
M2模型性能测试单元,分别连接数据划分单元和M2联邦分类模型训练单元,用于将分属于各联邦学习参与方的测试集输入到联邦分类模型M2中,得到联邦分类模型M2区分各归属方输入数据的若干个本地性能评价指标;
数值聚合计算单元,连接M2模型性能测试单元,用于对联邦分类模型M2为区分输入数据归属方得到的各本地性能评价指标的指标值进行聚合计算,得到一全局评价指标值;
数据分布一致性判断单元,连接数值聚合计算单元,用于根据全局评价指标值,判断出各联邦学习参与方分别提供的原始数据的数据分布是否一致。
需要声明的是,上述具体实施方式仅仅为本发明的较佳实施例及所运用技术原理。本领域技术人员应该明白,还可以对本发明做各种修改、等同替换、变化等等。但是,这些变换只要未背离本发明的精神,都应在本发明的保护范围之内。另外,本申请说明书和权利要求书所使用的一些术语并不是限制,仅仅是为了便于描述。
Claims (4)
- 一种在联邦学习场景下的数据集划分方法,其特征在于,包括如下步骤:步骤S1,判断各联邦学习参与方提供的原始数据的数据分布是否一致;步骤S2,使用各所述联邦学习参与方提供的数据分布一致的所述原始数据以及模型测试数据训练、使用验证集优化得到最优的联邦分类模型M1;步骤S3,将归属于各所述联邦学习参与方的所述原始数据输入到所述联邦分类模型M1中,所述联邦分类模型M1输出模型输入数据为所述模型测试数据的概率;步骤S4,按照预测概率由高到低选取指定数量的所述模型输入数据划分为数据归属的所述联邦学习参与方提供的用于验证模型性能的验证集,剩余的所述模型输入数据划分为数据归属的所述联邦学习参与方提供的用于训练模型的训练集。
- 根据权利要求1所述的在联邦学习场景下的数据划分方法,其特征在于,所述步骤S1中,判断各所述联邦学习参与方提供的所述原始数据的数据分布是否一致的方法具体包括:步骤S11,将所述联邦学习参与方提供的所述原始数据划分为与所述原始数据的数据分布一致的训练集、验证集和测试集;步骤S12,为划分的分属于各所述联邦学习参与方的训练集、验证集赋予相对应的数据标签;步骤S13,使用带有数据标签的分属于各所述联邦学习参与方的所述训练集训练、使用验证集优化得到最优的联邦分类模型M2;步骤S14,将分属于各所述联邦学习参与方的所述测试集输入到所述联邦分类模型M2中,得到所述联邦分类模型M2区分各归属方输入数据的若干个本地性能评价指标;步骤S15,对所述联邦分类模型M2为区分输入数据归属方得到的各所述本地性能评价指标的值进行聚合计算,得到一全局评价指标值,并根据所述全局评价指标值判断出各所述联邦学习参与方分别提供的所述原始数据的数据分布是否一致。
- 一种在联邦学习场景下的数据集划分系统,可实现如权利要求1或2任意一项所述的数据集划分方法,其特征在于,所述数据集划分系统包括:数据分布一致性判断模块,用于判断各联邦学习参与方提供的原始数据的数据分布是否一致;数据标签赋予模块,用于为数据分布一致的各所述联邦学习参与方提供的所述原始数据赋予相对应的数据标签并存储,并为模型测试数据赋予相对应的数据标签并存储;数据获取模块,连接所述数据标签赋予模块,用于获取经标签赋予后的的所述原始数据作为模型训练样本,获取所述模型测试数据作为模型验证样本;M1联邦分类模型训练模块,连接所述数据获取模块,用于使用获取的各所述联邦学习参与方提供的所述原始数据和所述模型测试数据训练,使用验证集优化得到最优的联邦分类模型M1;M1模型性能测试模块,分别连接所述数据获取模块和所述M1联邦分类模型训练模块,用于将获取的归属于各所述联邦学习参与方的所述原始数据输入到所述联邦分类模型M1中,所述联邦分类模型M1输出模型输入数据为所述模型测试数据的概率;验证集选定模块,连接所述M1模型性能测试模块和所述数据获取模块,用于按照预测概率由高到低选取指定数量的所述模型输入数据作为数据归属的所述联邦学习参与方提供的用于验证模型性能的验证集,剩余的所述模型输入 数据作为数据归属的所述联邦学习参与方提供的用于训练模型的训练集。
- 根据权利要求3所述的在联邦学习场景下的数据集划分系统,其特征在于,所述数据分布一致性判断模块中具体包括:数据划分单元,用于将各所述联邦学习参与方提供的所述原始数据划分为与所述原始数据的数据分布一致的训练集、验证集和测试集;数据标签赋予单元,连接所述数据划分单元,用于为划分的分属于各所述联邦学习参与方的训练集、验证集赋予相对应的数据标签,并为所述模型测试数据赋予相对应的数据标签;M2联邦分类模型训练单元,连接所述数据标签赋予单元,用于使用带有数据标签的分属于各所述联邦学习参与方的所述训练集训练、使用验证集优化得到最优的联邦分类模型M2;M2模型性能测试单元,分别连接所述数据划分单元和所述M2联邦分类模型训练单元,用于将分属于各所述联邦学习参与方的所述测试集输入到所述联邦分类模型M2中,得到所述联邦分类模型M2区分各归属方输入数据的若干个本地性能评价指标;数值聚合计算单元,连接所述M2模型性能测试单元,用于对所述联邦分类模型M2为区分输入数据归属方得到的各所述本地性能评价指标的值进行聚合计算,得到一全局评价指标值;数据分布一致性判断单元,连接所述数值聚合计算单元,用于根据所述全局评价指标值,判断出各所述联邦学习参与方分别提供的所述原始数据的数据分布是否一致。
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202011455586.2A CN112686388A (zh) | 2020-12-10 | 2020-12-10 | 一种在联邦学习场景下的数据集划分方法及系统 |
CN202011455586.2 | 2020-12-10 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2022121032A1 true WO2022121032A1 (zh) | 2022-06-16 |
Family
ID=75448904
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2020/140882 WO2022121032A1 (zh) | 2020-12-10 | 2020-12-29 | 一种在联邦学习场景下的数据集划分方法及系统 |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN112686388A (zh) |
WO (1) | WO2022121032A1 (zh) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115168210A (zh) * | 2022-07-13 | 2022-10-11 | 浙江大学 | 一种联邦学习中在黑盒场景下基于对抗样本的鲁棒水印遗忘验证方法 |
CN116307066A (zh) * | 2023-01-09 | 2023-06-23 | 中国科学院空天信息创新研究院 | 一种低碳园区态势综合分析方法与系统 |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113591486B (zh) * | 2021-07-29 | 2022-08-23 | 浙江大学 | 一种联邦学习中基于语义数据损失的遗忘验证方法 |
CN115310130B (zh) * | 2022-08-15 | 2023-11-17 | 南京航空航天大学 | 一种基于联邦学习的多站点医学数据分析方法及系统 |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110442457A (zh) * | 2019-08-12 | 2019-11-12 | 北京大学深圳研究生院 | 基于联邦学习的模型训练方法、装置及服务器 |
CN110852396A (zh) * | 2019-11-15 | 2020-02-28 | 苏州中科华影健康科技有限公司 | 一种宫颈图像的样本数据处理方法 |
CN111582313A (zh) * | 2020-04-09 | 2020-08-25 | 上海淇毓信息科技有限公司 | 样本数据生成方法、装置及电子设备 |
CN111582315A (zh) * | 2020-04-09 | 2020-08-25 | 上海淇毓信息科技有限公司 | 样本数据处理方法、装置及电子设备 |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR102052624B1 (ko) * | 2018-11-09 | 2019-12-05 | 주식회사 루닛 | 기계 학습 방법 및 장치 |
CN111275491B (zh) * | 2020-01-21 | 2023-12-26 | 深圳前海微众银行股份有限公司 | 一种数据处理方法及装置 |
CN111652379B (zh) * | 2020-05-29 | 2024-04-16 | 京东城市(北京)数字科技有限公司 | 模型管理方法、装置、电子设备及存储介质 |
CN111898764A (zh) * | 2020-06-23 | 2020-11-06 | 华为技术有限公司 | 联邦学习的方法、装置和芯片 |
CN111898768A (zh) * | 2020-08-06 | 2020-11-06 | 深圳前海微众银行股份有限公司 | 数据处理方法、装置、设备及介质 |
-
2020
- 2020-12-10 CN CN202011455586.2A patent/CN112686388A/zh active Pending
- 2020-12-29 WO PCT/CN2020/140882 patent/WO2022121032A1/zh active Application Filing
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110442457A (zh) * | 2019-08-12 | 2019-11-12 | 北京大学深圳研究生院 | 基于联邦学习的模型训练方法、装置及服务器 |
CN110852396A (zh) * | 2019-11-15 | 2020-02-28 | 苏州中科华影健康科技有限公司 | 一种宫颈图像的样本数据处理方法 |
CN111582313A (zh) * | 2020-04-09 | 2020-08-25 | 上海淇毓信息科技有限公司 | 样本数据生成方法、装置及电子设备 |
CN111582315A (zh) * | 2020-04-09 | 2020-08-25 | 上海淇毓信息科技有限公司 | 样本数据处理方法、装置及电子设备 |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115168210A (zh) * | 2022-07-13 | 2022-10-11 | 浙江大学 | 一种联邦学习中在黑盒场景下基于对抗样本的鲁棒水印遗忘验证方法 |
CN116307066A (zh) * | 2023-01-09 | 2023-06-23 | 中国科学院空天信息创新研究院 | 一种低碳园区态势综合分析方法与系统 |
Also Published As
Publication number | Publication date |
---|---|
CN112686388A (zh) | 2021-04-20 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2022121032A1 (zh) | 一种在联邦学习场景下的数据集划分方法及系统 | |
WO2017133492A1 (zh) | 一种风险评估方法和系统 | |
WO2017143921A1 (zh) | 一种多重抽样模型训练方法及装置 | |
CN108090902A (zh) | 一种基于多尺度生成对抗网络的无参考图像质量客观评价方法 | |
CN110442516B (zh) | 信息处理方法、设备及计算机可读存储介质 | |
CN102609406B (zh) | 学习装置、判断装置、学习方法和判断方法 | |
CN109598931B (zh) | 基于交通安全风险的群体划分与差异性分析方法及系统 | |
CN111815169B (zh) | 业务审批参数配置方法及装置 | |
CN108022146A (zh) | 征信数据的特征项处理方法、装置、计算机设备 | |
WO2019242627A1 (zh) | 一种数据处理方法及其装置 | |
Samawi et al. | Notes on the overlap measure as an alternative to the Youden index: How are they related? | |
Colak et al. | Comparison of semiparametric, parametric, and nonparametric ROC analysis for continuous diagnostic tests using a simulation study and acute coronary syndrome data | |
Cao et al. | Specification of a performance indicator using the evidential-reasoning approach | |
CN107392217B (zh) | 计算机实现的信息处理方法及装置 | |
Picho et al. | 7 deadly sins in educational research | |
TWI677830B (zh) | 模型中關鍵變量的探測方法及裝置 | |
CN115587670A (zh) | 一种基于指标图谱的产品质量诊断方法及装置 | |
WO2023029065A1 (zh) | 数据集质量评估方法、装置、计算机设备及存储介质 | |
CN103970651A (zh) | 基于组件安全属性的软件体系结构安全性评估方法 | |
CN112884480A (zh) | 异常交易识别模型的构造方法、装置、计算机设备和介质 | |
US11367311B2 (en) | Face recognition method and apparatus, server, and storage medium | |
CN105303194A (zh) | 一种电网指标体系建立方法、装置以及计算设备 | |
Borges et al. | Towards two-tier citizen sensing | |
CN114328174A (zh) | 一种基于对抗学习的多视图软件缺陷预测方法及系统 | |
CN115310606A (zh) | 基于数据集敏感属性重构的深度学习模型去偏方法及装置 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20964961 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20964961 Country of ref document: EP Kind code of ref document: A1 |