WO2024036880A1 - 一种数据处理方法、装置、电子设备及存储介质 - Google Patents

一种数据处理方法、装置、电子设备及存储介质 Download PDF

Info

Publication number
WO2024036880A1
WO2024036880A1 PCT/CN2023/071564 CN2023071564W WO2024036880A1 WO 2024036880 A1 WO2024036880 A1 WO 2024036880A1 CN 2023071564 W CN2023071564 W CN 2023071564W WO 2024036880 A1 WO2024036880 A1 WO 2024036880A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
label
residual
model parameter
data sample
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2023/071564
Other languages
English (en)
French (fr)
Inventor
张远健
周雍恺
陈钟正
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Unionpay Co Ltd
Original Assignee
China Unionpay Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Unionpay Co Ltd filed Critical China Unionpay Co Ltd
Priority to US19/104,157 priority Critical patent/US12572694B2/en
Publication of WO2024036880A1 publication Critical patent/WO2024036880A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
    • G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
    • G06F21/6245—Protecting personal data, e.g. for financial or medical purposes

Definitions

  • the present disclosure relates to the field of data security technology, specifically, to a data processing method, device, electronic equipment and storage medium.
  • the multi-party data can come from different organizations.
  • transaction data can be dispersed in financial structures, and medical diagnosis records can come from medical institutions.
  • medical diagnosis records can come from medical institutions.
  • it can also come from different industries.
  • the same transaction data can It can come from e-commerce or finance.
  • Multi-party data elements are subject to compliance constraints from regulatory authorities during the circulation process, making it unfeasible to centrally collect detailed data and then train models for business operations, which constitutes a data barrier.
  • federated learning methods can be used to allow each participant to obtain a global model without sharing their private data.
  • the private data here can be label information about the target user, for example, whether the target user is a high-net-worth user. . While reasonably exploring the application value of data, it is also necessary to prevent data abuse and protect private data.
  • model-related information can be exchanged in an encrypted manner to achieve collaborative optimization of federated models.
  • the tagged party can make a differential privacy in the form of addition before sending model-related information, so that after the differential privacy conditions are met, the The labeled party sends it to the unlabeled party in the form of homomorphic encryption. Due to the existence of noise, the unlabeled party cannot reconstruct the real residuals based on the mastered model-related parameters, thereby achieving the purpose of protecting private data.
  • Embodiments of the present disclosure at least provide a data processing method, device, electronic device and storage medium.
  • the data sender encrypts the private data of the label through residual decomposition, and the data receiver performs encryption based on the model parameter correction obtained by encryption. Model parameters are reconstructed, and the reconstructed model has higher accuracy.
  • embodiments of the present disclosure provide a data processing method, including:
  • each label residual value is used to characterize the degree of deviation between the real label of the corresponding data sample and the label predicted by the target model;
  • the model parameter correction amount is sent to the data receiver, so that the data receiver reconstructs the model parameter information of the target model according to the model parameter correction amount.
  • performing residual decomposition on each data sample based on each label residual value to obtain a model parameter correction amount corresponding to each data sample includes:
  • the model parameter correction amount corresponding to each data sample is determined.
  • the data sample corresponding to the target label residual value is determined as the target data sample.
  • the label residual value of the sorted target data sample is changed to obtain the Describes the residual change information corresponding to each data sample, including:
  • the residual change values corresponding to each of the data samples are collected to determine the residual change vector; wherein the residual change vector is used to characterize whether the label residual value of each data sample changes.
  • determining the model parameter correction amount corresponding to each data sample based on the residual change information corresponding to each data sample and the real label of each data sample includes:
  • the model parameter correction amount corresponding to each data sample is determined.
  • the method further includes:
  • sending the model parameter information to the data receiver includes:
  • the sending of the model parameter correction amount to the data receiver includes:
  • the model parameter correction amount is sent to the data receiver.
  • the present disclosure also provides a data processing method, including:
  • model parameter correction amount corresponding to each data sample sent by the data sender wherein the model parameter correction amount is determined by residual decomposition based on the label residual value determined for each data sample; each of the label residual values The degree of deviation between the true label used to characterize the corresponding data sample and the label predicted by the target model;
  • Model parameter information of the target model is reconstructed according to the model parameter correction amount.
  • reconstructing the model parameter information of the target model according to the model parameter correction amount includes:
  • model parameter information of the target model is determined.
  • determining the model parameter information of the target model based on the data feature information and the model parameter correction amount includes: :
  • Model parameter information of the target model is determined based on the third operator, the fourth operator and the model parameter correction amount.
  • the present disclosure also provides a data processing device, including:
  • the acquisition module is used to obtain the label residual value determined for each data sample; each label residual value is used to characterize the degree of deviation between the real label of the corresponding data sample and the label predicted by the target model;
  • a decomposition module configured to perform residual decomposition on each of the data samples based on each of the label residual values, and obtain the model parameter correction amount corresponding to each of the data samples;
  • a sending module configured to send the model parameter correction amount to the data receiver, so that the data receiver reconstructs the model parameter information of the target model according to the model parameter correction amount.
  • the present disclosure also provides a data processing device, including:
  • the receiving module is used to receive the model parameter correction amount corresponding to each data sample sent by the data sender; wherein the model parameter correction amount is determined by residual decomposition based on the label residual value determined for each data sample; each The label residual value is used to characterize the degree of deviation between the true label of the corresponding data sample and the label predicted by the target model;
  • a reconstruction module configured to reconstruct the model parameter information of the target model according to the model parameter correction amount.
  • the present disclosure also provides an electronic device, including: a processor, a memory, and a bus.
  • the memory stores machine-readable instructions executable by the processor.
  • the processor Communicates with the memory through a bus, and when the machine-readable instructions are executed by the processor, they execute as described in any one of the first aspect and its various implementations, the second aspect and its various implementations. Data processing methods.
  • the present disclosure also provides a computer-readable storage medium.
  • a computer program is stored on the computer-readable storage medium. The computer program is executed when the processor is run as in the first aspect and its various implementations, and the third aspect.
  • the residual decomposition of each data sample can be performed based on each of the label residual values.
  • the model parameter corrections corresponding to each data sample obtained by residual decomposition can be sent to the data receiver, and the data receiver can reconstruct the model parameter information based on the model parameter corrections.
  • the data sender that is, the labeled party
  • the data receiver that is, the unlabeled party
  • the data sender can effectively perturb the residuals during the residual decomposition process, so that the data receiver (that is, the unlabeled party) can only construct a model with incorrect label information. , which allows the data sender to protect the real labels it owns.
  • the labeled party can pass the necessary correction information (i.e., the model parameter correction amount) to the unlabeled party, so that the unlabeled party can reconstruct accurately according to the corrected information.
  • the model parameter information further ensures the training performance of the model.
  • Figure 1 shows a flow chart of a data processing method provided by an embodiment of the present disclosure
  • Figure 2 shows a flow chart of a specific method for determining residual change information in the data processing method provided by the embodiment of the present disclosure
  • Figure 3 shows a sequence flow chart of a data processing method provided by an embodiment of the present disclosure
  • Figure 4 shows a schematic application diagram of a data processing method provided by an embodiment of the present disclosure
  • Figure 5 shows a flow chart of another data processing method provided by an embodiment of the present disclosure
  • Figure 6 shows a schematic diagram of a data processing device provided by an embodiment of the present disclosure
  • Figure 7 shows a schematic diagram of another data processing device provided by an embodiment of the present disclosure.
  • FIG. 8 shows a schematic diagram of an electronic device provided by an embodiment of the present disclosure.
  • A/B can mean A or B;
  • and/or in this article is just an association relationship describing related objects, indicating that three relationships can exist, For example, A and/or B can mean: A exists alone, A and B exist simultaneously, and B exists alone.
  • first, second, etc. are used for descriptive purposes only and shall not be understood as indicating or implying relative importance or implicitly indicating the quantity of indicated technical features. Thus, features qualified by “first,” “second,” etc. may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present disclosure, unless otherwise specified, "plurality" means two or more.
  • model-related information can be exchanged in an encrypted manner to achieve collaborative optimization of federated models.
  • the tagged party can make a differential privacy in the form of addition before sending model-related information, so that after the differential privacy conditions are met, the The labeled party sends it to the unlabeled party in the form of homomorphic encryption.
  • the differential privacy that is, the process of adding noise to the residual r i
  • the differential privacy satisfies the condition noise ⁇ Lap(2 ⁇ -1 ) (that is, the noise comes from the Laplacian random variable generated with a standard deviation of 2 ⁇ -1 ). Due to the existence of noise, the unlabeled party cannot reconstruct the real residual r i based on the mastered model-related parameters, thereby achieving the purpose of protecting private data.
  • Another encryption method is localized differential privacy random response. The idea is to randomly perturb the composition of the batch so that the dimensions of the interactive residuals remain unchanged in form, but the actual rank is smaller than the number of samples in the batch, so the resulting residuals are not unique.
  • the labeled party selects a subset C from the labeled party's samples based on the batch size agreed upon by both parties, generates a random response vector m about the label y ⁇ 0,1 ⁇ based on the category distribution of the label, and uses the difference Privacy processing gets m rr sent to the unlabeled party.
  • the unlabeled party builds a local model based on the feature
  • the constructed residual matrix is homomorphically encrypted and sent to the unlabeled party.
  • the composition of the sent content D is formally expressed as:
  • ⁇ ( ⁇ ) is the sigmoid function
  • ⁇ > is homomorphic encryption.
  • This method is called a label protection strategy that mixes differential privacy and homomorphic encryption, and can achieve ⁇ differential privacy when the following conditions are met:
  • q is the proportion of m marked as 1.
  • the present disclosure provides at least one data processing solution.
  • the labeled party encrypts the private data of the label through residual decomposition, and the unlabeled party encrypts the private data of the label through residual decomposition.
  • Model parameters are reconstructed based on the model parameter corrections obtained by encryption, and the reconstructed model has higher accuracy.
  • both the tagged party and the untagged party need to perform local calculations and collaboratively modify information interactively, and the computational overhead does not increase significantly.
  • the labeled side here corresponds to the data sender, and the unlabeled side corresponds to the data receiver. Residual decomposition is performed on the labeled side, and parameter reconstruction is completed on the unlabeled side.
  • the data sender here refers to the initiator with real labels and its processing equipment used to perform including residual decomposition operations
  • the data receiver here refers to the initiator that has model input features but does not A responder with a real label and its processing device, including performing parameter reconstruction operations.
  • the execution subject of the data processing method provided by the embodiment of the present disclosure is generally an electronic device with certain computing capabilities.
  • the electronic device Devices include, for example, terminal devices or other processing devices.
  • the terminal devices may be user equipment (User Equipment, UE), mobile devices, user terminals, cellular phones, personal digital assistants (Personal Digital Assistant, PDA), handheld devices, etc.
  • the data processing method can be implemented by the processor calling computer-readable instructions stored in the memory.
  • FIG. 1 there is a flow chart of a data processing method provided by an embodiment of the present disclosure.
  • the method is mainly executed on the data sender and specifically includes steps S101 to S103, in which:
  • S101 Obtain the label residual value determined for each data sample; each label residual value is used to characterize the degree of deviation between the real label of the corresponding data sample and the label predicted by the target model;
  • S102 Perform residual decomposition on each data sample based on each label residual value to obtain the model parameter correction amount corresponding to each data sample;
  • the data processing method in the embodiment of the present disclosure can be mainly applied in the field of federated learning.
  • the federated learning here can correspond to the fusion learning method of multi-party data.
  • it can be transaction data derived from financial structures and medical diagnosis records of medical institutions. The resulting fused data.
  • longitudinal logistic regression methods are mostly used to build joint models. Due to the characteristic constraints of the longitudinal logistic regression method itself, one party's private data is likely to be inferred by the other party. For example, label data generated within a financial institution regarding whether a user is a high-net-worth user is likely to be leaked to medical institutions with the sharing of model parameter information. In this case, it will bring certain insecurity factors to user privacy. .
  • embodiments of this disclosure provide a collaborative data processing solution, which mainly corresponds to two stages, one is the model training stage, and the other is Model revision stage.
  • the process of residual decomposition is mainly performed on the label side. After decomposition, only a part of the residual-related confusion information is exchanged with the unlabeled party under the protection of homomorphic encryption, while the other part of the residual retained information is held solely by the labeled party and does not participate in the model training process, so that even if the unlabeled party follows
  • the relevant label leakage attack method (which satisfies the full rank condition of the column) solves the residual, and because the correspondence between the residual and the real label no longer has a one-to-one correspondence, the accuracy of the label prediction of the labeled party is different from that of random guessing.
  • the classification results are equivalent, and the true label of the labeled party cannot be inferred based on the sign of the residual obtained by known label leakage attacks, thus ensuring the security of private data.
  • the process of parameter reconstruction is mainly performed on the unlabeled side.
  • the model training of the labeled party After the model training of the labeled party has converged, it is initiated by the labeled party and responded by the unlabeled party. It only requires an additional round of interaction between the labeled party and the unlabeled party about the weight information of the unlabeled party to complete each of the unlabeled parties. Correction of feature weights. Since the additional information provided by the labeled party cannot be obtained through the model training stage, and this information is the whole of the residual retention implemented by the real label and the protected label, the unlabeled party cannot reconstruct a high-precision model without The label information of the tagged party is obtained during the model modification phase, thereby further ensuring the security of private data.
  • the data sender in the embodiment of the present disclosure after obtaining the label residual value determined for each data sample, can perform residual decomposition on each data sample based on each label residual value to obtain a model corresponding to each data sample. Parameter correction amount, and finally send the model parameter correction amount to the data receiver.
  • the model parameter correction here is the additional information provided by the unlabeled side in the labeled direction. This additional information is determined after decomposing the residuals.
  • the labeled side randomly performs residual decomposition locally based on the degree of fitting of its own label by the unlabeled side to achieve the effect of random replacement of label information, so that the unlabeled side cannot infer the true value based on the residual information processed by the labeled side. Label.
  • the unlabeled party by perturbing the residual, the unlabeled party can only construct a model with wrong label information, and the error information contained is generated by the local decomposition of the residual information by the labeled party.
  • the unlabeled party There is no way to distinguish this; then, the labeled party passes the necessary correction information to the unlabeled party, so that the unlabeled party can correct the weight of the feature, but cannot obtain the label of the labeled party.
  • the unlabeled party corrects the biased feature weights with the cooperation of the labeled party and obtains correct model parameter information. Since this information consists of the label of the labeled party and the perturbation information on the label, where the perturbation information has nothing to do with the information known by the unlabeled party during the model training stage, the unlabeled party cannot obtain the label information of the labeled party.
  • model parameter corrections plays a key role in label-free reconstruction of model parameters
  • the following can focus on the process of determining the model parameter corrections, which can be achieved through the following steps:
  • Step 1 Sort the residual values of each label according to the order of data size to obtain the residual values of each sorted label;
  • Step 2 After selecting the target data sample that needs to be residually decomposed from each data sample, change the label residual value of the sorted target data sample to obtain the residual change information corresponding to each data sample;
  • Step 3 Determine the model parameter correction amount corresponding to each data sample based on the residual change information corresponding to each data sample and the true label of each data sample.
  • the residual values of each label can be sorted first, and the residual change information can be obtained, and then the model parameter correction amount can be determined based on the residual change information and the real label.
  • the above-mentioned residual change information is used to indicate whether the label residual value has changed.
  • the changed label residual value can be marked first, and the unchanged label residual value can be marked secondly, so that random replacement can be achieved.
  • the purpose of the real label is to consider that the relevant model parameter correction is obtained by combining the residual change information and the real label. That is, what the unlabeled party obtains is not the direct real label but the correction after label mapping. Based on this A correction amount and corresponding input features can be used to construct model parameter information.
  • the labeled party needs to perform a local operation and synthesis of the real label and the created label mapping before sending it to the unlabeled party.
  • the sorted label residual values can be grouped according to the preset grouping number to obtain the grouped label residual values, and then a preset number of target label residual values are selected from each grouped label residual value. Difference, and finally the data sample corresponding to the target label residual value is determined as the target data sample.
  • the grouping can be preset, for example, it can be divided into two groups, four groups, etc. There are no specific restrictions here.
  • the purpose is to make it impossible for the unlabeled party to obtain the label of the labeled party based on the similarity between the features. Additional information for the information.
  • random and quantitative extraction can be done to ensure the randomness of label replacement and further improve the security of the data.
  • the residual change information in the embodiment of the present disclosure may point to residual change vectors, each vector corresponding to the residual change of a data sample.
  • a data sample when it is determined that a data sample is not a target data sample, it can be determined that the data sample corresponds to the first residual change value; when it is determined that a data sample is a target data sample, and the label residual value of the data sample is greater than zero
  • when it is determined that a data sample is a target data sample and the label residual value of the data sample is less than zero it is determined that the data sample corresponds to the third residual change value .
  • the residual change values corresponding to each data sample can be collected to determine the residual change vector.
  • the model parameter correction amount can be determined, specifically through the following steps:
  • Step 1 Perform a dot multiplication operation on the residual change vector and the transposed result of the residual change vector to determine the first operator; and, after determining the label value corresponding to the real label of each data sample, determine the label value The corresponding second operator;
  • Step 2 Based on the first operator and the second operator, determine the model parameter correction amount corresponding to each data sample.
  • the model parameter correction amount can be determined based on the first operator and the second operator.
  • the model parameter correction amount can also be a multi-dimensional vector, each dimension pointing to the correction amount of a data sample, thereby achieving an overall correction for the target model.
  • the changed label residual value can also be determined based on the product operation between the residual change information corresponding to each data sample and the sorted label residual values, and then the changed label residual value can be determined based on the changed residual change information.
  • the label residual value determines the model parameter information sent to the data receiver, and sends the model parameter information to the data receiver.
  • the above-mentioned calculation process of the changed label residual value can be a step that only needs to be performed once before the model converges.
  • the labeled party can perform the above steps in the first round to change the residual value of the randomly selected sample. The effect is equivalent to changing the true label of the sample, which reduces the computational cost while ensuring the security of the label.
  • model parameter information here can be in the middle process of training, that is, when the model convergence condition has not been reached, the labeled direction is sent to the unlabeled party. Since the residual changes occur here, even if the unlabeled party receives it The model parameter information cannot be deduced from the real label.
  • the labeled party can send corrections to the unlabeled party so that the unlabeled party can reconstruct accurate model parameter information.
  • the relevant model convergence can be associated with the training batch.
  • it can be determined whether the model meets the convergence conditions. If it converges, follow the above steps. method to send the correction amount, otherwise continue to perform model training.
  • Party B only knows that Party A’s labels are in the binary 0/1 form. Other information about Party A’s labels is unknown to Party B, but labels of any category are of interest to Party B (that is, Party B wants to know anything about Party A’s labels). label information of a sample);
  • the labeled party and the unlabeled party agree in a certain secure way that both parties will use longitudinal logistic regression for joint modeling to determine the size
  • Party B knows the characteristics of the aligned sample ⁇ (xi ) ⁇
  • Party A knows the label ⁇ y i ⁇ of the aligned sample.
  • Each round is initiated by Party B, Party A responds, and Party B updates the local model. parameters end.
  • Party B uses its own model parameters and feature set Construct local model (Calculated as ),Then Send to Party A with homomorphic encryption (such as paillier algorithm).
  • Party A will modify the model parameters by Sent to Party B, Party B reconstructs the real model parameters w as follows:
  • X corresponds to the data feature vector
  • y corresponds to the value of the real label
  • S is the label mapping vector obtained by splicing S B in batches, and corresponds to the residual change information.
  • the above residual change information can correspond to the retained information of the sample residuals of this batch.
  • the retained information can be determined in the first round of training. Specifically, this can be achieved by following the following steps:
  • Party A sorts all the samples in the batch according to the residual r i B(t) (from small to large or from large to small).
  • Party A performs randomized residual decomposition on the residuals of this batch of samples as follows. Randomly select a number m ⁇ 2,4 ⁇ as the group size composed of samples in this batch.
  • every m samples form a group where the symbol Represents the smallest integer not less than
  • G j formed randomly select from within the group samples form the candidate set C B , and And for all samples in C B , determine the corresponding retention information in the following way
  • sort(i) is the order of the i-th sample in the batch sorted according to the residual ⁇ r i B.
  • Party B can rearrange the data samples in their original order.
  • the residual change vector S B can be determined. Each element in S B reflects whether the residual information of the sample changes in the first round of residual decomposition. When the residual changes, otherwise Right now:
  • the residual change vector and the transposed result of the residual change vector can be subjected to a dot multiplication operation to determine the first operator ss T ; and, after determining the true label of each data sample In the case of the corresponding label value, determine the second operator corresponding to the label value Then determine the model parameter correction amount
  • model parameter correction amount can be sent by Party A to Party B after the model training is completed.
  • pseudo-residual error sent by Party A to Party B can be calculated as follows:
  • Party B updates its own model parameters in the dense state
  • the calculation method is:
  • the embodiment of the present disclosure performs two steps: a residual decomposition (only the first round of joint modeling) and a model parameter reconstruction information transmission on the labeled side. At the same time, a weighting step is required on the unlabeled side.
  • a residual decomposition only the first round of joint modeling
  • a model parameter reconstruction information transmission on the labeled side.
  • a weighting step is required on the unlabeled side.
  • the unlabeled party can build a local model based on the model parameters given by the labeled party, and then the labeled party determines whether it is in the first round of training. If so, the labeled party can construct it according to the above steps. The residuals and label transformations are retained, and the label-free method can be updated based on pseudo-residuals. Then, when it is determined that model convergence is achieved, the labeled party can send additional information to the unlabeled party so that the unlabeled party can reconstruct the weights.
  • the embodiments of the present disclosure do not limit the specific implementation of the two modules of batch initialization and reaching convergence conditions, nor does the implementation of sample alignment of both parties before batch initialization.
  • the focus of this disclosed embodiment is that the labeled side implements residual decomposition for the unlabeled side and implements weight reconstruction in collaboration with the unlabeled side.
  • Other step calculations in the model training phase can be solved using the classic gradient descent method.
  • model training the residual retention information is held solely by the labeled party and does not participate in the model training process.
  • the unlabeled party cannot determine whether the labeled party has modified a specific example label based on known information.
  • this security is at the information theory level. Its effect is that the unlabeled party cannot obtain its own label by increasing computing power, and only needs to perform one round of local operations. The additional computing overhead is relative to the entire joint modeling overhead. can be ignored. Therefore, the unlabeled party cannot obtain the true label y of the labeled party based on the semi-honest attack.
  • the usability of the model can be analyzed from the perspective of weight reconstruction. Since the background of the problem can be limited to two categories, if the party with labels in the first round changes the residual value of the randomly selected sample, the effect is equivalent to changing the true label of the sample, from the positive class (negative class) to the negative class ( Positive type).
  • the significance of sorting is to prevent unlabeled parties from leaking more label information from labeled parties due to differences in random initialization parameters. Thus imposing S or There is no obvious information difference, and the final result does not obfuscate the information, including differential privacy, before sending it to the tagged party.
  • the weight can achieve better results in distinguishing sample categories.
  • FIG. 5 is a flow chart of a data processing method provided by an embodiment of the present disclosure.
  • the method is mainly executed on the data receiver, and specifically includes steps S501 to S502, in which:
  • S501 Receive the model parameter correction amount corresponding to each data sample sent by the data sender; wherein, the model parameter correction amount is determined by residual decomposition based on the label residual value determined for each data sample; each label residual value is used for Characterizes the degree of deviation between the true label of the corresponding data sample and the label predicted by the target model;
  • S502 Reconstruct the model parameter information of the target model according to the model parameter correction amount.
  • parameter reconstruction of the target model can be achieved based on the received model parameter correction. Since the model parameter correction here is determined based on the real label of the data sender and the corresponding residual change information, the corresponding label can be marked. Replacement situation, and the real label cannot be parsed based on the correction amount, so the model parameters can be accurately reconstructed while protecting the real label of the data sender.
  • Step 1 Perform a dot multiplication operation on the data feature vector and the transposed result of the data feature vector to determine the third operator; and perform a dot multiplication operation on the transposed result of the data feature vector and the data feature vector to determine the fourth operator. ;
  • Step 2 Determine the model parameter information of the target model based on the third operator, the fourth operator and the model parameter correction amount.
  • xx T corresponds to the third operator
  • x T x corresponds to the fourth operator
  • the model parameters can then be determined.
  • the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process.
  • the specific execution order of each step should be based on its function and possible The internal logic is determined.
  • the embodiments of the present disclosure also provide a data processing device corresponding to the data processing method. Since the principle of solving the problem of the device in the embodiment of the present disclosure is similar to the above-mentioned data processing method in the embodiment of the present disclosure, the implementation of the device Please refer to the implementation of the method, and the repeated parts will not be repeated.
  • the device includes: an acquisition module 601, a decomposition module 602, and a sending module 603; wherein,
  • the acquisition module 601 is used to obtain the label residual value determined for each data sample; each label residual value is used to characterize the degree of deviation between the real label of the corresponding data sample and the label predicted by the target model;
  • the decomposition module 602 is used to perform residual decomposition on each data sample based on each label residual value to obtain the model parameter correction amount corresponding to each data sample;
  • the sending module 603 is used to send the model parameter correction amount to the data receiver, so that the data receiver reconstructs the model parameter information of the target model according to the model parameter correction amount.
  • each data sample when the label residual value determined for each data sample is obtained, each data sample can be residually decomposed based on each label residual value, and then each data sample obtained by residual decomposition can be The corresponding model parameter corrections are sent to the data receiver, and the data receiver can reconstruct the model parameter information based on the model parameter corrections.
  • the data sender that is, the labeled party
  • the data receiver that is, the unlabeled party
  • the data sender can effectively perturb the residuals during the residual decomposition process, so that the data receiver (that is, the unlabeled party) can only construct a model with incorrect label information. , which allows the data sender to protect the real labels it owns.
  • the labeled party can pass the necessary correction information (i.e., the model parameter correction amount) to the unlabeled party, so that the unlabeled party can reconstruct accurately according to the corrected information.
  • the model parameter information further ensures the training performance of the model.
  • the decomposition module 602 is used to perform residual decomposition on each data sample based on each label residual value according to the following steps to obtain the model parameter correction amount corresponding to each data sample, including
  • the label residual value of the sorted target data sample is changed to obtain the residual change information corresponding to each data sample;
  • the model parameter correction amount corresponding to each data sample is determined.
  • the decomposition module 602 is used to select target data samples that require residual decomposition from each data sample according to the following steps:
  • the decomposition module 602 when the residual change information corresponding to each data sample corresponds to the residual change vector, the decomposition module 602 is used to change the label residual value of the sorted label where the target data sample is located according to the following steps: , get the residual change information corresponding to each data sample:
  • the residual change values corresponding to each data sample are collected to determine the residual change vector; where the residual change vector is used to characterize whether the label residual value of each data sample has changed.
  • the decomposition module 602 is used to determine the model parameter correction amount corresponding to each data sample based on the residual change information corresponding to each data sample and the real label of each data sample according to the following steps:
  • the model parameter correction amount corresponding to each data sample is determined.
  • the sending module 603 is also used to:
  • the changed label residual value is determined based on the product operation between the residual change information corresponding to each data sample and the sorted label residual values;
  • the sending module 603 is configured to send model parameter information to the data receiver according to the following steps:
  • the sending module 603 is used to send the model parameter correction amount to the data receiver according to the following steps:
  • the model parameter corrections are sent to the data receiver.
  • the device includes: a receiving module 701 and a reconstruction module 702; wherein,
  • the receiving module 701 is used to receive the model parameter correction amount corresponding to each data sample sent by the data sender; wherein the model parameter correction amount is determined by residual decomposition based on the label residual value determined for each data sample; each label residual The difference is used to characterize the degree of deviation between the true label of the corresponding data sample and the label predicted by the target model;
  • the reconstruction module 702 is used to reconstruct the model parameter information of the target model according to the model parameter correction amount.
  • the reconstruction module 702 is used to reconstruct the model parameter information of the target model according to the model parameter correction amount according to the following steps:
  • the model parameter information of the target model is determined.
  • the reconstruction module 702 is configured to determine the model parameter information of the target model based on the data feature information and the model parameter correction amount according to the following steps:
  • the model parameter information of the target model is determined.
  • FIG. 8 is a schematic structural diagram of the electronic device provided by an embodiment of the present disclosure, it includes: a processor 801, a memory 802, and a bus 803.
  • the memory 802 stores machine-readable instructions executable by the processor 801 (for example, the execution instructions corresponding to the acquisition module 601, the decomposition module 602, and the sending module 603 in the device in Figure 6; and the receiving module in the device in Figure 7 701, execution instructions corresponding to the reconstruction module 702, etc.).
  • the processor 801 and the memory 802 communicate through the bus 803.
  • the machine-readable instructions are executed by the processor 801, they are executed as shown in Figure 1 or Figure 5 The steps of the data processing method are shown.
  • Embodiments of the present disclosure also provide a computer-readable storage medium.
  • a computer program is stored on the computer-readable storage medium.
  • the storage medium may be a volatile or non-volatile computer-readable storage medium.
  • Embodiments of the present disclosure also provide a computer program product.
  • the computer program product carries program code.
  • the instructions included in the program code can be used to execute the steps of the data processing method described in the above method embodiment. For details, please refer to the above method. The embodiments will not be described again here.
  • the above-mentioned computer program product can be specifically implemented by hardware, software or a combination thereof.
  • the computer program product is embodied as a computer storage medium.
  • the computer program product is embodied as a software product, such as a Software Development Kit (SDK), etc. wait.
  • SDK Software Development Kit
  • the devices, equipment, computer-readable storage media and methods provided by the embodiments of the present application are in one-to-one correspondence. Therefore, the devices, equipment and computer-readable storage media also have beneficial technical effects similar to their corresponding methods. Since the above has been discussed The beneficial technical effects of the method have been described in detail. Therefore, the beneficial technical effects of the device, equipment and computer-readable storage media will not be described in detail here.
  • embodiments of the present disclosure may be provided as methods, apparatuses (devices or systems), or computer-readable storage media. Accordingly, the disclosure may take the form of an entirely hardware implementation, an entirely software implementation, or an implementation combining software and hardware aspects. Furthermore, the present disclosure may take the form of a computer-readable storage medium embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
  • These computer program instructions may also be stored in a computer-readable memory that causes a computer or other programmable data processing apparatus to operate in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction means, the instructions
  • the device implements the functions specified in a process or processes of the flowchart and/or a block or blocks of the block diagram.
  • These computer program instructions may also be loaded onto a computer or other programmable data processing device, causing a series of operating steps to be performed on the computer or other programmable device to produce computer-implemented processing, thereby executing on the computer or other programmable device.
  • Instructions provide steps for implementing the functions specified in a process or processes of a flowchart diagram and/or a block or blocks of a block diagram.
  • a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
  • processors CPUs
  • input/output interfaces network interfaces
  • memory volatile and non-volatile memory
  • Memory may include non-permanent storage in computer-readable media, random access memory (RAM) and/or non-volatile memory in the form of read-only memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.
  • RAM random access memory
  • ROM read-only memory
  • flash RAM flash random access memory
  • Computer-readable media includes both persistent and non-volatile, removable and non-removable media that can be implemented by any method or technology for storage of information.
  • Information may be computer-readable instructions, data structures, modules of programs, or other data.
  • Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), and read-only memory.
  • PRAM phase change memory
  • SRAM static random access memory
  • DRAM dynamic random access memory
  • RAM random access memory
  • read-only memory read-only memory
  • ROM read-only memory
  • EEPROM electrically erasable programmable read-only memory
  • flash memory or other memory technology
  • compact disc read-only memory CD-ROM
  • DVD digital versatile disc
  • Magnetic tape cassettes tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium can be used to store information that can be accessed by a computing device.
  • certain steps may be omitted, multiple steps may be combined into one step for execution, and/or one step may be broken down into multiple steps for execution.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Bioethics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Computer Hardware Design (AREA)
  • Computer Security & Cryptography (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Storage Device Security (AREA)
  • Signal Processing For Digital Recording And Reproducing (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Complex Calculations (AREA)

Abstract

本公开提供了一种数据处理方法、装置、电子设备及存储介质，其中，该方法包括：获取针对各个数据样本确定的标签残差值；每个标签残差值用于表征对应的数据样本的真实标签与目标模型所预测标签之间的偏差程度；基于各个标签残差值对各个数据样本进行残差分解，得到各个数据样本对应的模型参数修正量；向数据接收方发送模型参数修正量，以使得数据接收方根据模型参数修正量重构目标模型的模型参数信息。本公开通过残差分解可以有效地对残差进行扰动，这使得数据发送方能够保护其拥有的真实标签，与此同时，数据发送方可以传递必要的修正信息(即模型参数修正量)给数据接收方以重构精准的模型参数信息，进一步确保了模型的训练性能。

Description

一种数据处理方法、装置、电子设备及存储介质
本申请要求于2022年08月15日提交的、申请号为CN202210975564.1、标题为“一种数据处理方法、装置、电子设备及存储介质”的中国专利申请的优先权，该中国专利申请的公开内容以引用的方式并入本文。
技术领域
本公开涉及数据安全技术领域，具体而言，涉及一种数据处理方法、装置、电子设备及存储介质。
背景技术
随着人工智能和大数据挖掘分析等应用领域的发展，对数据量的需求越来越大。为了进行更大价值的挖掘，往往需要多方数据的融合。这里的多方数据可以是来源于不同的组织，例如，交易数据可以分散于金融结构，医疗诊断记录可以是来源于医疗机构，此外，还可以是来自于不同的行业，例如，同样是交易数据可以来自于电商还可以来自于金融。
多方数据要素在流通过程中受到监管部门的合规约束，使得将明细数据集中式收集后训练模型用于业务经营的方式并不可行，这构成了数据壁垒。为了打破数据壁垒，可以采用联邦学习的方法使得各个参与方在无需共享其隐私数据的前提下获得全局模型，这里的隐私数据可以是有关目标用户的标签信息，例如，目标用户是否为高净值用户。在合理挖掘数据的应用价值的同时，也需要防止出现数据滥用，实现对隐私数据的保护。在实际应用中，可以通过加密的方式交换模型相关信息以实现协同优化联邦模型。
相关技术中提供了一种残差加密方式，以标签数据作为隐私数据为例，有标签方可以在发送模型相关信息之前，以加法的形式作一个差分隐私，使得满足差分隐私条件后，再由有标签方以同态加密的形式发送给无标签 方。由于噪声的存在，无标签方无法根据掌握的模型相关参数重构真实的残差，从而达到保护隐私数据的目的。
然而，由于差分隐私自带噪声的原因，这使得无标签方后续在模型精度上存在一定的性能损失。
发明内容
本公开实施例至少提供一种数据处理方法、装置、电子设备及存储介质，数据发送方通过残差分解的方式对标签这一隐私数据进行加密，数据接收方基于加密得到的模型参数修正量进行模型参数重构，重构的模型精度较高。
第一方面，本公开实施例提供了一种数据处理方法，包括：
获取针对各个数据样本确定的标签残差值；每个所述标签残差值用于表征对应的数据样本的真实标签与目标模型所预测标签之间的偏差程度；
基于各个所述标签残差值对所述各个数据样本进行残差分解，得到所述各个数据样本对应的模型参数修正量；
向数据接收方发送所述模型参数修正量，以使得所述数据接收方根据所述模型参数修正量重构所述目标模型的模型参数信息。
在一种可能的实施方式中，所述基于各个所述标签残差值对所述各个数据样本进行残差分解，得到所述各个数据样本对应的模型参数修正量，包括：
按照数据大小的排列顺序将各个所述标签残差值进行排序，得到排序的各个标签残差值；
在从所述各个数据样本中选取出需要进行残差分解的目标数据样本的情况下，对所述目标数据样本所在排序的标签残差值进行变化，得到所述各个数据样本对应的残差变化信息；
基于所述各个数据样本对应的残差变化信息以及所述各个数据样本的 真实标签，确定所述各个数据样本对应的模型参数修正量。
在一种可能的实施方式中，按照如下步骤从所述各个数据样本中选取需要进行残差分解的目标数据样本：
按照预设分组数量将所述排序的各个标签残差值进行分组，得到分组后的标签残差值；
从每个分组后的标签残差值中选取预设数量个目标标签残差值；
将所述目标标签残差值对应的数据样本确定为所述目标数据样本。
在一种可能的实施方式中，在所述各个数据样本对应的残差变化信息对应残差变化向量的情况下，所述对所述目标数据样本所在排序的标签残差值进行变化，得到所述各个数据样本对应的残差变化信息，包括：
针对每个所述数据样本，在判断出所述数据样本非所述目标数据样本的情况下，确定所述数据样本对应第一残差变化值；或者，
在判断出所述数据样本是所述目标数据样本，且所述数据样本的标签残差值大于零的情况下，确定所述数据样本对应第二残差变化值；或者，
在判断出所述数据样本是所述目标数据样本，且所述数据样本的标签残差值小于零的情况下，确定所述数据样本对应第三残差变化值；
将各个所述数据样本分别对应的残差变化值进行集合，确定所述残差变化向量；其中，所述残差变化向量用于表征所述各个数据样本的标签残差值是否发生变化。
在一种可能的实施方式中，所述基于所述各个数据样本对应的残差变化信息以及所述各个数据样本的真实标签，确定所述各个数据样本对应的模型参数修正量，包括：
将所述残差变化向量以及所述残差变化向量的转置结果进行点乘运算，确定第一算子；以及，在确定所述各个数据样本的真实标签对应的标签值的情况下，确定与所述标签值对应的第二算子；
基于所述第一算子和所述第二算子，确定所述各个数据样本对应的模型参数修正量。
在一种可能的实施方式中，在得到所述各个数据样本对应的残差变化信息之后，所述方法还包括：
基于所述各个数据样本对应的残差变化信息和所述排序的各个标签残差值之间的乘积运算，确定变化后的标签残差值；
基于所述变化后的标签残差值确定向所述数据接收方发送的模型参数信息；
向所述数据接收方发送所述模型参数信息。
在一种可能的实施方式中，所述向所述数据接收方发送所述模型参数信息，包括：
在未达到模型收敛条件的情况下，向所述数据接收方发送所述模型参数信息；
所述向数据接收方发送所述模型参数修正量，包括：
在达到模型收敛条件的情况下，向所述数据接收方发送所述模型参数修正量。
第二方面，本公开还提供了一种数据处理方法，包括：
接收数据发送方发送的各个数据样本对应的模型参数修正量；其中，所述模型参数修正量是基于各个数据样本确定的标签残差值进行残差分解确定的；每个所述标签残差值用于表征对应的数据样本的真实标签与目标模型所预测标签之间的偏差程度；
根据所述模型参数修正量重构所述目标模型的模型参数信息。
在一种可能的实施方式中，所述根据所述模型参数修正量重构所述目标模型的模型参数信息，包括：
获取针对目标模型输入的数据特征信息；
基于所述数据特征信息和所述模型参数修正量，确定所述目标模型的模型参数信息。
在一种可能的实施方式中，在所述数据特征信息包括数据特征向量的情况下，所述基于所述数据特征信息和所述模型参数修正量，确定所述目标模型的模型参数信息，包括：
将所述数据特征向量以及所述数据特征向量的转置结果进行点乘运算，确定第三算子；以及，将所述数据特征向量的转置结果以及所述数据特征向量进行点乘运算，确定第四算子；
基于所述第三算子、所述第四算子以及所述模型参数修正量，确定所述目标模型的模型参数信息。
第三方面，本公开还提供了一种数据处理装置，包括：
获取模块，用于获取针对各个数据样本确定的标签残差值；每个所述标签残差值用于表征对应的数据样本的真实标签与目标模型所预测标签之间的偏差程度；
分解模块，用于基于各个所述标签残差值对所述各个数据样本进行残差分解，得到所述各个数据样本对应的模型参数修正量；
发送模块，用于向数据接收方发送所述模型参数修正量，以使得所述数据接收方根据所述模型参数修正量重构所述目标模型的模型参数信息。
第四方面，本公开还提供了一种数据处理装置，包括：
接收模块，用于接收数据发送方发送的各个数据样本对应的模型参数修正量；其中，所述模型参数修正量是基于各个数据样本确定的标签残差值进行残差分解确定的；每个所述标签残差值用于表征对应的数据样本的真实标签与目标模型所预测标签之间的偏差程度；
重构模块，用于根据所述模型参数修正量重构所述目标模型的模型参 数信息。
第五方面，本公开还提供了一种电子设备，包括：处理器、存储器和总线，所述存储器存储有所述处理器可执行的机器可读指令，当电子设备运行时，所述处理器与所述存储器之间通过总线通信，所述机器可读指令被所述处理器执行时执行如第一方面及其各种实施方式、第二方面及其各种实施方式任一项所述的数据处理方法。
第六方面，本公开还提供了一种计算机可读存储介质，该计算机可读存储介质上存储有计算机程序，该计算机程序被处理器运行时执行如第一方面及其各种实施方式、第二方面及其各种实施方式任一项所述的数据处理方法。
采用上述数据处理方法、装置、电子设备及存储介质，在获取到针对各个数据样本确定的标签残差值的情况下，可以基于各个所述标签残差值对所述各个数据样本进行残差分解，而后可以将残差分解得到的各个数据样本对应的模型参数修正量发送给数据接收方，数据接收方则可以根据模型参数修正量重构模型参数信息。本公开中，数据发送方(即有标签方)在进行残差分解的过程中，可以有效地对残差进行扰动，使得数据接收方(即无标签方)仅能构造带错误标签信息的模型，这使得数据发送方能够保护其拥有的真实标签，与此同时，有标签方可以传递必要的修正信息(即模型参数修正量)给无标签方，使得无标签方可以按照修正信息重构精准的模型参数信息，进一步确保了模型的训练性能。
本公开的其他优点将配合以下的说明和附图进行更详细的解说。
应当理解，上述说明仅是本公开技术方案的概述，以便能够更清楚地了解本公开的技术手段，从而可依照说明书的内容予以实施。为了让本公开的上述和其它目的、特征和优点能够更明显易懂，以下特举例说明本公开的具体实施方式。
附图说明
为了更清楚地说明本公开实施例的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，此处的附图被并入说明书中并构成本说明书中的一部分，这些附图示出了符合本公开的实施例，并与说明书一起用于说明本公开的技术方案。应当理解，以下附图仅示出了本公开的某些实施例，因此不应被看作是对范围的限定，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他相关的附图。而且在整个附图中，用相同的标号表示相同的部件。在附图中：
图1示出了本公开实施例所提供的一种数据处理方法的流程图；
图2示出了本公开实施例所提供的数据处理方法中，确定残差变化信息具体方法的流程图；
图3示出了本公开实施例所提供的一种数据处理方法的时序流程图；
图4示出了本公开实施例所提供的一种数据处理方法的应用示意图；
图5示出了本公开实施例所提供的另一种数据处理方法的流程图；
图6示出了本公开实施例所提供的一种数据处理装置的示意图；
图7示出了本公开实施例所提供的另一种数据处理装置的示意图；
图8示出了本公开实施例所提供的一种电子设备的示意图。
具体实施方式
下面将参照附图更详细地描述本公开的示例性实施方式。虽然附图中显示了本公开的示例性实施方式，然而应当理解，可以以各种形式实现本公开而不应被这里阐述的实施方式所限制。相反，提供这些实施方式是为了能够更透彻地理解本公开，并且能够将本公开的范围完整的传达给本领域的技术人员。
在本公开实施方式的描述中，应理解，诸如“包括”或“具有”等术语旨在指示本说明书中所公开的特征、数字、步骤、行为、部件、部分或其组合的存在，并且不旨在排除一个或多个其他特征、数字、步骤、行为、 部件、部分或其组合存在的可能性。
除非另有说明，“/”表示或的意思，例如，A/B可以表示A或B；本文中的“和/或”仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。
术语“第一”、“第二”等仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”等的特征可以明示或者隐含地包括一个或者更多个这一特征。在本公开实施方式的描述中，除非另有说明，“多个”的含义是两个或两个以上。
经研究发现，相关技术中可以通过加密的方式交换模型相关信息以实现协同优化联邦模型。
相关技术中提供了一种残差加密方式，以标签数据作为隐私数据为例，有标签方可以在发送模型相关信息之前，以加法的形式作一个差分隐私，使得满足差分隐私条件后，再由有标签方以同态加密的形式发送给无标签方。
其中，在进行差分隐私的过程，也即是给残差r i增加噪声的过程，使得差分隐私
Figure PCTCN2023071564-appb-000001
其中所增加的噪声满足条件noise～Lap(2ε -1)(即，噪声来自于标准差为2ε -1产生的拉普拉斯随机变量)。由于噪声的存在，无标签方无法根据掌握的模型相关参数重构真实的残差r i，从而达到保护隐私数据的目的。
另外一种加密方式是本地化差分隐私随机响应。其思想为通过随机扰动批(batch)的组成，使得交互的残差在形式上维度不变，但是实际的秩小于批中样本的个数，从而得到的残差不唯一。
具体来说，有标签方根据双方商定的batch大小，从有标签方样本选择一个子集C，以标签的类别分布生成一个关于标签y∈{0,1}的随机响应向量 m，并将差分隐私处理得到m rr发送给无标签方。当无标签方根据特征x b构建局部模型后，有标签方将随机响应得到的m i＝1残差给出真实的残差值r i，而对于不在批中的样本则置0，在保证交互的残差维度仍然为批的大小(|B|)下，所构造的残差矩阵经同态加密后发送给无标签方。发送的内容D的组成形式化表示为：
Figure PCTCN2023071564-appb-000002
其中
Figure PCTCN2023071564-appb-000003
σ(·)为sigmoid函数，<·>为同态加密。该方法称为差分隐私和同态加密混合的标签保护策略，且在满足如下条件下能实现ε差分隐私：
Figure PCTCN2023071564-appb-000004
q为m中标注为1的比例。通过有标签方的上述操作，使得无标签方无法构造关于残差的线性方程组。
然而，上述方式虽然实现了标签保护，而且分类的精度损失相比第一种加密方式而言有所下降，但是由于差分隐私和随机置零的存在，没有充分利用批中的样本信息，因而相对于明文计算而言依然有性能损失，且分类精度损失的减小以增加训练成本为代价，暂不具备短期内投入生产使用的可能性。
为了至少部分地解决上述问题以及其他潜在问题中的一个或者多个，本公开提供了至少一种数据处理方案，有标签方通过残差分解的方式对标签这一隐私数据进行加密，无标签方基于加密得到的模型参数修正量进行模型参数重构，重构的模型精度较高。
需要说明的是，本公开实施例中的有标签方和无标签方均需要进行本地计算，并协作式交互修正信息，计算开销并没有显著增加。这里的有标签方对应的是数据发送方，无标签方对应的是数据接收方，残差分解是在有标签方进行，参数重构则在无标签方完成。
在实际应用中，这里的数据发送方指的是具有真实标签的发起方及其用于执行包括残差分解操作在内的处理设备，这里的数据接收方指的是具有模型输入特征，但不具备真实标签的响应方及其用于执行参数重构操作在内的处理设备。
为便于对本实施例进行理解，首先对本公开实施例所公开的一种数据处理方法进行详细介绍，本公开实施例所提供的数据处理方法的执行主体一般为具有一定计算能力的电子设备，该电子设备例如包括：终端设备或其它处理设备，终端设备可以为用户设备(User Equipment，UE)、移动设备、用户终端、蜂窝电话、个人数字助理(Personal Digital Assistant，PDA)、手持设备等。在一些可能的实现方式中，该数据处理方法可以通过处理器调用存储器中存储的计算机可读指令的方式来实现。
参见图1所示，为本公开实施例提供的数据处理方法的流程图，该方法主要是在数据发送方执行，具体包括步骤S101～S103，其中：
S101：获取针对各个数据样本确定的标签残差值；每个标签残差值用于表征对应的数据样本的真实标签与目标模型所预测标签之间的偏差程度；
S102：基于各个标签残差值对各个数据样本进行残差分解，得到各个数据样本对应的模型参数修正量；
S103：向数据接收方发送模型参数修正量，以使得数据接收方根据模型参数修正量重构目标模型的模型参数信息。
为了便于理解本公开实施例提供的数据处理方法，接下来首先对该方法的应用场景进行具体说明。本公开实施例中的数据处理方法主要可以应用于联邦学习领域，这里的联邦学习对应的可以是多方数据的融合学习方式，例如，可以是来源于金融结构的交易数据以及医疗机构的医疗诊断记录所形成的融合数据。
在相关技术中，大多采用纵向逻辑回归方法来构建联合模型。而由于 纵向逻辑回归方法本身的特性约束，导致一方的隐私数据很可能被另一方反推得到。例如，金融机构内产生的有关一个用户是否为高净值用户的标签数据很可能随着模型参数信息的共享而泄漏到医疗机构，在这种情况下，将对用户隐私带来一定的不安全因素。
为了在更好的推进联邦学习的同时，提升隐私数据的保护力度，本公开实施例提供了一种协作式的数据处理方案，该方案主要对应两个阶段，一个是模型训练阶段，另一个是模型修正阶段。
针对模型训练阶段，主要是在有标签方执行残差分解的过程。分解后仅有一部分残差相关的混淆信息在同态加密保护下与无标签方交换，而另一部分残差保留信息由有标签方单独持有，不参与模型训练过程，使得无标签方即使按照相关标签泄露攻击方式(满足列满秩条件)解出残差，也会因为该残差与真实标签对应关系不再具备一一对应关系，使得其关于有标签方标签推测的精度与随机猜测二分类结果相当，无法根据已知标签泄露攻击方式获得的残差的符号反推有标签方的真实标签，从而确保了隐私数据的安全性。
针对模型修正阶段，主要是在无标签方执行参数重构的过程。在有标签方模型训练收敛后，由有标签方发起，无标签方响应，仅需要有标签方和无标签方额外进行一轮次关于无标签方权重信息的交互即可完成无标签方每个特征权值的修正。由于有标签方提供的额外信息无法通过模型训练阶段获得，且该信息系真实标签与保护标签所实施的残差保留的整体，无标签方在重构出高精度的模型的前提下，也无法在模型修正阶段获得有标签方的标签信息，从而进一步确保了隐私数据的安全性。
本公开实施例中的数据发送方，在获取到针对各个数据样本确定的标签残差值的情况下，可以基于各个标签残差值对各个数据样本进行残差分解，得到各个数据样本对应的模型参数修正量，最后向数据接收方发送模型参数修正量。这里的模型参数修正量即是有标签方向无标签方提供的额外信息，该额外信息是在对残差进行分解后确定的。
有标签方根据无标签方对自身标签的拟合程度在本地随机执行残差分解，以达到随机化替换标签信息的效果，使得无标签方不能根据有标签方处理后的残差信息反推真实标签。
可知的是，本公开实施例中，通过扰动残差使得无标签方仅能构造带错误标签信息的模型，而所包含的错误信息系有标签方本地对残差信息作分解产生，无标签方对此无法区分；然后，有标签方传递必要的修正信息给无标签方，使得无标签方可以修正特征的权值，但不能获得有标签方的标签。
无标签方在有标签方的配合下修正存在偏差的特征权值，得到正确的模型参数信息。由于该信息由有标签方的标签及对标签的扰动信息组成，其中扰动信息与模型训练阶段无标签方已知的信息无关，使得无标签方无法获得有标签方的标签信息。
考虑到模型参数修正量的确定对于无标签方重构模型参数的关键作用，接下来可以重点说明有关确定模型参数修正量的过程，具体可以通过如下步骤来实现：
步骤一、按照数据大小的排列顺序将各个标签残差值进行排序，得到排序的各个标签残差值；
步骤二、在从各个数据样本中选取出需要进行残差分解的目标数据样本的情况下，对目标数据样本所在排序的标签残差值进行变化，得到各个数据样本对应的残差变化信息；
步骤三、基于各个数据样本对应的残差变化信息以及各个数据样本的真实标签，确定各个数据样本对应的模型参数修正量。
这里，首先可以对各个标签残差值进行排序，并得到残差变化信息，而后基于残差变化信息以及真实标签确定模型参数修正量。
其中，上述残差变化信息用于指示标签残差值是否发生变化，对于变化的标签残差值可以进行第一标记，对于未变化的标签残差值可以进行第 二标记，从而可以达到随机替换真实标签的目的，又考虑到有关模型参数修正量是结合残差变化信息和真实标签得到的，也即，无标签方获得的并非直接的真实标签而是经过标签映射后的修正量，基于这一修正量以及对应的输入特征即可以构建出模型参数信息。
需要说明的是，有标签方在模型收敛后才需要将真实的标签和所作的标签映射在本地做运算合成后再发送给无标签方。
为了达到随机替换真实标签的目的，在进行残差变化之前，需要通过分组、选取等一系列操作选择出目标样本。具体地，首先可以按照预设分组数量将排序的各个标签残差值进行分组，得到分组后的标签残差值，而后从每个分组后的标签残差值中选取预设数量个目标标签残差值，最后将目标标签残差值对应的数据样本确定为目标数据样本。这里的分组可以是预设好的，例如，可以分为两组、四组等，在此不做具体的限制，目的在于使得无标签方无法根据特征之间的相似性获得关于有标签方标签信息的额外信息。与此同时，在从各分组中选取需要进行残差变化的数据样本的过程中，可以是随机定量的抽取，以确保标签替换的随机性，进一步提升数据的安全性。
本公开实施例中的残差变化信息可以指向的是残差变化向量，每个向量对应一个数据样本的残差变化情况。其中，在判断出一个数据样本非目标数据样本的情况下，可以确定数据样本对应第一残差变化值；在判断出一个数据样本是目标数据样本，且数据样本的标签残差值大于零的情况下，可以确定数据样本对应第二残差变化值；在判断出一个数据样本是目标数据样本，且数据样本的标签残差值小于零的情况下，确定数据样本对应第三残差变化值。而后，可以将各个数据样本分别对应的残差变化值进行集合，确定残差变化向量。
基于上述残差变化向量，可以确定模型参数修正量，具体通过如下步骤来实现：
步骤一、将残差变化向量以及残差变化向量的转置结果进行点乘运算，确定第一算子；以及，在确定各个数据样本的真实标签对应的标签值的情况下，确定与标签值对应的第二算子；
步骤二、基于第一算子和第二算子，确定各个数据样本对应的模型参数修正量。
这里可以基于第一算子和第二算子确定模型参数修正量。该模型参数修正量也可以是一个多维向量，每一个维度指向的是一个数据样本的修正量，继而实现针对目标模型的整体修正。
在确定残差变化信息的过程中，还可以基于各个数据样本对应的残差变化信息和排序的各个标签残差值之间的乘积运算，确定变化后的标签残差值，而后可以基于变化后的标签残差值确定向数据接收方发送的模型参数信息，并向数据接收方发送模型参数信息。
其中，上述有关变化后的标签残差值的计算过程可以是在模型收敛之前仅需要执行一次的步骤，例如，有标签方可以是在首轮执行上述步骤，改变随机选择样本的残差值，其效果相当于改变了样本真实标签，在确保了标签安全性的前提下，还较低了计算成本。
需要说明的是，这里的模型参数信息可以是训练中间过程，也即未达到模型收敛条件的情况下，有标签方向无标签方发送的，由于这里发生了残差变化，即使无标签方接收到了模型参数信息，也无法反推出真实标签。在完成模型训练，也即达到模型收敛条件的情况下，有标签方则可以向无标签方发送修正量以便于无标签方重构出准确的模型参数信息。
在实际应用中，有关模型收敛可以是与训练批次相关联的，在一个批次内的所有数据样本都完成一轮参数更新的情况下，可以判定模型是否满足收敛条件，若收敛则按照上述方式发送修正量，否则继续执行模型训练。
为了便于理解上述有关模型参数修正量的确定过程，可以结合如下实施例进一步进行说明。
首先，前提条件示例如下：
1)建模需在两方之间完成，A方承担数据应用方(Guest)的角色，B方承担数据提供方(Host)和协作者(Arbiter)的角色。标签在A方(即有标签方)，特征在B方(即无标签方)；
2)A方的标签为二分类(0/1)；
3)B方仅已知A方的标签为二分类0/1形式，A方标签的其他信息B方未知，但任一类别的标签均为B方感兴趣(即B方希望了解A方任何一个样本的标签信息)；
4)双方约定按照纵向逻辑回归的基本步骤完成联合建模，B方仅可对A方发起半诚实攻击。
这里，有标签方和无标签方以某种安全方式约定双方联合建模均采用纵向逻辑回归，确定每轮建模的批B的大小|B|及组成、模型参数学习率η，且通过安全求交确定对齐样本。其中B方已知对齐样本的特征{(x i)}，A方已知对齐样本的标签{y i}。每一轮由B方发起，A方响应，以B方更新局部模型
Figure PCTCN2023071564-appb-000005
的参数结束。同时A方和B方约定当模型收敛(flag＝1)后，双方协作式完成B方真实权值W的重构。
在模型训练阶段，当模型未达到收敛条件(此时flag＝0)时，执行以下步骤：
在一个批B中，B方根据自有的模型参数
Figure PCTCN2023071564-appb-000006
以及特征集合
Figure PCTCN2023071564-appb-000007
构造局部模型
Figure PCTCN2023071564-appb-000008
(计算方式为
Figure PCTCN2023071564-appb-000009
)，然后将
Figure PCTCN2023071564-appb-000010
以同态加密(例如paillier算法)发送给A方。
当模型达到收敛条件(此时flag＝0)时，此时双方已完成关于带错误标签的模型训练，并执行如下操作：
A方将模型参数修正量
Figure PCTCN2023071564-appb-000011
发送给B方，B方按照如下方式重构真实的模型参数w：
Figure PCTCN2023071564-appb-000012
其中，X对应的是数据特征向量，y对应的是真实标签的值，S是S B按批拼接得到的标签映射向量，对应的是残差变化信息。
需要说明的是，上述残差变化信息对应的可以是本批样本残差的保留信息
Figure PCTCN2023071564-appb-000013
且该保留信息可以是在训练的第一轮确定的。具体可以按照如下步骤来实现：
1)A方根据残差r i B(t)排序(从小到大或从大到小均可)所有批内的样本。
2)A方对本批样本的残差按如下方式作随机化残差分解。随机选择一个数m∈{2,4}作为本批内由样本构成的组大小。
对于排序后的样本序列，从第一个样本起，每m个样本形成一个组
Figure PCTCN2023071564-appb-000014
其中符号
Figure PCTCN2023071564-appb-000015
代表不小于|B|/m的最小整数。对于所形成的任意一个组G j，随机从组内选择
Figure PCTCN2023071564-appb-000016
个样本形成候选集C B，且
Figure PCTCN2023071564-appb-000017
且对于所有在C B内的样本，按照以下方式确定对应的保留信息
Figure PCTCN2023071564-appb-000018
其中，若r i B(t)＞0，则
Figure PCTCN2023071564-appb-000019
若r i B(t)＜0，则
Figure PCTCN2023071564-appb-000020
不管是1还是-1，后续均将会执行残差变化。对于所有不在C B内的样本x i，有
Figure PCTCN2023071564-appb-000021
后续不会执行残差变化。具体可以按照如下公式来确定：
Figure PCTCN2023071564-appb-000022
其中sort(i)为按照残差Δr i B排序得到的第i个样本在批中的顺序。这里，为了防止B方反推，可以按照B方针对数据样本的原始顺序进行重排。
为了便于进一步说明有关重排操作所对应的残差分解，可以结合图2所示的残差分解示意图进一步进行说明。
如图2所示，批大小|B|＝8，按残差值由小到大顺序排列后，第二列和 第四列的粗体分别为
Figure PCTCN2023071564-appb-000023
在m＝2和m＝4下按钮随机先择的样本非零表示，相应的随机化残差分解可见第三列和第五列的1/-1部分。
这里，基于标签保留信息
Figure PCTCN2023071564-appb-000024
可以确定残差变化向量S B。S B中的每个元素体现样本在首轮残差分解时残差信息是否发生变化。当残差发生变化时，
Figure PCTCN2023071564-appb-000025
否则
Figure PCTCN2023071564-appb-000026
即：
Figure PCTCN2023071564-appb-000027
在确定残差变化向量S B的情况下，可以将残差变化向量以及残差变化向量的转置结果进行点乘运算，确定第一算子ss T；以及，在确定各个数据样本的真实标签对应的标签值的情况下，确定与标签值对应的第二算子
Figure PCTCN2023071564-appb-000028
而后确定模型参数修正量
Figure PCTCN2023071564-appb-000029
需要说明的是，上述模型参数修正量可以是在结束模型训练之后A方向B方发送的，在模型训练过程中，A方发送给B方的伪残差可以按下式计算：
Figure PCTCN2023071564-appb-000030
这样，B方在密态下更新本方的模型参数
Figure PCTCN2023071564-appb-000031
计算方式为：
Figure PCTCN2023071564-appb-000032
其中，
Figure PCTCN2023071564-appb-000033
为B方根据身份信息求解得的关于参数
Figure PCTCN2023071564-appb-000034
的梯度信息，该信息等价于A方按照特征及对应的伪残差在每个样本上乘积得到的伪梯度信息
Figure PCTCN2023071564-appb-000035
对每一个批B重复以上步骤，直至所有样本都完成一轮参数更新，然后判定模型是否满足收敛条件，若收敛则置flag＝1，否则继续。主要操作的时序图如图3所示。
可知的是，本公开实施例在有标签方进行了一次残差分解(仅联合建模的第一轮)和一次模型参数重构信息发送这两个步骤，同时在无标签方需进行一次权值构造，具体可以参照图4进行说明。
如图4所示，在完成批初始化之后，无标签方可以基于有标签方给予的模型参数建立局部模型，而后由有标签方判断是否处于首轮训练，若是，有标签方可以按照上述步骤构造保留残差以及标签变换，无标签方则可以基于伪残差进行更新。而后，在确定达到模型收敛的情况下，有标签方可以发送额外信息给无标签方以便无标签方重构权值。
需要说明的是，本公开实施例不限制批初始化和达到收敛条件两个模块的具体实现方式，批初始化前的双方样本对齐亦不限定实现方式。本公开实施例的重点在于有标签方实施针对无标签方的残差分解和与无标签方协作式实现权值重构，其他在模型训练阶段的步骤计算可以采用经典的梯度下降法求解。
此外，有关数据安全性可以从模型训练和模型修正两部分分析。模型训练过程中，残差保留信息由有标签方单独持有，不参与模型训练过程。无标签方无法根据已知的信息判定有标签方对特定的示例标签是否修改。
对有标签方而言，这种安全是信息论级别，其效果使得无标签方无法通过增加算力获得本方的标签，且仅需执行一轮本地操作，额外计算开销相对于整个联合建模开销而言可以忽略不计。因此无标签方无法依据半诚实攻击获取有标签方的真实标签y。
而权值重构部分，理论依据为等式XWS＝YS。由于真实标签及其变换信息整体
Figure PCTCN2023071564-appb-000036
发送给无标签方，该过程不涉及梯度信息交互，与已知标签泄露方式无关。这使得无标签方也无法计算有标签方对标签所作的标签映射S以及实际标签Y。因此，若攻击者沿用已知的标签泄露攻击方式，则自始至终无法消除残差与标签之间的随机对应性。换而言之，不论无标签方获得的结果为正的残差值还是负的残差值，其对应的真实标签可能为正类(1)也可能为负类(0)。
再者，有关模型可用性可从权值重构的角度分析。鉴于问题的背景可以限定为二分类，因此若在首轮有标签方改变了随机选择样本的残差值， 其效果相当于改变了样本真实标签，从正类(负类)变化为负类(正类)。其中排序的意义在于对抗无标签方因随机初始化参数的差异泄露更多有标签方标签信息。因而在真实标签Y上施加S或
Figure PCTCN2023071564-appb-000037
没有明显的信息差异，而且最终得到的结果在有标签方发送前并未对信息作包括差分隐私在内的混淆化处理，理论上权值对样本类别的区分能力可以达到更佳效果。
参见图5所示，为本公开实施例提供的数据处理方法的流程图，该方法主要是在数据接收方执行，具体包括步骤S501～S502，其中：
S501：接收数据发送方发送的各个数据样本对应的模型参数修正量；其中，模型参数修正量是基于各个数据样本确定的标签残差值进行残差分解确定的；每个标签残差值用于表征对应的数据样本的真实标签与目标模型所预测标签之间的偏差程度；
S502：根据模型参数修正量重构目标模型的模型参数信息。
这里，基于接收的模型参数修正量可以实现有关目标模型的参数重构，由于这里的模型参数修正量是基于数据发送方的真实标签以及对应的残差变化信息确定的，因而可以标记对应的标签替换情况，且基于修正量无法解析出真实标签，因而可以在对数据发送方的真实标签进行保护的前提下，实现模型参数的准确重构。
其中，有关模型参数修正量的确定过程可以参照上述实施例的相关描述，在此不做赘述。
在重构模型参数信息的过程中，除了需要依赖于上述模型参数修正量，还需要考虑针对目标模型输入的数据特征信息，具体可以通过如下步骤来实现：
步骤一、将数据特征向量以及数据特征向量的转置结果进行点乘运算，确定第三算子；以及，将数据特征向量的转置结果以及数据特征向量进行点乘运算，确定第四算子；
步骤二、基于第三算子、第四算子以及模型参数修正量，确定目标模 型的模型参数信息。
本公开实施例中有关模型参数信息的确定过程可以参照上述模型参数w的公式：
Figure PCTCN2023071564-appb-000038
这里的xx T对应的是第三算子，x Tx对应的是第四算子，
Figure PCTCN2023071564-appb-000039
对应的是修正量，继而可以确定出模型参数。
在本说明书的描述中，参考术语“一些可能的实施方式”、“一些实施方式”、“示例”、“具体示例”、或“一些示例”等的描述意指结合该实施方式或示例描述的具体特征、结构、材料或者特点包含于本公开的至少一个实施方式或示例中。在本说明书中，对上述术语的示意性表述不必须针对的是相同的实施方式或示例。而且，描述的具体特征、结构、材料或者特点可以在任一个或多个实施方式或示例中以合适的方式结合。此外，在不相互矛盾的情况下，本领域的技术人员可以将本说明书中描述的不同实施方式或示例以及不同实施方式或示例的特征进行结合和组合。
关于本公开实施方式的方法流程图，将某些操作描述为以一定顺序执行的不同的步骤。这样的流程图属于说明性的而非限制性的。可以将在本文中所描述的某些步骤分组在一起并且在单个操作中执行、可以将某些步骤分割成多个子步骤、并且可以以不同于在本文中所示出的顺序来执行某些步骤。可以由任何电路结构和/或有形机制(例如，由在计算机设备上运行的软件、硬件(例如，处理器或芯片实现的逻辑功能)等、和/或其任何组合)以任何方式来实现在流程图中所示出的各个步骤。
本领域技术人员可以理解，在具体实施方式的上述方法中，各步骤的撰写顺序并不意味着严格的执行顺序而对实施过程构成任何限定，各步骤的具体执行顺序应当以其功能和可能的内在逻辑确定。
基于同一发明构思，本公开实施例中还提供了与数据处理方法对应的 数据处理装置，由于本公开实施例中的装置解决问题的原理与本公开实施例上述数据处理方法相似，因此装置的实施可以参见方法的实施，重复之处不再赘述。
参照图6所示，为本公开实施例提供的一种数据处理装置的示意图，装置包括：获取模块601、分解模块602、发送模块603；其中，
获取模块601，用于获取针对各个数据样本确定的标签残差值；每个标签残差值用于表征对应的数据样本的真实标签与目标模型所预测标签之间的偏差程度；
分解模块602，用于基于各个标签残差值对各个数据样本进行残差分解，得到各个数据样本对应的模型参数修正量；
发送模块603，用于向数据接收方发送模型参数修正量，以使得数据接收方根据模型参数修正量重构目标模型的模型参数信息。
采用上述数据处理装置，在获取到针对各个数据样本确定的标签残差值的情况下，可以基于各个标签残差值对各个数据样本进行残差分解，而后可以将残差分解得到的各个数据样本对应的模型参数修正量发送给数据接收方，数据接收方则可以根据模型参数修正量重构模型参数信息。本公开中，数据发送方(即有标签方)在进行残差分解的过程中，可以有效地对残差进行扰动，使得数据接收方(即无标签方)仅能构造带错误标签信息的模型，这使得数据发送方能够保护其拥有的真实标签，与此同时，有标签方可以传递必要的修正信息(即模型参数修正量)给无标签方，使得无标签方可以按照修正信息重构精准的模型参数信息，进一步确保了模型的训练性能。
在一种可能的实施方式中，分解模块602，用于按照如下步骤基于各个标签残差值对各个数据样本进行残差分解，得到各个数据样本对应的模型参数修正量，包括
将各个标签残差值进行排序，得到排序的各个标签残差值；
在从各个数据样本中选取出需要进行残差分解的目标数据样本的情况下，对目标数据样本所在排序的标签残差值进行变化，得到各个数据样本对应的残差变化信息；
基于各个数据样本对应的残差变化信息以及各个数据样本的真实标签，确定各个数据样本对应的模型参数修正量。
在一种可能的实施方式中，分解模块602，用于按照如下步骤从各个数据样本中选取需要进行残差分解的目标数据样本：
按照预设分组数量将排序的各个标签残差值进行分组，得到分组后的标签残差值；
从每个分组后的标签残差值中选取预设数量个目标标签残差值；
将目标标签残差值对应的数据样本确定为目标数据样本。
在一种可能的实施方式中，在各个数据样本对应的残差变化信息对应残差变化向量的情况下，分解模块602，用于按照如下步骤对目标数据样本所在排序的标签残差值进行变化，得到各个数据样本对应的残差变化信息：
针对每个数据样本，在判断出数据样本非目标数据样本的情况下，确定数据样本对应第一残差变化值；或者，
在判断出数据样本是目标数据样本，且数据样本的标签残差值大于零的情况下，确定数据样本对应第二残差变化值；或者，
在判断出数据样本是目标数据样本，且数据样本的标签残差值小于零的情况下，确定数据样本对应第三残差变化值；
将各个数据样本分别对应的残差变化值进行集合，确定残差变化向量；其中，残差变化向量用于表征各个数据样本的标签残差值是否发生变化。
在一种可能的实施方式中，分解模块602，用于按照如下步骤基于各个数据样本对应的残差变化信息以及各个数据样本的真实标签，确定各个数据样本对应的模型参数修正量：
将残差变化向量以及残差变化向量的转置结果进行点乘运算，确定第一算子；以及，在确定各个数据样本的真实标签对应的标签值的情况下，确定与标签值对应的第二算子；
基于第一算子和第二算子，确定各个数据样本对应的模型参数修正量。
在一种可能的实施方式中，发送模块603，还用于：
在得到各个数据样本对应的残差变化信息之后，基于各个数据样本对应的残差变化信息和排序的各个标签残差值之间的乘积运算，确定变化后的标签残差值；
基于变化后的标签残差值确定向数据接收方发送的模型参数信息；
向数据接收方发送模型参数信息。
在一种可能的实施方式中，发送模块603，用于按照如下步骤向数据接收方发送模型参数信息：
在未达到模型收敛条件的情况下，向数据接收方发送模型参数信息；
发送模块603，用于按照如下步骤向数据接收方发送模型参数修正量：
在达到模型收敛条件的情况下，向数据接收方发送模型参数修正量。
参照图7所示，为本公开实施例提供的另一种数据处理装置的示意图，装置包括：接收模块701、重构模块702；其中，
接收模块701，用于接收数据发送方发送的各个数据样本对应的模型参数修正量；其中，模型参数修正量是基于各个数据样本确定的标签残差值进行残差分解确定的；每个标签残差值用于表征对应的数据样本的真实标签与目标模型所预测标签之间的偏差程度；
重构模块702，用于根据模型参数修正量重构目标模型的模型参数信息。
在一种可能的实施方式中，重构模块702，用于按照如下步骤根据模型 参数修正量重构目标模型的模型参数信息：
获取针对目标模型输入的数据特征信息；
基于数据特征信息和模型参数修正量，确定目标模型的模型参数信息。
在一种可能的实施方式中，在数据特征信息包括数据特征向量的情况下，重构模块702，用于按照如下步骤基于数据特征信息和模型参数修正量，确定目标模型的模型参数信息：
将数据特征向量以及数据特征向量的转置结果进行点乘运算，确定第三算子；以及，将数据特征向量的转置结果以及数据特征向量进行点乘运算，确定第四算子；
基于第三算子、第四算子以及模型参数修正量，确定目标模型的模型参数信息。
需要说明的是，本申请实施方式中的装置可以实现前述方法的实施方式的各个过程，并达到相同的效果和功能，这里不再赘述。
本公开实施例还提供了一种电子设备，如图8所示，为本公开实施例提供的电子设备结构示意图，包括：处理器801、存储器802、和总线803。存储器802存储有处理器801可执行的机器可读指令(比如，图6中的装置中获取模块601、分解模块602、发送模块603对应的执行指令等；再如图7中的装置中接收模块701、重构模块702对应的执行指令等)，当电子设备运行时，处理器801与存储器802之间通过总线803通信，机器可读指令被处理器801执行时执行如图1或者如图5所示的数据处理方法的步骤。
本公开实施例还提供一种计算机可读存储介质，该计算机可读存储介质上存储有计算机程序，该计算机程序被处理器运行时执行上述方法实施例中所述的数据处理方法的步骤。其中，该存储介质可以是易失性或非易失的计算机可读取存储介质。
本公开实施例还提供一种计算机程序产品，该计算机程序产品承载有程序代码，所述程序代码包括的指令可用于执行上述方法实施例中所述的数据处理方法的步骤，具体可参见上述方法实施例，在此不再赘述。
其中，上述计算机程序产品可以具体通过硬件、软件或其结合的方式实现。在一个可选实施例中，所述计算机程序产品具体体现为计算机存储介质，在另一个可选实施例中，计算机程序产品具体体现为软件产品，例如软件开发包(Software Development Kit，SDK)等等。
本申请中的各个实施方式均采用递进的方式描述，各个实施方式之间相同相似的部分互相参见即可，每个实施方式重点说明的都是与其他实施方式的不同之处。尤其，对于装置、设备和计算机可读存储介质实施方式而言，由于其基本相似于方法实施方式，所以其描述进行了简化，相关之处可参见方法实施方式的部分说明即可。
本申请实施方式提供的装置、设备和计算机可读存储介质与方法是一一对应的，因此，装置、设备和计算机可读存储介质也具有与其对应的方法类似的有益技术效果，由于上面已经对方法的有益技术效果进行了详细说明，因此，这里不再赘述装置、设备和计算机可读存储介质的有益技术效果。
本领域内的技术人员应明白，本公开的实施方式可提供为方法、装置(设备或系统)、或计算机可读存储介质。因此，本公开可采用完全硬件实施方式、完全软件实施方式、或结合软件和硬件方面的实施方式的形式。而且，本公开可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机可读存储介质的形式。
本公开是参照根据本公开实施方式的方法、装置(设备或系统)、和计算机可读存储介质的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和 /或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
在一个典型的配置中，计算设备包括一个或多个处理器(CPU)、输入/输出接口、网络接口和内存。
内存可能包括计算机可读介质中的非永久性存储器，随机存取存储器(RAM)和/或非易失性内存等形式，如只读存储器(ROM)或闪存(flash RAM)。内存是计算机可读介质的示例。
计算机可读介质包括永久性和非永久性、可移动和非可移动媒体可以由任何方法或技术来实现信息存储。信息可以是计算机可读指令、数据结构、程序的模块或其他数据。计算机的存储介质的例子包括，但不限于相变内存(PRAM)、静态随机存取存储器(SRAM)、动态随机存取存储器(DRAM)、其他类型的随机存取存储器(RAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、快闪记忆体或其他内存技术、只读光盘只读存储器(CD-ROM)、数字多功能光盘(DVD)或其他光学存储、磁 盒式磁带，磁带磁磁盘存储或其他磁性存储设备或任何其他非传输介质，可用于存储可以被计算设备访问的信息。此外，尽管在附图中以特定顺序描述了本公开方法的操作，但是，这并非要求或者暗示必须按照该特定顺序来执行这些操作，或是必须执行全部所示的操作才能实现期望的结果。附加地或备选地，可以省略某些步骤，将多个步骤合并为一个步骤执行，和/或将一个步骤分解为多个步骤执行。
虽然已经参考若干具体实施方式描述了本公开的精神和原理，但是应该理解，本公开并不限于所公开的具体实施方式，对各方面的划分也不意味着这些方面中的特征不能组合以进行受益，这种划分仅是为了表述的方便。本公开旨在涵盖所附权利要求的精神和范围内所包括的各种修改和等同布置。

Claims (14)

  1. 一种数据处理方法，包括：
    获取针对各个数据样本确定的标签残差值；每个所述标签残差值用于表征对应的数据样本的真实标签与目标模型所预测标签之间的偏差程度；
    基于各个所述标签残差值对所述各个数据样本进行残差分解，得到所述各个数据样本对应的模型参数修正量；
    向数据接收方发送所述模型参数修正量，以使得所述数据接收方根据所述模型参数修正量重构所述目标模型的模型参数信息。
  2. 根据权利要求1所述的方法，其中，所述基于各个所述标签残差值对所述各个数据样本进行残差分解，得到所述各个数据样本对应的模型参数修正量，包括：
    按照数据大小的排列顺序将各个所述标签残差值进行排序，得到排序的各个标签残差值；
    在从所述各个数据样本中选取出需要进行残差分解的目标数据样本的情况下，对所述目标数据样本所在排序的标签残差值进行变化，得到所述各个数据样本对应的残差变化信息；
    基于所述各个数据样本对应的残差变化信息以及所述各个数据样本的真实标签，确定所述各个数据样本对应的模型参数修正量。
  3. 根据权利要求2所述的方法，其中，按照如下步骤从所述各个数据样本中选取需要进行残差分解的目标数据样本：
    按照预设分组数量将所述排序的各个标签残差值进行分组，得到分组后的标签残差值；
    从每个分组后的标签残差值中选取预设数量个目标标签残差值；
    将所述目标标签残差值对应的数据样本确定为所述目标数据样本。
  4. 根据权利要求2或3所述的方法，其中，在所述各个数据样本对应的残差变化信息对应残差变化向量的情况下，所述对所述目标数据样本所在排序的标签残差值进行变化，得到所述各个数据样本对应的残差变化信息，包括：
    针对每个所述数据样本，在判断出所述数据样本非所述目标数据样本的情况下，确定所述数据样本对应第一残差变化值；或者，
    在判断出所述数据样本是所述目标数据样本，且所述数据样本的标签残差值大于零的情况下，确定所述数据样本对应第二残差变化值；或者，
    在判断出所述数据样本是所述目标数据样本，且所述数据样本的标签残差值小于零的情况下，确定所述数据样本对应第三残差变化值；
    将各个所述数据样本分别对应的残差变化值进行集合，确定所述残差变化向量；其中，所述残差变化向量用于表征所述各个数据样本的标签残差值是否发生变化。
  5. 根据权利要求4所述的方法，其中，所述基于所述各个数据样本对应的残差变化信息以及所述各个数据样本的真实标签，确定所述各个数据样本对应的模型参数修正量，包括：
    将所述残差变化向量以及所述残差变化向量的转置结果进行点乘运算，确定第一算子；以及，在确定所述各个数据样本的真实标签对应的标签值的情况下，确定与所述标签值对应的第二算子；
    基于所述第一算子和所述第二算子，确定所述各个数据样本对应的模型参数修正量。
  6. 根据权利要求2至5任一项所述的方法，其中，在得到所述各个数据样本对应的残差变化信息之后，所述方法还包括：
    基于所述各个数据样本对应的残差变化信息和所述排序的各个标签残差值之间的乘积运算，确定变化后的标签残差值；
    基于所述变化后的标签残差值确定向所述数据接收方发送的模型参数信息；
    向所述数据接收方发送所述模型参数信息。
  7. 根据权利要求6所述的方法，其中，所述向所述数据接收方发送所述模型参数信息，包括：
    在未达到模型收敛条件的情况下，向所述数据接收方发送所述模型参数信息；
    所述向数据接收方发送所述模型参数修正量，包括：
    在达到模型收敛条件的情况下，向所述数据接收方发送所述模型参数修正量。
  8. 一种数据处理方法，包括：
    接收数据发送方发送的各个数据样本对应的模型参数修正量；其中，所述模型参数修正量是基于各个数据样本确定的标签残差值进行残差分解确定的；每个所述标签残差值用于表征对应的数据样本的真实标签与目标模型所预测标签之间的偏差程度；
    根据所述模型参数修正量重构所述目标模型的模型参数信息。
  9. 根据权利要求8所述的方法，其中，所述根据所述模型参数修正量重构所述目标模型的模型参数信息，包括：
    获取针对目标模型输入的数据特征信息；
    基于所述数据特征信息和所述模型参数修正量，确定所述目标模型的模型参数信息。
  10. 根据权利要求9所述的方法，其中，在所述数据特征信息包括数据特征向量的情况下，所述基于所述数据特征信息和所述模型参数修正量，确定所述目标模型的模型参数信息，包括：
    将所述数据特征向量以及所述数据特征向量的转置结果进行点乘运算，确定第三算子；以及，将所述数据特征向量的转置结果以及所述数据特征向量进行点乘运算，确定第四算子；
    基于所述第三算子、所述第四算子以及所述模型参数修正量，确定所述目标模型的模型参数信息。
  11. 一种数据处理装置，包括：
    获取模块，用于获取针对各个数据样本确定的标签残差值；每个所述标签残差值用于表征对应的数据样本的真实标签与目标模型所预测标签之间的偏差程度；
    分解模块，用于基于各个所述标签残差值对所述各个数据样本进行残差分解，得到所述各个数据样本对应的模型参数修正量；
    发送模块，用于向数据接收方发送所述模型参数修正量，以使得所述数据接收方根据所述模型参数修正量重构所述目标模型的模型参数信息。
  12. 一种数据处理装置，包括：
    接收模块，用于接收数据发送方发送的各个数据样本对应的模型参数修正量；其中，所述模型参数修正量是基于各个数据样本确定的标签残差值进行残差分解确定的；每个所述标签残差值用于表征对应的数据样本的真实标签与目标模型所预测标签之间的偏差程度；
    重构模块，用于根据所述模型参数修正量重构所述目标模型的模型参数信息。
  13. 一种电子设备，包括：处理器、存储器和总线，所述存储器存储有所述处理器可执行的机器可读指令，当电子设备运行时，所述处理器与所述存储器之间通过总线通信，所述机器可读指令被所述处理器执行时执行如权利要求1至10任一项所述的数据处理方法。
  14. 一种计算机可读存储介质，该计算机可读存储介质上存储有计算 机程序，该计算机程序被处理器运行时执行如权利要求1至10任一项所述的数据处理方法。
PCT/CN2023/071564 2022-08-15 2023-01-10 一种数据处理方法、装置、电子设备及存储介质 Ceased WO2024036880A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US19/104,157 US12572694B2 (en) 2022-08-15 2023-01-10 Data processing method and apparatus, electronic device, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210975564.1 2022-08-15
CN202210975564.1A CN115422574A (zh) 2022-08-15 2022-08-15 一种数据处理方法、装置、电子设备及存储介质

Publications (1)

Publication Number Publication Date
WO2024036880A1 true WO2024036880A1 (zh) 2024-02-22

Family

ID=84197742

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/071564 Ceased WO2024036880A1 (zh) 2022-08-15 2023-01-10 一种数据处理方法、装置、电子设备及存储介质

Country Status (4)

Country Link
US (1) US12572694B2 (zh)
CN (1) CN115422574A (zh)
TW (1) TWI878788B (zh)
WO (1) WO2024036880A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12572694B2 (en) 2022-08-15 2026-03-10 China Unionpay Co., Ltd. Data processing method and apparatus, electronic device, and storage medium

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117009775A (zh) * 2023-09-28 2023-11-07 苏州元脑智能科技有限公司 模型训练数据获取方法、模型训练方法和装置

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200143080A1 (en) * 2017-07-07 2020-05-07 Advanced New Technologies Co., Ltd. Privacy protection based training sample generation method and device
CN111435452A (zh) * 2019-01-11 2020-07-21 百度在线网络技术（北京）有限公司 模型训练方法、装置、设备和介质
CN112347476A (zh) * 2020-11-13 2021-02-09 脸萌有限公司 数据保护方法、装置、介质及设备
CN114611128A (zh) * 2022-03-15 2022-06-10 北京字节跳动网络技术有限公司 一种纵向联邦学习方法、装置、系统、设备及存储介质
CN114723068A (zh) * 2022-04-15 2022-07-08 京东科技控股股份有限公司 联邦模型训练方法及装置
CN115422574A (zh) * 2022-08-15 2022-12-02 中国银联股份有限公司 一种数据处理方法、装置、电子设备及存储介质

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11972329B2 (en) * 2018-12-31 2024-04-30 Xerox Corporation Method and system for similarity-based multi-label learning
CN110990857B (zh) * 2019-12-11 2021-04-06 支付宝(杭州)信息技术有限公司 保护隐私安全的多方联合进行特征评估的方法及装置
CN113065610B (zh) * 2019-12-12 2022-05-17 支付宝(杭州)信息技术有限公司 基于联邦学习的孤立森林模型构建和预测方法和装置
CN115081640B (zh) * 2020-12-06 2024-10-18 支付宝(杭州)信息技术有限公司 基于差分隐私的联邦学习方法、装置及电子设备
CN113159310A (zh) * 2020-12-21 2021-07-23 江西理工大学 一种基于残差稀疏宽度学习系统的入侵检测方法
CN113807538B (zh) * 2021-04-09 2024-02-06 京东科技控股股份有限公司 联邦学习方法、装置、电子设备及存储介质
CN114785608B (zh) * 2022-05-09 2023-08-15 中国石油大学(华东) 一种基于去中心化联邦学习的工业控制网络入侵检测方法
US12503329B2 (en) 2022-07-22 2025-12-23 Ricoh Company, Ltd. Image forming apparatus

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200143080A1 (en) * 2017-07-07 2020-05-07 Advanced New Technologies Co., Ltd. Privacy protection based training sample generation method and device
CN111435452A (zh) * 2019-01-11 2020-07-21 百度在线网络技术（北京）有限公司 模型训练方法、装置、设备和介质
CN112347476A (zh) * 2020-11-13 2021-02-09 脸萌有限公司 数据保护方法、装置、介质及设备
CN114611128A (zh) * 2022-03-15 2022-06-10 北京字节跳动网络技术有限公司 一种纵向联邦学习方法、装置、系统、设备及存储介质
CN114723068A (zh) * 2022-04-15 2022-07-08 京东科技控股股份有限公司 联邦模型训练方法及装置
CN115422574A (zh) * 2022-08-15 2022-12-02 中国银联股份有限公司 一种数据处理方法、装置、电子设备及存储介质

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12572694B2 (en) 2022-08-15 2026-03-10 China Unionpay Co., Ltd. Data processing method and apparatus, electronic device, and storage medium

Also Published As

Publication number Publication date
TWI878788B (zh) 2025-04-01
US20250258959A1 (en) 2025-08-14
TW202409870A (zh) 2024-03-01
US12572694B2 (en) 2026-03-10
CN115422574A (zh) 2022-12-02

Similar Documents

Publication Publication Date Title
Li et al. Privacy-preserving feature selection with secure multiparty computation
CN112085159B (zh) 一种用户标签数据预测系统、方法、装置及电子设备
Fenner et al. Privacy-preserving Gaussian process regression–a modular approach to the application of homomorphic encryption
WO2024036880A1 (zh) 一种数据处理方法、装置、电子设备及存储介质
CN116094686B (zh) 用于量子卷积计算的同态加密方法、系统、设备及终端
Bi et al. Outsourced and privacy-preserving collaborative k-prototype clustering for mixed data via additive secret sharing
Tang et al. IHVFL: a privacy-enhanced intention-hiding vertical federated learning framework for medical data
Cai et al. Privacy‐preserving CNN feature extraction and retrieval over medical images
Cao et al. Neural link prediction over aligned networks
Chen et al. Privacy-preserving SVM on outsourced genomic data via secure multi-party computation
Dong et al. FlexBNN: Fast private binary neural network inference with flexible bit-width
Zhang et al. Fast robust location and scatter estimation: a depth-based method
Jia et al. Privacy‐Preserving Blockchain‐Based Nonlinear SVM Classifier Training for Social Networks
Luo et al. Privacy-preserving federated learning with hierarchical clustering to improve training on non-iid data
Yu et al. Quantum-aided secure deep neural network inference on real quantum computers
Ünal et al. A privacy-preserving approach for cloud-based protein fold recognition
CN112101609B (zh) 关于用户还款及时性的预测系统、方法、装置及电子设备
Ge et al. Practical two-party privacy-preserving neural network based on secret sharing
Rajasekar et al. Algorithmic strategies for solving complex problems in financial cryptography
Zhang et al. Functionality and data stealing by pseudo-client attack and target defenses in split learning
Sasy et al. PRAC: round-efficient 3-party MPC for dynamic data structures
Galvan et al. An alternating augmented Lagrangian method for constrained nonconvex optimization
Sun et al. A secure distributed machine learning protocol against static semi-honest adversaries
Song et al. Private Learning for Vertical Decision Trees: A Secure, Accurate, and Fast Realization
HK40077934A (zh) 一种数据处理方法、装置、电子设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23853801

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 23853801

Country of ref document: EP

Kind code of ref document: A1