WO2023216899A1 - 用于模型性能评估的方法、装置、设备和介质 - Google Patents

用于模型性能评估的方法、装置、设备和介质 Download PDF

Info

Publication number
WO2023216899A1
WO2023216899A1 PCT/CN2023/091142 CN2023091142W WO2023216899A1 WO 2023216899 A1 WO2023216899 A1 WO 2023216899A1 CN 2023091142 W CN2023091142 W CN 2023091142W WO 2023216899 A1 WO2023216899 A1 WO 2023216899A1
Authority
WO
WIPO (PCT)
Prior art keywords
protected
labels
category
prediction
scores
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2023/091142
Other languages
English (en)
French (fr)
Inventor
孙建凯
杨鑫
王崇
解浚源
吴迪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing ByteDance Network Technology Co Ltd
Lemon Inc Cayman Island
Original Assignee
Beijing ByteDance Network Technology Co Ltd
Lemon Inc Cayman Island
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing ByteDance Network Technology Co Ltd, Lemon Inc Cayman Island filed Critical Beijing ByteDance Network Technology Co Ltd
Priority to US18/865,611 priority Critical patent/US20250335327A1/en
Publication of WO2023216899A1 publication Critical patent/WO2023216899A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
    • G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
    • G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3466—Performance evaluation by tracing or monitoring
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/20—Ensemble learning

Definitions

  • Example embodiments of the present disclosure relate generally to the field of computers, and in particular to methods, apparatus, devices and computer-readable storage media for model performance evaluation.
  • a scheme for model performance evaluation is provided.
  • a method for model performance evaluation includes, at a client node, obtaining multiple prediction scores output by a machine learning model for multiple data samples, the multiple prediction scores respectively indicating predicted probabilities that the multiple data samples belong to the first category or the second category; based on random responses Mechanism to modify multiple ground-truth labels to obtain multiple protected labels. Multiple ground-truth labels respectively mark multiple data samples as belonging to the first category or to the second category; based on multiple protected labels and multiple prediction scores , determining error metric information related to a predetermined performance indicator of the machine learning model; and sending the error metric information to the service node.
  • a method for model performance evaluation includes receiving, at a service node, error metric information related to predetermined performance indicators of a machine learning model from a plurality of client nodes, respectively, the error metric information being determined by the respective client nodes based on a plurality of respective protected labels.
  • the plurality of protected tags are generated by applying a random response mechanism to a plurality of true value tags; based on the error metric information, an error value of the predetermined performance indicator is determined; and the error value is determined by correcting the error value. Corrected values for predetermined performance indicators.
  • an apparatus for model performance evaluation includes a score obtaining module configured to obtain a plurality of prediction scores output by the machine learning model for a plurality of data samples, the plurality of prediction scores respectively indicating predicted probabilities that the plurality of data samples belong to the first category or the second category; label modification A module configured to modify multiple truth labels based on a random response mechanism to obtain multiple protected labels.
  • the multiple truth labels respectively mark whether multiple data samples belong to the first category or belong to the second category; information determination module , configured to determine error metric information related to predetermined performance indicators of the machine learning model based on the plurality of protected labels and the plurality of prediction scores; and an information sending module configured to send the error metric information to the service node.
  • an apparatus for model performance evaluation includes: an information receiving module configured to receive error metric information related to predetermined performance indicators of the machine learning model from a plurality of client nodes respectively, the error metric information being generated by the corresponding client nodes based on a plurality of respective protected labels. Determining, a plurality of protected labels are generated by applying a random response mechanism to a plurality of true value labels; an indicator determination module is configured to determine an error value of a predetermined performance indicator based on the error measurement information; and an indicator correction module is configured to pass The error values are corrected to determine corrected values for predetermined performance indicators.
  • an electronic device in a fifth aspect of the present disclosure, includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit.
  • the instructions when executed by at least one processing unit, cause the device to perform the method of the first aspect.
  • an electronic device in a sixth aspect of the present disclosure, includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit.
  • the instructions when executed by at least one processing unit, cause the device to perform the method of the second aspect.
  • a computer-readable storage medium is provided.
  • a computer program is stored on the medium, and the computer program is executed by the processor to implement the method of the first aspect.
  • a computer-readable storage medium is provided.
  • a computer program is stored on the medium, and the computer program is executed by the processor to implement the method of the second aspect.
  • Figure 1 shows a schematic diagram of an example environment in which embodiments of the present disclosure can be applied
  • Figure 2 illustrates a flow diagram of signaling flow for model performance evaluation according to some embodiments of the present disclosure
  • Figure 3 shows a schematic diagram of an example of applying a random response mechanism to a truth label according to some embodiments of the present disclosure
  • FIG. 4 illustrates a flowchart of a process for type performance evaluation at a client node in accordance with some embodiments of the present disclosure
  • Figure 5 illustrates a flowchart of a process for model performance evaluation at a service node in accordance with some embodiments of the present disclosure
  • FIG. 6 illustrates a block diagram of an apparatus for model performance evaluation at a client node in accordance with some embodiments of the present disclosure
  • FIG. 7 illustrates a block diagram of an apparatus for model performance evaluation at a service node in accordance with some embodiments of the present disclosure.
  • FIG. 8 illustrates a block diagram of a computing device/system capable of implementing one or more embodiments of the present disclosure.
  • a prompt message is sent to the user to clearly remind the user that the operation requested will require the acquisition and use of the user's personal information. Therefore, users can autonomously choose whether to provide personal information to software or hardware such as electronic devices, applications, servers or storage media that perform the operations of the technical solution of the present disclosure based on the prompt information.
  • the method of sending prompt information to the user can be, for example, a pop-up window, and the prompt information can be presented in the form of text in the pop-up window.
  • the pop-up window can also host a selection control for the user to choose "agree” or "disagree” to provide personal information to the electronic device.
  • model can learn the association between the corresponding input and output from the training data, so that the corresponding output can be generated for the given input after the training is completed. Model generation can be based on machine learning techniques. Deep learning is a machine learning algorithm that uses multiple layers of processing units to process inputs and provide corresponding outputs. Neural network models are an example of deep learning-based models. In this article, a “model” may also be called a “machine learning model,” “learning model,” “machine learning network,” or “learning network,” and these terms are used interchangeably in this article.
  • a "neural network” is a machine learning network based on deep learning. Neural networks are capable of processing inputs and providing corresponding outputs, and typically include an input layer and an output layer and one or more hidden layers between the input layer and the output layer. Neural networks used in deep learning applications often include many hidden layers, thereby increasing the depth of the network.
  • the layers of a neural network are connected in sequence such that the output of the previous layer is provided as the input of the subsequent layer, where the input layer receives the input of the neural network and the output of the output layer serves as the final output of the neural network.
  • Each layer of a neural network consists of one or more nodes (also called processing nodes or neurons), each processing input from the previous layer.
  • machine learning can roughly include three stages, namely the training stage, the testing stage and the application stage (also called the inference stage).
  • the training phase a given model can be trained using a large amount of training data, and parameter values are updated iteratively until the model can obtain consistent inferences from the training data that meet the expected goals.
  • the model can be thought of as being able to learn the association between inputs and outputs (also known as input-to-output mapping) from the training data.
  • the parameter values of the trained model are determined.
  • test inputs are applied to the trained model to test whether the model can provide the correct output, thereby determining the performance of the model.
  • the model can be used based on the parameter values obtained from training, Process the actual input and determine the corresponding output.
  • FIG. 1 shows a schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented.
  • Client nodes 110-1...110-k,...110-N can maintain respective local data sets 112-1...112-k,...112-N respectively.
  • client nodes 110-1...110-k,...110-N may be collectively or individually referred to as client nodes 110
  • local data sets 112-1...112-k,...112- N may be referred to collectively or individually as local data set 112 .
  • the client node 110 and/or the service node 120 may be implemented at a terminal device or a server.
  • the terminal device can be any type of mobile terminal, fixed terminal or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, personal communication system (PCS) devices , personal navigation device, personal digital assistant (PDA), audio/video player, digital camera/camcorder, positioning device, television receiver, radio receiver, e-book device, gaming device, or any combination of the foregoing, including Accessories and peripherals for these devices or any combination thereof.
  • the terminal device is also able to support any type of interface to the user (such as "wearable" circuitry, etc.).
  • Servers are various types of computing systems/servers capable of providing computing capabilities, including but not limited to mainframes, edge computing nodes, computing devices in cloud environments, and so on.
  • the client node refers to the node that provides part of the training data for the machine learning model.
  • the client node may also be called a client, a terminal node, a terminal device, a user device, etc.
  • a service node refers to a node that aggregates training results at client nodes.
  • N client nodes 110 jointly participate in training the machine learning model 130 and aggregate the intermediate results in the training to the service node 120 so that the service node 120 updates the parameter set of the machine learning model 130 .
  • the complete set of local data for these client nodes 110 constitutes the complete training data set for the machine learning model 130 . Therefore, according to the federated learning mechanism, the service node 120 will generate a global machine learning model 130.
  • the local data set 112 at the client node 110 may include data samples and ground truth labels.
  • Figure 1 specifically shows a local data set 112-k at a certain client node 110-k, which includes a data sample set and a ground truth label set.
  • the data sample set includes a plurality (M) of data samples 102-1, 102-i, ... 102-M (collectively or individually referred to as data samples 102), and the true value label set includes a corresponding plurality (M ) Ground-truth label (ground-truth label) 105-1, 105-i,...105-M (collectively or individually referred to as ground-truth label 105).
  • Each data sample 102 may be annotated with a corresponding ground truth label 105 .
  • Data samples 102 may correspond to inputs to machine learning model 130, with ground truth labels 105 indicating the true output of the corresponding data samples 102.
  • Ground truth labels are an important part of supervised machine learning.
  • the machine learning model 130 may be built based on various machine learning or deep learning model architectures, and may be configured to implement various prediction tasks, such as various classification tasks, recommendation tasks, and so on.
  • the machine learning model 130 may also be called a prediction model, a recommendation model, a classification model, etc.
  • Data samples 102 may include input information related to a specific task of the machine learning model 130, with truth labels 105 related to the desired output of the task.
  • the machine learning model 130 may be configured to predict whether the input data sample belongs to the first category or the second category, and the ground truth label is used to label whether the data sample actually belongs to the first category or the second category.
  • Category 2 Many practical applications can be classified as such two-category tasks, such as whether the recommended items are converted (for example, clicks, purchases, registrations or other demand behaviors) in the recommendation task, etc.
  • Figure 1 only shows an example federated learning environment. Depending on the federated learning algorithm and actual application needs, the environment can also be different.
  • the service node 120 may serve as a client node in addition to serving as a central node to provide partial data for model training, model performance evaluation, etc. Embodiments of the present disclosure are not limited in this respect.
  • the client node 110 does not need to disclose local data samples or label data, but sends gradient data calculated based on local training data to the service node 120 so that the service node 120 can update the parameters of the machine learning model 130 set.
  • the performance of a machine learning model can be measured through one or more performance metrics. Different performance indicators can measure the difference between the predicted output given by the machine learning model for the data sample set and the real output indicated by the true value label set from different perspectives. Generally, if the difference between the predicted output given by the machine learning model and the real output is small, it means that the performance of the machine learning model is better. It can be seen that it is usually necessary to determine the performance indicators of the machine learning model based on the set of ground-truth labels of the data samples.
  • the requirements for data privacy protection are also getting higher and higher, including the need to protect the true value labels of data samples to avoid leakage.
  • the user's real conversion behavior of the recommended items involves user privacy, which is sensitive information and needs to be protected.
  • a model performance evaluation solution which can protect label data local to a client node.
  • the true value label set corresponding to the data sample set is modified by applying a Randomized Response (RR) mechanism to obtain a protected label set.
  • the client node determines metric information related to the performance indicators of the machine learning model based on the protected label set and the prediction score output by the machine learning model for the data sample set.
  • the label set is a modified protected label set
  • the determined metric information is not accurate metric information and is called "error metric information”.
  • the client node sends error metric information to the service node.
  • the service node receives their respective error metric information from the plurality of client nodes and determines an error value of the performance indicator based on the error metric information. The service node further corrects the error value to obtain the corrected value of the performance index.
  • each client node does not need to expose the local set of true value labels, and the service node can also calculate the value of the performance indicator based on the feedback information of the client node. In this way, while achieving model performance evaluation, the purpose of privacy protection for the local label data of the client node is achieved.
  • FIG. 2 illustrates a schematic block diagram of signaling flow 200 for model performance evaluation in accordance with some embodiments of the present disclosure. For ease of discussion, reference is made to environment 100 of FIG. 1 .
  • Signaling flow 200 involves client node 110 and service node 120.
  • the machine learning model 130 to be evaluated may be a global machine learning model determined based on the training process of federated learning.
  • the client node 110 and the service node 120 participate in the training process of the machine learning model 130 .
  • the machine learning model 130 may also be a model obtained in any other manner, and the client node 110 and the service node 120 may not participate in the training process of the machine learning model 130 .
  • the scope of the present disclosure is not limited in this regard.
  • service node 120 sends 205 machine learning model 130 to N client nodes 110.
  • each client node 110 may perform a subsequent evaluation process based on the machine learning model 130.
  • the machine learning model 130 to be evaluated may also be provided to the client node 110 in any other suitable manner.
  • operations measured by a client node are described from the perspective of a single client node.
  • the client node 110 obtains 215 a plurality of prediction scores output by the machine learning model 130 for a plurality of data samples 102 .
  • Each prediction score may indicate a predicted probability that the corresponding data sample 102 belongs to the first category or the second category. Both categories can be configured according to actual task needs.
  • the value range of the prediction score output by the machine learning model 130 can be set arbitrarily.
  • the prediction score can be a value in a certain continuous value interval (for example, a value between 0 and 1), or it can be one of multiple discrete values (for example, it can be 0, One of discrete values such as 1, 2, 3, 4, 5).
  • a higher prediction score may indicate that the data sample 102 has a greater predicted probability of belonging to the first category and a smaller predicted probability of belonging to the second category.
  • the opposite setting is also possible.
  • a higher prediction score may indicate a greater prediction probability that the data sample 102 belongs to the second category, and a smaller prediction probability that the data sample 102 belongs to the first category.
  • the client node 110 also modifies multiple true value labels 105 (also called true value labels) corresponding to each of the more than 220 data samples 102 based on a random response mechanism to obtain multiple protected labels.
  • the truth label 105 is used to label whether the corresponding data sample 102 belongs to the first category or the second category.
  • data samples belonging to the first category are sometimes called positive samples, positive examples, or positive class samples
  • data samples belonging to the second category are sometimes called negative samples, negative examples, or negative class samples.
  • each truth label 105 may have one of two values, indicating the first category or the second category respectively.
  • the value of the true value label 105 corresponding to the first category may be set to “1”, which indicates that the data sample belongs to the first category and is a positive sample.
  • the value of the ground truth label 105 corresponding to the second category can be set to “0”, which indicates that the data sample belongs to the second category and is a negative sample.
  • the true value label in order to achieve privacy protection of the true value label while determining the performance index of the machine learning model 130, is converted into a protected label through a random response mechanism.
  • Figure 3 shows an example of a protected label obtained after applying a random response mechanism to the true value label 105 according to some embodiments of the present disclosure.
  • the M true value labels 105 corresponding to the M data samples 102 will correspond to the protected labels 305-1,...305-i,...305-M (collectively as or individually referred to as a protected tag 305).
  • the random response mechanism is one of the Differential Privacy (DP) mechanisms.
  • DP Differential Privacy
  • ⁇ and ⁇ are real numbers greater than or equal to 0, that is, ⁇ , and It is a random mechanism (random algorithm).
  • the so-called random mechanism refers to that for a specific input, the output of the mechanism is not a fixed value, but obeys a certain distribution.
  • For the random mechanism It can be considered a random mechanism if the following conditions are met With ( ⁇ , ⁇ )-differential privacy: for any two adjacent training data sets D, D′, and for An arbitrary subset S of possible outputs exists:
  • the random mechanism can also be considered With ⁇ -differential privacy ( ⁇ -DP).
  • ⁇ -DP ⁇ -differential privacy
  • differential privacy mechanisms for random mechanisms with ( ⁇ , ⁇ )-differential privacy or ⁇ -differential privacy It is expected that the distribution of the two outputs obtained after acting on two adjacent data sets respectively is indistinguishable. In this case, observers can hardly detect small changes in the input data set of the algorithm by observing the output results, thus achieving the purpose of protecting privacy. If the random mechanism If applied to any adjacent data set, the probability of obtaining a specific output S is almost the same, then it will be considered that the algorithm is difficult to achieve the effect of differential privacy.
  • label differential privacy can be defined. Specifically, assume that ⁇ and ⁇ are real numbers greater than or equal to 0, that is, and It is a random mechanism (random algorithm). It can be considered a random mechanism if the following conditions are met With ( ⁇ , ⁇ )-label differential privacy (label differential privacy): For any two adjacent training data sets D, D′, their difference is only that the label of a single data sample is different, and for An arbitrary subset S of possible outputs exists:
  • the random mechanism can also be considered With ⁇ -differential privacy ( ⁇ -DP).
  • ⁇ -DP ⁇ -differential privacy
  • the random response mechanism is a random mechanism applied for the purpose of differential privacy protection.
  • the random response mechanism is positioned as follows: Suppose ⁇ is a parameter, and y ⁇ [0, 1] is a known value of the truth label in the random response mechanism. If for the value y of the true value label, the random response mechanism derives a random value from the following probability distribution
  • the random response mechanism After applying the random response mechanism, the random value There is a certain probability that it is equal to y, and there is also a certain probability that it is not equal to y.
  • the random response mechanism will satisfy ⁇ -differential privacy.
  • the protected tag 305 may also sometimes be called a noise tag or an interfering tag.
  • the truth label 105 of the i-th data sample 102 at the client node 110-k is represented as Protected tag 305 is represented as
  • the values of some truth labels 105 may be changed (i.e., ), some truth labels 105 may remain unchanged (i.e., ).
  • is the number of data samples of the client node 110-k.
  • the change to the truth label 105 can be considered as reversing the value of the truth label 105 . For example, if the truth label 105 The value is 1. After inversion, the protected label is 305 The value of is 0.
  • the client node 110 determines 225 metric information related to a predetermined performance indicator of the machine learning model 130 .
  • the metric information determined here is not an accurate metric based on a modified set of protected tags and is referred to as an "erroneous metric.”
  • individual client nodes 110 determine metric information related to performance indicators of the model based on local data sets (data samples and ground truth labels). Metric information for multiple client nodes 110 may be aggregated to service node 120 . In this way, the performance of the machine learning model 130 is evaluated based on the complete data set of multiple client nodes.
  • the type of error metric information provided by the client node may depend on the performance metrics to be calculated, and on whether the client node 110 is to provide the protected label 305 to the service node.
  • the prediction score given by the machine learning model 130 for a certain data sample is usually compared with a certain score threshold, and based on the comparison result, it is determined whether the data sample is predicted to belong to the first category or the second category. There are four possible outcomes in the prediction of the machine learning model 130 used to implement the binary classification task.
  • the true value label 105 indicates that it belongs to the first category (positive sample)
  • the machine learning model 130 also predicts that it is a positive sample
  • the data sample is considered to be a true sample (True Positive, TP).
  • the true value label 105 indicates that it belongs to the first category (positive sample) and the machine learning model 130 predicts that it is a negative sample
  • the data sample is considered to be a false negative sample (False Negative, FN).
  • the truth label 105 indicates that it belongs to the second category (negative sample)
  • the machine learning model 130 also predicts that it is a negative sample
  • the data sample is considered to be a True Negative (TN).
  • the performance index can be calculated based on the prediction results of the complete set of data samples of multiple client nodes 110 and the complete set of ground truth labels.
  • the performance metric of the machine learning model 130 may include the area under the curve (AUC) of the receiver operating characteristic curve (ROC).
  • the ROC curve is a curve drawn on the coordinate axis based on different classification methods (setting different score thresholds), with the false positive sample ratio (FPR) as the X-axis and the true sample ratio (TPR) as the Y-axis.
  • AUC refers to the area under the ROC curve.
  • AUC can be calculated by calculating the area under the ROC curve with an approximate algorithm.
  • the AUC may also be determined from a probabilistic perspective.
  • AUC can be thought of as: randomly selecting a positive sample and a negative sample, the probability that the machine learning model gives the positive sample a higher prediction score than the negative sample. That is to say, in the data sample set, positive and negative samples are combined to form a positive and negative sample pair, in which the prediction score of the positive sample is greater than the prediction score of the negative sample. If the model can give more positive samples a higher prediction score than the negative samples, it can be considered that the AUC is higher and the model has better performance.
  • the value range of AUC is between 0.5 and 1. The closer the AUC is to 1, the better the performance of the model.
  • the performance indicators of the machine learning model 130 may also include a P-R curve, which has recall as the horizontal axis and precision as the vertical axis. The closer the P-R curve is to the upper right corner, the better the performance of the model. The area under the curve is called the AP score (Average Precision Score).
  • the client node 110 after determining the error metric information, the client node 110 sends 230 the determined error metric information to the service node 120 .
  • the client node 110 may choose to send multiple protected labels 305 to the service node as part of the error metric information, or may choose not to send the protected labels 305, but continue further on this basis.
  • the value of the metric parameter may be chosen to send multiple protected labels 305 to the service node as part of the error metric information, or may choose not to send the protected labels 305, but continue further on this basis.
  • client node 110 may directly determine the plurality of prediction scores and the plurality of protected labels 305 as error metric information, and sent to service node 120. As shown in FIG. 2 , in the error metric information sending method 236 , the client node 110 sends more than 240 prediction scores and a plurality of protected labels to the service node 120 . Thus, the service node 120 may receive 242 the predicted score and protected label to the client node 110 . In these embodiments, for each data sample 102, the corresponding prediction score and protected label may be sent to the service node 120 in pairs.
  • Figure 2 also shows another way of sending error metric information 238.
  • the client node 110 may determine the plurality of prediction scores as a first portion of the error metric information and send 244 this portion of the information to the service node 120 .
  • the client node 110 may randomly adjust the order of the multiple predicted scores and send the multiple predicted scores to the service node in the adjusted order.
  • the output prediction scores have a certain order, such as from large to small or from small to large. , which may lead to certain information leakage. Random sequence adjustment can further enhance data privacy protection.
  • the service node 120 sorts 248 the prediction score sets from the plurality of client nodes 110 to obtain the ranking result of the prediction scores from each client node 110 in the prediction score set.
  • the service node 120 may sort the set of predicted scores in ascending order and assign (Predicted score of i-th data sample of client node 110-k) Assign ranking value
  • the sorting value Can indicate predicted score The number of other predicted scores exceeded in the set of predicted scores. For example, in ascending order, the lowest predicted score is assigned a rank value of 0, indicating that it does not exceed (larger than) any other predicted score; the next predicted score is assigned a rank value of 1, indicating that it is greater than 1 predicted score in the set, to And so on. Such assignment of sorted values facilitates subsequent calculations.
  • the service node 120 sends 250 the sorting results of its multiple prediction scores in the overall prediction score set to the corresponding client node 110.
  • the client node 110 may determine 254 the second portion of the error metric information based on the ranking results for each of the local plurality of protected tags 305 and the plurality of prediction scores.
  • the second part of the error metric information refers to the values of the metric parameters required to calculate a specific performance metric of the machine learning model 130 in addition to the prediction score.
  • client node 110 may determine the number of protected tags of the first type in the plurality of protected tags 305 (referred to as the "first number"), where the first type of protected tags are The label 305 indicates that the corresponding data sample 102 belongs to the first category, for example, indicates that the data sample 102 is a positive sample.
  • the client node 110 may also determine the number of second-type protected labels (referred to as the “second number”) among the plurality of protected labels 305, where the second-type protected labels indicate that the corresponding data sample belongs to the The second category, for example, indicates that the data sample is a negative sample.
  • the determination of the first number and the second number may be expressed as follows:
  • the client node 110 may also determine, based on the respective sorting results of multiple prediction scores, the prediction score of the data sample (ie, the positive sample) corresponding to the first type of protected label that exceeds the prediction score set.
  • the number of points scored (called the third number). This number may indicate the number of sample pairs in the set of data samples of the client node 110 for which positive samples are ranked higher than the remaining samples (in the case of ascending order).
  • the third number may be determined by:
  • localSum k represents the third number, Represents the value of the protected label corresponding to the i-th data sample, Indicates the ranking value of the prediction score corresponding to the i-th data sample.
  • the sort value Can be set to indicate the predicted score The number of other predicted scores exceeded in the set of predicted scores.
  • the value of is 1, for negative samples, The value is 0. In this way, by The sum of can determine the number of samples in which the prediction score ranking of the positive sample exceeds the prediction score ranking of the remaining samples (also the number of such prediction scores).
  • localSum k may be determined as the value of another metric parameter (error value) in the error metric information at client node 110-k.
  • Client node 110 may send 256 the values of these three metric parameters to service node 120 as a second part of the error metric information. After receiving 258 the second part of the error metric information, the service node 120 may perform subsequent operations accordingly.
  • different client nodes 110 may choose mode 236 or mode 238 to send respective error metric information to the service node 120 .
  • the true value label can obtain privacy protection. This is because the stochastic response mechanism is immune to post-processing. In other words, after applying the random response mechanism, no matter how the protected label and its related statistical information are subsequently processed, that is, regardless of whether the protected label data is sent from the client node, the differential privacy protection capability will not be eliminated.
  • the service node 120 After receiving 235 the error metric information sent by each client node 110, the service node 120 determines 260 the value of the performance indicator of the machine learning model 130 based on the error metric information from the plurality of client nodes 110.
  • the determined value of the performance index is also called an error value.
  • the calculation of performance indicators depends on the measurement information obtained and the type of performance indicators to be determined.
  • AUC there are also different algorithms that can be used for flexible determination.
  • the service node 120 may send the multiple client nodes 110
  • the values of these measurement parameters are aggregated by parameters to obtain the aggregate value (global value) of each measurement parameter, as follows:
  • first total number Indicates the total number of second-type protected labels (labels indicating negative samples) among all protected labels of the plurality of client nodes 110 (referred to as the "second total number")
  • second total number Indicates the total number of second-type protected labels (labels indicating negative samples) among all protected labels of the plurality of client nodes 110
  • localSum represents the number of protected labels in the first type
  • the predicted score of the corresponding data sample exceeds the third total number of predicted scores in the set of predicted scores. Since all statistics are based on protected tags, and globalSum may differ from values calculated based on the ground truth labels of client nodes.
  • the service node 120 may transfer these clients to The predicted scores and protected labels for the end node 110 are aggregated and tallied directly in a similar manner as discussed above for the client node and globalSum.
  • the error metric information received by the service node 120 from a certain client node 110 or a certain part of the client nodes 110 is a predicted score and a protected label, for example, such error metric information is received through manner 236, then
  • the service node 120 calculates localP k , localN k , and localSum k corresponding to each client node in a similar manner as discussed above for client nodes.
  • the service node 120 can also bring together the predicted scores and protected labels of some client nodes 110, and calculate the number of first-type protected labels corresponding to these client nodes in a similar manner as discussed above for client nodes.
  • the service node 120 aggregates the statistical information with localP k , localN k and localSum k received directly from other client nodes, thereby determining and globalSum.
  • the service node 120 can calculate the value of AUC in the following way (the value calculated here is the error value, expressed as AUC_corr):
  • the service node 120 may also calculate the AUC through other methods. Specifically, the service node 120 may aggregate the received prediction scores and protected labels. The service node 120 may determine the number of positive samples indicated by the protected labels and the number of negative samples indicated by the protected labels in the set of protected labels. In addition, the service node 120 may determine, based on the prediction score set, the number of prediction scores of positive samples that are greater than the prediction scores of negative samples among all data samples. The service node 120 can then calculate the value of the AUC (ie, the error value) based on these three numbers.
  • the AUC ie, the error value
  • the total number of data samples at N client nodes 110 is L, and the number of positive samples indicated by the protected labels is m and the number of negative samples is n.
  • the prediction score corresponding to each data sample is s i , i ⁇ [1,L].
  • AUC can also be determined from a probabilistic and statistical perspective based on other methods.
  • performance metrics of the machine learning model 130 may be evaluated in addition to AUC, as long as such performance metrics can be determined from multiple prediction scores and multiple protected labels. Embodiments of the present disclosure are not limited in this respect.
  • the service node 120 determines 265 the correction value of the predetermined performance index by correcting the error value.
  • the mapping relationship between the error value and the correction value of the performance indicator can be determined, and the error value can be corrected based on this.
  • the mapping relationship between error values and correction values may be based on a first total number of protected labels of the first type and a second type of protected labels in the set of protected labels involved in the N client nodes 110 Determined by the second total number of protected tags.
  • AUC_corr the mapping relationship between the error value of AUC (AUC_corr) and the correction value (denoted as AUC_real) can be expressed as follows:
  • N are the numbers of positive and negative samples in the data sample set indicated by the ground truth labels, is a first total number of protected labels of the first type and a second total number of protected labels of the second type determined from the error metric information provided by the client node 110 . can be sure That is, the total number of samples or labels remains unchanged. In addition, it can be determined From these two equations, we can get:
  • AUC_real can be calculated from AUC_corr when ⁇ , ⁇ + and ⁇ - are known.
  • values for other performance metrics may also be calculated.
  • the service node 120 can also correct the error values of these performance indicators by setting other mapping relationships to obtain more accurate performance indicator values.
  • FIG. 4 illustrates a flow diagram of a process 400 for model performance evaluation at a client node, in accordance with some embodiments of the present disclosure.
  • Process 400 may be implemented at client node 110.
  • the client node 110 obtains a plurality of prediction scores output by the machine learning model for a plurality of data samples.
  • the plurality of prediction scores respectively indicate the prediction probabilities that the plurality of data samples belong to the first category or the second category.
  • the client node 110 modifies the plurality of truth labels based on a random response mechanism to obtain a plurality of protected labels.
  • Multiple ground truth labels respectively label multiple data samples as belonging to the first category or to the second category.
  • the client node 110 determines error metric information related to a predetermined performance indicator of the machine learning model based on the plurality of protected labels and the plurality of prediction scores.
  • the client node 110 sends error metric information to the service node.
  • determining error metric information includes determining a plurality of prediction scores and a plurality of protected labels as error metric information.
  • a plurality of prediction scores are determined as a first portion of the error metric information and sent to the service node.
  • determining the error metric information further includes: after sending the plurality of prediction scores to the service node, receiving from the service node a ranking result of each of the plurality of prediction scores in a prediction score set, where the prediction score set includes a plurality of a prediction score sent by the client node, the plurality of client nodes including the client node; and determining a second portion of the error metric information based on the respective ranking results of the plurality of protected labels and the plurality of prediction scores.
  • determining the second portion of the error metric information includes: determining a first number of first-type protected tags among the plurality of protected tags, the first-type protected tags indicating that the corresponding data sample belongs to the first Category; determine a second number of protected labels of the second category among the plurality of protected labels, the second category of protected labels indicating that the corresponding data sample belongs to the second category; and based on the respective sorting results of the multiple prediction scores, determine the second category of protected labels.
  • the prediction score of the data sample corresponding to a type of protected label exceeds the third number of prediction scores in the prediction score set.
  • sending the error metric information includes: adjusting an order of the plurality of prediction scores; and sending the plurality of prediction scores to the service node in the adjusted order.
  • the predetermined performance metric includes at least the area under the receiver operating characteristic curve (ROC) curve (AUC).
  • ROC receiver operating characteristic curve
  • FIG. 5 illustrates a flow diagram of a process 500 for model performance evaluation at a service node, in accordance with some embodiments of the present disclosure.
  • Process 500 may be implemented at service node 120.
  • the service node 120 receives error metric information related to predetermined performance indicators of the machine learning model from the plurality of client nodes respectively. Error metric information is determined by the respective client nodes based on their respective multiple protected labels. Multiple protected labels are generated by applying a random response mechanism to multiple ground truth labels.
  • the service node 120 determines an error value for the predetermined performance indicator based on the error metric information.
  • the service node 120 determines a corrected value for the predetermined performance indicator by correcting the error value.
  • receiving the error metric information includes, for a given client node among the plurality of client nodes, receiving a plurality of protected labels and a plurality of predicted scores from the given client node, the plurality of predicted scores determined by the machine
  • the learning model is determined based on the plurality of data samples, and the plurality of prediction scores respectively indicate predicted probabilities that the plurality of data samples belong to the first category or the second category.
  • determining an error value for the predetermined performance indicator includes determining a first total number of protected labels of the first type and a number of protected labels of the second type in the set of protected labels received from the plurality of client nodes.
  • the second total number, the first type of protected label indicates that the corresponding data sample belongs to the first category, and the second type of protected label indicates that the corresponding data sample belongs to the second category; for a set of prediction scores received from multiple client nodes Perform sorting; based on the sorting result of each prediction score in the prediction score set, determine the third total number of prediction scores that the prediction score of the data sample corresponding to the first type of protected label exceeds in the prediction score set; and based on the first total number , the second total number and the third total number to calculate the error value of the predetermined performance index.
  • receiving the error metric information includes, for a given client node among the plurality of client nodes, receiving a plurality of prediction scores from the given client node as a first portion of the error metric information, the plurality of prediction scores The score is determined by the machine learning model based on the plurality of data samples, and the plurality of prediction scores respectively indicate predicted probabilities that the plurality of data samples belong to the first category or the second category.
  • process 500 further includes: determining a ranking of a plurality of predicted scores from a given client node in a set of predicted scores, the set of predicted scores including predicted scores sent by the plurality of client nodes; and placing the plurality of predicted scores from a given client node. The sorted results of prediction scores are sent to the given client node.
  • receiving error metric information further includes receiving, from the given client node, a first number of protected labels of the first type in a plurality of protected labels at the given client node, and a plurality of protected labels at the given client node. a second number of second-type protected labels in the label, where the first-type protected label indicates that the corresponding data sample belongs to the first category, and the second-type protected label indicates that the corresponding data sample belongs to the second category; and from the given The client node receives a third number, and the third number indicates the number of prediction scores that the prediction score of the data sample corresponding to the first type of protected label exceeds in the prediction score set.
  • determining the error value of the predetermined performance indicator includes: obtaining a first total number of protected labels of the first type by aggregating a first number of protected labels of the first type received from a plurality of client nodes; A second total number of protected labels of the second type is obtained by aggregating the second number of protected labels of the second type received from the plurality of client nodes; Three numbers, obtain the third total number of prediction scores that the prediction score of the data sample corresponding to the first type of protected label exceeds in the prediction score set; and calculate based on the first total number, the second total number and the third total number The error value of the predetermined performance indicator.
  • determining the correction value of the predetermined performance indicator includes: obtaining a first total number of protected labels of the first type and a second total number of protected labels of the second type in the protected label set of the plurality of client nodes.
  • the first type of protected label indicates that the corresponding data sample belongs to the first category
  • the second type of protected label indicates that the corresponding data sample belongs to the second category
  • the error of the predetermined performance indicator is determined based on the first total number and the second total number.
  • a mapping relationship between the value and the correction value and based on the mapping relationship, calculating the correction value of the predetermined performance index from the error value.
  • Figure 6 shows a block diagram of an apparatus 600 for model performance evaluation at a client node, in accordance with some embodiments of the present disclosure.
  • Apparatus 600 may be implemented as or included in client node 110 .
  • Each module/component in the device 600 may be implemented by hardware, software, firmware, or any combination thereof.
  • the apparatus 600 includes a score obtaining module 610 configured to obtain a plurality of prediction scores output by a machine learning model for a plurality of data samples.
  • the plurality of prediction scores respectively indicate the prediction probabilities that the plurality of data samples belong to the first category or the second category.
  • the device 600 also includes a label modification module 620 configured to modify a plurality of truth labels based on a random response mechanism to obtain a plurality of protected labels.
  • the plurality of truth labels respectively mark that the plurality of data samples belong to the first category or belong to the first category. Category II.
  • the apparatus 600 further includes an information determination module 630 configured to determine error metric information related to a predetermined performance indicator of the machine learning model based on a plurality of protected labels and a plurality of prediction scores; and an information sending module 640 configured to Send error metric information to the service node.
  • an information determination module 630 configured to determine error metric information related to a predetermined performance indicator of the machine learning model based on a plurality of protected labels and a plurality of prediction scores
  • an information sending module 640 configured to Send error metric information to the service node.
  • the information determination module 630 includes a first determination module configured to determine a plurality of prediction scores and a plurality of protected labels as error metric information.
  • a plurality of prediction scores are determined as a first portion of the error metric information and sent to the service node.
  • the information determination module 630 includes: a ranking result receiving module configured to receive, from the service node, the ranking results of each of the plurality of prediction scores in the prediction score set after sending the plurality of prediction scores to the service node, the set of prediction scores includes prediction scores sent by a plurality of client nodes, the plurality of client nodes including the client node; and a second determination module configured to rank results based on respective rankings of the plurality of protected labels and the plurality of prediction scores, Determine the second part of the error metric information.
  • the second determining module includes: a first number determining module configured to determine a first number of a first type of protected tags among the plurality of protected tags, the first type of protected tags indicating corresponding data The sample belongs to the first category; the second number determination module is configured to determine the second number of the second category of protected tags among the plurality of protected tags, and the second category of protected tags indicates that the corresponding data sample belongs to the second category; and a third number determination module configured to determine, based on the respective sorting results of the plurality of prediction scores, a third number of prediction scores that the prediction scores of the data samples corresponding to the first type of protected labels exceed in the prediction score set.
  • the information sending module 640 includes: a sequence adjustment module configured to adjust the order of multiple predicted scores; and a sequential sending module configured to send the multiple predicted scores to the service node in the adjusted order. .
  • the predetermined performance metric includes at least the area under the receiver operating characteristic curve (ROC) curve (AUC).
  • ROC receiver operating characteristic curve
  • Figure 7 shows a block diagram of an apparatus 700 for model performance evaluation at a service node, according to some embodiments of the present disclosure.
  • Apparatus 700 may be implemented as or included in service node 120 .
  • Each module/component in the device 700 may be implemented by hardware, software, firmware, or any combination thereof.
  • the apparatus 700 includes an information receiving module 710 configured to respectively receive error metric information related to predetermined performance indicators of the machine learning model from a plurality of client nodes. Error metric information is determined by the respective client nodes based on their respective multiple protected labels. Multiple protected labels are generated by applying a random response mechanism to multiple ground truth labels.
  • the apparatus 700 further includes an index determination module 720 configured to determine an error value of a predetermined performance index based on the error metric information; and an index correction module 730 configured to determine a correction value of the predetermined performance index by correcting the error value.
  • the information receiving module 710 includes: a first receiving module configured to, for a given client node among the plurality of client nodes, receive a plurality of protected labels and a plurality of predictions from the given client node. Scores, a plurality of prediction scores are determined by a machine learning model based on a plurality of data samples, and the plurality of prediction scores respectively indicate predicted probabilities that the plurality of data samples belong to the first category or the second category.
  • the indicator determination module 720 includes: a first total number determination module configured to determine a first total number and a first total number of protected tags of the first type in the set of protected tags received from the plurality of client nodes.
  • the second total number of the second type of protected labels, the first type of protected label indicates that the corresponding data sample belongs to the first category, and the second type of protected label indicates that the corresponding data sample belongs to the second category;
  • the sorting module is configured to The prediction score sets received from multiple client nodes are sorted; the second total number determination module is configured to determine the prediction score of the data sample corresponding to the first type of protected label based on the sorting result of each prediction score in the prediction score set. a third total number of prediction scores exceeded in the set of prediction scores; and a first total number-based metric determination module configured to calculate an error in the predetermined performance metric based on the first total number, the second total number, and the third total number. value.
  • the information receiving module 710 includes: a second receiving module configured to, for a given client node among the plurality of client nodes, receive a plurality of prediction scores from the given client node as error metric information.
  • the plurality of prediction scores are determined by the machine learning model based on the plurality of data samples, and the plurality of prediction scores respectively indicate predicted probabilities that the plurality of data samples belong to the first category or the second category.
  • the apparatus 700 further includes: a ranking determination module configured to determine a ranking result of a plurality of predicted scores from a given client node in a set of predicted scores, the set of predicted scores including those sent by the plurality of client nodes prediction scores; and a second sending module configured to send the sorted results of the plurality of prediction scores to the given client node.
  • a ranking determination module configured to determine a ranking result of a plurality of predicted scores from a given client node in a set of predicted scores, the set of predicted scores including those sent by the plurality of client nodes prediction scores
  • a second sending module configured to send the sorted results of the plurality of prediction scores to the given client node.
  • the information receiving module 710 further includes: a third receiving module configured to receive, from the given client node, a third protected label of the first type among the plurality of protected labels at the given client node. A number, and a second number of second-type protected labels among the plurality of protected labels, the first-type protected label indicates that the corresponding data sample belongs to the first category, and the second-type protected label indicates that the corresponding data sample belongs to a second category; and a fourth receiving module configured to receive a third number from a given client node, the third number indicating a predicted score that the predicted score of the data sample corresponding to the first type of protected label exceeds in the predicted score set Number of.
  • the indicator determination module 720 includes: a first aggregation module configured to obtain a first number of protected labels of the first type by aggregating a first number of protected labels of the first type received from multiple client nodes. a first total number; a second aggregation module configured to obtain a second total number of second type protected labels by aggregating the second number of second type protected labels received from multiple client nodes; and a third The aggregation module is configured to obtain, by aggregating a third number of prediction scores received from multiple client nodes, a prediction score of the data sample corresponding to the first type of protected label that exceeds the third number of prediction scores in the prediction score set. a total number; and a second total number-based indicator determination module configured to calculate an error value of the predetermined performance indicator based on the first total number, the second total number, and the third total number.
  • the indicator correction module 730 includes: a number obtaining module configured to obtain a first total number of protected labels of the first type and a number of protected labels of the second type in the protected label set of the plurality of client nodes.
  • the second total number, the first type of protected label indicates that the corresponding data sample belongs to the first category, and the second type of protected label indicates that the corresponding data sample belongs to the second category;
  • the mapping determination module is configured to be based on the first total number and The second total number is used to determine the mapping relationship between the error value and the correction value of the predetermined performance index; and the correction value determination module is configured to calculate the correction value of the predetermined performance index from the error value based on the mapping relationship.
  • Figure 8 illustrates a block diagram of a computing device/system 800 in which one or more embodiments of the present disclosure may be implemented. It should be understood that the computing device/system 800 shown in Figure 8 is exemplary only and should not constitute any limitation on the functionality and scope of the embodiments described herein. The computing device/system 800 shown in FIG. 8 may be used to implement the client node 110 or the service node 120 of FIG. 1 .
  • computing device/system 800 is in the form of a general purpose computing device.
  • Components of computing device/system 800 may include, but are not limited to, one or more processors or processing units 810, memory 820, storage devices 830, one or more communication units 840, one or more input devices 850, and one or more Output device 860.
  • the processing unit 810 may be a real or virtual processor and can perform various processes according to a program stored in the memory 820 . In a multi-processor system, multiple processing units execute computer-executable instructions in parallel to increase the parallel processing capabilities of the computing device/system 800.
  • Computing device/system 800 typically includes a plurality of computer storage media. Such media may be any available media that is accessible to computing device/system 800, including, but not limited to, volatile and nonvolatile media, removable and non-removable media.
  • Memory 820 may be volatile memory (e.g., registers, cache, random access memory (RAM)), Non-volatile memory (eg, read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory) or some combination thereof.
  • Storage device 830 may be a removable or non-removable medium and may include machine-readable media such as a flash drive, a magnetic disk, or any other medium that may be capable of storing information and/or data (e.g., training data for training ) and can be accessed within computing device/system 800.
  • machine-readable media such as a flash drive, a magnetic disk, or any other medium that may be capable of storing information and/or data (e.g., training data for training ) and can be accessed within computing device/system 800.
  • Computing device/system 800 may further include additional removable/non-removable, volatile/non-volatile storage media.
  • a disk drive may be provided for reading from or writing to a removable, non-volatile disk (eg, a "floppy disk") and for reading from or writing to a removable, non-volatile optical disk. Read or write to optical disc drives.
  • each drive may be connected to the bus (not shown) by one or more data media interfaces.
  • Memory 820 may include a computer program product 825 having one or more program modules configured to perform various methods or actions of various embodiments of the present disclosure.
  • the communication unit 840 implements communication with other computing devices through communication media. Additionally, the functionality of the components of computing device/system 800 may be implemented as a single computing cluster or as multiple computing machines capable of communicating over a communications connection. Accordingly, computing device/system 800 may operate in a networked environment using logical connections to one or more other servers, networked personal computers (PCs), or another network node.
  • PCs networked personal computers
  • Input device 850 may be one or more input devices, such as a mouse, keyboard, trackball, etc.
  • Output device 860 may be one or more output devices, such as a display, speakers, printer, etc.
  • the computing device/system 800 may also communicate via the communication unit 840 with one or more external devices (not shown), such as storage devices, display devices, etc., as needed, and with one or more external devices that enable the user to interact with the computing device/system. 800 interacts with devices, or communicates with any device (e.g., network card, modem, etc.) that enables computing device/system 800 to communicate with one or more other computing devices. Such communication may be performed via an input/output (I/O) interface (not shown).
  • I/O input/output
  • a computer-readable storage medium is provided with computer-executable instructions or computer programs stored thereon, wherein the computer-executable instructions or computer programs are executed by a processor to implement the method described above. .
  • a computer program product is also provided, the computer program product is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.
  • These computer-readable program instructions may be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, thereby producing a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, the computer-readable program instructions , resulting in an apparatus that implements the functions/actions specified in one or more blocks in the flowchart and/or block diagram.
  • These computer-readable program instructions can also be stored in a computer-readable storage medium. These instructions cause the computer, programmable data processing device and/or other equipment to work in a specific manner. Therefore, the computer-readable medium storing the instructions includes An article of manufacture that includes instructions that implement aspects of the functions/acts specified in one or more blocks of the flowcharts and/or block diagrams.
  • Computer-readable program instructions may be loaded onto a computer, other programmable data processing apparatus, or other equipment, causing a series of operating steps to be performed on the computer, other programmable data processing apparatus, or other equipment to produce a computer-implemented process, Thereby, instructions executed on a computer, other programmable data processing apparatus, or other equipment implement the functions/actions specified in one or more blocks of the flowcharts and/or block diagrams.
  • each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions that contains one or more executable functions for implementing the specified logical functions instruction.
  • the functions noted in the block may occur out of the order noted in the figures. For example, two consecutive blocks may actually execute substantially in parallel, or they may sometimes execute in the reverse order, depending on the functionality involved.
  • each block of the block diagram and/or flowchart illustration, and combinations of blocks in the block diagram and/or flowchart illustration can be implemented by special purpose hardware-based systems that perform the specified functions or acts. , or can be implemented using a combination of specialized hardware and computer instructions.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Computer Hardware Design (AREA)
  • Data Mining & Analysis (AREA)
  • Bioethics (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Quality & Reliability (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Computer Security & Cryptography (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Databases & Information Systems (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Storage Device Security (AREA)

Abstract

根据本公开的实施例，提供了用于模型性能评估的方法、装置、设备和介质。该方法包括：在客户端节点处，获得机器学习模型针对多个数据样本输出的多个预测得分，多个预测得分分别指示多个数据样本属于第一类别或第二类别的预测概率；基于随机响应机制来修改多个真值标签，以获得多个受保护标签，多个真值标签分别标注多个数据样本属于第一类别或是属于第二类别；基于多个受保护标签和多个预测得分，确定与机器学习模型的预定性能指标相关的差错度量信息；以及向服务节点发送差错度量信息。由此，在实现模型性能评估的同时，达到了对客户端节点本地标签数据的隐私保护目的。

Description

用于模型性能评估的方法、装置、设备和介质
本申请要求于2022年5月13日递交的，标题为“用于模型性能评估的方法、装置、设备和介质”、申请号为202210524005.9的中国发明专利申请的优先权。
技术领域
本公开的示例实施例总体涉及计算机领域，特别地涉及用于模型性能评估的方法、装置、设备和计算机可读存储介质。
背景技术
当前机器学习已经得到了广泛的应用，其性能通常是随着数据量的增加而提高。在理想情况下，可以认为能够集中收集到高质量数据样本和充足的标签数据，用于机器学习模型的训练。然而，在很多现实场景中，存在着所谓的数据孤岛问题，即数据通常是分散隔离的，存储在不同的实体(例如，企业、用户端)上。随着数据隐私保护问题越来越受到重视，导致目前集中式的机器学习系统难以进一步提高。因此，联邦学习得以兴起。联邦学习可以在数据离开本地节点的情况下，在加密的环境中实现和传统机器学习算法一致的性能。
在联邦学习中，期望能够更好地保护数据隐私，包括数据样本对应的标签数据的隐私。
发明内容
根据本公开的示例实施例，提供了一种用于模型性能评估的方案。
在本公开的第一方面，提供了一种用于模型性能评估的方法。该方法包括在客户端节点处，获得机器学习模型针对多个数据样本输出的多个预测得分，多个预测得分分别指示多个数据样本属于第一类别或第二类别的预测概率；基于随机响应机制来修改多个真值标签，以获得多个受保护标签，多个真值标签分别标注多个数据样本属于第一类别或是属于第二类别；基于多个受保护标签和多个预测得分，确定与机器学习模型的预定性能指标相关的差错度量信息；以及向服务节点发送差错度量信息。
在本公开的第二方面，提供了一种用于模型性能评估的方法。该方法包括在服务节点处，从多个客户端节点分别接收与机器学习模型的预定性能指标相关的差错度量信息，所述差错度量信息由相应客户端节点分别基于各自的多个受保护标签确定，所述多个受保护标签通过对多个真值标签应用随机响应机制来生成；基于所述差错度量信息，确定所述预定性能指标的差错值；以及通过校正所述差错值来确定所述预定性能指标的校正值。
在本公开的第三方面，提供了一种用于模型性能评估的装置。该装置包括得分获得模块，被配置为获得机器学习模型针对多个数据样本输出的多个预测得分，多个预测得分分别指示多个数据样本属于第一类别或第二类别的预测概率；标签修改模块，被配置为基于随机响应机制来修改多个真值标签，以获得多个受保护标签，多个真值标签分别标注多个数据样本属于第一类别或是属于第二类别；信息确定模块，被配置为基于多个受保护标签和多个预测得分，确定与机器学习模型的预定性能指标相关的差错度量信息；以及信息发送模块，被配置为向服务节点发送差错度量信息。
在本公开的第四方面，提供了一种用于模型性能评估的装置。该装置包括：信息接收模块，被配置为从多个客户端节点分别接收与机器学习模型的预定性能指标相关的差错度量信息，差错度量信息由相应客户端节点分别基于各自的多个受保护标签确定，多个受保护标签通过对多个真值标签应用随机响应机制来生成；指标确定模块，被配置为基于差错度量信息，确定预定性能指标的差错值；以及指标校正模块，被配置为通过校正差错值来确定预定性能指标的校正值。
在本公开的第五方面，提供了一种电子设备。该设备包括至少一个处理单元；以及至少一个存储器，至少一个存储器被耦合到至少一个处理单元并且存储用于由至少一个处理单元执行的指令。指令在由至少一个处理单元执行时使设备执行第一方面的方法。
在本公开的第六方面，提供了一种电子设备。该设备包括至少一个处理单元；以及至少一个存储器，至少一个存储器被耦合到至少一个处理单元并且存储用于由至少一个处理单元执行的指令。指令在由至少一个处理单元执行时使设备执行第二方面的方法。
在本公开的第七方面，提供了一种计算机可读存储介质。介质上存储有计算机程序，计算机程序被处理器执行以实现第一方面的方法。
在本公开的第八方面，提供了一种计算机可读存储介质。介质上存储有计算机程序，计算机程序被处理器执行以实现第二方面的方法。
应当理解，本发明内容部分中所描述的内容并非旨在限定本公开的实施例的关键特征或重要特征，也不用于限制本公开的范围。本公开的其它特征将通过以下的描述而变得容易理解。
附图说明
结合附图并参考以下详细说明，本公开各实施例的上述和其他特征、优点及方面将变得更加明显。在附图中，相同或相似的附图标记表示相同或相似的元素，其中：
图1示出了本公开的实施例能够在其中应用的示例环境的示意图；
图2示出了根据本公开的一些实施例的用于模型性能评估的信令流的流程图；
图3示出了根据本公开的一些实施例的对真值标签应用随机响应机制的示例的示意图；
图4示出根据本公开的一些实施例的在客户端节点处的型性能评估的过程的流程图；
图5示出根据本公开的一些实施例的在服务节点处的模型性能评估的过程的流程图；
图6示出了根据本公开的一些实施例的在客户端节点处用于模型性能评估的装置的框图；
图7示出了根据本公开的一些实施例的在服务节点处用于模型性能评估的装置的框图；以及
图8示出了能够实施本公开的一个或多个实施例的计算设备/系统的框图。
具体实施方式
下面将参照附图更详细地描述本公开的实施例。虽然附图中示出了本公开的某些实施例，然而应当理解的是，本公开可以通过各种形式来实现，而且不应该被解释为限于这里阐述的实施例，相反，提供这些实施例是为了更加透彻和完整地理解本公开。应当理解的是，本公开的附图及实施例仅用于示例性作用，并非用于限制本公开的保护范围。
在本公开的实施例的描述中，术语“包括”及其类似用语应当理解为开放性包含，即“包括但不限于”。术语“基于”应当理解为“至少部分地基于”。术语“一个实施例”或“该实施例”应当理解为“至少一个实施例”。术语“一些实施例”应当理解为“至少一些实施例”。下文还可能包括其他明确的和隐含的定义。
可以理解的是，本技术方案所涉及的数据(包括但不限于数据本身、数据的获取或使用)应当遵循相应法律法规及相关规定的要求。
可以理解的是，在使用本公开各实施例公开的技术方案之前，均应当根据相关法律法规通过适当的方式对本公开所涉及个人信息的类型、使用范围、使用场景等告知用户并获得用户的授权。
例如，在响应于接收到用户的主动请求时，向用户发送提示信息，以明确地提示用户，其请求执行的操作将需要获取和使用到用户的个人信息。从而，使得用户可以根据提示信息来自主地选择是否向执行本公开技术方案的操作的电子设备、应用程序、服务器或存储介质等软件或硬件提供个人信息。
作为一种可选的但非限制性的实现方式，响应于接收到用户的主动请求，向用户发送提示信息的方式，例如可以是弹窗的方式，弹窗中可以以文字的方式呈现提示信息。此外，弹窗中还可以承载供用户选择“同意”或“不同意”向电子设备提供个人信息的选择控件。
可以理解的是，上述通知和获取用户授权过程仅是示意性的，不对本公开的实现方式构成限定，其他满足相关法律法规的方式也可应用于本公开的实现方式中。
如本文中所使用的，术语“模型”可以从训练数据中学习到相应的输入与输出之间的关联，从而在训练完成后可以针对给定的输入，生成对应的输出。模型的生成可以基于机器学习技术。深度学习是一种机器学习算法，通过使用多层处理单元来处理输入和提供相应输出。神经网络模型是基于深度学习的模型的一个示例。在本文中，“模型”也可以被称为“机器学习模型”、“学习模型”、“机器学习网络”或“学习网络”，这些术语在本文中可互换地使用。
“神经网络”是一种基于深度学习的机器学习网络。神经网络能够处理输入并且提供相应输出，其通常包括输入层和输出层以及在输入层与输出层之间的一个或多个隐藏层。在深度学习应用中使用的神经网络通常包括许多隐藏层，从而增加网络的深度。神经网络的各个层按顺序相连，从而前一层的输出被提供作为后一层的输入，其中输入层接收神经网络的输入，而输出层的输出作为神经网络的最终输出。神经网络的每个层包括一个或多个节点(也称为处理节点或神经元)，每个节点处理来自上一层的输入。
通常，机器学习大致可以包括三个阶段，即训练阶段、测试阶段和应用阶段(也称为推理阶段)。在训练阶段，给定的模型可以使用大量的训练数据进行训练，不断迭代更新参数值，直到模型能够从训练数据中获取一致的满足预期目标的推理。通过训练，模型可以被认为能够从训练数据中学习从输入到输出之间的关联(也称为输入到输出的映射)。训练后的模型的参数值被确定。在测试阶段，将测试输入应用到训练后的模型，测试模型是否能够提供正确的输出，从而确定模型的性能。在应用阶段，模型可以被用于基于训练得到的参数值， 对实际的输入进行处理，确定对应的输出。
图1示出了本公开的实施例能够在其中实现的示例环境100的示意图。环境100涉及联邦学习环境，其中包括N个客户端节点110-1……110-k、……110-N(其中N为大于1的整数，k＝1、2、……N)以及服务节点120。客户端节点110-1……110-k、……110-N可以分别维护各自的本地数据集112-1……112-k、……112-N。为便于讨论，客户端节点110-1……110-k、……110-N可以被统称为或单独称为客户端节点110，本地数据集112-1……112-k、……112-N可以被统称为或单独称为本地数据集112。
在一些实施例中，客户端节点110和/或服务节点120可以被实现在终端设备或服务器处。终端设备可以是任意类型的移动终端、固定终端或便携式终端，包括移动手机、台式计算机、膝上型计算机、笔记本计算机、上网本计算机、平板计算机、媒体计算机、多媒体平板、个人通信系统(PCS)设备、个人导航设备、个人数字助理(PDA)、音频/视频播放器、数码相机/摄像机、定位设备、电视接收器、无线电广播接收器、电子书设备、游戏设备或者前述各项的任意组合，包括这些设备的配件和外设或者其任意组合。在一些实施例中，终端设备也能够支持任意类型的针对用户的接口(诸如“可佩戴”电路等)。服务器是能够提供计算能力的各种类型的计算系统/服务器，包括但不限于大型机、边缘计算节点、云环境中的计算设备，等等。
在联邦学习中，客户端节点指的是提供机器学习模型的部分训练数据的节点。客户端节点也可称为客户端、终端节点、终端设备、用户设备等。在联邦学习中，服务节点指的是聚合客户端节点处的训练结果的节点。
在图1的示例中，假设N个客户端节点110共同参与对机器学习模型130的训练，并将训练中的中间结果汇集到服务节点120，以由服务节点120更新机器学习模型130的参数集。这些客户端节点110的本地数据的全集构成机器学习模型130的完整训练数据集。因此，根据联邦学习的机制，服务节点120将生成全局的机器学习模型130。
针对机器学习模型130，客户端节点110处的本地数据集112可以包括数据样本和真值标签。图1具体示出了某个客户端节点110-k处的本地数据集112-k，其包括数据样本集和真值标签集。数据样本集包括多个(M个)数据样本102-1、102-i、……102-M(统称为或单独称为数据样本102)，并且真值标签集包括对应的多个(M个)真值标签(ground-truth label)105-1、105-i、……105-M(统称为或单独称为真值标签105)。其中，M为大于1的整数，i＝1、2、……M。每个数据样本102可以被标注有对应的真值标签105。数据样本102可以对应于机器学习模型130的输入，真值标签105指示对应的数据样本102的真实输出。真值标签是有监督机器学习中的重要部分。
在本公开的实施例中，机器学习模型130可以基于各种机器学习或深度学习的模型架构来构建，并且可以被配置为实现各种预测任务，诸如各种分类任务、推荐任务等等。机器学习模型130也可以被称为预测模型、推荐模型、分类模型，等等。
数据样本102可以包括与机器学习模型130的具体任务相关的输入信息，真值标签105与任务的期望输出有关。作为一个示例，在二分类任务中，机器学习模型130可以被配置为预测输入的数据样本属于第一类别或是第二类别，真值标签用于标注该数据样本实际属于第一类别或是第二类别。很多实际应用均可以被归类为这样的二分类任务，例如在推荐任务中对推荐项目的转化(例如，点击、购买、注册或其他需求行为)与否，等等。
应当理解，图1仅示出了示例的联邦学习环境。根据联邦学习算法和实际应用需要，环境还可以不同。例如，虽然被示出为单独的节点，在某些应用中，服务节点120除了作为中央节点外，还可以作为客户端节点，以提供部分数据用于模型训练、模型性能评估等。本公开的实施例在此方面不受限制。
在机器学习模型130的训练阶段，已有一些机制保护各个客户端节点110的本地数据不泄露。例如，在模型训练过程中，客户端节点110不必透漏本地的数据样本或标签数据，而是向服务节点120发送根据本地训练数据计算的梯度数据，以供服务节点120更新机器学习模型130的参数集。
在一些情况下，还希望全局地评估训练出的机器学习模型的性能。机器学习模型的性能可以通过一个或多个性能指标来衡量。不同性能指标能够从不同角度，衡量机器学习模型针对数据样本集给出的预测输出与真值标签集所指示的真实输出之间的差异。通常，如果机器学习模型给出的预测输出与真实输出之间的差异较小，那意味着机器学习模型的性能较好。可以看出，通常需要基于数据样本的真值标签集来确定机器学习模型的性能指标。
随着数据监管体系不断加强，对数据隐私保护的要求也越来越高，包括对数据样本的真值标签也需要保护，避免被泄露。例如，对于推荐任务中的数据拥有方，用户对推荐项目的真实转化行为涉及用户隐私，属于敏感信息，需要被保护。
因此，如何既能够确定机器学习模型的性能指标，又保护客户端节点本地的标签数据不被泄露，是一项具有挑战性的任务。当前还没有非常有效的方案能够解决该问题。
根据本公开的实施例，提供了一种模型性能评估方案，其能够保护客户端节点本地的标签数据。具体地，在客户端节点处，通过应用随机响应(Randomized Response，RR)机制来修改数据样本集对应的真值标签集，以获得受保护标签集。客户端节点基于受保护标签集和机器学习模型针对数据样本集输出的预测得分，确定与机器学习模型的性能指标相关的度量信息。此处由于标签集是经过修改后的受保护标签集，因此所确定的度量信息并非准确度量信息，称为“差错度量信息”。客户端节点向服务节点发送差错度量信息。
在服务节点处，服务节点从多个客户端节点接收到它们各自的差错度量信息，并基于差错度量信息确定性能指标的差错值。服务节点进一步校正差错值，以获得性能指标的校正值。
根据本公开的实施例，各个客户端节点无需暴露本地的真值标签集，同时服务节点还能够基于客户端节点的反馈信息计算出性能指标的值。以此方式，在实现模型性能评估的同时，达到了对客户端节点本地标签数据的隐私保护目的。
以下将继续参考附图描述本公开的一些示例实施例。
图2示出了根据本公开的一些实施例的用于模型性能评估的信令流200的示意框图。为便于讨论，参考图1的环境100进行讨论。信令流200涉及客户端节点110和服务节点120。
在本公开的实施例中，假设要评估机器学习模型130的性能。在一些实施例中，待评估的机器学习模型130可以是基于联邦学习的训练过程确定的全局机器学习模型，例如客户端节点110和服务节点120参与了机器学习模型130的训练过程。在一些实施例中，机器学习模型130也可以是以任何其他方式获得的模型，并且客户端节点110和服务节点120可以未参与了机器学习模型130的训练过程。本公开的范围在此方面不受限制。
在一些实施例中，如信令流200所示，服务节点120将机器学习模型130发送205给N个客户端节点110。在接收210到机器学习模型130后，各个客户端节点110可以基于机器学习模型130来执行后续评估过程。在一些实施例中，也可以以任何其他适当的方式将要评估的机器学习模型130提供给客户端节点110。
在本公开的实施例中，将单个客户端节点的角度来描述客户端节点测的操作。
在执行模型性能评估的过程中，客户端节点110获得215机器学习模型130针对多个数据样本102输出的多个预测得分。在一些实施例中，客户端节点110可以将各个数据样本102应用到机器学习模型130，作为模型的输入，并获得机器学习模型130输出的预测得分。例如，假设客户端节点110-k的数据样本集是Xk，机器学习模型130被表示为f()，那么针对数据样本集的预测得分集合可以被表示为sk＝f(Xk)，其中k＝1、2、……N。
在本公开的实施例中，特别关注于在实现二分类任务的机器学习模型的性能指标。每个预测得分可以指示对应的数据样本102属于第一类别或第二类别的预测概率。两种类别可以根据实际任务需要配置。
机器学习模型130输出的预测得分的取值范围可以任意设置。例如，预测得分可以是在某个连续取值区间中的取值(例如，0到1之间的取值)，或者可以是多个离散取值中的一个取值(例如，可以是0、1、2、3、4、5等离散取值之一)。在一些示例中，越高的预测得分可以指示数据样本102属于第一类别的预测概率越大，属于第二类别的预测概率越小。当然，相反设置也是可以的，例如越高的预测得分可以指示数据样本102属于第二类别的预测概率越大，属于第一类别的预测概率越小。
客户端节点110还基于随机响应机制来修改220多个数据样本102各自对应的多个真值标签105(也可称为真实值标签)，以获得多个受保护标签。
应当理解，虽然按顺序描述在215处预测得分的获得和在220处对真值标签应用的随机响应机制，但这两处操作可以按任意顺序来执行，而不受限制。
真值标签105用于标注对应的数据样本102属于第一类别或是第二类别。在下文中，为了方便讨论，将由属于第一类别的数据样本有时称为正样本、正例或正类样本，将属于第二类别的数据样本有时称为负样本、负例或负类样本。在一些实施例中，每个真值标签105可以具有两个取值之一，分别用于指示第一类别或第二类别。在下文的一些实施例中，为了方便讨论，可以将第一类别对应的真值标签105的取值设置为“1”，其指示数据样本属于第一类别，是正样本。此外，可以将第二类别对应的真值标签105的取值设置为“0”，其指示数据样本属于第二类别，是负样本。
在本公开的实施例中，为了在确定机器学习模型130的性能指标的同时实现对真值标签的隐私保护，通过随机响应机制来使真值标签转换为受保护标签。图3示出了根据本公开的一些实施例的对真值标签105应用随机响应机制后得到的受保护标签的示例。如图3所示，经过随机响应机制的应用后，M个数据样本102对应的M个真值标签105将对应于受保护标签305-1、……305-i、……305-M(统称为或单独称为受保护标签305)。
随机响应机制是差分隐私(Differential Privacy，DP)机制中的一种。为更好地理解本公开的实施例，下文将首先简单介绍差分隐私和随机响应机制。
假设ε，δ是大于等于0的实数，即ε，并且是一个随机机制(随机算法)。所谓随机机制，指的是对于特定输入，该机制的输出不是固定值，而是服从某一分布。对于随机机制如果满足以下情况则可以认为随机机制具有(ε，δ)-差分隐私：对于任意两个相邻训练数据集D，D′，并且对于的可能的输出的任意子集S，存在：
此外，如果δ＝0，还可以认为随机机制具有ε-差分隐私(ε-DP)。在差分隐私机制中，对于具有(ε，δ)-差分隐私或ε-差分隐私的随机机制期望其分别作用于两个相邻数据集后得到的两个输出的分布难以区分。这样的话，观察者可以通过观察输出结果，很难察觉到算法的输入数据集中的微小变化，从而达到保护隐私的目的。如果随机机制作用于任何相邻数据集，能得到特定输出S的概率均差不多，那么将会认为该算法难以达到差分隐私的效果。
在本文的实施例中，关注于对数据样本的标签的差分隐私，且标签指示二分类结果。因此，遵循差分隐私的设置，可以定义标签差分隐私。具体地，假设ε，δ是大于等于0的实数，即并且是一个随机机制(随机算法)。如果满足以下情况则可以认为随机机制具有(ε，δ)-标签差分隐私(label differential privacy)：对于任意两个相邻训练数据集D，D′，它们的差异仅在于单个数据样本的标签不同，并且对于的可能的输出的任意子集S，存在：
此外，如果δ＝0，还可以认为随机机制具有ε-差分隐私(ε-DP)。也就是说，期望在改变数据样本的标签后，从随机机制的输出结果的分布仍较小，使得观察者难以察觉到标签的改变。
随机响应机制是为达到差分隐私保护的目的所应用的一种随机机制。随机响应机制被定位如下：假设ε是一个参数，并且y∈[0，1]是在随机响应机制中真值标签的已知取值。如果对于真值标签的取值y，随机响应机制从以下的概率分布中导出随机值
也就是说，在应用随机响应机制后，随机值有一定概率等于y，也有一定概率不等于y。以上随机响应机制被认为具有δ＝0的标签差分隐私((ε，0)-标签差分隐私)，因为：
也就是说，随机响应机制将满足ε-差分隐私。
以上讨论了差分隐私和随机响应机制。在客户端节点110处应用到对多个真值标签105的修改时，多个真值标签105的取值将按照一定概率分布被随机改变。也相当于向多个真值标签105的集合添加噪音或干扰。因此，受保护标签305有时也可称为噪音标签或干扰标签。
假设客户端节点110-k处第i个数据样本102的真值标签105被表示为受保护标签305被表示为在应用随机响应机制后， 从结果来看，有些真值标签105的取值可能会被改变(即，)，有些真值标签105可能会保持不变(即，)。其中，k＝1、2、……N，i＝1、2、……、|Xk|，且|Xk|是客户端节点110-k的数据样本的数目。
由于在二分类问题下的真值标签105是从两个取值中选择，因此，对真值标签105的改变可以认为是反转真值标签105的取值。例如，如果真值标签105的取值是1，反转后，受保护标签305的取值是0。
经过随机响应机制，由于真值标签105的取值被随机改变，从受保护标签305将无法推导出真值标签105。
继续返回参考图2，在获得多个预测得分和多个受保护标签后，客户端节点110确定225与机器学习模型130的预定性能指标相关的度量信息。如前文提及的，在经过修改的受保护标签集基础上，此处所确定的度量信息并非准确度量信息，称为“差错度量信息”。
在本公开的实施例中，个体客户端节点110根据本地数据集(数据样本和真值标签)确定与模型的性能指标相关的度量信息。多个客户端节点110的度量信息可以被汇总到服务节点120。这样，相当于在多个客户端节点的完整数据集基础上评估出机器学习模型130的性能。
由客户端节点提供的差错度量信息的类型可以取决于要计算的性能指标，以及取决于客户端节点110是否要将受保护标签305提供给服务节点。
在下文中，首先介绍用于实现二分类任务的机器学习模型130的一些示例性能指标，然后详细讨论客户端节点110如何向服务节点反馈差错度量信息。
机器学习模型130针对某个数据样本给出的预测得分，通常会与某个得分阈值相比较，并根据比较结果确定该数据样本被预测为属于第一类别或是第二类别。用于实现二分类任务的机器学习模型130的预测可能会出现四种结果。
具体地，对于某个数据样本102，假设真值标签105指示其属于第一类别(正样本)，机器学习模型130也预测出其为正样本，那认为该数据样本是真正样本(True Positive，TP)。如果真值标签105指示其属于第一类别(正样本)，机器学习模型130预测出其为负样本，那认为该数据样本是假负样本(False Negative，FN)。如果真值标签105指示其属于第二类别(负样本)，但机器学习模型130也预测出其为负样本，那认为该数据样本是真负本(True Negative，TN)。如果真值标签105指示其属于第二类别(负样本)，但机器学习模型130预测出其为正样本，那认为该数据样本是假正本(False Positive，FP)。这四种结果可以由以下表1的混淆矩阵指示。
表1
在衡量机器学习模型130的性能时，期望能够在多个客户端节点110的数据样本全集的预测结果以及真值标签全集基础上计算性能指标。
在一些实施例中，机器学习模型130的性能指标可以包括受试者工作特征曲线(ROC)的曲线下面积(AUC)。
ROC曲线是根据不同的分类方式(设置不同得分阈值)，以假正样本比率(FPR)为X轴，真正样本比率(TPR)为Y轴，在坐标轴上绘制出的曲线。FPR可以被定义为：在实际为负例的数据样本中，被模型错误地判断为阳性的比率，表示为FPR＝FP/(FP+TN)，其中FP、TN表示在数据样本全集中统计出的FP、TN的数目。TPR：实际为阳性的样本中，被正确地判断为阳性的比率，表示为TPR＝TP/(TP+FN)。根据每个可能的得分阈值，可以计算出多个(FPR,TPR)对的坐标点， 将这些点连成线，就成为特定模型的ROC曲线。
从定义上理解，AUC指的是ROC曲线下方面积。在计算AUC时，一种可能的方式是，可以根据AUC的定义，可以通过用近似算法计算ROC曲线下的面积来计算AUC。
在一些实施例中，还可以从概率视角来确定AUC。AUC可以被认为是：随机选择一个正样本和一个负样本，机器学习模型给正样本的预测得分高于负样本的预测得分的概率。也就是说，在数据样本集中，将正、负样本两两组合形成正负样本对，其中正样本的预测得分大于负样本的预测得分的占比。如果模型能够给更多正样本输出高于负样本的预测得分，可以认为AUC更高，模型的性能更好。AUC的取值范围在0.5和1之间。AUC越接近1，说明模型的性能越好。
在上述AUC计算中，均需要基于数据样本的标签数据来确定一些度量参数的值。
除AUC之外，机器学习模型130的性能指标还可以包括精确率(Precision)，其被表示为Precision＝TP/TP+FP。精确度表示，被预测为正样本的数据样本子集中，由标签标注为正样本的概率。机器学习模型130的性能指标还可以包括召回率(Recall)，其被表示为Recall＝TP/TP+FN，即正样本被预测的概率。机器学习模型130的性能指标还可以包括P-R曲线，其以召回率为横轴，精确度为纵轴。P-R曲线越靠近右上角，说明模型的性能越好。曲线下面积称作AP分数(Average Precision Score，平均精确率分数)。
在下文中，将主要以AUC的确定作为示例来进行讨论。
继续参考图2，在确定差错度量信息后，客户端节点110向服务节点120发送230所确定的差错度量信息。
如前文提及的，客户端节点110可以选择将多个受保护标签305作为差错度量信息的一部分发送给服务节点，也可以选择不发送受保护标签305，而是在此基础上继续更进一步的度量参数的值。
在直接发送受保护标签305的一些实施例中，客户端节点110可以将多个预测得分和多个受保护标签305直接确定为差错度量信息， 并发送给服务节点120。如图2所示，在差错度量信息的发送方式236中，客户端节点110向服务节点120发送240多个预测得分和多个受保护标签。由此，服务节点120可以接收242到客户端节点110的预测得分和受保护标签。在这些实施例中，针对每个数据样本102，对应的预测得分与受保护标签可以成对被发送给服务节点120。
图2还示出了差错度量信息的另一发送方式238。在该方式下，客户端节点110可以将多个预测得分确定为差错度量信息的第一部分，并将这部分信息发送244给服务节点120。
在一些实施例中，在向服务节点120发送预测得分之前，客户端节点110可以随机调整多个预测得分的顺序，并按调整后的顺序将多个预测得分发送给服务节点。通过随机调整顺序，可以避免在一些特殊情况下，在客户端节点处将多个数据样本102顺序输入模型后，所输出的预测得分具有一定的顺序，例如从大到小或从小到大的顺序，这可能会导致一定的信息泄露。随机顺序调整可以进一步加强数据隐私保护。
在接收246预测得分后，服务节点120对来自多个客户端节点110的预测得分集合进行排序248，得到来自每个客户端节点110的预测得分在预测得分集合中的排序结果。
在一些实施例中，服务节点120可以对预测得分集合升序排序，并对每个预测得分(客户端节点110-k的第i个数据样本的预测得分)分配排序值在一些实施例中，排序值可以指示预测得分在预测得分集合中超过的其他预测得分的数目。例如，按升序排序，最低的预测得分被分配排序值0，指示其未超过(大于)任何其他预测得分；下一个预测得分被分配排序值1，指示其大于集合中的1个预测得分，以此类推。这样的排序值的分配有利于后续的计算。
在方式238中，对于接收到预测得分的客户端节点110，服务节点120将它的多个预测得分在整体的预测得分集合中的排序结果发送250给对应的客户端节点110。在接收252排序结果后，客户端节点110可以基于本地的多个受保护标签305以及多个预测得分各自的所述排序结果，确定254差错度量信息中的第二部分。差错度量信息的第二部分指的是除预测得分之外，计算机器学习模型130的特定性能度量所需要的度量参数的值。
在确定AUC作为性能指标的一些实施例中，客户端节点110可以确定多个受保护标签305中第一类受保护标签的数目(称为“第一数目”)，这里的第一类受保护标签305指示对应的数据样本102属于第一类别，例如指示数据样本102是正样本。此外，客户端节点110还可以确定多个受保护标签305中的第二类受保护标签的数目(称为“第二数目”)，这里的第二类受保护标签指示对应的数据样本属于第二类别，例如指示数据样本是负样本。在客户端节点110-k处，对第一数目和第二数目的确定可以被表示为如下：

其中|Xk|表示客户端节点110-k的数据样本的数目；表示第i个数据样本对应的受保护标签的值；localPk表示在客户端节点110-k处受保护标签中第一类受保护标签(指示正样本的标签)的数目，localNk表示在客户端节点110-k处受保护标签中第二类受保护标签(指示正样本的标签)的数目。
在上式(5)和(6)中，假设对于正样本，的取值为1，对于负样本，的取值为0。这样，通过对的加和，可以统计出由受保护标签指示的正样本的数目，通过对的加和，可以统计出由受保护标签指示的负样本的数目。在其他示例中，如果受保护标签用其他取值来指示正样本和负样本，还可以通过其他方式来统计localPk和localNk，本文对此不做限制。localPk和localNk可以被确定为客户端节点110-k处的差错度量信息中的两个度量参数的值(差错值)。
在一些实施例中，客户端节点110还可以基于多个预测得分各自的排序结果，确定第一类受保护标签对应的数据样本(即，正样本)的预测得分在预测得分集合中超过的预测得分的数目(称为第三数目)。这个数目可以指示在客户端节点110的数据样本的集合中，正样本的排序高于其余样本的排序的样本对个数(在升序排序的情况下)。在一些实施例中，在客户端节点110-k处，第三数目可以通过以下来确定：
其中localSumk表示第三数目，表示第i个数据样本对应的受保护标签的值，表示第i个数据样本对应的预测得分的排序值。如前所述，排序值可以被设置为指示预测得分在预测得分集合中超过的其他预测得分的数目。在上式(7)中，也假设对于正样本，的取值为1，对于负样本，的取值为0。这样，通过对的加和，可以确定出正样本的预测得分排序超过其余样本的预测得分排序的样本数目(也是这样的预测得分的数目)。localSumk可以被确定为客户端节点110-k处的差错度量信息中的另一个度量参数的值(差错值)。
localPk、localNk和localSumk均是在AUC的示例计算方式中需要确定的度量参数。客户端节点110可以将这三个度量参数的值发送256给服务节点120，作为差错度量信息中的第二部分。在接收258到差错度量信息的第二部分后，服务节点120可以据此进行后续操作。
在一些实施例中，不同客户端节点110可以选择方式236或方式238，向服务节点120发送各自的差错度量信息。
无论受保护标签305是否离开客户端节点110，真值标签均能获得隐私保护。这是因为随机响应机制对后处理免疫。换言之，在应用随机响应机制之后，无论后续如何处理受保护标签以及它的相关统计信息，即无论受保护标签数据是否从客户端节点被发出，差分隐私保护能力不会消除。
在接收235到各个客户端节点110发送的差错度量信息后，服务节点120基于来自多个客户端节点110的差错度量信息，确定260机器学习模型130的性能指标的值。此处，由于使用的是差错度量信息，所确定的性能指标的值也称为差错值。
在度量信息基础上，对性能指标的计算取决于所获得的度量信息、所要确定的性能指标的类型。对于AUC，还可以有不同算法可以用于灵活确定。
在一些实施例中，如果服务节点120从多个客户端节点110分别接收到通过方式238发送的度量参数的值localPk、localNk和localSumk，那么服务节点120可以将多个客户端节点110的这些度量参数的值分别按参数进行聚合，得到各个度量参数的聚合值(全局值)，如下：


其中表示在多个客户端节点110的全部受保护标签中第一类受保护标签(指示正样本的标签)的总数目(称为“第一总数目”)，表示在多个客户端节点110的全部受保护标签中第二类受保护标签(指示负样本的标签)的总数目(称为“第二总数目”)，localSum表示在第一类受保护标签对应的数据样本的预测得分在预测得分集合中超过的预测得分的第三总数目。由于均是在受保护标签基础上进行的统计，因此，和globalSum可能与在客户端节点的真值标签基础上统计出的值之间存在误差。
在一些实施例中，如果服务节点120从多个客户端节点110接收到差错度量信息是预测得分和受保护标签，例如通过方式236接收到这样的差错度量信息，那么服务节点120可以将这些客户端节点110的预测得分和受保护标签汇集在一起，并按以上客户端节点处所讨论的类似方法直接统计出和globalSum。
在一些实施例中，如果服务节点120从某个客户端节点110或某部分客户端节点110接收到差错度量信息是预测得分和受保护标签，例如通过方式236接收到这样的差错度量信息，那么服务节点120按以上客户端节点处所讨论的类似方法来计算每个客户端节点对应的localPk、localNk和localSumk。服务节点120也可以将部分客户端节点110的预测得分和受保护标签汇集在一起，按以上客户端节点处所讨论的类似方法，统计出这部分客户端节点对应的第一类受保护标签的数目、第二类受保护标签的数目、以及第一类受保护标签对应的数据样本的预测得分在预测得分集合中超过的预测得分的数目。然后，服务节点120将统计的信息与从其余客户端节点直接接收到的localPk、localNk和localSumk进行聚合，从而确定和globalSum。
在一些实施例中，基于和globalSum，服务节点120可以通过以下方式计算AUC的值(此处计算出的值是差错值，表示为AUC_corr)：
在一些实施例中，如果服务节点120通过方式236从多个客户端节点110接收到预测得分以及受保护标签，服务节点120还可以通过其他方式来计算AUC。具体地，服务节点120可以将接收到的预测得分以及受保护标签汇总。服务节点120可以确定在受保护标签集合中，由受保护标签指示的正样本的数目和由受保护标签指示的负样本的数目。此外，服务节点120可以确定基于预测得分集合，确定在全部数据样本中，正样本的预测得分大于负样本的预测得分的数目。服务节点120可以进而可以基于这三个数目来计算AUC的值(即差错值)。
假设基于受保护标签或预测得分的数目，可以确定N个客户端节点110处的数据样本的总数是L，并且其中由受保护标签指示的正样本的数目为m个，负样本为n个。此外，每个数据样本对应的预测得分为si，i∈[1,L]。通过遍历正样本和负样本的两两组合，可以形成m*n个样本对Pi，i∈[1,m*n个]，那么AUC可以被确定如下：
其中    (11B)
以上讨论了对于AUC的一些示例计算方式。如果适应的话，还可以基于其他方式从概率统计的角度来确定AUC。
在一些实施例中，除AUC之外，还可以评估机器学习模型130的其他性能指标，只要这样的性能指标是能够从多个预测得分和多个受保护标签中确定出。本公开的实施例在此方面不受限制。
为了获得性能指标的更准确值，在性能指标的差错值的基础上，服务节点120通过校正差错值来确定265预定性能指标的校正值。
在一些实施例中，可以确定性能指标的差错值与校正值之间存在的映射关系，并基于此来校正差错值。
在一些实施例中，对于AUC，其差错值与校正值之间的映射关系可以基于在N个客户端节点110涉及的受保护标签集中第一类受保护标签的第一总数目和第二类受保护标签的第二总数目来确定。作为一个示例，AUC的差错值(AUC_corr)与校正值(表示为AUC_real)之间的映射关系可以被表示如下：
其中    (12)

其中π＝P(Y＝1)，指的是由真值标签指示的数据样本集中正样本的比例，ρ+和ρ-分别指示在随机响应机制的应用中对指示正样本和负样本的真值标签的改变率。
对于π，由于服务节点120不知道真值标签的情况，可以通过受保护标签来估算由真值标签指示的数据样本集中正样本的比例。假设M、N是由真值标签指示的数据样本集中正样本和负样本的数目，是从客户端节点110提供的差错度量信息确定的第一类受保护标签的第一总数目和第二类受保护标签的第二总数目。可以确定即样本或标签的总数目不变。此外，还可以确定由这两个等式，可以获得：

相应地，可以确定出
因此，通过上述式(12)，在已知π、ρ+和ρ-的情况下，可以从AUC_corr计算出AUC_real。
可以理解，在对AUC的差错值进行校正后，计算出的AUC_real与基于真值标签统计出的AUC之间可能仍存在一些误差。然而，根据发明人多次反复试验的结果，可以确定这样的误差较小，在允许范围内。实际上，严格来说，即使拥有真值标签，在计算AUC的很多算法中，均是通过近似的方式去逼近AUC的真实值，即ROC曲线下方面积。因此，在需要对标签数据进行隐私保护的场景中，根据本公开的各个实施例，能够在获得数据的差分隐私保护的同时，允许服务节点确定出较准确的性能指标。
在一些实施例中，除AUC之外，还可以计算其他性能指标的值。服务节点120也可以通过设置其他映射关系来矫正这些性能指标的差错值，以获得更准确的性能指标值。
图4示出根据本公开的一些实施例的在客户端节点处用于模型性能评估的过程400的流程图。过程400可以被实现在客户端节点110处。
在框410，客户端节点110获得机器学习模型针对多个数据样本输出的多个预测得分。多个预测得分分别指示多个数据样本属于第一类别或第二类别的预测概率。
在框420，客户端节点110基于随机响应机制来修改多个真值标签，以获得多个受保护标签。多个真值标签分别标注多个数据样本属于第一类别或是属于第二类别。
在框430，客户端节点110基于多个受保护标签和多个预测得分，确定与机器学习模型的预定性能指标相关的差错度量信息。在框440，客户端节点110向服务节点发送差错度量信息。
在一些实施例中，确定差错度量信息包括：将多个预测得分和多个受保护标签确定为差错度量信息。
在一些实施例中，多个预测得分被确定为差错度量信息中的第一部分并且被发送给服务节点。在一些实施例中，确定差错度量信息还包括：在将多个预测得分发送给服务节点之后，从服务节点接收多个预测得分各自在预测得分集合中的排序结果，预测得分集合包括由多个客户端节点发送的预测得分，多个客户端节点包括客户端节点；以及基于多个受保护标签和多个预测得分各自的排序结果，确定差错度量信息中的第二部分。
在一些实施例中，确定差错度量信息中的第二部分包括：确定多个受保护标签中的第一类受保护标签的第一数目，第一类受保护标签指示对应的数据样本属于第一类别；确定多个受保护标签中的第二类受保护标签的第二数目，第二类受保护标签指示对应的数据样本属于第二类别；以及基于多个预测得分各自的排序结果，确定第一类受保护标签对应的数据样本的预测得分在预测得分集合中超过的预测得分的第三数目。
在一些实施例中，发送差错度量信息包括：调整多个预测得分的顺序；以及按调整后的顺序将多个预测得分发送给服务节点。
在一些实施例中，预定性能度量指标至少包括受试者工作特征曲线(ROC)的曲线下面积(AUC)。
图5示出根据本公开的一些实施例的在服务节点处用于模型性能评估的过程500的流程图。过程500可以被实现在服务节点120处。
在框510，服务节点120从多个客户端节点分别接收与机器学习模型的预定性能指标相关的差错度量信息。差错度量信息由相应客户端节点分别基于各自的多个受保护标签确定。多个受保护标签通过对多个真值标签应用随机响应机制来生成。
在框520，服务节点120基于差错度量信息，确定预定性能指标的差错值。在框530，服务节点120通过校正差错值来确定预定性能指标的校正值。
在一些实施例中，接收差错度量信息包括：对于多个客户端节点中的给定客户端节点，从给定客户端节点接收多个受保护标签和多个预测得分，多个预测得分由机器学习模型基于多个数据样本确定，并且多个预测得分分别指示多个数据样本属于第一类别或第二类别的预测概率。
在一些实施例中，确定预定性能指标的差错值包括：在从多个客户端节点接收到的受保护标签集中，确定第一类受保护标签的第一总数目和第二类受保护标签的第二总数目，第一类受保护标签指示对应的数据样本属于第一类别，第二类受保护标签指示对应的数据样本属于第二类别；对从多个客户端节点接收到的预测得分集合进行排序；基于预测得分集合中各个预测得分的排序结果，确定第一类受保护标签对应的数据样本的预测得分在预测得分集合中超过的预测得分的第三总数目；以及基于第一总数目、第二总数目和第三总数目来计算预定性能指标的差错值。
在一些实施例中，接收差错度量信息包括：对于多个客户端节点中的给定客户端节点，从给定客户端节点接收多个预测得分，作为差错度量信息中的第一部分，多个预测得分由机器学习模型基于多个数据样本确定，并且多个预测得分分别指示多个数据样本属于第一类别或第二类别的预测概率。
在一些实施例中，过程500还包括：确定来自给定客户端节点的多个预测得分在预测得分集合中的排序结果，预测得分集合包括由多个客户端节点发送的预测得分；以及将多个预测得分的排序结果发送给该给定客户端节点。
在一些实施例中，接收差错度量信息还包括：从给定客户端节点接收给定客户端节点处的多个受保护标签中的第一类受保护标签的第一数目、以及多个受保护标签中的第二类受保护标签的第二数目，第一类受保护标签指示对应的数据样本属于第一类别，第二类受保护标签指示对应的数据样本属于第二类别；以及从给定客户端节点接收第三数目，第三数目指示第一类受保护标签对应的数据样本的预测得分在预测得分集合中超过的预测得分的数目。
在一些实施例中，确定预定性能指标的差错值包括：通过聚合从多个客户端节点接收到的第一类受保护标签的第一数目，得到第一类受保护标签的第一总数目；通过聚合从多个客户端节点接收到的第二类受保护标签的第二数目，得到第二类受保护标签的第二总数目；通过聚合从多个客户端节点接收到的预测得分的第三数目，得到第一类受保护标签对应的数据样本的预测得分在预测得分集合中超过的预测得分的第三总数目；以及基于第一总数目、第二总数目和第三总数目来计算预定性能指标的差错值。
在一些实施例中，确定预定性能指标的校正值包括：获得在多个客户端节点的受保护标签集中第一类受保护标签的第一总数目和第二类受保护标签的第二总数目，第一类受保护标签指示对应的数据样本属于第一类别，第二类受保护标签指示对应的数据样本属于第二类别；基于第一总数目和第二总数目来确定预定性能指标的差错值与校正值之间的映射关系；以及基于映射关系，从差错值计算预定性能指标的校正值。
图6示出了根据本公开的一些实施例的在客户端节点处用于模型性能评估的装置600的框图。装置600可以被实现为或者被包括在客户端节点110中。装置600中的各个模块/组件可以由硬件、软件、固件或者它们的任意组合来实现。
如图所示，装置600包括得分获得模块610，被配置为获得机器学习模型针对多个数据样本输出的多个预测得分。多个预测得分分别指示多个数据样本属于第一类别或第二类别的预测概率。装置600还包括标签修改模块620，被配置为基于随机响应机制来修改多个真值标签，以获得多个受保护标签，多个真值标签分别标注多个数据样本属于第一类别或是属于第二类别。此外，装置600还包括信息确定模块630，被配置为基于多个受保护标签和多个预测得分，确定与机器学习模型的预定性能指标相关的差错度量信息；以及信息发送模块640，被配置为向服务节点发送差错度量信息。
在一些实施例中，信息确定模块630包括：第一确定模块，被配置为将多个预测得分和多个受保护标签确定为差错度量信息。
在一些实施例中，多个预测得分被确定为差错度量信息中的第一部分并且被发送给服务节点。在一些实施例中，信息确定模块630包括：排序结果接收模块，被配置为在将多个预测得分发送给服务节点之后，从服务节点接收多个预测得分各自在预测得分集合中的排序结果，预测得分集合包括由多个客户端节点发送的预测得分，多个客户端节点包括客户端节点；以及第二确定模块，被配置为基于多个受保护标签和多个预测得分各自的排序结果，确定差错度量信息中的第二部分。
在一些实施例中，第二确定模块包括：第一数目确定模块，被配置为确定多个受保护标签中的第一类受保护标签的第一数目，第一类受保护标签指示对应的数据样本属于第一类别；第二数目确定模块，被配置为确定多个受保护标签中的第二类受保护标签的第二数目，第二类受保护标签指示对应的数据样本属于第二类别；以及第三数目确定模块，被配置为基于多个预测得分各自的排序结果，确定第一类受保护标签对应的数据样本的预测得分在预测得分集合中超过的预测得分的第三数目。
在一些实施例中，信息发送模块640包括：顺序调整模块，被配置为调整多个预测得分的顺序；以及按顺序发送模块，被配置为按调整后的顺序将多个预测得分发送给服务节点。
在一些实施例中，预定性能度量指标至少包括受试者工作特征曲线(ROC)的曲线下面积(AUC)。
图7示出了根据本公开的一些实施例的在服务节点处用于模型性能评估的装置700的框图。装置700可以被实现为或者被包括在服务节点120中。装置700中的各个模块/组件可以由硬件、软件、固件或者它们的任意组合来实现。
如图所示，装置700包括信息接收模块710，被配置为从多个客户端节点分别接收与机器学习模型的预定性能指标相关的差错度量信息。差错度量信息由相应客户端节点分别基于各自的多个受保护标签确定。多个受保护标签通过对多个真值标签应用随机响应机制来生成。装置700还包括指标确定模块720，被配置为基于差错度量信息，确定预定性能指标的差错值；以及指标校正模块730，被配置为通过校正差错值来确定预定性能指标的校正值。
在一些实施例中，信息接收模块710包括：第一接收模块，被配置为对于多个客户端节点中的给定客户端节点，从给定客户端节点接收多个受保护标签和多个预测得分，多个预测得分由机器学习模型基于多个数据样本确定，并且多个预测得分分别指示多个数据样本属于第一类别或第二类别的预测概率。
在一些实施例中，指标确定模块720包括：第一总数确定模块，被配置为在从多个客户端节点接收到的受保护标签集中，确定第一类受保护标签的第一总数目和第二类受保护标签的第二总数目，第一类受保护标签指示对应的数据样本属于第一类别，第二类受保护标签指示对应的数据样本属于第二类别；排序模块，被配置为对从多个客户端节点接收到的预测得分集合进行排序；第二总数确定模块，被配置为基于预测得分集合中各个预测得分的排序结果，确定第一类受保护标签对应的数据样本的预测得分在预测得分集合中超过的预测得分的第三总数目；以及基于总数的第一指标确定模块，被配置为基于第一总数目、第二总数目和第三总数目来计算预定性能指标的差错值。
在一些实施例中，信息接收模块710包括：第二接收模块，被配置为对于多个客户端节点中的给定客户端节点，从给定客户端节点接收多个预测得分，作为差错度量信息中的第一部分。多个预测得分由机器学习模型基于多个数据样本确定，并且多个预测得分分别指示多个数据样本属于第一类别或第二类别的预测概率。
在一些实施例中，装置700还包括：排序确定模块，被配置为确定来自给定客户端节点的多个预测得分在预测得分集合中的排序结果，预测得分集合包括由多个客户端节点发送的预测得分；以及第二发送模块，被配置为将多个预测得分的排序结果发送给该给定客户端节点。
在一些实施例中，信息接收模块710还包括：第三接收模块，被配置为从给定客户端节点接收给定客户端节点处的多个受保护标签中的第一类受保护标签的第一数目、以及多个受保护标签中的第二类受保护标签的第二数目，第一类受保护标签指示对应的数据样本属于第一类别，第二类受保护标签指示对应的数据样本属于第二类别；以及第四接收模块，被配置为从给定客户端节点接收第三数目，第三数目指示第一类受保护标签对应的数据样本的预测得分在预测得分集合中超过的预测得分的数目。
在一些实施例中，指标确定模块720包括：第一聚合模块，被配置为通过聚合从多个客户端节点接收到的第一类受保护标签的第一数目，得到第一类受保护标签的第一总数目；第二聚合模块，被配置为通过聚合从多个客户端节点接收到的第二类受保护标签的第二数目，得到第二类受保护标签的第二总数目；第三聚合模块，被配置为通过聚合从多个客户端节点接收到的预测得分的第三数目，得到第一类受保护标签对应的数据样本的预测得分在预测得分集合中超过的预测得分的第三总数目；以及基于总数的第二指标确定模块，被配置为基于第一总数目、第二总数目和第三总数目来计算预定性能指标的差错值。
在一些实施例中，指标校正模块730包括：数目获得模块，被配置为获得在多个客户端节点的受保护标签集中第一类受保护标签的第一总数目和第二类受保护标签的第二总数目，第一类受保护标签指示对应的数据样本属于第一类别，第二类受保护标签指示对应的数据样本属于第二类别；映射确定模块，被配置为基于第一总数目和第二总数目来确定预定性能指标的差错值与校正值之间的映射关系；以及校正值确定模块，被配置为基于映射关系，从差错值计算预定性能指标的校正值。
图8示出了其中可以实施本公开的一个或多个实施例的计算设备/系统800的框图。应当理解，图8所示出的计算设备/系统800仅仅是示例性的，而不应当构成对本文所描述的实施例的功能和范围的任何限制。图8所示出的计算设备/系统800可以用于实现图1的客户端节点110或服务节点120。
如图8所示，计算设备/系统800是通用计算设备的形式。计算设备/系统800的组件可以包括但不限于一个或多个处理器或处理单元810、存储器820、存储设备830、一个或多个通信单元840、一个或多个输入设备850以及一个或多个输出设备860。处理单元810可以是实际或虚拟处理器并且能够根据存储器820中存储的程序来执行各种处理。在多处理器系统中，多个处理单元并行执行计算机可执行指令，以提高计算设备/系统800的并行处理能力。
计算设备/系统800通常包括多个计算机存储介质。这样的介质可以是计算设备/系统800可访问的任何可以获得的介质，包括但不限于易失性和非易失性介质、可拆卸和不可拆卸介质。存储器820可以是易失性存储器(例如寄存器、高速缓存、随机访问存储器(RAM))、 非易失性存储器(例如，只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、闪存)或它们的某种组合。存储设备830可以是可拆卸或不可拆卸的介质，并且可以包括机器可读介质，诸如闪存驱动、磁盘或者任何其他介质，其可以能够用于存储信息和/或数据(例如用于训练的训练数据)并且可以在计算设备/系统800内被访问。
计算设备/系统800可以进一步包括另外的可拆卸/不可拆卸、易失性/非易失性存储介质。尽管未在图8中示出，可以提供用于从可拆卸、非易失性磁盘(例如“软盘”)进行读取或写入的磁盘驱动和用于从可拆卸、非易失性光盘进行读取或写入的光盘驱动。在这些情况中，每个驱动可以由一个或多个数据介质接口被连接至总线(未示出)。存储器820可以包括计算机程序产品825，其具有一个或多个程序模块，这些程序模块被配置为执行本公开的各种实施例的各种方法或动作。
通信单元840实现通过通信介质与其他计算设备进行通信。附加地，计算设备/系统800的组件的功能可以以单个计算集群或多个计算机器来实现，这些计算机器能够通过通信连接进行通信。因此，计算设备/系统800可以使用与一个或多个其他服务器、网络个人计算机(PC)或者另一个网络节点的逻辑连接来在联网环境中进行操作。
输入设备850可以是一个或多个输入设备，例如鼠标、键盘、追踪球等。输出设备860可以是一个或多个输出设备，例如显示器、扬声器、打印机等。计算设备/系统800还可以根据需要通过通信单元840与一个或多个外部设备(未示出)进行通信，外部设备诸如存储设备、显示设备等，与一个或多个使得用户与计算设备/系统800交互的设备进行通信，或者与使得计算设备/系统800与一个或多个其他计算设备通信的任何设备(例如，网卡、调制解调器等)进行通信。这样的通信可以经由输入/输出(I/O)接口(未示出)来执行。
根据本公开的示例性实现方式，提供了一种计算机可读存储介质，其上存储有计算机可执行指令或计算机程序，其中计算机可执行指令或计算机程序被处理器执行以实现上文描述的方法。
根据本公开的示例性实现方式，还提供了一种计算机程序产品，计算机程序产品被有形地存储在非瞬态计算机可读介质上并且包括计算机可执行指令，而计算机可执行指令被处理器执行以实现上文描述的方法。
在本文中参照根据本公开实现的方法、装置、设备和计算机程序产品的流程图和/或框图描述了本公开的各个方面。应当理解，流程图和/或框图的每个方框以及流程图和/或框图中各方框的组合，都可以由计算机可读程序指令实现。
这些计算机可读程序指令可以提供给通用计算机、专用计算机或其他可编程数据处理装置的处理单元，从而生产出一种机器，使得这些指令在通过计算机或其他可编程数据处理装置的处理单元执行时，产生了实现流程图和/或框图中的一个或多个方框中规定的功能/动作的装置。也可以把这些计算机可读程序指令存储在计算机可读存储介质中，这些指令使得计算机、可编程数据处理装置和/或其他设备以特定方式工作，从而，存储有指令的计算机可读介质则包括一个制造品，其包括实现流程图和/或框图中的一个或多个方框中规定的功能/动作的各个方面的指令。
可以把计算机可读程序指令加载到计算机、其他可编程数据处理装置、或其他设备上，使得在计算机、其他可编程数据处理装置或其他设备上执行一系列操作步骤，以产生计算机实现的过程，从而使得在计算机、其他可编程数据处理装置、或其他设备上执行的指令实现流程图和/或框图中的一个或多个方框中规定的功能/动作。
附图中的流程图和框图显示了根据本公开的多个实现的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或指令的一部分，模块、程序段或指令的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个连续的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或动作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
以上已经描述了本公开的各实现，上述说明是示例性的，并非穷尽性的，并且也不限于所公开的各实现。在不偏离所说明的各实现的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实现的原理、实际应用或对市场中的技术的改进，或者使本技术领域的其他普通技术人员能理解本文公开的各个实现方式。

Claims (20)

  1. 一种模型性能评估的方法，包括：
    在客户端节点处，获得机器学习模型针对多个数据样本输出的多个预测得分，所述多个预测得分分别指示所述多个数据样本属于第一类别或第二类别的预测概率；
    基于随机响应机制来修改多个真值标签，以获得多个受保护标签，所述多个真值标签分别标注所述多个数据样本属于所述第一类别或是属于所述第二类别；
    基于所述多个受保护标签和所述多个预测得分，确定与所述机器学习模型的预定性能指标相关的差错度量信息；以及
    向服务节点发送所述差错度量信息。
  2. 根据权利要求1所述的方法，其中确定所述差错度量信息包括：
    将所述多个预测得分和所述多个受保护标签确定为所述差错度量信息。
  3. 根据权利要求1或2所述的方法，其中所述多个预测得分被确定为所述差错度量信息中的第一部分并且被发送给所述服务节点，并且其中确定所述差错度量信息还包括：
    在将所述多个预测得分发送给服务节点之后，从所述服务节点接收所述多个预测得分各自在预测得分集合中的排序结果，所述预测得分集合包括由多个客户端节点发送的预测得分，所述多个客户端节点包括所述客户端节点；以及
    基于所述多个受保护标签和所述多个预测得分各自的所述排序结果，确定所述差错度量信息中的第二部分。
  4. 根据权利要求3所述的方法，其中确定所述差错度量信息中的第二部分包括：
    确定所述多个受保护标签中的第一类受保护标签的第一数目，所述第一类受保护标签指示对应的数据样本属于所述第一类别；
    确定所述多个受保护标签中的第二类受保护标签的第二数目，所述第二类受保护标签指示对应的数据样本属于所述第二类别；以及
    基于所述多个预测得分各自的所述排序结果，确定所述第一类受保护标签对应的数据样本的预测得分在所述预测得分集合中超过的预测得分的第三数目。
  5. 根据权利要求3或4所述的方法，其中发送所述差错度量信息包括：
    调整所述多个预测得分的顺序；以及
    按调整后的顺序将所述多个预测得分发送给所述服务节点。
  6. 根据权利要求1至5中任一项所述的方法，其中所述预定性能度量指标至少包括受试者工作特征曲线(ROC)的曲线下面积(AUC)。
  7. 一种模型性能评估的方法，包括：
    在服务节点处，从多个客户端节点分别接收与机器学习模型的预定性能指标相关的差错度量信息，所述差错度量信息由相应客户端节点分别基于各自的多个受保护标签确定，所述多个受保护标签通过对多个真值标签应用随机响应机制来生成；
    基于所述差错度量信息，确定所述预定性能指标的差错值；以及
    通过校正所述差错值来确定所述预定性能指标的校正值。
  8. 根据权利要求7所述的方法，其中接收所述差错度量信息包括：对于所述多个客户端节点中的给定客户端节点，
    从所述给定客户端节点接收所述多个受保护标签和多个预测得分，所述多个预测得分由所述机器学习模型基于多个数据样本确定，并且所述多个预测得分分别指示所述多个数据样本属于第一类别或第二类别的预测概率。
  9. 根据权利要求8所述的方法，其中确定所述预定性能指标的所述差错值包括：
    在从所述多个客户端节点接收到的受保护标签集中，确定第一类受保护标签的第一总数目和第二类受保护标签的第二总数目，所述第一类受保护标签指示对应的数据样本属于所述第一类别，所述第二类受保护标签指示对应的数据样本属于所述第二类别；
    对从所述多个客户端节点接收到的预测得分集合进行排序；
    基于所述预测得分集合中各个预测得分的排序结果，确定所述第一类受保护标签对应的数据样本的预测得分在所述预测得分集合中超过的预测得分的第三总数目；以及
    基于所述第一总数目、第二总数目和所述第三总数目来计算所述预定性能指标的所述差错值。
  10. 根据权利要求7至9中任一项所述的方法，其中接收所述差错度量信息包括：对于所述多个客户端节点中的给定客户端节点，
    从所述给定客户端节点接收多个预测得分，作为所述差错度量信息中的第一部分，所述多个预测得分由所述机器学习模型基于多个数据样本确定，并且所述多个预测得分分别指示所述多个数据样本属于第一类别或第二类别的预测概率。
  11. 根据权利要求10所述的方法，还包括：
    确定来自所述给定客户端节点的所述多个预测得分在预测得分集合中的排序结果，所述预测得分集合包括由所述多个客户端节点发送的预测得分；以及
    将所述多个预测得分的所述排序结果发送给所述给定客户端节点。
  12. 根据权利要求11所述的方法，其中接收所述差错度量信息还包括：
    从所述给定客户端节点接收所述给定客户端节点处的所述多个受保护标签中的第一类受保护标签的第一数目、以及所述多个受保护标签中的第二类受保护标签的第二数目，所述第一类受保护标签指示对应的数据样本属于所述第一类别，所述第二类受保护标签指示对应的数据样本属于所述第二类别；以及
    从所述给定客户端节点接收第三数目，所述第三数目指示所述第一类受保护标签对应的数据样本的预测得分在所述预测得分集合中超过的预测得分的数目。
  13. 根据权利要求7至12中任一项所述的方法，其中确定所述预定性能指标的所述差错值包括：
    通过聚合从所述多个客户端节点接收到的所述第一类受保护标签的所述第一数目，得到所述第一类受保护标签的第一总数目；
    通过聚合从所述多个客户端节点接收到的所述第二类受保护标签的所述第二数目，得到所述第二类受保护标签的第二总数目；
    通过聚合从所述多个客户端节点接收到的预测得分的所述第三数目，得到所述第一类受保护标签对应的数据样本的预测得分在所述预测得分集合中超过的预测得分的第三总数目；以及
    基于所述第一总数目、第二总数目和所述第三总数目来计算所述预定性能指标的所述差错值。
  14. 根据权利要求7至13中任一项所述的方法，其中确定所述预定性能指标的校正值包括：
    获得在所述多个客户端节点的受保护标签集中第一类受保护标签的第一总数目和第二类受保护标签的第二总数目，所述第一类受保护标签指示对应的数据样本属于所述第一类别，所述第二类受保护标签指示对应的数据样本属于所述第二类别；
    基于所述第一总数目和所述第二总数目来确定所述预定性能指标的差错值与校正值之间的映射关系；以及
    基于所述映射关系，从所述差错值计算所述预定性能指标的所述校正值。
  15. 一种用于模型性能评估的装置，包括：
    得分获得模块，被配置为获得机器学习模型针对多个数据样本输出的多个预测得分，所述多个预测得分分别指示所述多个数据样本属于第一类别或第二类别的预测概率；
    标签修改模块，被配置为基于随机响应机制来修改多个真值标签，以获得多个受保护标签，所述多个真值标签分别标注所述多个数据样本属于所述第一类别或是属于所述第二类别；
    信息确定模块，被配置为基于所述多个受保护标签和所述多个预测得分，确定与所述机器学习模型的预定性能指标相关的差错度量信息；以及
    信息发送模块，被配置为向服务节点发送所述差错度量信息。
  16. 一种用于模型性能评估的装置，包括：
    信息接收模块，被配置为从多个客户端节点分别接收与机器学习模型的预定性能指标相关的差错度量信息，所述差错度量信息由相应客户端节点分别基于各自的多个受保护标签确定，所述多个受保护标签通过对多个真值标签应用随机响应机制来生成；
    指标确定模块，被配置为基于所述差错度量信息，确定所述预定性能指标的差错值；以及
    指标校正模块，被配置为通过校正所述差错值来确定所述预定性能指标的校正值。
  17. 一种电子设备，包括：
    至少一个处理单元；以及
    至少一个存储器，所述至少一个存储器被耦合到所述至少一个处理单元并且存储用于由所述至少一个处理单元执行的指令，所述指令在由所述至少一个处理单元执行时使所述设备执行根据权利要求1至6中任一项所述的方法。
  18. 一种电子设备，包括：
    至少一个处理单元；以及
    至少一个存储器，所述至少一个存储器被耦合到所述至少一个处理单元并且存储用于由所述至少一个处理单元执行的指令，所述指令在由所述至少一个处理单元执行时使所述设备执行根据权利要求7至14中任一项所述的方法。
  19. 一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行以实现根据权利要求1至6中任一项所述的方法。
  20. 一种计算机可读存储介质，其上存储有计算机程序，所述计 算机程序被处理器执行以实现根据权利要求7至14中任一项所述的方法。
PCT/CN2023/091142 2022-05-13 2023-04-27 用于模型性能评估的方法、装置、设备和介质 Ceased WO2023216899A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/865,611 US20250335327A1 (en) 2022-05-13 2023-04-27 Methods, apparatuses, devices and medium for model performance evaluation

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210524005.9A CN117113386B (zh) 2022-05-13 2022-05-13 用于模型性能评估的方法、装置、设备和介质
CN202210524005.9 2022-05-13

Publications (1)

Publication Number Publication Date
WO2023216899A1 true WO2023216899A1 (zh) 2023-11-16

Family

ID=88729632

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/091142 Ceased WO2023216899A1 (zh) 2022-05-13 2023-04-27 用于模型性能评估的方法、装置、设备和介质

Country Status (3)

Country Link
US (1) US20250335327A1 (zh)
CN (1) CN117113386B (zh)
WO (1) WO2023216899A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN120234190A (zh) * 2023-12-29 2025-07-01 华为技术有限公司 故障预测方法、装置及相关设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150379429A1 (en) * 2014-06-30 2015-12-31 Amazon Technologies, Inc. Interactive interfaces for machine learning model evaluations
CN111488995A (zh) * 2020-04-08 2020-08-04 北京字节跳动网络技术有限公司 用于评估联合训练模型的方法和装置
CN111861099A (zh) * 2020-06-02 2020-10-30 光之树(北京)科技有限公司 联邦学习模型的模型评估方法及装置
CN113222180A (zh) * 2021-04-27 2021-08-06 深圳前海微众银行股份有限公司 联邦学习建模优化方法、设备、介质及计算机程序产品
CN114169010A (zh) * 2021-12-13 2022-03-11 安徽理工大学 一种基于联邦学习的边缘隐私保护方法

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10102480B2 (en) * 2014-06-30 2018-10-16 Amazon Technologies, Inc. Machine learning service

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150379429A1 (en) * 2014-06-30 2015-12-31 Amazon Technologies, Inc. Interactive interfaces for machine learning model evaluations
CN111488995A (zh) * 2020-04-08 2020-08-04 北京字节跳动网络技术有限公司 用于评估联合训练模型的方法和装置
CN111861099A (zh) * 2020-06-02 2020-10-30 光之树(北京)科技有限公司 联邦学习模型的模型评估方法及装置
CN113222180A (zh) * 2021-04-27 2021-08-06 深圳前海微众银行股份有限公司 联邦学习建模优化方法、设备、介质及计算机程序产品
CN114169010A (zh) * 2021-12-13 2022-03-11 安徽理工大学 一种基于联邦学习的边缘隐私保护方法

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
CHEN, CHUAN ET AL.: "FedGL: Federated Graph Learning Framework with Global Self-Supervision", BAIDU, [ONLINE], [RETRIEVAL DATE 2023-7-6].[RETRIEVAL ON THE INTERNET]: URL: HTTPS://ARXIV.ORG/PDF/2105.03170.PDF, 7 May 2021 (2021-05-07) *
SUN JIANKAI, YANG XIN, YAO YUANSHUN, XIE JUNYUAN, WU DI, WANG CHONG: "Differentially Private AUC Computation in Vertical Federated Learning", ARXIV (CORNELL UNIVERSITY), CORNELL UNIVERSITY LIBRARY, ARXIV.ORG, ITHACA, 24 May 2022 (2022-05-24), Ithaca, XP093107217, Retrieved from the Internet <URL:https://arxiv.org/pdf/2205.12412.pdf> [retrieved on 20231130], DOI: 10.48550/arxiv.2205.12412 *

Also Published As

Publication number Publication date
CN117113386A (zh) 2023-11-24
US20250335327A1 (en) 2025-10-30
CN117113386B (zh) 2025-02-11

Similar Documents

Publication Publication Date Title
CN113196303B (zh) 不适当神经网络输入检测和处理
WO2019169704A1 (zh) 一种数据分类方法、装置、设备及计算机可读存储介质
CN109635206B (zh) 融合隐式反馈和用户社会地位的个性化推荐方法及系统
Jiang et al. Efficient federated unlearning with adaptive differential privacy preservation
CN112329837A (zh) 一种对抗样本检测方法、装置、电子设备及介质
US20260099726A1 (en) Method, apparatus, device and medium for information classification
Bien et al. Non-convex global minimization and false discovery rate control for the TREX
Bauckhage et al. Kernel archetypal analysis for clustering web search frequency time series
WO2023216899A1 (zh) 用于模型性能评估的方法、装置、设备和介质
Song et al. Collusion detection and ground truth inference in crowdsourcing for labeling tasks
WO2023216902A1 (zh) 用于模型性能评估的方法、装置、设备和介质
WO2023216900A1 (zh) 用于模型性能评估的方法、装置、设备和存储介质
WO2025189356A1 (en) Training of a model for question answering
CN119514648A (zh) 一种联邦学习优化方法、装置及介质
CN115358827B (zh) 账户数据处理方法、装置、计算机设备和存储介质
CN115081642B (zh) 一种多方协同更新业务预测模型的方法及系统
CN111679959A (zh) 计算机性能数据确定方法、装置、计算机设备及存储介质
CN117851854A (zh) 基于最小先验知识的标签分配方法、电子设备及存储介质
CN116049733A (zh) 基于神经网络的效能评估方法、系统、设备与存储介质
CN111291838B (zh) 实体对象分类结果的解释方法和装置
CN117171791A (zh) 一种图像分类模型的隐私泄露风险评估方法及系统
WO2024136905A1 (en) Clustering techniques for machine learning models
CN118228295A (zh) 数据泄露的风险评估方法、装置及存储介质
Zhao Integrating Machine Learning and Optimization for Problems in Contextual Decision-Making and Dynamic Learning
US12405930B2 (en) System and method for identifying poisoned data during data curation using data source characteristics

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23802671

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 11202405884Y

Country of ref document: SG

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 06-03-2025)

122 Ep: pct application non-entry in european phase

Ref document number: 23802671

Country of ref document: EP

Kind code of ref document: A1

WWP Wipo information: published in national office

Ref document number: 18865611

Country of ref document: US