WO2023098546A1 - 一种联邦学习方法及相关设备 - Google Patents

一种联邦学习方法及相关设备 Download PDF

Info

Publication number
WO2023098546A1
WO2023098546A1 PCT/CN2022/133861 CN2022133861W WO2023098546A1 WO 2023098546 A1 WO2023098546 A1 WO 2023098546A1 CN 2022133861 W CN2022133861 W CN 2022133861W WO 2023098546 A1 WO2023098546 A1 WO 2023098546A1
Authority
WO
WIPO (PCT)
Prior art keywords
random
quantization
model
quantizer
multiple terminals
Prior art date
Application number
PCT/CN2022/133861
Other languages
English (en)
French (fr)
Inventor
李银川
邵云峰
吴骏
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to EP22900351.2A priority Critical patent/EP4365785A1/en
Publication of WO2023098546A1 publication Critical patent/WO2023098546A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/20—Ensemble learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/098—Distributed learning, e.g. federated learning

Definitions

  • the embodiments of the present application relate to the field of machine learning, and in particular, to a federated learning method and related equipment.
  • Federated learning is a machine learning framework proposed for the existence of "data islands”, which can effectively help all parties to conduct joint training and establish shared machine learning without sharing data resources, that is, without training data locally. Model.
  • federated learning technology needs to be combined with privacy protection technology to prevent malicious attackers from obtaining client data distribution.
  • federated learning process requires frequent communication between clients and cloud servers to transmit models, a large amount of communication will increase overhead, so it is also necessary to reduce the amount of communication during transmission.
  • differential privacy technology, multi-party secure computing technology or homomorphic encryption technology can be used to achieve privacy protection, and quantization technology can also be used to reduce communication traffic.
  • Embodiments of the present application provide a federated learning method and related equipment, which are used to meet accuracy and privacy in the case of reduced communication traffic of federated learning.
  • Embodiments of the present application also provide corresponding servers, terminals, computer-readable storage media, computer equipment, and the like.
  • the first aspect of the present application provides a federated learning method, including: the server sends a global model to multiple terminals, the global model includes multiple local models, and the multiple local models correspond to multiple terminals one by one; the server sends a global model to multiple terminals Send random quantization instructions; the server receives the encoded data sent by multiple terminals, and decodes the encoded data to obtain the quantization model update amount of multiple terminals.
  • the model update amount of the terminal is quantified, and the model update amount of multiple terminals is obtained by multiple terminals training multiple local models; the server aggregates the quantized model update amounts of multiple terminals to obtain a new global model, and A new global model is sent to multiple terminals for iterative update until the global model converges.
  • the server in this application can be a cloud server, each terminal can be regarded as a client, and each terminal has a local model corresponding to itself.
  • the random quantization instructions in this application are issued by the server, and multiple terminals can determine different random quantization methods according to different random quantization instructions.
  • the server when performing federated learning, sends random quantization instructions to multiple terminals, and multiple terminals randomly quantize the training update data according to the random quantization instructions and upload them to the server.
  • Quantization can introduce disturbances to protect user privacy, and after the server aggregates the training update data after random quantization, it can eliminate the additional quantization error introduced by random quantization, so that federated learning can meet the requirements of accuracy and accuracy while reducing its traffic. Privacy.
  • the random quantization instruction includes a random step size quantization instruction and/or a random quantizer instruction
  • the quantization model update amounts of multiple terminals are the multiple terminals according to the random step size quantization instruction and/or
  • the random quantizer instruction is obtained by using a random step size quantization method and/or a random quantizer for model update amounts of multiple terminals.
  • the random quantization instruction may specifically be a random step size quantization instruction, a random quantizer instruction, or a combination of a random step size quantization instruction and a random quantizer instruction, which improves the feasibility of the solution.
  • the random quantization step size used by the random step size quantization method satisfies a random uniform distribution.
  • the introduced quantization error can be eliminated, and the accuracy of data transmission can be improved.
  • the random quantizer includes an up-quantizer and a down-quantizer, and the numbers of the up-quantizer and the down-quantizer are the same.
  • the random quantization instructions used each time in the iterative update can be different, which improves the feasibility of the solution.
  • the second aspect of the present application provides a federated learning method, including: the terminal receives the local model and random quantization instruction issued by the server; the terminal trains the local model to obtain the model update amount; the terminal randomly quantizes the model update amount according to the random quantization instruction , to obtain the update amount of the quantization model; the terminal encodes the update amount of the quantization model to obtain encoded data, and sends the encoded data to the server; the terminal receives the new local model and new random quantization instructions issued by the server to iteratively update until the global model Convergence, the new local model is obtained by the server through aggregation of quantized model update amounts of multiple terminals, and the quantized model update amounts of multiple terminals are obtained by the server decoding encoded data sent by multiple terminals.
  • the server in this application may be a cloud server, each terminal may be regarded as a client, and each terminal has a local model corresponding to itself.
  • the random quantization instructions in this application are issued by the server, and multiple terminals can determine different random quantization methods according to different random quantization instructions.
  • multiple terminals when performing federated learning, multiple terminals receive the local model and random quantization instructions issued by the server, and perform random quantization on the training update data according to the random quantization instructions, and then upload them to the server.
  • Quantization can reduce communication traffic , random quantization can introduce disturbances to protect user privacy, and after the server aggregates the training update data after random quantization, it can eliminate the additional quantization error introduced by random quantization, so that federated learning can meet the accuracy requirements while reducing its traffic. sex and privacy.
  • the random quantization instruction includes a random step size quantization instruction and/or a random quantizer instruction
  • the quantization model update amounts of multiple terminals are the multiple terminals according to the random step size quantization instruction and/or
  • the random quantizer instruction is obtained by using a random step size quantization method and/or a random quantizer for model update amounts of multiple terminals.
  • the random quantization step size used by the random step size quantization method satisfies a random uniform distribution.
  • the introduced quantization error can be eliminated, and the accuracy of data transmission can be improved.
  • the random quantizer includes an up-quantizer and a down-quantizer, and the numbers of the up-quantizer and the down-quantizer are the same.
  • the random quantization instructions used each time in the iterative update can be different, which improves the feasibility of the solution.
  • a third aspect of the present application provides a server configured to execute the method in the foregoing first aspect or any possible implementation manner of the first aspect.
  • the server includes modules or units for performing the method in the first aspect or any possible implementation of the first aspect, such as: a first delivery unit, a second delivery unit, a receiving unit, and an aggregation unit .
  • a fourth aspect of the present application provides a terminal configured to execute the method in the foregoing second aspect or any possible implementation manner of the second aspect.
  • the terminal includes modules or units for performing the method in the above second aspect or any possible implementation of the second aspect, such as: a first receiving unit, a training unit, a quantization unit, a coding unit, and a second receiving unit unit.
  • the fifth aspect of the present application provides a computer device, including: a processor, a communication interface, and a memory, the memory is used to store program codes, and the processor is used to call the program codes in the memory so that the controller executes the first aspect or the first aspect A method in any possible implementation.
  • a sixth aspect of the present application provides a computer device, including: a processor, a communication interface, and a memory, the memory is used to store program codes, and the processor is used to call the program codes in the memory so that the controller executes the second aspect or the second aspect A method in any possible implementation.
  • a seventh aspect of the present application provides a computer-readable storage medium, storing instructions, and when the instructions are run on a computer, the computer executes the method in the first aspect or any possible implementation manner of the first aspect.
  • the eighth aspect of the present application provides a computer-readable storage medium, which stores instructions, and when the instructions are run on a computer, the computer executes the method in the second aspect or any possible implementation manner of the second aspect.
  • the ninth aspect of the present application provides a computer program product that stores one or more computer-executable instructions.
  • the processor executes any of the possible implementations of the above-mentioned first aspect or the first aspect. Methods.
  • the tenth aspect of the present application provides a computer program product that stores one or more computer-executable instructions.
  • the processor executes any of the above-mentioned second aspect or any possible implementation of the second aspect. Methods.
  • the eleventh aspect of the present application provides a chip system, the chip system includes at least one processor and an interface, the interface is used to receive data and/or signals, and at least one processor is used to support the computer device to implement the above first aspect or the first aspect
  • the system-on-a-chip may further include a memory, and the memory is used for storing necessary program instructions and data of the computer device.
  • the system-on-a-chip may consist of chips, or may include chips and other discrete devices.
  • the twelfth aspect of the present application provides a chip system, the chip system includes at least one processor and an interface, the interface is used to receive data and/or signals, and the at least one processor is used to support the computer device to implement the above-mentioned second aspect or the first The functions involved in any possible implementation of the second aspect.
  • the system-on-a-chip may further include a memory, and the memory is used for storing necessary program instructions and data of the computer device.
  • the system-on-a-chip may consist of chips, or may include chips and other discrete devices.
  • the server when performing federated learning, sends random quantization instructions to multiple terminals, and multiple terminals randomly quantize the training update data according to the random quantization instructions and upload them to the server.
  • Quantization can reduce communication traffic. Random quantization can introduce disturbances to protect user privacy, and the server can eliminate the additional quantization error introduced by random quantization after the server aggregates the training update data after random quantization, so as to meet the requirements of accurate federated learning while reducing its communication traffic. sex and privacy.
  • Figure 1 is a framework diagram of the federated learning provided by the embodiment of the present application.
  • Fig. 2 is a schematic diagram of an embodiment of the federated learning method provided by the embodiment of the present application
  • FIG. 3 is a schematic diagram of the quantization of the random step size quantization instruction provided by the embodiment of the present application.
  • FIG. 4 is a schematic diagram of a quantization example of a random quantizer instruction provided in an embodiment of the present application
  • FIG. 5 is a schematic diagram of the combination of the random step size quantization instruction and the random quantizer instruction provided by the embodiment of the present application;
  • FIG. 6 is a schematic diagram of an embodiment of a server provided in an embodiment of the present application.
  • FIG. 7 is a schematic diagram of an embodiment of a terminal provided in an embodiment of the present application.
  • FIG. 8 is a schematic diagram of an embodiment of a computer device provided by an embodiment of the present application.
  • Embodiments of the present application provide a federated learning method and related equipment, which are used to meet accuracy and privacy in the case of reduced communication traffic of federated learning.
  • Embodiments of the present application also provide corresponding servers, terminals, computer-readable storage media, computer equipment, and the like.
  • Federated learning Suppose there is a server and K terminals in a federated learning system, the main steps of its training are as follows:
  • Step 1 At a certain time t, each terminal receives the global model W t issued by the server.
  • the model can be a neural network parameter model, or other machine learning model parameters, as the local model of each terminal where k represents the kth terminal;
  • Step 2 Each terminal updates the local model according to the local data to obtain a new round of local models Each terminal will have a new round of local model or model updates upload to the server;
  • Step 3 The server receives the local model or model update amount of each terminal, and performs model aggregation to obtain a new round of global model.
  • the server is updated as follows:
  • the server update is:
  • ⁇ k represents the weight coefficient, repeat this process until the global model converges.
  • Quantization technology refers to the process of discretizing continuous signal values into a limited number of values. Through quantization, data can be mapped to a predefined data grid. In federated learning, each terminal quantifies and encodes the model data and uploads it to the server.
  • the quantization and encoding process can be expressed as:
  • Q k is a quantizer, which can be any quantizer
  • Encode is an encoding operation, which can be any encoding method
  • the encoded data is obtained Uploading encoded data reduces the amount of communication compared to uploading the entire model.
  • Differential privacy technology refers to the protection of potential user privacy information in published data by adding interference noise to the data.
  • each participant prevents malicious attackers from deriving the data distribution of the participant from the model by adding noise to the model.
  • differential noise is added to the model to obtain:
  • Multi-party secure computing is a theoretical framework proposed for a group of mutually distrustful participants to protect private information and collaborative computing without a trusted third party.
  • Homomorphic encryption technology is a cryptographic technique based on computational complexity theory of mathematical problems. In federated learning, each participant uploads the model with homomorphic encryption, the cloud server aggregates and distributes the model, and the participant decrypts the model to obtain the output. This encryption technology can ensure that the model aggregation is not affected during the encryption and decryption process, so it can ensure the convergence of the global model.
  • each terminal can be regarded as a client, for example, there are client 1, client 2 and client 3, and more clients can be set
  • the server can be a cloud server, which stores the global model, can execute the aggregation and delivery of the global model, and configures the quantization instructions of each client.
  • the local model is quantified.
  • an embodiment of the federated learning method provided by the embodiment of the present application includes:
  • the server delivers the global model to multiple terminals.
  • the server issues random quantization instructions to multiple terminals.
  • the terminal trains the local model to obtain a model update amount.
  • the global model issued by the server includes multiple local models, and the multiple local models correspond to multiple terminals one by one, that is, each terminal has a local model corresponding to itself.
  • the global model includes local model 1 and local model 2,
  • the server sends the local model 1 to the terminal 1, and sends the local model 2 to the terminal 2.
  • the server may be a cloud server, and the terminal may be embodied as a client.
  • the terminal receives the global model and the random quantization instruction issued by the server, that is, receives the corresponding local model and the quantization instruction for the local model issued by the server.
  • each terminal uses local data to undergo ⁇ times of training for updating, where ⁇ is a positive integer greater than or equal to 1, then the model update amount can be expressed as:
  • step 102 and step 103 can be reversed, that is, the terminal can complete the training to obtain the model update amount and then receive the random quantization instruction, which is not limited in this embodiment of the present application.
  • the terminal performs random quantization on the model update amount according to the random quantization instruction, to obtain a quantized model update amount.
  • the terminal encodes the quantization model update amount to obtain encoded data.
  • the server receives encoded data sent by multiple terminals.
  • the server decodes the encoded data to obtain quantized model update amounts of multiple terminals.
  • Quantized model update amounts of multiple terminals are obtained by multiple terminals quantifying model update amounts of multiple terminals according to random quantization instructions, and model update amounts of multiple terminals are obtained by multiple terminals training multiple local models;
  • the server aggregates the quantitative model update amounts of multiple terminals to obtain a new global model.
  • the server delivers a new global model to multiple terminals for iterative update until the global model converges.
  • the random quantization instruction can have multiple types, which are described below:
  • the random quantization instruction is a random step size quantization instruction:
  • Terminal k is the mean value of the quantization step size issued by the quantization controller of the server by Sampling from a random distribution for the mean yields a random step size Quantify the model update amount, and obtain the quantified model update amount as:
  • Q is expressed as a general-purpose nearest quantizer, Denotes the additional error introduced by this random quantization method, which plays a role in privacy protection.
  • each terminal encodes the update amount of the quantization model to obtain the encoded data:
  • the server receives the encoded data sent by each terminal, and decodes the encoded data to obtain the quantized model update amount of multiple terminals, that is, the local model of each terminal:
  • the server aggregates the decoded gradients, that is, the quantized model updates of multiple terminals, to obtain a new global model:
  • ⁇ k represents the learning rate
  • the random quantization step size used by each terminal is randomly and uniformly distributed.
  • the additional error introduced by the random quantization step size method satisfies in Represents the terminal set, so the additional quantization error introduced by the random quantization method can be effectively eliminated after model aggregation, and will not affect the convergence of the global model.
  • the server downloads the latest global model to each terminal, and configures new quantization instructions for each terminal and sends them to the client. Quantization step average Then each terminal performs a new round of quantization according to the quantization instruction until the global model converges.
  • the traditional nearby quantization method that is, the upper left part of Figure 3 shows the original data to be uploaded by 16 terminals, and the lower left part shows the data uploaded by 16 terminals after quantization.
  • the distribution of the participants is similar, the difference is small, and the privacy protection effect is poor.
  • the right side of Figure 3 shows the method of introducing a random quantization step size. After the random quantization step size is introduced, data privacy can be effectively protected.
  • the average data obtained after aggregation of the random quantization technology is the same, but the distributed random quantization technology in the right figure has a better privacy protection effect.
  • the random quantization instruction is a random quantizer instruction:
  • the server first generates a random set of quantizers
  • the random quantizer includes an up-quantizer and a down-quantizer, and the number of up-quantizers and down-quantizers used is the same.
  • Terminal k quantizes the model update amount with the quantizer selected in the quantizer set according to the random quantizer instruction, and obtains the quantized model update amount as:
  • Q k represents the quantizer randomly selected by terminal k, and the random quantizer introduces random perturbation, which can replace the differential privacy method.
  • each terminal encodes the update amount of the quantization model to obtain the encoded data:
  • the server receives the encoded data sent by each terminal, and decodes the encoded data to obtain the quantized model update amount of multiple terminals, that is, the local model of each terminal:
  • the server aggregates the decoded gradients, that is, the quantized model updates of multiple terminals, to obtain a new global model:
  • ⁇ k represents the learning rate.
  • the methods of up-quantization and down-quantization respectively introduce additional errors e u and ed d for privacy protection.
  • the error term satisfies Therefore, global model aggregation can reduce the random perturbation brought by the quantizer, thereby reducing the loss of accuracy.
  • the server downloads the latest global model to each terminal, and configures a new quantization instruction for each terminal and sends it to the terminal.
  • the quantization instruction is the quantizer index value, which is used to guide each terminal to randomly quantize according to the index.
  • a random quantizer is selected from the set of quantizers to quantize the local model. For example, if the base of the set of quantizers is J, and the quantization mode configured by the server for a certain terminal is the jth quantizer, then the server sends the binary representation of index j to the terminal as a quantization instruction.
  • Random quantization instructions are random step quantization instructions and random quantizer instructions:
  • Terminal k introduces a random quantization step size, randomly perturbs the model gradient to obtain the perturbed model gradient, and then performs random quantization on the perturbed model gradient to obtain the quantized model update amount as:
  • Q k represents the quantizer randomly selected by terminal k, Indicates that the extra error introduced by the random quantization method combines the privacy protection methods in the above two methods, so as to achieve better privacy protection.
  • each terminal encodes the update amount of the quantization model to obtain the encoded data:
  • the server receives the encoded data sent by each terminal, and decodes the encoded data to obtain the quantized model update amount of multiple terminals, that is, the local model of each terminal:
  • the server aggregates the decoded gradients, that is, the quantized model updates of multiple terminals, to obtain a new global model:
  • model aggregation can reduce the quantization error introduced by random quantization step size and random quantizer quantization, and will not affect the convergence of the global model.
  • the random quantization instructions issued by the server to the terminal can be different, for example, for terminal 1, the server The random quantization command issued in the first round is a random step size quantization command. After the model is aggregated, the random quantization command issued in the second round is a random step size quantization command and a random quantizer command. The issued random quantization instructions are random quantizer instructions. After the model is aggregated, the random quantization instructions issued in the third round are random quantizer instructions until the global model converges.
  • the above three ways to encode the update amount of the quantized model can use dictionary matrix encoding, that is, the update amount of the quantized model Multiplied by a dictionary matrix and then uploaded, so that disturbances can be introduced.
  • the server and the terminal have the same matrix dictionary.
  • the server sends the matrix dictionary index as an instruction to the terminal, and the terminal selects the dictionary matrix according to the index to introduce disturbances.
  • the matrix dictionary stores different random quantization step sizes d
  • the server can send the dictionary index as a quantization instruction to the terminal, and the terminal configures the quantization step size d of the current round according to the index and the matrix dictionary.
  • the stop condition may be that the number of rounds of iterative update reaches a preset value set by the user, that is, the iterative update stops after the preset number of times.
  • the stop condition may also be whether the global model converges, that is, whether the difference between the output value of the current global model and the target value is smaller than a preset value, and if it is smaller, stop.
  • the server when performing federated learning, sends random quantization instructions to multiple terminals, and the multiple terminals randomly quantize the training update data according to the random quantization instructions and upload them to the server.
  • Quantization can reduce communication traffic, and random Quantization can introduce disturbances to protect user privacy, and the server can eliminate the additional quantization error introduced by random quantization after aggregating the training update data after random quantization, so as to meet the accuracy and Privacy.
  • an embodiment of the server 600 provided by the embodiment of the present application includes:
  • the first delivery unit 601 is configured to deliver the global model to multiple terminals, the global model includes multiple local models, and the multiple local models correspond to multiple terminals one by one; the first delivery unit 601 can execute the above method to implement Step 201 in the example.
  • the second delivery unit 602 is configured to deliver random quantization instructions to multiple terminals; the second delivery unit 602 can execute step 202 in the above method embodiment.
  • the receiving unit 603 is configured to receive encoded data sent by multiple terminals, and decode the encoded data to obtain quantization model update amounts of multiple terminals.
  • the model update amount of the terminal is quantified, and the model update amount of multiple terminals is obtained by multiple terminals training multiple local models; the receiving unit 603 can execute step 206 and step 207 in the above method embodiment.
  • the aggregation unit 604 is configured to aggregate the quantized model update amounts of multiple terminals to obtain a new global model, and deliver the new global model to multiple terminals for iterative update until the global model converges.
  • the aggregation unit 604 may execute step 208 in the foregoing method embodiment.
  • the server 600 when performing federated learning, sends random quantization instructions to multiple terminals, and the multiple terminals randomly quantize the training update data according to the random quantization instructions and upload them to the server 600.
  • Quantization can reduce communication traffic. Random quantization can introduce disturbances to protect user privacy, and the server 600 can eliminate the additional quantization error introduced by random quantization after aggregating the randomly quantized training update data, so as to meet the requirements of federated learning while reducing its communication traffic. Accuracy and Privacy.
  • the random quantization instruction includes a random step size quantization instruction and/or a random quantizer instruction
  • the quantization model update amount of multiple terminals is that multiple terminals perform a random step size quantization instruction and/or a random quantizer instruction on multiple terminals
  • the amount of model updates for is obtained using a random step quantization method and/or a random quantizer.
  • the random quantization step size used by the random step size quantization method satisfies a random uniform distribution.
  • the random quantizer includes an up-quantizer and a down-quantizer, and the numbers of the up-quantizer and the down-quantizer are the same.
  • an embodiment of the terminal 700 provided by the embodiment of the present application includes:
  • the first receiving unit 701 is configured to receive the local model and the random quantization instruction issued by the server; the first receiving unit 701 can execute step 201 and step 202 in the above method embodiment.
  • the training unit 702 is configured to train the local model to obtain the model update amount; the training unit 702 can execute step 203 in the above method embodiment.
  • the quantization unit 703 is configured to perform random quantization on the model update amount according to the random quantization instruction to obtain a quantized model update amount; the quantization unit 703 can execute step 204 in the above method embodiment.
  • the encoding unit 704 is configured to encode the update amount of the quantization model to obtain encoded data, and send the encoded data to the server; the encoding unit 704 can execute step 205 in the above method embodiment.
  • the second receiving unit 705 is configured to receive the new local model and the new random quantization instruction issued by the server to iteratively update until the global model converges, and the new local model is obtained by aggregating the quantization model update amounts of multiple terminals by the server The quantization model update amounts of the multiple terminals are obtained by the server decoding the encoded data sent by the multiple terminals.
  • the second receiving unit 705 may execute step 209 in the foregoing method embodiment.
  • the server when federated learning is performed, sends random quantization instructions to multiple terminals 700, and multiple terminals 700 perform random quantization on the training update data according to the random quantization instructions and then upload them to the server.
  • Quantization can reduce communication traffic. Random quantization can introduce disturbances to protect user privacy, and the server can eliminate the additional quantization error introduced by random quantization after the server aggregates the training update data after random quantization, so as to meet the requirements of accurate federated learning while reducing its communication traffic. sex and privacy.
  • the random quantization instruction includes a random step size quantization instruction and/or a random quantizer instruction
  • the quantization unit 703 is specifically configured to use a random step size quantization method for the model update amount according to the random step size quantization instruction and/or the random quantizer instruction and/or a random quantizer for random quantization.
  • the random quantization step size used by the random step size quantization method satisfies a random uniform distribution.
  • the random quantizer includes an up-quantizer and a down-quantizer, and the numbers of the up-quantizer and the down-quantizer are the same.
  • FIG. 8 it is a schematic diagram of a possible logical structure of a computer device 800 provided by an embodiment of the present application.
  • the computer device 800 includes: a processor 801 , a communication interface 802 , a storage system 803 and a bus 804 .
  • the processor 801 , the communication interface 802 and the storage system 803 are connected to each other through a bus 804 .
  • the processor 801 is used to control and manage the actions of the computer device 800, for example, the processor 801 is used to execute the federated learning method performed by the server or the terminal described in the above embodiments.
  • the communication interface 802 is used to support the computer device 800 to communicate.
  • the storage system 803 is used for storing program codes and data of the computer device 800 .
  • the computer device 800 can specifically be the server or terminal in the above-mentioned embodiments
  • the processor 801 can be a central processing unit, a general processor, a digital signal processor, an application specific integrated circuit, a field programmable gate array or other programmable Logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various illustrative logical blocks, modules and circuits described in connection with the present disclosure.
  • the processor 801 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, and the like.
  • the bus 804 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (Extended Industry Standard Architecture, EISA) bus, etc.
  • PCI Peripheral Component Interconnect
  • EISA Extended Industry Standard Architecture
  • the bus can be divided into address bus, data bus, control bus and so on. For ease of representation, only one thick line is used in FIG. 8 , but it does not mean that there is only one bus or one type of bus.
  • a computer-readable storage medium in which computer-executable instructions are stored, and when at least one processor of the device executes the computer-executable instructions, the device executes the above implementation The federated learning method described in the example.
  • a computer program product in another embodiment, includes computer-executable instructions, and the computer-executable instructions are stored in a computer-readable storage medium; Reading the storage medium reads the computer-executable instructions, and at least one processor executes the computer-executable instructions so that the device executes the federated learning method described in the above-mentioned embodiments.
  • a chip system in another embodiment, is also provided.
  • the chip system includes at least one processor and an interface, the interface is used to receive data and/or signals, and the at least one processor is used to support the implementation of the above-mentioned embodiment. Describes the federated learning approach.
  • the system-on-a-chip may further include a memory, and the memory is used for storing necessary program instructions and data of the computer device.
  • the system-on-a-chip may consist of chips, or may include chips and other discrete devices.
  • the disclosed system, device and method can be implemented in other ways.
  • the device embodiments described above are only illustrative.
  • the division of the units is only a logical function division. In actual implementation, there may be other division methods.
  • multiple units or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.
  • the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.
  • the technical solution of the present application is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), magnetic disk or optical disc, etc., which can store program codes. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Medical Informatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Information Transfer Between Computers (AREA)
  • Electrically Operated Instructional Devices (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

本申请实施例公开了一种联邦学习方法及相关设备，应用在联邦学习过程中，该方法具体包括: 服务器向多个终端下发随机量化指令，多个终端根据随机量化指令对训练更新数据进行随机量化后上传给服务器，其中量化可以实现降低通信量，随机的量化可以引入扰动来实现保护用户隐私，而服务器对随机量化后的训练更新数据进行聚合后可以消除随机量化引入的额外量化误差，从而实现在联邦学习在降低其通信量的情况下满足准确性和隐私性。

Description

一种联邦学习方法及相关设备
本申请要求于2021年12月02日提交中国专利局、申请号为202111463299.0、发明名称为“一种联邦学习方法及相关设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及机器学习领域，尤其涉及一种联邦学习方法及相关设备。
背景技术
随着人工智能技术的不断发展，联邦学习(federated learning)作为一种新兴的人工智能技术，应用范围越来越广。联邦学习是针对“数据孤岛”的存在而提出的一种机器学习框架，能够有效帮助各参与方在无需共享数据资源，即训练数据不出本地的情况下，进行联合训练，建立共享的机器学习模型。
但是由于联邦学习的过程需要客户端上传本地模型，恶意攻击者可以从上传的本地模型中反推数据分布，这可能会泄露客户端的数据隐私。因此，联邦学习技术需要和隐私保护技术相结合，从而防止恶意攻击者得到客户端的数据分布。另一方面，由于联邦学习过程需要各客户端和云服务器频繁通信以传输模型，较大的通信量会增大开销，因此还需要在进行传输时降低通信量。基于上述问题，可以使用差分隐私技术、多方安全计算技术或同态加密技术实现隐私保护，还可以使用量化技术降低通信量。
但是传统的量化方法虽然可以降低联邦学习过程中的通信量，但是在对于模型数据进行量化的时候，模型数据的分布几乎没有变化，无法保护用户隐私。而差分隐私技术在噪声引入过程中带来了额外的误差，同态加密技术和多方安全计算技术引入了额外的计算开销，因此无法实现在联邦学习在降低其通信量的情况下满足准确性和隐私性。
发明内容
本申请实施例提供一种联邦学习方法及相关设备，用于在联邦学习在降低其通信量的情况下满足准确性和隐私性。本申请实施例还提供了相应的服务器、终端、计算机可读存储介质及计算机设备等。
本申请第一方面提供一种联邦学习方法，包括：服务器向多个终端下发全局模型，全局模型包括多个本地模型，多个本地模型与多个终端一一对应；服务器向多个终端下发随机量化指令；服务器接收多个终端发送的编码数据，并对编码数据进行解码得到多个终端的量化模型更新量，多个终端的量化模型更新量为多个终端根据随机量化指令对多个终端的模型更新量进行量化得到的，多个终端的模型更新量为多个终端对多个本地模型进行训练得到的；服务器对多个终端的量化模型更新量进行聚合得到新的全局模型，并向多个终端下发新的全局模型进行迭代更新，直至全局模型收敛。
本申请中的服务器可以是云服务器，每个终端可以看作一个客户端，每个终端都有对应自身的一个本地模型。
本申请中的随机量化指令由服务器下发，多个终端可以根据不同的随机量化指令确定不同的随机量化方式。
本申请中的多个终端利用本地数据对自身的本地模型训练后，根据服务器下发的随机量化指令对模型更新量进行量化和编码，实现降低通信量，服务器聚合后可以消除量化带来的误差，得到新的全局模型，并重复上述步骤进行迭代更新，完成联邦学习。
该第一方面，在进行联邦学习时，服务器向多个终端下发随机量化指令，多个终端根据随机量化指令对训练更新数据进行随机量化后上传给服务器，其中量化可以实现降低通信量，随机的量化可以引入扰动来实现保护用户隐私，而服务器对随机量化后的训练更新数据进行聚合后可以消除随机量化引入的额外量化误差，从而实现联邦学习在降低其通信量的情况下满足准确性和隐私性。
在第一方面的一种可能的实现方式中，随机量化指令包括随机步长量化指令和/或随机量化器指令，多个终端的量化模型更新量为多个终端根据随机步长量化指令和/或随机量化器指令对多个终端的模型更新量使用随机步长量化方法和/或随机量化器得到的。
该种可能的实现方式中，随机量化指令具体可以为随机步长量化指令、随机量化器指令或随机步长量化指令结合随机量化器指令，提升了方案的可实现性。
在第一方面的一种可能的实现方式中，随机步长量化方法使用的随机量化步长满足随机均匀分布。
该种可能的实现方式中，当随机量化步长满足随机均匀分布时，可以消除引入的量化误差，提升了数据传输的准确性。
在第一方面的一种可能的实现方式中，随机量化器包括向上量化器和向下量化器，向上量化器和向下量化器的数量相同。
该种可能的实现方式中，当向上量化器和向下量化器的数量相同时，可以消除引入的量化误差，提升了数据传输的准确性。
在第一方面的一种可能的实现方式中，在迭代更新中使用的随机量化指令为多种。
该种可能的实现方式中，在迭代更新中每次使用的随机量化指令都可以不同，提升了方案的可实现性。
本申请第二方面提供一种联邦学习方法，包括：终端接收服务器下发的本地模型和随机量化指令；终端对本地模型进行训练得到模型更新量；终端根据随机量化指令对模型更新量进行随机量化，得到量化模型更新量；终端对量化模型更新量进行编码得到编码数据，并将编码数据发送给服务器；终端接收服务器下发的新的本地模型和新的随机量化指令进行迭代更新，直至全局模型收敛，新的本地模型为服务器对多个终端的量化模型更新量进行聚合得到的，多个终端的量化模型更新量为服务器对多个终端发送的编码数据进行解码得到的。
本申请中的服务器可以是云服务器，每个终端可以看作一个客户端，每个终端都有对应自身的一个本地模型。
本申请中的随机量化指令由服务器下发，多个终端可以根据不同的随机量化指令确定不同的随机量化方式。
本申请中的多个终端利用本地数据对自身的本地模型训练后，根据服务器下发的随机量化指令对模型更新量进行量化和编码，实现降低通信量，服务器聚合后可以消除量化带来的误差，得到新的全局模型，并重复上述步骤进行迭代更新，完成联邦学习。
该第二方面，在进行联邦学习时，多个终端接收服务器下发的本地模型和随机量化指令，并根据随机量化指令对训练更新数据进行随机量化后上传给服务器，其中量化可以实现降低通信量，随机的量化可以引入扰动来实现保护用户隐私，而服务器对随机量化后的训练更新数据进行聚合后可以消除随机量化引入的额外量化误差，从而实现联邦学习在降低其通信量的情况下满足准确性和隐私性。
在第二方面的一种可能的实现方式中，随机量化指令包括随机步长量化指令和/或随机量化器指令，多个终端的量化模型更新量为多个终端根据随机步长量化指令和/或随机量化器指令对多个终端的模型更新量使用随机步长量化方法和/或随机量化器得到的。
该种可能的实现方式中，
在第二方面的一种可能的实现方式中，随机步长量化方法使用的随机量化步长满足随机均匀分布。
该种可能的实现方式中，当随机量化步长满足随机均匀分布时，可以消除引入的量化误差，提升了数据传输的准确性。
在第二方面的一种可能的实现方式中，随机量化器包括向上量化器和向下量化器，向上量化器和向下量化器的数量相同。
该种可能的实现方式中，当向上量化器和向下量化器的数量相同时，可以消除引入的量化误差，提升了数据传输的准确性。
在第二方面的一种可能的实现方式中，在迭代更新中使用的随机量化指令为多种。
该种可能的实现方式中，在迭代更新中每次使用的随机量化指令都可以不同，提升了方案的可实现性。
本申请第三方面，提供了一种服务器，用于执行上述第一方面或第一方面的任意可能的实现方式中的方法。具体地，该服务器包括用于执行上述第一方面或第一方面的任意可能的实现方式中的方法的模块或单元，如：第一下发单元、第二下发单元、接收单元和聚合单元。
本申请第四方面，提供了一种终端，用于执行上述第二方面或第二方面的任意可能的实现方式中的方法。具体地，该终端包括用于执行上述第二方面或第二方面的任意可能的实现方式中的方法的模块或单元，如：第一接收单元、训练单元、量化单元、编码单元和第二接收单元。
本申请第五方面提供一种计算机设备，包括：处理器、通信接口和存储器，存储器用于存储程序代码，处理器用于调用存储器中的程序代码以使得控制器执行第一方面或第一方面的任意可能的实现方式中的方法。
本申请第六方面提供一种计算机设备，包括：处理器、通信接口和存储器，存储器用于存储程序代码，处理器用于调用存储器中的程序代码以使得控制器执行第二方面或第二方面的任意可能的实现方式中的方法。
本申请第七方面提供一种计算机可读存储介质，存储有指令，当指令在计算机上运行时，使得计算机执行如第一方面或第一方面的任意可能的实现方式中的方法。
本申请第八方面提供一种计算机可读存储介质，存储有指令，当指令在计算机上运行时，使得计算机执行如第二方面或第二方面的任意可能的实现方式中的方法。
本申请第九方面提供一种存储一个或多个计算机执行指令的计算机程序产品，当计算机执行指令被处理器执行时，处理器执行如上述第一方面或第一方面任意一种可能的实现方式的方法。
本申请第十方面提供一种存储一个或多个计算机执行指令的计算机程序产品，当计算机执行指令被处理器执行时，处理器执行如上述第二方面或第二方面任意一种可能的实现方式的方法。
本申请第十一方面提供了一种芯片系统，该芯片系统包括至少一个处理器和接口，该接口用于接收数据和/或信号，至少一个处理器用于支持计算机设备实现上述第一方面或第一方面任意一种可能的实现方式中所涉及的功能。在一种可能的设计中，芯片系统还可以包括存储器，存储器，用于保存计算机设备必要的程序指令和数据。该芯片系统，可以由芯片构成，也可以包含芯片和其他分立器件。
本申请第十二方面提供了一种芯片系统，该芯片系统包括至少一个处理器和接口，该接口用于接收数据和/或信号，至少一个处理器用于支持计算机设备实现上述第二方面或第二方面任意一种可能的实现方式中所涉及的功能。在一种可能的设计中，芯片系统还可以包括存储器，存储器，用于保存计算机设备必要的程序指令和数据。该芯片系统，可以由芯片构成，也可以包含芯片和其他分立器件。
本申请实施例中，在进行联邦学习时，服务器向多个终端下发随机量化指令，多个终端根据随机量化指令对训练更新数据进行随机量化后上传给服务器，其中量化可以实现降低通信量，随机的量化可以引入扰动来实现保护用户隐私，而服务器对随机量化后的训练更新数据进行聚合后可以消除随机量化引入的额外量化误差，从而实现在联邦学习在降低其通信量的情况下满足准确性和隐私性。
附图说明
图1为本申请实施例提供的联邦学习框架图；
图2为本申请实施例提供的联邦学习方法一个实施例示意图；
图3为本申请实施例提供的随机步长量化指令的量化示意图；
图4为本申请实施例提供的随机量化器指令的量化例示意图；
图5为本申请实施例提供的随机步长量化指令和随机量化器指令结合的量化示意图；
图6为本申请实施例提供的服务器的一个实施例示意图；
图7为本申请实施例提供的终端的一个实施例示意图；
图8为本申请实施例提供的计算机设备的一实施例示意图。
具体实施方式
下面结合附图，对本申请的实施例进行描述，显然，所描述的实施例仅仅是本申请一部分的实施例，而不是全部的实施例。本领域普通技术人员可知，随着技术发展和新场景的出现，本申请实施例提供的技术方案对于类似的技术问题，同样适用。
本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的实施例能够以除了在这里图示或描述的内容以外的顺序实施。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。
在这里专用的词“示例性”意为“用作例子、实施例或说明性”。这里作为“示例性”所说明的任何实施例不必解释为优于或好于其它实施例。
另外，为了更好的说明本申请，在下文的具体实施方式中给出了众多的具体细节。本领域技术人员应当理解，没有某些具体细节，本申请同样可以实施。在一些实例中，对于本领域技术人员熟知的方法、手段、元件和电路未作详细描述，以便于凸显本申请的主旨。
本申请实施例提供一种联邦学习方法及相关设备，用于在联邦学习在降低其通信量的情况下满足准确性和隐私性。本申请实施例还提供了相应的服务器、终端、计算机可读存储介质及计算机设备等。
下面对本申请实施例提到或涉及的概念进行解释：
联邦学习：假设一个联邦学习系统存在一个服务器和K个终端，其训练的主要步骤如下所示：
步骤一：在某一时刻t,各终端接收服务器下发的全局模型W t，该模型可为神经网络参数模型，也可以为其他机器学习模型参数，作为各终端的本地模型
Figure PCTCN2022133861-appb-000001
其中k表示为第k个终端；
步骤二：各终端根据本地数据更新本地模型，得到新一轮的本地模型
Figure PCTCN2022133861-appb-000002
各终端将新一轮的本地模型
Figure PCTCN2022133861-appb-000003
或者模型更新量
Figure PCTCN2022133861-appb-000004
上传到服务器；
步骤三：服务器接收各终端的本地模型或者模型更新量，进行模型聚合得到新一轮的全局模型。
对于各终端上传本地模型的情况，服务器更新为：
Figure PCTCN2022133861-appb-000005
对于各终端上传本地模型更新量的情况，服务器更新为：
Figure PCTCN2022133861-appb-000006
其中，α k表示权重系数,重复此过程直到全局模型收敛。
量化：量化技术指的是将连续的信号取值离散化为有限多个取值的过程，通过量化可 以将数据映射到预先定义的数据网格上。在联邦学习中，各终端将模型数据进行量化并编码，并上传服务器，量化编码过程可以表示为：
Figure PCTCN2022133861-appb-000007
其中，Q k为量化器，可为任意一种量化器，Encode为编码运算，其可以为任意一种编码方法，得到编码后的数据
Figure PCTCN2022133861-appb-000008
相比于上传整个模型，上传编码数据降低了通信量。
差分隐私：差分隐私技术指的是通过对数据添加干扰噪声的方式保护所发布数据中潜在的用户隐私信息。在联邦学习中，各参与方通过对模型添加噪声的方法防止恶意攻击者从模型反推参与方的数据分布，在模型上传时，对模型添加差分噪声，得到：
Figure PCTCN2022133861-appb-000009
其中，
Figure PCTCN2022133861-appb-000010
表示均值为0，方差为σ 2的高斯噪声，该方法通过了对模型添加高斯噪声实现了隐私保护。
多方安全计算：多方安全计算技术是为了一组互不信任的参与方之前在保护隐私信息及没有可信第三方的前提下协同计算问题而提出的理论框架。
同态加密：同态加密技术是一种基于数学难题的计算复杂性理论的密码学技术。在联邦学习中，各参与方对模型进行同态加密上传，云服务器进行模型聚合并下发，参与方进行解密得到输出。该加密技术可以保证在加密解密过程不影响模型聚合，因此可以保证全局模型的收敛。
如图1所示，在本申请实施例提供的联邦学习架构中，每个终端可以看作一个客户端，例如存在客户端1、客户端2和客户端3，此外还可以设置更多客户端，服务器可以为云服务器，存储有全局模型，可以执行全局模型的聚合和下发，并配置每个客户端的量化指令，每个客户端根据对应的量化指令选择相应的量化方式，对训练后的本地模型进行量化。
下面结合上述概念的描述和学习框架对本申请实施例提供的连邦学习方法进行描述。
如图2所示，本申请实施例提供的联邦学习方法的一实施例包括：
201、服务器向多个终端下发全局模型。
202、服务器向多个终端下发随机量化指令。
203、终端对本地模型进行训练得到模型更新量。
服务器下发的全局模型包括多个本地模型，其中多个本地模型与多个终端一一对应，即每个终端都有对应自身的一个本地模型，例如全局模型包括本地模型1和本地模型2，服务器将本地模型1下发给终端1，将本地模型2下发给终端2。服务器具体可以为云服务器，终端可以体现为客户端。
在t时刻，终端接收服务器下发的全局模型和随机量化指令，即接收服务器下发的对应的本地模型和针对该本地模型的量化指令。具体的，每个终端都利用本地数据经过τ次训练以进行更新，其中τ为大于或等于1的正整数，则模型更新量可以表示为：
Figure PCTCN2022133861-appb-000011
其中，
Figure PCTCN2022133861-appb-000012
表示第k个终端在t时刻的本地模型，
Figure PCTCN2022133861-appb-000013
为经过τ次训练后的本地模型。
需要说明的是，步骤102和步骤103的执行顺序可以调换，即终端可以完成训练得到模型更新量后再收到随机量化指令，本申请实施例对此不作限制。
204、终端根据随机量化指令对模型更新量进行随机量化，得到量化模型更新量。
205、终端对量化模型更新量进行编码得到编码数据。
206、服务器接收多个终端发送的编码数据。
207、服务器对编码数据进行解码得到多个终端的量化模型更新量。
多个终端的量化模型更新量为多个终端根据随机量化指令对多个终端的模型更新量进行量化得到的，多个终端的模型更新量为多个终端对多个本地模型进行训练得到的；
208、服务器对多个终端的量化模型更新量进行聚合得到新的全局模型。
209、服务器向多个终端下发新的全局模型进行迭代更新，直至全局模型收敛。
每个终端得到模型更新量后，根据自身接收到的随机量化指令对模型更新量进行随机量化，其中随机量化指令可以有多种类型，以下分别进行说明：
一、随机量化指令为随机步长量化指令：
终端k根据服务器的量化控制器下发的量化步长的均值
Figure PCTCN2022133861-appb-000014
以
Figure PCTCN2022133861-appb-000015
为均值的随机分布中采样得到随机步长
Figure PCTCN2022133861-appb-000016
对模型更新量进行量化，得到量化模型更新量为：
Figure PCTCN2022133861-appb-000017
其中，Q表示为通用的就近量化器，
Figure PCTCN2022133861-appb-000018
表示为该随机量化方式引入的额外误差，起到隐私保护的作用。
然后各个终端对量化模型更新量进行编码得到编码数据：
Figure PCTCN2022133861-appb-000019
服务器接收各个终端发送的编码数据，并对编码数据进行解码得到多个终端的量化模型更新量，即各个终端的本地模型：
Figure PCTCN2022133861-appb-000020
最后服务器对解码后的梯度，即对多个终端的量化模型更新量进行聚合，得到新的全局模型：
Figure PCTCN2022133861-appb-000021
其中，α k表示学习率,对于每个终端使用的随机量化步长是随机均匀分布的，当参与联邦学习的终端足够多时，随机量化步长方法引入的额外误差满足
Figure PCTCN2022133861-appb-000022
其中
Figure PCTCN2022133861-appb-000023
表示终端集合，因此在模型聚合后可以有效消除随机量化方法引入的额外量化误差，不会影响全局模型的收敛。模型聚合后，服务器将最新的全局模型下给各终端，并为各终端配置新的量化指令并下发给客户端，该量化指令包含选取随机步长量化方式的指令，以及新一轮迭代的量化步长均值
Figure PCTCN2022133861-appb-000024
然后各终端根据该量化指令进行新一轮的量化，直至全局模型收敛。
具体的，如图3所示，传统的就近量化方法，即图3左侧上部分图表示16个终端所需上传的原始数据，左侧下部分图表示16个终端量化后上传的数据，两者分布相似，差异性很小，隐私保护效果差。图3右侧为引入随机量化步长的方法，引入随机量化步长后，可以有效的保护数据隐私，16个终端采用左侧图传统量化技术聚合后得到的均值数据和采用右侧图分布式随机量化技术聚合后得到的均值数据相同，但右侧图的分布式随机量化技术隐私保护效果更好。
二、随机量化指令为随机量化器指令：
服务器首先生成随机量化器集合
Figure PCTCN2022133861-appb-000025
随机量化器包括向上量化器和向下量化器，使用的向上量化器和向下量化器的数量相同。
向上量化器将连续数据映射到数据上方的网格点，表示为Q(x)＝d×ceil(x/d)，其中ceil表示为向上取整。向下量化器：将连续数据映射到数据下方的网格点，表示为Q(x)＝d×floor(x/d)，其中floor表示向下取整。
终端k根据随机量化器指令在量化器集合中选择的量化器对模型更新量进行量化，得到量化模型更新量为：
Figure PCTCN2022133861-appb-000026
其中，Q k表示为终端k随机选取的量化器，随机量化器引入了随机扰动，可以代替差分隐私方法。
然后各个终端对量化模型更新量进行编码得到编码数据：
Figure PCTCN2022133861-appb-000027
服务器接收各个终端发送的编码数据，并对编码数据进行解码得到多个终端的量化模型更新量，即各个终端的本地模型：
Figure PCTCN2022133861-appb-000028
最后服务器对解码后的梯度，即对多个终端的量化模型更新量进行聚合，得到新的全局模型：
Figure PCTCN2022133861-appb-000029
其中，α k表示学习率。如图4所示，对于随机量化器选取的量化方法，向上量化和向下量化的方法分别引入额外的误差e u和e d来进行隐私保护，当终端数量足够大时，并且满足使用向上量化器的终端数量
Figure PCTCN2022133861-appb-000030
和使用向下量化器的终端数量
Figure PCTCN2022133861-appb-000031
相等，假设模型数据是随机均匀分布的，因此误差项满足
Figure PCTCN2022133861-appb-000032
因此全局模型聚合可减少量化器带来的随机扰动，从而减少精度损失。
模型聚合后，服务器将最新的全局模型下给各终端，并为各终端配置新的量化指令并下发给终端，该量化指令为量化器索引值，用于指导各终端根据该索引从随机量化器集合中选择随机量化器对本地模型进行量化。例如，如果量化器集合的基数为J，服务器为某一个终端配置的量化方式为第j个量化器，那么服务器将索引j的二进制表示作为量化指令下发给该终端。
三、随机量化指令为随机步长量化指令和随机量化器指令：
终端k引入随机量化步长，对模型梯度进行随机扰动，得到扰动后的模型梯度，然后对扰动后的模型梯度进行随机量化器量化得到量化模型更新量为：
Figure PCTCN2022133861-appb-000033
其中，Q k表示为终端k随机选取的量化器，
Figure PCTCN2022133861-appb-000034
表示为该随机量化方式引入的额外误差，结合了上述两种方式中的隐私保护方法，从而更好的实现隐私保护。
然后各个终端对量化模型更新量进行编码得到编码数据：
Figure PCTCN2022133861-appb-000035
服务器接收各个终端发送的编码数据，并对编码数据进行解码得到多个终端的量化模型更新量，即各个终端的本地模型：
Figure PCTCN2022133861-appb-000036
最后服务器对解码后的梯度，即对多个终端的量化模型更新量进行聚合，得到新的全局模型：
Figure PCTCN2022133861-appb-000037
如图5所示，用户对应终端，类似上述两种方式，模型聚合可以减少随机量化步长和随机量化器量化引入的量化误差，不会影响全局模型的收敛。
需要说明的是，在迭代更新中使用的随机量化指令为多种，即在进行每一轮的迭代更新时，服务器下发给终端的随机量化指令都可以不同，例如对应终端1来说，服务器在第一轮下发的随机量化指令为随机步长量化指令，聚合模型后，第二轮下发的随机量化指令为随机步长量化指令和随机量化器指令，聚合模型后，在第三轮下发的随机量化指令为随机量化器指令，聚合模型后，在第三轮下发的随机量化指令为随机量化器指令，直至全局模型收敛。
此外，上述三种方式对量化模型更新量的编码方式可以采用字典矩阵编码，即将量化模型更新量
Figure PCTCN2022133861-appb-000038
乘以一个字典矩阵后进行上传，从而可以引入扰动，服务器与终端有同样的矩阵字典，服务器令矩阵字典索引作为指令下发给终端，终端根据索引选择字典矩阵引入扰动。例如，矩阵字典存储了不同的随机量化步长d，服务器可以将字典索引作为量化指令下发给终端，终端根据索引以及矩阵字典配置本轮的量化步长d。
在服务器对聚合全局模型后，可以设置是否达到停止条件的判断，若达到，则输出当前聚合后的全局模型，若未达到，则继续下发全局模型以进行迭代更新。其中，停止条件可以是迭代更新的轮数达到用户设定的预设值，即迭代更新了预设次数就停止。停止条件还可以是全局模型是否收敛，即当前的全局模型的输出值和目标值的差值是否小于预设值，若小于则停止。
本申请实施例在进行联邦学习时，服务器向多个终端下发随机量化指令，多个终端根据随机量化指令对训练更新数据进行随机量化后上传给服务器，其中量化可以实现降低通信量，随机的量化可以引入扰动来实现保护用户隐私，而服务器对随机量化后的训练更新数据进行聚合后可以消除随机量化引入的额外量化误差，从而实现在联邦学习在降低其通信量的情况下满足准确性和隐私性。
如图6所示，本申请实施例提供的服务器600的一实施例包括：
第一下发单元601，用于向多个终端下发全局模型，全局模型包括多个本地模型，多个本地模型与多个终端一一对应；该第一下发单元601可以执行上述方法实施例中的步骤201。
第二下发单元602，用于向多个终端下发随机量化指令；该第二下发单元602可以执 行上述方法实施例中的步骤202。
接收单元603，用于接收多个终端发送的编码数据，并对编码数据进行解码得到多个终端的量化模型更新量，多个终端的量化模型更新量为多个终端根据随机量化指令对多个终端的模型更新量进行量化得到的，多个终端的模型更新量为多个终端对多个本地模型进行训练得到的；该接收单元603可以执行上述方法实施例中的步骤206和步骤207。
聚合单元604，用于对多个终端的量化模型更新量进行聚合得到新的全局模型，并向多个终端下发新的全局模型进行迭代更新，直至全局模型收敛。该聚合单元604可以执行上述方法实施例中的步骤208。
本申请实施例在进行联邦学习时，服务器600向多个终端下发随机量化指令，多个终端根据随机量化指令对训练更新数据进行随机量化后上传给服务器600，其中量化可以实现降低通信量，随机的量化可以引入扰动来实现保护用户隐私，而服务器600对随机量化后的训练更新数据进行聚合后可以消除随机量化引入的额外量化误差，从而实现在联邦学习在降低其通信量的情况下满足准确性和隐私性。
可选的，随机量化指令包括随机步长量化指令和/或随机量化器指令，多个终端的量化模型更新量为多个终端根据随机步长量化指令和/或随机量化器指令对多个终端的模型更新量使用随机步长量化方法和/或随机量化器得到的。
可选的，随机步长量化方法使用的随机量化步长满足随机均匀分布。
可选的，随机量化器包括向上量化器和向下量化器，向上量化器和向下量化器的数量相同。
可选的，在迭代更新中使用的随机量化指令为多种。
如图7所示，本申请实施例提供的终端700的一实施例包括：
第一接收单元701，用于接收服务器下发的本地模型和随机量化指令；该第一接收单元701可以执行上述方法实施例中的步骤201和步骤202。
训练单元702，用于对本地模型进行训练得到模型更新量；该训练单元702可以执行上述方法实施例中的步骤203。
量化单元703，用于据随机量化指令对模型更新量进行随机量化，得到量化模型更新量；该量化单元703可以执行上述方法实施例中的步骤204。
编码单元704，用于对量化模型更新量进行编码得到编码数据，并将编码数据发送给服务器；该编码单元704可以执行上述方法实施例中的步骤205。
第二接收单元705，用于接收服务器下发的新的本地模型和新的随机量化指令进行迭代更新，直至全局模型收敛，新的本地模型为服务器对多个终端的量化模型更新量进行聚合得到的，多个终端的量化模型更新量为服务器对多个终端发送的编码数据进行解码得到的。该第二接收单元705可以执行上述方法实施例中的步骤209。
本申请实施例在进行联邦学习时，服务器向多个终端700下发随机量化指令，多个终端700根据随机量化指令对训练更新数据进行随机量化后上传给服务器，其中量化可以实现降低通信量，随机的量化可以引入扰动来实现保护用户隐私，而服务器对随机量化后的训练更新数据进行聚合后可以消除随机量化引入的额外量化误差，从而实现在联邦学习在 降低其通信量的情况下满足准确性和隐私性。
可选的，随机量化指令包括随机步长量化指令和/或随机量化器指令，量化单元703具体用于根据随机步长量化指令和/或随机量化器指令对模型更新量使用随机步长量化方法和/或随机量化器进行随机量化。
可选的，随机步长量化方法使用的随机量化步长满足随机均匀分布。
可选的，随机量化器包括向上量化器和向下量化器，向上量化器和向下量化器的数量相同。
可选的，在迭代更新中使用的随机量化指令为多种。
如图8计算机设备所示，为本申请的实施例提供的计算机设备800的一种可能的逻辑结构示意图。计算机设备800包括：处理器801、通信接口802、存储系统803以及总线804。处理器801、通信接口802以及存储系统803通过总线804相互连接。在本申请的实施例中，处理器801用于对计算机设备800的动作进行控制管理，例如，处理器801用于执行图上述实施例所描述的服务器或终端执行的联邦学习方法。通信接口802用于支持计算机设备800进行通信。存储系统803，用于存储计算机设备800的程序代码和数据。
其中，该计算机设备800具体可以为上述实施例中的服务器或终端，处理器801可以是中央处理器单元，通用处理器，数字信号处理器，专用集成电路，现场可编程门阵列或者其他可编程逻辑器件、晶体管逻辑器件、硬件部件或者其任意组合。其可以实现或执行结合本申请公开内容所描述的各种示例性的逻辑方框，模块和电路。处理器801也可以是实现计算功能的组合，例如包含一个或多个微处理器组合，数字信号处理器和微处理器的组合等等。总线804可以是外设部件互连标准(Peripheral Component Interconnect，PCI)总线或扩展工业标准结构(Extended Industry Standard Architecture，EISA)总线等。总线可以分为地址总线、数据总线、控制总线等。为便于表示，图8中仅用一条粗线表示，但并不表示仅有一根总线或一种类型的总线。
在本申请的另一实施例中，还提供一种计算机可读存储介质，计算机可读存储介质中存储有计算机执行指令，当设备的至少一个处理器执行该计算机执行指令时，设备执行上述实施例所描述的联邦学习方法。
在本申请的另一实施例中，还提供一种计算机程序产品，该计算机程序产品包括计算机执行指令，该计算机执行指令存储在计算机可读存储介质中；设备的至少一个处理器可以从计算机可读存储介质读取该计算机执行指令，至少一个处理器执行该计算机执行指令使得设备执行上述实施例所描述的联邦学习方法。
在本申请的另一实施例中，还提供一种芯片系统，该芯片系统包括至少一个处理器和接口，该接口用于接收数据和/或信号，至少一个处理器用于支持实现上述实施例所描述的联邦学习方法。在一种可能的设计中，芯片系统还可以包括存储器，存储器，用于保存计算机设备必要的程序指令和数据。该芯片系统，可以由芯片构成，也可以包含芯片和其他分立器件。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统，装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统，装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，read-only memory)、随机存取存储器(RAM，random access memory)、磁碟或者光盘等各种可以存储程序代码的介质。

Claims (22)

  1. 一种联邦学习方法，其特征在于，包括：
    服务器向多个终端下发全局模型，所述全局模型包括多个本地模型，所述多个本地模型与所述多个终端一一对应；
    所述服务器向所述多个终端下发随机量化指令；
    所述服务器接收所述多个终端发送的编码数据，并对所述编码数据进行解码得到所述多个终端的量化模型更新量，所述多个终端的量化模型更新量为所述多个终端根据所述随机量化指令对所述多个终端的模型更新量进行量化得到的，所述多个终端的模型更新量为所述多个终端对所述多个本地模型进行训练得到的；
    所述服务器对所述多个终端的量化模型更新量进行聚合得到新的全局模型，并向所述多个终端下发新的所述全局模型进行迭代更新，直至所述全局模型收敛。
  2. 根据权利要求1所述的方法，其特征在于，所述随机量化指令包括随机步长量化指令和/或随机量化器指令，所述多个终端的量化模型更新量为所述多个终端根据所述随机步长量化指令和/或随机量化器指令对所述多个终端的模型更新量使用随机步长量化方法和/或随机量化器得到的。
  3. 根据权利要求2所述的方法，其特征在于，所述随机步长量化方法使用的随机量化步长满足随机均匀分布。
  4. 根据权利要求2所述的方法，其特征在于，所述随机量化器包括向上量化器和向下量化器，所述向上量化器和所述向下量化器的数量相同。
  5. 根据权利要求1-4中任一项所述的方法，其特征在于，在所述迭代更新中使用的所述随机量化指令为多种。
  6. 一种联邦学习方法，其特征在于，包括：
    终端接收服务器下发的本地模型和随机量化指令；
    所述终端对所述本地模型进行训练得到模型更新量；
    所述终端根据所述随机量化指令对所述模型更新量进行随机量化，得到量化模型更新量；
    所述终端对所述量化模型更新量进行编码得到编码数据，并将所述编码数据发送给所述服务器；
    所述终端接收所述服务器下发的新的本地模型和新的随机量化指令进行迭代更新，直至所述全局模型收敛，新的所述本地模型为所述服务器对多个所述终端的量化模型更新量进行聚合得到的，多个所述终端的量化模型更新量为所述服务器对多个所述终端发送的所述编码数据进行解码得到的。
  7. 根据权利要求6所述的方法，其特征在于，所述随机量化指令包括随机步长量化指令和/或随机量化器指令，所述终端根据所述随机量化指令对所述模型更新量进行随机量化包括：
    所述终端根据所述随机步长量化指令和/或随机量化器指令对所述模型更新量使用随机步长量化方法和/或随机量化器进行随机量化。
  8. 根据权利要求7所述的方法，其特征在于，所述随机步长量化方法使用的随机量化步长满足随机均匀分布。
  9. 根据权利要求7所述的方法，其特征在于，所述随机量化器包括向上量化器和向下量化器，所述向上量化器和所述向下量化器的数量相同。
  10. 根据权利要求6-9中任一项所述的方法，其特征在于，在所述迭代更新中使用的所述随机量化指令为多种。
  11. 一种服务器，其特征在于，包括：
    第一下发单元，用于向多个终端下发全局模型，所述全局模型包括多个本地模型，所述多个本地模型与所述多个终端一一对应；
    第二下发单元，用于向所述多个终端下发随机量化指令；
    接收单元，用于接收所述多个终端发送的编码数据，并对所述编码数据进行解码得到所述多个终端的量化模型更新量，所述多个终端的量化模型更新量为所述多个终端根据所述随机量化指令对所述多个终端的模型更新量进行量化得到的，所述多个终端的模型更新量为所述多个终端对所述多个本地模型进行训练得到的；
    聚合单元，用于对所述多个终端的量化模型更新量进行聚合得到新的全局模型，并向所述多个终端下发新的所述全局模型进行迭代更新，直至所述全局模型收敛。
  12. 根据权利要求11所述的服务器，其特征在于，所述随机量化指令包括随机步长量化指令和/或随机量化器指令，所述多个终端的量化模型更新量为所述多个终端根据所述随机步长量化指令和/或随机量化器指令对所述多个终端的模型更新量使用随机步长量化方法和/或随机量化器得到的。
  13. 根据权利要求12所述的服务器，其特征在于，所述随机步长量化方法使用的随机量化步长满足随机均匀分布。
  14. 根据权利要求12所述的服务器，其特征在于，所述随机量化器包括向上量化器和向下量化器，所述向上量化器和所述向下量化器的数量相同。
  15. 根据权利要求11-14中任一项所述的服务器，其特征在于，在所述迭代更新中使用的所述随机量化指令为多种。
  16. 一种终端，其特征在于，包括：
    第一接收单元，用于接收服务器下发的本地模型和随机量化指令；
    训练单元，用于对所述本地模型进行训练得到模型更新量；
    量化单元，用于据所述随机量化指令对所述模型更新量进行随机量化，得到量化模型更新量；
    编码单元，用于对所述量化模型更新量进行编码得到编码数据，并将所述编码数据发送给所述服务器；
    第二接收单元，用于接收所述服务器下发的新的本地模型和新的随机量化指令进行迭代更新，直至所述全局模型收敛，新的所述本地模型为所述服务器对多个所述终端的量化模型更新量进行聚合得到的，多个所述终端的量化模型更新量为所述服务器对多个所述终端发送的所述编码数据进行解码得到的。
  17. 根据权利要求16所述的终端，其特征在于，所述随机量化指令包括随机步长量化指令和/或随机量化器指令，所述终端根据所述随机量化指令对所述模型更新量进行随机量化包括：
    所述终端根据所述随机步长量化指令和/或随机量化器指令对所述模型更新量使用随机步长量化方法和/或随机量化器进行随机量化。
  18. 根据权利要求17所述的终端，其特征在于，所述随机步长量化方法使用的随机量化步长满足随机均匀分布。
  19. 根据权利要求17所述的终端，其特征在于，所述随机量化器包括向上量化器和向下量化器，所述向上量化器和所述向下量化器的数量相同。
  20. 根据权利要求16-19中任一项所述的终端，其特征在于，在所述迭代更新中使用的所述随机量化指令为多种。
  21. 一种计算机设备，包括：处理器、通信接口和存储器，所述存储器用于存储程序代码，所述处理器用于调用所述存储器中的程序代码以使得所述处理器执行如权利要求1至5或6至10中任一项所述的方法。
  22. 一种计算机可读存储介质，存储有指令，当所述指令在计算机上运行时，使得计算机执行如权利要求1至5或6至10中任一项所述的方法。
PCT/CN2022/133861 2021-12-02 2022-11-24 一种联邦学习方法及相关设备 WO2023098546A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP22900351.2A EP4365785A1 (en) 2021-12-02 2022-11-24 Federated learning method and related device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111463299.0 2021-12-02
CN202111463299.0A CN116227630A (zh) 2021-12-02 2021-12-02 一种联邦学习方法及相关设备

Publications (1)

Publication Number Publication Date
WO2023098546A1 true WO2023098546A1 (zh) 2023-06-08

Family

ID=86585930

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/133861 WO2023098546A1 (zh) 2021-12-02 2022-11-24 一种联邦学习方法及相关设备

Country Status (3)

Country Link
EP (1) EP4365785A1 (zh)
CN (1) CN116227630A (zh)
WO (1) WO2023098546A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117424765A (zh) * 2023-12-19 2024-01-19 天津医康互联科技有限公司 分布式独热编码方法、装置、电子设备及计算机存储介质
CN117521856A (zh) * 2023-12-29 2024-02-06 南京邮电大学 一种基于本地特征的大模型切割联邦学习方法及系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112906046A (zh) * 2021-01-27 2021-06-04 清华大学 一种利用单比特压缩感知技术的模型训练方法和装置
CN113098806A (zh) * 2021-04-16 2021-07-09 华南理工大学 一种联邦学习下边端协同的信道适应性梯度压缩方法
CN113258935A (zh) * 2021-05-25 2021-08-13 山东大学 一种联邦学习中基于模型权值分布的通信压缩方法
CN113315604A (zh) * 2021-05-25 2021-08-27 电子科技大学 一种联邦学习自适应梯度量化方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112906046A (zh) * 2021-01-27 2021-06-04 清华大学 一种利用单比特压缩感知技术的模型训练方法和装置
CN113098806A (zh) * 2021-04-16 2021-07-09 华南理工大学 一种联邦学习下边端协同的信道适应性梯度压缩方法
CN113258935A (zh) * 2021-05-25 2021-08-13 山东大学 一种联邦学习中基于模型权值分布的通信压缩方法
CN113315604A (zh) * 2021-05-25 2021-08-27 电子科技大学 一种联邦学习自适应梯度量化方法

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117424765A (zh) * 2023-12-19 2024-01-19 天津医康互联科技有限公司 分布式独热编码方法、装置、电子设备及计算机存储介质
CN117424765B (zh) * 2023-12-19 2024-03-22 天津医康互联科技有限公司 分布式独热编码方法、装置、电子设备及计算机存储介质
CN117521856A (zh) * 2023-12-29 2024-02-06 南京邮电大学 一种基于本地特征的大模型切割联邦学习方法及系统
CN117521856B (zh) * 2023-12-29 2024-03-15 南京邮电大学 一种基于本地特征的大模型切割联邦学习方法及系统

Also Published As

Publication number Publication date
CN116227630A (zh) 2023-06-06
EP4365785A1 (en) 2024-05-08

Similar Documents

Publication Publication Date Title
WO2023098546A1 (zh) 一种联邦学习方法及相关设备
US11017322B1 (en) Method and system for federated learning
Gafni et al. Federated learning: A signal processing perspective
CN111553483B (zh) 基于梯度压缩的联邦学习的方法、装置及系统
US10664169B2 (en) Performance of object storage system by reconfiguring storage devices based on latency that includes identifying a number of fragments that has a particular storage device as its primary storage device and another number of fragments that has said particular storage device as its replica storage device
CN110262819B (zh) 一种联邦学习的模型参数更新方法及装置
US20180356999A1 (en) Securely recovering stored data in a dispersed storage network
CN110741573A (zh) 在区块链网络中选择性使用网络编码传播交易的方法和系统
WO2019227831A1 (zh) 编码方法、设备及可读存储介质
CN113435534A (zh) 一种基于相似度度量的数据异构处理方法、装置、计算机设备及计算机可读存储介质
CN115841133A (zh) 一种联邦学习方法、装置、设备及存储介质
CN111246462B (zh) 一种终端与电能表间的数据安全传输方法、系统及设备
Chen et al. Communication-efficient design for quantized decentralized federated learning
US20190065766A1 (en) Controlling access when processing intents in a dispersed storage network
CN112104867A (zh) 一种视频处理方法、视频处理装置、智能设备及存储介质
Chen et al. Rate distortion optimization for adaptive gradient quantization in federated learning
CN115952539A (zh) 多数恶意下鲁棒隐私性联邦学习方法、系统、设备及介质
CN113887746A (zh) 基于联合学习的降低通信压力的方法及装置
WO2022068200A1 (zh) 缺陷预测方法和装置、存储介质和电子装置
CN116340959A (zh) 一种面向断点隐私保护的方法、装置、设备及介质
CN114780224A (zh) 一种应用于元宇宙的资源调度方法及系统
CN114707663A (zh) 分布式机器学习方法、装置、电子设备及存储介质
CN115277031A (zh) 一种数据处理的方法和装置
WO2023124312A1 (zh) 联合学习中的预测方法及装置
Chen et al. NQFL: Nonuniform Quantization for Communication Efficient Federated Learning

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22900351

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2022900351

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2022900351

Country of ref document: EP

Effective date: 20240129