WO2023284387A1

WO2023284387A1 - Model training method, apparatus, and system based on federated learning, and device and medium

Info

Publication number: WO2023284387A1
Application number: PCT/CN2022/091868
Authority: WO
Inventors: 陈录城; 诸葛慧玲; 张成龙; 孙明; 贾淇超; 李晓璐
Original assignee: 卡奥斯工业智能研究院(青岛)有限公司; 海尔卡奥斯物联生态科技有限公司; 海尔数字科技(青岛)有限公司
Priority date: 2021-07-15
Filing date: 2022-05-10
Publication date: 2023-01-19
Also published as: CN113537513A

Abstract

The embodiments of the present application relate to a model training method, apparatus, and system based on federated learning, and a device and a medium. The method comprises: implementing training of a local model on the basis of local data, and sending algorithm parameters of the trained local model to a public cloud server, such that the public cloud server verifies whether the algorithm parameters of a joint model need to be updated using the received algorithm parameters of the trained local model; receiving updated algorithm parameters of the joint model pushed by the public cloud server; verifying whether the algorithm parameters of the trained local model need to be updated using the received updated algorithm parameters of the joint model; and, when verifying that the algorithm parameters of the trained local model need to be updated using the received updated algorithm parameters of the joint model, updating the algorithm parameters of the trained local model to the received updated algorithm parameters of the joint model.

Description

Model training method, device, system, equipment and medium based on federated learning

This application claims the priority of the Chinese patent application with application number 202110799024.8 submitted to the China Patent Office on July 15, 2021, the entire content of which is incorporated herein by reference.

technical field

The embodiments of the present application relate to the technical field of artificial intelligence, for example, to a model training method, device, system, electronic device, and storage medium based on federated learning.

Background technique

Industrial data is the core of industrial informatization, especially manufacturing enterprises have a high degree of dependence on industrial data in production operations, such as process parameters, equipment operation data, production data and a series of industrial data are key data affecting manufacturing production, these Data security is directly related to the stable operation of the manufacturing production line. Data loss, malicious tampering or errors will lead to the shutdown of the entire production line and cause huge losses to industrial production. In addition, the leakage of industrial data of key enterprises related to the national economy and people's livelihood will also affect national security. Out of the protection of industrial data, manufacturing companies do not share or transmit data externally due to data security considerations, resulting in the formation of data islands, which has become a challenge for the implementation and continuous optimization of artificial intelligence technology in industrial scenarios. In the process of industrial intelligence, how to effectively protect data security and use sufficient data volume for model training and solve the problem of continuous optimization of artificial intelligence models has become the key to the development of manufacturing technology.

In order to ensure the data security of industrial enterprises, most of them adopt private cloud solutions. The data does not leave the factory, and the data is safe and reliable. However, because the data is not open and shared, the industrial data is scattered and the high-quality data is scarce, which affects the data model training and optimization, which is not enough to support The implementation of artificial intelligence technology.

Contents of the invention

The embodiments of the present application provide a model training method, device, electronic equipment, and storage medium based on federated learning, so as to ensure the effect of model training while ensuring the security of industrial data.

The embodiment of the present application provides a model training method based on federated learning, which is executed by multiple private cloud servers, including: training the local model based on local data, and sending the algorithm parameters of the trained local model to the public cloud server, To enable the public cloud server to verify whether it is necessary to update the algorithm parameters of the joint model using the received algorithm parameters of the trained local model; receive the algorithm parameters of the updated joint model pushed by the public cloud server; verify whether The received algorithm parameters of the updated joint model need to be used to update the algorithm parameters of the trained local model; the received algorithm parameters of the updated joint model need to be used to update the trained local model. In the case of the algorithm parameters of the model, update the algorithm parameters of the trained local model to the received algorithm parameters of the updated joint model.

In an embodiment, the verifying whether it is necessary to update the algorithm parameters of the trained local model by using the received algorithm parameters of the updated joint model includes: calculating the trained local model by using a priori data set. The effect index of the model obtains the first index value; after replacing the algorithm parameters of the trained local model with the received algorithm parameters of the updated joint model, the algorithm is replaced by using the prior data set calculation The effect index of the parameterized local model obtains a second index value; determine whether to use the received algorithm parameters of the updated joint model to update the Algorithm parameters of the trained local model.

In one embodiment, the public cloud server verifying whether it is necessary to use the received algorithm parameters of the trained local model to update the algorithm parameters of the joint model includes: calculating the effect of the joint model using a priori data set The index obtains a third index value; after replacing the algorithm parameters of the joint model with the algorithm parameters of the received local model after training, using the prior data set to calculate the value of the joint model after replacing the algorithm parameters The effect index obtains a fourth index value; determine whether to use the received algorithm parameters of the trained local model to update the algorithm parameters of the joint model according to the magnitude of the third index value and the fourth index value.

In one embodiment, the performance index includes precision and/or recall.

In one embodiment, before the training of the local model based on the local data, it also includes receiving the algorithm parameters of the initial model issued by the public cloud server; the training of the local model based on the local data includes: based on the initial Algorithmic parameters of the model and the local data train the local model.

The embodiment of the present application also provides a model training device based on federated learning, which is configured in multiple private cloud servers. The device includes: a local training and parameter uploading unit, which is configured to train the local model based on local data. The algorithm parameters of the trained local model are sent to the public cloud server, so that the public cloud server verifies whether the algorithm parameters of the joint model need to be updated with the received algorithm parameters; the joint model parameter receiving unit is configured to receive the The algorithm parameters of the updated joint model pushed by the public cloud server; the verification and update unit is configured to verify whether the received algorithm parameters of the updated joint model need to be used to update the algorithm parameters of the trained local model, In the case that the algorithm parameters of the trained local model need to be updated with the received algorithm parameters of the joint model, update the algorithm parameters of the trained local model to the received Algorithm parameters for the updated joint model.

In an embodiment, the verifying and updating unit is configured to verify whether it is necessary to use the received algorithm parameters of the updated joint model to update the algorithm parameters of the trained local model, including: using a priori data set to calculate The effect index of the trained local model obtains a first index value; after replacing the algorithm parameters of the trained local model with the received algorithm parameters of the updated joint model, the prior data is used Set calculating the effect index of the local model after replacing the algorithm parameters to obtain a second index value; determine whether to use the received updated joint model according to the size of the first index value and the second index value Algorithm parameters for updating the algorithm parameters of the trained local model.

In one embodiment, the verification by the public cloud server in the local training and parameter uploading unit whether the received algorithm parameters need to be used to update the algorithm parameters of the joint model includes: using a priori data set to calculate the algorithm parameters of the joint model The effect index obtains the third index value; after replacing the algorithm parameters of the joint model with the received algorithm parameters of the trained local model, the joint model after replacing the algorithm parameters is calculated by using the prior data set Obtaining the fourth index value of the effect index; determining whether to use the received algorithm parameters of the trained local model to update the algorithm parameters of the joint model according to the size of the third index value and the fourth index value .

In one embodiment, the performance index includes precision and/or recall.

In one embodiment, the device further includes an initial model parameter receiving unit configured to receive the algorithm parameters of the initial model issued by the public cloud server before the local model is trained based on the local data; the local training and The parameter uploading unit is configured to train the local model based on the algorithm parameters of the initial model and the local data.

Also provided is a model training system based on federated learning, including a public cloud server and multiple private cloud servers; the multiple private cloud servers train the local model based on local data, and send the algorithm parameters of the trained local model to To the public cloud server; the public cloud server verifies whether the received algorithm parameters of the trained local model need to be used to update the algorithm parameters of the joint model, and the received trained local model needs to be used for verification In the case of updating the algorithm parameters of the joint model using the algorithm parameters of the local model, update the algorithm parameters of the joint model by using the received algorithm parameters of the local model, and push the updated algorithm parameters of the joint model to the multiple Private cloud server; when the plurality of private cloud servers receive the algorithm parameters of the updated joint model pushed by the public cloud server, verify whether the received updated joint model needs to be used Algorithm parameters update the model parameters of the trained local model, and in the case that the received algorithm parameters of the updated joint model need to be used to update the trained local model algorithm parameters, the trained The algorithm parameters of the local model are updated to the received algorithm parameters of the updated joint model.

In one embodiment, before the multiple private cloud servers train the local model based on local data, the system further includes: the public cloud server sends the algorithm parameters of the initial model to the multiple private cloud servers, The multiple private cloud servers train the local model based on the algorithm parameters of the initial model and the local data.

An electronic device is also provided. The electronic device includes: a processor; and a memory configured to store executable instructions, and when the executable instructions are executed by the processor, the electronic device executes the methods of the foregoing embodiments.

A computer-readable storage medium is also provided, on which a computer program is stored, and when the computer program is executed by a processor, the methods of the above-mentioned embodiments are implemented.

Description of drawings

The following will give a brief introduction to the drawings that need to be used in the description of the embodiments of the present application. Obviously, the drawings in the following description are only part of the embodiments of the present application. For those of ordinary skill in the art, On the premise of no creative work, other drawings can also be obtained according to the content of the embodiment of the present application and these drawings.

Fig. 1 is a schematic flowchart of a model training method based on federated learning provided according to an embodiment of the present application;

FIG. 2 is a schematic flowchart of a method of a federated learning-based model training system provided according to an embodiment of the present application;

FIG. 3A is a schematic diagram of another federated learning-based model training system method provided according to an embodiment of the present application;

FIG. 3B is a schematic flowchart of another federated learning-based model training system method provided according to an embodiment of the present application;

Fig. 4 is a schematic structural diagram of a model training device based on federated learning provided according to an embodiment of the present application;

FIG. 5 is a schematic structural diagram of another model training device based on federated learning provided according to an embodiment of the present application;

FIG. 6 shows a schematic structural diagram of an electronic device suitable for implementing the embodiments of the present application.

detailed description

The technical solutions of the embodiments of the present application will be described below with reference to the accompanying drawings. The described embodiments may be part of the embodiments of the present application. Based on the embodiments of the present application, those skilled in the art can obtain other embodiments without making creative efforts, and these embodiments should all belong to the protection scope of the present application.

It should be noted that the terms "system" and "network" are often used interchangeably herein, and "and/or" mentioned herein refers to any and all combinations including one or more related listed items. The terms "first", "second", etc. in are used to distinguish different objects, not to limit a specific order.

It should also be noted that the following multiple embodiments can be implemented independently, and multiple embodiments can also be implemented in combination with each other, which is not limited in this embodiment of the present application.

The names of messages or information exchanged between multiple devices herein are for illustrative purposes only, and are not used to limit the scope of these messages or information.

The technical solutions of the embodiments of the present application will be described below in conjunction with the accompanying drawings and through specific implementation methods.

FIG. 1 shows a schematic flowchart of a model training method based on federated learning provided by an embodiment of the present application. This embodiment is applicable to the situation where multiple private cloud servers train models through federated learning, and the method can be executed by a model training device based on federated learning configured on multiple private cloud servers. As shown in FIG. 1 , the model training method based on federated learning described in this embodiment includes the following steps.

In step S110, the local model is trained based on local data, and the algorithm parameters of the trained local model are sent to the public cloud server, so that the public cloud server verifies whether the received trained local model needs to be adopted The algorithm parameters of update the algorithm parameters of the joint model.

When the public cloud server verifies whether it is necessary to use the received algorithm parameters of the trained local model to update the algorithm parameters of the joint model, it can use a priori data set for verification to determine if the joint model uses the algorithm parameters of the private cloud server. The algorithm parameters of the trained local model, and whether the training effect of the joint model is better (for example, whether the model accuracy is higher). The effect index of the joint model can be calculated by using the prior data set to obtain the third index value, and after the algorithm parameters of the joint model are replaced with the received algorithm parameters of the local model after training, the replacement is calculated using the prior data set The fourth index value is obtained from the effect index of the joint model after the algorithm parameters, and it is determined whether it is necessary to use the received algorithm parameters of the trained local model to update the joint model according to the size of the third index value and the fourth index value. Algorithmic parameters for the model.

In step S120, the algorithm parameters of the updated joint model pushed by the public cloud server are received.

In step S130, verify whether it is necessary to use the received algorithm parameters of the updated joint model to update the algorithm parameters of the trained local model, if necessary, use the received algorithm parameters of the updated joint model To update the algorithm parameters of the trained local model, execute step S140, if it is not necessary to update the algorithm parameters of the trained local model by using the received algorithm parameters of the updated joint model, then return to step S110 .

For example, a priori data set may be used to calculate the effect index of the trained local model to obtain a first index value, and the algorithm parameters of the trained local model may be replaced with the received algorithm of the updated joint model After parameterizing, use the prior data set to calculate the effect index of the local model after replacing the algorithm parameters to obtain the second index value, and determine whether to use the received The algorithm parameters of the updated joint model update the algorithm parameters of the trained local model.

In step S140, the algorithm parameters of the trained local model are updated to the received algorithm parameters of the updated joint model.

The effect index may include various types, including but not limited to the accuracy rate of model prediction, recall rate, and the like.

According to one or more embodiments of the present disclosure, before training the local model based on local data, multiple private cloud servers can also receive the algorithm parameters of the initial model issued by the public cloud server, based on the algorithm parameters of the initial model and The local data trains the local model to synchronize the initial state of the local model on multiple private cloud servers.

In this embodiment, multiple private cloud servers are used to train the local model based on local data, and the algorithm parameters of the trained local model are sent to the public cloud server, so that the public cloud server verifies whether the received algorithm needs to be used Parameters update the algorithm parameters of the joint model; receive the algorithm parameters of the updated joint model pushed by the public cloud server; verify whether the received algorithm parameters of the updated joint model need to be used to update the trained local model ; If the verification needs to use the received algorithm parameters of the updated joint model to update the trained local model, update the algorithm parameters of the trained local model to the received updated joint model The algorithm parameters of the model can ensure the effect of model training while ensuring the security of industrial data.

FIG. 2 is a schematic flowchart of a model training method of a federated learning-based model training system provided according to an embodiment of the present application. The model training system based on federated learning described in this embodiment includes a public cloud server and multiple private cloud servers. As shown in FIG. 2 , the model training method of the federated learning-based model training system described in this embodiment includes the following steps.

In step S210, multiple private cloud servers train local models based on local data, and send algorithm parameters of the trained local models to the public cloud server.

In step S220, the public cloud server verifies whether it is necessary to use the received algorithm parameters of the trained local model to update the algorithm parameters of the joint model, and if it needs to use the received algorithm parameters of the trained local model Updating the algorithm parameters of the joint model uses the received algorithm parameters of the trained local model to update the algorithm parameters of the joint model, and pushes the updated algorithm parameters of the joint model to multiple private cloud servers.

In step S230, if multiple private cloud servers receive the algorithm parameters of the updated joint model pushed by the public cloud server, step S240 is executed.

In step S240, verify whether it is necessary to use the received algorithm parameters of the updated joint model to update the algorithm parameters of the trained local model, if necessary, use the received algorithm parameters of the updated joint model To update the algorithm parameters of the trained local model, execute step S250, and return to step S210 if the received algorithm parameters of the updated joint model do not need to be used to update the algorithm parameters of the trained local model .

In step S250, the algorithm parameters of the trained local model are updated to the received algorithm parameters of the updated joint model.

According to one or more embodiments of the present disclosure, before step S210, the public cloud server may also send the algorithm parameters of the initial model to multiple private cloud servers, and the multiple private cloud servers are based on the algorithm parameters of the initial model and local data to train the local model.

While ensuring the security of industrial data, the technical solution of this embodiment satisfies the continuous optimization of industrial data models and improves the use effect of artificial intelligence (AI) technology in industrial applications without the need for open data.

FIG. 3A is a schematic diagram of a model training method of another federated learning-based model training system provided according to an embodiment of the present application. This embodiment is a safe data model training solution that performs local model training on a private cloud server and integrates and optimizes the models in multiple places into a joint model, and then feeds back to the respective private cloud servers in multiple places.

As shown in FIG. 3A , this embodiment mainly uses technologies such as federated learning, distributed computing, and algorithm model integration and optimization. Federated learning is a paradigm of distributed collaborative training of machine learning models, which can be used for collaborative training of machine learning models on a large number of edge devices (clients) without centralized training data. It is characterized by a large number of decentralized stages linked to centralized server, these participants have zero trust in each other and only have access to local training data. Distributed computing is to split a large computing task into multiple small computing tasks and distribute them to multiple machines for calculation, and then summarize the results. Distributed computing in the federated learning process is to perform calculations in multiple places to form a data model, and then Upload the model to summarize the results. The integration and optimization technology of the algorithm model is based on the definition of the same parameter, through federated learning and distributed computing, and integrates local computing and models into a joint model. In a federated machine learning setting, the global model is initialized and maintained by a central parameter server, which is then shared to edge devices. In order to train the global model through distributed collaboration, the client uses local private data to calculate and update the model, and then uploads the updated model to the server, while keeping the privacy-sensitive training data on its own device, through distributed security aggregation Over multiple iterations, the federated learning system trains the integrated model.

FIG. 3B shows a schematic flowchart of another model training method based on federated learning provided by the embodiment of the present application. This embodiment is based on the foregoing embodiments, and an improved description is made. As shown in FIG. 3B , the model training method based on federated learning described in this embodiment includes the following steps.

In step S301, the factory private cloud starts model training.

In step S302, the factory private cloud shares the algorithm parameters of the model with the joint model.

In step S303, whether the joint model verification needs to update its algorithm parameters, if the joint model verification needs to update the algorithm parameters, then perform step S305, if the joint model verification does not need to update the algorithm parameters, then perform step S304.

In step S304, the algorithm parameters of the joint model are not updated, and step S306 is executed.

In step S305, the joint parameters of the joint model are updated.

In step S306, the joint model pushes the algorithm parameters of the joint model to the factory private cloud.

In step S307, the factory private cloud judges whether the joint model is optimal, if the joint model is optimal, execute step S310, and if the joint model is not optimal, execute step S308.

In step S308, the model is not updated, and step S309 is executed.

In step S309, continue the local training and end.

In step S310, the local model is updated iteratively.

The private cloud of the factory starts the local algorithm model training and shares the trained model with the joint model. The joint model needs to perform model verification before receiving the model shared by multiple factories. If the verification does not need to update the algorithm parameters of the joint model, the algorithm will not be updated Parameters, the factory private cloud uses the information shared by multiple factories to optimize and update the model, and when it corresponds to the sharing of multiple factories, it will use the shared information of multiple factories to optimize the model. The joint model will regularly push the joint model to (multiple) factories, and the factory will verify the model after receiving the push. If the pushed joint model is better than the local model, the local model will be iteratively updated, otherwise the local model will be updated continuously. local training. The push process is that multiple local models push the updated algorithm parameters after learning to the joint model in real time during the continuous learning process to update the algorithm parameters. After the joint model is updated according to the real-time algorithm parameters, the updated joint model algorithm The parameters are reversely pushed to multiple factories. These parameters are based on unified definition rules, and the algorithm model of the factory is continuously pushed iteratively with the update of the joint model.

The technical solution of this embodiment mainly uses technologies such as federated learning, distributed computing, and algorithm model joint integration. The joint model can be placed on the public cloud. There is no need to share data in multiple places, but only the data model. The joint model is continuously optimized through the public cloud. Shared with multiple regions for continuous model optimization. The entire process data does not leave the factory, and the training results of the local data model are shared to improve the use effect of industrial AI technology. While ensuring the security of industrial data, it satisfies the continuous optimization of industrial data models and improves the use effect of AI technology in industrial applications without the need for open data.

As an implementation of the methods shown in the above figures, this application provides an embodiment of a model training device based on federated learning. Figure 4 shows the structure of a model training device based on federated learning provided in this embodiment Schematic diagram, the device embodiment corresponds to the method embodiment shown in FIG. 1 , and the device can be applied to various electronic devices in multiple private cloud servers. As shown in FIG. 4 , the model training device based on federated learning described in this embodiment includes a local training and parameter uploading unit 410 , a joint model parameter receiving unit 420 and a verification and updating unit 430 .

The local training and parameter uploading unit 410 is configured to train the local model based on local data, and send the algorithm parameters of the trained local model to the public cloud server, so that the public cloud server verifies whether the received The algorithm parameters of the update joint model algorithm parameters.

The joint model parameter receiving unit 420 is configured to receive the updated algorithm parameters of the joint model pushed by the public cloud server.

The verification and updating unit 430 is configured to verify whether it is necessary to use the received algorithm parameters of the updated joint model to update the algorithm parameters of the trained local model, and if the verification needs to use the received update If the algorithm parameters of the updated joint model update the algorithm parameters of the trained local model, the algorithm parameters of the trained local model are updated to the received algorithm parameters of the updated joint model.

According to one or more embodiments of the present disclosure, the verification and updating unit 430 is configured to: use the prior data set to calculate the effect index of the trained local model to obtain a first index value; After the algorithm parameters of the updated local model are replaced with the received algorithm parameters of the updated joint model, the effect index of the local model after replacing the algorithm parameters is calculated using the prior data set to obtain a second index value; according to the The values of the first index value and the second index value determine whether to use the received algorithm parameters of the updated joint model to update the algorithm parameters of the trained local model.

According to one or more embodiments of the present disclosure, the verification by the public cloud server in the local training and parameter uploading unit 410 whether it is necessary to use the received algorithm parameters to update the algorithm parameters of the joint model includes: using a priori data set Calculating the effect index of the joint model to obtain the third index value; after replacing the algorithm parameters of the joint model with the received algorithm parameters of the trained local model, using the prior data set to calculate the joint model after replacing the algorithm parameters Obtaining a fourth index value based on the effect index of the result index; determining whether to use the received algorithm parameters of the trained local model to update the algorithm parameters of the joint model according to the magnitude of the third index value and the fourth index value.

According to one or more embodiments of the present disclosure, the performance index includes precision and/or recall.

According to one or more embodiments of the present disclosure, the device further includes an initial model parameter receiving unit configured to receive the algorithm parameters of the initial model issued by the public cloud server before training the local model based on the local data;

The local training and parameter uploading unit being configured to train the local model based on local data includes: training the local model based on algorithm parameters of the initial model and local data.

The model training device based on federated learning provided in this embodiment can execute the model training method based on federated learning provided in the method embodiment of the present disclosure, and has corresponding functional modules for executing the method.

FIG. 5 shows a schematic structural diagram of another model training device based on federated learning provided by an embodiment of the present application. As shown in FIG. 5 , the model training device based on federated learning in this embodiment includes an initial model parameter receiving unit 510 , a local training and parameter uploading unit 520 , a joint model parameter receiving unit 530 and a verification and updating unit 540 .

The initial model parameter receiving unit 510 is configured to receive the algorithm parameters of the initial model issued by the public cloud server.

The local training and parameter uploading unit 520 is configured such that training the local model based on local data includes: training the local model based on the algorithm parameters of the initial model and local data, and sending the algorithm parameters of the trained local model to to the public cloud server, so that the public cloud server verifies whether the received algorithm parameters need to be used to update the algorithm parameters of the joint model.

The joint model parameter receiving unit 530 is configured to receive the updated algorithm parameters of the joint model pushed by the public cloud server.

The verification and updating unit 540 is configured to, if the verification needs to use the received algorithm parameters of the updated joint model to update the algorithm parameters of the trained local model, then update the algorithm parameters of the trained local model to The algorithm parameters are updated to the received algorithm parameters of the updated joint model.

According to one or more embodiments of the present disclosure, the verification and updating unit 540 is configured to verify whether it is necessary to use the received algorithm parameters of the updated joint model to update the algorithm parameters of the trained local model including : Using the prior data set to calculate the effect index of the trained local model to obtain the first index value; after replacing the algorithm parameters of the trained local model with the received algorithm parameters of the updated joint model , using the prior data set to calculate the effect index of the local model after replacing the algorithm parameters to obtain a second index value; determine whether to use the received The updated algorithm parameters of the joint model update the algorithm parameters of the trained local model.

According to one or more embodiments of the present disclosure, the verification of whether the public cloud server in the local training and parameter uploading sheet 520 needs to use the received algorithm parameters to update the algorithm parameters of the joint model includes: using a priori data set Calculating the effect index of the joint model to obtain the third index value; after replacing the algorithm parameters of the joint model with the received algorithm parameters of the trained local model, using the prior data set to calculate the joint model after replacing the algorithm parameters Obtaining a fourth index value based on the effect index of the method; determining whether to use the received algorithm parameters of the trained local model to update the algorithm parameters of the joint model according to the magnitude of the third index value and the fourth index value.

Referring to FIG. 6 , it shows a schematic structural diagram of an electronic device 600 suitable for implementing the embodiment of the present application. The above-mentioned terminal device in the embodiment of the present application is, for example, a mobile device, a computer, or a vehicle-mounted device built in a floating car, or any combination thereof. In some embodiments, the mobile device may include, for example, a mobile phone, a smart home device, a wearable device, a smart mobile device, a virtual reality device, etc., or any combination thereof. The electronic device shown in FIG. 6 is only an example, and should not limit the functions and scope of use of this embodiment of the present application.

As shown in FIG. 6, an electronic device 600 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) Various appropriate actions and processes are executed by a program loaded into a random access memory (Random Access Memory, RAM) 603 by 608 . In the RAM 603, various programs and data necessary for the operation of the electronic device 600 are also stored. The processing device 601, ROM 602, and RAM 603 are connected to each other through a bus 604. An input/output (Input/Output, I/O) interface 605 is also connected to the bus 604 .

Typically, the following devices can be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, a liquid crystal display (LCD), speaker, vibration an output device 607 such as a computer; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication means 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. While FIG. 6 shows electronic device 600 having various means, it should be understood that implementing or possessing all of the means shown is not a requirement. More or fewer means may alternatively be implemented or provided.

According to an embodiment of an embodiment of the present application, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, the embodiments of the present application include a computer program product, which includes a computer program carried on a computer-readable medium, where the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network via communication means 609, or from storage means 608, or from ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present application are performed.

It should be noted that the above-mentioned computer-readable medium in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. Examples of computer readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, RAM, ROM, Erasable Programmable Read-Only Memory, EPROM) or flash memory, optical fiber, portable compact disk read-only memory (Compact Disc Read-Only Memory, CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In the embodiments of the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, and the program may be used by or in combination with an instruction execution system, device or device. However, in the embodiment of the present application, the computer-readable signal medium may include a data signal propagated in the baseband or as a part of the carrier wave, and the computer-readable program code is carried therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can transmit, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device . The program code contained on the computer readable medium can be transmitted by any appropriate medium, including but not limited to: electric wire, optical cable, radio frequency (Radio Frequency, RF), etc., or any suitable combination of the above.

The above-mentioned computer-readable medium may be included in the above-mentioned electronic device, or may exist independently without being incorporated into the electronic device.

The above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the electronic device, the electronic device: trains the local model based on local data, and converts the algorithm parameters of the trained local model to Send to the public cloud server, so that the public cloud server verifies whether the algorithm parameters of the joint model need to be updated by using the received algorithm parameters of the trained local model; receive the updated joint model pushed by the public cloud server Algorithm parameters; verify whether it is necessary to use the received algorithm parameters of the updated joint model to update the algorithm parameters of the trained local model, if the verification needs to use the received algorithm parameters of the updated joint model Updating the algorithm parameters of the trained local model is updating the algorithm parameters of the trained local model to the received algorithm parameters of the updated joint model.

Computer program codes for performing the operations of the embodiments of the present application may be written in one or more programming languages or a combination thereof, and the above-mentioned programming languages include object-oriented programming languages—such as Java, Smalltalk, C++, and A conventional procedural programming language - such as the "C" language or a similar programming language. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user computer via any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or, alternatively, may be connected to an external computer (e.g., via the Internet) service provider via Internet connection).

The flow charts and block diagrams in the accompanying drawings illustrate the system architecture, functions and operations of possible implementations of systems, methods and computer program products according to various embodiments of the embodiments of the present application. In this regard, each block in a flowchart or block diagram may represent a module, program segment, or portion of code that contains one or more logical functions for implementing specified executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by a dedicated hardware-based system that performs the specified functions or operations , or may be implemented by a combination of dedicated hardware and computer instructions.

The units involved in the embodiments described in the present application may be implemented by means of software or by means of hardware. The name of the unit does not limit the unit itself in some cases, for example, the first obtaining unit may also be described as "a unit for obtaining at least two Internet Protocol addresses".

Claims

A model training method based on federated learning, executed by multiple private cloud servers, including:

The local model is trained based on local data, and the algorithm parameters of the trained local model are sent to the public cloud server, so that the public cloud server verifies whether it is necessary to use the received algorithm parameters of the trained local model to update the joint Algorithmic parameters of the model;

receiving the algorithm parameters of the updated joint model pushed by the public cloud server;

Verifying whether the received algorithm parameters of the updated joint model need to be used to update the algorithm parameters of the trained local model;

In the case that the algorithm parameters of the trained local model need to be updated with the received algorithm parameters of the joint model, update the algorithm parameters of the trained local model to the received Algorithm parameters for the updated joint model.
The method according to claim 1, wherein the verifying whether the received algorithm parameters of the updated joint model need to be used to update the algorithm parameters of the trained local model comprises:

calculating an effect index of the trained local model by using a priori data set to obtain a first index value;

After replacing the algorithm parameters of the trained local model with the received algorithm parameters of the updated joint model, using the prior data set to calculate the effect index of the local model after replacing the algorithm parameters to obtain the first Two index values;

Determine whether to use the received algorithm parameters of the updated joint model to update the algorithm parameters of the trained local model according to the magnitudes of the first index value and the second index value.
The method according to claim 1, wherein the verification by the public cloud server whether it is necessary to use the received algorithm parameters of the trained local model to update the algorithm parameters of the joint model comprises:

Using the prior data set to calculate the effect index of the joint model to obtain the third index value;

After replacing the algorithm parameters of the joint model with the received algorithm parameters of the trained local model, using the prior data set to calculate the effect index of the joint model after replacing the algorithm parameters to obtain a fourth index value;

Determine whether to use the received algorithm parameters of the trained local model to update the algorithm parameters of the joint model according to the magnitude of the third index value and the fourth index value.
The method according to claim 2 or 3, wherein the effect index includes at least one of precision rate and recall rate.
The method according to claim 1, before said training the local model based on the local data, further comprising, receiving the algorithm parameters of the initial model sent by the public cloud server;

The training the local model based on local data includes: training the local model based on algorithm parameters of the initial model and the local data.
A model training device based on federated learning, configured in multiple private cloud servers, including:

The local training and parameter uploading unit is configured to train the local model based on local data, and send the algorithm parameters of the trained local model to the public cloud server, so that the public cloud server verifies whether it needs to adopt the received training Algorithm parameters of the subsequent local model update the algorithm parameters of the joint model;

The joint model parameter receiving unit is configured to receive the algorithm parameters of the updated joint model pushed by the public cloud server;

The verification and update unit is configured to verify whether the received algorithm parameters of the updated joint model need to be used to update the algorithm parameters of the trained local model, and the received updated joint model needs to be used for verification In a case where the algorithm parameters of the trained local model are updated, the algorithm parameters of the trained local model are updated to the received algorithm parameters of the updated joint model.
A model training system based on federated learning, including a public cloud server and multiple private cloud servers;

The plurality of private cloud servers train the local model based on local data, and send the algorithm parameters of the trained local model to the public cloud server;

The public cloud server verifies whether the algorithm parameters of the joint model need to be updated by using the received algorithm parameters of the trained local model, and the algorithm parameters of the joint model need to be updated by using the received algorithm parameters of the trained local model. In the case of a model, update the algorithm parameters of the joint model by using the received algorithm parameters of the trained local model, and push the updated algorithm parameters of the joint model to the plurality of private cloud servers;

When the plurality of private cloud servers receive the algorithm parameters of the updated joint model pushed by the public cloud server, verify whether it is necessary to use the received algorithm parameters of the updated joint model to update the Algorithm parameters of the trained local model, in the case that the algorithm parameters of the trained local model need to be updated by using the received algorithm parameters of the updated joint model, the trained local model The algorithm parameters of are updated to the received algorithm parameters of the updated joint model.
The system according to claim 1, before the plurality of private cloud servers train the local model based on local data, it also includes:

The public cloud server sends the algorithm parameters of the initial model to the multiple private cloud servers, and the multiple private cloud servers train the local model based on the algorithm parameters of the initial model and the local data.
An electronic device comprising:

at least one processor; and

A memory configured to store executable instructions, which, when executed by the at least one processor, cause the electronic device to execute the method according to any one of claims 1-5.
A computer-readable storage medium, on which a computer program is stored, wherein, when the computer program is executed by a processor, the method according to any one of claims 1-5 is implemented.