CN116126451A - Federal learning workflow construction method based on blockchain network and related equipment - Google Patents

Federal learning workflow construction method based on blockchain network and related equipment Download PDF

Info

Publication number
CN116126451A
CN116126451A CN202310370151.5A CN202310370151A CN116126451A CN 116126451 A CN116126451 A CN 116126451A CN 202310370151 A CN202310370151 A CN 202310370151A CN 116126451 A CN116126451 A CN 116126451A
Authority
CN
China
Prior art keywords
training
initiator
task
blockchain
local
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310370151.5A
Other languages
Chinese (zh)
Inventor
王光宇
高天润
吕鹏帅
张平
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing University of Posts and Telecommunications
Original Assignee
Beijing University of Posts and Telecommunications
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing University of Posts and Telecommunications filed Critical Beijing University of Posts and Telecommunications
Priority to CN202310370151.5A priority Critical patent/CN116126451A/en
Publication of CN116126451A publication Critical patent/CN116126451A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/451Execution arrangements for user interfaces
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00Arrangements for software engineering
    • G06F8/20Software design
    • G06F8/24Object-oriented
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00Machine learning
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Human Computer Interaction (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The application provides a federal learning workflow construction method based on a blockchain network and related equipment; the method comprises the following steps: enabling an initiator of the training task to deploy an operation environment of the blockchain and to join the blockchain as a computing node; enabling an initiator to create a training task, wherein the training task comprises a task for training a preset federal learning model; enabling the initiator to select federate members added to the blockchain and transmitting training tasks to each federate member selected; enabling each federal member to receive training tasks to the local; verifying the training data of the local training tasks by the initiator and each federal member, and adding the training data to the training of the federal learning model by the verified federal member; and enabling the sponsor and all the federate members added into the training to execute the local training tasks, and downloading the trained federate learning model to the local of the sponsor after the local training tasks are completed.

Description

Federal learning workflow construction method based on blockchain network and related equipment
Technical Field
The embodiment of the application relates to the technical field of federation learning, in particular to a federation learning workflow construction method based on a blockchain network and related equipment.
Background
In the related federal learning training mode, a single server is often adopted to aggregate and distribute data, but the centralized aggregate architecture is easy to generate obvious single-point faults, so that the fault tolerance of the whole system is reduced.
When the blockchain network is applied to the training of federation learning, the logic of the workflow is not clear enough due to the complex training process, so that the displayed operation interface is difficult to observe the workflow with clear logic when training operation is performed, and when federation learning is performed in the blockchain, the operation interface capable of clearly displaying the workflow is not available at present, so that the training operation is very inconvenient to implement.
Disclosure of Invention
In view of this, an object of the present application is to provide a federal learning workflow construction method and related devices based on a blockchain network.
Based on the above objects, the present application provides a federal learning workflow construction method based on a blockchain network, including:
enabling an initiator of a training task to deploy an operation environment of a blockchain and to join the blockchain as a computing node;
enabling the initiator to create a training task, wherein the training task comprises a task for training a preset federal learning model;
causing the initiator to select federate members to join the blockchain and send the training task to each federate member selected;
causing each federate member to receive the training task locally;
verifying the training data of the local training tasks by the initiator and each federation member, and adding the training data to the training of the federation learning model by the verified federation member;
and enabling the sponsor and all the sponsor to be added into the trained federate members to execute the local training tasks, and downloading the trained federate learning model to the local of the sponsor after the local training tasks are completed.
Further, adding the blockchain includes:
enabling the initiator to send task information into the blockchain, wherein the task information comprises relevant information in the training task;
and letting the initiator accept the verification of the authorization permission of the block chain to the initiator;
in response to verification of the initiator by the authorization permission, the initiator is taken as a computing node in the blockchain and added to the blockchain.
Further, after the initiator creates the training task, the method further includes:
a preprocessing program for causing the initiator to set data;
the preprocessing program comprises a feature sampling program for feature sampling the data, a feature cleaning program for feature cleaning the data, a missing value filling program for filling missing values in the data, an abnormal value processing program for processing abnormal values in the data and a feature normalizing program for normalizing the features of the data.
Further, causing the initiator to select a federation member to join the blockchain, comprising:
verifying authorized permissions of federal members who accept the training tasks;
in response to verification of any federal member passing the grant, the federal member is selected as a compute node in the blockchain and added to the blockchain.
Further, after the initiator and each federal member verify the training data of the respective local training task, the method further includes:
enabling the initiator and each federation member to analyze the data characteristics of the respective training data and define respective characteristic projects according to the respective data characteristics;
the sponsor and each federation member are caused to define a model structure for a respective federal learning model and a training strategy for a respective training task.
Further, performing the respective local training tasks, comprising:
during each training round, electing an aggregation node among the sponsor and each federal member;
the aggregation node aggregates the local training parameters of other computing nodes in the block chain to obtain the common training parameters of the round, and the other computing nodes complete the training of the round by using the common training parameters of the round;
and sending the local training parameters of the own round of training to the aggregation node by each other calculation node to aggregate to obtain the shared common training parameters in the next round of training, and carrying out the next round of training until the federal learning model training is completed.
Further, the aggregating node aggregates the local training parameters of other computing nodes in the blockchain to obtain a common training parameter of the present round, and the other computing nodes complete the training of the present round by using the common training parameters of the present round, including:
enabling the aggregation node to aggregate local training parameters of other computing nodes in the block chain respectively in the previous round of training to obtain common training parameters of the previous round of training;
enabling the aggregation node to share the round of common training parameters of the round of training to the other computing nodes according to a preset intelligent contract;
and enabling the other computing nodes to perform the respective local training tasks of the round by using the common training parameters of the round training, and obtaining the local training parameters of the round training.
Based on the same inventive concept, the application also provides a federal learning workflow construction device based on a blockchain network, comprising: the system comprises a registration module, a task creation module, a node selection module, a task sharing module, a verification module and a calculation module;
the registration module is configured to enable an initiator of a training task to deploy an operation environment of a blockchain and to join the blockchain as a computing node;
the task creation module is configured to enable the initiator to create training tasks, wherein the training tasks comprise tasks for training a preset federal learning model;
the node selection module is configured to enable the initiator to select federate members added to the blockchain and send the training task to each federate member selected;
the task sharing module is configured to enable each federate member to receive the training task to be local;
the verification module is configured to enable the initiator and each federal member to verify the training data of the local training tasks, and add the training data to the training of the federal learning model through the verified federal members;
the calculation module is configured to enable the initiator and all federate members added into training to execute the local training tasks respectively, and download the trained federate learning model to the local of the initiator after the local training tasks are completed.
Based on the same inventive concept, the application also provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the federal learning workflow construction method based on the blockchain network when executing the program.
Based on the same inventive concept, the present application also provides a non-transitory computer readable storage medium, wherein the non-transitory computer readable storage medium stores computer instructions for causing the computer to perform the federal learning workflow construction method based on the blockchain network as described above.
As can be seen from the above description, the federation learning workflow construction method and related device based on the blockchain network provided in the present application, based on the deployed blockchain operation environment, perform the training of the federation learning model in the blockchain, and in the workflow, divide the creation of the training task, the selection of federation members, the verification of the training data, and the execution of the computing task, comprehensively consider the different roles of different operations in the workflow, perform the design of the workflow, and design the operation interface according to the logic of the workflow, thereby implementing the design of the workflow from the deployment operation environment to the end of the training, and the design of the operation interface.
Drawings
In order to more clearly illustrate the technical solutions of the present application or related art, the drawings that are required to be used in the description of the embodiments or related art will be briefly described below, and it is apparent that the drawings in the following description are only embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort to those of ordinary skill in the art.
FIG. 1 is a flow chart of a federal learning workflow construction method based on a blockchain network in an embodiment of the present application;
FIG. 2 is an interface schematic diagram of a workflow for training a federal learning model according to an embodiment of the present application;
FIG. 3 is a schematic structural diagram of a federal learning workflow construction device based on a blockchain network according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
For the purposes of making the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail below with reference to the accompanying drawings.
It should be noted that unless otherwise defined, technical or scientific terms used in the embodiments of the present application should be given a general meaning as understood by one of ordinary skill in the art to which the present application belongs. The terms "first," "second," and the like, as used in the embodiments of the present application, do not denote any order, quantity, or importance, but rather are used to distinguish one element from another. The word "comprising" or "comprises", and the like, means that elements or items preceding the word are included in the element or item listed after the word and equivalents thereof, but does not exclude other elements or items. The terms "connected" or "connected," and the like, are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "upper", "lower", "left", "right", etc. are used merely to indicate relative positional relationships, which may also be changed when the absolute position of the object to be described is changed.
As described in the background section, it is also difficult for related blockchain-based federal learning training methods to meet the needs of actual training tasks.
The applicant found in the course of implementing the present application that the main problems of the relevant federal learning training method are: in the related federal learning training mode, a single server is often adopted to aggregate and distribute data, but the centralized aggregate architecture is easy to generate obvious single-point faults, so that the fault tolerance of the whole system is reduced.
In addition, in the federal learning system, malicious behaviors of malicious or semi-honest nodes or attack resistance bring great potential safety hazards to the whole system, and particularly, in medical or financial scenes, how to identify, punish and reject malicious participants and prevent attacks from the malicious nodes is a problem faced by federal learning.
Therefore, the characteristics of decentralization of the blockchain and the common algorithm of the blockchain are utilized, and the blockchain is applied to federation learning to solve the problem, but when the blockchain is applied to federation learning, the training process is complex, so that operation logic, flow and the like are difficult to clearly establish, and therefore, a workflow is introduced at the moment to solve the problem, namely, business rules among various operation steps are abstracted and summarized into logic to model and provide an operation interface of the workflow, and the established business is realized to automatically transfer information among a plurality of participants according to certain preset rules. Based on this, one or more embodiments in the present application provide a federal learning workflow construction method based on a blockchain network.
Embodiments of the present application are described in detail below with reference to the accompanying drawings.
Referring to fig. 1, a federal learning workflow construction method based on a blockchain network according to an embodiment of the present application includes the following steps:
step S101, enabling an initiator of a training task to deploy a running environment of a blockchain and to join the blockchain as a computing node.
In an embodiment of the present application, for a node that holds a federal learning model, the node may be considered an initiator when the node initiates a training task for the federal learning model.
Further, when the initiator of the training task needs to use the blockchain to organize other nodes to train the federation learning model together, the initiator can deploy the running environment of the blockchain first and join the blockchain after the deployment of the running environment of the blockchain is completed.
In a specific example, the initiator may install a blockchain operating environment, such as a Docker, where Docker is an application container engine.
Further, the framework of the blockchain is configured by a Docker.
Further, fig. 2 shows an operation interface of a workflow when training the liberty learning module, wherein a specific example is horizontal federal learning, which includes an initiator and two federal members, and after the initiator deploys the running environment of the blockchain, a computing mode may be selected on a top page of the workflow operation interface, and training tasks may be set in a task list.
Specifically, when the computing mode is selected, the initiator may register task information of the training task into the blockchain through the intelligent contract, where the task information may specifically include: basic information of the training task, description of training data, requirements for other nodes participating in the training task, etc.
Based on this, as shown in fig. 2, the initiator can create a task, i.e., perform the creation of a training task.
Specifically, the blockchain is caused to perform verification of the authorization permission for the initiator, and the initiator is caused to accept the verification of the authorization permission.
Further, after the initiator passes the verification of the authorized license, the initiator can be considered as a node without security risk and added to the blockchain network.
Step S102, enabling the initiator to create training tasks, wherein the training tasks comprise tasks for training a preset federal learning model.
In the embodiment of the present application, the initiator may create the training task set in the task list in the blockchain based on the running environment of the blockchain deployed in step S101 and the task information registered to the blockchain.
Further, as shown in fig. 2, after the initiator creates the training task, when executing the training task of the federal learning model, the first step may create a computing task in an interface for creating the federal task, set a preprocessing program, and the like.
In this particular example, as shown in fig. 2, creating the calculation task may include, for example, filling in Metainfo (initial information) or the like required for calculation in the training task.
Based on this, a data pre-handler may be set up as described in the create federation task interface of FIG. 2.
Specifically, the initiator may upload a program for data preprocessing into the blockchain network and define the program as a data preprocessing program.
The data preprocessing program specifically may include: a feature sampling program for feature sampling data, a feature cleaning program for feature cleaning data, a missing value filling program for filling missing values in data, an abnormal value processing program for processing abnormal values in data, and a feature normalizing program for normalizing the features of data.
Step S103, enabling the initiator to select federation members added to the blockchain, and sending the training task to each selected federation member.
In an embodiment of the present application, as shown in FIG. 2, after creating a computing task, an initiator may select federation members that are co-participating in the computing task and add the selected federation members to the blockchain.
Specifically, verification of authorized permissions among other federal members that may receive training tasks is performed.
Further, when any federal member passes the validation of the authorized license, the federal member can be considered as a node without security risk and added to the blockchain network.
In this embodiment, each federal member joining the blockchain network is used as a computing node in the blockchain network to perform a training task on the bang learning module together with the initiator, for example, to perform a computing task therein together.
Step S104, enabling each federate member to receive the training task to the local.
In this embodiment, for each federate member selected, the federate member is enabled to receive the training task sent by the initiator, and store the received training task to the local area of the federate member.
Specifically, the initiator includes task information and the like related to the training task in the training task transmitted to each federal member.
Step 105, the sponsor and each federal member verify the training data of the local training tasks, and the federal member passing the verification of the training data is added to the training of the federal learning model.
In an embodiment of the present application, based on the step S104 performed, the respective training data of the initiator and each federal member may be validated.
In a specific example, as shown in fig. 2, a second step of the training task may be performed: and verifying the node data.
Specifically, fig. 2 shows an operation interface of a workflow when two federation members respectively train a federation learning module, taking federation member 1 as an example, after an initiator starts verifying node data, federation member 1 will open a task list from a front page of the workflow operation interface, and set the training task in the task list.
Further, federal member 1 can view the training tasks that the initiator invites from its task list.
Further, for the training task, the federal member 1 is preset with training data for local training thereof, and it can be seen that the locally preset training data of each of the initiator, federal member 1, and federal member 2 may be different.
Further, federal member 1 will verify each data set in its local training data.
In particular, the training data local thereto may be verified with a data verification code previously acquired from the initiator.
Further, as shown in fig. 2, when verification fails, it is necessary to check whether there is an error in the directory, format, or the like of the data in each data set, and execute the data verification code again to verify the training data.
Further, when the training data passes the verification, the federal member 1 can be added into the training of the federal learning model, and the computing tasks in the training tasks are jointly executed.
Wherein each federal member needs to verify the respective local training data and perform the same procedure as described above.
Further, the initiator also needs to verify its local training data and perform the same procedure as described above.
And step S106, enabling the sponsor and all the federate members added into the training to execute the local training tasks, and downloading the trained federate learning model to the local of the sponsor after the local training tasks are completed.
In the embodiment of the application, based on the determined all federal members and the initiator added to the training, the computing task in the training task can be executed on the federation learning model, and the federation learning model after training is saved to the local of the initiator.
In this embodiment, after determining all federate members to join the training task, each federate member and initiator to join the training task may be considered as a computing node in the blockchain network.
Further, each computing node, including the initiator and federation members, is caused to build a federal learning model locally at the respective node.
Specifically, analyzing data characteristics in respective training data, and defining respective characteristic projects according to the analyzed data characteristics; wherein, the feature engineering represents the operation of extracting features from the training data to the maximum extent.
Further, each computing node is enabled to define each item of details of the local federal learning model, for example, a model structure is selected, and a corresponding training strategy is defined for the local training task.
Based on this, each computing node may be caused to initialize the operating environment required by the respective local federal learning model.
Further, based on the initialized local federal learning model, each computing node may perform a training task under the control of a preset smart contract, that is, perform the third step of performing the training task shown in fig. 2: a calculation is performed.
Specifically, in performing a training task, multiple rounds of training may be performed, with each round of training dynamically electing an aggregate node from all computing nodes including the initiator and all federal members.
The aggregation node is used as a master node and dynamically sends a message to other computing nodes to prove that the master node still exists in the blockchain network, and if one of the computing nodes does not receive the message due to timeout, the other computing node is dynamically elected to replace the current aggregation node.
Further, the selected aggregation node performs training of the round under a preset intelligent contract, and shares the updated public training parameters in each round with each other computing node.
Specifically, other computing nodes except the aggregation node send respective local training parameters obtained in the previous round of training to the aggregation node, and the aggregation node aggregates after receiving the local training parameters to obtain common training parameters of the round of training.
Further, the aggregation node shares common training parameters of the round to other computing nodes.
Further, after receiving the common training parameters of the round of training, each other computing node performs a local training task on each federal learning model locally at each computing node by using the common training parameters, and obtains the local training parameters of the round of training.
Further, each other computing node transmits the respective local training parameters to the aggregation node so as to enable the aggregation node to aggregate, thereby obtaining the common training parameters required by the next round of training.
Further, when the common training parameters enable the federal learning model to reach a preset convergence threshold, the training task is considered to be completed, and data transmission among the computing nodes is stopped.
Further, when the training task is completed, the sponsor can download the trained federal learning model to its local and apply it to other new services.
In this embodiment, in order to ensure the completeness of the blockchain network, when data is communicated between the computing nodes, a cryptographic key system such as RSA (asymmetric encryption algorithm) may be used for communication.
Therefore, according to the federation learning workflow construction method based on the blockchain network, training of the federation learning model is performed in the blockchain based on the running environment of the deployed blockchain, in the workflow, the training task is created, the federation members are selected, the training data are verified, and the execution of the computing task is divided, different actions of different operations in the workflow are comprehensively considered, the workflow is designed, and the operation interface is designed according to the logic of the workflow, so that the workflow design from the deployment running environment to the end of training is integrated, and the design of the operation interface is realized.
It should be noted that, the method of the embodiments of the present application may be performed by a single device, such as a computer or a server. The method of the embodiment can also be applied to a distributed scene, and is completed by mutually matching a plurality of devices. In the case of such a distributed scenario, one of the devices may perform only one or more steps of the methods of embodiments of the present application, which interact with each other to complete the methods.
It should be noted that some embodiments of the present application are described above. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments described above and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
Based on the same inventive concept, the embodiment of the application also provides a federal learning workflow construction device based on a blockchain network, which corresponds to the method of any embodiment.
Referring to fig. 3, the federal learning workflow construction apparatus based on a blockchain network includes: a registration module 301, a task creation module 302, a node selection module 303, a task sharing module 304, a verification module 305, and a calculation module 306;
wherein the registration module 301 is configured to enable an initiator of a training task to deploy an operating environment of a blockchain and join the blockchain as a computing node;
the task creation module 302 is configured to enable the initiator to create a training task, where the training task includes a task of training a preset federal learning model;
the node selection module 303 is configured to enable the initiator to select federation members joining the blockchain and send the training task to each federation member selected;
the task sharing module 304 is configured to enable each federate member to receive the training task locally;
the verification module 305 is configured to enable the initiator and each federal member to verify the training data of the local training task, and add the training data to the training of the federal learning model through the verified federal member;
the computing module 306 is configured to enable the initiator and all federation members that join in training to execute the local training tasks, and download the trained federation learning model to the local of the initiator after the local training tasks are completed.
For convenience of description, the above devices are described as being functionally divided into various modules, respectively. Of course, the functions of each module may be implemented in the same piece or pieces of software and/or hardware when implementing the embodiments of the present application.
The device of the foregoing embodiment is configured to implement the federal learning workflow construction method based on the blockchain network in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which are not described herein.
Based on the same inventive concept, corresponding to the method of any embodiment, the embodiment of the application further provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the program to realize the federal learning workflow construction method based on the blockchain network according to any embodiment.
Fig. 4 shows a more specific hardware architecture of an electronic device according to this embodiment, where the device may include: a processor 1010, a memory 1020, an input/output interface 1030, a communication interface 1040, and a bus 1050. Wherein processor 1010, memory 1020, input/output interface 1030, and communication interface 1040 implement communication connections therebetween within the device via a bus 1050.
The processor 1010 may be implemented by a general-purpose CPU (Central Processing Unit ), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, etc. for executing relevant programs to implement the technical solutions provided in the embodiments of the present application.
The Memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory ), static storage device, dynamic storage device, or the like. Memory 1020 may store an operating system and other application programs, and when the solutions provided by the embodiments of the present application are implemented in software or firmware, the relevant program code is stored in memory 1020 and invoked for execution by processor 1010.
The input/output interface 1030 is used to connect with an input/output module for inputting and outputting information. The input/output module may be configured as a component in a device (not shown in the figure) or may be external to the device to provide corresponding functionality. Wherein the input devices may include a keyboard, mouse, touch screen, microphone, various types of sensors, etc., and the output devices may include a display, speaker, vibrator, indicator lights, etc.
Communication interface 1040 is used to connect communication modules (not shown) to enable communication interactions of the present device with other devices. The communication module may implement communication through a wired manner (such as USB, network cable, etc.), or may implement communication through a wireless manner (such as mobile network, WIFI, bluetooth, etc.).
Bus 1050 includes a path for transferring information between components of the device (e.g., processor 1010, memory 1020, input/output interface 1030, and communication interface 1040).
It should be noted that although the above-described device only shows processor 1010, memory 1020, input/output interface 1030, communication interface 1040, and bus 1050, in an implementation, the device may include other components necessary to achieve proper operation. Furthermore, it will be understood by those skilled in the art that the above-described apparatus may include only the components necessary to implement the embodiments of the present application, and not all the components shown in the drawings.
The device of the foregoing embodiment is configured to implement the federal learning workflow construction method based on the blockchain network in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which are not described herein.
Based on the same inventive concept, corresponding to any of the above embodiments of the method, the present application further provides a non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the blockchain network-based federal learning workflow construction method according to any of the above embodiments.
The computer readable media of the present embodiments, including both permanent and non-permanent, removable and non-removable media, may be used to implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device.
The storage medium of the foregoing embodiments stores computer instructions for causing the computer to execute the federal learning workflow construction method based on the blockchain network according to any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which are not described herein.
Those of ordinary skill in the art will appreciate that: the discussion of any of the embodiments above is merely exemplary and is not intended to suggest that the scope of the application (including the claims) is limited to these examples; the technical features of the above embodiments or in the different embodiments may also be combined under the idea of the present application, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in details for the sake of brevity.
Additionally, well-known power/ground connections to Integrated Circuit (IC) chips and other components may or may not be shown within the provided figures, in order to simplify the illustration and discussion, and so as not to obscure the embodiments of the present application. Furthermore, the devices may be shown in block diagram form in order to avoid obscuring the embodiments of the present application, and this also takes into account the fact that specifics with respect to implementation of such block diagram devices are highly dependent upon the platform on which the embodiments of the present application are to be implemented (i.e., such specifics should be well within purview of one skilled in the art). Where specific details (e.g., circuits) are set forth in order to describe example embodiments of the application, it should be apparent to one skilled in the art that embodiments of the application can be practiced without, or with variation of, these specific details. Accordingly, the description is to be regarded as illustrative in nature and not as restrictive.
While the present application has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of those embodiments will be apparent to those skilled in the art in light of the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.
The embodiments of the present application are intended to embrace all such alternatives, modifications and variances which fall within the broad scope of the appended claims. Any omissions, modifications, equivalents, improvements, and the like, which are within the spirit and principles of the embodiments of the present application, are therefore intended to be included within the scope of the present application.

Claims (10)

1. The federal learning workflow construction method based on the blockchain network is characterized by comprising the following steps of:
enabling an initiator of a training task to deploy an operation environment of a blockchain and to join the blockchain as a computing node;
enabling the initiator to create a training task, wherein the training task comprises a task for training a preset federal learning model;
causing the initiator to select federate members to join the blockchain and send the training task to each federate member selected;
causing each federate member to receive the training task locally;
verifying the training data of the local training tasks by the initiator and each federation member, and adding the training data to the training of the federation learning model by the verified federation member;
and enabling the sponsor and all the sponsor to be added into the trained federate members to execute the local training tasks, and downloading the trained federate learning model to the local of the sponsor after the local training tasks are completed.
2. The method of claim 1, wherein after the joining the blockchain, further comprising:
enabling the initiator to send task information into the blockchain, wherein the task information comprises relevant information in the training task;
and letting the initiator accept the verification of the authorization permission of the block chain to the initiator;
in response to verification of the initiator by the authorization permission, the initiator is taken as a computing node in the blockchain and added to the blockchain.
3. The method of claim 1, wherein the causing the initiator to create a training task comprises:
a preprocessing program for causing the initiator to set data;
the preprocessing program comprises a feature sampling program for feature sampling the data, a feature cleaning program for feature cleaning the data, a missing value filling program for filling missing values in the data, an abnormal value processing program for processing abnormal values in the data and a feature normalizing program for normalizing the features of the data.
4. The method of claim 1, wherein the causing the initiator to select a federal member to join the blockchain comprises:
verifying authorized permissions of federal members who accept the training tasks;
in response to verification of any federal member passing the grant, the federal member is selected as a compute node in the blockchain and added to the blockchain.
5. The method of claim 1, wherein said causing said initiator and said each federal member to validate training data for a respective local training task further comprises:
enabling the initiator and each federation member to analyze the data characteristics of the respective training data and define respective characteristic projects according to the respective data characteristics;
the sponsor and each federation member are caused to define a model structure for a respective federal learning model and a training strategy for a respective training task.
6. The method of claim 1, wherein the performing the respective local training tasks comprises:
during each training round, electing an aggregation node among the sponsor and each federal member;
the aggregation node aggregates the local training parameters of other computing nodes in the block chain to obtain the common training parameters of the round, and the other computing nodes complete the training of the round by using the common training parameters of the round;
and sending the local training parameters of the own round of training to the aggregation node by each other calculation node to aggregate to obtain the shared common training parameters in the next round of training, and carrying out the next round of training until the federal learning model training is completed.
7. The method of claim 6, wherein the causing the aggregation node to aggregate the local training parameters of other computing nodes in the blockchain to obtain the common training parameters of the present round, and causing the other computing nodes to complete the training of the present round using the common training parameters of the present round, comprises:
enabling the aggregation node to aggregate local training parameters of other computing nodes in the block chain respectively in the previous round of training to obtain common training parameters of the previous round of training;
enabling the aggregation node to share the round of common training parameters of the round of training to the other computing nodes according to a preset intelligent contract;
and enabling the other computing nodes to perform the respective local training tasks of the round by using the common training parameters of the round training, and obtaining the local training parameters of the round training.
8. The utility model provides a federal learning workflow construction device based on blockchain network which characterized in that includes: the system comprises a registration module, a task creation module, a node selection module, a task sharing module, a verification module and a calculation module;
the registration module is configured to enable an initiator of a training task to deploy an operation environment of a blockchain and to join the blockchain as a computing node;
the task creation module is configured to enable the initiator to create training tasks, wherein the training tasks comprise tasks for training a preset federal learning model;
the node selection module is configured to enable the initiator to select federate members added to the blockchain and send the training task to each federate member selected;
the task sharing module is configured to enable each federate member to receive the training task to be local;
the verification module is configured to enable the initiator and each federal member to verify the training data of the local training tasks, and add the training data to the training of the federal learning model through the verified federal members;
the calculation module is configured to enable the initiator and all federate members added into training to execute the local training tasks respectively, and download the trained federate learning model to the local of the initiator after the local training tasks are completed.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable by the processor, wherein the processor implements the method of any one of claims 1 to 7 when executing the computer program.
10. A non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the method of any one of claims 1 to 7.
CN202310370151.5A 2023-04-10 2023-04-10 Federal learning workflow construction method based on blockchain network and related equipment Pending CN116126451A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310370151.5A CN116126451A (en) 2023-04-10 2023-04-10 Federal learning workflow construction method based on blockchain network and related equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310370151.5A CN116126451A (en) 2023-04-10 2023-04-10 Federal learning workflow construction method based on blockchain network and related equipment

Publications (1)

Publication Number Publication Date
CN116126451A true CN116126451A (en) 2023-05-16

Family

ID=86301275

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310370151.5A Pending CN116126451A (en) 2023-04-10 2023-04-10 Federal learning workflow construction method based on blockchain network and related equipment

Country Status (1)

Country Link
CN (1) CN116126451A (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200272945A1 (en) * 2019-02-21 2020-08-27 Hewlett Packard Enterprise Development Lp System and method of decentralized model building for machine learning and data privacy preserving using blockchain
CN113988318A (en) * 2021-10-21 2022-01-28 北京天融信网络安全技术有限公司 Federal learning method, apparatus, electronic device, and medium
CN114443754A (en) * 2020-11-03 2022-05-06 中国电信股份有限公司 Block chain-based federated learning processing method, device, system and medium

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200272945A1 (en) * 2019-02-21 2020-08-27 Hewlett Packard Enterprise Development Lp System and method of decentralized model building for machine learning and data privacy preserving using blockchain
CN114443754A (en) * 2020-11-03 2022-05-06 中国电信股份有限公司 Block chain-based federated learning processing method, device, system and medium
CN113988318A (en) * 2021-10-21 2022-01-28 北京天融信网络安全技术有限公司 Federal learning method, apparatus, electronic device, and medium

Similar Documents

Publication Publication Date Title
US11669832B2 (en) Blockchain-implemented method and system for access control on remote internet-enabled resources
CN107592292B (en) A kind of block chain communication method between nodes and device
CN107770154B (en) Block chain reliable data storage method, terminal and system based on cloud storage
CN110601896B (en) Data processing method and equipment based on block chain nodes
CN111985007A (en) Contract signing and executing method and device based on block chain
Vermesan Advancing IoT platforms interoperability
CN103577180A (en) Data processing method and data processing device
CN104462934B (en) A kind of information processing method and electronic equipment
CN109272378A (en) A kind of discovery method and apparatus of risk group
CN111444090A (en) Contract testing method and device in block chain, electronic equipment and storage medium
Xing et al. Talaria: A framework for simulation of permissioned blockchains for logistics and beyond
CN116126451A (en) Federal learning workflow construction method based on blockchain network and related equipment
CN116684120A (en) Block chain-based data credibility verification method, device, equipment and storage medium
CN112613063B (en) Data verification system construction method and device
CN108713199A (en) Right management method, system, mobile terminal, shared charging equipment and server
CN116521509A (en) Intelligent contract testing method, device, equipment, storage medium and product
CN112311767A (en) Car networking vulnerability analysis method, device, medium and equipment based on correlation analysis
CN110544125A (en) Block chain based cut price data evidence storing method and device
CN107102946A (en) A kind of detection method of program, device and electronic equipment
CN109428723A (en) Verification method, subscriber card and verifying system
CN116502272B (en) Method and device for saving CAD metafile of computer aided design
CN114398082B (en) Compatible operation method and device for frame type block chain application
CN116032494B (en) Data interaction method, blockchain predictor, device and medium
CN109219035A (en) The configuration platform and data altering system, method, apparatus of embedded chip card
CN116151968A (en) Data processing method, device, equipment and readable storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20230516