WO2020082611A1 - Method for carrying out deep learning on basis of blockchain platform and electronic device - Google Patents

Method for carrying out deep learning on basis of blockchain platform and electronic device Download PDF

Info

Publication number
WO2020082611A1
WO2020082611A1 PCT/CN2019/070289 CN2019070289W WO2020082611A1 WO 2020082611 A1 WO2020082611 A1 WO 2020082611A1 CN 2019070289 W CN2019070289 W CN 2019070289W WO 2020082611 A1 WO2020082611 A1 WO 2020082611A1
Authority
WO
WIPO (PCT)
Prior art keywords
deep learning
task
learning task
blockchain platform
computing node
Prior art date
Application number
PCT/CN2019/070289
Other languages
French (fr)
Chinese (zh)
Inventor
金龙
Original Assignee
平安科技(深圳)有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 平安科技(深圳)有限公司 filed Critical 平安科技(深圳)有限公司
Publication of WO2020082611A1 publication Critical patent/WO2020082611A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00Administration; Management
    • G06Q10/06Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G06Q10/063Operations research, analysis or management
    • G06Q10/0631Resource planning, allocation, distributing or scheduling for enterprises or organisations
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network

Definitions

  • the present application relates to the field of blockchain technology, in particular to a method for deep learning based on a blockchain platform, an electronic device, and a computer-readable storage medium.
  • Deep learning requires the training of models through neural network calculations.
  • the data model training process consumes a lot of computing resources. If artificial intelligence products want to achieve better product indicators, in addition to algorithms, massive amounts of data are required for training.
  • the more data, in the case of equal computing resources means longer training, for example: more than a week or even a month to several months.
  • the purpose of the present application is to provide a method for deep learning based on a blockchain platform, an electronic device and a computer-readable storage medium, so as to solve the existing technical problem of excessively high cost of deep learning.
  • a method for deep learning based on a blockchain platform including the following steps:
  • distributing the deep learning task to the computing nodes in the blockchain platform includes:
  • the release information carries the price corresponding to the unit task volume
  • the computing node corresponding to the claim request is assigned a deep learning task of the requested task amount.
  • publishing the deep learning task on the blockchain platform includes:
  • the deep learning task is issued to a computing node that can apply for the deep learning task.
  • performing deep learning on the deep learning task through the distributed computing node includes:
  • the computing node receives a deep learning task distributed to itself, where the deep learning task distributed to itself carries: a deep learning model and a training set;
  • the computing node inputs the training set as training samples into the deep learning model for training to obtain the trained deep learning model
  • the computing nodes in the blockchain platform include at least one of the following: full-featured nodes running on the FPGA server cluster, idle GPU computing nodes in enterprises, and idle GPU computing nodes in individuals.
  • the method further includes:
  • an electronic device including a memory and a processor for storing a deep learning system based on a blockchain platform that can be executed by the processor.
  • Deep learning systems include:
  • the first receiving module is used to receive the deep learning model selected by the task publisher on the blockchain platform;
  • a second receiving module configured to receive the training set input by the task publisher
  • a generating module configured to generate a deep learning task according to the training set and the deep learning model
  • a distribution module for distributing the deep learning tasks to the computing nodes in the blockchain platform
  • a deep learning module is used to perform deep learning on the deep learning task through the computing node to which it is distributed.
  • the distribution module includes:
  • a release unit configured to release the deep learning task on the blockchain platform, wherein the release information carries a price corresponding to a unit task volume
  • a receiving unit configured to receive a computing node's claim request for the issued deep learning task, where the claim request carries the claimed task amount;
  • the allocation unit is used for allocating the deep learning task of the requested task amount to the computing node corresponding to the claim request.
  • the deep learning module includes:
  • the receiving unit is used for the computing node to receive the deep learning task distributed to itself, wherein the deep learning task distributed to itself carries: deep learning model and training set;
  • the input unit is used by the computing node to input the training set as a training sample into the deep learning model for training to obtain the trained deep learning model;
  • the computing nodes in the blockchain platform include at least one of the following: full-featured nodes running on the FPGA server cluster, idle GPU computing nodes in enterprises, and idle GPU computing nodes in individuals.
  • a computer device including: a memory and a processor, wherein the processor is used to perform the steps of the above method.
  • a non-volatile computer-readable storage medium on which computer instructions are stored, and when the instructions are executed, the steps of the above method are implemented.
  • the positive progress of this application lies in: receiving the deep learning model and input training set selected by the task publisher on the blockchain platform to generate a deep learning task; after the task publisher confirms the release of the deep learning task, the The deep learning task is distributed to the computing nodes in the blockchain platform for deep learning.
  • the training tasks are distributed and processed through the blockchain platform, which solves the existing technical problems of too high deep learning costs and low training efficiency, and achieves the technical effects of reducing training costs and improving training efficiency.
  • FIG. 1 is a flowchart of a deep learning method based on blockchain according to an embodiment of the present application
  • FIG. 2 is a structural block diagram of a deep learning system based on blockchain according to an embodiment of the present application
  • FIG. 3 is a structural block diagram of a deep learning system based on blockchain according to an embodiment of the present application.
  • FIG. 4 is a schematic diagram of an optional hardware architecture of an electronic device according to an embodiment of the present application.
  • the three elements of artificial intelligence are computing power, algorithms, and data.
  • the amount of data is a very important factor that affects the indicators of artificial intelligence products. Companies that make artificial intelligence products need to continue to manually label low-quality data or directly purchase high-quality Data, but a lot of data not only involves user privacy, but also for the data provider also wants the data to be non-reproducible. It only sells the right to use the data and does not sell the ownership. However, if the data receiver does not have access to the data, then It is impossible to train based on data.
  • FIG. 1 is a schematic diagram of an optional process of the deep learning method based on blockchain in this application. As shown in FIG. 1, the method may include the following steps:
  • Step 101 Receive the deep learning model selected by the task publisher on the blockchain platform
  • the deep learning model packaged and released by the user may be received, wherein the deep learning model carries: operating environment, input data format, output Data format; publish the deep learning model to the blockchain platform for the task publisher to choose.
  • Step 102 Receive the training set input by the task publisher
  • Step 103 Generate a deep learning task according to the training set and the deep learning model
  • the task processing information of the deep learning task can be generated and displayed.
  • the task processing information includes at least one of the following: estimated cost, available computing nodes, and unit cost; wherein, by dynamically adjusting the unit cost of all deep learning models, all computing nodes in the blockchain platform perform any The overall rewards for learning tasks are the same.
  • Step 104 Distribute the deep learning task to the computing nodes in the blockchain platform
  • the computing nodes in the above-mentioned blockchain platform may include, but are not limited to, at least one of the following: full-featured nodes running on FPGA server clusters, idle GPU computing nodes in enterprises, and idle GPU computing nodes in individuals.
  • Step 105 Perform deep learning on the deep learning task through the distributed computing node.
  • the execution information and completion status of the deep learning task can be obtained; the execution information and the completion status are recorded in the form of a transaction
  • the processing status of these tasks can be traced and queried later.
  • distributing the deep learning task to the computing nodes in the blockchain platform may include:
  • the deep learning task is released on the blockchain platform, wherein the release information carries the price corresponding to the unit task volume;
  • S2 Receive a claim request for a deep learning task issued by a computing node, where the claim request carries the amount of claimed tasks;
  • the task is distributed to the claim platform, and the computing node makes the claim independently according to its own situation.
  • the deep learning task can be divided into multiple unit task volumes; the price of the unit task volume is calculated; and the deep learning task can be claimed according to the computing environment required by the deep learning task
  • the computing node of publishes the deep learning task to the computing node that can claim the deep learning task.
  • performing deep learning on the deep learning task through the computing node distributed to may include:
  • the computing node receives the deep learning task distributed to itself, where the deep learning task distributed to itself carries: a deep learning model and a training set;
  • the computing node inputs the training set as a training sample into the deep learning model for training to obtain the trained deep learning model.
  • the deep learning computing nodes on the chain can be composed of many forms, for example, it can include: large Full-featured nodes (permanent nodes) running on GPU or FPGA server clusters, idle GPU server computing nodes for small and medium-sized enterprises, and personal idle GPU computing nodes.
  • the deep learning blockchain conducts transactions based on smart contracts, and incentivizes mining nodes according to the reward system designed by smart contracts. While ensuring the safe and stable operation of the system, it also allows all participants to get rewards from it, so that artificial intelligence manufacturers can obtain neural network deep learning computing power at a low cost.
  • the deep learning blockchain can be designed according to the following principles:
  • the above-mentioned deep learning blockchain platform may include the following modules:
  • the deep learning calculation module is used to complete: distribution of calculation tasks and distribution of container calculations. Specifically, after the computing container is successfully deployed, the verification calculation is performed. After the verification calculation is passed, the calculation task is distributed across the entire network through the blockchain network.
  • It is used to manage the entire life cycle of container instances for a single user or use a group, provide virtual services according to user needs, and is responsible for container creation, suspension, suspension, adjustment, migration, restart, destruction and other operations.
  • the container computing engine When the computing capacity of a user's concurrent request reaches a specific value of the container allocation (this value can be set by the user), the container computing engine will start an alarm and will automatically deploy containers to other normal nodes for expansion.
  • each DB server will have n coroutines to process the request.
  • a set of virtual container image search and retrieval system with functions of creating image, uploading image, deleting image, editing basic information of image.
  • Image management system mainly consists of two services: image management API and image management register.
  • the mirror management API is the entrance of the mirror management system service, and is responsible for receiving user API requests.
  • the mirror management register processes mirror metadata related requests.
  • the mirror management API receives the user's API request, if it is determined that the request is related to metadata, the request is transferred to the mirror management register service. Then, the mirror management register will parse the content of the user's metadata request, and interact with the database to access and update the metadata of the mirror.
  • deep learning can be performed as follows:
  • Training task release The publisher selects the published model algorithm, and releases the training / test task after packing the data. Before submission, you can display: estimated cost consumption, available nodes, unit cost and other information. By dynamically adjusting the unit cost of all model algorithms, you can achieve the same overall return for all nodes in the entire network to perform any deep calculation task. Among them, the task execution status and task completion status are recorded in the block in the form of transactions.
  • S3 task allocation and collection receive the tasks broadcast to the deep learning blockchain and the running status of other nodes, create a block every certain time and select the tasks to run according to the algorithm and execute them. In addition to the fees paid by the task publisher, the final reward of the node will be calculated in the blockchain after the task is completed, and distributed in proportion to the execution of each node.
  • the deep learning model selected by the task publisher on the blockchain platform and the input training set are received to generate a deep learning task; after the task publisher confirms the release of the deep learning task, the depth The learning task is distributed to the computing nodes in the blockchain platform for deep learning.
  • the training tasks are distributed and processed through the blockchain platform, which solves the existing technical problems of too high deep learning costs and low training efficiency, and achieves the technical effects of reducing training costs and improving training efficiency.
  • FIG. 2 and FIG. 3 show the An optional structural block diagram of a deep learning system for a blockchain platform.
  • the deep learning system based on a blockchain platform is divided into one or more program modules, and one or more program modules are stored in a storage medium. It is executed by one or more processors to complete this application.
  • the program module referred to in this application refers to a series of computer program instruction segments that can perform specific functions. It is more suitable than the program itself to describe the execution process of a deep learning system based on a blockchain platform in a storage medium.
  • the following description will specifically introduce Functions of each program module in this embodiment:
  • a system 20 for deep learning based on a blockchain platform includes:
  • the first receiving module 201 is used to receive the deep learning model selected by the task publisher on the blockchain platform;
  • the second receiving module 202 is configured to receive the training set input by the task publisher
  • a generating module 203 configured to generate a deep learning task according to the training set and the deep learning model
  • the distribution module 204 is used to distribute the deep learning task to the computing nodes in the blockchain platform;
  • the deep learning module 205 is configured to perform deep learning on the deep learning task through the distributed computing node.
  • the computing nodes in the above-mentioned blockchain platform may include, but are not limited to, at least one of the following: a full-featured node running on an FPGA server cluster, an idle GPU computing node in an enterprise, and an individual idle GPU computing node.
  • the above-mentioned system may further include: a third receiving module 301, configured to receive the depth of the packaged release by the user before receiving the deep learning model selected by the task publisher on the blockchain platform Learning model, wherein the deep learning model carries: operating environment, input data format, output data format; publishing module 302, used to publish the deep learning model to the blockchain platform, for the task publisher select.
  • a third receiving module 301 configured to receive the depth of the packaged release by the user before receiving the deep learning model selected by the task publisher on the blockchain platform Learning model, wherein the deep learning model carries: operating environment, input data format, output data format
  • publishing module 302 used to publish the deep learning model to the blockchain platform, for the task publisher select.
  • the above system may also generate task processing information of the deep learning task, where the task processing information includes at least one of the following One: estimated costs, available computing nodes, and unit costs; where, by dynamically adjusting the unit costs of all deep learning models, the overall return of all computing nodes in the blockchain platform to perform any learning task is the same ; Display the task processing information.
  • the above-mentioned system can also obtain execution information and completion status of the deep learning task; Recorded in the blockchain platform in the form of transactions.
  • the above-mentioned distribution module may include: a publishing unit for publishing the deep learning task on the blockchain platform, wherein the publishing information carries a price corresponding to a unit task amount; a receiving unit is used for Receiving a claim request for a deep learning task issued by a computing node, wherein the claim request carries the claimed task amount; an allocation unit is configured to allocate a deep learning task requesting the claimed task amount to the computing node corresponding to the claim request.
  • the publishing unit includes: a division subunit for dividing the deep learning task into a plurality of unit task amounts; a calculation subunit for calculating the price of the unit task amount; determining a subunit, using To determine the computing nodes that can apply for the deep learning task according to the computing environment required by the deep learning task; issue a subunit for publishing the deep learning task to the computing nodes that can apply for the deep learning task .
  • the deep learning module is specifically used for the computing node to receive a deep learning task distributed to itself, where the deep learning task distributed to itself carries: a deep learning model and a training set; the computing node will The training set is input as a training sample into the deep learning model for training to obtain the trained deep learning model.
  • the deep learning model selected by the task publisher on the blockchain platform and the input training set are received to generate a deep learning task; after the task publisher confirms to release the deep learning task, Distribute the deep learning task to the computing nodes in the blockchain platform for deep learning.
  • the training tasks are distributed and processed through the blockchain platform, which solves the existing technical problems of too high deep learning costs and low training efficiency, and achieves the technical effects of reducing training costs and improving training efficiency.
  • the electronic device 2 is a device that can automatically perform numerical calculation and / or information processing according to an instruction set or stored in advance.
  • the electronic device 2 may be a smart phone, tablet computer, notebook computer, desktop computer, rack server, blade server, tower server, or rack server (including an independent server or a server cluster composed of multiple servers).
  • the electronic device 2 includes at least but not limited to: a memory 21, a processor 22, a network interface 23, and a system 20 for deep learning based on a blockchain platform that can communicate with each other via a system bus. among them:
  • the memory 21 includes at least one type of computer-readable storage medium.
  • the readable storage medium includes a flash memory, a hard disk, a multimedia card, a card-type memory (for example, SD or DX memory, etc.), a random access memory (RAM), and a static random access memory.
  • SRAM static random access memory
  • ROM read-only memory
  • EEPROM electrically erasable programmable read-only memory
  • PROM programmable read-only memory
  • magnetic memory magnetic disk, optical disk, etc.
  • the memory 21 may be an internal storage module of the electronic device 2, such as a hard disk or a memory of the electronic device 2.
  • the memory 21 may also be an external storage device of the electronic device 2, such as a plug-in hard disk equipped on the electronic device 2, a smart memory card (Smart Media, Card, SMC), and secure digital (Secure Digital, SD) card, flash card (Flash Card), etc.
  • the memory 21 may also include both the internal storage module of the electronic device 2 and its external storage device.
  • the memory 21 is generally used to store an operating system installed in the electronic device 2 and various application software, such as program codes of a system 20 for deep learning based on a blockchain platform.
  • the memory 21 may also be used to temporarily store various types of data that have been output or will be output.
  • the processor 22 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments.
  • the processor 22 is generally used to control the overall operation of the electronic device 2, such as performing control and processing related to data interaction or communication with the electronic device 2.
  • the processor 22 is used to run the program code stored in the memory 21 or process data, such as a system 20 for deep learning based on a blockchain platform.
  • the network interface 23 may include a wireless network interface or a wired network interface, and the network interface 23 is generally used to establish a communication connection between the electronic device 2 and other electronic devices.
  • the network interface 23 is used to connect the electronic device 2 with an external terminal through a network, and establish a data transmission channel and a communication connection between the electronic device 2 and the external terminal.
  • the network can be Intranet, Internet, Global System of Mobile (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Wireless or wired networks such as Bluetooth and Wi-Fi.
  • FIG. 4 only shows an electronic device having components 21-23, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.
  • the system 20 for deep learning based on the blockchain platform stored in the memory 21 can also be divided into one or more program modules, one or more program modules are stored in the memory 21, and One or more processors (the processor 22 in this embodiment) are executed to complete the application.
  • This embodiment also provides a computer-readable storage medium in which a system for deep learning based on a blockchain platform is stored.
  • the system for monitoring and dialing tasks can be executed by at least one processor, so that at least one The processor executes the steps of the deep learning method based on the blockchain platform as in the first embodiment.
  • the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc.
  • the computer-readable storage medium may be an internal storage unit of the computer device, such as a hard disk or memory of the computer device.
  • the computer-readable storage medium may also be an external storage device of the computer device, for example, a plug-in hard disk equipped on the computer device, a smart memory card (Smart Media (SMC), Secure Digital (Secure Digital) , SD) card, flash card (Flash Card), etc.
  • the computer-readable storage medium may also include both the internal storage unit of the computer device and the external storage device thereof.
  • the computer-readable storage medium is generally used to store the operating system and various application software installed on the computer device, for example, the program code of the customer guarantee analysis system in the second embodiment.
  • the computer-readable storage medium can also be used to temporarily store various types of data that have been output or are to be output.
  • modules or steps in the embodiments of the present application described above can be implemented by a general-purpose computing device, and they can be concentrated on a single computing device or distributed among multiple computing devices.
  • they can be implemented with program code executable by the computing device, so that they can be stored in the storage device and executed by the computing device, and in some cases, can be different from here
  • the steps shown or described are executed in the order of, or they are made into individual integrated circuit modules separately, or multiple modules or steps among them are made into a single integrated circuit module for implementation. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

Landscapes

  • Business, Economics & Management (AREA)
  • Human Resources & Organizations (AREA)
  • Engineering & Computer Science (AREA)
  • Economics (AREA)
  • Strategic Management (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Development Economics (AREA)
  • Quality & Reliability (AREA)
  • Signal Processing (AREA)
  • Game Theory and Decision Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Marketing (AREA)
  • Operations Research (AREA)
  • Educational Administration (AREA)
  • Tourism & Hospitality (AREA)
  • Physics & Mathematics (AREA)
  • General Business, Economics & Management (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

Disclosed in the present application are a method for carrying out deep learning on the basis of a blockchain platform, and an electronic device, the method comprising: receiving a deep learning model selected by a task distributor at a blockchain platform; receiving a training set inputted by the task distributor; generating a deep learning task according to the training set and the deep learning model; distributing the deep learning task to computing nodes in the blockchain platform; and carrying out deep learning for the deep learning task by means of the computing nodes to which the task is distributed. Employing the solution above solves the existing technical problems of the cost of deep learning being too high and the efficiency of training being low, and achieves the technical effects of reducing the cost of training and increasing the efficiency of training.

Description

基于区块链平台进行深度学习的方法、电子装置Deep learning method and electronic device based on blockchain platform
相关申请的交叉引用Cross-reference of related applications
本申请申明享有2018年10月25日递交的申请号为CN 201811252535.2、名称为“基于区块链平台进行深度学习的方法、电子装置”的中国专利申请的优先权,该中国专利申请的整体内容以参考的方式结合在本申请中。This application declares to enjoy the priority of the Chinese patent application submitted on October 25, 2018, with the application number CN 201811252535.2 and titled "Deep learning method and electronic device based on blockchain platform". Incorporated by reference in this application.
技术领域Technical field
本申请涉及区块链技术领域,具体涉及一种基于区块链平台进行深度学习的方法、电子装置及计算机可读存储介质。The present application relates to the field of blockchain technology, in particular to a method for deep learning based on a blockchain platform, an electronic device, and a computer-readable storage medium.
背景技术Background technique
深度学习需要进行通过神经网络计算来训练模型,然而,数据模型训练过程需要消耗大量的计算资源,人工智能产品如果希望达到更好的产品指标,除了算法之外就需要海量的数据来进行训练。然而,越多的数据,在同等计算资源的情况下,意味着更长时间的训练,例如:超过一周甚至一个月到几个月的时间。Deep learning requires the training of models through neural network calculations. However, the data model training process consumes a lot of computing resources. If artificial intelligence products want to achieve better product indicators, in addition to algorithms, massive amounts of data are required for training. However, the more data, in the case of equal computing resources, means longer training, for example: more than a week or even a month to several months.
如果训练过程中有参数不正确,需要反复地进行训练,训练时间太长,对于企业产品迭代更新是极其不利的,很可能在行业竞争中失败,从而导致很多厂商不得不投入大量资金购置GPU、FPGA等硬件资源,成本投入过高。If the parameters in the training process are incorrect, the training needs to be repeated. The training time is too long, which is extremely unfavorable for the iterative update of enterprise products. It is likely to fail in the industry competition, resulting in many manufacturers having to invest a lot of money to purchase GPU The cost of hardware resources such as FPGA is too high.
针对上述问题,目前尚未提出有效的解决方案。In response to the above problems, no effective solution has been proposed yet.
发明内容Summary of the invention
本申请的目的在于提供一种基于区块链平台进行深度学习的方法、电子装置及计算机可读存储介质,进而解决现有的深度学习的成本过高的技术问题。The purpose of the present application is to provide a method for deep learning based on a blockchain platform, an electronic device and a computer-readable storage medium, so as to solve the existing technical problem of excessively high cost of deep learning.
本申请是通过下述技术方案来解决上述技术问题:This application solves the above technical problems through the following technical solutions:
根据本申请的一个方面,提供了一种基于区块链平台进行深度学习的方法,包括如下步骤:According to an aspect of this application, a method for deep learning based on a blockchain platform is provided, including the following steps:
接收任务发布者在区块链平台选定的深度学习模型;Receive the deep learning model selected by the task publisher on the blockchain platform;
接收所述任务发布者输入的训练集;Receiving the training set input by the task publisher;
根据所述训练集和所述深度学习模型,生成深度学习任务;Generate a deep learning task according to the training set and the deep learning model;
将所述深度学习任务分发至所述区块链平台中的计算节点;Distribute the deep learning task to the computing nodes in the blockchain platform;
通过分发至的计算节点,对所述深度学习任务进行深度学习。Perform deep learning on the deep learning task through the distributed computing nodes.
在一个实施方式中,将所述深度学习任务分发至所述区块链平台中的计算节点,包括:In one embodiment, distributing the deep learning task to the computing nodes in the blockchain platform includes:
在所述区块链平台发布所述深度学习任务,其中,发布信息中携带有单位任务量对应 的价格;Publish the deep learning task on the blockchain platform, wherein the release information carries the price corresponding to the unit task volume;
接收计算节点对发布的深度学习任务的认领请求,其中,所述认领请求中携带有认领的任务量;Receiving a claim request for a deep learning task issued by a computing node, where the claim request carries the amount of claimed tasks;
向认领请求对应的计算节点分配请求认领的任务量的深度学习任务。The computing node corresponding to the claim request is assigned a deep learning task of the requested task amount.
在一个实施方式中,将所述深度学习任务在所述区块链平台进行发布,包括:In one embodiment, publishing the deep learning task on the blockchain platform includes:
将所述深度学习任务划分为多个单位任务量;Divide the deep learning task into multiple unit task volumes;
计算单位任务量的价格;Calculate the price of unit task volume;
根据所述深度学习任务所需的计算环境,确定可申领所述深度学习任务的计算节点;Determine the computing nodes that can apply for the deep learning task according to the computing environment required by the deep learning task;
向可申领所述深度学习任务的计算节点发布所述深度学习任务。The deep learning task is issued to a computing node that can apply for the deep learning task.
在一个实施方式中,通过分发至的计算节点,对所述深度学习任务进行深度学习,包括:In one embodiment, performing deep learning on the deep learning task through the distributed computing node includes:
所述计算节点接收分发至自身的深度学习任务,其中,分发至自身的深度学习任务中携带有:深度学习模型和训练集;The computing node receives a deep learning task distributed to itself, where the deep learning task distributed to itself carries: a deep learning model and a training set;
所述计算节点将所述训练集作为训练样本输入所述深度学习模型中进行训练,得到训练后的深度学习模型;The computing node inputs the training set as training samples into the deep learning model for training to obtain the trained deep learning model;
其中,所述区块链平台中的计算节点包括以下至少之一:FPGA服务器集群运行的全功能节点、企业中空闲的GPU计算节点、个人闲置的GPU计算节点。Wherein, the computing nodes in the blockchain platform include at least one of the following: full-featured nodes running on the FPGA server cluster, idle GPU computing nodes in enterprises, and idle GPU computing nodes in individuals.
在一个实施方式中,通过分发至的计算节点,对所述深度学习任务进行深度学习之后,所述方法还包括:In one embodiment, after performing deep learning on the deep learning task through the distributed computing node, the method further includes:
获取所述深度学习任务的执行信息和完成状态;Acquiring execution information and completion status of the deep learning task;
将所述执行信息和所述完成状态,以交易的形式记录在所述区块链平台中。Record the execution information and the completion status in the form of transactions in the blockchain platform.
另一方面,提供了一种电子装置,包括存储器和处理器,所述存储器用于存储可被所述处理器执行的基于区块链平台进行深度学习的系统,所述基于区块链平台进行深度学习的系统包括:On the other hand, there is provided an electronic device including a memory and a processor for storing a deep learning system based on a blockchain platform that can be executed by the processor. Deep learning systems include:
第一接收模块,用于接收任务发布者在区块链平台选定的深度学习模型;The first receiving module is used to receive the deep learning model selected by the task publisher on the blockchain platform;
第二接收模块,用于接收所述任务发布者输入的训练集;A second receiving module, configured to receive the training set input by the task publisher;
生成模块,用于根据所述训练集和所述深度学习模型,生成深度学习任务;A generating module, configured to generate a deep learning task according to the training set and the deep learning model;
分发模块,用于将所述深度学习任务分发至所述区块链平台中的计算节点;A distribution module for distributing the deep learning tasks to the computing nodes in the blockchain platform;
深度学习模块,用于通过分发至的计算节点,对所述深度学习任务进行深度学习。A deep learning module is used to perform deep learning on the deep learning task through the computing node to which it is distributed.
在一个实施方式中,所述分发模块包括:In one embodiment, the distribution module includes:
发布单元,用于在所述区块链平台发布所述深度学习任务,其中,发布信息中携带有单位任务量对应的价格;A release unit, configured to release the deep learning task on the blockchain platform, wherein the release information carries a price corresponding to a unit task volume;
接收单元,用于接收计算节点对发布的深度学习任务的认领请求,其中,所述认领请求中携带有认领的任务量;A receiving unit, configured to receive a computing node's claim request for the issued deep learning task, where the claim request carries the claimed task amount;
分配单元,用于向认领请求对应的计算节点分配请求认领的任务量的深度学习任务。The allocation unit is used for allocating the deep learning task of the requested task amount to the computing node corresponding to the claim request.
在一个实施方式中,所述深度学习模块包括:In one embodiment, the deep learning module includes:
接收单元,用于计算节点接收分发至自身的深度学习任务,其中,分发至自身的深度学习任务中携带有:深度学习模型和训练集;The receiving unit is used for the computing node to receive the deep learning task distributed to itself, wherein the deep learning task distributed to itself carries: deep learning model and training set;
输入单元,用于所述计算节点将所述训练集作为训练样本输入所述深度学习模型中进行训练,得到训练后的深度学习模型;The input unit is used by the computing node to input the training set as a training sample into the deep learning model for training to obtain the trained deep learning model;
其中,所述区块链平台中的计算节点包括以下至少之一:FPGA服务器集群运行的全功能节点、企业中空闲的GPU计算节点、个人闲置的GPU计算节点。Wherein, the computing nodes in the blockchain platform include at least one of the following: full-featured nodes running on the FPGA server cluster, idle GPU computing nodes in enterprises, and idle GPU computing nodes in individuals.
根据本申请的又一个方面,提供了一种计算机设备,包括:存储器和处理器,其中,所述处理器用于执行上述方法的步骤。According to yet another aspect of the present application, a computer device is provided, including: a memory and a processor, wherein the processor is used to perform the steps of the above method.
根据本申请的又一个方面,提供了一种非易失性计算机可读存储介质,其上存储有计算机指令,所述指令被执行时实现上述方法的步骤。According to yet another aspect of the present application, there is provided a non-volatile computer-readable storage medium on which computer instructions are stored, and when the instructions are executed, the steps of the above method are implemented.
本申请的积极进步效果在于:接收任务发布者在区块链平台选定的深度学习模型和输入的训练集,以生成深度学习任务;在任务发布者确认发布所述深度学习任务之后,将所述深度学习任务分发至所述区块链平台中的计算节点进行深度学习。在上述方案中通过区块链平台对训练任务进行分发处理,从而解决了现有的深度学习成本过高、训练效率低下的技术问题,达到了降低训练成本、提升训练效率的技术效果。The positive progress of this application lies in: receiving the deep learning model and input training set selected by the task publisher on the blockchain platform to generate a deep learning task; after the task publisher confirms the release of the deep learning task, the The deep learning task is distributed to the computing nodes in the blockchain platform for deep learning. In the above scheme, the training tasks are distributed and processed through the blockchain platform, which solves the existing technical problems of too high deep learning costs and low training efficiency, and achieves the technical effects of reducing training costs and improving training efficiency.
附图说明BRIEF DESCRIPTION
图1是根据本申请实施例的基于区块链的深度学习的方法流程图;1 is a flowchart of a deep learning method based on blockchain according to an embodiment of the present application;
图2是根据本申请实施例的基于区块链的深度学习的系统的结构框图;2 is a structural block diagram of a deep learning system based on blockchain according to an embodiment of the present application;
图3是根据本申请实施例的基于区块链的深度学习的系统的结构框图;3 is a structural block diagram of a deep learning system based on blockchain according to an embodiment of the present application;
图4是根据本申请实施例的电子装置的一种可选的硬件架构示意图。4 is a schematic diagram of an optional hardware architecture of an electronic device according to an embodiment of the present application.
具体实施方式detailed description
为了使本申请的目的、技术方案及优点更加清楚明白,以下结合附图及实施例,对本申请进行进一步详细说明。应当理解,此处所描述的具体实施例仅用以解释本申请,并不用于限定本申请。基于本申请中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例,都属于本申请保护的范围。In order to make the purpose, technical solutions and advantages of the present application more clear, the following describes the present application in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application. Based on the embodiments in this application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the scope of protection of this application.
考虑到现有的人工产品在上线过后仍然需要通过神经网络计算来进行解码,用户量越大需要的计算量就越大,成本也就越高,且用户在不同时间周期中访问频率也会有所变化,一次性购买大量的计算资源,必然会出现很多时候的闲置浪费。人工智能三要素是算力、 算法、数据,其中,数据量是影响人工智能产品指标的很重要因素,做人工智能产品的公司需要不断通过对低质量的数据进行人工标注处理或者直接购买高质量的数据,但是很多数据不但涉及到用户隐私,同时对于数据提供方来说也希望数据具有不可复制性,仅仅出售数据使用权,不出售拥有权,然而,如果数据接收方接触不到数据,那么就无法基于数据进行训练。Considering that the existing artificial products still need to be decoded by neural network calculation after going online, the greater the number of users, the greater the amount of calculation required, the higher the cost, and the frequency of user access in different time periods. As a result, purchasing large amounts of computing resources at one time will inevitably result in idle waste. The three elements of artificial intelligence are computing power, algorithms, and data. Among them, the amount of data is a very important factor that affects the indicators of artificial intelligence products. Companies that make artificial intelligence products need to continue to manually label low-quality data or directly purchase high-quality Data, but a lot of data not only involves user privacy, but also for the data provider also wants the data to be non-reproducible. It only sells the right to use the data and does not sell the ownership. However, if the data receiver does not have access to the data, then It is impossible to train based on data.
基于此,在本例中提供了基于区块链的深度学习的方法、电子设备,下面对该方法和电子设备进行具体说明如下:Based on this, a deep learning method and electronic device based on blockchain are provided in this example, and the method and the electronic device are specifically described as follows:
实施例1Example 1
下面结合附图对本申请提供的基于区块链的深度学习的方法进行说明。The method for deep learning based on blockchain provided by the present application will be described below with reference to the drawings.
图1为本申请基于区块链的深度学习的方法的一种可选的流程示意图,如图1所示,该方法可以包括以下步骤:FIG. 1 is a schematic diagram of an optional process of the deep learning method based on blockchain in this application. As shown in FIG. 1, the method may include the following steps:
步骤101:接收任务发布者在区块链平台选定的深度学习模型;Step 101: Receive the deep learning model selected by the task publisher on the blockchain platform;
具体的,在接收任务发布者在区块链平台选定的深度学习模型之前,可以接收用户打包发布的深度学习模型,其中,所述深度学习模型中携带有:运行环境、输入数据格式、输出数据格式;将所述深度学习模型发布至区块链平台,用于任务发布者进行选择。Specifically, before receiving the deep learning model selected by the task publisher on the blockchain platform, the deep learning model packaged and released by the user may be received, wherein the deep learning model carries: operating environment, input data format, output Data format; publish the deep learning model to the blockchain platform for the task publisher to choose.
在实际实现的时候,研究者可以在针对某一人工智能应用研发出了深度学习模型之后,开源并打包发布至深度学习区块链,提供运行环境以及输入输出数据格式标准,并可以选择用公开数据集提交训练/测试任务。其中,发布的深度学习模型被他人使用时,发布者可获得相应回报。In actual implementation, after researching and developing a deep learning model for an artificial intelligence application, researchers can open source and package and publish to the deep learning blockchain to provide the operating environment and input and output data format standards, and can choose to use public The data set submits training / test tasks. Among them, when the released deep learning model is used by others, the publisher can get corresponding rewards.
步骤102:接收所述任务发布者输入的训练集;Step 102: Receive the training set input by the task publisher;
步骤103:根据所述训练集和所述深度学习模型,生成深度学习任务;Step 103: Generate a deep learning task according to the training set and the deep learning model;
为了使得任务发布者可以知道训练任务的费用以及处理情况等,在根据所述训练集和所述深度学习模型,生成深度学习任务之后,可以生成并显示深度学习任务的任务处理信息,其中,所述任务处理信息包括以下至少之一:预估的费用、可用的计算节点、单位成本;其中,通过动态调整所有深度学习模型的单位成本,使得所述区块链平台中所有计算节点执行任一学习任务的总体回报是相同的。In order to enable the task publisher to know the cost and processing status of the training task, after generating the deep learning task according to the training set and the deep learning model, the task processing information of the deep learning task can be generated and displayed. The task processing information includes at least one of the following: estimated cost, available computing nodes, and unit cost; wherein, by dynamically adjusting the unit cost of all deep learning models, all computing nodes in the blockchain platform perform any The overall rewards for learning tasks are the same.
步骤104:将所述深度学习任务分发至所述区块链平台中的计算节点;Step 104: Distribute the deep learning task to the computing nodes in the blockchain platform;
其中,上述所述区块链平台中的计算节点可以包括但不限于以下至少之一:FPGA服务器集群运行的全功能节点、企业中空闲的GPU计算节点、个人闲置的GPU计算节点。The computing nodes in the above-mentioned blockchain platform may include, but are not limited to, at least one of the following: full-featured nodes running on FPGA server clusters, idle GPU computing nodes in enterprises, and idle GPU computing nodes in individuals.
步骤105:通过分发至的计算节点,对所述深度学习任务进行深度学习。Step 105: Perform deep learning on the deep learning task through the distributed computing node.
具体的,在通过分发至的计算节点,对所述深度学习任务进行深度学习之后,可以获取深度学习任务的执行信息和完成状态;将所述执行信息和所述完成状态,以交易的形式 记录在所述区块链平台中,因为以交易的形式记录在区块链平台中,因此,后续可以对这些任务的处理情况进行追溯和查询。Specifically, after performing deep learning on the deep learning task through the distributed computing node, the execution information and completion status of the deep learning task can be obtained; the execution information and the completion status are recorded in the form of a transaction In the blockchain platform, because it is recorded in the blockchain platform in the form of transactions, therefore, the processing status of these tasks can be traced and queried later.
在实际实现的过程中,将所述深度学习任务分发至所述区块链平台中的计算节点,可以包括:In the actual implementation process, distributing the deep learning task to the computing nodes in the blockchain platform may include:
S1:在所述区块链平台发布所述深度学习任务,其中,发布信息中携带有单位任务量对应的价格;S1: The deep learning task is released on the blockchain platform, wherein the release information carries the price corresponding to the unit task volume;
S2:接收计算节点对发布的深度学习任务的认领请求,其中,所述认领请求中携带有认领的任务量;S2: Receive a claim request for a deep learning task issued by a computing node, where the claim request carries the amount of claimed tasks;
S3:向认领请求对应的计算节点分配请求认领的任务量的深度学习任务。S3: Assign the deep learning task of the requested task amount to the computing node corresponding to the claim request.
即,将任务分发至申领平台,由计算节点按照自身情况自主进行申领。That is, the task is distributed to the claim platform, and the computing node makes the claim independently according to its own situation.
具体的,在发布的过程中,可以将深度学习任务划分为多个单位任务量;计算单位任务量的价格;根据所述深度学习任务所需的计算环境,确定可申领所述深度学习任务的计算节点;向可申领所述深度学习任务的计算节点发布所述深度学习任务。Specifically, during the release process, the deep learning task can be divided into multiple unit task volumes; the price of the unit task volume is calculated; and the deep learning task can be claimed according to the computing environment required by the deep learning task The computing node of publishes the deep learning task to the computing node that can claim the deep learning task.
对于计算节点而言,通过分发至的计算节点,对所述深度学习任务进行深度学习,可以包括:For the computing node, performing deep learning on the deep learning task through the computing node distributed to may include:
S1:计算节点接收分发至自身的深度学习任务,其中,分发至自身的深度学习任务中携带有:深度学习模型和训练集;S1: The computing node receives the deep learning task distributed to itself, where the deep learning task distributed to itself carries: a deep learning model and a training set;
S2:计算节点将所述训练集作为训练样本输入所述深度学习模型中进行训练,得到训练后的深度学习模型。S2: The computing node inputs the training set as a training sample into the deep learning model for training to obtain the trained deep learning model.
下面结合一个具体实施例对上述方法进行说明,然而,值得注意的是,该具体实施例仅是为了更好地说明本申请,并不构成对本申请的不当限定。The above method will be described below in conjunction with a specific embodiment. However, it is worth noting that this specific embodiment is only for better description of the present application, and does not constitute an undue limitation on the present application.
在本例中,提供了一种基于区块链技术的去中心化的、低成本的、隐私的深度学习计算平台,链上的深度学习计算节点可由多种形态组成,例如,可以包括:大型GPU或者FPGA服务器集群运行的全功能节点(永久节点)、中小型企业闲散的空余GPU服务器计算节点以及个人闲置GPU计算节点。In this example, a decentralized, low-cost, private deep learning computing platform based on blockchain technology is provided. The deep learning computing nodes on the chain can be composed of many forms, for example, it can include: large Full-featured nodes (permanent nodes) running on GPU or FPGA server clusters, idle GPU server computing nodes for small and medium-sized enterprises, and personal idle GPU computing nodes.
具体的,深度学习区块链基于智能合约进行交易,并根据智能合约设计的奖励系统对挖矿节点进行激励。在保证系统安全、稳定运行的同时,也让所有的参与者都能从中获取回报,从而使得人工智能厂商可以在低成本的情况下就能获取到神经网络深度学习计算能力。Specifically, the deep learning blockchain conducts transactions based on smart contracts, and incentivizes mining nodes according to the reward system designed by smart contracts. While ensuring the safe and stable operation of the system, it also allows all participants to get rewards from it, so that artificial intelligence manufacturers can obtain neural network deep learning computing power at a low cost.
具体的,深度学习区块链可以按照如下原则设计:Specifically, the deep learning blockchain can be designed according to the following principles:
1)扩展原则:每一个模块是松耦合的,很容易添加新的模块进来,每个模块自身的更新不需要依赖其他模块接口的变化。1) Extension principle: each module is loosely coupled, it is easy to add new modules, and the update of each module itself does not need to depend on the changes of other module interfaces.
2)伸缩原则:客户产品用户访问是波动的,如果大量用户访问到一个节点的时候,必然会带来节点的服务崩溃,因此设置节点的容器本身是可以自动化部署的,当用户请求出现压力时,可以快速实现横向扩展。2) Scaling principle: Customer product user access is fluctuating. If a large number of users access a node, it will inevitably bring down the service of the node. Therefore, the container that sets the node itself can be automatically deployed. , You can quickly achieve horizontal expansion.
3)隐私原则:生态的各方参与者、挖矿节点、人工智能厂商、数据提供者等可以得到隐私保护,参与者可以根据自己的需求来选择性开放。3) Privacy principle: All participants in the ecology, mining nodes, artificial intelligence manufacturers, data providers, etc. can be protected by privacy, and participants can selectively open according to their own needs.
上述的深度学习区块链平台可以包括如下模块:The above-mentioned deep learning blockchain platform may include the following modules:
1)深度学习计算模块:1) Deep learning calculation module:
深度学习计算模块用于完成:计算任务分发、容器计算量分发。具体的,计算容器被部署成功之后,做验证计算,在验证计算通过之后,计算任务通过区块链网络进行全网分发。The deep learning calculation module is used to complete: distribution of calculation tasks and distribution of container calculations. Specifically, after the computing container is successfully deployed, the verification calculation is performed. After the verification calculation is passed, the calculation task is distributed across the entire network through the blockchain network.
2)计算容器:2) Calculation container:
用于为单个用户或者使用群组管理容器实例的整个生命周期,根据用户需求来提供虚拟服务,负责容器的创建、挂起、暂停、调整、迁移、重启、销毁等操作。It is used to manage the entire life cycle of container instances for a single user or use a group, provide virtual services according to user needs, and is responsible for container creation, suspension, suspension, adjustment, migration, restart, destruction and other operations.
当用户的并发请求计算容量达到容器分配量特定值(该值可以由用户自己设定)的时候,容器计算引擎会启动报警工作,并且会开始自动化部署容器进入其他正常节点进行扩容。When the computing capacity of a user's concurrent request reaches a specific value of the container allocation (this value can be set by the user), the container computing engine will start an alarm and will automatically deploy containers to other normal nodes for expansion.
具体的,可以先读取配置文件,读取配置参数,并且根据配置完成初始化消息队列,用以后续与别的组件进行内部消息交互。同时,根据配置文件中的配置项启动DB服务器,配置文件中的每一个API对应一个服务器。另外根据系统的GPU核心数n,每个DB服务器都会有n个协程去处理请求。Specifically, you can first read the configuration file, read the configuration parameters, and complete the initialization of the message queue according to the configuration for subsequent internal message interaction with other components. At the same time, the DB server is started according to the configuration items in the configuration file, and each API in the configuration file corresponds to a server. In addition, according to the number of GPU cores of the system n, each DB server will have n coroutines to process the request.
3)镜像管理系统:3) Image management system:
一套虚拟容器镜像查找和检索系统,有创建镜像、上传镜像、删除镜像、编辑镜像基本信息的功能镜像管理系统主要由镜像管理API和镜像管理寄存器这两种服务组成。A set of virtual container image search and retrieval system, with functions of creating image, uploading image, deleting image, editing basic information of image. Image management system mainly consists of two services: image management API and image management register.
其中,镜像管理API是镜像管理系统服务的入口,负责接收用户的API请求。镜像管理寄存器处理的是镜像元数据相关的请求,当镜像管理API收到用户的API请求后,如果判断该请求是与元数据相关,就把该请求转给镜像管理寄存器服务。然后,镜像管理寄存器会解析用户元数据请求的内容,并和数据库交互存取和更新镜像的元数据。Among them, the mirror management API is the entrance of the mirror management system service, and is responsible for receiving user API requests. The mirror management register processes mirror metadata related requests. When the mirror management API receives the user's API request, if it is determined that the request is related to metadata, the request is transferred to the mirror management register service. Then, the mirror management register will parse the content of the user's metadata request, and interact with the database to access and update the metadata of the mirror.
基于上述深度学习区块链平台可以按照如下方式进行深度学习:Based on the above deep learning blockchain platform, deep learning can be performed as follows:
S1,研究者针对某一人工智能应用研发了深度学习模型后,开源并打包发布至深度学习区块链,提供运行环境以及输入输出数据格式标准,并可以选择用公开数据集提交训练/测试任务。其中,发布的深度学习模型被他人使用时,发布者可获得相应回报。S1, after researchers have developed a deep learning model for an artificial intelligence application, open source and package it and release it to the deep learning blockchain, provide the operating environment and input and output data format standards, and can choose to submit training / test tasks with public data sets . Among them, when the released deep learning model is used by others, the publisher can get corresponding rewards.
S2,训练任务发布:发布者选择已发布的模型算法,打包数据后发布训练/测试任务。 提交前可以显示:预估的费用消耗、可用的节点、单位成本等信息,通过动态调整所有模型算法的单位成本,可达到全网所有节点执行任一深度计算任务的总体回报相同。其中,任务的执行情况及任务完成状态以交易形式记录在区块中。S2. Training task release: The publisher selects the published model algorithm, and releases the training / test task after packing the data. Before submission, you can display: estimated cost consumption, available nodes, unit cost and other information. By dynamically adjusting the unit cost of all model algorithms, you can achieve the same overall return for all nodes in the entire network to perform any deep calculation task. Among them, the task execution status and task completion status are recorded in the block in the form of transactions.
S3,任务分配与领取:接收广播到深度学习区块链中的任务以及其他节点的运行状态,每隔一定时间创建一个区块并根据算法选择运行的任务并加以执行。节点最终的回报除了任务发布者支付的费用外,任务完成后会在区块链中统计总的计算量,并按每个节点的执行情况按比例进行分配。S3, task allocation and collection: receive the tasks broadcast to the deep learning blockchain and the running status of other nodes, create a block every certain time and select the tasks to run according to the algorithm and execute them. In addition to the fees paid by the task publisher, the final reward of the node will be calculated in the blockchain after the task is completed, and distributed in proportion to the execution of each node.
在上例中,为了解决现有的深度学习集群资源耗费巨大、不能共享、存在一定程度浪费以及深度学习集群弹性不足的问题,提出了一种基于区块链的深度学习集群系统,在区块链网络中,各方具备深度学习资源和能力的平台可接入到整个深度学习网络中,依靠智能算法,对于深度学习的任务进行编排调度,从而有效提升了平台资源利用率。且任务之间相互竞争节点能力,使得集群更加高效。因为建立的是区块链系统,从而使得任务具备隐私性,信息安全得到有效保护。进一步的,根据区块链平台产生的任务调度费用,受到多方记账并认可,提升了信任度。网络中任务均由各深度学习节点协同完成,避免了中心化节点泄露任务内容和结果的问题。In the above example, in order to solve the problems of huge resource consumption, inability to share, a certain degree of waste, and insufficient flexibility of deep learning clusters in existing deep learning clusters, a blockchain-based deep learning cluster system was proposed. In the chain network, platforms with deep learning resources and capabilities of all parties can be connected to the entire deep learning network, and rely on intelligent algorithms to arrange and schedule deep learning tasks, thereby effectively improving platform resource utilization. In addition, tasks compete for node capabilities, making the cluster more efficient. Because the blockchain system is established, the task is made private and information security is effectively protected. Further, according to the task scheduling fees generated by the blockchain platform, it is accounted and recognized by multiple parties, which improves trust. All the tasks in the network are coordinated by each deep learning node, which avoids the problem that the centralized node leaks the task content and results.
在上述实施例中,接收任务发布者在区块链平台选定的深度学习模型和输入的训练集,以生成深度学习任务;在任务发布者确认发布所述深度学习任务之后,将所述深度学习任务分发至所述区块链平台中的计算节点进行深度学习。在上述方案中通过区块链平台对训练任务进行分发处理,从而解决了现有的深度学习成本过高、训练效率低下的技术问题,达到了降低训练成本、提升训练效率的技术效果。In the above embodiment, the deep learning model selected by the task publisher on the blockchain platform and the input training set are received to generate a deep learning task; after the task publisher confirms the release of the deep learning task, the depth The learning task is distributed to the computing nodes in the blockchain platform for deep learning. In the above scheme, the training tasks are distributed and processed through the blockchain platform, which solves the existing technical problems of too high deep learning costs and low training efficiency, and achieves the technical effects of reducing training costs and improving training efficiency.
通过以上的实施方式的描述,本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现,当然也可以通过硬件,但很多情况下前者是更佳的实施方式。Through the description of the above embodiments, those skilled in the art can clearly understand that the methods in the above embodiments can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware, but in many cases the former is better Implementation.
实施例2Example 2
基于上述实施例一中提供的基于区块链平台进行深度学习的方法,本实施例中提供一种基于区块链平台进行深度学习的系统,具体地,图2和图3示出了该基于区块链平台进行深度学习的系统的可选的结构框图,该基于区块链平台进行深度学习的系统被分割成一个或多个程序模块,一个或者多个程序模块被存储于存储介质中,并由一个或多个处理器所执行,以完成本申请。本申请所称的程序模块是指能够完成特定功能的一系列计算机程序指令段,比程序本身更适合描述基于区块链平台进行深度学习的系统在存储介质中的执行过程,以下描述将具体介绍本实施例各程序模块的功能:Based on the deep learning method based on the blockchain platform provided in the first embodiment above, this embodiment provides a system for deep learning based on the blockchain platform. Specifically, FIG. 2 and FIG. 3 show the An optional structural block diagram of a deep learning system for a blockchain platform. The deep learning system based on a blockchain platform is divided into one or more program modules, and one or more program modules are stored in a storage medium. It is executed by one or more processors to complete this application. The program module referred to in this application refers to a series of computer program instruction segments that can perform specific functions. It is more suitable than the program itself to describe the execution process of a deep learning system based on a blockchain platform in a storage medium. The following description will specifically introduce Functions of each program module in this embodiment:
如图2所示,基于区块链平台进行深度学习的系统20包括:As shown in FIG. 2, a system 20 for deep learning based on a blockchain platform includes:
第一接收模块201,用于接收任务发布者在区块链平台选定的深度学习模型;The first receiving module 201 is used to receive the deep learning model selected by the task publisher on the blockchain platform;
第二接收模块202,用于接收所述任务发布者输入的训练集;The second receiving module 202 is configured to receive the training set input by the task publisher;
生成模块203,用于根据所述训练集和所述深度学习模型,生成深度学习任务;A generating module 203, configured to generate a deep learning task according to the training set and the deep learning model;
分发模块204,用于将所述深度学习任务分发至所述区块链平台中的计算节点;The distribution module 204 is used to distribute the deep learning task to the computing nodes in the blockchain platform;
深度学习模块205,用于通过分发至的计算节点,对所述深度学习任务进行深度学习。The deep learning module 205 is configured to perform deep learning on the deep learning task through the distributed computing node.
在一个实施方式中,上述区块链平台中的计算节点可以包括但不限于以下至少之一:FPGA服务器集群运行的全功能节点、企业中空闲的GPU计算节点、个人闲置的GPU计算节点。In one embodiment, the computing nodes in the above-mentioned blockchain platform may include, but are not limited to, at least one of the following: a full-featured node running on an FPGA server cluster, an idle GPU computing node in an enterprise, and an individual idle GPU computing node.
在一个实施方式中,如图3所示,上述系统还可以包括:第三接收模块301,用于在接收任务发布者在区块链平台选定的深度学习模型之前,接收用户打包发布的深度学习模型,其中,所述深度学习模型中携带有:运行环境、输入数据格式、输出数据格式;发布模块302,用于将所述深度学习模型发布至区块链平台,用于任务发布者进行选择。In one embodiment, as shown in FIG. 3, the above-mentioned system may further include: a third receiving module 301, configured to receive the depth of the packaged release by the user before receiving the deep learning model selected by the task publisher on the blockchain platform Learning model, wherein the deep learning model carries: operating environment, input data format, output data format; publishing module 302, used to publish the deep learning model to the blockchain platform, for the task publisher select.
在一个实施方式中,根据所述训练集和所述深度学习模型,生成深度学习任务之后,上述系统还可以生成所述深度学习任务的任务处理信息,其中,所述任务处理信息包括以下至少之一:预估的费用、可用的计算节点、单位成本;其中,通过动态调整所有深度学习模型的单位成本,使得所述区块链平台中所有计算节点执行任一学习任务的总体回报是相同的;显示所述任务处理信息。In one embodiment, after generating a deep learning task according to the training set and the deep learning model, the above system may also generate task processing information of the deep learning task, where the task processing information includes at least one of the following One: estimated costs, available computing nodes, and unit costs; where, by dynamically adjusting the unit costs of all deep learning models, the overall return of all computing nodes in the blockchain platform to perform any learning task is the same ; Display the task processing information.
在一个实施方式中,通过分发至的计算节点,对所述深度学习任务进行深度学习之后,上述系统还可以获取深度学习任务的执行信息和完成状态;将所述执行信息和所述完成状态,以交易的形式记录在所述区块链平台中。In one embodiment, after performing deep learning on the deep learning task through the computing node to which it is distributed, the above-mentioned system can also obtain execution information and completion status of the deep learning task; Recorded in the blockchain platform in the form of transactions.
在一个实施方式中,上述分发模块可以包括:发布单元,用于在所述区块链平台发布所述深度学习任务,其中,发布信息中携带有单位任务量对应的价格;接收单元,用于接收计算节点对发布的深度学习任务的认领请求,其中,所述认领请求中携带有认领的任务量;分配单元,用于向认领请求对应的计算节点分配请求认领的任务量的深度学习任务。In one embodiment, the above-mentioned distribution module may include: a publishing unit for publishing the deep learning task on the blockchain platform, wherein the publishing information carries a price corresponding to a unit task amount; a receiving unit is used for Receiving a claim request for a deep learning task issued by a computing node, wherein the claim request carries the claimed task amount; an allocation unit is configured to allocate a deep learning task requesting the claimed task amount to the computing node corresponding to the claim request.
在一个实施方式中,所述发布单元包括:划分子单元,用于将所述深度学习任务划分为多个单位任务量;计算子单元,用于计算单位任务量的价格;确定子单元,用于根据所述深度学习任务所需的计算环境,确定可申领所述深度学习任务的计算节点;发布子单元,用于向可申领所述深度学习任务的计算节点发布所述深度学习任务。In one embodiment, the publishing unit includes: a division subunit for dividing the deep learning task into a plurality of unit task amounts; a calculation subunit for calculating the price of the unit task amount; determining a subunit, using To determine the computing nodes that can apply for the deep learning task according to the computing environment required by the deep learning task; issue a subunit for publishing the deep learning task to the computing nodes that can apply for the deep learning task .
在一个实施方式中,深度学习模块具体用于所述计算节点接收分发至自身的深度学习任务,其中,分发至自身的深度学习任务中携带有:深度学习模型和训练集;所述计算节点将所述训练集作为训练样本输入所述深度学习模型中进行训练,得到训练后的深度学习模型。In one embodiment, the deep learning module is specifically used for the computing node to receive a deep learning task distributed to itself, where the deep learning task distributed to itself carries: a deep learning model and a training set; the computing node will The training set is input as a training sample into the deep learning model for training to obtain the trained deep learning model.
关于上述实施例中的装置,其中各个单元、模块执行操作的具体方式已经在有关该方法的实施例中进行了详细描述,此处将不做详细阐述说明。Regarding the device in the above embodiment, the specific manner in which the operations of each unit and module have been described in detail in the embodiment of the method, and will not be elaborated here.
在本实施例的各个实施方式中,接收任务发布者在区块链平台选定的深度学习模型和输入的训练集,以生成深度学习任务;在任务发布者确认发布所述深度学习任务之后,将所述深度学习任务分发至所述区块链平台中的计算节点进行深度学习。在上述方案中通过区块链平台对训练任务进行分发处理,从而解决了现有的深度学习成本过高、训练效率低下的技术问题,达到了降低训练成本、提升训练效率的技术效果。In various implementations of this embodiment, the deep learning model selected by the task publisher on the blockchain platform and the input training set are received to generate a deep learning task; after the task publisher confirms to release the deep learning task, Distribute the deep learning task to the computing nodes in the blockchain platform for deep learning. In the above scheme, the training tasks are distributed and processed through the blockchain platform, which solves the existing technical problems of too high deep learning costs and low training efficiency, and achieves the technical effects of reducing training costs and improving training efficiency.
实施例3Example 3
在本申请优选的实施例三中提供一种电子装置。图4是根据本申请电子装置一实施例的硬件架构示意图。本实施例中,电子装置2是一种能够按照事先设定或者存储的指令,自动进行数值计算和/或信息处理的设备。例如,可以是智能手机、平板电脑、笔记本电脑、台式计算机、机架式服务器、刀片式服务器、塔式服务器或机柜式服务器(包括独立的服务器,或者多个服务器所组成的服务器集群)等。如图所示,电子装置2至少包括但不限于:可通过系统总线相互通信连接存储器21、处理器22、网络接口23、以及基于区块链平台进行深度学习的系统20。其中:An electronic device is provided in the third preferred embodiment of the present application. 4 is a schematic diagram of a hardware architecture according to an embodiment of an electronic device of the present application. In this embodiment, the electronic device 2 is a device that can automatically perform numerical calculation and / or information processing according to an instruction set or stored in advance. For example, it may be a smart phone, tablet computer, notebook computer, desktop computer, rack server, blade server, tower server, or rack server (including an independent server or a server cluster composed of multiple servers). As shown in the figure, the electronic device 2 includes at least but not limited to: a memory 21, a processor 22, a network interface 23, and a system 20 for deep learning based on a blockchain platform that can communicate with each other via a system bus. among them:
存储器21至少包括一种类型的计算机可读存储介质,可读存储介质包括闪存、硬盘、多媒体卡、卡型存储器(例如,SD或DX存储器等)、随机访问存储器(RAM)、静态随机访问存储器(SRAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、可编程只读存储器(PROM)、磁性存储器、磁盘、光盘等。在一些实施例中,存储器21可以是电子装置2的内部存储模块,例如该电子装置2的硬盘或内存。在另一些实施例中,存储器21也可以是电子装置2的外部存储设备,例如该电子装置2上配备的插接式硬盘,智能存储卡(Smart Media Card,SMC),安全数字(Secure Digital,SD)卡,闪存卡(Flash Card)等。当然,存储器21还可以既包括电子装置2的内部存储模块也包括其外部存储设备。本实施例中,存储器21通常用于存储安装于电子装置2的操作系统和各类应用软件,例如基于区块链平台进行深度学习的系统20的程序代码等。此外,存储器21还可以用于暂时地存储已经输出或者将要输出的各类数据。The memory 21 includes at least one type of computer-readable storage medium. The readable storage medium includes a flash memory, a hard disk, a multimedia card, a card-type memory (for example, SD or DX memory, etc.), a random access memory (RAM), and a static random access memory. (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 21 may be an internal storage module of the electronic device 2, such as a hard disk or a memory of the electronic device 2. In other embodiments, the memory 21 may also be an external storage device of the electronic device 2, such as a plug-in hard disk equipped on the electronic device 2, a smart memory card (Smart Media, Card, SMC), and secure digital (Secure Digital, SD) card, flash card (Flash Card), etc. Of course, the memory 21 may also include both the internal storage module of the electronic device 2 and its external storage device. In this embodiment, the memory 21 is generally used to store an operating system installed in the electronic device 2 and various application software, such as program codes of a system 20 for deep learning based on a blockchain platform. In addition, the memory 21 may also be used to temporarily store various types of data that have been output or will be output.
处理器22在一些实施例中可以是中央处理器(Central Processing Unit,CPU)、控制器、微控制器、微处理器、或其他数据处理芯片。该处理器22通常用于控制电子装置2的总体操作,例如执行与电子装置2进行数据交互或者通信相关的控制和处理等。本实施例中,处理器22用于运行存储器21中存储的程序代码或者处理数据,例如运行的基于区块链平台进行深度学习的系统20等。The processor 22 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments. The processor 22 is generally used to control the overall operation of the electronic device 2, such as performing control and processing related to data interaction or communication with the electronic device 2. In this embodiment, the processor 22 is used to run the program code stored in the memory 21 or process data, such as a system 20 for deep learning based on a blockchain platform.
网络接口23可包括无线网络接口或有线网络接口,该网络接口23通常用于在电子装 置2与其他电子装置之间建立通信连接。例如,网络接口23用于通过网络将电子装置2与外部终端相连,在电子装置2与外部终端之间的建立数据传输通道和通信连接等。网络可以是企业内部网(Intranet)、互联网(Internet)、全球移动通讯系统(Global System of Mobile communication,GSM)、宽带码分多址(Wideband Code Division Multiple Access,WCDMA)、4G网络、5G网络、蓝牙(Bluetooth)、Wi-Fi等无线或有线网络。The network interface 23 may include a wireless network interface or a wired network interface, and the network interface 23 is generally used to establish a communication connection between the electronic device 2 and other electronic devices. For example, the network interface 23 is used to connect the electronic device 2 with an external terminal through a network, and establish a data transmission channel and a communication connection between the electronic device 2 and the external terminal. The network can be Intranet, Internet, Global System of Mobile (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Wireless or wired networks such as Bluetooth and Wi-Fi.
需要指出的是,图4仅示出了具有部件21-23的电子装置,但是应理解的是,并不要求实施所有示出的部件,可以替代的实施更多或者更少的部件。It should be noted that FIG. 4 only shows an electronic device having components 21-23, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.
在本实施例中,存储于存储器21中的基于区块链平台进行深度学习的系统20还可以被分割为一个或者多个程序模块,一个或者多个程序模块被存储于存储器21中,并由一个或多个处理器(本实施例为处理器22)所执行,以完成本申请。In this embodiment, the system 20 for deep learning based on the blockchain platform stored in the memory 21 can also be divided into one or more program modules, one or more program modules are stored in the memory 21, and One or more processors (the processor 22 in this embodiment) are executed to complete the application.
实施例4Example 4
本实施例还提供一种计算机可读存储介质,计算机可读存储介质内存储有基于区块链平台进行深度学习的系统,监控拨打任务的系统可被至少一个处理器所执行,以使至少一个处理器执行如实施例一的基于区块链平台进行深度学习的方法的步骤。This embodiment also provides a computer-readable storage medium in which a system for deep learning based on a blockchain platform is stored. The system for monitoring and dialing tasks can be executed by at least one processor, so that at least one The processor executes the steps of the deep learning method based on the blockchain platform as in the first embodiment.
本实施例中,计算机可读存储介质包括闪存、硬盘、多媒体卡、卡型存储器(例如,SD或DX存储器等)、随机访问存储器(RAM)、静态随机访问存储器(SRAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、可编程只读存储器(PROM)、磁性存储器、磁盘、光盘等。在一些实施例中,计算机可读存储介质可以是计算机设备的内部存储单元,例如该计算机设备的硬盘或内存。在另一些实施例中,计算机可读存储介质也可以是计算机设备的外部存储设备,例如该计算机设备上配备的插接式硬盘,智能存储卡(Smart Media Card,SMC),安全数字(Secure Digital,SD)卡,闪存卡(Flash Card)等。当然,计算机可读存储介质还可以既包括计算机设备的内部存储单元也包括其外部存储设备。本实施例中,计算机可读存储介质通常用于存储安装于计算机设备的操作系统和各类应用软件,例如实施例二的客户保障分析系统的程序代码等。此外,计算机可读存储介质还可以用于暂时地存储已经输出或者将要输出的各类数据。In this embodiment, the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory ( ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the computer-readable storage medium may be an internal storage unit of the computer device, such as a hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium may also be an external storage device of the computer device, for example, a plug-in hard disk equipped on the computer device, a smart memory card (Smart Media (SMC), Secure Digital (Secure Digital) , SD) card, flash card (Flash Card), etc. Of course, the computer-readable storage medium may also include both the internal storage unit of the computer device and the external storage device thereof. In this embodiment, the computer-readable storage medium is generally used to store the operating system and various application software installed on the computer device, for example, the program code of the customer guarantee analysis system in the second embodiment. In addition, the computer-readable storage medium can also be used to temporarily store various types of data that have been output or are to be output.
显然,本领域的技术人员应该明白,上述的本申请实施例的各模块或各步骤可以用通用的计算装置来实现,它们可以集中在单个的计算装置上,或者分布在多个计算装置所组成的网络上,可选地,它们可以用计算装置可执行的程序代码来实现,从而,可以将它们存储在存储装置中由计算装置来执行,并且在某些情况下,可以以不同于此处的顺序执行所示出或描述的步骤,或者将它们分别制作成各个集成电路模块,或者将它们中的多个模块或步骤制作成单个集成电路模块来实现。这样,本申请实施例不限制于任何特定的硬件和软件结合。Obviously, those skilled in the art should understand that the modules or steps in the embodiments of the present application described above can be implemented by a general-purpose computing device, and they can be concentrated on a single computing device or distributed among multiple computing devices. On the Internet, optionally, they can be implemented with program code executable by the computing device, so that they can be stored in the storage device and executed by the computing device, and in some cases, can be different from here The steps shown or described are executed in the order of, or they are made into individual integrated circuit modules separately, or multiple modules or steps among them are made into a single integrated circuit module for implementation. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.
以上仅为本申请的优选实施例,并非因此限制本申请的专利范围,凡是利用本申请说明书及附图内容所作的等效结构或等效流程变换,或直接或间接运用在其他相关的技术领域,均同理包括在本申请的专利保护范围内。The above are only the preferred embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by using the description and drawings of this application, or directly or indirectly used in other related technical fields , The same reason is included in the scope of patent protection of this application.

Claims (20)

  1. 一种基于区块链平台进行深度学习的方法,其特征在于,包括如下步骤:A method for deep learning based on a blockchain platform is characterized by the following steps:
    接收任务发布者在区块链平台选定的深度学习模型;Receive the deep learning model selected by the task publisher on the blockchain platform;
    接收所述任务发布者输入的训练集;Receiving the training set input by the task publisher;
    根据所述训练集和所述深度学习模型,生成深度学习任务;Generate a deep learning task according to the training set and the deep learning model;
    将所述深度学习任务分发至所述区块链平台中的计算节点;Distribute the deep learning task to the computing nodes in the blockchain platform;
    通过分发至的计算节点,对所述深度学习任务进行深度学习。Perform deep learning on the deep learning task through the distributed computing nodes.
  2. 根据权利要求1所述的方法,其特征在于,将所述深度学习任务分发至所述区块链平台中的计算节点,包括:The method according to claim 1, wherein distributing the deep learning task to the computing nodes in the blockchain platform includes:
    在所述区块链平台发布所述深度学习任务,其中,发布信息中携带有单位任务量对应的价格;Publish the deep learning task on the blockchain platform, wherein the release information carries the price corresponding to the unit task volume;
    接收计算节点对发布的深度学习任务的认领请求,其中,所述认领请求中携带有认领的任务量;Receiving a claim request for a deep learning task issued by a computing node, where the claim request carries the amount of claimed tasks;
    向认领请求对应的计算节点分配请求认领的任务量的深度学习任务。The computing node corresponding to the claim request is assigned a deep learning task of the requested task amount.
  3. 根据权利要求2所述的方法,其特征在于,将所述深度学习任务在所述区块链平台进行发布,包括:The method of claim 2, wherein publishing the deep learning task on the blockchain platform includes:
    将所述深度学习任务划分为多个单位任务量;Divide the deep learning task into multiple unit task volumes;
    计算单位任务量的价格;Calculate the price of unit task volume;
    根据所述深度学习任务所需的计算环境,确定可申领所述深度学习任务的计算节点;Determine the computing nodes that can apply for the deep learning task according to the computing environment required by the deep learning task;
    向可申领所述深度学习任务的计算节点发布所述深度学习任务。The deep learning task is issued to a computing node that can apply for the deep learning task.
  4. 根据权利要求1所述的方法,其特征在于,通过分发至的计算节点,对所述深度学习任务进行深度学习,包括:The method according to claim 1, wherein performing deep learning on the deep learning task through the distributed computing node includes:
    所述计算节点接收分发至自身的深度学习任务,其中,分发至自身的深度学习任务中携带有:深度学习模型和训练集;The computing node receives a deep learning task distributed to itself, where the deep learning task distributed to itself carries: a deep learning model and a training set;
    所述计算节点将所述训练集作为训练样本输入所述深度学习模型中进行训练,得到训练后的深度学习模型;The computing node inputs the training set as training samples into the deep learning model for training to obtain the trained deep learning model;
    其中,所述区块链平台中的计算节点包括以下至少之一:FPGA服务器集群运行的全功能节点、企业中空闲的GPU计算节点、个人闲置的GPU计算节点。Wherein, the computing nodes in the blockchain platform include at least one of the following: full-featured nodes running on the FPGA server cluster, idle GPU computing nodes in enterprises, and idle GPU computing nodes in individuals.
  5. 根据权利要求1所述的方法,其特征在于,通过分发至的计算节点,对所述深度学习任务进行深度学习之后,所述方法还包括:The method according to claim 1, wherein after performing deep learning on the deep learning task through the distributed computing node, the method further comprises:
    获取所述深度学习任务的执行信息和完成状态;Acquiring execution information and completion status of the deep learning task;
    将所述执行信息和所述完成状态,以交易的形式记录在所述区块链平台中。Record the execution information and the completion status in the form of transactions in the blockchain platform.
  6. 一种电子装置,包括存储器和处理器,其特征在于,所述存储器用于存储可被所述处理器执行的基于区块链平台进行深度学习的系统,所述基于区块链平台进行深度学习的系统包括:An electronic device includes a memory and a processor, wherein the memory is used to store a deep learning system based on a blockchain platform executable by the processor, and the deep learning based on the blockchain platform The system includes:
    第一接收模块,用于接收任务发布者在区块链平台选定的深度学习模型;The first receiving module is used to receive the deep learning model selected by the task publisher on the blockchain platform;
    第二接收模块,用于接收所述任务发布者输入的训练集;A second receiving module, configured to receive the training set input by the task publisher;
    生成模块,用于根据所述训练集和所述深度学习模型,生成深度学习任务;A generating module, configured to generate a deep learning task according to the training set and the deep learning model;
    分发模块,用于将所述深度学习任务分发至所述区块链平台中的计算节点;A distribution module for distributing the deep learning tasks to the computing nodes in the blockchain platform;
    深度学习模块,用于通过分发至的计算节点,对所述深度学习任务进行深度学习。A deep learning module is used to perform deep learning on the deep learning task through the computing node to which it is distributed.
  7. 根据权利要求6所述的电子装置,其特征在于,所述分发模块包括:The electronic device according to claim 6, wherein the distribution module comprises:
    发布单元,用于在所述区块链平台发布所述深度学习任务,其中,发布信息中携带有单位任务量对应的价格;A release unit, configured to release the deep learning task on the blockchain platform, wherein the release information carries a price corresponding to a unit task volume;
    接收单元,用于接收计算节点对发布的深度学习任务的认领请求,其中,所述认领请求中携带有认领的任务量;A receiving unit, configured to receive a computing node's claim request for the issued deep learning task, where the claim request carries the claimed task amount;
    分配单元,用于向认领请求对应的计算节点分配请求认领的任务量的深度学习任务。The allocation unit is used for allocating the deep learning task of the requested task amount to the computing node corresponding to the claim request.
  8. 根据权利要求7所述的电子装置,其特征在于,将所述深度学习任务在所述区块链平台进行发布,包括:The electronic device according to claim 7, wherein publishing the deep learning task on the blockchain platform includes:
    将所述深度学习任务划分为多个单位任务量;Divide the deep learning task into multiple unit task volumes;
    计算单位任务量的价格;Calculate the price of unit task volume;
    根据所述深度学习任务所需的计算环境,确定可申领所述深度学习任务的计算节点;Determine the computing nodes that can apply for the deep learning task according to the computing environment required by the deep learning task;
    向可申领所述深度学习任务的计算节点发布所述深度学习任务。The deep learning task is issued to a computing node that can apply for the deep learning task.
  9. 根据权利要求6所述的电子装置,其特征在于,所述深度学习模块包括:The electronic device according to claim 6, wherein the deep learning module comprises:
    接收单元,用于计算节点接收分发至自身的深度学习任务,其中,分发至自身的深度学习任务中携带有:深度学习模型和训练集;The receiving unit is used for the computing node to receive the deep learning task distributed to itself, wherein the deep learning task distributed to itself carries: deep learning model and training set;
    输入单元,用于所述计算节点将所述训练集作为训练样本输入所述深度学习模型中进行训练,得到训练后的深度学习模型;The input unit is used by the computing node to input the training set as a training sample into the deep learning model for training to obtain the trained deep learning model;
    其中,所述区块链平台中的计算节点包括以下至少之一:FPGA服务器集群运行的全功能节点、企业中空闲的GPU计算节点、个人闲置的GPU计算节点。Wherein, the computing nodes in the blockchain platform include at least one of the following: full-featured nodes running on the FPGA server cluster, idle GPU computing nodes in enterprises, and idle GPU computing nodes in individuals.
  10. 根据权利要求6所述的电子装置,其特征在于,还用于通过分发至的计算节点,对所述深度学习任务进行深度学习之后,获取所述深度学习任务的执行信息和完 成状态;将所述执行信息和所述完成状态,以交易的形式记录在所述区块链平台中。The electronic device according to claim 6, characterized in that it is further used to obtain execution information and completion status of the deep learning task after performing deep learning on the deep learning task through the computing node to which it is distributed; The execution information and the completion status are recorded in the blockchain platform in the form of transactions.
  11. 一种计算机设备,包括:存储器和处理器,其中,所述处理器用于执行如下步骤:A computer device includes: a memory and a processor, wherein the processor is used to perform the following steps:
    接收任务发布者在区块链平台选定的深度学习模型;Receive the deep learning model selected by the task publisher on the blockchain platform;
    接收所述任务发布者输入的训练集;Receiving the training set input by the task publisher;
    根据所述训练集和所述深度学习模型,生成深度学习任务;Generate a deep learning task according to the training set and the deep learning model;
    将所述深度学习任务分发至所述区块链平台中的计算节点;Distribute the deep learning task to the computing nodes in the blockchain platform;
    通过分发至的计算节点,对所述深度学习任务进行深度学习。Perform deep learning on the deep learning task through the distributed computing nodes.
  12. 根据权利要求11所述的计算机设备,其特征在于,将所述深度学习任务分发至所述区块链平台中的计算节点,包括:The computer device according to claim 11, wherein the distribution of the deep learning task to the computing nodes in the blockchain platform includes:
    在所述区块链平台发布所述深度学习任务,其中,发布信息中携带有单位任务量对应的价格;Publish the deep learning task on the blockchain platform, wherein the release information carries the price corresponding to the unit task volume;
    接收计算节点对发布的深度学习任务的认领请求,其中,所述认领请求中携带有认领的任务量;Receiving a claim request for a deep learning task issued by a computing node, where the claim request carries the amount of claimed tasks;
    向认领请求对应的计算节点分配请求认领的任务量的深度学习任务。The computing node corresponding to the claim request is assigned a deep learning task of the requested task amount.
  13. 根据权利要求12所述的计算机设备,其特征在于,将所述深度学习任务在所述区块链平台进行发布,包括:The computer device according to claim 12, wherein publishing the deep learning task on the blockchain platform includes:
    将所述深度学习任务划分为多个单位任务量;Divide the deep learning task into multiple unit task volumes;
    计算单位任务量的价格;Calculate the price of unit task volume;
    根据所述深度学习任务所需的计算环境,确定可申领所述深度学习任务的计算节点;Determine the computing nodes that can apply for the deep learning task according to the computing environment required by the deep learning task;
    向可申领所述深度学习任务的计算节点发布所述深度学习任务。The deep learning task is issued to a computing node that can apply for the deep learning task.
  14. 根据权利要求11所述的计算机设备,其特征在于,通过分发至的计算节点,对所述深度学习任务进行深度学习,包括:The computer device according to claim 11, wherein performing deep learning on the deep learning task through the distributed computing node includes:
    所述计算节点接收分发至自身的深度学习任务,其中,分发至自身的深度学习任务中携带有:深度学习模型和训练集;The computing node receives a deep learning task distributed to itself, where the deep learning task distributed to itself carries: a deep learning model and a training set;
    所述计算节点将所述训练集作为训练样本输入所述深度学习模型中进行训练,得到训练后的深度学习模型;The computing node inputs the training set as training samples into the deep learning model for training to obtain the trained deep learning model;
    其中,所述区块链平台中的计算节点包括以下至少之一:FPGA服务器集群运行的全功能节点、企业中空闲的GPU计算节点、个人闲置的GPU计算节点。Wherein, the computing nodes in the blockchain platform include at least one of the following: full-featured nodes running on the FPGA server cluster, idle GPU computing nodes in enterprises, and idle GPU computing nodes in individuals.
  15. 根据权利要求11所述的计算机设备,其特征在于,通过分发至的计算节点,对所述深度学习任务进行深度学习之后,所述方法还包括:The computer device according to claim 11, wherein after performing deep learning on the deep learning task through the distributed computing node, the method further comprises:
    获取所述深度学习任务的执行信息和完成状态;Acquiring execution information and completion status of the deep learning task;
    将所述执行信息和所述完成状态,以交易的形式记录在所述区块链平台中。Record the execution information and the completion status in the form of transactions in the blockchain platform.
  16. 一种非易失性计算机可读存储介质,其上存储有计算机指令,所述指令被执行时实现如下步骤:A non-volatile computer-readable storage medium having computer instructions stored thereon, the instructions implement the following steps when executed:
    接收任务发布者在区块链平台选定的深度学习模型;Receive the deep learning model selected by the task publisher on the blockchain platform;
    接收所述任务发布者输入的训练集;Receiving the training set input by the task publisher;
    根据所述训练集和所述深度学习模型,生成深度学习任务;Generate a deep learning task according to the training set and the deep learning model;
    将所述深度学习任务分发至所述区块链平台中的计算节点;Distribute the deep learning task to the computing nodes in the blockchain platform;
    通过分发至的计算节点,对所述深度学习任务进行深度学习。Perform deep learning on the deep learning task through the distributed computing nodes.
  17. 根据权利要求16所述的非易失性计算机可读存储介质,其特征在于,将所述深度学习任务分发至所述区块链平台中的计算节点,包括:The non-volatile computer-readable storage medium of claim 16, wherein distributing the deep learning task to computing nodes in the blockchain platform includes:
    在所述区块链平台发布所述深度学习任务,其中,发布信息中携带有单位任务量对应的价格;Publish the deep learning task on the blockchain platform, wherein the release information carries the price corresponding to the unit task volume;
    接收计算节点对发布的深度学习任务的认领请求,其中,所述认领请求中携带有认领的任务量;Receiving a claim request for a deep learning task issued by a computing node, where the claim request carries the amount of claimed tasks;
    向认领请求对应的计算节点分配请求认领的任务量的深度学习任务。The computing node corresponding to the claim request is assigned a deep learning task of the requested task amount.
  18. 根据权利要求17所述的非易失性计算机可读存储介质,其特征在于,将所述深度学习任务在所述区块链平台进行发布,包括:The non-volatile computer-readable storage medium of claim 17, wherein publishing the deep learning task on the blockchain platform includes:
    将所述深度学习任务划分为多个单位任务量;Divide the deep learning task into multiple unit task volumes;
    计算单位任务量的价格;Calculate the price of unit task volume;
    根据所述深度学习任务所需的计算环境,确定可申领所述深度学习任务的计算节点;Determine the computing nodes that can apply for the deep learning task according to the computing environment required by the deep learning task;
    向可申领所述深度学习任务的计算节点发布所述深度学习任务。The deep learning task is issued to a computing node that can apply for the deep learning task.
  19. 根据权利要求16所述的非易失性计算机可读存储介质,其特征在于,通过分发至的计算节点,对所述深度学习任务进行深度学习,包括:The non-volatile computer-readable storage medium according to claim 16, wherein performing deep learning on the deep learning task through the computing node to which it is distributed includes:
    所述计算节点接收分发至自身的深度学习任务,其中,分发至自身的深度学习任务中携带有:深度学习模型和训练集;The computing node receives a deep learning task distributed to itself, where the deep learning task distributed to itself carries: a deep learning model and a training set;
    所述计算节点将所述训练集作为训练样本输入所述深度学习模型中进行训练,得到训练后的深度学习模型;The computing node inputs the training set as training samples into the deep learning model for training to obtain the trained deep learning model;
    其中,所述区块链平台中的计算节点包括以下至少之一:FPGA服务器集群运行的全功能节点、企业中空闲的GPU计算节点、个人闲置的GPU计算节点。Wherein, the computing nodes in the blockchain platform include at least one of the following: full-featured nodes running on the FPGA server cluster, idle GPU computing nodes in enterprises, and idle GPU computing nodes in individuals.
  20. 根据权利要求16所述的非易失性计算机可读存储介质,其特征在于,通过分 发至的计算节点,对所述深度学习任务进行深度学习之后,所述方法还包括:The non-volatile computer-readable storage medium according to claim 16, wherein after performing deep learning on the deep learning task through the distributed computing node, the method further includes:
    获取所述深度学习任务的执行信息和完成状态;Acquiring execution information and completion status of the deep learning task;
    将所述执行信息和所述完成状态,以交易的形式记录在所述区块链平台中。Record the execution information and the completion status in the form of transactions in the blockchain platform.
PCT/CN2019/070289 2018-10-25 2019-01-03 Method for carrying out deep learning on basis of blockchain platform and electronic device WO2020082611A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811252535.2 2018-10-25
CN201811252535.2A CN109409738A (en) 2018-10-25 2018-10-25 Method, the electronic device of deep learning are carried out based on block platform chain

Publications (1)

Publication Number Publication Date
WO2020082611A1 true WO2020082611A1 (en) 2020-04-30

Family

ID=65469643

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/070289 WO2020082611A1 (en) 2018-10-25 2019-01-03 Method for carrying out deep learning on basis of blockchain platform and electronic device

Country Status (2)

Country Link
CN (1) CN109409738A (en)
WO (1) WO2020082611A1 (en)

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110197285B (en) * 2019-05-07 2021-03-23 清华大学 Block chain-based safe cooperation deep learning method and device
CN111327692A (en) * 2020-02-05 2020-06-23 北京百度网讯科技有限公司 Model training method and device and cluster system
CN111679905B (en) * 2020-05-11 2022-03-08 天津大学 Calculation network fusion network model system
CN111753984A (en) * 2020-06-28 2020-10-09 中国银行股份有限公司 Distributed AI training method, device and system based on block chain
CN112150152B (en) * 2020-10-09 2023-08-08 浙江专线宝网阔物联科技有限公司 B-F neural network traceable algorithm based on fusion of block chain and fuzzy cognitive map
CN112181599B (en) * 2020-10-16 2023-05-16 中国联合网络通信集团有限公司 Model training method, device and storage medium
US20220138550A1 (en) * 2020-10-29 2022-05-05 International Business Machines Corporation Blockchain for artificial intelligence training
CN114819195A (en) * 2021-01-28 2022-07-29 华为技术有限公司 Training method, device and system of ensemble learning model and related equipment
CN112968897B (en) * 2021-02-25 2022-04-08 浙江清华长三角研究院 Container calculation method operating in decentralized system

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106529682A (en) * 2016-10-28 2017-03-22 北京奇虎科技有限公司 Method and apparatus for processing deep learning task in big-data cluster
CN106529673A (en) * 2016-11-17 2017-03-22 北京百度网讯科技有限公司 Deep learning network training method and device based on artificial intelligence
CN107864198A (en) * 2017-11-07 2018-03-30 济南浪潮高新科技投资发展有限公司 A kind of block chain common recognition method based on deep learning training mission

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105808500A (en) * 2016-02-26 2016-07-27 山西牡丹深度智能科技有限公司 Realization method and device of deep learning
WO2019144353A1 (en) * 2018-01-25 2019-08-01 深圳前海达闼云端智能科技有限公司 Blockchain-based data training method and device, storage medium and blockchain node

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106529682A (en) * 2016-10-28 2017-03-22 北京奇虎科技有限公司 Method and apparatus for processing deep learning task in big-data cluster
CN106529673A (en) * 2016-11-17 2017-03-22 北京百度网讯科技有限公司 Deep learning network training method and device based on artificial intelligence
CN107864198A (en) * 2017-11-07 2018-03-30 济南浪潮高新科技投资发展有限公司 A kind of block chain common recognition method based on deep learning training mission

Also Published As

Publication number Publication date
CN109409738A (en) 2019-03-01

Similar Documents

Publication Publication Date Title
WO2020082611A1 (en) Method for carrying out deep learning on basis of blockchain platform and electronic device
Agmon Ben-Yehuda et al. Deconstructing Amazon EC2 spot instance pricing
Van den Bossche et al. Online cost-efficient scheduling of deadline-constrained workloads on hybrid clouds
US8589929B2 (en) System to provide regular and green computing services
US10680975B2 (en) Method of dynamic resource allocation for public clouds
US11388232B2 (en) Replication of content to one or more servers
US8880671B2 (en) Releasing computing infrastructure components in a networked computing environment
CN102185926A (en) Cloud computing resource management system and method
US9275408B1 (en) Transferring ownership of computing resources
CN109254836B (en) Deadline constraint cost optimization scheduling method for priority dependent tasks of cloud computing system
CN106815254A (en) A kind of data processing method and device
US20210208948A1 (en) System and method for operating a service to arrange automatic resource capacity marketplace between Kubernetes clusters.
WO2021244343A1 (en) Cloud computing power allocation method, user terminal, cloud computing power platform and system
Yao et al. Cutting your cloud computing cost for deadline-constrained batch jobs
US9699114B1 (en) Providing use of local or private cloud infrastructure resources to public cloud providers
CN112182089B (en) Report generation method, device and equipment based on data warehouse model
CN115509754A (en) Business data processing method and device, electronic equipment and storage medium
US11176506B2 (en) Blockchain expense and resource utilization optimization
CN116074541B (en) Resource processing method, system, device and electronic equipment
Venkateswaran et al. Architecture of a time‐sensitive provisioning system for cloud‐native software
Sallo et al. Towards a DISSECT-CF extension for simulating function-as-a-service
CN114792186A (en) Production scheduling simulation method and device
CN115794378A (en) Batch task processing method, device and system and computer equipment
US20140188530A1 (en) Provision of customer attributes to an organization
CN117389715A (en) Resource allocation method, resource allocation device, computer device, and storage medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19875999

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19875999

Country of ref document: EP

Kind code of ref document: A1