WO2023082575A1 - 一种面向神经网络模型计算的图执行流水并行方法和装置 - Google Patents

一种面向神经网络模型计算的图执行流水并行方法和装置 Download PDF

Info

Publication number
WO2023082575A1
WO2023082575A1 PCT/CN2022/092481 CN2022092481W WO2023082575A1 WO 2023082575 A1 WO2023082575 A1 WO 2023082575A1 CN 2022092481 W CN2022092481 W CN 2022092481W WO 2023082575 A1 WO2023082575 A1 WO 2023082575A1
Authority
WO
WIPO (PCT)
Prior art keywords
execution
execution body
data
memory block
batch
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2022/092481
Other languages
English (en)
French (fr)
Inventor
王宏升
谭博文
鲍虎军
陈�光
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang Lab
Original Assignee
Zhejiang Lab
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang Lab filed Critical Zhejiang Lab
Priority to US17/838,342 priority Critical patent/US12468921B2/en
Publication of WO2023082575A1 publication Critical patent/WO2023082575A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
    • G06F12/023—Free address space management
    • G06F12/0253—Garbage collection, i.e. reclamation of unreferenced memory
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/901—Indexing; Data structures therefor; Storage structures
    • G06F16/9024—Graphs; Linked lists
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/30—Arrangements for executing machine instructions, e.g. instruction decode
    • G06F9/38—Concurrent instruction execution, e.g. pipeline or look ahead
    • G06F9/3885—Concurrent instruction execution, e.g. pipeline or look ahead using a plurality of independent parallel functional units
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
    • G06F9/5016—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
    • G06F9/5022—Mechanisms to release resources
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/50—Indexing scheme relating to G06F9/50
    • G06F2209/5017—Task decomposition
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Definitions

  • the invention relates to the technical field of deep learning, in particular to a graph execution pipeline parallel method and device for neural network model calculation.
  • the graph execution pipeline parallel method for neural network model calculation isolates each batch of training data and different subgraphs, and each batch of training data flows sequentially in a 1F1B forward-to-backward manner Through the forward calculation graph and the reverse calculation graph.
  • the invention can make each equipment process have a batch of data being processed, keep all equipment processes busy without pipeline suspension, and the entire assembly line is relatively balanced. At the same time, it can ensure that the parameter update on each subgraph is performed at a fixed cycle, which also helps to prevent too many small batches from being processed at the same time and ensures model convergence.
  • the object of the present invention is to provide a graph execution pipeline parallel method and device oriented to neural network model calculation, so as to overcome the deficiencies in the prior art.
  • the present invention provides the following technical solutions:
  • the present application discloses a graph execution pipeline parallel method oriented to neural network model calculation.
  • the neural network model is provided with several executors, and there are 2*N executors in total, and N is a positive integer.
  • the execution body is provided with several memory blocks; the method specifically includes the following steps:
  • the n-th executive executes the S2 operation on the i+1-th batch of sub-data, and at the same time, sends the address of the memory block where the i-th batch is located to the n-th +1 execution body; the n+1th execution body analyzes the memory block where the i-th batch is located, obtains the execution result of the n-th execution body on the i-th batch of sub-data, and uses the execution result of the n-th execution body as the n-th execution body The input data of +1 execution body, execute its own kernel function calculation, and write the execution result into the free memory block of the n+1th execution body; then input the sub-data of the i+2th batch;
  • the nth executive executes the S2 operation on the i+2th batch of sub-data
  • the nth executive and the n+1th executive execute the i+1th batch
  • the second sub-data executes the operation of S3; at the same time, the n+1th execution body sends the address of the memory block where the i-th batch is located to the n+2th execution body, and the n+2th execution body analyzes the memory where the i-th batch is located block, obtain the execution result of the n+1th executive body on the i-th batch of sub-data, and use the execution result of the n+1th executive body as the input data of the n+2th executive body, perform its own kernel function calculation, and Write the execution result to the free memory block of the n+2th execution body;
  • the nth execution body reclaims the memory block sent to the n+1th execution body
  • the last execution body executes its own kernel function calculation, and writes the execution result into the memory block of the last execution body, and reclaims the memory block by itself immediately after execution.
  • the executive body checks whether there is a free memory block in itself; if it exists, then executes its own kernel function calculation on the i-th batch of sub-data; The batch waits for a free block of memory to exist.
  • the executable will check whether the execution of the sub-data of the N*(n-1)+1 batch is completed before executing its own kernel function calculation , the n is a positive integer.
  • step S5 specifically includes the following operations:
  • the n+1th execution body notifies the nth execution body that the memory block sent to the n+1th execution body has been consumed;
  • the nth execution body reclaims the memory block sent to the n+1th execution body, and marks it as free.
  • the construction of the execution body specifically includes the following sub-steps:
  • S02. Create the thread of the executive body: the thread of the executive body is responsible for sequentially obtaining the current pending tasks from the kernel function task queue, and submitting them to the thread pool;
  • S03. Create an executive body of kernel function create an executive body used for operator kernel function calculation according to the context information of the current kernel function task and the current thread; and use the executive body to run the kernel function task in the task queue;
  • the thread of the event recall queue is responsible for taking out and returning the processed tasks in the event recall queue in turn.
  • the invention also discloses a graph execution device oriented to neural network model calculation, which includes an execution body construction module and an execution body pipeline parallel work module, the execution body construction module is used for the construction of the execution body, and the execution body pipeline parallel work The module is used to implement the above-mentioned graph execution pipeline parallel method for neural network model calculation.
  • the invention also discloses a graph execution device oriented to neural network model calculation, which includes memory and one or more processors, executable codes are stored in the memory, and the executable code is executed by the one or more processors.
  • the code is used, it is used in the above-mentioned graph execution pipeline parallel method for neural network model calculation.
  • the invention also discloses a computer-readable storage medium, on which a program is stored.
  • a program is stored.
  • the above-mentioned graph execution pipeline parallel method oriented to neural network model calculation is realized.
  • the present invention Provides a graph execution pipeline parallel method and device for neural network model calculations, creates graph execution bodies on the machine based on the physical computing graphs compiled and generated by the deep learning framework, and allocates multiple free memories for each graph execution body by design
  • the block scheme realizes that the entire calculation graph participates in the deep learning training tasks of different batches of data in a parallel manner.
  • the graph executive parallel execution method based on multiple free tensor storage blocks disclosed by the present invention is easier to implement distributed training of large models than existing methods.
  • the present invention has a low threshold for users, and enables the model to learn the internal correlation of a large number of data flowing into the neural network in batches, thereby obtaining the "intelligence" in the corresponding scenario. "Perception and judgment ability.
  • the present invention provides a set of simple and easy-to-use operating device of the neural network model for algorithm engineers related to deep learning, so that the deep learning model can be trained conveniently.
  • Figure 1 is an architecture diagram of a graph execution pipeline parallel method for neural network model calculation
  • Fig. 2 is a flow chart of creating a management task executive body thread module
  • FIG. 1 The basic actions of the task execution body pipeline parallel work module
  • FIG. 5 is a schematic structural diagram of a graph execution device oriented to neural network model calculation in the present invention.
  • FIG. 1 the architecture diagram of the neural network model calculation-oriented graph execution pipeline parallel method.
  • the training data is fed into the neural network model in batches, and the graph execution body on the machine is created according to the physical computing graph compiled and generated by the deep learning framework, and multiple free memory blocks are allocated to each graph execution body, so that the entire calculation
  • the graph simultaneously participates in the deep learning training task in a pipelined parallel manner.
  • the specific operations are as follows:
  • the n-th executive executes the S2 operation on the i+1-th batch of sub-data, and at the same time, sends the address of the memory block where the i-th batch is located to the n-th +1 execution body; the n+1th execution body analyzes the memory block where the i-th batch is located, obtains the execution result of the n-th execution body on the i-th batch of sub-data, and uses the execution result of the n-th execution body as the n-th execution body The input data of +1 execution body, execute its own kernel function calculation, and write the execution result into the free memory block of the n+1th execution body; then input the sub-data of the i+2th batch;
  • the nth executive executes the S2 operation on the i+2th batch of sub-data
  • the nth executive and the n+1th executive execute the i+1th batch
  • the second sub-data executes the operation of S3; at the same time, the n+1th execution body sends the address of the memory block where the i-th batch is located to the n+2th execution body, and the n+2th execution body analyzes the memory where the i-th batch is located block, obtain the execution result of the n+1th executive body on the i-th batch of sub-data, and use the execution result of the n+1th executive body as the input data of the n+2th executive body, perform its own kernel function calculation, and Write the execution result to the free memory block of the n+2th execution body;
  • the nth execution body reclaims the memory block sent to the n+1th execution body.
  • the last execution body executes its own kernel function calculation, and writes the execution result into the memory block of the last execution body, and reclaims the memory block by itself immediately after execution.
  • the executive body checks whether there is a free memory block; if there is, then executes its own kernel function calculation on the i-th batch of sub-data; exists, make the i-th batch wait for a free memory block.
  • the executive will check the location of the sub-data of the N*(n-1)+1 batch before executing its own kernel function calculation. Whether the execution of the execution body is completed, and the n is a positive integer.
  • step S5 specifically includes the following operations:
  • the n+1th execution body notifies the nth execution body that the memory block sent to the n+1th execution body has been consumed;
  • the nth execution body reclaims the memory block sent to the n+1th execution body, and marks it as free.
  • the construction of the execution body specifically includes the following sub-steps:
  • S02. Create the thread of the executive body: the thread of the executive body is responsible for sequentially obtaining the current pending tasks from the kernel function task queue, and submitting them to the thread pool;
  • S03. Create an executive body of kernel function create an executive body used for operator kernel function calculation according to the context information of the current kernel function task and the current thread; and use the executive body to run the kernel function task in the task queue;
  • the present invention is a graph execution device oriented to neural network model calculation, comprising an execution body construction module and an execution body pipeline parallel work module,
  • the executive body construction module includes the following basic actions:
  • Create the task queue of the operator kernel function add the calculation tasks of the current operator kernel function to the current kernel function task queue in turn; create the thread of the task executive body: create the thread of the task executive body.
  • the thread of the task executor is responsible for sequentially obtaining the current pending tasks from the task queue; when the server receives a request, it submits the request to the thread pool and continues to wait for other requests. If there is an available thread in the pool, it will be woken up and the request will be serviced immediately. If there are no threads available in the pool, tasks are queued until a free thread becomes available. Once a thread has finished its service, it returns to the pool to wait for more work. Thread pools work well when the tasks submitted to them can be executed asynchronously.
  • Create the task executive body of the kernel function Create a task executive body for operator kernel function calculation according to the context information of the current kernel function task and the current thread. And use the task executor to run the kernel function tasks in the task queue.
  • the thread that creates the event recall queue the thread that creates the event recall queue.
  • the threads of the event recall queue are responsible for sequentially taking out and returning the processed tasks in the event recall queue.
  • the execution body pipeline parallel work module includes the following basic actions: the execution body inputs data, the current execution body sends a message to the downstream execution body, the downstream execution body prepares the tensor data to be consumed, and the current execution body sends the upstream execution body Send a message, the upstream execution body reclaims the tensor data that has been consumed, and the tail execution body recycles the calculation data by itself.
  • Executive input data at time t, for the i-th batch of data, the executive inputs the i-th batch of data, loads its internal operator kernel function calculation task, executes the kernel function calculation, and generates the output tensor of the kernel function calculation task Data, write the execution result to the free memory block;
  • the current executive sends a message to the downstream executive: at time t, for the i-th batch of data, store the tensor data produced by the current executive in an empty storage unit, and then match the address of the storage unit with the current executive
  • the identity number of the downstream executable is packaged into a message, and then the message is sent to the target executable, and the target executable is the downstream executable corresponding to the current executable;
  • the downstream execution body prepares the tensor data to be consumed: at time t, for the i-th batch of data, the downstream execution body receives the message, parses the tensor data produced by the above-mentioned current execution body from the message, and the tensor data will be As the input tensor when the downstream execution body runs its operator kernel function, and checks whether there are free memory blocks available in the memory block produced by itself, if there is an available free memory block, the downstream execution body executes the kernel function of the corresponding operator Computing tasks, read free memory blocks, and the downstream execution body writes the output tensor results generated by execution into memory blocks;
  • the current execution body sends a message to the upstream execution body: at time t, for the i-th batch of data, the execution body sends a message to the upstream producer execution body to notify the upstream producer execution body that the execution body consumes the upstream producer execution body.
  • Memory block the storage unit where the upstream executive can reclaim its output tensor data;
  • the upstream execution body recycles the consumed data: at time t, for the i-th batch of data, the upstream execution body receives the recycling message sent by the downstream execution body, and starts to check whether the memory block has been consumed by all consumer execution bodies After the consumption is completed, if so, the memory block is reclaimed and marked as a free block;
  • the tail execution body reclaims the calculation data by itself: at time t, for the i-th batch of data, the tail execution body executes the kernel function calculation task of the corresponding operator, writes its own free memory block, and executes body A to reclaim the memory block by itself immediately after execution.
  • An embodiment of a graph execution device oriented to neural network model calculation in the present invention can be applied to any device with data processing capability, and any device with data processing capability can be a device or device such as a computer.
  • the device embodiments can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by reading the corresponding computer program instructions in the non-volatile memory into the memory for operation by the processor of any device capable of data processing.
  • any device with data processing capability where the neural network model calculation-oriented graph execution device is located, except for the processor shown in Figure 5,
  • any device with data processing capability where the device in the embodiment is usually based on the actual function of any device with data processing capability may also include other hardware. Let me repeat. For the implementation process of the functions and effects of each unit in the above device, please refer to the implementation process of the corresponding steps in the above method for details, and will not be repeated here.
  • the device embodiment since it basically corresponds to the method embodiment, please refer to the part description of the method embodiment for relevant parts.
  • the device embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in One place, or it can be distributed to multiple network elements. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of the present invention. It can be understood and implemented by those skilled in the art without creative effort.
  • An embodiment of the present invention also provides a computer-readable storage medium, on which a program is stored.
  • a program is stored.
  • a graph execution device oriented to neural network model calculation in the above-mentioned embodiments is implemented.
  • the computer-readable storage medium may be an internal storage unit of any device capable of data processing described in any of the foregoing embodiments, such as a hard disk or a memory.
  • the computer-readable storage medium can also be an external storage device of any device with data processing capabilities, such as a plug-in hard disk, a smart media card (Smart Media Card, SMC), an SD card, and a flash memory card equipped on the device. (Flash Card) etc.
  • the computer-readable storage medium may also include both an internal storage unit of any device capable of data processing and an external storage device.
  • the computer-readable storage medium is used to store the computer program and other programs and data required by any device capable of data processing, and may also be used to temporarily store data that has been output or will be output.
  • building a physical calculation graph consists of forward operator x->forward operator y->forward operator z and reverse operator Z->reverse operator Y->reverse operator X, According to each operator, create an execution body that runs its own kernel function, corresponding to the execution calculation graph that constitutes execution body a->execution body b->execution body c->execution body C->execution body B->execution body A; start The execution body at runtime runs the entire computation graph in parallel.
  • Execution body b, execution body c, execution body C, execution body B, and execution body A have no readable input tensor data, so execution bodies b, c, C, B, and A are in the waiting state.
  • Step 3 Run the entire computational graph in parallel.
  • execution body a For the second batch of data, execution body a inputs data: execution body a also checks whether it has free blocks to write, and finds that there are, and execution body a is also executing the second batch of input data at T2, and writes the execution result into Memory free block r12.
  • the current execution body a sends a message to the downstream execution body b, and the downstream execution body b prepares the tensor data to be consumed: the execution body a sends a message to the execution body b to notify the execution body b to read the execution body a
  • the output memory block r11, the execution body b receives the message, and checks whether the memory block b produced by itself has a free memory block available, and finds that there is an available free memory block r21, so the execution body b executes the forward operator b at time T2
  • the kernel function calculation task reads the memory block r11, and the execution body b writes the output tensor result generated by the execution into the memory block r21.
  • execution body a inputs data: execution body a will also check whether it has free blocks to write, if found, execution body a is also executing the third batch of input data, and writes the execution result into the memory Free block r13.
  • the current execution body b sends a message to the downstream execution body c, the downstream execution body c prepares the tensor data to be consumed, the current execution body b sends a message to the upstream execution body a, and the upstream execution body a recycles Consumed tensor data:
  • Executor b produced memory block r21, so it sent a message to downstream consumer executive c to notify executive c to read the memory block r21 produced by executive b, and executive c received it
  • the memory block r21 finds that the memory block r31 is free, so the execution body c starts to execute, reads the memory block r21, and writes to the memory block r31.
  • execution body b sends a message to the upstream producer execution body a to inform execution body a that execution body b has used up the memory block r11 of execution body a, and execution body a has received the memory block r11 returned by execution body b after running out, and checks the memory All consumers of block r11 are used up, so the memory block r11 is reclaimed and marked as a free block.
  • the current execution body a sends a message to the downstream execution body b, and the downstream execution body b prepares the tensor data to be consumed: the execution body a sends a message to the execution body b to notify the execution body b to read the execution body a
  • the output memory block r12 the execution body b receives the message, and checks whether the memory block b produced by itself has a free memory block available, and finds that there is an available free memory block r22, so the execution body b executes the kernel of the forward operator b
  • the function computing task reads the memory block r12, and the execution body b writes the output tensor result generated by the execution into the memory block r22.
  • execution body a inputs data: execution body a will also check whether it has free blocks to write and execution body A has finished executing. If it finds no, it will wait and not enter the pipeline.
  • the current execution body c sends a message to the downstream execution body C, the downstream execution body C prepares the tensor data to be consumed, the current execution body c sends a message to the upstream execution body b, and the upstream execution body b recycles Consumed tensor data: executor c produced memory block r31, then sent a message to downstream consumer executor C to notify executor C to read the memory block r31 produced by executor c, and executor C received
  • the memory block r31 finds that the memory block r11 is free, so the execution body C starts to execute, reads the memory block r31, and writes to the memory block r11.
  • execution body c sends a message to the upstream producer execution body b to inform execution body b that execution body c has used up the memory block r21 of execution body b, and execution body b has received the memory block r21 returned by execution body c after running out, and checks the memory All consumers of block r21 are used up, so the memory block r21 is reclaimed and marked as a free block.
  • the current execution body b sends a message to the downstream execution body c, the downstream execution body c prepares tensor data to be consumed, the current execution body b sends a message to the upstream execution body a, and the upstream execution body a recycles Consumed tensor data:
  • Executor b produced memory block r22, so it sent a message to downstream consumer executive c to notify executive c to read the memory block r22 produced by executive b, and executive c received it
  • the memory block r22 finds that the memory block r32 is free, so the execution body c starts to execute, reads the memory block r22, and writes to the memory block r32.
  • execution body b sends a message to the upstream producer execution body a to inform execution body a that execution body b has used up the memory block r12 of execution body a, and execution body a has received the memory block r12 returned by execution body b after running out, and checks the memory All consumers of block r12 are used up, so the memory block r12 is reclaimed and marked as a free block.
  • the current execution body a sends a message to the downstream execution body b, and the downstream execution body b prepares the tensor data to be consumed: the execution body a sends a message to the execution body b to notify the execution body b to read the execution body a
  • the output memory block r13 the execution body b receives the message, and checks whether the memory block b produced by itself has a free memory block available, and finds that there is an available free memory block r23, so the execution body b executes the kernel of the forward operator b
  • the function computing task reads the memory block r13, and the execution body b writes the output tensor result generated by the execution into the memory block r23.
  • execution body a inputs data: execution body a will also check whether it has free blocks to write and execution body A has finished executing. If it finds no, it will wait and not enter the pipeline.
  • the current execution body c sends a message to the downstream execution body C, the downstream execution body C prepares the tensor data to be consumed, the current execution body c sends a message to the upstream execution body b, and the upstream execution body b recycles Consumed tensor data: executor c produces memory block r11, then sends a message to downstream consumer executor B to notify executor B to read the memory block r11 produced by executor C, and executor B receives Memory block r11 finds that memory block r21 is free, so executive body B starts to execute, reads memory block r11, and writes to memory block r21.
  • execution body C sends a message to the upstream producer execution body c to inform execution body c that execution body C has used up the memory block r31 of execution body c, and execution body c has received the memory block r31 returned by execution body C after running out, and checks the memory All consumers of block r31 are used up, so the memory block r31 is reclaimed and marked as a free block.
  • the current execution body c sends a message to the downstream execution body C, the downstream execution body C prepares tensor data to be consumed, the current execution body c sends a message to the upstream execution body b, and the upstream execution body b recycles Consumed tensor data: executor c produces memory block r32, so it sends a message to downstream consumer executor C to notify executor C to read the memory block r32 produced by executor c, and executor C receives The memory block r32 finds that the memory block r12 is free, so the execution body C starts to execute, reads the memory block r32, and writes to the memory block r12.
  • execution body c sends a message to the upstream producer execution body b to inform execution body b that execution body c has used up the memory block r22 of execution body b, and execution body b has received the memory block r22 returned by execution body c after running out, and checks the memory All consumers of block r22 are used up, so the memory block r22 is reclaimed and marked as a free block.
  • the current execution body b sends a message to the downstream execution body c
  • the downstream execution body c prepares tensor data to be consumed
  • the current execution body b sends a message to the upstream execution body a
  • the upstream execution body a recycles Consumed tensor data:
  • Executor b produced memory block r23, and then sent a message to downstream consumer executive c to notify executive c to read the memory block r23 produced by executive b, and executive c received it
  • the memory block r23 finds that the memory block r33 is free, so the execution body c starts to execute, reads the memory block r23, and writes to the memory block r33.
  • execution body b sends a message to the upstream producer execution body a to inform execution body a that execution body b has used up the memory block r13 of execution body a, and execution body a has received the memory block r13 returned by execution body b after running out, and checks the memory All consumers of block r13 are used up, so the memory block r13 is reclaimed and marked as a free block.
  • execution body a inputs data: execution body a will also check whether it has free blocks to write and execution body A has finished executing. If it finds no, it will wait and not enter the pipeline.
  • the current execution body B sends a message to the downstream execution body A, the downstream execution body A prepares the tensor data to be consumed, and the tail execution body A recycles the calculation data by itself.
  • the current execution body B sends it to the upstream execution body C Message, upstream executive body C reclaims the tensor data that has been consumed: executive body B produces memory block r21, and then sends a message to downstream consumer executive body A to notify executive body A to read the memory produced by executive body B Block r21, Executor A receives memory block r21 and finds that it has free memory block r31, so Executor A starts to execute, reads memory block r21, writes memory block r31, and executes A to recycle memory block r31 immediately after execution.
  • execution body B sends a message to the upstream producer execution body C to inform execution body C that execution body B has used up the memory block r11 of execution body C, and execution body C has received the memory block r11 returned by execution body B after running out, and checks the memory All consumers of block r11 are used up, so the memory block r11 is reclaimed and marked as a free block.
  • the current execution body C sends a message to the downstream execution body B, the downstream execution body B prepares the tensor data to be consumed, the current execution body C sends a message to the upstream execution body c, and the upstream execution body C recycles Consumed tensor data:
  • Executor C produces memory block r12, and then sends a message to downstream consumer Executor B to notify Executor B to read the memory block r12 produced by Executor C, and Executor B receives
  • the memory block r12 finds that the memory block r22 is free, so the execution body B starts to execute, reads the memory block r12, and writes to the memory block r22.
  • execution body C sends a message to the upstream producer execution body c to inform execution body c that execution body C has used up the memory block r32 of execution body c, and execution body c has received the memory block r32 returned by execution body C after running out, and checks the memory All consumers of block r32 are used up, so the memory block r32 is reclaimed and marked as a free block.
  • the current execution body c sends a message to the downstream execution body C, the downstream execution body C prepares the tensor data to be consumed, the current execution body c sends a message to the upstream execution body b, and the upstream execution body b recycles Consumed tensor data: executor c produced memory block r33, then sent a message to downstream consumer executor C to notify executor C to read the memory block r33 produced by executor c, and executor C received
  • the memory block r33 finds that the memory block r13 is free, so the execution body C starts to execute, reads the memory block r33, and writes to the memory block r13.
  • execution body c sends a message to the upstream producer execution body b to inform execution body b that execution body c has used up the memory block r23 of execution body b, and execution body b has received the memory block r23 returned by execution body c after running out, and checks the memory All consumers of block r23 are used up, so the memory block r23 is reclaimed and marked as a free block.
  • execution body a execution body b, execution body c, execution body C, execution body B, and execution body A all start to work in parallel.
  • execution body a inputs data: execution body a also checks whether it has free blocks to write and execution body A finishes execution, and if found, execution body a is also executing the fourth batch of input data , write the execution result into the memory free block r11.
  • the current execution body B sends a message to the downstream execution body A, the downstream execution body A prepares tensor data to be consumed, the current execution body B sends a message to the upstream execution body C, and the upstream execution body C recycles Consumed tensor data:
  • Executor B produces memory block r22, then sends a message to downstream consumer Executor A to notify Executor A to read the memory block r22 produced by Executor B, Executor A receives Memory block r22 finds that memory block r32 is free, so executive body A starts to execute, reads memory block r22, writes memory block r32, and executes body A to recycle memory block r32 immediately after execution.
  • execution body B sends a message to the upstream producer execution body C to notify execution body C that execution body B has used up the memory block r12 of execution body C, and execution body C has received the memory block r12 returned by execution body B after running out, and checks the memory All consumers of block r11 are used up, so the memory block r12 is reclaimed and marked as a free block.
  • the current execution body C sends a message to the downstream execution body B, the downstream execution body B prepares tensor data to be consumed, the current execution body C sends a message to the upstream execution body c, and the upstream execution body C recycles Consumed tensor data: the current execution body c sends a message to the downstream execution body C, the downstream execution body C prepares tensor data to be consumed, the current execution body c sends a message to the upstream execution body b, and the upstream execution body b recycles Consumed tensor data: executor c produced memory block r13, then sent a message to downstream consumer executor B to notify executor B to read the memory block r13 produced by executor C, and executor B received Memory block r13 finds that memory block r23 is free, so executive body B starts to execute, reads memory block r13, and writes to memory block r23.
  • execution body C sends a message to the upstream producer execution body c to inform execution body c that execution body C has used up the memory block r33 of execution body c, and execution body c has received the memory block r33 returned by execution body C after running out, and checks the memory All consumers of block r33 are used up, so the memory block r32 is reclaimed and marked as a free block. So executive a, executive b, executive c, and executive C start to work in parallel. Executors B and A are still waiting because there is no data to read.
  • Execution bodies a, b, c, C, B, and A are all working. At this time, a batch of data execution bodies are all executed and the next batch of data is input. Through the design of multiple free memory blocks, the execution body realizes pipeline parallelism.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Biophysics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Databases & Information Systems (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Neurology (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Complex Calculations (AREA)

Abstract

本发明提出了一种面向神经网络模型计算的图执行流水并行方法和装置，提供了一种深度学习训练系统中面向神经网络模型计算的图执行流水并行方法和装置。包括面向神经网络模型计算过程中的图执行流程和各功能模块协同工作的过程。所述面向神经网络模型计算的图执行流水并行方法是根据深度学习框架编译生成的物理计算图创建本机上的图执行体，通过设计为每个图执行体分配多个空闲内存块的方案，实现了整张计算图以流水并行的方式同时参与到不同批次数据的深度学习训练任务中，充分提高了内存的使用率和数据的并行速率。

Description

一种面向神经网络模型计算的图执行流水并行方法和装置
本发明要求于2022年4月27日向中国国家知识产权局提交的申请号为202210447287.7、发明名称为“一种面向神经网络模型计算的图执行流水并行方法和装置”中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及深度学习技术领域，特别涉及一种面向神经网络模型计算的图执行流水并行方法和装置。
背景技术
随着人工智能产业化应用的快速发展，实际应用场景对大模型的需求变得越来越紧迫，机器学习工作负载的结构越来越趋于复杂的大模型，导致用于大模型计算的图的执行成本非常高。已有的用于神经网络模型计算的图执行方法多数基于同步的方法，导致整个图执行系统的资源利用率不高，限制了分布式系统的加速比和吞吐率。
为解决以上问题，本发明提供的用于神经网络模型计算的图执行流水并行方法隔离了各批次的训练数据与不同子图，每批次的训练数据按照1F1B前向后向的方式依次流过前向计算图和反向计算图。本发明可使得每个设备进程上都会有一个批次的数据正在被处理，使所有设备进程保持忙碌，而不会出现管道暂停，整个流水线是比较均衡的。同时能确保以固定周期执行每个子图上的参数更新，也有助于防止同时处理过多小批量并确保模型收敛。
发明内容
本发明的目的在于提供一种面向神经网络模型计算的图执行流水并行方法和装置，以克服现有技术中的不足。
为实现上述目的，本发明提供如下技术方案：
本申请公开了一种面向神经网络模型计算的图执行流水并行方法，所述神经网络模型中设有若干个执行体，所述执行体共有2*N个，所述N为正整数，所述执行体设有若干个内存块；所述方法具体包括如下步骤：
S1、将训练数据分成若干个批次子数据；
S2、若干个批次子数据依次输入神经网络模型中，当第i批次的子数据输入后，第n执行体对第i批次的子数据执行自身核函数计算，并将执行结果写入第n执行体空闲的内存块中；接着输入第i+1批次的子数据；所述i，n均为正整数；
S3、当第i+1批次的子数据输入后，第n执行体对第i+1批次的子数据执行S2操作的同时，将第i批次所在的内存块的地址发送至第n+1执行体；第n+1执行体解析第i批次所在的内存块，得到第n执行体对第i批次的子数据的执行结果，并将第n执行体的执行结果作为第n+1执行体的输入数据，执行自身核函数计算，并将执行结果写入第n+1执行体空闲的内存块中；接着输入第i+2批次的子数据；
S4、当第i+2批次的子数据输入后，第n执行体对第i+2批次的子数据执行S2操作，第n执行体和第n+1执行体对第i+1批次的子数据执行S3的操作；同时第n+1执行体将第i批次所在的内存块的地址发送至第n+2执行体，第n+2执行体解析第i批次所在的内存块，得到第n+1执行体对第i批次的子数据的执行结果，并将第n+1执行体的执行结果作为第n+2执行体的输入数据，执行自身核函数计算，并将执行结果写入第n+2执行体空闲的内存块中；
S5、第n执行体回收发送给第n+1执行体的内存块；
S6、最后一个执行体执行自身核函数计算，并将执行结果写入最后一个执行体的内存块，执行完毕即刻自行回收内存块。
作为优选，执行体在执行自身核函数计算前，执行体会检查自身是否存在空闲的内存块；若存在，则对第i批次的子数据执行自身核函数计算；若不存在，则令第i批次等待存在空闲的内存块。
作为优选，对于第N*n+1批次的子数据，执行体在执行自身核函数计算前，会检查第N*(n-1)+1批次的子数据所在的执行体是否执行完毕，所述n为正整数。
作为优选，步骤S5具体包括如下操作：
S51、第n+1执行体通知第n执行体已消费完发送给第n+1执行体的内存块；
S52、第n执行体回收发送给第n+1执行体的内存块，并将其标记为空闲。
作为优选，还包括执行体的构造，所述执行体的构造具体包括如下子步骤：
S01、创建算子核函数的任务队列：将当前算子核函数的计算任务依次加入当前核函数任务队列；
S02、创建执行体的线程：所述执行体的线程负责从所述核函数任务队列中依次获取当前待处理任务，并提交给线程池；
S03、创建核函数的执行体：根据当前核函数任务和当前线程的上下文信息创建用于算子核函数计算的执行体；并使用执行体运行任务队列中的核函数任务；
S04、创建事件召回队列：将任务执行体处理完的任务添加到事件召回队列中；
S05、创建事件召回队列的线程：所述事件召回队列的线程负责将事件召回队列中已处理的任 务依次取出并返回。
本发明还公开了一种面向神经网络模型计算的图执行装置，包括执行体构造模块和执行体流水并行工作模块，所述执行体构造模块用于执行体的构造，所述执行体流水并行工作模块用于执行上述一种面向神经网络模型计算的图执行流水并行方法。
本发明还公开了一种面向神经网络模型计算的图执行装置，包括存储器和一个或多个处理器，所述存储器中存储有可执行代码，所述一个或多个处理器执行所述可执行代码时，用于上述一种面向神经网络模型计算的图执行流水并行方法。
本发明还公开了一种计算机可读存储介质，其上存储有程序，该程序被处理器执行时，实现上述的一种面向神经网络模型计算的图执行流水并行方法。
本发明的有益效果：
提供了一种面向神经网络模型计算的图执行流水并行方法和装置，根据深度学习框架编译生成的物理计算图创建本机上的图执行体，通过设计为每个图执行体分配多个空闲内存块的方案，实现了整张计算图以流水并行的方式同时参与到不同批次数据的深度学习训练任务中。本发明公开的基于多个空闲张量存储块的图执行体并行执行方法比已有方法更加容易实现大模型的分布式训练。在大规模深度神经网络的分布式应用场景下，本发明对用户的使用门槛较低，并且能够使模型学习到大量分批次流入神经网络的数据的内在关联，从而获得对应场景中的“智能”感知与判断能力。本发明为深度学习相关的算法工程师提供了一套简洁易用的神经网络模型的运行装置，使之能方便地训练深度学习模型。
本发明的特征及优点将通过实施例结合附图进行详细说明。
附图说明
图1面向神经网络模型计算的图执行流水并行方法的架构图；
图2创建管理任务执行体线程模块的流程图；
图3任务执行体流水并行工作模块基本动作；
图4执行体流水并行的执行过程；
图5是本发明一种面向神经网络模型计算的图执行装置的结构示意图。
具体实施方式
为使本发明的目的、技术方案和优点更加清楚明了，下面通过附图及实施例，对本发明进行进一步详细说明。但是应该理解，此处所描述的具体实施例仅仅用以解释本发明，并不用于限制本发明的范围。此外，在以下说明中，省略了对公知结构和技术的描述，以避免不必要地混淆本发明的概念。
如图1所示，所述一种面向神经网络模型计算的图执行流水并行方法的架构图。如图将训练数据分批次喂入神经网络模型，根据深度学习框架编译生成的物理计算图创建本机上的图执行体，为每个图执行体分配多个空闲内存块，使得整张计算图以流水并行的方式同时参与深度学习训练任务中，具体操作如下：
S1、将训练数据分成若干个批次子数据；
S2、若干个批次子数据依次输入神经网络模型中，当第i批次的子数据输入后，第n执行体对第i批次的子数据执行自身核函数计算，并将执行结果写入第n执行体空闲的内存块中；接着输入第i+1批次的子数据；所述i，n均为正整数；
S3、当第i+1批次的子数据输入后，第n执行体对第i+1批次的子数据执行S2操作的同时，将第i批次所在的内存块的地址发送至第n+1执行体；第n+1执行体解析第i批次所在的内存块，得到第n执行体对第i批次的子数据的执行结果，并将第n执行体的执行结果作为第n+1执行体的输入数据，执行自身核函数计算，并将执行结果写入第n+1执行体空闲的内存块中；接着输入第i+2批次的子数据；
S4、当第i+2批次的子数据输入后，第n执行体对第i+2批次的子数据执行S2操作，第n执行体和第n+1执行体对第i+1批次的子数据执行S3的操作；同时第n+1执行体将第i批次所在的内存块的地址发送至第n+2执行体，第n+2执行体解析第i批次所在的内存块，得到第n+1执行体对第i批次的子数据的执行结果，并将第n+1执行体的执行结果作为第n+2执行体的输入数据，执行自身核函数计算，并将执行结果写入第n+2执行体空闲的内存块中；
S5、第n执行体回收发送给第n+1执行体的内存块。
S6、最后一个执行体执行自身核函数计算，并将执行结果写入最后一个执行体的内存块，执行完毕即刻自行回收内存块。
在一种可行的实施例中，执行体在执行自身核函数计算前，执行体会检查自身是否存在空闲的内存块；若存在，则对第i批次的子数据执行自身核函数计算；若不存在，则令第i批次等待存在空闲的内存块。
在一种可行的实施例中，对于第N*n+1批次的子数据，执行体在执行自身核函数计算前，会检查第N*(n-1)+1批次的子数据所在的执行体是否执行完毕，所述n为正整数。
在一种可行的实施例中，步骤S5具体包括如下操作：
S51、第n+1执行体通知第n执行体已消费完发送给第n+1执行体的内存块；
S52、第n执行体回收发送给第n+1执行体的内存块，并将其标记为空闲。
在一种可行的实施例中，还包括执行体的构造，所述执行体的构造具体包括如下子步 骤：
S01、创建算子核函数的任务队列：将当前算子核函数的计算任务依次加入当前核函数任务队列；
S02、创建执行体的线程：所述执行体的线程负责从所述核函数任务队列中依次获取当前待处理任务，并提交给线程池；
S03、创建核函数的执行体：根据当前核函数任务和当前线程的上下文信息创建用于算子核函数计算的执行体；并使用执行体运行任务队列中的核函数任务；
S04、创建事件召回队列：将任务执行体处理完的任务添加到事件召回队列中；
S05、创建事件召回队列的线程：所述事件召回队列的线程负责将事件召回队列中已处理的任务依次取出并返回。
本发明一种面向神经网络模型计算的图执行装置，包括执行体构造模块和执行体流水并行工作模块，
参阅图2，所述执行体构造模块包括如下基本动作：
创建算子核函数的任务队列：将当前算子核函数的计算任务依次加入当前核函数任务队列；创建任务执行体的线程：创建任务执行体的线程。所述任务执行体的线程负责从所述任务队列中依次获取当前待处理任务；当服务器收到一个请求时，将请求提交给线程池，并继续等待其他请求。如果池中有一个可用的线程，它就会被唤醒，请求就会立即得到服务。如果池子里没有可用的线程，任务就会被排队，直到有一个空闲的线程。一旦一个线程完成了它的服务，它就会返回到池子里，等待更多的工作。当提交给线程池的任务可以异步执行时，线程池就能很好地工作。
创建核函数的任务执行体：根据当前核函数任务和当前线程的上下文信息创建用于算子核函数计算的任务执行体。并使用所述任务执行体运行任务队列中的核函数任务。
创建事件召回队列：当处理完上述任务队列中的全部任务执行体时，创建事件召回队列，依次将上述任务执行体处理完的任务添加到事件召回队列中；
创建事件召回队列的线程：创建事件召回队列的线程。所述事件召回队列的线程负责将事件召回队列中已处理的任务依次取出并返回。
参阅图3，所述执行体流水并行工作模块包括如下基本动作：执行体输入数据、当前执行体向下游执行体发送消息、下游执行体准备待消费的张量数据、当前执行体向上游执行体发送消息、上游执行体回收已被消费完的张量数据、尾执行体自行回收计算数据。
执行体输入数据：在t时刻，对于第i批次数据，执行体输入第i批次数据，加载其内 部的算子核函数计算任务，执行核函数计算，生成核函数计算任务的输出张量数据，将执行结果写入内存空闲块；
当前执行体向下游执行体发送消息：在t时刻，对于第i批次数据，将当前执行体生产所得的张量数据存储到空的存储单元中，再将存储单元的地址和当前执行体对应的下游执行体的身份标识号打包成消息，之后发送消息至目标执行体，所述目标执行体就是当前执行体对应的下游执行体；
下游执行体准备待消费的张量数据：在t时刻，对于第i批次数据，下游执行体收到消息，从消息中解析出上述当前执行体生产的张量数据，所述张量数据将作为下游执行体运行其算子核函数时的输入张量，并检查自己生产的内存块是否有空闲内存块可用，如发现有可用的空闲内存块，则下游执行体执行对应算子的核函数计算任务，读取空闲内存块，下游执行体将执行生成的输出张量结果写入内存块；
当前执行体向上游执行体发送消息：在t时刻，对于第i批次数据，执行体给上游的生产者执行体发消息通知上游的生产者执行体执行体消费完了上游的生产者执行体的内存块，上游执行体可以回收其输出张量数据的存储单元；
上游执行体回收已被消费完的数据：在t时刻，对于第i批次数据，上游执行体收到下游执行体发送的回收消息，就开始检查内存块是否都已被所有的消费者执行体消费完毕，如是，则将内存块回收，标记为空闲块；
尾执行体自行回收计算数据：在t时刻，对于第i批次数据，尾执行体执行对应算子的核函数计算任务，写入自己内存块空闲,执行体A执行完毕即刻自行回收内存块。
本发明一种面向神经网络模型计算的图执行装置的实施例可以应用在任意具备数据处理能力的设备上，该任意具备数据处理能力的设备可以为诸如计算机等设备或装置。装置实施例可以通过软件实现，也可以通过硬件或者软硬件结合的方式实现。以软件实现为例，作为一个逻辑意义上的装置，是通过其所在任意具备数据处理能力的设备的处理器将非易失性存储器中对应的计算机程序指令读取到内存中运行形成的。从硬件层面而言，如图5所示，为本发明一种面向神经网络模型计算的图执行装置所在任意具备数据处理能力的设备的一种硬件结构图，除了图5所示的处理器、内存、网络接口、以及非易失性存储器之外，实施例中装置所在的任意具备数据处理能力的设备通常根据该任意具备数据处理能力的设备的实际功能，还可以包括其他硬件，对此不再赘述。上述装置中各个单元的功能和作用的实现过程具体详见上述方法中对应步骤的实现过程，在此不再赘述。
对于装置实施例而言，由于其基本对应于方法实施例，所以相关之处参见方法实施例 的部分说明即可。以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本发明方案的目的。本领域普通技术人员在不付出创造性劳动的情况下，即可以理解并实施。
本发明实施例还提供一种计算机可读存储介质，其上存储有程序，该程序被处理器执行时，实现上述实施例中的一种面向神经网络模型计算的图执行装置。
所述计算机可读存储介质可以是前述任一实施例所述的任意具备数据处理能力的设备的内部存储单元，例如硬盘或内存。所述计算机可读存储介质也可以是任意具备数据处理能力的设备的外部存储设备，例如所述设备上配备的插接式硬盘、智能存储卡(Smart Media Card，SMC)、SD卡、闪存卡(Flash Card)等。进一步的，所述计算机可读存储介质还可以既包括任意具备数据处理能力的设备的内部存储单元也包括外部存储设备。所述计算机可读存储介质用于存储所述计算机程序以及所述任意具备数据处理能力的设备所需的其他程序和数据，还可以用于暂时地存储已经输出或者将要输出的数据。
实施例：
参阅图4，构建物理计算图由正向算子x->正向算子y->正向算子z及反向算子Z->反向算子Y->反向算子X组成，分别根据各个算子创建运行自身核函数的执行体，对应构成执行体a->执行体b->执行体c->执行体C->执行体B->执行体A的执行计算图；启动运行时的执行体，并行运行整张计算图。
T1时刻：
输入第1批次数据，执行体a输入数据：执行体a运行正向算子x的核函数，将运行结果的输出张量写入内存空闲块r11。
执行体b，执行体c，执行体C，执行体B，执行体A由于没有可读的输入张量数据，所以执行体b，c，C，B，A处在等待状态。
步骤3:并行运行整张计算图。
T2时刻：
对于第2批次数据，执行体a输入数据：执行体a还会检查自己是否有空闲块可写，发现有，T2时刻执行体a也在执行第2批次输入数据，将执行结果写入内存空闲块r12。
同时对于第1批次数据，当前执行体a向下游执行体b发送消息、下游执行体b准备待消费的张量数据：执行体a给执行体b发消息通知执行体b读取执行体a产出的内存块r11， 执行体b收到消息，并检查自己生产的内存块b是否有空闲内存块可用，发现有可用的空闲内存块r21，于是T2时刻执行体b执行前向算子b的核函数计算任务，读取内存块r11，执行体b将执行生成的输出张量结果写入内存块r21。
于是执行体a和执行体b就开始并行工作。执行体c，C，B，A由于没有数据可读，仍在等待。
T3时刻：
对于第3批次数据，执行体a输入数据：执行体a还会检查自己是否有空闲块可写，发现有，则执行体a也在执行第3批次输入数据，将执行结果写入内存空闲块r13。
同时对于第1批次数据，当前执行体b向下游执行体c发送消息、下游执行体c准备待消费的张量数据、当前执行体b向上游执行体a发送消息、上游执行体a回收已被消费完的张量数据：执行体b生产出了内存块r21，于是给下游的消费者执行体c发消息通知执行体c读取执行体b产出的内存块r21，执行体c收到内存块r21，发现自己有内存块r31空闲，于是执行体c开始执行，读内存块r21,写入内存块r31。同时执行体b给上游的生产者执行体a发消息通知执行体a执行体b用完了执行体a的内存块r11，执行体a收到了执行体b用完还回来的内存块r11，检查内存块r11所有的消费者都用完了，于是将内存块r11回收，标记为空闲块。
同时对于第2批次数据，当前执行体a向下游执行体b发送消息、下游执行体b准备待消费的张量数据：执行体a给执行体b发消息通知执行体b读取执行体a产出的内存块r12，执行体b收到消息，并检查自己生产的内存块b是否有空闲内存块可用，发现有可用的空闲内存块r22，于是执行体b执行前向算子b的核函数计算任务，读取内存块r12，执行体b将执行生成的输出张量结果写入内存块r22。
于是执行体a，执行体b，执行体c就开始并行工作。执行体C，B，A由于没有数据可读，仍在等待。
T4时刻：
对于第4批次数据，执行体a输入数据：执行体a还会同时检查自己是否有空闲块可写和执行体A执行完毕，发现没有，则等待不进入流水线。
同时对于第1批次数据，当前执行体c向下游执行体C发送消息、下游执行体C准备待消费的张量数据、当前执行体c向上游执行体b发送消息、上游执行体b回收已被消费完的张量数据：执行体c生产出了内存块r31，于是给下游的消费者执行体C发消息通知执行体C读取执行体c产出的内存块r31，执行体C收到内存块r31，发现自己有内存块r11空闲， 于是执行体C开始执行，读内存块r31,写入内存块r11。同时执行体c给上游的生产者执行体b发消息通知执行体b执行体c用完了执行体b的内存块r21，执行体b收到了执行体c用完还回来的内存块r21，检查内存块r21所有的消费者都用完了，于是将内存块r21回收，标记为空闲块。
同时对于第2批次数据，当前执行体b向下游执行体c发送消息、下游执行体c准备待消费的张量数据、当前执行体b向上游执行体a发送消息、上游执行体a回收已被消费完的张量数据：执行体b生产出了内存块r22，于是给下游的消费者执行体c发消息通知执行体c读取执行体b产出的内存块r22，执行体c收到内存块r22，发现自己有内存块r32空闲，于是执行体c开始执行，读内存块r22,写入内存块r32。同时执行体b给上游的生产者执行体a发消息通知执行体a执行体b用完了执行体a的内存块r12，执行体a收到了执行体b用完还回来的内存块r12，检查内存块r12所有的消费者都用完了，于是将内存块r12回收，标记为空闲块。
同时对于第3批次数据，当前执行体a向下游执行体b发送消息、下游执行体b准备待消费的张量数据：执行体a给执行体b发消息通知执行体b读取执行体a产出的内存块r13，执行体b收到消息，并检查自己生产的内存块b是否有空闲内存块可用，发现有可用的空闲内存块r23，于是执行体b执行前向算子b的核函数计算任务，读取内存块r13，执行体b将执行生成的输出张量结果写入内存块r23。
于是执行体a，执行体b，执行体c，执行体C就开始并行工作。执行体B，A由于没有数据可读，仍在等待。
T5时刻：
对于第4批次数据，执行体a输入数据：执行体a还会同时检查自己是否有空闲块可写和执行体A执行完毕，发现没有，则等待不进入流水线。
同时对于第1批次数据，当前执行体c向下游执行体C发送消息、下游执行体C准备待消费的张量数据、当前执行体c向上游执行体b发送消息、上游执行体b回收已被消费完的张量数据：执行体c生产出了内存块r11，于是给下游的消费者执行体B发消息通知执行体B读取执行体C产出的内存块r11，执行体B收到内存块r11，发现自己有内存块r21空闲，于是执行体B开始执行，读内存块r11,写入内存块r21。同时执行体C给上游的生产者执行体c发消息通知执行体c执行体C用完了执行体c的内存块r31，执行体c收到了执行体C用完还回来的内存块r31，检查内存块r31所有的消费者都用完了，于是将内存块r31回收，标记为空闲块。
同时对于第2批次数据，当前执行体c向下游执行体C发送消息、下游执行体C准备待消费的张量数据、当前执行体c向上游执行体b发送消息、上游执行体b回收已被消费完的张量数据：执行体c生产出了内存块r32，于是给下游的消费者执行体C发消息通知执行体C读取执行体c产出的内存块r32，执行体C收到内存块r32，发现自己有内存块r12空闲，于是执行体C开始执行，读内存块r32,写入内存块r12。同时执行体c给上游的生产者执行体b发消息通知执行体b执行体c用完了执行体b的内存块r22，执行体b收到了执行体c用完还回来的内存块r22，检查内存块r22所有的消费者都用完了，于是将内存块r22回收，标记为空闲块。
同时对于第3批次数据，当前执行体b向下游执行体c发送消息、下游执行体c准备待消费的张量数据、当前执行体b向上游执行体a发送消息、上游执行体a回收已被消费完的张量数据：执行体b生产出了内存块r23，于是给下游的消费者执行体c发消息通知执行体c读取执行体b产出的内存块r23，执行体c收到内存块r23，发现自己有内存块r33空闲，于是执行体c开始执行，读内存块r23,写入内存块r33。同时执行体b给上游的生产者执行体a发消息通知执行体a执行体b用完了执行体a的内存块r13，执行体a收到了执行体b用完还回来的内存块r13，检查内存块r13所有的消费者都用完了，于是将内存块r13回收，标记为空闲块。
于是执行体a，执行体b，执行体c，执行体C,执行体B就开始并行工作。执行体A由于没有数据可读，仍在等待。
T6时刻：
对于第4批次数据，执行体a输入数据：执行体a还会同时检查自己是否有空闲块可写和执行体A执行完毕，发现没有，则等待不进入流水线。
同时对于第1批次数据，当前执行体B向下游执行体A发送消息、下游执行体A准备待消费的张量数据、尾执行体A自行回收计算数据当前执行体B向上游执行体C发送消息、上游执行体C回收已被消费完的张量数据：执行体B生产出了内存块r21，于是给下游的消费者执行体A发消息通知执行体A读取执行体B产出的内存块r21，执行体A收到内存块r21，发现自己有内存块r31空闲，于是执行体A开始执行，读内存块r21,写入内存块r31,执行体A执行完毕即刻自行回收内存块r31。同时执行体B给上游的生产者执行体C发消息通知执行体C执行体B用完了执行体C的内存块r11，执行体C收到了执行体B用完还回来的内存块r11，检查内存块r11所有的消费者都用完了，于是将内存块r11回收，标记为空闲块。
同时对于第2批次数据，当前执行体C向下游执行体B发送消息、下游执行体B准 备待消费的张量数据、当前执行体C向上游执行体c发送消息、上游执行体c回收已被消费完的张量数据：执行体C生产出了内存块r12，于是给下游的消费者执行体B发消息通知执行体B读取执行体C产出的内存块r12，执行体B收到内存块r12，发现自己有内存块r22空闲，于是执行体B开始执行，读内存块r12,写入内存块r22。同时执行体C给上游的生产者执行体c发消息通知执行体c执行体C用完了执行体c的内存块r32，执行体c收到了执行体C用完还回来的内存块r32，检查内存块r32所有的消费者都用完了，于是将内存块r32回收，标记为空闲块。
同时对于第3批次数据，当前执行体c向下游执行体C发送消息、下游执行体C准备待消费的张量数据、当前执行体c向上游执行体b发送消息、上游执行体b回收已被消费完的张量数据：执行体c生产出了内存块r33，于是给下游的消费者执行体C发消息通知执行体C读取执行体c产出的内存块r33，执行体C收到内存块r33，发现自己有内存块r13空闲，于是执行体C开始执行，读内存块r33,写入内存块r13。同时执行体c给上游的生产者执行体b发消息通知执行体b执行体c用完了执行体b的内存块r23，执行体b收到了执行体c用完还回来的内存块r23，检查内存块r23所有的消费者都用完了，于是将内存块r23回收，标记为空闲块。
于是至此执行体a，执行体b，执行体c，执行体C，执行体B，执行体A全部开始并行工作。
T7时刻：
对于第4批次数据，执行体a输入数据：执行体a还会同时检查自己是否有空闲块可写和执行体A执行完毕，发现有，则执行体a也在执行第4批次输入数据，将执行结果写入内存空闲块r11。
同时对于第1批次数据，全部执行体执行完毕。
同时对于第2批次数据，当前执行体B向下游执行体A发送消息、下游执行体A准备待消费的张量数据、当前执行体B向上游执行体C发送消息、上游执行体C回收已被消费完的张量数据：执行体B生产出了内存块r22，于是给下游的消费者执行体A发消息通知执行体A读取执行体B产出的内存块r22，执行体A收到内存块r22，发现自己有内存块r32空闲，于是执行体A开始执行，读内存块r22,写入内存块r32,执行体A执行完毕即刻自行回收内存块r32。同时执行体B给上游的生产者执行体C发消息通知执行体C执行体B用完了执行体C的内存块r12，执行体C收到了执行体B用完还回来的内存块r12，检查内存块r11所有的消费者都用完了，于是将内存块r12回收，标记为空闲块。
同时对于第3批次数据，当前执行体C向下游执行体B发送消息、下游执行体B准备待消费的张量数据、当前执行体C向上游执行体c发送消息、上游执行体c回收已被消费完的张量数据：当前执行体c向下游执行体C发送消息、下游执行体C准备待消费的张量数据、当前执行体c向上游执行体b发送消息、上游执行体b回收已被消费完的张量数据：执行体c生产出了内存块r13，于是给下游的消费者执行体B发消息通知执行体B读取执行体C产出的内存块r13，执行体B收到内存块r13，发现自己有内存块r23空闲，于是执行体B开始执行，读内存块r13,写入内存块r23。同时执行体C给上游的生产者执行体c发消息通知执行体c执行体C用完了执行体c的内存块r33，执行体c收到了执行体C用完还回来的内存块r33，检查内存块r33所有的消费者都用完了，于是将内存块r32回收，标记为空闲块。于是执行体a，执行体b，执行体c，执行体C就开始并行工作。执行体B，A由于没有数据可读，仍在等待。
T8时刻：
执行体a，b，c，C，B，A都在工作，此时一个批次数据执行体全部执行完毕并完成输入下一组批次数据。通过多个空闲内存块的设计，执行体就实现了流水并行。
以上所述仅为本发明的较佳实施例而已，并不用以限制本发明，凡在本发明的精神和原则之内所作的任何修改、等同替换或改进等，均应包含在本发明的保护范围之内。

Claims (8)

  1. 一种面向神经网络模型计算的图执行流水并行方法，其特征在于，所述神经网络模型中设有若干个执行体，所述执行体共有2*N个，所述N为正整数，所述执行体设有若干个内存块；所述方法具体包括如下步骤：
    S1、将训练数据分成若干个批次子数据；
    S2、若干个批次子数据依次输入神经网络模型中，当第i批次的子数据输入后，第n执行体对第i批次的子数据执行自身核函数计算，并将执行结果写入第n执行体空闲的内存块中；接着输入第i+1批次的子数据；所述i，n均为正整数；
    S3、当第i+1批次的子数据输入后，第n执行体对第i+1批次的子数据执行S2操作的同时，将第i批次所在的内存块的地址发送至第n+1执行体；第n+1执行体解析第i批次所在的内存块，得到第n执行体对第i批次的子数据的执行结果，并将第n执行体的执行结果作为第n+1执行体的输入数据，执行自身核函数计算，并将执行结果写入第n+1执行体空闲的内存块中；接着输入第i+2批次的子数据；
    S4、当第i+2批次的子数据输入后，第n执行体对第i+2批次的子数据执行S2操作，第n执行体和第n+1执行体对第i+1批次的子数据执行S3的操作；同时第n+1执行体将第i批次所在的内存块的地址发送至第n+2执行体，第n+2执行体解析第i批次所在的内存块，得到第n+1执行体对第i批次的子数据的执行结果，并将第n+1执行体的执行结果作为第n+2执行体的输入数据，执行自身核函数计算，并将执行结果写入第n+2执行体空闲的内存块中；
    S5、第n执行体回收发送给第n+1执行体的内存块；
    S6、最后一个执行体执行自身核函数计算，并将执行结果写入最后一个执行体的内存块，执行完毕即刻自行回收内存块。
  2. 如权利要求1所述的一种面向神经网络模型计算的图执行流水并行方法，其特征在于：执行体在执行自身核函数计算前，执行体会检查自身是否存在空闲的内存块；若存在，则对第i批次的子数据执行自身核函数计算；若不存在，则令第i批次等待存在空闲的内存块。
  3. 如权利要求2所述的一种面向神经网络模型计算的图执行流水并行方法，其特征在于，对于第N*n+1批次的子数据，执行体在执行自身核函数计算前，会检查第N*(n-1)+1批次的子数据所在的执行体是否执行完毕，所述n为正整数。
  4. 如权利要求1所述的一种面向神经网络模型计算的图执行流水并行方法，其特征在于：步骤S5具体包括如下操作：
    S51、第n+1执行体通知第n执行体已消费完发送给第n+1执行体的内存块；
    S52、第n执行体回收发送给第n+1执行体的内存块，并将其标记为空闲。
  5. 如权利要求1所述的一种面向神经网络模型计算的图执行流水并行方法，其特征在于：还包括执行体的构造，所述执行体的构造具体包括如下子步骤：
    S01、创建算子核函数的任务队列：将当前算子核函数的计算任务依次加入当前核函数任务队列；
    S02、创建执行体的线程：所述执行体的线程负责从所述核函数任务队列中依次获取当前待处理任务，并提交给线程池；
    S03、创建核函数的执行体：根据当前核函数任务和当前线程的上下文信息创建用于算子核函数计算的执行体；并使用执行体运行任务队列中的核函数任务；
    S04、创建事件召回队列：将任务执行体处理完的任务添加到事件召回队列中；
    S05、创建事件召回队列的线程：所述事件召回队列的线程负责将事件召回队列中已处理的任务依次取出并返回。
  6. 一种面向神经网络模型计算的图执行装置，其特征在于：包括执行体构造模块和执行体流水并行工作模块，所述执行体构造模块用于执行体的构造，所述执行体流水并行工作模块用于执行如权利要求1-4任一项所述的一种面向神经网络模型计算的图执行流水并行方法。
  7. 一种面向神经网络模型计算的图执行装置，其特征在于：包括存储器和一个或多个处理器，所述存储器中存储有可执行代码，所述一个或多个处理器执行所述可执行代码时，用于实现权利要求1-5任一项所述的一种面向神经网络模型计算的图执行流水并行方法。
  8. 一种计算机可读存储介质，其特征在于：其上存储有程序，该程序被处理器执行时,实现权利要求1-5任一项所述的一种面向神经网络模型计算的图执行流水并行方法。
PCT/CN2022/092481 2022-04-27 2022-05-12 一种面向神经网络模型计算的图执行流水并行方法和装置 Ceased WO2023082575A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/838,342 US12468921B2 (en) 2022-04-27 2022-06-13 Pipelining and parallelizing graph execution method for neural network model computation and apparatus thereof

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210447287.7A CN114548383A (zh) 2022-04-27 2022-04-27 一种面向神经网络模型计算的图执行流水并行方法和装置
CN202210447287.7 2022-04-27

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/838,342 Continuation US12468921B2 (en) 2022-04-27 2022-06-13 Pipelining and parallelizing graph execution method for neural network model computation and apparatus thereof

Publications (1)

Publication Number Publication Date
WO2023082575A1 true WO2023082575A1 (zh) 2023-05-19

Family

ID=81667147

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/092481 Ceased WO2023082575A1 (zh) 2022-04-27 2022-05-12 一种面向神经网络模型计算的图执行流水并行方法和装置

Country Status (3)

Country Link
US (1) US12468921B2 (zh)
CN (1) CN114548383A (zh)
WO (1) WO2023082575A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117032954A (zh) * 2023-07-17 2023-11-10 北京泛睿科技合伙企业(有限合伙) 针对终端训练模型的内存优化方法、系统、设备及介质

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114548383A (zh) 2022-04-27 2022-05-27 之江实验室 一种面向神经网络模型计算的图执行流水并行方法和装置
CN115408157A (zh) * 2022-08-31 2022-11-29 北京中科睿信科技有限公司 一种基于线程池的模型并行化数据仿真方法
CN115688893B (zh) * 2022-10-19 2024-09-03 北京百度网讯科技有限公司 内存调度方法及装置、电子设备和存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190362227A1 (en) * 2018-05-23 2019-11-28 Microsoft Technology Licensing, Llc Highly performant pipeline parallel deep neural network training
CN112884086A (zh) * 2021-04-06 2021-06-01 北京百度网讯科技有限公司 模型训练方法、装置、设备、存储介质以及程序产品
CN114139702A (zh) * 2021-11-25 2022-03-04 广东浪潮智慧计算技术有限公司 一种深度神经网络训练方法、系统、装置、设备及介质
CN114186687A (zh) * 2022-02-17 2022-03-15 之江实验室 一种面向神经网络模型计算的中间表示方法和装置
CN114237918A (zh) * 2022-02-28 2022-03-25 之江实验室 一种面向神经网络模型计算的图执行方法和装置
CN114548383A (zh) * 2022-04-27 2022-05-27 之江实验室 一种面向神经网络模型计算的图执行流水并行方法和装置

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022104223A1 (en) * 2020-11-16 2022-05-19 Lightmatter, Inc. Parallelization and pipelining strategies for an efficient analog neural network accelerator
US12321855B2 (en) * 2021-04-02 2025-06-03 Tenstorrent Ai Ulc Graph execution using access request response dynamic batch assembly
US20230004871A1 (en) * 2021-06-30 2023-01-05 Advanced Micro Devices, Inc. Machine learning cluster pipeline fusion
US20230083345A1 (en) * 2021-09-07 2023-03-16 Nvidia Corporation Multi-architecture execution graphs
US20230084951A1 (en) * 2021-09-16 2023-03-16 Nvidia Corporation Synchronizing graph execution
US20230169408A1 (en) * 2021-11-30 2023-06-01 International Business Machines Corporation Annotation of a Machine Learning Pipeline with Operational Semantics

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190362227A1 (en) * 2018-05-23 2019-11-28 Microsoft Technology Licensing, Llc Highly performant pipeline parallel deep neural network training
CN112884086A (zh) * 2021-04-06 2021-06-01 北京百度网讯科技有限公司 模型训练方法、装置、设备、存储介质以及程序产品
CN114139702A (zh) * 2021-11-25 2022-03-04 广东浪潮智慧计算技术有限公司 一种深度神经网络训练方法、系统、装置、设备及介质
CN114186687A (zh) * 2022-02-17 2022-03-15 之江实验室 一种面向神经网络模型计算的中间表示方法和装置
CN114237918A (zh) * 2022-02-28 2022-03-25 之江实验室 一种面向神经网络模型计算的图执行方法和装置
CN114548383A (zh) * 2022-04-27 2022-05-27 之江实验室 一种面向神经网络模型计算的图执行流水并行方法和装置

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117032954A (zh) * 2023-07-17 2023-11-10 北京泛睿科技合伙企业(有限合伙) 针对终端训练模型的内存优化方法、系统、设备及介质
CN117032954B (zh) * 2023-07-17 2024-04-26 北京泛睿科技合伙企业(有限合伙) 针对终端训练模型的内存优化方法、系统、设备及介质

Also Published As

Publication number Publication date
CN114548383A (zh) 2022-05-27
US12468921B2 (en) 2025-11-11
US20230351145A1 (en) 2023-11-02

Similar Documents

Publication Publication Date Title
US20230236888A1 (en) Memory allocation method, related device, and computer-readable storage medium
US11609792B2 (en) Maximizing resource utilization of neural network computing system
Chen et al. Accelerating mapreduce on a coupled cpu-gpu architecture
US11941514B2 (en) Method for execution of computational graph in neural network model and apparatus thereof
US12468921B2 (en) Pipelining and parallelizing graph execution method for neural network model computation and apparatus thereof
CN112711478B (zh) 基于神经网络的任务处理方法、装置、服务器和存储介质
CN114217966A (zh) 基于资源调整的深度学习模型动态批处理调度方法和系统
CN104714785A (zh) 任务调度装置、方法及并行处理数据的设备
CN110866610A (zh) 一种深度学习模型分布式运算的方法及装置
CN112162854A (zh) 一种cpu-gpu间计算任务调度方法、系统及介质
CN111708639A (zh) 任务调度系统及方法、存储介质及电子设备
CN114637536B (zh) 任务处理方法、计算协处理器、芯片及计算机设备
CN111190741A (zh) 基于深度学习节点计算的调度方法、设备及存储介质
US20180046474A1 (en) Method for executing child kernels invoked on device side utilizing dynamic kernel consolidation and related non-transitory computer readable medium
WO2021008260A1 (zh) 数据执行体及其数据处理方法
CN118313458A (zh) 数据处理方法、数据处理器、电子设备、存储介质
CN112860396B (zh) 一种基于分布式深度学习的gpu调度方法及系统
CN118535332A (zh) 基于算子选择与细粒度融合的深度学习推理方法及装置
CN109840151B (zh) 一种用于多核处理器的负载均衡方法和装置
CN106155799B (zh) 基于遗传算法的Codelet调度方法
CN112395062A (zh) 任务处理方法、装置、设备及计算机可读存储介质
CN116185615A (zh) 减少gpu和cpu交互开销的方法及系统
CN116301874A (zh) 代码编译方法、电子设备及存储介质
CN119847790A (zh) 一种异步并行执行计算任务的方法及装置
CN117492973A (zh) 高效任务分配

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22891386

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22891386

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 02.05.2025)

122 Ep: pct application non-entry in european phase

Ref document number: 22891386

Country of ref document: EP

Kind code of ref document: A1