WO2023201893A1 - 一种计算任务调度方法、装置、电子设备及可读存储介质 - Google Patents

一种计算任务调度方法、装置、电子设备及可读存储介质 Download PDF

Info

Publication number
WO2023201893A1
WO2023201893A1 PCT/CN2022/102769 CN2022102769W WO2023201893A1 WO 2023201893 A1 WO2023201893 A1 WO 2023201893A1 CN 2022102769 W CN2022102769 W CN 2022102769W WO 2023201893 A1 WO2023201893 A1 WO 2023201893A1
Authority
WO
WIPO (PCT)
Prior art keywords
task
network
target
network layer
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2022/102769
Other languages
English (en)
French (fr)
Inventor
贾兆荣
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Suzhou Wave Intelligent Technology Co Ltd
Original Assignee
Suzhou Wave Intelligent Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suzhou Wave Intelligent Technology Co Ltd filed Critical Suzhou Wave Intelligent Technology Co Ltd
Priority to US18/725,590 priority Critical patent/US12498971B2/en
Publication of WO2023201893A1 publication Critical patent/WO2023201893A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/48—Indexing scheme relating to G06F9/48
    • G06F2209/486—Scheduler internals

Definitions

  • the present application relates to the field of neural network technology, and in particular to a computing task scheduling method, a computing task scheduling device, electronic equipment and a computer-readable storage medium.
  • the artificial intelligence acceleration engine is an accelerated computing component specially designed for deep learning algorithms.
  • the convolutional neural network structure Convolutional Neural Networks, CNN commonly used in deep learning algorithms usually contains multiple convolution combination layers. Each convolution The combined layer also includes small layers such as convolution, activation, and pooling. Convolutional combination layers are used to extract different features in images.
  • a CNN usually contains dozens or even hundreds of convolutional combination layers.
  • This application provides a computing task scheduling method, which is applied to the task scheduler.
  • the method includes:
  • network description data is used to describe the network layer tasks corresponding to each network layer in the target neural network
  • each functional module on the artificial intelligence accelerated computing component corresponds to at least two register groups
  • the network describes the data generation process, including:
  • generate network description data based on the target neural network and hardware architecture description data including:
  • the optimization graph is compiled and processed to obtain network description data.
  • generate network description data based on the target neural network and hardware architecture description data including:
  • detect free register groups on the artificial intelligence accelerated computing component including:
  • detect free register groups on the artificial intelligence accelerated computing component including:
  • the free register group corresponding to the second functional module is determined; wherein the free register group is one of the two ping-pong register groups corresponding to the second functional module.
  • a target register configuration based on dependencies, network description data, and free register group types including:
  • the target register configuration is generated using the candidate descriptor data and the hardware architecture description data corresponding to the artificial intelligence accelerated computing component.
  • determine the next task of the target network layer task completion notification corresponding to the target network layer task based on the dependency relationship including:
  • the target subsequent task whose execution wait minus one is zero is determined as the next task.
  • each target function module is enabled, so that the enabled target function module performs data calculation preparation operations based on the corresponding non-idle register group.
  • This application also provides a computing task scheduling device, which is applied to the task scheduler.
  • the device includes:
  • the network description acquisition module is used to obtain network description data; the network description data is used to describe the network layer tasks corresponding to each network layer in the target neural network;
  • the dependency determination module is used to determine the dependencies between tasks in each network layer based on network description data
  • An idle detection module is used to detect idle register groups on the artificial intelligence accelerated computing component; each functional module on the artificial intelligence accelerated computing component corresponds to at least two register groups;
  • a register configuration module used to generate a target register configuration based on dependencies, network description data and free register group types, and write the target register configuration to the free register group;
  • a startup module configured to respond to detecting the target network layer task completion notification, determine the next task of the target network layer task corresponding to the target network layer task completion notification based on the dependency relationship, and correspond the next task on the artificial intelligence accelerated computing component
  • the first functional module is started to perform the next task.
  • This application also provides an electronic device, including a memory and one or more processors.
  • Computer-readable instructions are stored in the memory.
  • the one or more processors cause Execute the steps of the above computing task scheduling method.
  • the present application also provides one or more non-volatile computer-readable storage media storing computer-readable instructions, wherein the computer-readable instructions, when executed by one or more processors, cause one or more processes to The processor executes the steps of the calculation task scheduling method.
  • Figure 1 is a schematic diagram of the hardware architecture of an AI engine provided by one or more embodiments of the present application;
  • Figure 2 is a flow chart of a computing task scheduling method provided by one or more embodiments of the present application.
  • Figure 3 is a hardware architecture description data structure diagram of an AI acceleration device provided by one or more embodiments of the present application.
  • Figure 4 is a data structure diagram of an AI engine provided by one or more embodiments of the present application.
  • Figure 5 is a data structure diagram of engine task data provided by one or more embodiments of the present application.
  • Figure 6 is a data structure diagram of engine configuration data provided by one or more embodiments of the present application.
  • Figure 7 is a flow chart of a network description data generation process provided by one or more embodiments of the present application.
  • Figure 8 is a data structure diagram of network description data provided by one or more embodiments of the present application.
  • Figure 9 is a data structure diagram of functional module description data provided by one or more embodiments of the present application.
  • Figure 10 is a data structure diagram of dependent description data provided by one or more embodiments of the present application.
  • Figure 11 is a data structure diagram of functional module description data provided by one or more embodiments of the present application.
  • Figure 12A is the first part of a data structure diagram of register description data provided by one or more embodiments of the present application.
  • Figure 12B is the second part of a data structure diagram of register description data provided by one or more embodiments of the present application.
  • Figure 13A is a data structure diagram of an input and output mode description data provided by one or more embodiments of the present application.
  • Figure 13B is a specific data structure diagram of convolution data description data provided by one or more embodiments of the present application.
  • Figure 13C is a specific data structure diagram of activation data description data provided by one or more embodiments of the present application.
  • Figure 14 is a schematic diagram of a convolutional neural network inference process provided by one or more embodiments of the present application.
  • Figure 15 is a specific task scheduling flow chart provided by one or more embodiments of the present application.
  • Figure 16 is a specific interrupt processing flow chart provided by one or more embodiments of the present application.
  • Figure 17 is a schematic structural diagram of a computing task scheduling device provided by one or more embodiments of the present application.
  • Figure 18 is a schematic structural diagram of an electronic device provided by one or more embodiments of the present application.
  • Artificial intelligence acceleration engine (or artificial intelligence acceleration component, AI Engine) is an accelerated computing component specially designed for deep learning algorithms.
  • the convolutional neural network structure commonly used in deep learning algorithms usually contains multiple convolution combination layers, each volume
  • the product combination layer also includes small layers such as convolution, activation, and pooling.
  • Convolutional combination layers are used to extract different features in images.
  • a CNN usually contains dozens or even hundreds of convolutional combination layers.
  • the general structure of a task-based AI acceleration engine is shown in Figure 1, including: interface bus AXI (Advanced eXtensible Interface) and APB (Advanced Peripheral Bus, peripheral bus), reading and writing external The DMA (Direct Memory Access, memory direct access controller) module of the storage medium, the local cache buffer module, the Data Loader module for parallel data loading, status monitoring (i.e. Status Collector) and interrupt controller (i.e. Interrupt Controller), which are responsible for calculations MAC unit (or called multiplication and addition calculation module, a kind of functional module), Activation unit (also called activation processing module, a kind of functional module), Pooling unit (also called pooling processing module, a kind of functional module) kind). In addition, it can also include data reshape module, normalization module, etc.
  • AXI Advanced eXtensible Interface
  • APB Advanced Peripheral Bus, peripheral bus
  • the DMA Direct Memory Access, memory direct access controller
  • the DMA Direct Memory Access, memory direct access controller
  • Registers i.e. registers, or register groups
  • Controllers can exist in many ways.
  • Figure 1 shows a centralized controller.
  • it can also be a decentralized controller, that is, each controller is located inside each module.
  • These modules are all mounted on the APB or connected in a daisy chain. .
  • the Status Collector monitors the information of each module, such as buffer free space size, operating status of each module, number of read and write data, blocking time or idle time, etc., and provides Interrupt Controller with interrupt pending of each module, such as DMA reading data completion Interrupt pending, MAC Unit calculation completes interrupt pending, Activation activation completes interrupt pending, Pooling completes interrupt pending, etc.
  • the interrupt pending register is set in the Interrupt Controller.
  • Each bit of the pending register maps to represent the pending signal of each functional module.
  • the interrupt mask register and the interrupt enable register are also set. When the interrupt is enabled and the interrupt is not masked, when the pending bit is 1, the interrupt signal Intr is generated. Specifically, which module of the AI engine completed the task and generated an interrupt, which can be determined by querying the pending register. It can be seen that the process of calling the AI acceleration engine to accelerate calculations is a process of constantly configuring registers and waiting for interrupts.
  • a CNN network has dozens or hundreds of combination layers. If the calculation amount of a single combination layer exceeds the computing power of the AI acceleration engine, a single combination layer needs to be split into multiple sub-combination layers. Each combination layer may include load data, multiplication and addition operations, activation, pooling, normalization, fusion and other operations. Multiple modules need to be called to complete the computing tasks of a combination layer. It can be seen that to complete a CNN inference task, each module needs to complete hundreds or thousands of tasks, and each module needs to cooperate with each other.
  • each functional module needs to be calculated in parallel as much as possible, and each computing task is processed in the form of an assembly line (i.e., pipeline).
  • assembly line i.e., pipeline
  • this application designs a task scheduler, which can be used to schedule tasks on the AI engine so that the AI engine can calculate efficiently.
  • This embodiment does not limit the specific form of the task scheduler.
  • the role of the task scheduler can be played by a processor in an AI device (artificial intelligence accelerated computing device), and the AI device can have any number of processors.
  • AI engine each AI engine can correspond to a processor, or multiple AI engines can correspond to a processor.
  • the task scheduler can have the following characteristics:
  • the combination and pipeline of each module within the AI engine The combination and pipeline order of each module in each layer of CNN are different.
  • the functions implemented by each module are different, and the internal registers that need to be configured are also different.
  • the configuration methods of different functional modules are also different. Therefore, the task scheduler should be able to automatically switch the modules that need to be configured according to any CNN network structure. It also needs to implement different configuration methods and switch synchronously with the modules.
  • the configuration of a certain functional module is usually configured during the operation of the previous layer of the CNN or other functional modules. When it is the module's turn to execute Enable again. That is, the configuration logic of the module is not related to the enable logic of the module.
  • two layers of CNN are both a combination of convolution + activation + pooling.
  • the current layer performs operations, it configures the configuration required for the next layer's operation. This mechanism allows the operation and configuration to be performed at the same time, saving configuration time. After the operation of the current layer is completed, the operation of the next layer starts directly, so that the module runs continuously and improves the module operation efficiency.
  • each functional module in this application corresponds to at least two sets of configuration registers, that is, two register sets.
  • each module is enabled in sequence.
  • the pooling module needs to be enabled first, then the activation module, and finally the convolution module. That is, the modules to be executed later are enabled first, so that subsequent modules can be enabled in advance. Be prepared to receive the operation results of the previous modules, otherwise data loss will occur.
  • module configuration and module enablement are two sets of unrelated logic. This mechanism effectively implements the pipeline between modules, and the task scheduler should be able to support this kind of scheduling.
  • the essence of the program is logic and data. Therefore, the task scheduler should be able to use data to describe each process node involved in the task scheduling process, such as input/output data, CNN structure, each functional module of the AI engine, and AI Describe the tasks of the engine, the operations of each step, the connections between the modules of the AI engine, and the enabling logic of the modules.
  • the working process of the task scheduler can be summarized as: configuration register-->module operation-->execution is completed and an interrupt is generated.
  • the AI engine uses each functional module to work according to the control of the task scheduler.
  • the process is: read task data-->initialize all available modules-->update module enable sequence-->enable sequence is 0.
  • specific steps will be described subsequently.
  • Figure 2 is a flow chart of a computing task scheduling method provided by an embodiment of the present application.
  • the method includes:
  • the network description data is used to describe the network layer tasks corresponding to each network layer in the target neural network.
  • the network layer task refers to the task of performing inference calculations on the network layer.
  • the network description data is affected by the network structure of the neural network and the hardware structure of the AI engine.
  • the network description data may be obtained from other electronic devices or may be generated by an electronic device with a task scheduler.
  • the hardware architecture of the artificial intelligence accelerated computing component is first determined, and based on the hardware architecture, corresponding hardware architecture description data is generated.
  • generation rules may be preset, and after the hardware architecture is obtained, the hardware architecture description data is generated based on the generation rules.
  • preset hardware architecture description data corresponding to various hardware architectures may be pre-stored, and after the hardware architecture is determined, the corresponding hardware architecture description data is determined therefrom.
  • Figure 3 shows the hardware architecture description data of an AI Device, which includes the device_config configuration data required for initialization, the pointer AI_engine_ptr describing the AI acceleration engine configured on the device device, and the task description Device_task data processed by the device.
  • Device task Device_task is usually a combination of a series of subtasks. Each subtask will have input data, output data and other parameter data. These data are written to the slave (i.e. AI Device) when the host (i.e. master) issues the task. In the accessed storage space memory, the slave needs to obtain the address of these task data when processing these tasks. These addresses are combined into address_list. The number of data addresses in address_list is address_num.
  • AI_device_ptr refers to the device to which the task belongs, and is used to identify the executor of the task (i.e. slave).
  • Device configuration Device_config is the data used to initialize the device device, including the base address base_address of the device on the system bus, the device's interrupt number Device_irq in the system, the device platform pointer, the number of AI engines included on the device, and other information.
  • the AI Eegine in the device can also be described in the form of data structures.
  • Figure 4 is a data description data structure diagram of an AI engine provided by an embodiment of the present application.
  • the description data of the AI engine includes the engine's task task, the engine's configuration config, the CNN network description network_desc (that is, the network description data) to be inferred by the engine, the description of the engine's internal function module processor, engine status, engine device pointer, etc. .
  • the status description and device pointer respectively describe the status of the AI engine and indicate the device to which the AI engine belongs.
  • Engine_task includes:
  • Device task pointer device_task_ptr This pointer points to the device task device_task, indicating the source of the task. It is also used to index device task device_task related information, such as the address list and number of addresses where the data involved in the task is located.
  • Task status task_status used to mark the status of task execution.
  • Task base address task_base The address offset used when indexing the task.
  • Pointer to the task operation description list operation_desc_ptr During CNN inference, it usually contains dozens or hundreds of convolutional layers, which require the convolution and other processors inside the AI engine to repeatedly process these convolutional layers, so each processor (i.e., functional module) There are dozens or hundreds of different task operation description data. These data are delivered to the storage space of the AI engine through the master. Therefore, the address of these task operation description data needs to be specified for the AI engine.
  • Task input/output data description pointer surface_desc_ptr The input/output data of the task is the description of the input/output data of each convolutional layer of CNN. It describes the dimension size of the data, whether it is tightly arranged, whether it is compressed, whether there is an address jump, etc. information.
  • Task dependency description data address dependency_graph_addr: dependency_graph is the dependency relationship between task operations. Only when all dependencies of the current task operation of the processor are resolved can the task operation be enabled. This configuration allows the AI engine to automatically switch between tasks and improve the applicability of the AI engine to different CNN networks.
  • Lookup table data address lut_data_addr The lookup table is used to implement nonlinear calculations, such as sigmoid nonlinear activation.
  • the lookup table data needs to be written by the master into the storage space of the AI engine.
  • Region of interest (roi) array address In the field of image processing, the region of interest (ROI, region of interest) is an image area selected from the image. This area is the focus of image analysis. Circle the area for further processing. Similarly, the ROI data needs to be sent by the master to the storage space of the AI engine.
  • ROI region of interest
  • Input/output data address surface_addr Input/output data refers to the image input during inference or the intermediate result data of the intermediate convolution layer.
  • the engine configuration data mainly includes:
  • Metadata size atom_size The bus bit width of the AI engine, which can usually be set to 256bit or 512bit. It has large data throughput capability and can process 32 or 64 int8 type data in parallel.
  • Data compression support compress_support If data compression is supported, the AI engine needs to implement the data decompression function.
  • the target neural network which is the neural network that needs to be inferred and calculated, and generate network description data based on the target neural network and the hardware architecture description data.
  • This embodiment does not limit the specific generation process of network description data.
  • preset network layer description data corresponding to each network layer can be preset.
  • determine the target neural network Each network layer and the dependencies between network layers are obtained, and the preset network layer description data corresponding to each network layer is obtained.
  • the synthesis process is performed based on the dependency relationship, and the network description can be obtained. data.
  • Figure 7. Model analysis can be performed on the target neural network to obtain an intermediate graph.
  • the target neural network can specifically be a caffe model, pytorch model, params and other models.
  • the target neural network model and its corresponding configuration parameters are parsed through Parser (a syntax parser) to convert the deep learning framework model into an intermediate graph representation defined by the compiler.
  • Parser a syntax parser
  • graph optimization is performed on the intermediate graph to obtain the optimized graph.
  • Optimizer an optimization program
  • the specific optimization method can be common subexpression elimination (common subexpression elimination, CSE), operator fusion, storage optimization, etc.
  • the optimization graph is compiled and processed to obtain network description data.
  • the optimized intermediate graph (i.e., optimization graph) is compiled by Compiler to generate a map file (i.e., network description data) that can be deployed on the hardware.
  • the map file is subsequently deployed on the Device through the runtime.
  • the Device's Driver or Firmware i.e., task scheduler
  • FIG 8 shows a data structure diagram of a specific neural network description data Network_desc.
  • These include processor operation description data operation_desc, input/output data description surface_desc, dependency description between tasks dependency_desc, lookup table data lut_data, area of interest data roi_array, status data stat_list, the starting number of each processor's operation_desc, etc. Since the CNN data is written by the master into the storage space of the AI engine, the AI engine needs to extract data from the storage space based on the CNN description Network_desc during inference calculation and participate in the calculation.
  • the process of data extraction by AI engine can be divided into two steps: First, obtain the address of the data in the storage space from the address_list of Device_task according to the index described by Network_desc, and write the address to the Engine task for subsequent data reading. ; The second is to read data from the storage space according to the address in engine_task and participate in inference operations.
  • the description data of the AI engine can also include the function module processor description data.
  • the processor description can be divided into three parts: processor attributes, processor input data and processor behavior functions (i.e. API).
  • the attribute data of the processor includes the name of the processor, the type of processor (convolution, activation, pooling, etc.), the result data receiver or consumer of the processor, or the next-level processor, etc.
  • each functional module that is, the processor, corresponds to two register groups, and the two are ping-pong register groups. Therefore, in this embodiment, the ID of the register group used last time can be used to calculate the ID of the register group to be used in this operation. Since there are only two sets of register groups, the ID is equal to 0 or 1.
  • the input combined data includes dependency description tail_op, processor_group, and status description stat_data_desc.
  • the three include all the data required for processor operation, including: denpendency (used to update pipeline status and automatically enable the processor), operation (processor operation data), surface_desc (information description such as input/output data dimensions), surface (Input data) etc.
  • Figure 10 shows a kind of dependency description data, which includes the index (i.e., index table) of this processor operation, the operation type op_type, the dependency count dependency_count, as well as the producers of input data and consumers of output data of this processor.
  • the dependency description of the processor operation is the key to the various processor pipelines inside the AI engine. Each processor operation has a dependency description. Each dependency description records the previous level and the next level of this operation, so that all the processes inside the AI engine
  • the processor forms a linked list.
  • dependency_count that is, execution wait count
  • the update operation is specifically the dependency (that is, dependency_count) minus 1 operation.
  • the processor's denpendency_count is 0, the processor is enabled.
  • the processor's program() is an operation on the corresponding module configuration register inside the AI engine
  • enable() is an operation to enable the processor to perform inference operations.
  • FIG 11 shows a specific data structure of processor group description data.
  • the data of the processor is transmitted through the processor group.
  • Each functional module processor inside the AI engine is set up with 2 sets of registers.
  • the ping-pong rotation mechanism controls the continuous operation of the processor without gaps. Therefore, the configuration operation of a register group is in Another set of registers controls what is done when the processor is running. This method shortens the time for the AI engine to execute tasks and improves the computing efficiency of the AI engine.
  • the register group controls the operation of the processor, processor group can be used to describe the specific information of the register group that needs to be configured and processor-related information. It should be noted that this information needs to be generated when configuring the register, and the specific generation method is not limited.
  • the id in Figure 11 describes the register group number. Since there are only two sets of registers, the id is 0 or 1. active records whether the current processor is enabled. The event records the bit number of the interrupt register that is set after the current processor is interrupted. That is, after the processor operation is completed, the event bit in the 32-bit interrupt register will be set to 1.
  • roi_index indicates the current location of ROI when ROI exists. is_rdma_needed refers to whether the current processor operation needs to read other parameters from the outside to participate in the operation. Pending refers to whether the current processor has an interrupt that has not been processed.
  • lut_index is the position of the address of the lookup table in address_list when the processor is active.
  • This level of denpendency is used to describe the dependencies of the current processor and record dependency-related information.
  • the upper-level dependency and the lower-level dependency are the dependencies of parents and consumers. After program() or enable(), the current processor needs to update the dependency_count of parent and consumers. It should be noted here that the upper level is not executed first and the next level is executed later. For example, convolution -> activation -> pooling, you need to enable 'pooling' first, then enable 'activation', and finally enable 'convolution', and the three basically perform operations at the same time because the three are pipelined. , 'Pooling' requires the result of 'Activation' as input, similarly, 'Activation' requires the result of 'Convolution' as input. Therefore, from the perspective of the entire task processing process, the three operations are performed at the same time.
  • operation_container and surface_container are the register data and input/output data information of the processor.
  • Figure 12A is the first part of a data structure diagram of register description data provided by an embodiment of the present application.
  • Figure 12B is a data structure diagram provided by an embodiment of the present application.
  • the second part of the data structure diagram of register describing data The two parts together form a complete data structure diagram of register describing data.
  • Figure 13A is a data structure diagram of an input and output mode description data provided by an embodiment of the present application.
  • Figure 13B is a specific data structure diagram of a convolution data description data provided by an embodiment of the present application.
  • Figure 13C is a data structure diagram of an input and output mode description data provided by an embodiment of the present application.
  • a specific data structure diagram of activation data description data is provided.
  • the operation container contains the information of all registers that need to be configured for the processor group.
  • the operation_container contains the operation descriptions of n processors. n is the number of processors in the internal function modules of the AI engine. In this example, the AI engine only has convolution, activation, and pooling. The content of the operation description of each processor is determined by the hardware design of the processor. For example, the convolution processor sets the convolution mode, data multiplexing, data type, batch size, data buffer size, convolution data and kernel size, Step size, padding or not, input and output accuracy, etc.
  • the program() operation of the processor is to configure these register data to the corresponding functional modules of the AI engine.
  • Surface_container describes the input/output data of the processor.
  • conv_surface_desc contains a description of input data, output data, weight data and compressed data when data compression exists. The description starts from data type, data address, data address offset, data size, data dimension, data The occupied space and other aspects describe the data information, which is used by the processor to read data or write data.
  • the description data of AI engine includes hardware information description of AI device->AI engine->processor->processor group, task description of device task->engine task, CNN description, operation description, and data description.
  • These descriptions include processor pipeline information, task automatic scheduling information and dependency information between tasks.
  • the entire description is an abstraction of hardware, CNN, tasks, operations, and data, and abstracts complex task scheduling logic and hardware configuration logic into data update iterations. Therefore, the entire CNN inference process is converted into a process in which the internal processor of the AI engine is continuously scheduled, configured, and enabled for execution.
  • dependency_desc describes the processor call enabling logic
  • operation_desc describes the processor's register group configuration information
  • surface_desc describes each processor input. /Output data information.
  • This information is generated by the Master through the compiler based on the hardware configuration and the trained CNN.
  • Master writes these data into the data storage space of AI engine, then combines the addresses of various data into address_list, and stores the address of address_list and the number of addresses in address_list in the device task.
  • Network_desc describes these data addresses in address_list. The index number in .
  • the address of Network_desc itself is stored at position 0 in address_list.
  • Figure 14 shows the process of converting the lenet model into the processor calculation flow of the AI engine. INDEX is the index of the processor.
  • the task scheduler needs to first obtain the Network_desc (that is, the network description data) from the 0 position of the address_list, and then obtain the storage address of the relevant data from the Address_list according to the index number in the Network_desc, and store it in engine_task.
  • the processor on the AI Engine performs inference, it reads the address from engine_task and assigns it to the data address in surface_desc.
  • the processor reads relevant data from the storage space based on the address, offset, size, and data type of the data.
  • the dependency relationships describe the order of tasks in the network layer. This embodiment does not limit the specific method of determining the dependency relationship.
  • the network descriptor data corresponding to each network layer task is determined from the network description data, and the corresponding network descriptor data is parsed from each network descriptor data.
  • the next network layer pointer that is, the pointer consumer_ptr of the next operation of the processor in the above-mentioned processor description data. Based on each next network layer pointer, the dependency relationship is obtained.
  • each functional module on the artificial intelligence accelerated computing component corresponds to at least two register groups.
  • each processor corresponds to two ping-pong register groups.
  • the free register group in this application refers to a register group without data and a register group that stores register group data corresponding to network layer tasks that have been completed.
  • the detection of free register groups can be performed in real time, according to a preset period, or when an interrupt is detected (that is, a network layer task is completed), so that the register group data can be loaded when other non-idle register groups are used for inference. ,Improve efficiency.
  • all register groups on the artificial intelligence accelerated computing component can be determined as free register groups, and then Configure all register groups.
  • the artificial intelligence accelerated computing component can also be started to start model inference. If it is not in the initialization stage and the target network layer task completion notification is detected, it means that the target network layer task has been executed and the register data in the corresponding register group is invalid. In this case, first determine the target network layer task corresponding The second functional module.
  • the free register group is one of the two ping-pong register groups corresponding to the second functional module.
  • each functional module needs to be enabled first.
  • the enabling is completed, the functional module that meets the conditions (that is, the first functional module in S105) is started.
  • operations such as data loading can be completed to improve the degree of parallelism and thereby improve computing efficiency.
  • the enabling sequence corresponding to each target function module on the artificial intelligence accelerated computing component can be determined based on the dependency relationship.
  • the target function module can be any functional module, and the enabling sequence is in the same direction as the dependency relationship. Based on the reverse order of enabling, each target function module is enabled, so that the enabled target function module performs data calculation preparation operations based on the corresponding non-idle register group.
  • Data calculation preparation operations can include operations such as data loading and transformation, and are not specifically limited.
  • S104 Based on the dependency relationship, network description data and free register group type, generate a target register configuration, and write the target register configuration to the free register group.
  • network layer tasks are executed in a sequence, and each functional module can only execute its corresponding type of network layer task.
  • candidate network layer tasks that match the free register group type can be determined based on dependencies.
  • the free register group type is the type of the corresponding functional module, such as convolution, activation, and pooling. Chemical etc. Based on the type, candidate tasks that match the type can be determined, such as tasks corresponding to the convolutional layer or tasks corresponding to the pooling layer.
  • the candidate descriptor data corresponding to the candidate network layer task in the network description data that is, the data related to the network layer task
  • the candidate descriptor data and the hardware architecture description data corresponding to the artificial intelligence accelerated computing component can be generated based on operation_container, suface_container and other data.
  • S105 If the target network layer task completion notification is detected, determine the next task of the target network layer task corresponding to the target network layer task completion notification based on the dependency relationship, and set the next task to the corresponding first task on the artificial intelligence accelerated computing component.
  • the function module is started to perform the next task.
  • the target network layer task completion notification (ie, interruption) is detected, it means that the target network layer task has been executed, and the target network layer task can be any network layer task.
  • the next task of the target network layer task corresponding to the target network layer task completion notification is determined based on the dependency relationship. That is, according to the dependency relationship, the next network layer task corresponding to the task can be determined, that is, the next task. Since the inference of the target network layer is performed serially, the output of the previous network layer task is the output of the next network layer task. enter. For a target neural network, its corresponding input data is processed by each network layer task in turn, and an output can be obtained.
  • each input data can respectively correspond to different network layer tasks of the target neural network.
  • different functions can be utilized at one time.
  • the module processes different input data to achieve parallel processing effects, thereby improving computing efficiency.
  • each network layer task it participates in is executed at different times. Therefore, for an input data, as long as the target network layer is not the last network layer, there must be its corresponding next task. .
  • the register configuration corresponding to the next task has already been written. In this case , you can directly execute the next task. Specifically, according to the type of the next task, its corresponding first functional module on the artificial intelligence accelerated computing component can be determined, the first functional module is started, and the first functional module can execute the next task according to the register configuration. This embodiment does not limit the specific method of determining the next task. In one implementation, the above-mentioned dependency_count can be used to determine the next task.
  • the target network layer task completion notification can be parsed to obtain the task identifier (such as event number) corresponding to the target network layer task, and based on the dependency relationship, the task identifier can be used to determine subsequent tasks corresponding to the target network layer task.
  • follow-up tasks refer to tasks that need to be performed after the target network layer task, and the specific number is not limited. Decrement the execution wait count (dependency_count) corresponding to each subsequent task by one, and determine the target subsequent task whose execution wait is reduced by one to zero as the next task. If the execution wait is reduced by one and becomes zero, it indicates that the target subsequent task needs to be executed immediately .
  • the AI engine task initialization includes: the Master delivers the task data to the slave-side storage space, and notifies the slave-side processor through an interrupt to start parsing the task, and starts the AI engine task scheduler for task scheduling.
  • the scheduler reads the delivered device_task data and obtains the address_list address and the number of addresses in the address_list.
  • the initialization and register group configuration of processor groups include: reading the address of Network_desc from position 0 in address_list, and reading the Network_desc data. According to the address index described by Network_desc, read the operation_des, surface_desc, dependency_desc, lut_list and other addresses, and use these addresses to initialize the engine_task data structure.
  • the scheduler can also read op_head from Network_desc.
  • This op_head indicates the index of the first dependency_desc of each processor, and based on the data size of index and dependency_desc, calculates the offset address offset of the dependency_desc of the current processor, and then According to the dependency_graph_addr in engine_task, get the address of reading dependency_desc, and then read the data. Obtain the op_type of the current processor based on the read dependency_desc.
  • the scheduler obtains the id of the idle register group of the processor group and whether to read the rdma id based on dependency_desc and processor, and calculates the address offset offset based on the index in dependency_desc, the size of the operation_desc data type, and the size of the surface_desc data type; and then based on the index in engine_task For operation_desc_addr and surface_desc_addr, calculate the specific addresses of the two data and read them.
  • the scheduler initializes the current processor group with the data read above and starts configuring the AI engine function module corresponding to the current processor.
  • the scheduler updates the tail_op at the end of dependency_desc of the current processor to the current dependency data, which is used to find the index of consumers of the current processor. Decrement the dependency_count of the current processor by 1 and determine whether it is 0; if it is 0, enable the function module of the AI engine corresponding to the processor and start computing.
  • the scheduler inverts the ID of the current processor group, configures another set of register groups according to the above steps, and configures other n-1 processors according to the above steps.
  • the interrupt service routine is a program that is executed immediately after an interrupt is generated.
  • the program reads the AI engine's interrupt status register, determines the processor type, and generates the corresponding event number. Then clear the interrupt status register.
  • the current events will be stored in the event of the processor group, and event_flag will be set to 1.
  • the events handler is located in the while(1) loop and waits until the task execution is completed before jumping out of the while loop.
  • the events handler is located after the process groups initialization and register configuration procedures.
  • event_flag When the events handler finds that event_flag is 1, it first clears event_flag, and then polls the two processor groups of all processors to determine which processor group has generated an interrupt, and handles it accordingly. If the DMA reading of image data or weight data is interrupted, the dependencies of all consumers of the current processor group will be directly updated. If other processors are interrupted, the group ID will be rotated first and the dependencies of all consumers will be updated. Then, the scheduler begins to configure the processor goup register group corresponding to the current group id. It should be noted that according to the above logic at this time, the configuration data of the current group's register group has been read and the processor group is ready.
  • the scheduler updates the denpency of all consumers. If there is a processor with a dependency_count of 0, it is enabled.
  • the scheduler switches processor groups, and runs the initialization steps of the processor groups and the response in the register group configuration process until the last interrupt is generated.
  • the number of operations run is equal to the number of operations in Netwok_desc, indicating that the task is completed.
  • the slave side generates an interrupt to notify the Master to read the final result.
  • the consumer and parent of the process group convert the entire reasoning process into the pipeline of the processor group (pipeline processing), and update the dependency of all consumers and parent processor groups after configuring the register group or after an interrupt is generated, and when dependency_count is When 0, the processor is enabled.
  • CNN is composed of multiple identical or similar convolutional layers, and various calculations within the convolutional layer can be pipelined (i.e., pipeline processing), combined with the hardware of the AI engine. Configuration, efficient scheduling of computing tasks, and improved computing efficiency.
  • the network description data After obtaining the network description data, if it is determined that a free register group exists, based on the dependency relationship and the free register group type, the network description data is used to generate a target register configuration that matches the free register group, and is written to the free register group.
  • each functional module corresponds to at least two registers group, and the configuration time of the register is shorter than the time for the functional module to use the register for data processing, so at any time, each functional module corresponds to at least one non-idle register group.
  • the first functional module corresponding to the next task can be started immediately, enabling seamless pipeline processing and improving computing efficiency; at the same time, because each function
  • the registers of the modules are all configured, so multiple functional modules can be calculated and processed in parallel, making full use of hardware resources and improving computing efficiency; in addition, the configuration of registers and the operation of functional modules can be executed in parallel.
  • the idle register group can be configured, or other register groups can perform data reading and writing operations to fully utilize the bus bandwidth, avoid delays caused by sudden large data transmission requirements, and improve computing efficiency.
  • the computing task scheduling device provided by the embodiment of the present application is introduced below.
  • the computing task scheduling device described below and the computing task scheduling method described above may be mutually referenced.
  • Figure 17 is a schematic structural diagram of a computing task scheduling device provided by an embodiment of the present application. It is applied to a task scheduler and includes:
  • the network description acquisition module 110 is used to obtain network description data; the network description data is used to describe the network layer tasks corresponding to each network layer in the target neural network;
  • the dependency determination module 120 is used to determine the dependency relationship between various network layer tasks based on the network description data
  • the idle detection module 130 is used to detect idle register groups on the artificial intelligence accelerated computing component; each functional module on the artificial intelligence accelerated computing component corresponds to at least two register groups;
  • the register configuration module 140 is used to generate a target register configuration based on dependencies, network description data and free register group types, and write the target register configuration to the free register group;
  • the startup module 150 is configured to, if a target network layer task completion notification is detected, determine the next task of the target network layer task corresponding to the target network layer task completion notification based on the dependency relationship, and set the next task on the artificial intelligence accelerated computing component The corresponding first functional module is started to perform the next task.
  • the hardware architecture determination module is used to determine the hardware architecture of the artificial intelligence acceleration computing component
  • the hardware description generation module is used to generate corresponding hardware architecture description data based on the hardware architecture
  • the network description generation module is used to obtain the target neural network and generate network description data based on the target neural network and hardware architecture description data.
  • the network description generation module includes:
  • the model analysis unit is used to perform model analysis on the target neural network and obtain the intermediate graph
  • the optimization unit is used to describe the data based on the hardware architecture, perform graph optimization processing on the intermediate graph, and obtain the optimized graph;
  • the compilation unit is used to compile the optimization graph and obtain network description data.
  • the network description generation module includes:
  • Dependency determination unit used to determine each network layer in the target neural network and the dependencies between network layers
  • the acquisition unit is used to acquire the preset network layer description data corresponding to each network layer;
  • the generation unit is used to generate network description data based on dependencies using preset network layer description data.
  • the dependency determination module 120 includes:
  • the sub-data determination unit is used to determine the network description sub-data corresponding to each network layer task from the network description data;
  • the pointer acquisition unit is used to parse and obtain the corresponding next network layer pointer from each network descriptor data
  • the relationship generation unit is used to obtain dependency relationships based on each next network layer pointer.
  • the idle detection module 130 includes:
  • An initial detection unit used to determine all register groups on the artificial intelligence accelerated computing component as free register groups if it is in the initialization stage;
  • the module determination unit is used to determine the second functional module corresponding to the target network layer task if it is not in the initialization stage and the target network layer task completion notification is detected;
  • An identification acquisition unit configured to update the free register group identification corresponding to the type in the network description data based on the type of the second functional module
  • the idle determination unit is used to determine the idle register group corresponding to the second functional module according to the updated idle register group identification; wherein the idle register group is one of the two ping-pong register groups corresponding to the second functional module.
  • the register configuration module 140 includes:
  • a candidate determination unit used to determine candidate network layer tasks that match the free register group type based on dependency relationships
  • a sub-data acquisition unit used to determine candidate descriptor data corresponding to the candidate network layer task in the network description data
  • the configuration generation unit is used to generate the target register configuration using the candidate descriptor data and the hardware architecture description data corresponding to the artificial intelligence accelerated computing component.
  • start module 150 including:
  • the identification determination unit is used to parse the target network layer task completion notification and obtain the task identification corresponding to the target network layer task;
  • the subsequent task determination unit is used to determine the subsequent tasks corresponding to the target network layer task based on the dependency relationship and using the task identifier;
  • the update unit is used to decrement the execution wait count corresponding to each subsequent task by one;
  • the next task determination unit is used to determine the target subsequent task whose execution wait minus one is zero as the next task.
  • the enabling sequence determination module is used to determine the enabling sequence corresponding to each target function module on the artificial intelligence accelerated computing component based on dependencies;
  • the enabling module is used to enable each target function module based on the reverse order of the enabling sequence, so that the enabled target function module performs data calculation preparation operations based on the corresponding non-idle register group.
  • the electronic device provided by the embodiment of the present application is introduced below.
  • the electronic device described below and the computing task scheduling method described above can be mutually referenced.
  • the electronic device 100 may include one or more processors 101 and memory 102, and may further include one or more of a multimedia component 103, an information input/output (I/O) interface 104, and a communication component 105.
  • a multimedia component 103 may include one or more of a multimedia component 103, an information input/output (I/O) interface 104, and a communication component 105.
  • I/O information input/output
  • the processor 101 is used to control the overall operation of the electronic device 100 to complete all or part of the steps in the above-mentioned computing task scheduling method;
  • the memory 102 is used to store various types of data to support the operations of the electronic device 100.
  • the data may include, for example, instructions for any application or method operating on the electronic device 100, as well as application-related data.
  • the memory 102 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (Read- Only Memory (ROM), magnetic memory, flash memory, one or more of magnetic disks or optical disks.
  • SRAM static random access memory
  • EEPROM Electrically erasable programmable read-only memory
  • EPROM Erasable Programmable Read-Only Memory
  • PROM Programmable Read-Only Memory
  • ROM Read-Only Memory
  • magnetic memory flash memory, one or more of magnetic disks or optical disks.
  • Multimedia components 103 may include screen and audio components.
  • the screen may be a touch screen, for example, and the audio component is used to output and/or input audio signals.
  • the audio component may include a microphone for receiving external audio signals.
  • the received audio signals may be further stored in memory 102 or sent via communication component 105 .
  • the audio component also includes at least one speaker for outputting audio signals.
  • the I/O interface 104 provides an interface between the processor 101 and other interface modules.
  • the other interface modules may be keyboards, mice, buttons, etc. These buttons can be virtual buttons or physical buttons.
  • the communication component 105 is used for wired or wireless communication between the electronic device 100 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G or 4G, or one or a combination of them, so the corresponding communication component 105 may include: Wi-Fi parts, Bluetooth parts, NFC parts.
  • the electronic device 100 may be configured by one or more application specific integrated circuits (Application Specific Integrated Circuit, ASIC for short), digital signal processor (Digital Signal Processor, DSP for short), digital signal processing device (Digital Signal Processing Device, DSPD for short), Programmable Logic Device (PLD for short), Field Programmable Gate Array (FPGA for short), controller, microcontroller, microprocessor or other electronic components are implemented for executing the above embodiments
  • ASIC Application Specific Integrated Circuit
  • DSP Digital Signal Processor
  • DSPD Digital Signal Processing Device
  • PLD Programmable Logic Device
  • FPGA Field Programmable Gate Array
  • the application also provides one or more non-volatile computer-readable storage media storing computer-readable instructions.
  • the computer-readable instructions When executed by one or more processors, the computer-readable instructions cause one or more processors to perform computing task scheduling.
  • the computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the above computing task scheduling method are implemented.
  • the computer-readable storage medium can include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk, etc., which can store program code medium.
  • RAM random access memory
  • ROM read-only memory
  • electrically programmable ROM electrically erasable programmable ROM
  • registers hard disks, removable disks, CD-ROMs, or anywhere in the field of technology. any other known form of storage media.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Artificial Intelligence (AREA)
  • Neurology (AREA)
  • Multi Processors (AREA)
  • Devices For Executing Special Programs (AREA)
  • Image Analysis (AREA)

Abstract

一种计算任务调度方法、装置、电子设备及计算机可读存储介质，应用于神经网络领域；方法包括：获取网络描述数据（S101）；基于网络描述数据，确定各个网络层任务之间的依赖关系（S102）；检测人工智能加速计算部件上的空闲寄存器组（S103）；基于依赖关系、网络描述数据和空闲寄存器组类型，生成目标寄存器配置，并将目标寄存器配置写入到空闲寄存器组（S104）；若检测到目标网络层任务完成通知，则基于依赖关系确定目标网络层任务完成通知对应的目标网络层任务的下一任务，并将下一任务在人工智能加速计算部件上对应的第一功能模块启动，以便执行下一任务（S105）。

Description

一种计算任务调度方法、装置、电子设备及可读存储介质
相关申请的交叉引用
本申请要求于2022年04月20日提交中国专利局，申请号为202210413206.1，申请名称为“一种计算任务调度方法、装置、电子设备及可读存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及神经网络技术领域，特别涉及一种计算任务调度方法、计算任务调度装置、电子设备及计算机可读存储介质。
背景技术
人工智能加速引擎(AI Engine)是专门为深度学习算法设计的加速计算部件，深度学习算法常用的卷积神经网络结构(Convolutional Neural Networks，CNN)通常包含多个卷积组合层，每个卷积组合层又包括卷积、激活、池化等小层。卷积组合层用于提取图像中的不同的特征，一个CNN通常包含数十甚至上百个卷积组合层。当AI engine的硬件结构定型后，其理论算力是固定不变的。但在实际应用中，由于配置时间、数据传输时间、模块间的不协调导致的延时等开销，导致AI engine的计算效率较低。
因此，相关技术存在的AI engine的计算效率较低的问题，是本领域技术人员需要解决的技术问题。
发明内容
本申请提供了一种计算任务调度方法，应用于任务调度器，方法包括：
获取网络描述数据；网络描述数据用于描述目标神经网络中各个网络层分别对应的网络层任务；
基于网络描述数据，确定各个网络层任务之间的依赖关系；
检测人工智能加速计算部件上的空闲寄存器组；人工智能加速计算部件上各个功能模块分别对应于至少两个寄存器组；
基于依赖关系、网络描述数据和空闲寄存器组类型，生成目标寄存器配置，并将目标寄存器配置写入到空闲寄存器组；和
响应于检测到目标网络层任务完成通知，基于依赖关系确定目标网络层任务完成通知对应的目标网络层任务的下一任务，并将下一任务在人工智能加速计算部件上对应的第一功能模块启动，以便执行下一任务。
可选地，网络描述数据的生成过程，包括：
确定人工智能加速计算部件的硬件架构；
基于硬件架构，生成对应的硬件架构描述数据；和
获取目标神经网络，并基于目标神经网络和硬件架构描述数据，生成网络描述数据。
可选地，基于目标神经网络和硬件架构描述数据，生成网络描述数据，包括：
对目标神经网络进行模型解析，得到中间图；
基于硬件架构描述数据，对中间图进行图优化处理，得到优化图；和
对优化图进行编译处理，得到网络描述数据。
可选地，基于目标神经网络和硬件架构描述数据，生成网络描述数据，包括：
确定目标神经网络中的各个网络层，以及网络层之间的依赖关系；
获取各个网络层分别对应的预设网络层描述数据；和
利用预设网络层描述数据，基于依赖关系，生成网络描述数据。
可选地，基于网络描述数据，确定各个网络层任务之间的依赖关系，包括：
从网络描述数据中确定各个网络层任务分别对应的网络描述子数据；
从各个网络描述子数据中解析得到对应的下一网络层指针；和
基于各个下一网络层指针，得到依赖关系。
可选地，检测人工智能加速计算部件上的空闲寄存器组，包括：
响应于处于初始化阶段，将人工智能加速计算部件上的所有寄存器组确定为空闲寄存器组。
可选地，检测人工智能加速计算部件上的空闲寄存器组，包括：
响应于不处于初始化阶段，且检测到目标网络层任务完成通知，确定目标网络层任务对应的第二功能模块；
基于第二功能模块的类型，更新网络描述数据中与类型对应的空闲寄存器组标识；和
根据更新后的空闲寄存器组标识，确定第二功能模块对应的空闲寄存器组；其中，空闲寄存器组为第二功能模块对应的两组乒乓寄存器组中的一个。
可选地，基于依赖关系、网络描述数据和空闲寄存器组类型，生成目标寄存器配置，包括：
基于依赖关系，确定与空闲寄存器组类型匹配的候选网络层任务；
在网络描述数据中确定与候选网络层任务对应的候选描述子数据；和
利用候选描述子数据和人工智能加速计算部件对应的硬件架构描述数据，生成目标寄存器配置。
可选地，基于依赖关系确定目标网络层任务完成通知对应的目标网络层任务的下一任务，包括：
解析目标网络层任务完成通知，得到目标网络层任务对应的任务标识；
基于依赖关系，利用任务标识确定目标网络层任务对应的后续任务；
将各个后续任务对应的执行等待计数减一；和
将执行等待减一后为零的目标后续任务确定为下一任务。
可选地，还包括：
基于依赖关系，确定人工智能加速计算部件上各个目标功能模块对应的使能顺序；和
基于使能顺序的倒序，对各个目标功能模块进行使能处理，以使被使能处理后的目标功能模块基于对应的非空闲寄存器组进行数据计算准备操作。
本申请还提供了一种计算任务调度装置，应用于任务调度器，装置包括：
网络描述获取模块，用于获取网络描述数据；网络描述数据用于描述目标神经网络中各个网络层分别对应的网络层任务；
依赖确定模块，用于基于网络描述数据，确定各个网络层任务之间的依赖关系；
空闲检测模块，用于检测人工智能加速计算部件上的空闲寄存器组；人工智能加速计算部件上各个功能模块分别对应于至少两个寄存器组；
寄存器配置模块，用于基于依赖关系、网络描述数据和空闲寄存器组类型，生成目标寄存器配置，并将目标寄存器配置写入到空闲寄存器组；和
启动模块，用于响应于检测到目标网络层任务完成通知，基于依赖关系确定目标网络层任务完成通知对应的目标网络层任务的下一任务，并将下一任务在人工智能加速计算部件上对应的第一功能模块启动，以便执行下一任务。
本申请还提供了一种电子设备，包括存储器及一个或多个处理器，存储器中储存有计算机可读指令，计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器执行上述的计算任务调度方法的步骤。
本申请还提供了一个或多个存储有计算机可读指令的非易失性计算机可读存储介质，其特征在于，计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器执行计算任务调度方法的步骤。
附图说明
为了更清楚地说明本申请实施例或相关技术中的技术方案，下面将对实施例或相关技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据提供的附图获得其他的附图。
图1为本申请一个或多个实施例提供的一种AI engine的硬件架构示意图；
图2为本申请一个或多个实施例提供的一种计算任务调度方法流程图；
图3为本申请一个或多个实施例提供的一种AI加速设备的硬件架构描述数据结构图；
图4为本申请一个或多个实施例提供的一种AI engine的数据结构图；
图5为本申请一个或多个实施例提供的一种引擎任务数据的数据结构图；
图6为本申请一个或多个实施例提供的一种引擎配置数据的数据结构图；
图7为本申请一个或多个实施例提供的一种网络描述数据生成过程流程图；
图8为本申请一个或多个实施例提供的一种网络描述数据的数据结构图；
图9为本申请一个或多个实施例提供的一种功能模块描述数据的数据结构图；
图10为本申请一个或多个实施例提供的一种依赖描述数据的数据结构图；
图11为本申请一个或多个实施例提供的一种功能模块描述数据的数据结构图；
图12A为本申请一个或多个实施例提供的一种寄存器描述数据的数据结构图的第一部分；
图12B为本申请一个或多个实施例提供的一种寄存器描述数据的数据结构图的第二部分；
图13A为本申请一个或多个实施例提供的一种输入输出方式描述数据的数据结构图；
图13B为本申请一个或多个实施例提供的一种卷积数据描述数据的具体数据结构图；
图13C为本申请一个或多个实施例提供的一种激活数据描述数据的具体数据结构图；
图14为本申请一个或多个实施例提供的一种卷积神经网络推理过程示意图；
图15为本申请一个或多个实施例提供的一种具体的任务调度流程图；
图16为本申请一个或多个实施例提供的一种具体的中断处理流程图；
图17为本申请一个或多个实施例提供的一种计算任务调度装置的结构示意图；
图18为本申请一个或多个实施例提供的一种电子设备的结构示意图。
具体实施方式
为使本申请实施例的目的、技术方案和优点更加清楚，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
人工智能加速引擎(或称为人工智能加速部件，AI Engine)是专门为深度学习算法设计的加速计算部件，深度学习算法常用的卷积神经网络结构通常包含多个卷积组合层，每个卷积组合层又包括卷积、激活、池化等小层。卷积组合层用于提取图像中的不同的特征，一个CNN通常包含数十甚至上百个卷积组合层。
综上，在一种实施方式中，一个任务型AI加速引擎的大致结构请参考图1所示，包含：接口总线AXI(Advanced eXtensible Interface)和APB(Advanced Peripheral Bus，外围总线)，读写外部存储介质的DMA(Direct Memory Access，存储器直接访问控制器)模块，本地缓存buffer模块，并行数据加载的Data Loader模块，状态监测(即Status Collector)及中断控制器(即Interrupt Controller)，负责计算的MAC单元(或称为乘加计算模块，功能模块的一种)、Activation单元(或称为激活处理模块，功能模块的一种)、Pooling单元(或称为池化处理模块，功能模块的一种)。此外还可以包含数据reshape模块，归一化模块等。
在应用过程中，Registers(即寄存器，或寄存器组)接收各个模块的配置信息，Controller根据这些配置控制各个模块的运转。Controller可以有多种存在方式，图1是集中式的controller，此外，还可以为分散式的controller，即各个controller分别位于各个模块内部，这些模块都挂载在APB上或用菊花链的方式连接。
Status Collector监控各个模块的信息，如buffer空闲空间大小、各模块运行状态、读写数据的数目、阻塞时长或空闲时长等信息，并为Interrupt Controller提供各模块的中断pending，如DMA读取数据完成中断pending、MAC Unit计算完成中断pending、Activation激活完成中断pending、Pooling完成中断pending等。Interrupt Controller内设置中断pending寄存器，pending寄存器的各个bit位映射表示各个功能模块的pending信号。此外，还设置了中断mask寄存器和中断使能寄存器，当中断使能且中断没有被mask时，pending位置1时，产生中断信号Intr。具体是AI engine的哪一个模块完成了任务产生了中断，可以通过查询pending寄存器确定。由此可见，调用AI加速引擎加速计算的过程是不断配置寄存器并等待中断的过程。
一个CNN网络有数十或上百个组合层，若单个组合层的计算量超过了AI加速引擎的计算能力，需要将单个组合层 拆分为多个子组合层。每个组合层可能包含load数据、乘加运算、激活、pooling、归一化、融合等多种操作，需要调用多个模块共同完成一个组合层的运算任务。由此可见，完成一次CNN的推理任务，各个模块需要完成成百上千个任务，且各个模块之间需要相互配合。
为了能够保证AI engine以较高的计算效率工作，需要尽可能地使各个功能模块并行计算，各个计算任务呈流水线的形式(即pipeline)处理，此外，还需要尽可能的减少功能模块未进行计算的时长，平衡地利用带宽资源等。为了实现该效果，本申请设计了一种任务调度器，该任务调度器可以用于对AI engine上的任务进行调度，使得AI engine能够高效计算。本实施例并不限定任务调度器的具体形式，在一种实施方式中，任务调度器的角色可以由AI device(人工智能加速计算设备)中的处理器担任，AI device中可以具有任意数量的AI engine，每个AI engine可以对应于一个处理器，或多个AI engine对应于一个处理器。
任务调度器为了能够起到灵活调度计算任务的功能，其可以具备如下特点：
1、通用性和可扩展行：不同的厂家、不同的应用场景以及适用的最优CNN网络的不同，且AI engine的功能模块以及组合方式也会有所不同，但是均具备相同的基本架构。因此，任务调度器可以考虑到通用性和可扩展性，以便适用不同的AI engine。
2、AI engine内部各模块的组合与pipeline：CNN每层中各模块的组合方式以及pipeline顺序是不同的。各模块实现的功能不同，内部需要配置的寄存器也就不同，不同的功能模块的配置方式也是不同的。因此，任务调度器应当能够根据任意一种CNN网络结构自动切换需要配置的模块，还需要实现不同的配置方式，与模块同步切换。
3、AI engine内部各模块的使能时机：为了提高AI engine的计算效率，某一功能模块的配置通常在CNN的前一层或其他功能模块运行的过程中配置，轮到该模块执行的时候再使能。即模块的配置逻辑与模块的使能逻辑是不相关的。如CNN某2层都为卷积+激活+Pooling组合，当前层在执行运算时，配置下一层运算所需要的配置，这种机制使得运算和配置同时进行，节省了配置时间。当前层运算结束后直接开始下一层的运算，使得模块运行连续不断，提高模块运算效率。为了实现该效果，本申请中各个功能模块对应于至少两组配置寄存器，即两个寄存器组。在一种实施方式中，可以有两个寄存器组，二者组成乒乓配置模式。在同一层内，各模块的使能也有先后，如上述举例，需要先使能pooling模块，再使能激活模块，最后使能卷积模块；即后执行的先使能，使得后续的模块提前做好接收前面模块的运算结果的准备，否则会出现数据丢失的后果。由此可见，模块的配置与模块的使能是两套不相关的逻辑，这种机制有效的实现了模块间的pipeline，任务调度器应当能够支撑这种调度。
4、数据描述：程序的本质是逻辑和数据，因此，任务调度器应当能够用数据描述任务调度流程涉及的各个流程节点，如对输入/输出数据、CNN结构、AI engine的各个功能模块、AI engine的任务、每一步的操作、AI engine的模块间的联系、模块的使能逻辑进行描述等。
综上，任务调度器在工作时的流程可以概括为：配置寄存器-->模块运行-->运行完成，产生中断。具体的，AI engine根据任务调度器的控制，利用各功能模块工作的过程为：读取任务数据-->初始化所有的可用模块-->更新模块使能序列-->使能序列为0的模块-->等待中断-->配置该模块的另外一组寄存器或下一个使用的模块的寄存器-->收到中断后更新模块的使能序列-->使能使能序列为0的模块-->等待中断-->……。在该过程中，具体的步骤将在后续描述。
请参考图2，图2为本申请实施例提供的一种计算任务调度方法流程图。该方法包括：
S101：获取网络描述数据。
其中，网络描述数据用于描述目标神经网络中各个网络层分别对应的网络层任务，网络层任务是指对网络层进行推理计算的任务。根据上述描述可知，网络描述数据受到神经网络的网络结构，以及AI engine的硬件结构的影响。网络描述数据可以从其他电子设备处获取，或者可以由具有任务调度器的电子设备生成。
在一种具体的实施方式中，在生成网络描述数据时，首先确定人工智能加速计算部件的硬件架构，并基于硬件架构，生成对应的硬件架构描述数据。本实施例并不限定硬件架构描述数据的具体生成方式，在一种实施方式中，可以预设有生成规则，在得到硬件架构后，基于生成规则生成硬件架构描述数据。在另一种实施方式中，可以预存有各种硬件架构对应的预设硬件架构描述数据，在确定硬件架构后，从中确定对应的硬件架构描述数据。在另一种实施方式中，在可到硬件架构后，可以将其输出，以便由人工输入硬件架构描述数据。请参考图3，图3示出了一种AI Device 的硬件架构描述数据，其中包括初始化所需的device_config配置数据，device设备上配置的AI加速引擎描述的指针AI_engine_ptr，设备处理的任务描述Device_task数据。设备任务Device_task通常是一系列子任务的组合，每个子任务都会有输入数据、输出数据和其他参数数据，这些数据都在主机(即master)下发任务时写入了slave(即AI Device)能够访问的存储空间memory中，slave处理这些任务时需要获取这些任务数据的地址。这些地址组合成address_list。address_list中的数据地址数目即address_num。AI_device_ptr是指该任务所属于的device，用于分辨任务的执行者(即slave)。设备配置Device_config是用于初始化设备device的数据，包括设备在系统总线上的基地址base_address、设备在系统中的中断号Device_irq、设备平台指针、设备上包含的AI engine的数目等信息。
除了对Device的描述外，还可以利用数据结构的形式对device中的AI Eegine进行描述。请参考图4，图4为本申请实施例提供的一种AI engine的数描述据结构图。其中，AI engine的描述数据包含引擎的任务task、引擎的配置config、引擎所要推理的CNN网络描述network_desc(即网络描述数据)、引擎的内部功能模块processor的描述、引擎状态、引擎设备指针等内容。状态描述和设备指针分别描述了AI engine的状态，并指示了AI engine所属的device。
具体的，AI engine的任务数据描述为Engine_task，请参考图5，Engine_task包括：
设备任务指针device_task_ptr：该指针指向设备任务device_task，指明了该任务的来源，同时也用于索引设备任务device_task相关信息，如任务涉及的数据所在的地址列表、地址数目等。
任务状态task_status：用于标记任务执行的状态。
任务基地址task_base：用于索引task时的地址偏移。
任务操作描述列表的指针operation_desc_ptr：在CNN推理时，通常包含数十上百的卷积层，需要AI engine内部的卷积等processor重复处理这些卷积层，因此每个processor(即功能模块)都有数十上百个不同的任务操作描述数据。这些数据是通过master下发到AI engine的存储空间中，因此，需要为AI engine指明这些任务操作描述数据的地址。
任务输入/输出数据描指针surface_desc_ptr：任务的输入/输出数据即CNN每个卷积层的输入/输出数据的描述，描述了数据的维度大小、是否紧密排列、是否压缩、有没有地址跳转等信息。
任务依赖关系描述数据地址dependency_graph_addr：dependency_graph是任务操作之间的依赖关系，只有processor当前的任务操作所有依赖都解决了，才能使能该任务操作。这种配置使得AI engine可以自动完成任务之间的切换，提高AI engine对不同CNN网络的适用性。
查找表数据地址lut_data_addr：查找表用于实现非线性计算，如sigmoid非线性激活。查找表数据需要由master写入到AI engine的存储空间中。
感兴趣区域(roi)阵列地址：在图像处理领域，感兴趣区域(ROI，region of interest)是从图像中选择的一个图像区域，这个区域是图像分析所关注的重点。圈定该区域以便进行进一步处理。同样ROI数据需要master下发到AI engine的存储空间中。
输入/输出数据地址surface_addr：输入/输出数据是指推理时的image(即图像)输入或中间卷积层的中间结果数据。
请参考图6，引擎配置数据主要包括：
元数据大小atom_size：AI engine的总线位宽，通常可以设置为如256bit或512bit，具有较大的数据吞吐能力，能够并行处理32个或64个int8类型的数据。
内部模块使能：不同的CNN推理使用的功能模块不同，并不是所有的AI engine的功能模块能够用到，所以设置了模块processor使能。
数据压缩支持compress_support：若支持数据压缩，则AI engine需要实现数据解压功能。
在得到硬件架构描述数据后，获取目标神经网络，目标神经网络即为需要被推理计算的神经网络，并基于目标神经网络和硬件架构描述数据，生成网络描述数据。本实施例并不限定网络描述数据的具体生成过程，在一种实施方式中，可以预设有对各个网络层对应的预设网络层描述数据，在获取目标神经网络后，确定目标神经网络中的各个网络层，以及网络层之间的依赖关系，并获取各个网络层分别对应的预设网络层描述数据，利用预设网络层描述数据，基于依赖关系进行合成的处理，即可得到网络描述数据。在另一种实施方式中，请参考图7，可以对目标神经网络进行模 型解析，得到中间图。如图7所示，目标神经网络具体可以为caffe model、pytorch model、params等模型。目标神经网络模型及其对应的配置参数经过Parser(一种语法解析器)进行模型解析，将深度学习框架模型转换为编译器定义的中间图表示。基于硬件架构描述数据，对中间图进行图优化处理，得到优化图，具体的，Optimizer(一种优化程序)对中间图进行图优化，优化的方式具体可以为，共同子表达式消除(common subexpression elimination，CSE)、算子融合、存储优化等。对优化图进行编译处理，得到网络描述数据。优化后的中间图(即优化图)经过Compiler的编译，生成可部署在硬件上的map文件(即网络描述数据)。map文件后续通过运行时Runtime部署在Device上，Device的Driver或Firmware(即任务调度器)根据map文件调用AI Engine的资源完成此次AI推理计算，还可以将结果反馈给Runtime。
请参考图8，其示出了一种具体的神经网络描述数据Network_desc的数据结构图。其中包括processor操作描述数据operation_desc、输入/输出数据描述surface_desc、任务间的依赖关系描述dependency_desc、查找表数据lut_data、感兴趣的区域数据roi_array、状态数据stat_list、每个processor的operation_desc的起始编号等。由于CNN数据是由master写入到AI engine的存储空间中，AI engine需要在推理计算时基于CNN描述Network_desc从存储空间中提取数据，并参与运算。AI engine提取数据的过程可以拆分为两步：一是根据Network_desc描述的index从Device_task的address_list中获取数据在存储空间中的地址，并将地址写入到Engine task中，以便后续的数据读取；二是根据engine_task中的地址从存储空间中读取数据，参与推理运算。
除此以外，AI engine的描述数据中还可以包括功能模块processor描述数据，请参考图9，processor描述可以分为三部分：processor的属性、processor的输入数据和processor的行为函数(即API)。processor的属性数据包括processor的名称、processor的类型(卷积、激活、池化等)、processor的结果数据接收者或消费者或下一级processor等。在本实施方式中，每个功能模块，即processor对应于两个寄存器组，二者互为乒乓寄存器组。因此，本实施方式中可以利用上一次使用的寄存器组的id计算本次操作需要使用的寄存器组的id，由于只有两组寄存器组，因此id等于0或1。输入的组合数据(即输入数据)包括dependency描述tail_op、processor_group、状态描述stat_data_desc。三者囊括了processor运算所需要的的所有数据，包括：denpendency(用于更新pipline状态，自动使能processor)、operation(processor的操作数据)、surface_desc(输入/输出数据维度等信息描述)、surface(输入数据)等。
图10示出了一种dependency描述数据，其中包括本次processor操作的index(即索引表)、操作类型op_type、依赖计数dependency_count，以及本次processor的输入数据的生产者和输出数据的消费者。processor操作的dependency描述是AI engine内部各种processor pipeline的关键，每个processor操作都有dependency描述，每个denpendency描述都记录了本次操作的前一级与后一级，这样AI engine内部所有的processor就组成了一个链表。dependency_count(即执行等待计数)则决定了该processor是否使能。每个processor在执行了program()或enable()之后都会更新dependency，更新操作具体为dependency(即dependency_count)减1操作。当该processor的denpendency_count为0时，使能该processor。其中，processor的program()是对AI engine内部相应的模块配置寄存器的操作，而enable()是使能该processor，进行推理运算的操作。
请参考图11，其示出了一种具体的processor group描述数据的数据结构。processor的数据都是通过processor group传入的，AI engine内部的每个功能模块processor都设置了2组寄存器，以乒乓轮换的机制，无间隙控制processor连续运行，因此一个寄存器组的配置操作是在另外一组寄存器控制该processor运行时完成的，该方法缩短了AI engine执行任务的时间，提高了AI engine的运算效率。由于寄存器组控制processor运行，因此可以用processor group描述需要配置的寄存器组的具体信息和processor相关信息。需要说明的是，该信息需要在配置寄存器时生成，具体生成方式不做限定。具体的，图11中的id描述了寄存器组号，由于只有两组寄存器，所以id为0或1。active记录了当前processor是否被enable。event记录了当前processor发生中断后，中断寄存器置位的bit的编号，即processor运算完成后会将32位的中断寄存器中的event位置1。roi_index是当存在ROI时，指明当前roi所在的位置。is_rdma_needed是指当前processor操作是否需要从外部读取其他参数参与运算。pending是指当前processor是否有中断没有处理。lut_index是当processor是激活操作时，查找表的地址在address_list中的位置。本级denpendency用于描述当前processor的依赖，记录了依赖的相关信息。上一级dependency和下一级dependency，即parent和consumers的dependency。当前processor 在program()或enable()之后，都需要更新parent和consumers的dependency_count。这里需要注意，并不是上一级先执行，下一级后执行。如卷积->激活->池化，需要先使能‘池化’，然后使能‘激活’，最后使能‘卷积’，而三者基本是同时执行运算，因为三者是pipeline的，‘池化’需要‘激活’的结果作为输入，同样，‘激活’需要‘卷积’的结果作为输入。所以从整个task处理过程来看，三者是同时执行运算的。
operation_container与surface_container是processor的寄存器数据和输入/输出数据信息。具体请参考图12A、图12B、图13A、图13B和图13C，图12A为本申请实施例提供的一种寄存器描述数据的数据结构图的第一部分，图12B为本申请实施例提供的一种寄存器描述数据的数据结构图的第二部分，两个部分共同组成完整的寄存器描述数据的数据结构图。图13A为本申请实施例提供的一种输入输出方式描述数据的数据结构图，图13B为本申请实施例提供的一种卷积数据描述数据的具体数据结构图，图13C为本申请实施例提供的一种激活数据描述数据的具体数据结构图。
operation container包含了该processor group需要配置的所有寄存器的信息，operation_container包含n个processor的操作描述，n为AI engine内部功能模块processor的个数，本例中AI engine只有卷积、激活、池化。每个processor的操作描述的内容是有该processor的硬件设计决定的，如卷积processor设置了卷积模式、数据复用、数据类型、batch大小、数据buffer大小、卷积数据和kernel的大小、步长、是否padding、输入输出精度等。processor的program()操作就是把这些寄存器数据配置给AI engine的相应的功能模块。surface_container则描述了processor的输入/输出数据，AI engine内部有多少个processor，surface_container就需要包含多少个surface_desc。本实施例中，只有卷积、激活、池化三个processor。以卷积为例，conv_surface_desc包含了对输入数据、输出数据、权重数据和当存在数据压缩时的压缩数据的描述，描述从数据类型、数据地址、数据地址偏移、数据大小、数据维度、数据占用的空间等方面说明了数据信息，用于processor读取数据或写出数据。
综上，AI engine的描述数据包括了AI device->AI engine->processor->processor group的硬件信息描述，device task->engine task的任务描述，CNN描述，操作描述，数据描述。这些描述中包含了processor pipeline的信息，任务自动调度的信息和任务之间的依赖信息。整个描述是对硬件、CNN、任务、操作、数据的抽象，将复杂的任务调度逻辑和硬件配置逻辑抽象为数据的更新迭代。因此，整个CNN推理过程转换为AI engine的内部processor不断被调度、配置、使能执行的过程，dependency_desc描述了processor调用使能逻辑，operation_desc描述了processor的寄存器组配置信息，surface_desc描述了各个processor输入/输出数据的信息。这些信息都是Master根据硬件配置和已经训练完成的CNN通过编译器生成的。Master将这些数据写入AI engine的数据存储空间，然后将各种数据的地址组合成address_list，并把address_list的地址和address_list中的地址数目存放在device task中，Network_desc则描述了这些数据地址在address_list中的索引编号。Network_desc本身的地址则存放在address_list中0位置处。请参考图14，其示出了lenet模型转换为AI engine的processor计算流程的过程。INDEX即为processor的index，通过14次的4个processor的轮流执行，即可完成对lenet卷积神经网络的推理。
由此可见，任务调度器在获取到devcie task后需要先从address_list的0位置处获取Network_desc(即网络描述数据)，然后根据Network_desc中的索引编号从Address_list中获取相关数据的存储地址，并存放在engine_task中。AI Engine上的processor在执行推理时，从engine_task中读取地址，赋值给surface_desc中的数据address。processor根据数据的address、offset、size、数据类型，进而从存储空间中读取相关数据。
S102：基于网络描述数据，确定各个网络层任务之间的依赖关系。
对于任务调度器来说，其在进行任务调度时，需要按照网络层任务的依赖关系进行调度，依赖关系说明了各个网络层任务之间的先后顺序。本实施例并不限定确定依赖关系的具体方式，例如在一种实施方式中，从网络描述数据中确定各个网络层任务分别对应的网络描述子数据，并从各个网络描述子数据中解析得到对应的下一网络层指针(即上述的processor描述数据中的processor下一次操作的指针consumer_ptr)。基于各个下一网络层指针，得到依赖关系。
S103：检测人工智能加速计算部件上的空闲寄存器组。
如先前的描述，人工智能加速计算部件上各个功能模块分别对应于至少两个寄存器组，例如在一种优选实施方式中，每个processor对应于两个乒乓寄存器组。本申请中的空闲寄存器组是指无数据的寄存器组和存储有已经执行完毕的网络层任务对应的寄存器组数据的寄存器组。对于空闲寄存器组的检测可以实时执行、按照预设周期执行，或者在 检测到中断(即有网络层任务完成完毕)时执行，以便在利用其他非空闲寄存器组进行推理时进行寄存器组数据的加载，提高效率。
具体的，在一种实施方式中，若处于初始化阶段，则可以确定所有的寄存器组中都没有有效的数据，因此，可以将人工智能加速计算部件上的所有寄存器组确定为空闲寄存器组，进而对所有寄存器组进行配置。在初始化配置完毕后，还可以启动人工智能加速计算部件，以便开始进行模型推理。若不处于初始化阶段，且检测到目标网络层任务完成通知，则说明目标网络层任务执行完毕，其对应的寄存器组中的寄存器数据无效了，在这种情况下，首先确定目标网络层任务对应的第二功能模块。基于第二功能模块的类型，更新网络描述数据中与类型对应的空闲寄存器组标识(即id)，并根据更新后的空闲寄存器组标识，确定第二功能模块对应的空闲寄存器组。可以理解的是，空闲寄存器组为第二功能模块对应的两组乒乓寄存器组中的一个。
需要说明的是，在配置完毕后，需要首先使能各个功能模块，在使能完毕后，对于满足条件的功能模块(即S105中的第一功能模块)进行启动，在使能到启动的过程中，可以完成数据加载等操作，提高并行程度，进而提高计算效率。在一种实施方式中，可以基于依赖关系，确定人工智能加速计算部件上各个目标功能模块对应的使能顺序，目标功能模块可以为任意功能模块，使能顺序为与依赖关系相同方向的顺序。基于使能顺序的倒序，对各个目标功能模块进行使能处理，以使被使能处理后的目标功能模块基于对应的非空闲寄存器组进行数据计算准备操作。通过先使能后执行的功能模块，使得后启动的功能模块预先做好准确，在启动时可以直接进行数据处理，无需执行数据计算准确操作。数据计算准备操作可以为数据加载、变形等操作，具体不做限定。
S104：基于依赖关系、网络描述数据和空闲寄存器组类型，生成目标寄存器配置，并将目标寄存器配置写入到空闲寄存器组。
在检测到空闲寄存器组后，需要对其进行配置。本实施来并不限定具体的配置方式，在一种实施方式中，网络层任务的执行具有先后顺序，且每个功能模块仅能够执行其对应类型的网络层任务。在这种情况下，生成目标寄存器配置时，可以基于依赖关系，确定与空闲寄存器组类型匹配的候选网络层任务，空闲寄存器组类型即为对应的功能模块的类型，例如卷积、激活、池化等。根据该类型，可以确定类型匹配的候选任务，例如卷积层对应的任务，或池化层对应的任务。在网络描述数据中确定与候选网络层任务对应的候选描述子数据，即与该网络层任务相关的数据，进而利用候选描述子数据和人工智能加速计算部件对应的硬件架构描述数据，生成目标寄存器配置，具体可以为基于operation_container、suface_container等数据生成对应的寄存器配置数据。
S105：若检测到目标网络层任务完成通知，则基于依赖关系确定目标网络层任务完成通知对应的目标网络层任务的下一任务，并将下一任务在人工智能加速计算部件上对应的第一功能模块启动，以便执行下一任务。
若检测到目标网络层任务完成通知(即中断)，则说明目标网络层任务执行完毕，目标网络层任务可以为任意一个网络层任务。响应于检测到目标网络层任务完成通知，基于依赖关系确定目标网络层任务完成通知对应的目标网络层任务的下一任务。即，根据依赖关系，可以确定该任务对应的下一各网络层任务，即下一任务，由于目标网络层的推理为串行进行，前一个网络层任务的输出为下一各网络层任务的输入。对于一个目标神经网络来说，其对应的一个输入数据依次经过各个网络层任务的处理，即可得到一个输出。在一种实施方式中，为了提高效率，对于多个输入数据来说，每个输入数据可以分别对应于目标神经网络的不同网络层任务，在这种情况下，可以在一个时刻利用不同的功能模块对不同的输入数据进行处理，实现并行处理的效果，进而提高计算效率。但是对于一个输入数据来说，其参与的各个网络层任务均在不同的时刻执行，因此，对于一个输入数据来说，只要目标网络层不是最后一个网络层，则必然存在其对应的下一任务。
由于本申请中所有的寄存器组中的寄存器配置均为有效的数据(因为一旦检测到空闲寄存器就会对其进行配置)，因此下一任务对应的寄存器配置已经被写入，在这种情况下，可以直接执行该下一任务。具体的，根据下一任务的类型，可以确定其在人工智能加速计算部件上对应的第一功能模块，启动第一功能模块，第一功能模块即可根据寄存器配置执行该下一任务。本实施例并不限定确定下一任务的具体方式，在一种实施方式中，可以利用上述的dependency_count确定下一任务。具体的，可以解析目标网络层任务完成通知，得到目标网络层任务对应的任务标识(例如event编号)，并基于依赖关系，利用任务标识确定目标网络层任务对应的后续任务。后续任务，是指需要在目 标网络层任务之后执行的任务，具体数量不做限定。将各个后续任务对应的执行等待计数(dependency_count)减一，并将执行等待减一后为零的目标后续任务确定为下一任务，执行等待减一后为零即表明目标后续任务需要被立即执行。
请参考图15，图15为本申请实施例提供的一种具体的任务调度流程图。其中说明了处理刷的过程，包括AI engine task初始化和processor groups的初始化与寄存器组配置。具体的，AI engine task初始化包括：Master将任务数据下发至slave端的存储空间，并通过中断通知slave端处理器开始解析任务，并启动AI engine任务调度器，以便进行任务调度。调度器读取下发的device_task数据，获取address_list地址以及address_list中的地址数目。
processor groups的初始化与寄存器组配置包括：从address_list中的0位置处读取Network_desc的地址，并读取Network_desc数据。根据Network_desc描述的地址index，读取operation_des、surface_desc、dependency_desc、lut_list等地址，并用这些地址初始化engine_task数据结构。此外，调度器还可以从Network_desc中读取op_head，此op_head指示了每个processor的第一个denpendency_desc的index，并根据index和dependency_desc的数据大小，计算出当前processor的dependency_desc的偏移地址offset，然后根据engine_task中的dependency_graph_addr，得到读取dependency_desc的地址，然后读取数据。根据读出的dependency_desc获取当前processor的op_type，该op_type是processor的编号，若有n个processor，则0<＝op_type<＝n-1。调度器根据dependency_desc和processor获取processor group空闲的寄存器组的id和是否读取rdma的id，并根据dependency_desc中的index以及operation_desc数据类型大小、surface_desc数据类型大小，计算地址偏移offset；再根据engine_task中的operation_desc_addr和surface_desc_addr，计算出者两种数据的具体地址，并读取。调度器用以上读取的数据初始化当前processor group，并开始配置当前processor对应的AI engine的功能模块。调度器更新当前processor的代表末尾dependency_desc的tail_op为当前的依赖数据，用于寻找当前processor的consumers的index。将当前processor的dependency_count减1，并判断是否为0；若为0，则使能该processor对应的AI engine的功能模块，开始进行运算。调度器将当前processor group的id取反，按照以上步骤配置另外一组寄存器组，并按照以上步骤配置其他n-1个processor。
由此可见，运行完以上步骤，所有的processor的所有的寄存器组都已经配置完成，并使能了第一个应该运行的processor。配置过程中，可能会有中断产生，此时会进入中断服务程序，产生相应的中断event编号。但不会立即进入中断event处理程序。必须先完成所有的procesor的2个寄存器组的配置后，在处理中断event。这种机制通用性更强，普遍使用所有的架构不同的AI engine。
请参考图16，图16为本申请实施例提供的一种具体的中断处理流程图。中断服务程序是中断产生后立刻执行的程序，程序中会读取AI engine的中断状态寄存器，判断processor的类型，并产生相应的events编号。然后将中断状态寄存器清零。当前events会存储在processor group的event中，并将event_flag置1。具体的，events处理程序位于while(1)循环中，一直等到task执行完成后才跳出while循环。events处理程序位于process groups初始化与寄存器配置程序之后。当events处理程序发现event_flag为1时，首先清零event_flag，然后轮询所有的processor的两组processor group，判断哪个processor group产生了中断，并随之做相应处理。若是DMA读取image数据或weight数据中断，就会直接更新当前processor group的所有consumers的dependency。若是其他processor中断，首先会轮换group id，并更新所有comsumers的dependency。然后，调度器开始配置当前对group id对应的processor goup寄存器组。需要注意的是，此时按照以上逻辑，当前group的寄存器组的配置数据已经读取并processor group准备完成，这也是用processor运算过程遮掩processor配置过程的机制，以便processor可以无间隙转换两次任务的执行。调度器更新所有consumers的denpency，若有dependency_count为0的processor，则使能。调度器切换processor group，并运行processor groups的初始化与寄存器组配置过程中响应的初始化步骤，直到最后一个中断产生，运行的操作次数等于Netwok_desc中的操作次数，说明task完成。slave端产生中断通知Master读取最终结果。
综上，process group的consumer和parent把整个推理过程转换成了processor group的pipeline(流水线处理)，并在配置寄存器组后或中断产生后更新所有consumer和parent的processor group的dependency，并在dependency_count为0时，使能processor。
应用本申请实施例提供的计算任务调度方法，利用CNN是由多个相同或相似的卷积层，且卷积层内部的各种计算 可以pipeline(即流水线处理)的特点，结合AI engine的硬件配置，高效调度计算任务，提高计算效率。在获取网络描述数据后，若确定存在空闲寄存器组，则基于依赖关系和空闲寄存器组类型，利用网络描述数据生成与空闲寄存器组相匹配的目标寄存器配置，并将其写入到空闲寄存器组。此外，当检测到目标网络层任务完成通知，则根据依赖关系确定流水线处理的下一任务，由于在检测到空闲寄存器组后会立即对其进行配置，且每个功能模块对应于至少两个寄存器组，且寄存器的配置时长比功能模块利用寄存器进行数据处理的时长短，所以在任意时刻，每个功能模块都对应于至少一个非空闲寄存器组。因此，当检测到目标网络层任务完成且下一任务被确定后，可以立即启动下一任务对应的第一功能模块，使得能够无缝衔接的进行流水处理，提高计算效率；同时，由于各个功能模块的寄存器均被配置好，因此多个功能模块可以并行计算处理，充分利用硬件资源，提高计算效率；此外，寄存器的配置和功能模块的运行可以并行执行，在某一功能模块运行过程中，可以对空闲寄存器组进行配置，或者由其他寄存器组进行数据读写操作，充分利用总线带宽，避免突发性的大量数据传输需求导致的延迟，提高计算效率。
下面对本申请实施例提供的计算任务调度装置进行介绍，下文描述的计算任务调度装置与上文描述的计算任务调度方法可相互对应参照。
请参考图17，图17为本申请实施例提供的一种计算任务调度装置的结构示意图，应用于任务调度器，包括：
网络描述获取模块110，用于获取网络描述数据；网络描述数据用于描述目标神经网络中各个网络层分别对应的网络层任务；
依赖确定模块120，用于基于网络描述数据，确定各个网络层任务之间的依赖关系；
空闲检测模块130，用于检测人工智能加速计算部件上的空闲寄存器组；人工智能加速计算部件上各个功能模块分别对应于至少两个寄存器组；
寄存器配置模块140，用于基于依赖关系、网络描述数据和空闲寄存器组类型，生成目标寄存器配置，并将目标寄存器配置写入到空闲寄存器组；
启动模块150，用于若检测到目标网络层任务完成通知，则基于依赖关系确定目标网络层任务完成通知对应的目标网络层任务的下一任务，并将下一任务在人工智能加速计算部件上对应的第一功能模块启动，以便执行下一任务。
可选地，包括：
硬件架构确定模块，用于确定人工智能加速计算部件的硬件架构；
硬件描述生成模块，用于基于硬件架构，生成对应的硬件架构描述数据；
网络描述生成模块，用于获取目标神经网络，并基于目标神经网络和硬件架构描述数据，生成网络描述数据。
可选地，网络描述生成模块，包括：
模型解析单元，用于对目标神经网络进行模型解析，得到中间图；
优化单元，用于基于硬件架构描述数据，对中间图进行图优化处理，得到优化图；
编译单元，用于对优化图进行编译处理，得到网络描述数据。
可选地，网络描述生成模块，包括：
依赖确定单元，用于确定目标神经网络中的各个网络层，以及网络层之间的依赖关系；
获取单元，用于获取各个网络层分别对应的预设网络层描述数据；
生成单元，用于利用预设网络层描述数据，基于依赖关系，生成网络描述数据。
可选地，依赖确定模块120，包括：
子数据确定单元，用于从网络描述数据中确定各个网络层任务分别对应的网络描述子数据；
指针获取单元，用于从各个网络描述子数据中解析得到对应的下一网络层指针；
关系生成单元，用于基于各个下一网络层指针，得到依赖关系。
可选地，空闲检测模块130，包括：
初始检测单元，用于若处于初始化阶段，则将人工智能加速计算部件上的所有寄存器组确定为空闲寄存器组；
模块确定单元，用于若不处于初始化阶段，且检测到目标网络层任务完成通知，则确定目标网络层任务对应的第 二功能模块；
标识获取单元，用于基于第二功能模块的类型，更新网络描述数据中与类型对应的空闲寄存器组标识；
空闲确定单元，用于根据更新后的空闲寄存器组标识，确定第二功能模块对应的空闲寄存器组；其中，空闲寄存器组为第二功能模块对应的两组乒乓寄存器组中的一个。
可选地，寄存器配置模块140，包括：
候选确定单元，用于基于依赖关系，确定与空闲寄存器组类型匹配的候选网络层任务；
子数据获取单元，用于在网络描述数据中确定与候选网络层任务对应的候选描述子数据；
配置生成单元，用于利用候选描述子数据和人工智能加速计算部件对应的硬件架构描述数据，生成目标寄存器配置。
可选地，启动模块150，包括：
标识确定单元，用于解析目标网络层任务完成通知，得到目标网络层任务对应的任务标识；
后续任务确定单元，用于基于依赖关系，利用任务标识确定目标网络层任务对应的后续任务；
更新单元，用于将各个后续任务对应的执行等待计数减一；
下一任务确定单元，用于将执行等待减一后为零的目标后续任务确定为下一任务。
可选地，还包括：
使能顺序确定模块，用于基于依赖关系，确定人工智能加速计算部件上各个目标功能模块对应的使能顺序；
使能模块，用于基于使能顺序的倒序，对各个目标功能模块进行使能处理，以使被使能处理后的目标功能模块基于对应的非空闲寄存器组进行数据计算准备操作。
下面对本申请实施例提供的电子设备进行介绍，下文描述的电子设备与上文描述的计算任务调度方法可相互对应参照。
请参考图18，图18为本申请实施例提供的一种电子设备的结构示意图。其中电子设备100可以包括一个或多个处理器101和存储器102，还可以进一步包括多媒体组件103、信息输入/信息输出(I/O)接口104以及通信组件105中的一种或多种。
其中，处理器101用于控制电子设备100的整体操作，以完成上述的计算任务调度方法中的全部或部分步骤；存储器102用于存储各种类型的数据以支持在电子设备100的操作，这些数据例如可以包括用于在该电子设备100上操作的任何应用程序或方法的指令，以及应用程序相关的数据。该存储器102可以由任何类型的易失性或非易失性存储设备或者它们的组合实现，例如静态随机存取存储器(Static Random Access Memory，SRAM)、电可擦除可编程只读存储器(Electrically Erasable Programmable Read-Only Memory，EEPROM)、可擦除可编程只读存储器(Erasable Programmable Read-Only Memory，EPROM)、可编程只读存储器(Programmable Read-Only Memory，PROM)、只读存储器(Read-Only Memory，ROM)、磁存储器、快闪存储器、磁盘或光盘中的一种或多种。
多媒体组件103可以包括屏幕和音频组件。其中屏幕例如可以是触摸屏，音频组件用于输出和/或输入音频信号。例如，音频组件可以包括一个麦克风，麦克风用于接收外部音频信号。所接收的音频信号可以被进一步存储在存储器102或通过通信组件105发送。音频组件还包括至少一个扬声器，用于输出音频信号。I/O接口104为处理器101和其他接口模块之间提供接口，上述其他接口模块可以是键盘，鼠标，按钮等。这些按钮可以是虚拟按钮或者实体按钮。通信组件105用于电子设备100与其他设备之间进行有线或无线通信。无线通信，例如Wi-Fi，蓝牙，近场通信(Near Field Communication，简称NFC)，2G、3G或4G，或它们中的一种或几种的组合，因此相应的该通信组件105可以包括：Wi-Fi部件，蓝牙部件，NFC部件。
电子设备100可以被一个或多个应用专用集成电路(Application Specific Integrated Circuit，简称ASIC)、数字信号处理器(Digital Signal Processor，简称DSP)、数字信号处理设备(Digital Signal Processing Device，简称DSPD)、可编程逻辑器件(Programmable Logic Device，简称PLD)、现场可编程门阵列(Field Programmable Gate Array，简称FPGA)、控制器、微控制器、微处理器或其他电子元件实现，用于执行上述实施例给出的计算任务调度方法。
下面对本申请实施例提供的计算机可读存储介质进行介绍，下文描述的计算机可读存储介质与上文描述的计算任务调度方法可相互对应参照。
本申请还提供一个或多个存储有计算机可读指令的非易失性计算机可读存储介质，计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器执行计算任务调度方法的步骤，计算机可读存储介质上存储有计算机可读指令，计算机可读指令被处理器执行时实现上述的计算任务调度方法的步骤。
该计算机可读存储介质可以包括：U盘、移动硬盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
本说明书中各个实施例采用递进的方式描述，每个实施例重点说明的都是与其它实施例的不同之处，各个实施例之间相同或相似部分互相参见即可。对于实施例公开的装置而言，由于其与实施例公开的方法相对应，所以描述的比较简单，相关之处参见方法部分说明即可。
本领域技术人员还可以进一步意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、计算机软件或者二者的结合来实现，为了清楚地说明硬件和软件的可互换性，在上述说明中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以硬件还是软件的方式来执行，取决于技术方案的特定应用和设计约束条件。本领域技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应该认为超出本申请的范围。
结合本文中所公开的实施例描述的方法或算法的步骤可以直接用硬件、处理器执行的软件模块，或者二者的结合来实施。软件模块可以置于随机存储器(RAM)、内存、只读存储器(ROM)、电可编程ROM、电可擦除可编程ROM、寄存器、硬盘、可移动磁盘、CD-ROM、或技术领域内所公知的任意其它形式的存储介质中。
最后，还需要说明的是，在本文中，诸如第一和第二等之类的关系属于仅仅用来将一个实体或者操作与另一个实体或者操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语包括、包含或者其他任何变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。
本文中应用了具体个例对本申请的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本申请的方法及其核心思想；同时，对于本领域的一般技术人员，依据本申请的思想，在具体实施方式及应用范围上均会有改变之处，综上，本说明书内容不应理解为对本申请的限制。

Claims (13)

  1. 一种计算任务调度方法，其特征在于，应用于任务调度器，所述方法包括：
    获取网络描述数据；所述网络描述数据用于描述目标神经网络中各个网络层分别对应的网络层任务；
    基于所述网络描述数据，确定各个所述网络层任务之间的依赖关系；
    检测人工智能加速计算部件上的空闲寄存器组；所述人工智能加速计算部件上各个功能模块分别对应于至少两个寄存器组；
    基于所述依赖关系、所述网络描述数据和空闲寄存器组类型，生成目标寄存器配置，并将所述目标寄存器配置写入到所述空闲寄存器组；和
    响应于检测到目标网络层任务完成通知，基于所述依赖关系确定所述目标网络层任务完成通知对应的目标网络层任务的下一任务，并将所述下一任务在所述人工智能加速计算部件上对应的第一功能模块启动，以便执行所述下一任务。
  2. 根据权利要求1所述的计算任务调度方法，其特征在于，所述网络描述数据的生成过程，包括：
    确定所述人工智能加速计算部件的硬件架构；
    基于所述硬件架构，生成对应的硬件架构描述数据；和
    获取所述目标神经网络，并基于所述目标神经网络和所述硬件架构描述数据，生成所述网络描述数据。
  3. 根据权利要求2所述的计算任务调度方法，其特征在于，所述基于所述目标神经网络和所述硬件架构描述数据，生成所述网络描述数据，包括：
    对所述目标神经网络进行模型解析，得到中间图；
    基于所述硬件架构描述数据，对所述中间图进行图优化处理，得到优化图；和
    对所述优化图进行编译处理，得到所述网络描述数据。
  4. 根据权利要求2所述的计算任务调度方法，其特征在于，所述基于所述目标神经网络和所述硬件架构描述数据，生成所述网络描述数据，包括：
    确定所述目标神经网络中的各个网络层，以及所述网络层之间的所述依赖关系；
    获取各个网络层分别对应的预设网络层描述数据；和
    利用所述预设网络层描述数据，基于所述依赖关系，生成所述网络描述数据。
  5. 根据权利要求1-4中任一项所述的计算任务调度方法，其特征在于，所述基于所述网络描述数据，确定各个所述网络层任务之间的依赖关系，包括：
    从所述网络描述数据中确定各个所述网络层任务分别对应的网络描述子数据；
    从各个所述网络描述子数据中解析得到对应的下一网络层指针；和
    基于各个所述下一网络层指针，得到所述依赖关系。
  6. 根据权利要求1-5中任一项所述的计算任务调度方法，其特征在于，所述检测人工智能加速计算部件上的空闲寄存器组，包括：
    响应于处于初始化阶段，将所述人工智能加速计算部件上的所有寄存器组确定为所述空闲寄存器组。
  7. 根据权利要求1-5中任一项所述的计算任务调度方法，其特征在于，所述检测人工智能加速计算部件上的空闲寄存器组，包括：
    响应于不处于所述初始化阶段，检测到目标网络层任务完成通知，确定所述目标网络层任务对应的第二功能模块；
    基于所述第二功能模块的类型，更新所述网络描述数据中与所述类型对应的空闲寄存器组标识；和
    根据更新后的所述空闲寄存器组标识，确定所述第二功能模块对应的所述空闲寄存器组；其中，所述空闲寄存器组为所述第二功能模块对应的两组乒乓寄存器组中的一个。
  8. 根据权利要求1-7中任一项所述的计算任务调度方法，其特征在于，所述基于所述依赖关系、所述网络描述数据和空闲寄存器组类型，生成目标寄存器配置，包括：
    基于所述依赖关系，确定与所述空闲寄存器组类型匹配的候选网络层任务；
    在所述网络描述数据中确定与所述候选网络层任务对应的候选描述子数据；和
    利用所述候选描述子数据和所述人工智能加速计算部件对应的硬件架构描述数据，生成所述目标寄存器配置。
  9. 根据权利要求1-8中任一项所述的计算任务调度方法，其特征在于，所述基于所述依赖关系确定所述目标网络层任务完成通知对应的目标网络层任务的下一任务，包括：
    解析所述目标网络层任务完成通知，得到所述目标网络层任务对应的任务标识；
    基于所述依赖关系，利用所述任务标识确定所述目标网络层任务对应的后续任务；
    将各个所述后续任务对应的执行等待计数减一；和
    将所述执行等待减一后为零的目标后续任务确定为所述下一任务。
  10. 根据权利要求1-9中任一项所述的计算任务调度方法，其特征在于，还包括：
    基于所述依赖关系，确定所述人工智能加速计算部件上各个目标功能模块对应的使能顺序；和
    基于所述使能顺序的倒序，对各个所述目标功能模块进行使能处理，以使被使能处理后的目标功能模块基于对应的非空闲寄存器组进行数据计算准备操作。
  11. 一种计算任务调度装置，其特征在于，应用于任务调度器，所述装置包括：
    网络描述获取模块，用于获取网络描述数据；所述网络描述数据用于描述目标神经网络中各个网络层分别对应的网络层任务；
    依赖确定模块，用于基于所述网络描述数据，确定各个所述网络层任务之间的依赖关系；
    空闲检测模块，用于检测人工智能加速计算部件上的空闲寄存器组；所述人工智能加速计算部件上各个功能模块分别对应于至少两个寄存器组；
    寄存器配置模块，用于基于所述依赖关系、所述网络描述数据和空闲寄存器组类型，生成目标寄存器配置，并将所述目标寄存器配置写入到所述空闲寄存器组；和
    启动模块，用于响应于检测到目标网络层任务完成通知，基于所述依赖关系确定所述目标网络层任务完成通知对应的目标网络层任务的下一任务，并将所述下一任务在所述人工智能加速计算部件上对应的第一功能模块启动，以便执行所述下一任务。
  12. 一种电子设备，其特征在于，包括存储器及一个或多个处理器，所述存储器中储存有计算机可读指令，所述计算机可读指令被所述一个或多个处理器执行时，使得所述一个或多个处理器执行如权利要求1-10任意一项所述的方法的步骤。
  13. 一个或多个存储有计算机可读指令的非易失性计算机可读存储介质，其特征在于，所述计算机可读指令被一个或多个处理器执行时，使得所述一个或多个处理器执行如权利要求1-10任意一项所述的方法的步骤。
PCT/CN2022/102769 2022-04-20 2022-06-30 一种计算任务调度方法、装置、电子设备及可读存储介质 Ceased WO2023201893A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/725,590 US12498971B2 (en) 2022-04-20 2022-06-30 Determination of a next task of a target network layer task for a task scheduling based on dependencies and register configuration

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210413206.1A CN114510339B (zh) 2022-04-20 2022-04-20 一种计算任务调度方法、装置、电子设备及可读存储介质
CN202210413206.1 2022-04-20

Publications (1)

Publication Number Publication Date
WO2023201893A1 true WO2023201893A1 (zh) 2023-10-26

Family

ID=81554635

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/102769 Ceased WO2023201893A1 (zh) 2022-04-20 2022-06-30 一种计算任务调度方法、装置、电子设备及可读存储介质

Country Status (3)

Country Link
US (1) US12498971B2 (zh)
CN (1) CN114510339B (zh)
WO (1) WO2023201893A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117290074A (zh) * 2023-11-23 2023-12-26 知行汽车科技(苏州)股份有限公司 一种车型配置中的进程设置方法、装置、设备及介质

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114510339B (zh) 2022-04-20 2022-07-29 苏州浪潮智能科技有限公司 一种计算任务调度方法、装置、电子设备及可读存储介质
CN114860412B (zh) * 2022-05-19 2023-04-04 北京百度网讯科技有限公司 任务处理方法、装置、电子设备和介质
CN115801424B (zh) * 2022-11-29 2025-08-26 北京威努特技术有限公司 一种流量快速转发的方法
CN118550659A (zh) * 2023-02-27 2024-08-27 华为技术有限公司 一种任务运行方法、装置及相关设备
CN117971437B (zh) * 2024-03-26 2025-01-24 摩尔线程智能科技(北京)股份有限公司 任务分配方法、电路、设备、介质及程序

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6871343B1 (en) * 1999-09-22 2005-03-22 Kabushiki Kaisha Toshiba Central processing apparatus and a compile method
CN112711475A (zh) * 2021-01-20 2021-04-27 上海交通大学 一种基于图卷积神经网络的工作流调度方法及系统
US20210157283A1 (en) * 2019-11-26 2021-05-27 Apex Artificial Intelligence Industries, Inc. Adaptively controlling groups of automated machines
CN113448692A (zh) * 2020-03-25 2021-09-28 杭州海康威视数字技术股份有限公司 分布式图计算的方法、装置、设备及存储介质
CN113961333A (zh) * 2021-12-22 2022-01-21 北京燧原智能科技有限公司 循环任务的生成、执行方法、装置、ai芯片及存储介质
CN114510339A (zh) * 2022-04-20 2022-05-17 苏州浪潮智能科技有限公司 一种计算任务调度方法、装置、电子设备及可读存储介质

Family Cites Families (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001188767A (ja) * 1999-12-28 2001-07-10 Fuji Xerox Co Ltd ニューラルネットワーク演算装置及びニューラルネットワークの演算方法
JP4204522B2 (ja) * 2004-07-07 2009-01-07 株式会社東芝 マイクロプロセッサ
US8423728B1 (en) * 2005-06-16 2013-04-16 Emc Corporation Physical based scheduler using run count and skip count values
US7508396B2 (en) * 2005-09-28 2009-03-24 Silicon Integrated Systems Corp. Register-collecting mechanism, method for performing the same and pixel processing system employing the same
JP4781133B2 (ja) * 2006-03-08 2011-09-28 コニカミノルタビジネステクノロジーズ株式会社 画像形成装置、情報処理装置、画像処理プログラム、および画像処理方法
TWI396132B (zh) * 2008-08-06 2013-05-11 Nat Univ Tsing Hua 非規則性暫存器集之指令管線化方法
US8868695B2 (en) * 2011-03-02 2014-10-21 Cleversafe, Inc. Configuring a generic computing device utilizing specific computing device operation information
US9154451B2 (en) * 2012-08-21 2015-10-06 Advanced Micro Devices, Inc. Systems and methods for sharing devices in a virtualization environment
JP6079208B2 (ja) * 2012-12-19 2017-02-15 株式会社ソシオネクスト マイクロコンピュータ，そのミドルウエア及びマイクロコンピュータの動作方法
US9672020B2 (en) * 2014-09-19 2017-06-06 Microsoft Technology Licensing, Llc Selectively loading precompiled header(s) and/or portion(s) thereof
CN105653258A (zh) * 2015-11-06 2016-06-08 乐视移动智能信息技术（北京）有限公司 一种代码处理方法及装置
CN105893159B (zh) * 2016-06-21 2018-06-19 北京百度网讯科技有限公司 数据处理方法和装置
CN108475347A (zh) * 2017-11-30 2018-08-31 深圳市大疆创新科技有限公司 神经网络处理的方法、装置、加速器、系统和可移动设备
CN109978129B (zh) * 2017-12-28 2020-08-25 中科寒武纪科技股份有限公司 调度方法及相关装置
US11636327B2 (en) * 2017-12-29 2023-04-25 Intel Corporation Machine learning sparse computation mechanism for arbitrary neural networks, arithmetic compute microarchitecture, and sparsity for training mechanism
US10719970B2 (en) * 2018-01-08 2020-07-21 Apple Inc. Low latency firmware command selection using a directed acyclic graph
US20200175396A1 (en) * 2018-12-04 2020-06-04 Intel Corporation Deterministic neural networking interoperability
CN109754073B (zh) * 2018-12-29 2020-03-10 中科寒武纪科技股份有限公司 数据处理方法、装置、电子设备和可读存储介质
CN112291293B (zh) * 2019-07-27 2023-01-06 华为技术有限公司 任务处理方法、相关设备及计算机存储介质
US20210073669A1 (en) * 2019-09-06 2021-03-11 American Express Travel Related Services Company Generating training data for machine-learning models
US11442794B1 (en) * 2019-09-27 2022-09-13 Amazon Technologies, Inc. Event assignment for synchronization of concurrent execution engines
US11797277B2 (en) * 2019-10-22 2023-10-24 Shenzhen Corerain Technologies Co., Ltd. Neural network model conversion method server, and storage medium
TWI727493B (zh) * 2019-11-08 2021-05-11 瑞昱半導體股份有限公司 一種閘道器控制晶片及其網路封包處理方法
US11068641B1 (en) * 2020-03-04 2021-07-20 Mythic, Inc. Systems and methods for intelligently buffer tracking for optimized dataflow within an integrated circuit architecture
US11080026B1 (en) * 2020-07-28 2021-08-03 Electronic Arts Inc. Combined building of dual representation program instructions
US12198037B2 (en) * 2020-11-06 2025-01-14 Samsung Electronics Co., Ltd. Hardware architecture determination based on a neural network and a network compilation process
US12574782B2 (en) * 2021-01-05 2026-03-10 Intel Corporation Network controlled small gap (NCSG) configurations to reduce interruptions due to intra-rat bandwidth part (BWP) transitions
CN112949847B (zh) * 2021-03-29 2023-07-25 上海西井科技股份有限公司 神经网络算法加速系统、调度系统及调度方法
KR20230040757A (ko) * 2021-09-16 2023-03-23 삼성전자주식회사 모델 메타 데이터 및 제어 데이터에 기초하여 뉴럴 네트워크 연산을 수행하는 전자 장치 및 뉴럴 네트워크 모듈
CN114239816B (zh) * 2021-12-09 2023-04-07 电子科技大学 一种卷积神经网络-图卷积神经网络可重构硬件加速架构

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6871343B1 (en) * 1999-09-22 2005-03-22 Kabushiki Kaisha Toshiba Central processing apparatus and a compile method
US20210157283A1 (en) * 2019-11-26 2021-05-27 Apex Artificial Intelligence Industries, Inc. Adaptively controlling groups of automated machines
CN113448692A (zh) * 2020-03-25 2021-09-28 杭州海康威视数字技术股份有限公司 分布式图计算的方法、装置、设备及存储介质
CN112711475A (zh) * 2021-01-20 2021-04-27 上海交通大学 一种基于图卷积神经网络的工作流调度方法及系统
CN113961333A (zh) * 2021-12-22 2022-01-21 北京燧原智能科技有限公司 循环任务的生成、执行方法、装置、ai芯片及存储介质
CN114510339A (zh) * 2022-04-20 2022-05-17 苏州浪潮智能科技有限公司 一种计算任务调度方法、装置、电子设备及可读存储介质

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117290074A (zh) * 2023-11-23 2023-12-26 知行汽车科技(苏州)股份有限公司 一种车型配置中的进程设置方法、装置、设备及介质
CN117290074B (zh) * 2023-11-23 2024-02-02 知行汽车科技(苏州)股份有限公司 一种车型配置中的进程设置方法、装置、设备及介质

Also Published As

Publication number Publication date
US20240385878A1 (en) 2024-11-21
US12498971B2 (en) 2025-12-16
CN114510339B (zh) 2022-07-29
CN114510339A (zh) 2022-05-17

Similar Documents

Publication Publication Date Title
WO2023201893A1 (zh) 一种计算任务调度方法、装置、电子设备及可读存储介质
US9043770B2 (en) Program module applicability analyzer for software development and testing for multi-processor environments
US20140006751A1 (en) Source Code Level Multistage Scheduling Approach for Software Development and Testing for Multi-Processor Environments
US20140007044A1 (en) Source Code Generator for Software Development and Testing for Multi-Processor Environments
KR20210021261A (ko) 가속기에서 이종 컴포넌트들을 구성하기 위한 방법들 및 장치
US11016769B1 (en) Method and apparatus for processing information
US12210438B1 (en) Breakpoints in neural network accelerator
CN116107669B (zh) 深度学习框架的算子注册方法、装置、设备及存储介质
WO2008113291A1 (en) Method and system for calling functions
WO2023124543A1 (zh) 用于大数据的数据处理方法和数据处理装置
JP2025521149A (ja) シミュレーション装置、シミュレーションシステム及びそのシミュレーション方法、記憶媒体
US20200226051A1 (en) Generating a debugging network for a synchronous digital circuit during compilation of program source code
CN109558226B (zh) 一种基于核间中断的dsp多核并行计算调度方法
WO2025124578A1 (zh) 张量处理装置、方法及计算机可读存储介质
CN119781850A (zh) 基于dpu异构架构的向量化执行引擎的算子卸载方法及系统
CN113918290B (zh) 一种api调用方法以及装置
CN116225821A (zh) 支持软件定义的性能监控事件的可编程性能监控单元
CN107820605A (zh) 用于动态低延迟优化的系统和方法
CN116107728A (zh) 一种任务执行方法、装置、存储介质及电子设备
CN115202666A (zh) 数据流架构优化方法、装置、电子设备
CN114880290A (zh) 一种虚拟文件系统的监控方法、监控装置及计算设备
US10803007B1 (en) Reconfigurable instruction
WO2025179868A1 (zh) 嵌入式系统的硬件资源访问方法、装置及服务器
CN100576175C (zh) 用于多个内核的并行执行的方法和系统
CN103345392B (zh) 处理信息的方法及其装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22938119

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 18725590

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22938119

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205 DATED 14/03/2025)

WWG Wipo information: grant in national office

Ref document number: 18725590

Country of ref document: US

122 Ep: pct application non-entry in european phase

Ref document number: 22938119

Country of ref document: EP

Kind code of ref document: A1