WO2024087513A1 - 应用场景的数据处理方法、系统、电子设备及存储介质 - Google Patents

应用场景的数据处理方法、系统、电子设备及存储介质 Download PDF

Info

Publication number
WO2024087513A1
WO2024087513A1 PCT/CN2023/085999 CN2023085999W WO2024087513A1 WO 2024087513 A1 WO2024087513 A1 WO 2024087513A1 CN 2023085999 W CN2023085999 W CN 2023085999W WO 2024087513 A1 WO2024087513 A1 WO 2024087513A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
computing
npu
cores
idle
Prior art date
Application number
PCT/CN2023/085999
Other languages
English (en)
French (fr)
Inventor
沈正福
黄同高
谢武锋
周玮
Original Assignee
深圳市中兴微电子技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市中兴微电子技术有限公司 filed Critical 深圳市中兴微电子技术有限公司
Publication of WO2024087513A1 publication Critical patent/WO2024087513A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means

Definitions

  • the present application relates to the field of artificial intelligence technology, and in particular to a data processing method, system, electronic device and storage medium for application scenarios.
  • AI Artificial Intelligence
  • CPU Central Processing Unit
  • GPU Graphics Processing Unit
  • NPU Neurological Network Processing Unit
  • NPU's simultaneous support for multiple operating systems has become an increasingly important performance indicator for NPU design.
  • OS Operating System
  • most NPU manufacturers currently still use hardware isolation to support multiple OS (Operating System), which strongly binds NPU computing resources to AI application scenarios.
  • OS Operating System
  • the computing resources bound to the specified AI application scenario remain idle, which seriously reduces the NPU's operating computing power in the actual operating environment of the AI application scenario.
  • the main purpose of this application is to provide a data processing method, system, electronic device and storage medium for an application scenario, aiming to solve the technical problem that the NPU computing power resources are strongly bound to the application scenario, resulting in low NPU operating computing power.
  • the present application provides a data processing method for an application scenario, including:
  • the task execution data is distributed to the required number of idle computing cores for computing, computing result data is obtained, and task response of the current application scenario is performed according to the computing result data.
  • the present application provides a data processing system for an application scenario, wherein the data processing system for the application scenario includes a central processing unit and a neural network processor connected to each other, wherein the central processing unit includes multiple virtual machine VM modules, and the neural network processor includes an NPU computing core, an NPU control module, and a virtualization control VMU module:
  • Each of the VM modules is respectively bound to at least one AI application, wherein one AI application maps to one application scenario;
  • the VMU module is configured to obtain task execution data generated by the AI application in the VM module, wherein the task execution data carries the required number of NPU computing cores required to be used;
  • the NPU control module is configured to detect the operation core running status information of each of the NPU operation cores, determine the idle operation cores in the NPU operation cores that are in an idle state according to the operation core running status information, and distribute the task execution data to the required number of idle operation cores for operation to obtain operation result data;
  • the VM module is configured to perform a task response for a current application scenario according to the operation result data.
  • the present application also provides an electronic device, which includes a central processing unit, a neural network processor, a memory as described above, and a data processing program for an application scenario stored on the memory and executable on the central processing unit and/or the neural network processor.
  • a data processing program for an application scenario stored on the memory and executable on the central processing unit and/or the neural network processor.
  • the present application also provides a readable storage medium, which is a computer-readable storage medium, and a data processing program for an application scenario is stored on the computer-readable storage medium.
  • a data processing program for an application scenario is executed by a processor, the steps of the data processing method for the application scenario as described above are implemented.
  • the embodiments of the present application propose a data processing method, system, electronic device and storage medium for an application scenario.
  • the technical solution of the embodiments of the present application is to obtain task execution data corresponding to the current application scenario (for example, smart vehicles may include application scenarios such as smart cockpits and assisted driving, smart phones may include application scenarios such as image recognition and voice recognition, and smart homes may include application scenarios such as linkage control and automatic adjustment of operating parameters), wherein the task execution data carries the required number of NPU computing cores required to be used, and detects the computing core operating status information of each NPU computing core, determines the idle computing cores in each NPU computing core in an idle state according to the computing core operating status information, and then distributes the task execution data to the required number of idle computing cores for calculation, obtains calculation result data, and then performs task response for the current application scenario according to the calculation result data.
  • the task execution data carries the required number of NPU computing cores required to be used, and detects the computing core operating status information of each NPU computing core, determines the idle
  • a hardware isolation method is used to achieve support for multiple OS (Operating System).
  • the hardware isolation method can be: strongly bind the NPU computing core in the NPU chip to the application scenario.
  • an NPU chip includes three NPU computing cores, among which NPU computing core 1 is bound to application scenario a, and NPU computing core 2 and NPU computing core 3 are bound to application scenario b.
  • application scenario a is enabled and application scenario b is not enabled, the computing power resources of NPU computing core 2 and NPU computing core 3 are idle, and application scenario a is only supported by running NPU computing core 1.
  • the embodiment of the present application determines the required number of NPU computing cores according to the task execution data of the specific AI application scenario, and selects the required number of idle NPU computing cores from each NPU computing core by combining the running status information of each NPU computing core to run and support the current application scenario, thereby realizing dynamic scheduling of NPU computing resources according to the specific AI application scenario, avoiding the strong binding of NPU computing resources and application scenarios, which results in the computing resources bound to the specified application scenario being idle when the specified application scenario is not enabled, causing the NPU to have reduced operating performance in the actual operating environment.
  • the embodiment of the present application provides an NPU multi-OS support mechanism that can dynamically allocate NPU core resources, which can reuse NPU computing cores in different application scenarios, thereby improving the utilization rate of NPU computing resources, and further solving the technical problem of low NPU computing power caused by the strong binding of NPU computing resources and application scenarios.
  • FIG1 is a flow chart of a first embodiment of a data processing method for an application scenario of the present application
  • FIG2 is a flow chart of a second embodiment of a data processing method for an application scenario of the present application
  • FIG3 is a flow chart of a third embodiment of a data processing method for an application scenario of the present application.
  • FIG4 is a schematic diagram of a structural module of a data processing system for an application scenario of an embodiment of the present application
  • FIG5 is a scene interaction diagram of a data processing system for an application scenario of an embodiment of the present application.
  • FIG6 is a schematic diagram of the module structure of the VMU according to an embodiment of the present application.
  • FIG7 is a timing diagram of a data processing method for an application scenario of an embodiment of the present application.
  • FIG. 8 is a schematic diagram of a task list mapping for physical address synchronization between a host side and a device side in an embodiment of the present application
  • FIG9 is a schematic diagram of a flow chart of performing a convolution operation on task execution data according to an embodiment of the present application.
  • FIG. 10 is a schematic diagram of the structure of an electronic device involved in an embodiment of the present application.
  • connection can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined.
  • fixation can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined.
  • NPU Neurological Network Processing Unit
  • NPU reduces the hardened design of control instructions and uses the MAC array as the main computing core.
  • MAC array designs of more than 10,000 on the market.
  • some NPUs have theoretical computing capabilities of hundreds of TOPs.
  • NPU provides a basic guarantee for the actual implementation of more AI scenarios.
  • the present application proposes a data processing method for an application scenario.
  • the data processing method for the application scenario of the present application please refer to FIG. 1 .
  • the data processing method for the application scenario includes:
  • Step S100 obtaining task execution data corresponding to the current application scenario, wherein the task execution data carries the required number of NPU computing cores required to be used;
  • the data processing method of the application scenario refers to the data processing method of the AI (Artificial Intelligence) application scenario, or more specifically, it can be an application scenario based on a neural network algorithm.
  • AI Artificial Intelligence
  • AI It is a technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to achieve the best results.
  • AI can specifically include computer vision technology, speech processing technology, natural language processing technology, and machine learning/deep learning.
  • the data processing method of the application scenario of the present application is applied to the vehicle field, and the types of the AI application scenarios include but are not limited to intelligent assisted driving ADAS (Advanced Driving Assistance System), intelligent cockpit and intelligent cabin air conditioning, etc.
  • the data processing method of the application scenario of the present application is applied to the mobile terminal field, and the types of the AI application scenarios include but are not limited to intelligent voice services, intelligent photography and facial recognition of mobile phones, etc.
  • the data processing method of the application scenario of the present application is applied to the home field, and the types of the AI application scenarios include but are not limited to intelligent air conditioners, intelligent refrigerators, intelligent lamps and intelligent range hoods, etc.
  • the data processing method of the application scenario of the present application can also be applied to the application scenarios supported by NPU multi-operating systems in the fields of drones (such as AI application scenarios such as path planning and intelligent obstacle avoidance), medical devices (such as AI application scenarios such as intelligent identification of infusion types and infusion bubble detection), and robots (such as AI application scenarios such as ground cleaning tasks and voice assistant services), which are not specifically limited in this embodiment.
  • drones such as AI application scenarios such as path planning and intelligent obstacle avoidance
  • medical devices such as AI application scenarios such as intelligent identification of infusion types and infusion bubble detection
  • robots such as AI application scenarios such as ground cleaning tasks and voice assistant services
  • the corresponding task execution data may be the driving environment information detected in real time by the sensors on the vehicle during the driving of the car.
  • the corresponding task execution data may be the face image captured by the camera on the vehicle, and the driving condition information obtained by the wireless communication module on the vehicle.
  • the corresponding task execution data may be the air parameter information inside and outside the cabin collected by the sensors on the vehicle (which may include carbon dioxide concentration, PM2.5 concentration, and carbon monoxide concentration, etc.).
  • Step S200 detecting the operation core running status information of each of the NPU operation cores, and determining an idle operation core in an idle state among the NPU operation cores according to the operation core running status information;
  • the computing core running status information refers to the running status information of whether the NPU computing core is in a working state or an idle state, or the running status information of whether the thread of the NPU computing core is occupied by a work task. Therefore, based on the computing core running status information, the idle computing cores in the idle state among the current NPU computing cores can be determined, and of course, the working computing cores in the working state among the current NPU computing cores can also be determined at the same time.
  • Step S300 distribute the task execution data to the required number of idle computing cores for computing, obtain computing result data, and perform task response of the current application scenario according to the computing result data.
  • the data processing method of the application scenario of the present application is applied to a vehicle.
  • the type of the current application scenario is intelligent assisted driving and the task execution data is the driving environment information detected in real time by the sensors mounted on the vehicle during the driving of the car
  • the task execution data can be allocated to the required number of idle computing cores for calculation to obtain calculation result data.
  • the calculation result data can be the result data of the identification, detection and tracking of static objects or dynamic objects around the vehicle.
  • task responses such as risk identification, risk warning or strategy execution of driving risks in the current application scenario can be performed. For example, when the risk of vehicle collision is identified, risk warning prompts or emergency intervention braking and other task responses can be performed, thereby effectively increasing the safety of car driving.
  • the task execution data when the current application scenario is a smart cockpit, and the task execution data is a facial image captured by a camera mounted on the vehicle, and the driving condition information obtained by a wireless communication module mounted on the vehicle, the task execution data can be distributed to the required number of idle computing cores for computing to obtain computing result data.
  • the computing result data can be the result data of facial expression, current weather information, and current road condition information. Then, the task response of the current application scenario can be performed according to the computing result data.
  • the task response can be: outputting a prompt message to open the car window so that the driver can breathe fresh air and relax, and ⁇ or outputting a prompt message of a place suitable for relaxation such as a surrounding park, and ⁇ or outputting a prompt message to play soothing music.
  • the task response can be: outputting a prompt message to close the car window, or an information push prompt message, to calm the driver's emotions.
  • the task response can be to push information about busy places such as entertainment venues to the driver, or output prompt information to play cheerful and exciting music, etc., so that Based on the management of the driver's expression and according to the driver's emotions and environmental data, it can provide the driver with corresponding vehicle service information, which is more humane and intelligent.
  • the task execution data when the current application scenario is intelligent cabin air adjustment, and the task execution data is the air parameter information inside and outside the cabin collected by the sensors mounted on the vehicle (which may include carbon dioxide concentration, PM2.5 concentration, carbon monoxide concentration, etc.), the task execution data can be distributed to the required number of idle computing cores for calculation to obtain calculation result data.
  • the calculation result data may be the air quality information inside and outside the cabin.
  • task responses such as automatically turning on the internal circulation mode or external circulation mode of the vehicle air conditioner can be performed, thereby maintaining good air quality in the cabin and providing a comfortable and healthy cabin air environment.
  • step S30 the task execution data is distributed to the required number of idle computing cores for computing.
  • the step of obtaining computing result data includes:
  • Step S310 obtaining a virtual space address corresponding to the task execution data, and converting the virtual space address into a host physical address;
  • Step S320 extracting the task execution data from the host physical address
  • Step S330 transferring the extracted task execution data to the required number of idle computing cores for computing to obtain computing result data.
  • multiple operating systems need to be able to run in parallel on a single terminal device (such as a vehicle), wherein one operating system corresponds to installing at least one AI application (APP, Application), and one AI application corresponds to supporting the running of one AI application scenario. That is, one operating system at least corresponds to supporting the running of one AI application scenario.
  • These operating systems may be referred to as "guest operating systems" (guest operating system), and they may include multiple instances of an operating system as well as different operating systems.
  • One virtual machine corresponds to running one operating system, and multiple virtual machines can run under the management of a virtual machine monitor (VMM, Virtual Machine Monitor).
  • Virtual machine monitors typically use a memory management unit (Memory Management Unit, MMU) to support address translation and memory protection of VMs.
  • MMU memory management unit
  • the upper-level virtual machine publishes AI applications, and the MMU abstracts and instantiates the application into a task list in DDR (Double Data Rate) and saves the address mapping relationship.
  • the task list records the number of cores bound to the lower layer (that is, the required number of NPU computing cores).
  • the MMU Memory Management Unit
  • the MMU is responsible for converting the virtual space address corresponding to the task execution data from the VM into the host physical address, so as to store the task execution data in DDR (Double Data Rate) according to the host physical address, and trigger the peripheral access of the NPU.
  • the VMU integrates the SMMU (System Memory Management Unit, device-side storage management unit) unit, interrupt control unit and DMA (Data Memory Access, direct storage access) unit.
  • the SMMU unit is responsible for synchronizing the address mapping relationship, converting the guest address sent by the VM into the host physical address, and then extracting the task execution data from the host physical address to the DDR through the DMA unit to the register unit (buffer) in the NPU (Neural Network Processing Unit), such as SRAM (Static Random-Access Memory), and then the NPU Control (NPU control module) of the NPU moves the task execution data in the register unit to the required number of idle computing cores for calculation, and obtains the calculation result data through the calculation of each idle computing core.
  • NPU Neurological Network Processing Unit
  • SRAM Static Random-Access Memory
  • the interrupt control unit realizes inter-core synchronization and moves the data stored in the register unit to SRAM through DMA for the corresponding running virtual machine VM to use, thereby effectively realizing the multi-OS function support on the NPU, which can be combined with Figure 5.
  • This embodiment obtains the virtual space address corresponding to the task execution data, converts the virtual space address into a host physical address, extracts the task execution data from the host physical address, and moves the extracted task execution data to the required number of idle computing cores for calculation to obtain calculation result data, thereby improving the IO (Input Output) real-time performance of the VM and NPU, and realizing dynamic scheduling of NPU computing resources according to specific AI application scenarios, avoiding the strong binding of NPU computing resources and application scenarios, which results in the computing resources bound to the specified application scenario being idle when the specified application scenario is not enabled, causing the NPU to have reduced operating performance in the actual operating environment, thereby effectively achieving the support of NPU multi-OS capabilities.
  • IO Input Output
  • the technical solution of the embodiment of the present application relates to an NPU multi-OS support mechanism, and in particular to a multi-functional fusion field of multiple AI applications. That is, when AI scenarios that require support from different operating systems all put forward computing requirements for the computing resource platform, the NPU needs to achieve synchronous support for multiple operating systems through a virtualization mechanism.
  • the technical solution of the embodiment of the present application is by: obtaining task execution data corresponding to the current application scenario (for example, smart vehicles may include application scenarios such as smart cockpits and assisted driving, smart phones may include application scenarios such as image recognition and voice recognition, and smart homes may include application scenarios such as linkage control and automatic adjustment of operating parameters), wherein the task execution data carries the required number of NPU computing cores required to be used, and detects the computing core operating status information of each NPU computing core, determines the idle computing cores in each NPU computing core in an idle state according to the computing core operating status information, and then distributes the task execution data to the required number of idle computing cores for calculation, obtains calculation result data, and then performs task response for the current application scenario according to the calculation result data.
  • the task execution data carries the required number of NPU computing cores required to be used, and detects the computing core operating status information of each NPU computing core, determines the idle computing cores in each NPU computing core in an idle state according to the computing core operating status information, and then distributes the
  • a hardware isolation method is used to achieve support for multiple OS (Operating System).
  • the hardware isolation method can be: strongly bind the NPU computing core in the NPU chip to the application scenario.
  • an NPU chip includes three NPU computing cores, among which NPU computing core 1 is bound to application scenario a of VM1, and NPU computing core 2 and NPU computing core 3 are bound to application scenario b of VM2.
  • application scenario a is enabled and application scenario b is not enabled, the computing power resources of NPU computing core 2 and NPU computing core 3 are idle, and application scenario a is only supported by running NPU computing core 1.
  • multi-OS support of NPU can also be achieved through manual core binding, that is, before the AI application (one AI application supports running one AI application scenario) is enabled, the binding relationship between the AI application and the NPU computing core is manually set.
  • This approach can achieve basic multi-OS support functions, but it requires manual settings at the user level, has poor flexibility, and cannot achieve dynamic scheduling of NPU computing resources according to specific application scenarios.
  • this method cannot provide strong security isolation for hardware resources, and has a poor security level, especially in real-time fields such as autonomous driving. There are security risks.
  • the embodiment of the present application determines the required number of NPU computing cores according to the task execution data of the specific AI application scenario, and selects the required number of idle NPU computing cores from each NPU computing core by combining the running status information of each NPU computing core to run and support the current application scenario, thereby realizing dynamic scheduling of NPU computing resources according to the specific AI application scenario, avoiding the strong binding of NPU computing resources with the application scenario, and causing the computing resources bound to the specified application scenario to be idle when the specified application scenario is not enabled, causing the problem of reduced NPU operating performance in the actual operating environment, while effectively realizing multi-OS function support on NPU, it also has higher performance isolation capability and resource reuse capability, improves system security, and meets higher levels of functional safety requirements.
  • the embodiment of the present application provides an NPU multi-OS support mechanism that can dynamically allocate NPU core resources, which can reuse NPU computing cores in different application scenarios, improve the utilization rate of NPU computing resources, and thus solve the technical problem of strong binding of NPU computing resources with application scenarios, resulting in low NPU running computing power.
  • step S300 the task execution data is distributed to the required number of idle computing cores for computing.
  • the step of obtaining computing result data includes:
  • Step S340 using the initial convolution operation node as the current convolution operation node, and using the task execution data as input data of the current convolution operation node;
  • Step S350 allocating the input data of the current convolution operation node to the required number of idle operation cores for operation, to obtain the output data of the current convolution operation node;
  • Step S360 taking the next convolution operation node corresponding to the current convolution operation node as a new current convolution operation node, and taking the original current convolution operation node as a previous convolution operation node;
  • Step S370 using the output data of the previous convolution operation node as the input data of the current convolution operation node, and returning to execute the step of distributing the input data of the current convolution operation node to the required number of idle operation cores for operation, until all convolution operation nodes are completed and the operation result data is obtained.
  • the NPU needs to perform four convolution operation nodes on the task execution data of the current application scenario, namely, convolution operation node a, convolution operation node b, convolution operation node c and convolution operation node d, wherein convolution operation node a is It is the initial convolution operation node.
  • the task execution data is first used as the input data of the convolution operation node a, and is distributed to the required number of idle operation cores for operation to obtain the output data of the convolution operation node a.
  • the output data of the convolution operation node a is used as the input data of the convolution operation node b, and is distributed to the required number of idle operation cores for operation to obtain the output data of the convolution operation node b.
  • the output data of the convolution operation node b is used as the input data of the convolution operation node c, and is distributed to the required number of idle operation cores for operation to obtain the output data of the convolution operation node c.
  • the output data of the convolution operation node c is used as the input data of the convolution operation node d to obtain the output data of the convolution operation node d.
  • the output data of the convolution operation node d is the operation result data. It should be noted that the example shown is only helpful for understanding the present application, and does not constitute a limitation on the convolution operation process of the present application. All technical principles or technical concepts based on this are within the protection scope of the present application.
  • This embodiment uses the initial convolution operation node as the current convolution operation node, and uses the task execution data as the input data of the current convolution operation node, and distributes the input data of the current convolution operation node to the required number of idle operation cores for operation to obtain the output data of the current convolution operation node, and then uses the next convolution operation node corresponding to the current convolution operation node as the new current convolution operation node, and uses the original current convolution operation node as the previous convolution operation node, and then uses the output data of the previous convolution operation node as the input data of the current convolution operation node, and returns to execute: the step of distributing the input data of the current convolution operation node to the required number of idle operation cores for operation until all convolution operation nodes are completed and the operation result data is obtained, thereby realizing data synchronization between the NPU operation cores inside the NPU, so that multi-core capabilities can be integrated, and the NPU multi-OS support function of dynamic allocation, synchronization, operation, and release of
  • step S350 the step of distributing the input data of the current convolution operation node to the required number of idle operation cores for operation, and obtaining the output data of the current convolution operation node includes:
  • Step A10 dividing the input data of the current convolution operation node into the required number of slice task data, and assigning each slice task data to the required number of idle operation cores for operation;
  • Step A20 after each idle computing core completes computing the slice task data, the computing slice data obtained by the idle computing cores are combined to obtain the output data of the current convolution computing node.
  • This embodiment divides the input data of the current convolution operation node into the required number of sliced task data, and distributes each sliced task data to the required number of idle operation cores for operation. After each idle operation core completes the operation of the sliced task data, the operation slice data obtained by each idle operation core are combined to obtain the output data of the current convolution operation node, thereby realizing large model segmentation operation, flexibly and dynamically scheduling NPU core resources, so that multi-core capabilities can be integrated, and further realizing the NPU multi-OS support function of dynamic allocation, synchronization, operation, and release of NPU operation cores.
  • the user process completes the work of sending task execution data.
  • the task structure corresponding to the task execution data is a custom task list. Its main content items can be combined with Figure 8.
  • the MMU Memory Management Unit
  • the task list mainly includes the number of cores required for the task (that is, the required number of NPU computing cores), the interrupt vector table for interaction between the host side and the device side, the task input data address, and the reserved output data space;
  • the MMU completes the virtual address translation on the host side, achieves the conversion of the virtual space address to the host physical address, and records the address mapping relationship in the multi-level page table.
  • the SMMU synchronizes the conversion relationship between the host side and the device side through the multi-level page table to achieve access to the unified physical address.
  • VM integrates SMMU unit, interrupt control unit and DMA unit
  • NPU through registers, mainly completing page table synchronization and instruction issuance. That is, according to specific task requirements, a new task table is created in the table entry of SMMU dedicated to NPU, and the address conversion involved in the MMU part is mapped to SMMU, so as to achieve the conversion of guest address to host physical address, and finally realize the unification of physical addresses on VM side and NPU side.
  • DMA moves the task execution data stored in the task input data address in DDR to the input buffer position of the computing core specified by NPU (i.e., the idle computing core in the NPU computing core that is in idle state) according to the address translation in SMMU.
  • the computing core here is determined by NPU control by retrieving the status bit of each core (this embodiment uses the interrupt mask bit in the interrupt control unit to implement) to determine whether each core is in working state or idle state, and binds the corresponding number of idle computing cores to the task execution data issued in (1) according to the core number specified in the task (i.e., the required number of NPU computing cores required to be used).
  • the specific process can be that VM accesses NPU through register configuration, NPU receives the access signal, NPU Control (NPU control module) checks the running status of each computing core, and then dynamically allocates the idle resources of the computing core, and moves the task execution data to the input buffer position corresponding to the allocated computing core through DMA to the specified position in the task list.
  • NPU Control NPU control module
  • NPU completes the filling of all task execution data in the specified buffer position. After the filling is completed, NPU control triggers the operation core to perform pipeline operation. After the operation is completed, the operation result is moved to the corresponding output buffer position.
  • the task object of NPU is a neural network model, it is a multi-layer graph structure composed of many business nodes, and there is a strict serial dependency relationship between the nodes.
  • the task object of NPU is a neural network model, it is a multi-layer graph structure composed of many business nodes, and there is a strict serial dependency relationship between the nodes.
  • the calculation result is output to SRAM, and an interrupt signal is sent to the interrupt control unit.
  • the interrupt control unit determines whether all cores have completed the calculation task. When all cores have completed the calculation task, the interrupt control unit sends an interrupt signal to each core (operation core). Each core fills the new raw data and the data in the associated SRAM and performs the calculation. Repeat the above steps until the computing tasks of all nodes in the network model are completed.
  • the interrupt control unit can be used to achieve inter-core synchronization between multiple cores. After all the operations are completed, the operation results are output to SRAM and an interrupt signal is sent to the interrupt control unit.
  • DMA moves the final calculation result to the output data address space in DDR through the address conversion relationship recorded in SMMU.
  • NPU Control maps the interrupt behavior through the interrupt signal table and moves the calculation result in SRAM to the DDR output address specified in the task list through DMA.
  • the NPU sends an IO interrupt signal to the VM.
  • the VM receives the interrupt signal, completes data extraction, and determines whether to continue processing. If there is no subsequent work, the memory space is reclaimed and the page table is released.
  • the embodiment of the present application further provides a data processing system 100 for an application scenario.
  • FIG4 is a schematic diagram of a structural module of the data processing system 100 for the application scenario of the embodiment of the present application, wherein the data processing system 100 for the application scenario includes a central processing unit 1 and a neural network processor 2 connected to each other, the central processing unit 1 includes a plurality of virtual machine VM modules 11, and the neural network processor 2 includes an NPU computing core 22, an NPU control module 21 and a virtualization control VMU module 23:
  • Each VM module 11 is respectively bound to at least one AI application, wherein one AI application maps to one application scenario;
  • the VMU module 23 is configured to obtain task execution data generated by the AI application in the VM module 11, wherein the task execution data carries the required number of NPU computing cores 22 required to be used;
  • the NPU control module 21 is configured to detect the operation status information of the operation cores of each NPU operation core 22, determine the idle operation cores in each NPU operation core 22 that are in an idle state according to the operation status information, and distribute the task execution data to the required number of idle operation cores for operation to obtain operation result data;
  • the VM module 11 is configured to perform task responses for the current application scenario according to the calculation result data.
  • the central processing unit 1 refers to the CPU
  • the neural network processor 2 refers to the NPU
  • a terminal device in order for a terminal device to run multiple AI application scenarios simultaneously, multiple operating systems (OS) need to be able to run in parallel on a single terminal device (such as a vehicle), where one operating system is bound to at least one AI application (APP), and one AI application maps to one application scenario.
  • the application scenario is an AI application scenario, and the type of the AI application scenario has been described in detail above, so it will not be repeated here.
  • one operating system at least supports running one AI application scenario.
  • These operating systems can be called “guest operating systems", which can include multiple instances of an operating system and different operating systems.
  • guest operating systems can include multiple instances of an operating system and different operating systems.
  • one VM (Virtual Machine) module 11 runs one operating system, and multiple VM modules 11 can run under the management of a virtual machine monitor (VMM).
  • the virtual machine monitor usually uses an MMU module 12 (Memory Management Unit) to support address translation and memory protection of the VM module 11.
  • MMU module 12 Memory Management Unit
  • the upper-level virtual machine VM module 11 publishes the AI application, and the MMU module 12 abstracts and instantiates the application into a task list in DDR (Double Data Rate) and saves the address mapping relationship.
  • DDR Double Data Rate
  • the task list records the number of cores bound to the lower layer (that is, the required number of NPU computing cores).
  • the MMU (Memory Management Unit) module 12 is responsible for converting the virtual space address corresponding to the task execution data from the VM module 11 into the host physical address, thereby storing the task execution data in the host physical address of DDR according to the host physical address, and triggering the peripheral access of the neural network processor 2.
  • the VMU module 23 integrates the SMMU (System Memory Management Unit) unit 231, the interrupt control unit 233 and the DMA (Data Memory Access) unit 232.
  • the SMMU unit 231 is responsible for synchronizing the address mapping relationship, converting the guest address sent by the VM module 11 into a host physical address, and then extracting the task execution data from the host physical address of the DDR to the register unit (buffer) in the neural network processor 2 through the DMA unit 232, such as SRAM (Static Random-Access Memory), and then the NPU control module 21 of the neural network processor 2 moves the task execution data in the register unit to the required number of idle computing cores for calculation, and obtains the calculation result data through the calculation of each idle computing core.
  • the register unit such as SRAM (Static Random-Access Memory)
  • the interrupt control unit realizes inter-core synchronization, and moves the calculation result data stored in the register unit to SRAM through the DMA unit 232 for the corresponding running virtual machine VM module 11 to use, thereby effectively realizing multi-OS function support on the NPU, which can be combined with Figure 5.
  • the data processing system 100 of the application scenario of the embodiment of the present application includes a central processing unit 1 and a neural network processor 2 that are interconnected, the central processing unit 1 includes multiple virtual machine VM modules 11, the neural network processor includes an NPU computing core 22, an NPU control module 21 and a virtualization control VMU module 23, wherein each VM module 11 is respectively bound to at least one AI application, wherein one AI application maps one application scenario, the VMU module 23 is configured to obtain task execution data generated by the AI application in the VM module 11, wherein the task execution data carries the required number of NPU computing cores 22 required to be used, the NPU control module 21 is configured to detect the computing core running status information of each NPU computing core 22, determine the idle computing cores in each NPU computing core 22 that are in an idle state according to the computing core running status information, and distribute the task execution data to the required number of idle computing cores for computing to obtain computing result data, and the VM module 11 is configured to perform task response for the current application scenario according to the computing result data.
  • the embodiment of the present application determines the required number of NPU computing cores 22 required to be used according to the task execution data of the specific AI application scenario, and selects the required number of idle NPU computing cores from each NPU computing core 22 by combining the running status information of each NPU computing core 22 to run and support the current application scenario, thereby realizing dynamic scheduling of NPU computing resources according to the specific AI application scenario, avoiding the strong binding of NPU computing resources and application scenarios, which results in the computing resources bound to the specified application scenario being idle when the specified application scenario is not enabled, causing the NPU to have reduced operating performance in the actual operating environment.
  • the embodiment of the present application provides an NPU multi-OS support mechanism that can dynamically allocate NPU core resources, so that the NPU computing cores can be reused in different application scenarios, thereby improving the utilization rate of NPU computing resources, and further solving the technical problem that the NPU computing resources are strongly bound to the application scenarios, resulting in low NPU operating computing power.
  • the CPU 1 further includes a memory management MMU module 12
  • the VMU module 23 includes a memory management SMMU unit 231 and a direct memory access DMA unit 232 .
  • the MMU module 12 is configured to obtain a virtual space address corresponding to the task execution data, convert the virtual space address into a host physical address, and send the guest physical address and the address mapping relationship between the guest physical address and the host physical address to the SMMU unit 231;
  • the SMMU unit 231 is configured to receive the guest physical address and address mapping relationship sent by the MMU module 12, and The mapping relationship maps the guest physical address to the host physical address, and sends the mapped host physical address to the DMA unit 232;
  • the DMA unit 232 is configured to receive the host physical address sent by the SMMU unit 231 and extract the task execution data from the sent host physical address;
  • the NPU control module 21 is configured to transfer the extracted task execution data to a required number of idle computing cores for computing, to obtain computing result data.
  • the central processor 1 is configured to include a memory management MMU module 12, and the VMU module 23 includes a memory management SMMU unit 231 and a direct memory access DMA unit 232, wherein the MMU module 12 is configured to obtain a virtual space address corresponding to task execution data, convert the virtual space address into a host physical address, and send the guest physical address and the address mapping relationship between the guest physical address and the host physical address to the SMMU unit 231; the SMMU unit 231 is configured to receive the guest physical address and the address mapping relationship sent by the MMU module 12, map the guest physical address to the host physical address according to the sent address mapping relationship, and send the mapped host physical address to the DMA unit 232; the DMA unit 232 32 is configured to receive the host physical address sent by the SMMU unit 231, and extract the task execution data from the sent host physical address; the NPU control module 21 is configured to move the extracted task execution data to the required number of idle computing cores for computing, and obtain the computing result data, thereby improving the IO (Input
  • FIG. 6 is a schematic diagram of the module structure of the VMU in the embodiment of the present application
  • FIG. 5 is a scene interaction diagram of the data processing system in the application scenario of the embodiment of the present application, including:
  • the virtual machine VM is responsible for publishing applications and binding the required operating system
  • 2MMU is the memory management unit, which is responsible for mapping the virtual machine system space to the physical space
  • 3DDR is the physical memory space, which is responsible for storing the instantiated NPU tasks, identifying the number of computing cores for each task, the execution data storage location, the data recovery location, and the interrupt reception location;
  • 4Hypervisor is a virtual machine monitor, responsible for page table sharing and synchronization between MMU and SMMU;
  • 5NPU Control is the NPU control core, responsible for allocating and binding NPU computing cores and controlling the pipeline operations of each module;
  • 6VMU integrates SMMU, interrupt control unit and DMA, which are responsible for device-side address mapping, inter-core data synchronization and data transfer respectively;
  • 7SRAM is a cache unit that stores intermediate and result data
  • the NPU computing core is the computing module in the NPU, responsible for accelerating the execution of specific operators in upper-level applications.
  • the VMU module 23 further includes an interrupt control unit 233.
  • the NPU control module 21 is configured to use the initial convolution operation node as the current convolution operation node, and use the task execution data as the input data of the current convolution operation node, and distribute the input data of the current convolution operation node to the required number of idle operation cores for operation;
  • the interrupt control unit 233 is configured to determine whether each idle computing core has completed computing the input data of the current convolution computing node, and after each idle computing core has completed computing all the input data of the current convolution computing node, send a first interrupt signal to the NPU control module 21;
  • the NPU control module 21 is configured to use the next convolution operation node corresponding to the current convolution operation node as the new current convolution operation node according to the first interrupt signal, use the original current convolution operation node as the previous convolution operation node, and use the output data of the previous convolution operation node as the input data of the current convolution operation node, distribute the input data of the current convolution operation node to the required number of idle operation cores for operation, and obtain the output data of the current convolution operation node, until all convolution operation nodes are completed and the operation result data is obtained.
  • the interrupt control unit 233 can optimize the interrupt mechanism and improve the IO real-time performance of the VM and NPU. At the same time, it can realize data synchronization between internal computing cores, so that multi-core capabilities can be integrated, thereby realizing large model segmentation operations.
  • the NPU control module 21 uses the initial convolution operation node as the current convolution operation node, and uses the task execution data as the input data of the current convolution operation node, and distributes the input data of the current convolution operation node to the required number of idle operation cores for operation, and uses the interrupt control unit 233 to determine whether each idle operation core has completed the operation of the input data of the current convolution operation node. After each idle operation core has completed the operation of the input data of the current convolution operation node, a first interrupt signal is sent to the NPU control module 21, and then the current convolution operation node is corresponding to the next convolution operation node according to the first interrupt signal.
  • the convolution operation node is used as the new current convolution operation node, and the original current convolution operation node is used as the previous convolution operation node, and the output data of the previous convolution operation node is used as the input data of the current convolution operation node.
  • the input data of the current convolution operation node is distributed to the required number of idle operation cores for operation to obtain the output data of the current convolution operation node, until all convolution operation nodes are completed and the operation result data is obtained, thereby realizing data synchronization between the NPU operation cores 22 inside the NPU, so that multi-core capabilities can be integrated, and the NPU multi-OS function of dynamic allocation, synchronization, operation, and release of the NPU operation core 22 is realized.
  • the task object of the NPU is a neural network model, it is a multi-layer graph structure composed of many business nodes, and there are strict serial dependencies between the nodes.
  • a model task needs to be split into multiple computing cores for execution; on the other hand, due to the existence of dependencies between layers, it is necessary to consider the synchronization problem between cores during multi-core execution.
  • Specific embodiment three is:
  • the NPU first divides the tasks and distributes the raw data of multiple channels to different cores for calculation
  • the interrupt control unit sends an interrupt signal to each core.
  • Each core then fills in new raw data and data in the associated SRAM and performs computations.
  • the NPU control module 21 is configured to divide the input data of the current convolution operation node into a required number of slice task data, and allocate each slice task data to a required number of idle operation cores for operation;
  • the interrupt control unit 233 is configured to determine whether each idle computing core has completed computing the slice task data, and after each idle computing core has completed computing all the slice task data, send a second interrupt signal to the NPU control module 21;
  • the NPU control module 21 is configured to combine the operation slice data obtained by the operations of each idle operation core according to the second interrupt signal to obtain the output data of the current convolution operation node.
  • the input data of the current convolution operation node is divided into the required number of slice task data through the NPU control module 21, and each slice task data is correspondingly allocated to the required number of idle operation cores for operation, and then the interrupt control unit 233 is used to determine whether each idle operation core has completed the operation of the slice task data.
  • a second interrupt signal is sent to the NPU control module 21, and then the NPU control module 21 combines the operation slice data obtained by the operation of each idle operation core according to the second interrupt signal to obtain the output data of the current convolution operation node, thereby realizing large model segmentation operation and flexible
  • the computing resources of the NPU computing core 22 are dynamically scheduled so that multi-core capabilities can be integrated, further realizing the NPU multi-OS support function of dynamic allocation, synchronization, computing, and release of the NPU computing core 22.
  • FIG. 10 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application.
  • the electronic device may include: a central processing unit (CPU) 1001, a communication bus 1002, a user interface 1003, a network interface 1004, a memory 1005, and a neural network processor (neural-network process units, NPU) 1006.
  • the communication bus 1002 is used to realize the connection and communication between these components.
  • the user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the user interface 1003 may also include a standard wired interface and a wireless interface.
  • the network interface 1004 may include a standard wired interface, a wireless interface (such as a Wireless-Fidelity (WI-FI interface).
  • the memory 1005 may be a high-speed random access memory (RAM) memory, or a stable non-volatile memory (NVM), such as a disk memory.
  • RAM random access memory
  • NVM stable non-volatile memory
  • the memory 1005 may also be a storage device independent of the aforementioned central processor 1001 and/or neural network processor 1006.
  • a memory 1005 as a storage medium may include an operating system, a data storage module, a network communication module, a user interface module, and a computer program.
  • the network interface 1004 is mainly used for data communication with other devices; the user interface 1003 is mainly used for data interaction with the user; the central processing unit 1001, the neural network processor 1006, and the memory 1005 in this embodiment can be set in the electronic device, and the electronic device calls the computer program stored in the memory 1005 through the processor 1001 and/or the neural network processor 1006, and executes the data processing method provided in any of the above embodiments for the application scenario of the electronic device.
  • the terminal proposed in this embodiment and the data processing method for application scenarios applied to electronic devices proposed in the above embodiments belong to the same inventive concept.
  • Technical details not fully described in this embodiment can be referred to any of the above embodiments, and this embodiment has the same beneficial effects as the data processing method for executing application scenarios.
  • the present application also provides a readable storage medium, which is a computer-readable storage medium, and the computer-readable storage medium stores one or more programs, and the one or more programs can also be executed by one or more processors to implement the steps of each embodiment of the above-mentioned memory access method.
  • the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.
  • a storage medium such as ROM/RAM, magnetic disk, optical disk
  • a terminal device which can be a mobile phone, computer, server, air conditioner, or network device, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Theoretical Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Neurology (AREA)
  • Stored Programmes (AREA)

Abstract

本申请公开了一种应用场景的数据处理方法、系统、电子设备及存储介质，属于人工智能技术领域。本申请通过获取当前应用场景对应的任务执行数据，其中，任务执行数据携带有需求使用NPU运算核的需求数量；检测各NPU运算核的运算核运行状态信息，根据运算核运行状态信息，确定各NPU运算核中处于空闲状态的空闲运算核；将任务执行数据分配至需求数量的空闲运算核上进行运算，得到运算结果数据，并根据运算结果数据进行当前应用场景的任务响应。

Description

应用场景的数据处理方法、系统、电子设备及存储介质
相关申请
本申请要求于2022年10月25号申请的、申请号为202211315722.7的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及人工智能技术领域，尤其涉及应用场景的数据处理方法、系统、电子设备及存储介质。
背景技术
AI(Artificial Intelligence，人工智能)是利用数字计算机或者数字计算机控制的机器模拟、延伸和扩展人的智能，感知环境、获取知识并使用知识获得最佳效果的理论、方法、技术及应用系统。传统的CPU(Central Processing Unit，中央处理器)、GPU(Graphics Processing Unit，图形处理器)等运算平台通常不能满足AI应用场景的算力需求，而NPU(Neural Network Processing Unit，神经网络处理器)的出现，为更多AI应用场景的实际落地提供了基础性保障。
随着AI应用场景的丰富以及芯片集中化设计的发展需求，NPU对于多种操作系统(或者说多种类型的AI应用场景)的同步支持，已成为NPU设计越来越重要的性能指标。然而，目前大部分NPU厂商仍采用硬件隔离式的方式来实现对多OS(Operating System，操作系统)的支持，该方式使得NPU算力资源与AI应用场景强绑定，在某个指定AI应用场景未启用时，该指定AI应用场景所绑定部分的算力资源一直处于闲置状态，从而严重降低了NPU在AI应用场景的实际运行环境下的运行算力。
发明内容
本申请的主要目的在于提供一种应用场景的数据处理方法、系统、电子设备及存储介质，旨在解决NPU算力资源与应用场景强绑定，导致NPU运行算力低的技术问题。
为实现上述目的，本申请提供一种应用场景的数据处理方法，包括：
获取当前应用场景对应的任务执行数据，其中，所述任务执行数据携带有需求使用NPU运算核的需求数量；
检测各所述NPU运算核的运算核运行状态信息，根据所述运算核运行状态信息，确定各所述NPU运算核中处于空闲状态的空闲运算核；
将所述任务执行数据分配至所述需求数量的空闲运算核上进行运算，得到运算结果数据，并根据所述运算结果数据进行当前应用场景的任务响应。
为实现上述目的，本申请提供一种应用场景的数据处理系统，所述应用场景的数据处理系统包括相互连接的中央处理器和神经网络处理器，所述中央处理器包括多个虚拟机VM模块，所述神经网络处理器包括NPU运算核、NPU控制模块和虚拟化控制VMU模块：
各所述VM模块分别绑定至少一种AI应用，其中，一种AI应用映射一种应用场景；
所述VMU模块设置为获取来自于所述VM模块中AI应用所产生的任务执行数据，其中，所述任务执行数据携带有需求使用NPU运算核的需求数量；
所述NPU控制模块设置为检测各所述NPU运算核的运算核运行状态信息，根据所述运算核运行状态信息，确定各所述NPU运算核中处于空闲状态的空闲运算核，并将所述任务执行数据分配至所述需求数量的空闲运算核上进行运算，得到运算结果数据；
所述VM模块设置为根据所述运算结果数据进行当前应用场景的任务响应。
此外，为实现上述目的，本申请还提供一种电子设备，所述电子设备包括如上述的中央处理器、神经网络处理器、存储器、以及存储在所述存储器上并可在所述中央处理器和/或所述神经网络处理器上运行的应用场景的数据处理程序，所述应用场景的数据处理程序被所述中央处理器和/或所述神经网络处理器执行时实现如上述应用场景的数据处理方法的步骤。
此外，为实现上述目的，本申请还提供一种可读存储介质，所述可读存储介质为计算机可读存储介质，所述计算机可读存储介质上存储有应用场景的数据处理程序，所述应用场景的数据处理程序被处理器执行时实现如上所述的应用场景的数据处理方法的步骤。
本申请实施例提出一种应用场景的数据处理方法、系统、电子设备及存储介质，本申请实施例的技术方案是通过：获取当前应用场景(例如智能车辆可包括智能座舱、辅助驾驶等应用场景，智能手机可包括图像识别、语音识别等应用场景，以及智能家居可包括联动控制、运行参数自动调节等应用场景)对应的任务执行数据，其中，任务执行数据携带有需求使用NPU运算核的需求数量，并检测各NPU运算核的运算核运行状态信息，根据运算核运行状态信息，确定各NPU运算核中处于空闲状态的空闲运算核，然后将任务执行数据分配至需求数量的空闲运算核上进行运算，得到运算结果数据，再根据运算结果数据进行当前应用场景的任务响应。
而目前采用硬件隔离式的方式来实现对多OS(Operating System，操作系统)的支持，该硬件隔离式的方式可为：将NPU芯片中的NPU运算核与应用场景进行强绑定，例如一个NPU芯片包括三个NPU运算核，其中，NPU运算核一与应用场景a绑定，NPU运算核二和NPU运算核三与应用场景b绑定，在应用场景a启用，而应用场景b未启用时，NPU运算核二和NPU运算核三的算力资源处于闲置状态，仅通过NPU运算核一运行支持应用场景a。而在应用场景a未启用，而应用场景b启用时，NPU运算核一的算力资源处于闲置状态，仅通过NPU运算核二和NPU运算核三运行支持应用场景b，从而严重降低了NPU在应用场景的实际运行环境下的运行算力。
相比于该硬件隔离式的方式，本申请实施例通过根据具体AI应用场景的任务执行数据，确定需求使用NPU运算核的需求数量，并通过结合各NPU运算核的运行状态信息，从各NPU运算核中选取该需求数量的空闲NPU运算核，来运行支持当前应用场景，从而实现根据具体AI应用场景对NPU算力资源进行动态调度，避免NPU算力资源与应用场景强绑定，而导致在指定应用场景未启用时，该指定应用场景所绑定部分的算力资源处于闲置状态，造成NPU在实际运行环境下运行性能降低的问题，而本申请实施例通过提供一种能动态分配NPU核资源的NPU多OS支持机制，可在不同应用场景下对NPU运算核进行复用，提高了NPU算力资源的利用率，进而解决了NPU算力资源与应用场景强绑定，导致NPU运行算力低的技术问题。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图示出的结构获得其他的附图。
图1为本申请应用场景的数据处理方法第一实施例的流程示意图；
图2为本申请应用场景的数据处理方法第二实施例的流程示意图；
图3为本申请应用场景的数据处理方法第三实施例的流程示意图；
图4为本申请实施例应用场景的数据处理系统的结构模块示意图；
图5为本申请实施例应用场景的数据处理系统的场景交互图；
图6为本申请实施例VMU的模块结构示意图；
图7为本申请实施例应用场景的数据处理方法的时序图；
图8为本申请实施例中主机侧与设备侧进行物理地址同步的任务列表映射示意图；
图9为本申请实施例对任务执行数据进行卷积运算的流程示意图；
图10为本申请实施例方案涉及的电子设备的结构示意图。
本申请目的的实现、功能特点及优点将结合实施例，参照附图做进一步说明。
具体实施方式
应当理解，此处所描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请的一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
本申请实施例中所有方向性指示(诸如上、下、左、右、前、后……)仅用于解释在某一特定姿态(如附图所示)下各部件之间的相对位置关系、运动情况等，如果该特定姿态发生改变时，则该方向性指示也相应地随之改变。
在本申请中，除非另有明确的规定和限定，术语“连接”、“固定”等应做广义理解，例如，“固定”可以是固定连接，也可以是可拆卸连接，或成一体；可以是机械连接，也可以是电连接；可以是直接相连，也可以通过中间媒介间接相连，可以是两个元件内部的连通或两个元件的相互作用关系，除非另有明确的限定。对于本领域的普通技术人员而言，可以根据具体情况理解上述术语在本申请中的具体含义。
另外，在本申请中如涉及“第一”、“第二”等的描述仅用于描述目的，而不能理解为指示或暗示其相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”的特征可以明示或者隐含地包括至少一个该特征。另外，各个实施例之间的技术方案可以相互结合，但是必须是以本领域普通技术人员能够实现为基础，当技术方案的结合出现相互矛盾或无法实现时应当认为这种技术方案的结合不存在，也不在本申请要求的保护范围之内。
随着全球智能化和数字化水平的提升，高速膨胀的数据量、算力、算法都对运算平台提出了更高的发展要求，而传统的CPU(Central Processing Unit，中央处理器)、GPU(Graphics Processing Unit，图形处理器)等运算平台为了保障运算的高度灵活，不得不集成更为复杂的指令结构，从而牺牲集群阵列化运算的可能，在这种形势下，能为神经网络提供专用化加速能力的NPU(Neural Network Processing Unit，神经网络处理器)加速平台就显得尤为重要。
在设计上，NPU减少了控制类指令的硬化设计，将MAC阵列作为主要的计算核心，目前市面上已出现万级以上的MAC阵列设计，结合多核形态，部分NPU已具备上百TOPs的理论计算能力，相对于CPU、GPU不足1TOPs的算力，NPU为更多AI场景的实际落地提供了基础性保障。
同时，随着AI场景的丰富以及芯片集中化设计的发展需求，单NPU对于多种操作系统的同步支持成为NPU设计时越来越重要的性能指标。然而，目前大部分NPU厂商仍采用硬件隔离式的方式来实现对多OS(Operating System，操作系统)的支持。在这种形式下，NPU的算力核心无法得到动态调度，这就导致大量的算力资源与AI场景强绑定，在指定AI场景未启用时，绑定部分的算力资源一直处于闲置状态，再结合NPU本身存在的MAC利用率问题，实际运行环境下的算力将不足理论算力的一半，严重降低了NPU在AI应用场景的实际运行环境下的运行算力。
基于此，本申请提出一种应用场景的数据处理方法，在本申请应用场景的数据处理方法的一实施例中，请参照图1，应用场景的数据处理方法包括：
步骤S100，获取当前应用场景对应的任务执行数据，其中，所述任务执行数据携带有需求使用NPU运算核的需求数量；
在本实施例中，该应用场景的数据处理方法是指AI(Artificial Intelligence，人工智能)应用场景的数据处理方法，或者更具体可为基于神经网络算法的应用场景。本领域技术人员可以理解的是，AI 是利用数字计算机或者数字计算机控制的机器模拟、延伸和扩展人的智能，感知环境、获取知识并使用知识获得最佳效果的技术及应用系统。其中，AI具体可包括计算机视觉技术、语音处理技术、自然语言处理技术，以及机器学习/深度学习等几大方向。
在一实施例中，本申请应用场景的数据处理方法应用于车辆领域，该AI应用场景的类型包括但不限于智能辅助驾驶ADAS(Advanced Driving Assistance System，高级驾驶辅助系统)、智能座舱和车舱空气智能调节等。在另一实施例中，本申请应用场景的数据处理方法应用于移动终端领域，该AI应用场景的类型包括但不限于手机的智能语音服务、智能拍照和面部识别等。在又一实施例中，本申请应用场景的数据处理方法应用于家居领域，该AI应用场景的类型包括但不限于智能空调、智能冰箱、智能灯具和智能油烟机等。当然，本申请应用场景的数据处理方法还可应用于无人机领域(例如路径规划、智能避障等AI应用场景)、医疗器械领域(例如输液种类的智能识别、输液气泡检测等AI应用场景)和机器人领域(例如地面清洁任务、语音助手服务等AI应用场景)等NPU多操作系统支持的应用场合，本实施例不作具体的限定。
不同的AI应用场景往往对应不同的任务执行数据，其中，该任务执行数据携带有需求使用NPU运算核的需求数量。例如，当前应用场景的类型为智能辅助驾驶时，其对应的任务执行数据可为车辆搭载的传感器在汽车行驶过程中所实时检测的行驶环境信息。又例如，当前应用场景的类型为智能座舱时，其对应的任务执行数据可为车辆搭载的摄像头所采集的人脸图像，以及车辆搭载的无线通讯模块获取到的行驶工况信息。还例如，当前应用场景的类型为车舱空气智能调节时，其对应的任务执行数据可为车辆搭载的传感器所采集的车舱内和车舱外的空气参数信息(可包括二氧化碳浓度、PM2.5浓度和一氧化碳浓度等)。
步骤S200，检测各所述NPU运算核的运算核运行状态信息，根据所述运算核运行状态信息，确定各所述NPU运算核中处于空闲状态的空闲运算核；
在本实施例中，该运算核运行状态信息是指NPU运算核是否处于工作状态还是空闲状态的运行状态信息，或者说NPU运算核的线程是否被工作任务所占据的运行状态信息，从而根据该运算核运行状态信息，可确定出当前的各NPU运算核中处于空闲状态的空闲运算核，当然同时也可确定出当前的各NPU运算核中处于工作状态的工作运算核。
步骤S300，将所述任务执行数据分配至所述需求数量的空闲运算核上进行运算，得到运算结果数据，并根据所述运算结果数据进行当前应用场景的任务响应。
在一实施方式中，假设本申请应用场景的数据处理方法应用于车辆，作为一种示例，当前应用场景的类型为智能辅助驾驶，且任务执行数据为车辆搭载的传感器在汽车行驶过程中所实时检测的行驶环境信息时，可通过将该任务执行数据分配至该需求数量的空闲运算核上进行运算得到运算结果数据，此时该运算结果数据可为车辆周围静态物体或动态物体的辨识、侦测和追踪的结果数据，根据该运算结果数据可进行当前应用场景的行车风险的风险识别、风险预警或应对风险的策略执行等任务响应，例如识别到车辆碰撞风险时进行风险预警提示或紧急干预制动等任务响应，从而有效增加汽车驾驶的安全性。
作为另一种示例，当前应用场景的类型为智能座舱，且任务执行数据为车辆搭载的摄像头所采集的人脸图像，以及车辆搭载的无线通讯模块获取到的行驶工况信息时，可通过将该任务执行数据分配至该需求数量的空闲运算核上进行运算得到运算结果数据，此时该运算结果数据可为人脸表情、当前天气信息和当前路况信息的结果数据，然后根据该运算结果数据可进行当前应用场景的任务响应，例如在人脸表情为悲伤表情，且当前天气信息表示天气状况良好，和\或当前路况信息表示路况良好时，任务响应可为：输出开启车窗的提示信息以使驾驶员呼吸新鲜空气放松心情，和\或可输出如周边公园等适合放松身心的地点的提示信息，和\或输出播放舒缓类音乐的提示信息。又例如在人脸表情为厌恶表情，且当前天气信息表示天气状况不佳时，任务响应可为：可输出闭合车窗的提示信息，或资讯推送提示信息，以平复驾驶员情绪。还例如在人脸表情为愉悦表情，且当前天气信息表示天气状况良好时，任务响应可为向驾驶员推送如娱乐场所等热闹地点的信息，或输出播放欢快激扬类音乐的提示信息等，从而能 够基于对驾驶员的表情管理，依照驾驶员的情绪和环境数据，为驾驶员提供相应的车机服务信息，更加人性化和智能化。
作为又一种示例，当前应用场景的类型为车舱空气智能调节时，且任务执行数据为车辆搭载的传感器所采集的车舱内和车舱外的空气参数信息(可包括二氧化碳浓度、PM2.5浓度和一氧化碳浓度等)时，可通过将该任务执行数据分配至该需求数量的空闲运算核上进行运算得到运算结果数据，此时该运算结果数据可为车舱内和车舱外的空气质量信息，根据该运算结果数据可进行自动开启车辆空调的内循环模式或外循环模式等任务响应，从而保持车舱内良好的空气质量，提供一种舒适健康的舱内空气环境。
作为一种可能的实施方式，请参照图2，在步骤S30中，将所述任务执行数据分配至所述需求数量的空闲运算核上进行运算，得到运算结果数据的步骤包括：
步骤S310，获取所述任务执行数据对应的虚拟空间地址，将所述虚拟空间地址转换为主机物理地址；
步骤S320，从所述主机物理地址中提取所述任务执行数据；
步骤S330，将提取的任务执行数据搬运至所述需求数量的空闲运算核上进行运算，得到运算结果数据。
在本实施例中，为了使终端设备同时运行多个AI应用场景，多个操作系统(OS，Operating System)需要能够在单个终端设备(例如车辆)上并行运行，其中，一个操作系统对应安装至少一个AI应用程序(APP，Application)，一个AI应用程序对应支持运行一种AI应用场景。也即，一个操作系统至少对应支持运行一种AI应用场景。这些操作系统可称为“客机操作系统”(guest operating system)，它们可以包括一个操作系统的多个实例以及不同的操作系统。一个虚拟机(VM，Virtual Machine)对应运行一个操作系统，多个虚拟机可以在虚拟机监控器(VMM，Virtual Machine Monitor)的管理下运行。虚拟机监控器通常使用存储器管理单元(Memory Management Unit，MMU)来支持VM的地址转换(translation)和存储器保护。在工作时，由上层虚拟机发布AI应用，MMU将应用抽象实例化为DDR(Double Data Rate，双倍速率同步动态随机存储器)中的任务列表，并保存地址映射关系，任务列表记录下层绑定的核number(即需求使用NPU运算核的需求数量)，MMU(Memory Management Unit，存储器管理单元)负责将来自VM的任务执行数据对应的虚拟空间地址转换为主机物理地址，从而根据该主机物理地址将任务执行数据对应存储至DDR(Double Data Rate，双倍速率同步动态随机存储器)中，并触发NPU的外设取用，VMU中集成了SMMU(System Memory Management Unit，设备端存储管理单元)单元、中断控制单元及DMA(Data Memory Access，直接存储访问)单元，SMMU单元负责同步地址映射关系，将VM发送的访客地址转换为主机物理地址，然后通过DMA单元从至DDR的主机物理地址中提取任务执行数据至NPU(Neural Network Processing Unit，神经网络处理器)内的寄存单元(buffer)，例如SRAM(Static Random-Access Memory，静态随机存取存储器)中，然后再由NPU的NPU Control(NPU控制模块)将寄存单元中的任务执行数据搬运至该需求数量的空闲运算核上进行运算，通过各空闲运算核运算得到运算结果数据，多个NPU运行核并行运行该任务执行数据时，中断控制单元实现核间同步，并通过DMA将存储在寄存单元中的搬运至SRAM，供对应运行的虚拟机VM取用，从而有效实现NPU上的多OS功能支持，可结合参照图5。
本实施例通过获取任务执行数据对应的虚拟空间地址，将该虚拟空间地址转换为主机物理地址，从主机物理地址中提取该任务执行数据，并将提取的任务执行数据搬运至该需求数量的空闲运算核上进行运算，得到运算结果数据，从而提升VM与NPU的IO(Input Output，输入输出)实时性，实现根据具体AI应用场景对NPU算力资源进行动态调度，避免NPU算力资源与应用场景强绑定，而导致在指定应用场景未启用时，该指定应用场景所绑定部分的算力资源处于闲置状态，造成NPU在实际运行环境下运行性能降低的问题，进而有效达成NPU多OS能力的支持。
本申请实施例的技术方案涉及一种NPU多OS支持机制，尤其涉及到多AI应用的多功能融合场 景。即需要不同操作系统支持的AI场景均对运算资源平台提出运算需求时，需要NPU通过虚拟化机制实现对多操作系统的同步支持。
本申请实施例的技术方案是通过：获取当前应用场景(例如智能车辆可包括智能座舱、辅助驾驶等应用场景，智能手机可包括图像识别、语音识别等应用场景，以及智能家居可包括联动控制、运行参数自动调节等应用场景)对应的任务执行数据，其中，任务执行数据携带有需求使用NPU运算核的需求数量，并检测各NPU运算核的运算核运行状态信息，根据运算核运行状态信息，确定各NPU运算核中处于空闲状态的空闲运算核，然后将任务执行数据分配至需求数量的空闲运算核上进行运算，得到运算结果数据，再根据运算结果数据进行当前应用场景的任务响应。
而目前采用硬件隔离式的方式来实现对多OS(Operating System，操作系统)的支持，该硬件隔离式的方式可为：将NPU芯片中的NPU运算核与应用场景进行强绑定，例如一个NPU芯片包括三个NPU运算核，其中，NPU运算核一与VM1的应用场景a绑定，NPU运算核二和NPU运算核三与VM2的应用场景b绑定，在应用场景a启用，而应用场景b未启用时，NPU运算核二和NPU运算核三的算力资源处于闲置状态，仅通过NPU运算核一运行支持应用场景a。而在应用场景a未启用，而应用场景b启用时，NPU运算核一的算力资源处于闲置状态，仅通过NPU运算核二和NPU运算核三运行支持应用场景b，从而严重降低了NPU在应用场景的实际运行环境下的运行算力。
另外，还可通过手动绑核来实现NPU的多OS支持，即在AI应用程序(一个AI应用程序对应支持运行一个AI应用场景)启用之前，手动设置该AI应用程序与NPU运算核的绑定关系，这种做法可以实现基础的多OS支持功能，但是需要用户层手动设置，灵活性较差，且无法根据具体的应用场景实现NPU算力资源的动态调度，最重要的，这种方式无法对硬件资源进行强有力的安全隔离，安全等级较差，尤其在自动驾驶等实时领域存在安全风险。
相比于该硬件隔离式和该手动绑核的方式，本申请实施例通过根据具体AI应用场景的任务执行数据，确定需求使用NPU运算核的需求数量，并通过结合各NPU运算核的运行状态信息，从各NPU运算核中选取该需求数量的空闲NPU运算核，来运行支持当前应用场景，从而实现根据具体AI应用场景对NPU算力资源进行动态调度，避免NPU算力资源与应用场景强绑定，而导致在指定应用场景未启用时，该指定应用场景所绑定部分的算力资源处于闲置状态，造成NPU在实际运行环境下运行性能降低的问题，同时在有效实现NPU上多OS功能支持的同时，还具有较高的性能隔离能力及资源复用能力，提升系统安全，达到更高级别的功能安全需求。而本申请实施例通过提供一种能动态分配NPU核资源的NPU多OS支持机制，可在不同应用场景下对NPU运算核进行复用，提高了NPU算力资源的利用率，进而解决了NPU算力资源与应用场景强绑定，导致NPU运行算力低的技术问题。
作为一种可能的实施方式，请参照图3，在步骤S300中，将所述任务执行数据分配至所述需求数量的空闲运算核上进行运算，得到运算结果数据的步骤包括：
步骤S340，将初始卷积运算节点作为当前卷积运算节点，并将所述任务执行数据作为当前卷积运算节点的输入数据；
步骤S350，将当前卷积运算节点的输入数据分配至所述需求数量的空闲运算核上进行运算，得到当前卷积运算节点的输出数据；
步骤S360，将当前卷积运算节点对应下一卷积运算节点作为新的当前卷积运算节点，并将原来的当前卷积运算节点作为上一卷积运算节点；
步骤S370，将上一卷积运算节点的输出数据作为当前卷积运算节点的输入数据，并返回执行所述将当前卷积运算节点的输入数据分配至所述需求数量的空闲运算核上进行运算的步骤，直至所有卷积运算节点均运算完成，得到运算结果数据。
在本实施例中，NPU对当前应用场景的任务执行数据需要进行运算的卷积运算节点包括四个，依次为卷积运算节点a、卷积运算节点b、卷积运算节点c和卷积运算节点d，其中，卷积运算节点a即 为初始卷积运算节点，此时首先将该任务执行数据作为卷积运算节点a的输入数据，分配至该需求数量的空闲运算核上进行运算，得到卷积运算节点a的输出数据，然后将卷积运算节点a的输出数据作为卷积运算节点b的输入数据，分配至该需求数量的空闲运算核上进行运算，得到卷积运算节点b的输出数据，再将卷积运算节点b的输出数据作为卷积运算节点c的输入数据，分配至该需求数量的空闲运算核上进行运算，得到卷积运算节点c的输出数据，最后将卷积运算节点c的输出数据作为卷积运算节点d的输入数据，得到卷积运算节点d的输出数据，由于卷积运算节点d为需要进行运算的最后一个卷积运算节点，因此卷积运算节点d的输出数据即为运算结果数据。需要说明的是，该示出的实例，仅助于理解本申请，并不构成对本申请卷积运算过程的限定，基于此技术原理或技术构思均在本申请的保护范围内。
本实施例通过将初始卷积运算节点作为当前卷积运算节点，并将该任务执行数据作为当前卷积运算节点的输入数据，并将当前卷积运算节点的输入数据分配至该需求数量的空闲运算核上进行运算，得到当前卷积运算节点的输出数据，然后将当前卷积运算节点对应下一卷积运算节点作为新的当前卷积运算节点，并将原来的当前卷积运算节点作为上一卷积运算节点，再将上一卷积运算节点的输出数据作为当前卷积运算节点的输入数据，并返回执行：将当前卷积运算节点的输入数据分配至所述需求数量的空闲运算核上进行运算的步骤，直至所有卷积运算节点均运算完成，得到运算结果数据，从而实现NPU内部的各NPU运算核间的数据同步，使得多核能力可以进行整合，实现NPU运算核动态分配、同步、运算、释放的NPU多OS支持功能。
作为一种可能的实施方式，步骤S350，将当前卷积运算节点的输入数据分配至所述需求数量的空闲运算核上进行运算，得到当前卷积运算节点的输出数据的步骤包括：
步骤A10，将当前卷积运算节点的输入数据切分成所述需求数量份的分片任务数据，并将各所述分片任务数据对应分配至所述需求数量的空闲运算核上进行运算；
步骤A20，在各空闲运算核对分片任务数据运算完毕后，将各空闲运算核运算得到的运算分片数据进行组合，得到当前卷积运算节点的输出数据。
本实施例通过将当前卷积运算节点的输入数据切分成该需求数量份的分片任务数据，并将各分片任务数据对应分配至该需求数量的空闲运算核上进行运算，然后在各空闲运算核对分片任务数据运算完毕后，将各空闲运算核运算得到的运算分片数据进行组合，得到当前卷积运算节点的输出数据，从而实现大模型切分运算，灵活地动态调度NPU核资源，使得多核能力可以进行整合，进一步实现了NPU运算核动态分配、同步、运算、释放的NPU多OS支持功能。
为了助于理解本申请，列举具体实施例一，请参照图7，图7为本申请实施例应用场景的数据处理方法的时序图，具体实施例一：
(1)由用户进程完成任务执行数据的下发工作，任务执行数据对应的任务结构为自定义的任务列表，其主要内容项可结合参照图8，MMU(Memory Management Unit，存储器管理单元)将应用抽象实例化为DDR(Double Data Rate，双倍速率同步动态随机存储器)中的任务列表，并保存地址映射关系，任务列表主要包括任务需要使用的核的数目(即需求使用NPU运算核的需求数量)、host(主机)侧与device(设备)侧交互的中断向量表、任务输入数据地址以及预留的输出数据空间；MMU完成host侧的虚拟地址翻译，达成虚拟空间地址到主机物理地址的转换，将地址映射关系记录到多级页表中，SMMU通过该多级页表同步host侧与device侧的转换关系，实现对统一物理地址的访问。
(2)VM(VM集成了SMMU单元、中断控制单元和DMA单元)通过寄存器下发调用NPU，主要完成页表同步及指令下发，即根据具体任务需求在SMMU专属于NPU的表项中新建一张任务表，将MMU部分涉及到的地址转换映射到SMMU中，达成访客地址到主机物理地址的转换，最终实现VM侧与NPU侧的物理地址统一；
(3)DMA根据SMMU中的地址翻译将DDR中任务输入数据地址中存储的任务执行数据搬移到NPU指定的运算核(即NPU运算核中处于空闲状态的空闲运算核)的输入buffer位置。具体地，此处的的运算核是由NPU control通过检索各核状态位(本实施例使用中断控制单元中的中断屏蔽位实现)来判断每个核处于工作状态还是空闲状态，根据任务中指定的core number(即需求使用NPU运算核的需求数量)来将对应数量的空闲运算核绑定给(1)中下发的任务执行数据。具体过程可为VM通过寄存器配置取用NPU，NPU接收取用信号，NPU Control(NPU控制模块)检查各运算核的运行状态，继而动态分配运算核的空闲资源，并通过DMA到任务列表的指定位置搬运任务执行数据到分配的运算核对应的输入buffer位置。
(4)NPU完成所有任务执行数据在指定buffer位置的填充，填充完毕后，NPU control触发运算核进行流水线运算，运算完成后，将运算结果搬运到相应的输出buffer位置。具体地，可结合参照图9，由于NPU的任务对象是神经网络模型，是一种多层的图结构，由众多业务节点组成，且节点之间存在严格的串行依赖关系。一方面，为了加速模型的执行且解决大模型的支持度问题，需要将一个模型任务拆分到多个运算核上执行；另一方面，由于各层依赖关系的存在，需要在多核执行时考虑核间的同步问题。具体地，当某一core(运算核)完成计算任务时，将运算结果输出到SRAM上，并发送中断信号给中断控制单元，中断控制单元判断是否所有core均完成计算任务，当所有core均完成计算任务时，中断控制单元发送中断信号给各core(运算核)，各core进行新的raw data以及关联的SRAM中数据的填充，并执行计算。重复上述步骤，直到完成网络模型全部节点的运算任务，其中，可利用中断控制单元实现多core间的核间同步，所有运算完成后，将运算结果输出到SRAM并发送中断信号给中断控制单元。
(5)DMA通过SMMU中记录的地址转换关系，将最终的运算结果搬运到DDR中的输出数据地址空间，具体地，NPU Control通过中断信号表映射中断行为，通过DMA将SRAM中的运算结果搬运到任务列表中指定的DDR输出地址。
(6)NPU发送IO中断信号到VM，VM接收中断信号，完成数据提取，判断是否继续处理，若无后续工作，回收内存空间，释放页表。
上述具体实施例一仅助于理解本申请的技术构思或技术原理，并不构成对本申请的限定，基于此进行更多形式的简单变换，均应在本申请的保护范围内。
此外，本申请实施例还提供一种应用场景的数据处理系统100，请参照图4，图4为本申请实施例应用场景的数据处理系统100的结构模块示意图，其中，应用场景的数据处理系统100包括相互连接的中央处理器1和神经网络处理器2，中央处理器1包括多个虚拟机VM模块11，神经网络处理器2包括NPU运算核22、NPU控制模块21和虚拟化控制VMU模块23：
各VM模块11分别绑定至少一种AI应用，其中，一种AI应用映射一种应用场景；
VMU模块23设置为获取来自于VM模块11中AI应用所产生的任务执行数据，其中，任务执行数据携带有需求使用NPU运算核22的需求数量；
NPU控制模块21设置为检测各NPU运算核22的运算核运行状态信息，根据运算核运行状态信息，确定各NPU运算核22中处于空闲状态的空闲运算核，并将任务执行数据分配至需求数量的空闲运算核上进行运算，得到运算结果数据；
VM模块11设置为根据运算结果数据进行当前应用场景的任务响应。
在本实施例中，中央处理器1是指CPU，神经网络处理器2是指NPU。
需要说明的是，为了使终端设备能同时运行多个AI应用场景，多个操作系统(OS，Operating System)需要能够在单个终端设备(例如车辆)上并行运行，其中，一个操作系统对应绑定至少一个AI应用(APP，Application)，一个AI应用映射一种应用场景。该应用场景为AI应用场景，该AI应用场景的类型已在前述中进行详细举例说明，在此不再赘述。
也即，一个操作系统至少对应支持运行一种AI应用场景。这些操作系统可称为“客机操作系统”(guest operating system)，它们可以包括一个操作系统的多个实例以及不同的操作系统。需要说明的是，一个VM(Virtual Machine，虚拟机)模块11对应运行一个操作系统，多个VM模块11可以在虚拟机监控器(VMM，Virtual Machine Monitor)的管理下运行。虚拟机监控器通常使用MMU模块12(Memory Management Unit，存储器管理单元)来支持VM模块11的地址转换(translation)和存储器保护。具体地，在工作时，由上层虚拟机VM模块11发布AI应用，MMU模块12将应用抽象实例化为DDR(Double Data Rate，双倍速率同步动态随机存储器)中的任务列表，并保存地址映射关系，任务列表记录下层绑定的核number(即需求使用NPU运算核的需求数量)，MMU(Memory Management Unit，存储器管理单元)模块12负责将来自VM模块11的任务执行数据对应的虚拟空间地址转换为主机物理地址，从而根据该主机物理地址将任务执行数据对应存储至DDR的主机物理地址中，并触发神经网络处理器2的外设取用，VMU模块23中集成了SMMU(System Memory Management Unit，设备端存储管理单元)单元231、中断控制单元233及DMA(Data Memory Access，直接存储访问)单元232。其中，SMMU单元231负责同步地址映射关系，将VM模块11发送的访客地址转换为主机物理地址，然后通过DMA单元232从DDR的主机物理地址中提取任务执行数据至神经网络处理器2内的寄存单元(buffer)，例如SRAM(Static Random-Access Memory，静态随机存取存储器)中，然后再由神经网络处理器2的NPU控制模块21将寄存单元中的任务执行数据搬运至该需求数量的空闲运算核上进行运算，通过各空闲运算核运算得到运算结果数据，多个NPU运行核22并行运行该任务执行数据时，中断控制单元实现核间同步，并通过DMA单元232将存储在寄存单元中的运算结果数据搬运至SRAM，供对应运行的虚拟机VM模块11取用，从而有效实现NPU上的多OS功能支持，可结合参照图5。
本申请实施例的应用场景的数据处理系统100包括相互连接的中央处理器1和神经网络处理器2，中央处理器1包括多个虚拟机VM模块11，神经网络处理器包括NPU运算核22、NPU控制模块21和虚拟化控制VMU模块23，其中，各VM模块11分别绑定至少一种AI应用，其中，一种AI应用映射一种应用场景，VMU模块23设置为获取来自于VM模块11中AI应用所产生的任务执行数据，其中，任务执行数据携带有需求使用NPU运算核22的需求数量，NPU控制模块21设置为检测各NPU运算核22的运算核运行状态信息，根据运算核运行状态信息，确定各NPU运算核22中处于空闲状态的空闲运算核，并将任务执行数据分配至需求数量的空闲运算核上进行运算，得到运算结果数据，VM模块11设置为根据运算结果数据进行当前应用场景的任务响应。
相比于目前一般采用的硬件隔离式和该手动绑核的方式，本申请实施例通过根据具体AI应用场景的任务执行数据，确定需求使用NPU运算核22的需求数量，并通过结合各NPU运算核22的运行状态信息，从各NPU运算核22中选取该需求数量的空闲NPU运算核，来运行支持当前应用场景，从而实现根据具体AI应用场景对NPU算力资源进行动态调度，避免NPU算力资源与应用场景强绑定，而导致在指定应用场景未启用时，该指定应用场景所绑定部分的算力资源处于闲置状态，造成NPU在实际运行环境下运行性能降低的问题，同时在有效实现NPU上多OS功能支持的同时，还具有较高的性能隔离能力及资源复用能力，提升系统安全，达到更高级别的功能安全需求。而本申请实施例通过提供一种能动态分配NPU核资源的NPU多OS支持机制，可在不同应用场景下对NPU运算核进行复用，提高了NPU算力资源的利用率，进而解决了NPU算力资源与应用场景强绑定，导致NPU运行算力低的技术问题。
请参照图4，在一种可能的实施方式中，中央处理器1还包括内存管理MMU模块12，VMU模块23包括内存管理SMMU单元231和直接存储访问DMA单元232，
MMU模块12设置为获取任务执行数据对应的虚拟空间地址，将虚拟空间地址转换为主机物理地址，并将访客物理地址，以及访客物理地址与主机物理地址的地址映射关系发送至SMMU单元231；
SMMU单元231设置为接收MMU模块12发送的访客物理地址和地址映射关系，根据发送的地址 映射关系将访客物理地址映射为主机物理地址，将映射为的主机物理地址发送至DMA单元232；
DMA单元232设置为接收SMMU单元231发送的主机物理地址，并从发送的主机物理地址中提取任务执行数据；
NPU控制模块21设置为将提取的任务执行数据搬运至需求数量的空闲运算核上运算，得到运算结果数据。
本实施例通过将中央处理器1设置包括内存管理MMU模块12，并将该VMU模块23包括内存管理SMMU单元231和直接存储访问DMA单元232，其中，MMU模块12设置为获取任务执行数据对应的虚拟空间地址，将虚拟空间地址转换为主机物理地址，并将访客物理地址，以及访客物理地址与主机物理地址的地址映射关系发送至SMMU单元231；SMMU单元231设置为接收MMU模块12发送的访客物理地址和地址映射关系，根据发送的地址映射关系将访客物理地址映射为主机物理地址，将映射为的主机物理地址发送至DMA单元232；DMA单元232设置为接收SMMU单元231发送的主机物理地址，并从发送的主机物理地址中提取任务执行数据；NPU控制模块21设置为将提取的任务执行数据搬运至需求数量的空闲运算核上运算，得到运算结果数据，从而提升了VM与NPU的IO(Input Output，输入输出)实时性，实现根据具体AI应用场景对NPU算力资源进行动态调度，避免NPU算力资源与应用场景强绑定，而导致在指定应用场景未启用时，该指定应用场景所绑定部分的算力资源处于闲置状态，造成NPU在实际运行环境下运行性能降低的问题，进而有效达成NPU多OS能力的支持。
为了助于理解本申请，列举具体实施例二，请参照图5和图6，图6为本申请实施例VMU的模块结构示意图，图5为本申请实施例应用场景的数据处理系统的场景交互图，包括：
①虚拟机VM；②MMU；③DDR；④Hypervisor(虚拟机监控器)；⑤NPU Control；⑥VMU；⑦SRAM；⑧NPU运算核，其中，
①虚拟机VM负责发布应用并绑定所需的操作系统；
②MMU为内存管理单元，负责完成虚拟机系统空间到物理空间的映射；
③DDR为物理内存空间，负责存储实例化后的NPU任务，标识每个任务的运算核数量、执行数据存储位置、数据回收位置、中断接收位置；
④Hypervisor为虚拟机监控器，负责MMU、SMMU间的页表共享和同步；
⑤NPU Control为NPU控制核心，负责分配和绑定NPU运算核，控制各模块流水线作业；
⑥VMU集成了SMMU、中断控制单元及DMA，分别负责设备端地址映射，核间数据同步及数据搬运；
⑦SRAM为缓存单元，存储中间和结果数据；
⑧NPU运算核为NPU中的运算模块，负责承担上层应用中具体算子的执行加速。
上述具体实施例二仅助于理解本申请的技术构思或技术原理，并不构成对本申请的限定，基于此进行更多形式的简单变换，均应在本申请的保护范围内。
在一种可能的实施方式中，请参照图4，VMU模块23还包括中断控制单元233，
NPU控制模块21设置为将初始卷积运算节点作为当前卷积运算节点，并将任务执行数据作为当前卷积运算节点的输入数据，将当前卷积运算节点的输入数据分配至需求数量的空闲运算核上进行运算；
中断控制单元233设置为判断各空闲运算核是否对当前卷积运算节点的输入数据运算完毕，在各空闲运算核对当前卷积运算节点的输入数据全部运算完毕后，发送第一中断信号至NPU控制模块21；
NPU控制模块21设置为根据第一中断信号将当前卷积运算节点对应下一卷积运算节点作为新的当前卷积运算节点，将原来的当前卷积运算节点作为上一卷积运算节点，并将上一卷积运算节点的输出数据作为当前卷积运算节点的输入数据，将当前卷积运算节点的输入数据分配至需求数量的空闲运算核上进行运算，得到当前卷积运算节点的输出数据，直至所有卷积运算节点均运算完成，得到运算结果数据。
在本实施例中，本领域技术人员可知的是，该中断控制单元233可对中断机制实现优化，提升VM与NPU的IO实时性，同时，可以实现内部运算核间的数据同步，使得多核能力可以进行整合，从而实现大模型切分运算。
本实施例通过NPU控制模块21将初始卷积运算节点作为当前卷积运算节点，并将该任务执行数据作为当前卷积运算节点的输入数据，并将当前卷积运算节点的输入数据分配至该需求数量的空闲运算核上进行运算，并通过中断控制单元233判断各空闲运算核是否对当前卷积运算节点的输入数据运算完毕，在各空闲运算核对当前卷积运算节点的输入数据全部运算完毕后，发送第一中断信号至NPU控制模块21，然后通过NPU控制模块21根据第一中断信号将当前卷积运算节点对应下一卷积运算节点作为新的当前卷积运算节点，将原来的当前卷积运算节点作为上一卷积运算节点，并将上一卷积运算节点的输出数据作为当前卷积运算节点的输入数据，将当前卷积运算节点的输入数据分配至需求数量的空闲运算核上进行运算，得到当前卷积运算节点的输出数据，直至所有卷积运算节点均运算完成，得到运算结果数据从而实现NPU内部的各NPU运算核22间的数据同步，使得多核能力可以进行整合，实现NPU运算核22动态分配、同步、运算、释放的NPU多OS功能。
为了助于理解，列举具体实施例三，可参照图9，由于NPU的任务对象是神经网络模型，是一种多层的图结构，由众多业务节点组成，且节点之间存在严格的串行依赖关系。一方面，为了加速模型的执行且解决大模型的支持度问题，需要将一个模型任务拆分到多个运算核上执行；另一方面，由于各层依赖关系的存在，需要在多核执行时考虑核间的同步问题。具体实施例三为：
(1)当任务列表指定的core(运算核)数量大于1时，多个NPU运算核需要组合在一起实现一个网络模型的运算加速。
(2)NPU首先进行任务的切割，将多通道的raw数据分配到不同core上运算；
(3)当某一core完成计算任务时，将运算结果输出到SRAM上，并发送中断信号给中断控制单元，中断控制单元判断是否所有core均完成计算任务；
(4)当所有core均完成计算任务时，中断控制单元发送中断信号给各core，各core进行新的raw data以及关联的SRAM中数据的填充，并执行计算；
(5)重复以上步骤，直到完成网络模型全部节点的运算任务，将SRAM中的运算结果数据通过DMA搬运到对应VM任务列表的output位置，从而实现NPU运算核22动态分配、同步、运算、释放的NPU多OS支持功能。
需要说明的是，上述具体实施例三仅助于理解本申请的技术构思或技术原理，并不构成对本申请的限定，基于此进行更多形式的简单变换，均应在本申请的保护范围内。
请参照图4，在一种可能的实施方式中，NPU控制模块21设置为将当前卷积运算节点的输入数据切分成需求数量份的分片任务数据，并将各分片任务数据对应分配至需求数量的空闲运算核上进行运算；
中断控制单元233设置为判断各空闲运算核是否对分片任务数据运算完毕，在各空闲运算核对分片任务数据全部运算完毕后，发送第二中断信号至NPU控制模块21；
NPU控制模块21设置为根据第二中断信号将各空闲运算核运算得到的运算分片数据进行组合，得到当前卷积运算节点的输出数据。
本实施例通过NPU控制模块21将当前卷积运算节点的输入数据切分成该需求数量份的分片任务数据，并将各分片任务数据对应分配至该需求数量的空闲运算核上进行运算，然后通过中断控制单元233判断各空闲运算核是否对分片任务数据运算完毕，在各空闲运算核对分片任务数据全部运算完毕后，发送第二中断信号至NPU控制模块21，再通过NPU控制模块21根据该第二中断信号将各空闲运算核运算得到的运算分片数据进行组合，得到当前卷积运算节点的输出数据，从而实现大模型切分运算，灵活 地动态调度NPU运算核22的算力资源，使得多核能力可以进行整合，进一步实现了NPU运算核22动态分配、同步、运算、释放的NPU多OS支持功能。
此外，本申请实施例还提供一种电子设备，请参照图10，图10为本申请一实施例提供的一种电子设备的硬件结构示意图。如图10所示，电子设备可以包括：中央处理器(Central Processing Unit，CPU)1001，通信总线1002、用户接口1003，网络接口1004，存储器1005，神经网络处理器(neural-network process units，NPU)1006。其中，通信总线1002用于实现这些组件之间的连接通信。用户接口1003可以包括显示屏(Display)、输入单元比如键盘(Keyboard)，用户接口1003还可以包括标准的有线接口、无线接口。网络接口1004可以包括标准的有线接口、无线接口(如无线保真(Wireless-Fidelity，WI-FI接口)。存储器1005可以是高速的随机存取存储器(Random Access Memory，RAM)存储器，也可以是稳定的非易失性存储器(Non-Volatile Memory，NVM)，例如磁盘存储器。存储器1005还可以是独立于前述中央处理器1001和/或神经网络处理器1006的存储设备。
本领域技术人员可以理解，图10中示出的结构并不构成对电子设备的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。如图10所示，作为一种存储介质的存储器1005中可以包括操作系统、数据存储模块、网络通信模块、用户接口模块以及计算机程序。
在图10所示的电子设备中，网络接口1004主要用于与其他设备进行数据通信；用户接口1003主要用于与用户进行数据交互；本实施例中的中央处理器1001、神经网络处理器1006、存储器1005可以设置在电子设备中，电子设备通过处理器1001和/或神经网络处理器1006调用存储器1005中存储的计算机程序，并执行上述任一实施例提供的应用于电子设备的应用场景的数据处理方法。
本实施例提出的终端与上述实施例提出的应用于电子设备的应用场景的数据处理方法属于同一发明构思，未在本实施例中详尽描述的技术细节可参见上述任意实施例，并且本实施例具备与执行应用场景的数据处理方法相同的有益效果。
此外，本申请还提供了一种可读存储介质，所述可读存储介质为计算机可读存储介质，所述计算机可读存储介质存储有一个或者一个以上程序，所述一个或者一个以上程序还可被一个或者一个以上的处理器执行以用于实现上述存储器访问方法各实施例的步骤。
本申请计算机可读存储介质具体实施方式与上述存储器访问方法各实施例基本相同，在此不再赘述。
需要说明的是，在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者系统不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者系统所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、方法、物品或者系统中还存在另外的相同要素。
上述本申请实施例序号仅仅为了描述，不代表实施例的优劣。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在如上所述的一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端设备(可以是手机，计算机，服务器，空调器，或者网络设备等)执行本申请各个实施例所述的方法。
以上仅为本申请的可选实施例，并非因此限制本申请的专利范围，凡是利用本申请说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本申请的专利保护范围内。

Claims (10)

  1. 一种应用场景的数据处理方法，包括：
    获取当前应用场景对应的任务执行数据，其中，所述任务执行数据携带有需求使用NPU运算核的需求数量；
    检测各所述NPU运算核的运算核运行状态信息，根据所述运算核运行状态信息，确定各所述NPU运算核中处于空闲状态的空闲运算核；
    将所述任务执行数据分配至所述需求数量的空闲运算核上进行运算，得到运算结果数据，并根据所述运算结果数据进行当前应用场景的任务响应。
  2. 如权利要求1所述的应用场景的数据处理方法，其中，所述将所述任务执行数据分配至所述需求数量的空闲运算核上进行运算，得到运算结果数据的步骤包括：
    获取所述任务执行数据对应的虚拟空间地址，将所述虚拟空间地址转换为主机物理地址；
    从所述主机物理地址中提取所述任务执行数据；
    将提取的任务执行数据搬运至所述需求数量的空闲运算核上进行运算，得到运算结果数据。
  3. 如权利要求1所述的应用场景的数据处理方法，其中，所述将所述任务执行数据分配至所述需求数量的空闲运算核上进行运算，得到运算结果数据的步骤包括：
    将初始卷积运算节点作为当前卷积运算节点，并将所述任务执行数据作为当前卷积运算节点的输入数据；
    将当前卷积运算节点的输入数据分配至所述需求数量的空闲运算核上进行运算，得到当前卷积运算节点的输出数据；
    将当前卷积运算节点对应下一卷积运算节点作为新的当前卷积运算节点，并将原来的当前卷积运算节点作为上一卷积运算节点；
    将上一卷积运算节点的输出数据作为当前卷积运算节点的输入数据，并返回执行所述将当前卷积运算节点的输入数据分配至所述需求数量的空闲运算核上进行运算的步骤，直至所有卷积运算节点均运算完成，得到运算结果数据。
  4. 如权利要求3所述的应用场景的数据处理方法，其中，所述将当前卷积运算节点的输入数据分配至所述需求数量的空闲运算核上进行运算，得到当前卷积运算节点的输出数据的步骤包括：
    将当前卷积运算节点的输入数据切分成所述需求数量份的分片任务数据，并将各所述分片任务数据对应分配至所述需求数量的空闲运算核上进行运算；
    在各空闲运算核对所述分片任务数据运算完毕后，将各空闲运算核运算得到的运算分片数据进行组合，得到当前卷积运算节点的输出数据。
  5. 一种应用场景的数据处理系统，其中，所述应用场景的数据处理系统包括相互连接的中央处理器和神经网络处理器，所述中央处理器包括多个虚拟机VM模块，所述神经网络处理器包括NPU运算核、NPU控制模块和虚拟化控制VMU模块：
    各所述VM模块分别绑定至少一种AI应用，其中，一种AI应用映射一种应用场景；
    所述VMU模块设置为获取来自于所述VM模块中AI应用所产生的任务执行数据，其中，所述任务执行数据携带有需求使用NPU运算核的需求数量；
    所述NPU控制模块设置为检测各所述NPU运算核的运算核运行状态信息，根据所述运算核运行状态信息，确定各所述NPU运算核中处于空闲状态的空闲运算核，并将所述任务执行数据分配至所述需求数量的空闲运算核上进行运算，得到运算结果数据；
    所述VM模块设置为根据所述运算结果数据进行当前应用场景的任务响应。
  6. 如权利要求5所述的应用场景的数据处理系统，其中，所述中央处理器还包括内存管理MMU模块，所述VMU模块包括内存管理SMMU单元和直接存储访问DMA单元，
    所述MMU模块设置为获取所述任务执行数据对应的虚拟空间地址，将所述虚拟空间地址转换为主机物理地址，并将访客物理地址，以及所述访客物理地址与主机物理地址的地址映射关系发送至所述SMMU单元；
    所述SMMU单元设置为接收所述MMU模块发送的所述访客物理地址和所述地址映射关系，根据发送的地址映射关系将所述访客物理地址映射为主机物理地址，将映射为的主机物理地址发送至DMA单元；
    所述DMA单元设置为接收所述SMMU单元发送的主机物理地址，并从发送的主机物理地址中提取所述任务执行数据；
    所述NPU控制模块设置为将提取的任务执行数据搬运至所述需求数量的空闲运算核上运算，得到运算结果数据。
  7. 如权利要求5所述的应用场景的数据处理系统，其中，所述VMU模块还包括中断控制单元，
    所述NPU控制模块设置为将初始卷积运算节点作为当前卷积运算节点，并将所述任务执行数据作为当前卷积运算节点的输入数据，将当前卷积运算节点的输入数据分配至所述需求数量的空闲运算核上进行运算；
    所述中断控制单元设置为判断各所述空闲运算核是否对当前卷积运算节点的输入数据运算完毕，在各所述空闲运算核对当前卷积运算节点的输入数据全部运算完毕后，发送第一中断信号至所述NPU控制模块；
    所述NPU控制模块设置为根据所述第一中断信号将当前卷积运算节点对应下一卷积运算节点作为新的当前卷积运算节点，将原来的当前卷积运算节点作为上一卷积运算节点，并将上一卷积运算节点的输出数据作为当前卷积运算节点的输入数据，将当前卷积运算节点的输入数据分配至所述需求数量的空闲运算核上进行运算，得到当前卷积运算节点的输出数据，直至所有卷积运算节点均运算完成，得到运算结果数据。
  8. 如权利要求7所述的应用场景的数据处理系统，其中，所述NPU控制模块设置为将当前卷积运算节点的输入数据切分成所述需求数量份的分片任务数据，并将各所述分片任务数据对应分配至所述需求数量的空闲运算核上进行运算；
    所述中断控制单元设置为判断各所述空闲运算核是否对所述分片任务数据运算完毕，在各所述空闲运算核对所述分片任务数据全部运算完毕后，发送第二中断信号至所述NPU控制模块；
    所述NPU控制模块设置为根据所述第二中断信号将各空闲运算核运算得到的运算分片数据进行组合，得到当前卷积运算节点的输出数据。
  9. 一种电子设备，包括如上权利要求5至8中任一项所述的中央处理器、神经网络处理器、存储器、以及存储在所述存储器上并可在所述中央处理器和/或所述神经网络处理器上运行的应用场景的数据处理程序，所述应用场景的数据处理程序被所述中央处理器和/或所述神经网络处理器执行时实现如权利要求1至4中任一项所述应用场景的数据处理方法的步骤。
  10. 一种存储介质，其中，所述存储介质为计算机可读存储介质，所述计算机可读存储介质上存储有应用场景的数据处理程序，所述应用场景的数据处理程序被处理器执行时实现如权利要求1至4中任一项所述的应用场景的数据处理方法的步骤。
PCT/CN2023/085999 2022-10-25 2023-04-03 应用场景的数据处理方法、系统、电子设备及存储介质 WO2024087513A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211315722.7A CN117973465A (zh) 2022-10-25 2022-10-25 应用场景的数据处理方法、系统、电子设备及存储介质
CN202211315722.7 2022-10-25

Publications (1)

Publication Number Publication Date
WO2024087513A1 true WO2024087513A1 (zh) 2024-05-02

Family

ID=90829838

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/085999 WO2024087513A1 (zh) 2022-10-25 2023-04-03 应用场景的数据处理方法、系统、电子设备及存储介质

Country Status (2)

Country Link
CN (1) CN117973465A (zh)
WO (1) WO2024087513A1 (zh)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160224053A1 (en) * 2015-01-30 2016-08-04 Qualcomm Incorporated Timer-based processing unit operational scaling employing timer resetting on idle process scheduling
CN113128673A (zh) * 2019-12-31 2021-07-16 Oppo广东移动通信有限公司 数据处理方法、存储介质、神经网络处理器及电子设备
CN114218147A (zh) * 2021-04-30 2022-03-22 无锡江南计算技术研究所 一种计算核心加速的通信优化方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160224053A1 (en) * 2015-01-30 2016-08-04 Qualcomm Incorporated Timer-based processing unit operational scaling employing timer resetting on idle process scheduling
CN113128673A (zh) * 2019-12-31 2021-07-16 Oppo广东移动通信有限公司 数据处理方法、存储介质、神经网络处理器及电子设备
CN114218147A (zh) * 2021-04-30 2022-03-22 无锡江南计算技术研究所 一种计算核心加速的通信优化方法

Also Published As

Publication number Publication date
CN117973465A (zh) 2024-05-03

Similar Documents

Publication Publication Date Title
US10180843B2 (en) Resource processing method and device for a multi-core operating system
US10191759B2 (en) Apparatus and method for scheduling graphics processing unit workloads from virtual machines
CN111258744A (zh) 一种基于异构计算的任务处理方法及软硬件框架系统
WO2018119952A1 (zh) 一种设备虚拟化方法、装置、系统及电子设备、计算机程序产品
US8219722B2 (en) DMA and graphics interface emulation
US11010859B2 (en) Display resource scheduling method and device for embedded system
CN106844007A (zh) 一种基于空间复用的虚拟化方法与系统
CN111309649B (zh) 一种数据传输和任务处理方法、装置及设备
US9201823B2 (en) Pessimistic interrupt affinity for devices
CN113641413B (zh) 目标模型加载更新方法及装置、可读介质和电子设备
US9003094B2 (en) Optimistic interrupt affinity for devices
WO2021136512A1 (zh) 基于深度学习节点计算的调度方法、设备及存储介质
CN104025185A (zh) 用于使用gpu控制器来预加载缓存的机制
WO2022105187A1 (zh) 一种内存管理的方法、设备及系统
CN111274019A (zh) 一种数据处理方法、装置及计算机可读存储介质
CN109254826A (zh) 虚拟化加速处理装置的挂起检测
CN116501512A (zh) 使用多播的高效多设备同步屏障
CN111338769A (zh) 一种数据处理方法、装置及计算机可读存储介质
US8402229B1 (en) System and method for enabling interoperability between application programming interfaces
US20230185595A1 (en) Method for realizing live migration, chip, board, and storage medium
CN113377529A (zh) 一种智能加速卡及基于智能加速卡的数据处理方法
WO2024087513A1 (zh) 应用场景的数据处理方法、系统、电子设备及存储介质
CN113568734A (zh) 基于多核处理器的虚拟化方法、系统、多核处理器和电子设备
CN111310638A (zh) 一种数据处理方法、装置及计算机可读存储介质
CN116360925A (zh) 一种半虚拟化实现方法、装置、设备及介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23881110

Country of ref document: EP

Kind code of ref document: A1