WO2018188463A1 - 面向神经网络异构计算平台的编程模型 - Google Patents

面向神经网络异构计算平台的编程模型 Download PDF

Info

Publication number
WO2018188463A1
WO2018188463A1 PCT/CN2018/079954 CN2018079954W WO2018188463A1 WO 2018188463 A1 WO2018188463 A1 WO 2018188463A1 CN 2018079954 W CN2018079954 W CN 2018079954W WO 2018188463 A1 WO2018188463 A1 WO 2018188463A1
Authority
WO
WIPO (PCT)
Prior art keywords
neural network
host
file
dedicated processor
computing platform
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2018/079954
Other languages
English (en)
French (fr)
Inventor
孙晓明
隋凌志
罗洪
单羿
姚颂
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Deephi Intelligent Technology Co Ltd
Original Assignee
Beijing Deephi Intelligent Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Deephi Intelligent Technology Co Ltd filed Critical Beijing Deephi Intelligent Technology Co Ltd
Publication of WO2018188463A1 publication Critical patent/WO2018188463A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/40—Transformation of program code
    • G06F8/41—Compilation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/30—Creation or generation of source code
    • G06F8/31—Programming languages or programming paradigms
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/30—Creation or generation of source code
    • G06F8/31—Programming languages or programming paradigms
    • G06F8/315—Object-oriented languages
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0495—Quantised networks; Sparse networks; Compressed networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/10—Interfaces, programming languages or software development kits, e.g. for simulating neural networks
    • G06N3/105—Shells for specifying net layout
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/40—Transformation of program code
    • G06F8/54—Link editing before load time
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/70—Software maintenance or management
    • G06F8/71—Version control; Configuration management

Definitions

  • the invention relates to a heterogeneous computing platform and a neural network, and more particularly to a programming model for a neural network heterogeneous computing platform.
  • a programming model for neural network heterogeneous computing platform is needed to effectively solve various difficulties in the development, compilation, deployment and operation phases of neural network applications in heterogeneous environments of CPU+neural network dedicated processors.
  • the present invention focuses on a series of efficient compilation and runtime support techniques for neural network heterogeneous computing platforms. More specifically, the object of the present invention is to provide a programming model for a neural network heterogeneous computing platform, thereby innovatively and effectively solving the development, compilation, deployment, and deployment of neural network applications in a heterogeneous environment of a CPU+neural network dedicated processor. Various difficulties in the operational phase.
  • a method for compiling a heterogeneous computing platform includes: inputting a trained neural network model to a neural network (NN) optimization compiler, and generating a corresponding neural network NN assembly file; input NN assembly file into NN assembler to generate NN binary file corresponding to neural network; compile and assemble user neural network application developed in high-level language using host compiler tool chain, and then generate corresponding host assembly File and host binaries; use the host linker to link NN binaries and host binaries to generate a single mixed-link executable.
  • NN neural network
  • the binary file is an ELF binary file
  • the single hybrid linked executable file is a single hybrid link ELF executable file
  • the high-level language is C/C++.
  • the single hybrid link executable file includes: an ELF standard executable file portion and an NN extension portion.
  • the ELF standard executable file portion further includes: an ELF file header, a segment table, a code segment, and a data segment.
  • the NN extension further includes: an NN header, a code segment, and a data segment.
  • a program execution support method for a heterogeneous computing platform including: a user implements a neural network dedicated processor core by calling a high-level language application programming interface (API) of a neural network dedicated processor development library.
  • API application programming interface
  • a compiling system for a heterogeneous computing platform comprising: a neural network (NN) optimization compiler, configured to receive a trained neural network model, and generate an NN assembly file corresponding to the neural network; NN assembler for receiving NN assembly files, generating NN binary files corresponding to neural networks; host compiler for compiling neural network applications developed by users in high-level languages, generating corresponding host assembly files; host assembler, Used to assemble host assembly files, generate host binaries; host linker for linking NN binaries and host binaries to generate a single hybrid link executable.
  • NN neural network
  • the binary file is an ELF binary file
  • the single hybrid linked executable file is a single hybrid link ELF executable file
  • the high-level language is C/C++.
  • the single hybrid link executable file includes: an ELF standard executable file portion and an NN extension portion.
  • the ELF standard executable file portion further includes: an ELF file header, a segment table, a code segment, and a data segment.
  • the NN extension further includes: an NN header, a code segment, and a data segment.
  • a program execution support system for a heterogeneous computing platform including: a host; a neural network dedicated processor; a neural network application; a neural network dedicated processor development library, which is used for encapsulating a neural network.
  • Various high-level language application programming interfaces of the processor for providing a user with a neural network application call, implementing initialization operations and runtime control of the neural network dedicated processor core; heterogeneous computing platform according to the third aspect of the present invention a compiling system for generating a hybrid link executable file; a standard loader of the operating system for loading a code segment and a data segment of a host in the executable file into a memory space of the host; a neural network dedicated processor loader a code segment and a data segment for loading a neural network dedicated processor in the executable file into a memory space of a neural network dedicated processor; a host operating system; a neural network dedicated processor driver for operating with the host
  • the system works together to make the instructions in the host code segment and neural network specific The instructions in the processor code segment are executed serially in the order set by the neural network application.
  • the technical solution of the invention has the characteristics of good calculation performance, strong scalability, strong compatibility and high flexibility.
  • Figure 1 is a hybrid compilation model of a heterogeneous computing platform for a CPU+neural network dedicated processor
  • FIG. 2 is a flowchart of a compiling method of a heterogeneous computing platform according to a first embodiment of the present invention
  • FIG. 3 is a schematic diagram of a CPU+neural network dedicated processor hybrid link ELF executable file
  • FIG. 4 is a program execution support model of a heterogeneous computing platform in accordance with a second embodiment of the present invention.
  • the present invention Compared with a single computing platform (ie computing platform with only host or CPU), the present invention also includes the following functional components: 1) neural network optimization compiler; 2) NN assembler; 3) CPU + neural network dedicated processor Hybrid link ELF (Executable and Linking Format) executable file; 4) neural network dedicated processor driver; 5) runtime neural network dedicated processor loader (Loader); 6) neural network dedicated processor development Library.
  • neural network dedicated processor as used in the present application may also be simply referred to as “neural network processor” or "NN processor”. Since deep learning is currently the most popular technical classification in neural network technology, a neural network dedicated processor can be implemented as a deep learning dedicated processor or a deep learning processor. However, those skilled in the art should understand that there are other technical branches of the neural network, such as Deep Neutral Network (DNN), so the neural network dedicated processor can also be implemented as a deep neural network dedicated processor or deep neural network. Network processor (DNN processor). That is, techniques relating to the compilation and operational support of "deep learning processors” or “deep neural network processors” in heterogeneous computing platforms are also within the scope of the present invention.
  • DNN processor Deep Neutral Network
  • the functional components 1), 2), 3) described above provide compilation support for neural networks on a heterogeneous platform of CPU + neural network dedicated processors, ie, relating to the present invention.
  • the functional components 4), 5), 6) provide relevant program operation support for the CPU + neural network dedicated processor heterogeneous neural network application, ie the second relating to the invention Example.
  • Figure 1 is a hybrid compilation model of heterogeneous computing platform for CPU+neural network dedicated processor.
  • a neural network (NN network) optimization compiler takes a neural network model as an input, and obtains control flow and data in the model by analyzing the topology of the network. Flow information, on the basis of which various optimization transformation techniques are applied to the model. Specifically, the compiler incorporates computational operations between different network layers in the neural network model to reduce computational strength. For both structured and unstructured sparse networks, the compiler will eliminate unnecessary calculations and data movements caused by sparse values. In addition, the compiler will fully multiplex the network parameters and feature map values stored in the on-chip memory of the neural network dedicated processor to reduce the memory bandwidth requirements and thus reduce power consumption. The compiler outputs the NN assembly file to the NN assembler.
  • the NN assembler receives the NN assembly file from the NN optimizing compiler.
  • the received NN assembly file (assembly instruction) is converted by the NN assembler to generate the corresponding binary machine code.
  • the machine code and data of the neural network dedicated processor are integrated to generate an ELF binary file in a standard format.
  • the host linker is used to generate a CPU+neural network dedicated processor heterogeneous hybrid link ELF executable.
  • the format of the ELF executable file can be extended to increase the support of the neural network dedicated processor.
  • the neural network dedicated processor and the host two ELF binary files are linked together to generate a single executable file (ELF executable file) to facilitate the deployment and operation of the neural network application.
  • ELF executable file an executable file
  • both the neural network dedicated processor and the host ELF binary are shown as ELF obiect files.
  • Figure 3 shows an example of a CPU+Neural Network Dedicated Processor Hybrid Link ELF executable file, which is described in further detail below.
  • the host ELF binary file used by the host linker is compiled by the host compiler for the C/C++ neural network program and compiled by the host assembler for the generated host assembly file.
  • the host compiler, the host assembler, and the host linker are all existing functional components in the compilation model of a single computing platform.
  • FIG. 2 is a flow chart of a method of compiling a heterogeneous computing platform in accordance with a first embodiment of the present invention.
  • the compiling method 200 of the heterogeneous computing platform begins in step S201.
  • the trained neural network model is input to the NN optimization compiler, and an NN assembly file corresponding to the neural network is generated.
  • step S203 the NN assembly file is input to the NN assembler, and an NN binary file corresponding to the neural network is generated.
  • step S205 the host compiler tool chain is used to compile and assemble the neural network application developed by the user in a high-level language, and the corresponding host assembly file and host binary file are sequentially generated.
  • the above-mentioned high-level languages are embodied as C/C++.
  • C/C++ is a high level language that is typical and commonly used in the art, thereby improving the programmability of the present invention.
  • the high-level language described herein may also be a high-level language other than C/C++, for example, including but not limited to: Java, Pascal, Python, Perl, and the like, an object-oriented high-level language. Or it may be typical or commonly used on a programming platform prior to the present invention, and may also be other high-level languages that are typical or commonly used on future programming platforms that the inventors of the present invention can foresee.
  • step S207 the NN binary file and the host binary file are linked using the host linker to generate a single hybrid link executable file.
  • the binary files mentioned above are all embodied as ELF binary files (or ELF files).
  • ELF Executable and Linking Format
  • the binary file described herein may also be a file format other than the ELF file, which may be typical or commonly used on the programming platform before the present invention, and may also be inventor of the present invention. Typical or commonly used on future programming platforms that can be foreseen.
  • the single hybrid link executable file generated in step S207 may be a single hybrid link ELF executable file.
  • Figure 3 shows an example of this.
  • the single hybrid link executable file generated in step S207 may include two parts: an ELF standard executable file part and an NN extension part.
  • the ELF standard executable file part is the same as the ordinary executable file, and further includes: an ELF file header (shown as "ELF Header” in FIG. 3), a segment table (shown as “Section Header Table” in FIG. 3), and a code segment ( Figure 3 shows ".text"), the data segment (shown as ".data” in Figure 3).
  • the NN extension is an extension to the ELF standard executable section, which is linked with the ELF standard executable section, and further includes: a NN header (shown as "NN Header” in Figure 3), and a code section (shown in Figure 3). .text”), data segment (shown in Figure 3 as ".data”).
  • the NN extension part is linked after the ELF standard executable file
  • the link mode and composition structure of the CPU+neural network dedicated processor hybrid link executable file may also be other than FIG. form. That is, the single hybrid link executable file according to the first embodiment of the present invention is not limited to the form of the example given in FIG. 3, and may be CPU+Neural Network-specific processing composed of any link form and content. Mixed link executable file.
  • compilation method 200 can be considered to be complete.
  • the generated executable can be launched and run under the support of a programming model for the neural network heterogeneous computing platform.
  • a compilation system of a heterogeneous computing platform may include an NN optimization compiler for receiving a trained neural network model and generating a NN assembly file corresponding to the neural network.
  • the NN assembler in the compilation system is used to receive the NN assembly file from the NN optimization compiler to generate a NN binary file corresponding to the neural network.
  • the host compiler of the compiling system is used to compile the neural network application developed by the user in a high-level language, and generate a corresponding host assembly file.
  • the host assembler is used to assemble the host assembly file and generate host binaries.
  • the high-level language can be C/C++ or other high-level language for improving system programmability.
  • the build system still uses the host linker to link NN binaries and host binaries, but generates a single mixed link executable.
  • the binary file may be an ELF binary file
  • the single hybrid link executable file may be a single hybrid link ELF executable file
  • the single hybrid link executable file can include: an ELF standard executable file portion and an NN extension portion.
  • the ELF standard executable file part may further include: an ELF file header, a segment table, a code segment, and a data segment.
  • the NN extension can further include: an NN header, a code segment, and a data segment.
  • the example given in Figure 3 can be seen again.
  • the single hybrid link executable file according to the first embodiment of the present invention is not limited to the form of the example given in FIG. 3, and may be a CPU+neural network dedicated processor combination of any link form and content composition. Link the executable file.
  • FIG. 4 is a program execution support model of a heterogeneous computing platform in accordance with a second embodiment of the present invention.
  • the neural network dedicated processor is simply referred to as a DPU to be distinguished from the host CPU.
  • DPU the neural network dedicated processor
  • Those skilled in the art will appreciate that such naming does not affect the generality of a neural network dedicated processor. That is, in the present specification and the drawings, "neural network dedicated processor” and “DPU” are terms that are used interchangeably to denote another processor on the heterogeneous computing platform that is different from the CPU.
  • the neural network dedicated processor driver (shown as "DPU driver” in FIG. 4) operates in the kernel space of the operating system and is responsible for dynamic management of the operation of the neural network dedicated processor resources. And scheduling, and responsible for the management and control of neural network code (hereinafter referred to as neural network dedicated processor core) running on a neural network dedicated processor, responsible for the management and allocation of the neural network dedicated processor memory space.
  • DPU driver the neural network dedicated processor driver
  • the neural network dedicated processor loader (shown as "DPU loader” in Figure 4), at runtime, works in the operating system user space (user space), is responsible for the dynamic loading and release of the neural network dedicated processor core, and is responsible for the nerve Relocation of network-specific processor core code.
  • the neural network dedicated processor development library (shown as "DPU Library” in Figure 4) operates in the operating system user space (user space), which is used to encapsulate various C/C++ application programming interfaces (APIs) for neural network dedicated processors. It is called when the user develops various neural network applications, thereby improving the programming efficiency on the neural network dedicated processor.
  • DPU Library operating system user space
  • APIs application programming interfaces
  • a host or CPU
  • a neural network dedicated processor shown as “DPU” in FIG. 4
  • a host operating system or an operating system standard loader (OS Loader)
  • OS Loader operating system standard loader
  • CPU + neural network dedicated processor heterogeneous computing platform developed neural network application, mainly composed of two parts: C / C + + code, trained neural network model (network topology and neural network parameters). From development to compilation to deployment and operation, the typical workflow is as follows:
  • Step 1 The user implements the initialization operation of the neural network dedicated processor core and various control of the runtime by calling the C/C++ API programming interface of the neural network dedicated processor development library, such as dpuInit(), dpuFini(), dpuCreateKernel(), dpuDestroyKernel(), dpuRunKernel(), etc.
  • the "dpu” therein represents a neural network dedicated processor.
  • Step 2 Enter the trained neural network model to optimize the compiler for the NN and generate a NN assembly file corresponding to the network.
  • Step 3 Enter the NN assembly file to the NN assembler to generate the ELF binary file corresponding to the network.
  • Step 4 Compile the user-developed C/C++ neural network application using the host compiler toolchain to generate the corresponding host assembly file and host ELF binary file.
  • Step 5 Use the host linker to link all of the ELF binary files of the neural network dedicated processor and host to generate a single hybrid link ELF executable.
  • Step 6 Compile the generated single hybrid link ELF executable file contains all the information needed for the host and neural network dedicated processor to run. It is executed in exactly the same way as a normal host ELF executable file, enabling one-click deployment.
  • Step 7 Start the hybrid link ELF executable file, the code segment and the data segment of the host in the ELF are loaded into the host memory space by the standard loader of the operating system; the code segment and the data segment of the neural network dedicated processor in the ELF are made by the nerve
  • the network dedicated processor loader is responsible for loading into the memory space of the neural network dedicated processor (where the memory space of the neural network dedicated processor is managed by the neural network dedicated processor driver).
  • Step 8 The instructions in the host code segment and the instructions in the neural network-specific processor code segment are serially executed in the order set by the neural network application.
  • the computational and data-intensive neural network algorithm is in the form of a neural network dedicated processor core. The host offloads to the neural network dedicated processor for accelerated operation and runs more efficiently.
  • the user implements initialization of the neural network dedicated processor core by calling a high-level language application programming interface (API) of the neural network dedicated processor development library. Operation and runtime control.
  • API application programming interface
  • the executable file is started.
  • the code segment and the data segment of the host in the executable file are loaded into the memory space of the host by the standard loader of the operating system; and the code segment and the data segment of the neural network dedicated processor in the executable file are used by the neural network
  • the dedicated processor loader loads into the memory space of the neural network dedicated processor.
  • the host operating system then cooperates with the neural network dedicated processor driver to serially execute the instructions in the host code segment and the instructions in the neural network specific processor code segment in the order set by the neural network application.
  • the program running support system of the heterogeneous computing platform includes not only a host, a host operating system, a standard loader of an operating system, but also a neural network dedicated processing in a single computing platform. , neural network application, neural network dedicated processor development library, compilation system, neural network dedicated processor loader, neural network dedicated processor driver.
  • a neural network dedicated processor development library is used to encapsulate various high-level language (for example, C/C++) application programming interfaces (APIs) of a neural network dedicated processor to provide a user to develop a neural network.
  • the application is called to implement the initialization operation and runtime control of the neural network dedicated processor core.
  • the compilation system in the program execution support system is a compilation system of the heterogeneous computing platform according to the first embodiment of the present invention for generating a hybrid link executable file.
  • a standard loader of the operating system is used to load code segments and data segments of the host in the executable file into the memory space of the host; and a neural network dedicated processor loader is used to process the neural network in the executable file.
  • the code segment and data segment of the device are loaded into the memory space of the neural network dedicated processor.
  • the neural network dedicated processor driver is used in conjunction with the host operating system to cause the instructions in the host code segment and the instructions in the neural network specific processor code segments to be executed serially in the order set by the neural network application.
  • the compiling and program running support technology for the heterogeneous computing platform of the neural network proposed by the present invention covers the stages of development, compilation, deployment and operation of the neural network application, and is a complete and practical one.
  • Technical solution The compiling and program running support technology for the heterogeneous computing platform of the neural network proposed by the present invention covers the stages of development, compilation, deployment and operation of the neural network application, and is a complete and practical one.
  • the neural network optimization compiler in the present invention can allocate resources for computing units and on-chip memory in a neural network dedicated processor, balance data read and write load and calculate operation load, and fully utilize the neural network dedicated processor hardware. Resource parallelism, accurately mining the computing power of a neural network dedicated processor, and generating the most efficient instruction code.
  • the compiler tool chain can compile various neural network models to generate executable code on a neural network dedicated processor, and is not limited to a specific neural network algorithm, and is suitable for neural network special processing.
  • the development of various neural network applications on the platform has great flexibility.
  • Ease of use Provides a standard C/C++ API programming interface to the user through a neural network dedicated processor development library, which eliminates users from CUDA and OpenCL by extending the C/C++ syntax to provide heterogeneous programming. Familiar with the additional burden of the new grammar, thus reducing the development threshold of the neural network dedicated processor platform and improving programming efficiency.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Mathematical Physics (AREA)
  • Stored Programmes (AREA)

Abstract

本公开提供一种面向神经网络异构计算平台的编程模型。具体地说，本公开提出一种异构计算平台的编译方法和系统及其程序运行支持方法和系统。训练后的神经网络模型输入到神经网络(NN)优化编译器，生成与神经网络对应的NN汇编文件。NN汇编文件输入到NN汇编器，生成与神经网络对应的NN二进制文件。使用主机编译器工具链编译和汇编用户用高级语言开发的神经网络应用程序，依次生成相应的主机汇编文件和主机二进制文件。使用主机链接器链接NN二进制文件和主机二进制文件，以生成单一的混合链接可执行文件。本发明的技术方案具有计算性能好、可扩展性强、兼容性强、灵活度高等特点。

Description

面向神经网络异构计算平台的编程模型 技术领域
本发明涉及异构计算平台与神经网络，更具体涉及面向神经网络异构计算平台的编程模型。
背景技术
人工智能近年得到迅速发展，极大地影响了人们的生活，世界各国都给予了高度的重视，并进行大规模研发投入。人工神经网络是人工智能应用的核心。深度学习神经网络算法是最常见的一种人工神经网络模型。它的工作负载特征是计算密集(乘加操作为G量级)和数据密集(参数为M至数百M字节)。基于传统通用处理器CPU的计算平台并不能很好地满足性能要求。近几年，以NVIDIA GPU为代表的加速神经网络计算的异构平台变得流行起来。CUDA SDK提供的编译工具链和开发包简化了用户在CPU+GPU异构环境下的应用开发。随着FPGA以及各种深度学习专用芯片ASIC(如谷歌TPU)等更具性价比的加速方案不断涌现，解决好CPU+FPGA/ASIC异构计算平台面临的以下问题变得迫切：
1.基于流行C/C++高级语言的可编程性
2.降低神经网络应用的开发门槛、提供编程效率
3.优化神经网络结构、编译生成高效的计算指令
4.提高数据复用、减少CPU与FPGA/ASIC之间的数据移动
因此，需要一种面向神经网络异构计算平台的编程模型，有效解决CPU+神经网络专用处理器的异构环境下神经网络应用的开发、编译、部署和运行阶段面临的各种困难。
发明内容
根据以上的讨论，本发明主要关注面向神经网络异构计算平台上的一系列高效编译和运行时支持技术。更具体地说，本发明的目的在于提供一种面 向神经网络异构计算平台的编程模型，从而创新性地有效解决CPU+神经网络专用处理器异构环境下神经网络应用的开发、编译、部署和运行阶段面临的各种困难。
为实现上述目的，根据本发明的第一方面，提供一种异构计算平台的编译方法，包括：将训练后的神经网络模型输入到神经网络(NN)优化编译器，生成与神经网络对应的NN汇编文件；将NN汇编文件输入到NN汇编器，生成与神经网络对应的NN二进制文件；使用主机编译器工具链编译和汇编用户用高级语言开发的神经网络应用程序，依次生成相应的主机汇编文件和主机二进制文件；使用主机链接器链接NN二进制文件和主机二进制文件，以生成单一的混合链接可执行文件。
根据本发明第一方面的异构计算平台的编译方法，优选地，所述的二进制文件是ELF二进制文件，所述的单一的混合链接可执行文件是单一的混合链接ELF可执行文件。
根据本发明第一方面的异构计算平台的编译方法，优选地，所述高级语言是C/C++。
根据本发明第一方面的异构计算平台的编译方法，优选地，所述的单一的混合链接可执行文件包括：ELF标准可执行文件部分和NN扩展部分。ELF标准可执行文件部分进一步包括：ELF文件头，段表，代码段，数据段。NN扩展部分进一步包括：NN头，代码段，数据段。
根据本发明的第二方面，提供一种异构计算平台的程序运行支持方法，包括：用户通过调用神经网络专用处理器开发库的高级语言应用编程接口(API)，实现神经网络专用处理器内核的初始化操作和运行时的控制；启动根据本发明第一方面的异构计算平台的编译方法所得到的混合链接可执行文件；所述可执行文件中的主机的代码段和数据段被操作系统的标准加载器加载到主机的内存空间；所述可执行文件中的神经网络专用处理器的代码段和数据段被神经网络专用处理器加载器加载到神经网络专用处理器的内存 空间；主机代码段中的指令和神经网络专用处理器代码段中的指令按照神经网络应用设定的顺序串行执行。
根据本发明的第三方面，提供一种异构计算平台的编译系统，包括：神经网络(NN)优化编译器，用于接收训练后的神经网络模型，生成与神经网络对应的NN汇编文件；NN汇编器，用于接收NN汇编文件，生成与神经网络对应的NN二进制文件；主机编译器，用于编译用户用高级语言开发的神经网络应用程序，生成相应的主机汇编文件；主机汇编器，用于汇编主机汇编文件，生成主机二进制文件；主机链接器，用于链接NN二进制文件和主机二进制文件，以生成单一的混合链接可执行文件。
根据本发明第三方面的异构计算平台的编译系统，优选地，所述的二进制文件是ELF二进制文件，所述的单一的混合链接可执行文件是单一的混合链接ELF可执行文件。
根据本发明第三方面的异构计算平台的编译系统，优选地，所述高级语言是C/C++。
根据本发明第三方面的异构计算平台的编译系统，优选地，所述的单一的混合链接可执行文件包括：ELF标准可执行文件部分和NN扩展部分。ELF标准可执行文件部分进一步包括：ELF文件头，段表，代码段，数据段。NN扩展部分进一步包括：NN头，代码段，数据段。
根据本发明的第四方面，提供一种异构计算平台的程序运行支持系统，包括：主机；神经网络专用处理器；神经网络应用程序；神经网络专用处理器开发库，用于封装神经网络专用处理器的各种高级语言应用编程接口，以便提供给用户开发神经网络应用时调用，实现神经网络专用处理器内核的初始化操作和运行时的控制；根据本发明第三方面的异构计算平台的编译系统，用于生成混合链接可执行文件；操作系统的标准加载器，用于将所述可执行文件中的主机的代码段和数据段加载到主机的内存空间；神经网络专用处理器加载器，用于将所述可执行文件中的神经网络专用处理器的代码段和数据 段加载到神经网络专用处理器的内存空间；主机操作系统；神经网络专用处理器驱动程序，用于与主机操作系统相配合，使主机代码段中的指令和神经网络专用处理器代码段中的指令按照神经网络应用设定的顺序串行执行。
本发明的技术方案具有计算性能好、可扩展性强、兼容性强、灵活度高等特点。
附图说明
下面参考附图结合实施例说明本发明。在附图中：
图1是CPU+神经网络专用处理器异构计算平台混合编译模型；
图2是根据本发明的第一实施例的异构计算平台的编译方法的流程图；
图3是CPU+神经网络专用处理器混合链接ELF可执行文件的示意图；
图4是根据本发明的第二实施例的异构计算平台的程序运行支持模型。
具体实施方式
附图仅用于示例说明，不能理解为对本专利的限制；下面结合附图和实施例对本发明的技术方案做进一步的说明。
与单一计算平台(即只有主机或CPU的计算平台)相比，本发明还包括了以下的功能部件：1)神经网络优化编译器；2)NN汇编器；3)CPU+神经网络专用处理器异构混合(hybrid)链接ELF(Executable and Linking Format)可执行文件；4)神经网络专用处理器驱动程序；5)运行时神经网络专用处理器加载器(Loader)；6)神经网络专用处理器开发库(Library)。
本领域技术人员应当理解，本申请中所使用的术语“神经网络专用处理器”，也可简称为“神经网络处理器”或“NN处理器”。由于深度学习是神经网络技术中目前最为流行的一个技术分类，因此神经网络专用处理器可以被实现为深度学习专用处理器或深度学习处理器。但是，本领域技术人员应该明白，神经网络还有其他的技术分支，例如深度神经网络(DNN，Deep Neutral Network)，因此神经网络专用处理器也可以被实现为深度神经网络专用处理器或深度神经网络处理器(DNN处理器)。也就是说，有关“深度学 习处理器”或“深度神经网络处理器”在异构计算平台中的编译和运行支持方面的技术也在本发明的范围之内。
如下面参考图1所述的内容中所提到的，上述的功能部件1)、2)、3)提供神经网络在CPU+神经网络专用处理器异构平台上的编译支持，即涉及本发明的第一实施例。如下面参考图4所述的内容中所提到的，功能部件4)、5)、6)为CPU+神经网络专用处理器异构神经网络应用提供相关程序运行支持，即涉及本发明的第二实施例。
第一实施例：
下面，将首先结合图1来描述第一实施例中的组成部件的功能。
图1是CPU+神经网络专用处理器异构计算平台混合编译模型。
在图1中，神经网络(Neural Network，可简称为NN)优化编译器(或称为“NN优化编译器”)以神经网络模型为输入，通过分析网络的拓扑结构获取模型中控制流和数据流信息，以此为基础在模型上施加各种优化变换技术。具体地说，该编译器合并神经网络模型中不同网络层之间的计算操作、降低计算强度。对于结构化和非结构化稀疏网络，该编译器将消除稀疏值带来的不必要计算和数据移动。此外，该编译器将充分复用神经网络专用处理器片上内存中存放的网络参数和特征图(feature map)数值，降低访存带宽需求，从而减少功耗。该编译器输出NN汇编文件到NN汇编器。
NN汇编器接收来自NN优化编译器的NN汇编文件。接收到的NN汇编文件(汇编指令)被NN汇编器转换而生成对应的二进制机器编码。此后，神经网络专用处理器的机器编码和数据被集成在一起生成标准格式的ELF二进制文件。
主机链接器用于生成CPU+神经网络专用处理器异构混合链接ELF可执行文件。在本发明的优选实施例中，可以扩展ELF可执行文件的格式，增 加神经网络专用处理器的支持。具体地说，把神经网络专用处理器和主机两种ELF二进制文件链接在一起，生成单一可执行文件(ELF可执行文件)，方便神经网络应用程序的部署和运行。图1中，神经网络专用处理器和主机ELF二进制文件都被示出为ELF obiect文件。图3给出了CPU+神经网络专用处理器混合链接ELF可执行文件的一个示例，下文将进一步详细描述。
主机链接器所使用的主机ELF二进制文件是通过主机编译器对C/C++神经网络程序进行编译，通过主机汇编器对生成的主机汇编文件进行汇编而得到的。其中，主机编译器、主机汇编器、主机链接器都是单一计算平台的编译模型中已有的功能部件。
下面从方法步骤的角度来进一步描述根据本发明第一实施例的编译模型的工作状态。
图2是根据本发明的第一实施例的异构计算平台的编译方法的流程图。
如图2中所示，根据本发明的第一实施例的异构计算平台的编译方法200开始于步骤S201。在该步骤S201，将训练后的神经网络模型输入到NN优化编译器，生成与神经网络对应的NN汇编文件。
在步骤S203，将NN汇编文件输入到NN汇编器，生成与神经网络对应的NN二进制文件。
在步骤S205，使用主机编译器工具链编译和汇编用户用高级语言开发的神经网络应用程序，依次生成相应的主机汇编文件和主机二进制文件。
在本发明的优选实施例中，以及参考图1的描述，将以上提到的高级语言都具体化为C/C++。本领域技术人员应该理解，C/C++是本领域中典型且常用的高级语言，从而使本发明的可编程性得到提升。但是，本领域技术人员也应该理解，这里所述的高级语言也可以是除了C/C++以外的其他高级语言，例如，包括但不限于：Java、Pascal、Python、Perl等等面向对象的高级 语言或可能在本发明之前的编程平台上典型或常用，也可能在本发明的发明人所能预见到的未来的编程平台上典型或常用的其他高级语言。
在步骤S207，使用主机链接器链接NN二进制文件和主机二进制文件，以生成单一的混合链接可执行文件。
在本发明的优选实施例中，以及参考图1的描述，将以上提到的二进制文件都具体化为ELF二进制文件(或称为ELF文件)。本领域技术人员应该理解，ELF(Executable and Linking Format)文件使用典型且常用的标准文件格式，从而使本发明的可编程性得到提升。但是，本领域技术人员也应该理解，这里所述的二进制文件也可以是除了ELF文件以外的其他文件格式，其可能在本发明之前的编程平台上典型或常用，也可能在本发明的发明人所能预见到的未来的编程平台上典型或常用。
此外，在本发明的优选实施方式中，在步骤S207生成的单一的混合链接可执行文件可以是单一的混合链接ELF可执行文件。图3给出了这样的一种示例。
图3是CPU+神经网络专用处理器混合链接ELF可执行文件的示意图。如图3所示，在步骤S207所生成的单一的混合链接可执行文件可以包括两个部分：ELF标准可执行文件部分和NN扩展部分。其中，ELF标准可执行文件部分与普通可执行文件一样，其进一步包括：ELF文件头(图3示为“ELF Header”)，段表(图3示为“Section Header Table”)，代码段(图3示为“.text”)，数据段(图3示为“.data”)。NN扩展部分是对ELF标准可执行文件部分的扩展，其与ELF标准可执行文件部分链接在一起，进一步包括：NN头(图3示为“NN Header”)，代码段(图3示为“.text”)，数据段(图3示为“.data”)。
尽管在图3中，NN扩展部分链接在ELF标准可执行文件之后，但本领域技术人员应该理解，CPU+神经网络专用处理器混合链接可执行文件的链接方式和组成结构也可以是除了图3以外的形式。也就是说，根据本发明的 第一实施例所述的单一的混合链接可执行文件不限于图3中所给出的示例的形式，还可以是任意链接形式和内容构成的CPU+神经网络专用处理器混合链接可执行文件。
回到图2，步骤S207执行完之后，编译方法200可视为结束。生成的可执行文件可以被启动，从而在整个面向神经网络异构计算平台的编程模型的支持下，得到运行。
在本发明的第一实施例中，除了上述的异构计算平台的编译方法，还提出了一种异构计算平台的编译系统。
参看图1和图2，根据本发明的第一实施例的异构计算平台的编译系统可以包括NN优化编译器，其用于接收训练后的神经网络模型，生成与神经网络对应的NN汇编文件。
编译系统中的NN汇编器用于从NN优化编译器接收NN汇编文件，从而生成与神经网络对应的NN二进制文件。
另一方面，针对主机方面，编译系统的主机编译器用于编译用户用高级语言开发的神经网络应用程序，生成相应的主机汇编文件。而主机汇编器则用于汇编主机汇编文件，生成主机二进制文件。所述高级语言可以是C/C++，也可以是用于提升系统可编程性的其他高级语言。
编译系统仍然使用主机链接器来链接NN二进制文件和主机二进制文件，但是生成的是单一的混合链接可执行文件。
如上在对图2的描述中所解释的，所述的二进制文件可以是ELF二进制文件，所述的单一的混合链接可执行文件可以是单一的混合链接ELF可执行文件。
如前所述，所述的单一的混合链接可执行文件可以包括：ELF标准可执 行文件部分和NN扩展部分。其中，ELF标准可执行文件部分可以进一步包括：ELF文件头，段表，代码段，数据段。NN扩展部分则可以进一步包括：NN头，代码段，数据段。可以再次参看图3所给出的示例。但是，根据本发明的第一实施例所述的单一的混合链接可执行文件不限于图3中所给出的示例的形式，还可以是任意链接形式和内容构成的CPU+神经网络专用处理器混合链接可执行文件。
第二实施例：
下面将结合图4来描述本发明的编程环境，即程序运行支持模型。
图4是根据本发明的第二实施例的异构计算平台的程序运行支持模型。
在图4中，将神经网络专用处理器简称为DPU，以与主机CPU相区别。本领域技术人员应该理解，这样的命名并不影响神经网络专用处理器的一般性。也就是说，本说明书与附图中，“神经网络专用处理器”与“DPU”是可以互换使用的术语，用于表示异构计算平台上与CPU相异的另一处理器。
如图4中所示，神经网络专用处理器驱动程序(图4中示出为“DPU驱动程序”)工作在操作系统内核态(kernel space)，负责神经网络专用处理器资源运行时的动态管理和调度，并且负责运行在神经网络专用处理器上神经网络代码(以下称为神经网络专用处理器内核)的管理和控制，负责神经网络专用处理器内存空间的管理和分配。
神经网络专用处理器加载器(图4中示为“DPU loader”)，在运行时，工作在操作系统用户态(user space)，负责神经网络专用处理器内核的动态加载和释放，并且负责神经网络专用处理器内核代码的重定位。
神经网络专用处理器开发库(图4中示为“DPU Library”)工作在操作系统用户态(user space)，用于封装神经网络专用处理器的各种C/C++应用 编程接口(API)提供给用户开发各种神经网络应用时调用，从而提高神经网络专用处理器上的编程效率。
此外，再加上本领域技术人员应该知道的主机(或CPU)、神经网络专用处理器(图4中示为“DPU”)、主机操作系统和操作系统的标准加载器(OS Loader)，用于配合以上提到的服务于神经网络专用处理器的部件，一起提供执行神经网络应用程序的运行支持环境。换句话说，实现神经网络应用程序的神经网络专用处理器加速。
下面分别从方法步骤的角度和系统构成的角度来进一步描述根据本发明第二实施例的异构计算平台的程序运行支持模型的工作状态。
CPU+神经网络专用处理器异构计算平台开发的神经网络应用程序，主要有两部分组成：C/C++代码、训练后的神经网络模型(网络拓扑结构和神经网络参数)。从开发、编译到部署和运行，典型的工作流程如下所示：
步骤1.用户通过调用神经网络专用处理器开发库的C/C++API编程接口，实现神经网络专用处理器内核的初始化操作和运行时的各种控制，例如dpuInit()，dpuFini()，dpuCreateKernel()，dpuDestroyKernel()，dpuRunKernel()等。其中的“dpu”表示神经网络专用处理器。
步骤2.输入训练后的神经网络模型给NN优化编译器，生成与网络对应的NN汇编文件。
步骤3.输入NN汇编文件给NN汇编器，生成与网络对应的ELF二进制文件。
步骤4.使用主机编译器工具链编译用户开发的C/C++神经网络应用程序，生成相应的主机汇编文件和主机ELF二进制文件。
步骤5.使用主机链接器链接神经网络专用处理器和主机的所有ELF二 进制文件，生成单一的混合链接ELF可执行文件。
步骤6.编译完生成的单一混合链接ELF可执行文件包含了主机和神经网络专用处理器运行所需的全部信息，执行方式与普通的主机ELF可执行文件完全一样，可实现一键式部署。
步骤7.启动混合链接ELF可执行文件，ELF中主机的代码段和数据段会被操作系统的标准加载器加载到主机的内存空间；ELF中神经网络专用处理器的代码段和数据段由神经网络专用处理器加载器负责加载到神经网络专用处理器的内存空间(其中神经网络专用处理器的内存空间由神经网络专用处理器驱动程序管理)。
步骤8.主机代码段中的指令和神经网络专用处理器代码段中的指令按照神经网络应用设定的顺序串行执行，计算和数据密集的神经网络算法以神经网络专用处理器内核的形态被主机卸载(offload)至神经网络专用处理器上加速运行，更加高效地运行。
可以将以上的过程总结一下。
在根据本发明的第二实施例的异构计算平台的程序运行支持方法中，用户通过调用神经网络专用处理器开发库的高级语言应用编程接口(API)，实现神经网络专用处理器内核的初始化操作和运行时的控制。
用户根据本发明第一实施例的异构计算平台的编译方法得到了混合链接可执行文件之后，启动该可执行文件。所述可执行文件中的主机的代码段和数据段被操作系统的标准加载器加载到主机的内存空间；而所述可执行文件中的神经网络专用处理器的代码段和数据段被神经网络专用处理器加载器加载到神经网络专用处理器的内存空间。
然后，主机操作系统与神经网络专用处理器驱动程序相配合，将主机代码段中的指令和神经网络专用处理器代码段中的指令按照神经网络应用设 定的顺序串行执行。
在本发明的第二实施例中，除了上述的异构计算平台的程序运行支持方法，还提出了一种异构计算平台的程序运行支持系统。
如图4所示，根据本发明第二实施例的异构计算平台的程序运行支持系统不仅包括单一计算平台中常见的主机、主机操作系统、操作系统的标准加载器，还包括神经网络专用处理器、神经网络应用程序、神经网络专用处理器开发库、编译系统、神经网络专用处理器加载器、神经网络专用处理器驱动程序。
在本发明的程序运行支持系统中，神经网络专用处理器开发库用于封装神经网络专用处理器的各种高级语言(例如C/C++)应用编程接口(API)，以便提供给用户开发神经网络应用时调用，实现神经网络专用处理器内核的初始化操作和运行时的控制。
程序运行支持系统中的编译系统，就是根据本发明第一实施例的异构计算平台的编译系统，用于生成混合链接可执行文件。
操作系统的标准加载器用于将所述可执行文件中的主机的代码段和数据段加载到主机的内存空间；而神经网络专用处理器加载器用于将所述可执行文件中的神经网络专用处理器的代码段和数据段加载到神经网络专用处理器的内存空间。
神经网络专用处理器驱动程序用于与主机操作系统相配合，使主机代码段中的指令和神经网络专用处理器代码段中的指令按照神经网络应用设定的顺序串行执行。
以下对本发明所提到的创新点进行进一步讨论。
目前面向神经网络领域专用处理器的设计和研发正处于百花齐放的初 期阶段，如何构建面向神经网络异构计算平台的专用编译工具链在学术界和工业界还没有形成标准和规范，也没有相关的完整技术方案公布，因此本发明具有一定的先进性和创新性，总结起来具有以下优点：
1.完备性好：本发明提出的面向神经网络异构计算平台的编译和程序运行支持技术，覆盖了神经网络应用的开发、编译、部署和运行各阶段，是一套完整的、实用性强的技术解决方案。
2.性能好：本发明中的神经网络优化编译器，能够针对神经网络专用处理器中计算单元和片上内存的资源配置，平衡数据读写负载和计算操作负载，充分发挥神经网络专用处理器硬件资源并行度，精确挖掘神经网络专用处理器的计算能力、生成最高效的指令代码。
3.灵活性高：本发明中编译工具链能把各种神经网络模型编译生成神经网络专用处理器上可运行指令码，不受限于某种特定的神经网络算法，适用于神经网络专用处理器平台各种神经网络应用程序的开发，具有较好的灵活性。
4.易用性强：通过神经网络专用处理器开发库向用户提供标准C/C++API编程接口，与CUDA、OpenCL通过扩展C/C++语法提供异构编程的方式相比，免除了用户熟悉新语法的额外负担，因此可降低神经网络专用处理器平台的开发门槛、提高编程效率。
上面已经描述了本发明的各种实施例和实施情形。但是，本发明的精神和范围不限于此。本领域技术人员将能够根据本发明的教导而做出更多的应用，而这些应用都在本发明的范围之内。
也就是说，本发明的上述实施例仅仅是为清楚说明本发明所做的举例，而非对本发明实施方式的限定。对于所属领域的普通技术人员来说，在上述说明的基础上还可以做出其他不同形式的变化或变动。这里无需也无法对所有的实施方式予以穷举。凡在本发明的精神和原则之内所作的任何修改、替 换或改进等，均应包含在本发明权利要求的保护范围之内。

Claims (10)

  1. 一种异构计算平台的编译方法，包括：
    将训练后的神经网络模型输入到神经网络(NN)优化编译器，生成与神经网络对应的NN汇编文件；
    将NN汇编文件输入到NN汇编器，生成与神经网络对应的NN二进制文件；
    使用主机编译器工具链编译和汇编用户用高级语言开发的神经网络应用程序，依次生成相应的主机汇编文件和主机二进制文件；
    使用主机链接器链接NN二进制文件和主机二进制文件，以生成单一的混合链接可执行文件。
  2. 根据权利要求1所述的异构计算平台的编译方法，其中，所述的二进制文件是ELF二进制文件，所述的单一的混合链接可执行文件是单一的混合链接ELF可执行文件。
  3. 根据权利要求1所述的异构计算平台的编译方法，其中，所述高级语言是C/C++。
  4. 根据权利要求1所述的异构计算平台的编译方法，其中，所述的单一的混合链接可执行文件包括：
    ELF标准可执行文件部分，其进一步包括：
    ELF文件头，
    段表，
    代码段，
    数据段，
    NN扩展部分，其进一步包括：
    NN头，
    代码段，
    数据段。
  5. 一种异构计算平台的程序运行支持方法，包括：
    用户通过调用神经网络专用处理器开发库的高级语言应用编程接口(API)，实现神经网络专用处理器内核的初始化操作和运行时的控制；
    启动根据权利要求1所述的异构计算平台的编译方法所得到的混合链接可执行文件；
    所述可执行文件中的主机的代码段和数据段被操作系统的标准加载器加载到主机的内存空间；
    所述可执行文件中的神经网络专用处理器的代码段和数据段被神经网络专用处理器加载器加载到神经网络专用处理器的内存空间；
    主机代码段中的指令和神经网络专用处理器代码段中的指令按照神经网络应用设定的顺序串行执行。
  6. 一种异构计算平台的编译系统，包括：
    神经网络(NN)优化编译器，用于接收训练后的神经网络模型，生成与神经网络对应的NN汇编文件；
    NN汇编器，用于接收NN汇编文件，生成与神经网络对应的NN二进制文件；
    主机编译器，用于编译用户用高级语言开发的神经网络应用程序，生成相应的主机汇编文件；
    主机汇编器，用于汇编主机汇编文件，生成主机二进制文件；
    主机链接器，用于链接NN二进制文件和主机二进制文件，以生成单一的混合链接可执行文件。
  7. 根据权利要求6所述的异构计算平台的编译系统，其中，所述的二进制文件是ELF二进制文件，所述的单一的混合链接可执行文件是单一的混合链接ELF可执行文件。
  8. 根据权利要求6所述的异构计算平台的编译系统，其中，所述高级语言是C/C++。
  9. 根据权利要求6所述的异构计算平台的编译系统，其中，所述的单 一的混合链接可执行文件包括：
    ELF标准可执行文件部分，其进一步包括：
    ELF文件头，
    段表，
    代码段，
    数据段，
    NN扩展部分，其进一步包括：
    NN头，
    代码段，
    数据段。
  10. 一种异构计算平台的程序运行支持系统，包括：
    主机；
    神经网络专用处理器；
    神经网络应用程序；
    神经网络专用处理器开发库，用于封装神经网络专用处理器的各种高级语言应用编程接口，以便提供给用户开发神经网络应用时调用，实现神经网络专用处理器内核的初始化操作和运行时的控制；
    根据权利要求6所述的异构计算平台的编译系统，用于生成混合链接可执行文件；
    操作系统的标准加载器，用于将所述可执行文件中的主机的代码段和数据段加载到主机的内存空间；
    神经网络专用处理器加载器，用于将所述可执行文件中的神经网络专用处理器的代码段和数据段加载到神经网络专用处理器的内存空间；
    主机操作系统；
    神经网络专用处理器驱动程序，用于与主机操作系统相配合，使主机代码段中的指令和神经网络专用处理器代码段中的指令按照神经网络应用设定的顺序串行执行。
PCT/CN2018/079954 2017-04-11 2018-03-22 面向神经网络异构计算平台的编程模型 Ceased WO2018188463A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710231894.9A CN107239315B (zh) 2017-04-11 2017-04-11 面向神经网络异构计算平台的编程模型
CN201710231894.9 2017-04-11

Publications (1)

Publication Number Publication Date
WO2018188463A1 true WO2018188463A1 (zh) 2018-10-18

Family

ID=59984035

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/079954 Ceased WO2018188463A1 (zh) 2017-04-11 2018-03-22 面向神经网络异构计算平台的编程模型

Country Status (3)

Country Link
US (1) US10732943B2 (zh)
CN (1) CN107239315B (zh)
WO (1) WO2018188463A1 (zh)

Families Citing this family (60)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11868896B2 (en) 2016-01-27 2024-01-09 Microsoft Technology Licensing, Llc Interface for working with simulations on premises
US11836650B2 (en) 2016-01-27 2023-12-05 Microsoft Technology Licensing, Llc Artificial intelligence engine for mixing and enhancing features from one or more trained pre-existing machine-learning models
US10733531B2 (en) * 2016-01-27 2020-08-04 Bonsai AI, Inc. Artificial intelligence engine having an architect module
US11775850B2 (en) 2016-01-27 2023-10-03 Microsoft Technology Licensing, Llc Artificial intelligence engine having various algorithms to build different concepts contained within a same AI model
US11841789B2 (en) 2016-01-27 2023-12-12 Microsoft Technology Licensing, Llc Visual aids for debugging
CN107239315B (zh) * 2017-04-11 2019-11-15 赛灵思公司 面向神经网络异构计算平台的编程模型
CN109496294A (zh) * 2018-01-15 2019-03-19 深圳鲲云信息科技有限公司 人工智能处理装置的编译方法及系统、存储介质及终端
US10892953B2 (en) * 2018-02-23 2021-01-12 Nec Corporation Network-based application acceleration
CN110308899B (zh) * 2018-03-27 2023-12-29 上海寒武纪信息科技有限公司 针对神经网络处理器的语言源程序生成方法和装置
CN110308909B (zh) * 2018-03-27 2023-08-01 上海寒武纪信息科技有限公司 针对神经网络处理器的可执行程序生成装置和方法
CN109643229B (zh) * 2018-04-17 2022-10-04 深圳鲲云信息科技有限公司 网络模型的应用开发方法、平台及计算机可读存储介质
CN109840117A (zh) * 2018-06-20 2019-06-04 中兴通讯股份有限公司 训练模型的实现方法、设备以及存储介质
CN111316227B (zh) * 2018-08-20 2021-08-13 华为技术有限公司 一种调试应用程序的方法及设备
CN109447256A (zh) * 2018-09-12 2019-03-08 上海交通大学 基于FPGA的Tensorflow系统加速的设计方法
CN111104120B (zh) * 2018-10-29 2023-12-22 赛灵思公司 神经网络编译方法、系统及相应异构计算平台
JP6950665B2 (ja) * 2018-11-02 2021-10-13 横河電機株式会社 エンジニアリング装置、エンジニアリング装置の制御方法及びプログラム
WO2020093304A1 (zh) * 2018-11-08 2020-05-14 北京比特大陆科技有限公司 神经网络编译方法、装置、设备、存储介质及程序产品
CN109284815B (zh) * 2018-11-30 2020-11-24 安徽寒武纪信息科技有限公司 神经网络模型算法编译方法、装置及相关产品
CN109558564B (zh) * 2018-11-30 2022-03-11 上海寒武纪信息科技有限公司 运算方法、装置及相关产品
CN109543825B (zh) * 2018-11-30 2020-12-01 安徽寒武纪信息科技有限公司 神经网络模型算法编译方法、装置及相关产品
US11423142B2 (en) * 2018-12-06 2022-08-23 Nec Corporation Confidential machine learning with program compartmentalization
CN109656566B (zh) * 2018-12-14 2020-01-10 中科寒武纪科技股份有限公司 异构计算系统可执行文件获取方法、运行方法及相关产品
CN109711540B (zh) * 2018-12-20 2021-09-21 中科寒武纪科技股份有限公司 一种计算装置及板卡
CN109766996A (zh) * 2018-12-29 2019-05-17 北京中科寒武纪科技有限公司 卷积神经网络的优化方法、装置、存储介质和系统
CN109711367B (zh) * 2018-12-29 2020-03-06 中科寒武纪科技股份有限公司 运算方法、装置及相关产品
CN109754084B (zh) * 2018-12-29 2020-06-12 中科寒武纪科技股份有限公司 网络结构的处理方法、装置及相关产品
CN109754073B (zh) * 2018-12-29 2020-03-10 中科寒武纪科技股份有限公司 数据处理方法、装置、电子设备和可读存储介质
US11157692B2 (en) * 2019-03-29 2021-10-26 Western Digital Technologies, Inc. Neural networks using data processing units
CN110018831B (zh) * 2019-04-04 2022-11-08 中科寒武纪科技股份有限公司 程序处理方法、装置及计算机可读存储介质
US11868901B1 (en) 2019-05-21 2024-01-09 Percieve Corporation Compiler for optimizing memory allocations within cores
US11175898B2 (en) * 2019-05-31 2021-11-16 Apple Inc. Compiling code for a machine learning model for execution on a specialized processor
CN112148291A (zh) * 2019-06-26 2020-12-29 中兴通讯股份有限公司 指令块的处理方法及装置、存储介质、电子装置
CN112183712B (zh) * 2019-07-03 2025-07-22 安徽寒武纪信息科技有限公司 深度学习算法的编译方法、装置及相关产品
DE102020118563A1 (de) * 2019-07-17 2021-01-21 Steering Solutions Ip Holding Corporation Middleware-system und -verfahren
CN110515626B (zh) * 2019-08-20 2023-04-18 Oppo广东移动通信有限公司 深度学习计算框架的代码编译方法及相关产品
CN110569106B (zh) * 2019-08-27 2022-11-15 Oppo广东移动通信有限公司 代码加载方法、装置、电子设备及计算机可读介质
CN110704040A (zh) * 2019-09-30 2020-01-17 上海寒武纪信息科技有限公司 信息处理方法、装置、计算机设备及可读存储介质
CN112711409B (zh) * 2019-10-25 2024-06-04 杭州海康威视数字技术股份有限公司 应用程序开发、运行方法、系统及智能分析设备
CN110908667B (zh) * 2019-11-18 2021-11-16 北京迈格威科技有限公司 神经网络联合编译的方法、装置和电子设备
US11720351B2 (en) * 2020-03-17 2023-08-08 Onspecta, Inc. Microkernel-based software optimization of neural networks
CN111831285B (zh) * 2020-06-23 2023-03-14 西安电子科技大学 一种面向内存计算平台的代码转换方法、系统及应用
CN111915016B (zh) * 2020-07-10 2022-03-25 深圳云天励飞技术股份有限公司 一种基于tvm编译器的异构平台的部署方法及装置
CN112529175B (zh) * 2020-11-05 2022-03-18 上海交通大学 神经网络的编译方法、系统、计算机存储介质及编译设备
US20220147813A1 (en) * 2020-11-06 2022-05-12 Micron Technology, Inc. Runtime optimization of computations of an artificial neural network compiled for execution on a deep learning accelerator
CN112434755B (zh) * 2020-12-15 2023-04-07 电子科技大学 一种基于异构系统的数据异常感知方法
CN114647612A (zh) * 2020-12-21 2022-06-21 平头哥(上海)半导体技术有限公司 处理单元、片上系统及处理方法
CN114764609B (zh) * 2021-01-12 2025-10-03 富泰华工业（深圳）有限公司 多神经网络模型载入方法、装置、电子设备及存储介质
US12443832B1 (en) 2021-04-30 2025-10-14 Xilinx, Inc. Neural network architecture with high bandwidth memory (HBM)
CN113031966B (zh) * 2021-05-20 2021-09-21 之江实验室 一种智能选择编译加速库的深度学习编译优化方法
CN113778458B (zh) * 2021-08-19 2024-04-05 北京爱芯科技有限公司 数据处理器功能开发系统、方法及计算设备
CN114201154B (zh) * 2021-12-10 2023-01-31 北京百度网讯科技有限公司 算子生成方法和装置
CN114942763A (zh) * 2021-12-28 2022-08-26 太初(无锡)电子科技有限公司 异构编译系统
CN114968186A (zh) * 2022-01-12 2022-08-30 厦门壹普智慧科技有限公司 一种面向通用神经网络张量处理器的统一编程方法
CN114282641B (zh) * 2022-03-07 2022-07-05 麒麟软件有限公司 一种通用异构加速框架的构建方法
CN114385233B (zh) * 2022-03-24 2022-08-02 山东省计算中心（国家超级计算济南中心） 一种跨平台自适应数据处理工作流系统及方法
US12417082B1 (en) * 2022-03-29 2025-09-16 Amazon Technologies, Inc. Frontier node-based data layout analysis framework
US12442074B2 (en) 2022-08-25 2025-10-14 Applied Materials, Inc. System and method for controlling foreline pressure
CN115981666B (zh) * 2023-03-21 2023-07-21 北京探境科技有限公司 神经网络信息整合方法、装置、系统及存储介质
CN116306856B (zh) * 2023-05-17 2023-09-05 之江实验室 一种基于搜索的深度学习模型部署方法及装置
CN116661779B (zh) * 2023-07-26 2023-09-19 北京麟卓信息科技有限公司 一种基于符号动态重定向的多编译器混合链接方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101441569A (zh) * 2008-11-24 2009-05-27 中国人民解放军信息工程大学 基于异构可重构体系结构面向任务流的新型编译方法
CN105426226A (zh) * 2015-11-24 2016-03-23 无锡江南计算技术研究所 一种异构代码融合的编译和生成方法
CN106155755A (zh) * 2015-06-03 2016-11-23 上海红神信息技术有限公司 程序编译方法和程序编译器
CN107239315A (zh) * 2017-04-11 2017-10-10 北京深鉴智能科技有限公司 面向神经网络异构计算平台的编程模型

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB0029115D0 (en) 2000-11-29 2001-01-10 Sgs Thomson Microelectronics Assembling an object code module
US6826550B2 (en) 2000-12-15 2004-11-30 International Business Machines Corporation Method, system, and program for converting application program code to executable code using neural networks based on characteristics of the inputs
US6964029B2 (en) * 2002-10-31 2005-11-08 Src Computers, Inc. System and method for partitioning control-dataflow graph representations
US8327316B2 (en) 2008-09-30 2012-12-04 Ics Triplex Isagraf Inc. Compilation model
KR101534037B1 (ko) 2010-09-24 2015-07-08 인텔 코포레이션 컴퓨팅 플랫폼의 이종 프로세서들 간의 공유 가상 메모리에서의 가상 함수들의 공유
KR20120031756A (ko) * 2010-09-27 2012-04-04 삼성전자주식회사 Ｃｐｕ와 ｇｐｕ를 사용하는 이종 시스템에서 가상화를 이용한 어플리케이션 컴파일 및 실행 방법 및 장치
US9582397B2 (en) * 2012-08-22 2017-02-28 Nxp Usa, Inc. Method and system for obtaining run-time information associated with executing an executable
US9448818B2 (en) * 2014-02-14 2016-09-20 Red Hat, Inc. Defining classes as singleton classes or non-singleton classes
KR101757407B1 (ko) * 2015-06-11 2017-07-12 라인 가부시키가이샤 바이너리 병합 장치, 방법 및 컴퓨터 프로그램
WO2017083399A2 (en) * 2015-11-09 2017-05-18 Google Inc. Training neural networks represented as computational graphs
WO2018058426A1 (zh) * 2016-09-29 2018-04-05 清华大学 硬件神经网络转换方法、计算装置、编译方法和神经网络软硬件协作系统
US10552207B2 (en) * 2016-12-21 2020-02-04 Intel Corporation Systems and methods for multi-architecture computing including program stack translation
US9990187B1 (en) 2017-01-27 2018-06-05 Sas Institute Inc. Analytic execution for automatic decision making

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101441569A (zh) * 2008-11-24 2009-05-27 中国人民解放军信息工程大学 基于异构可重构体系结构面向任务流的新型编译方法
CN106155755A (zh) * 2015-06-03 2016-11-23 上海红神信息技术有限公司 程序编译方法和程序编译器
CN105426226A (zh) * 2015-11-24 2016-03-23 无锡江南计算技术研究所 一种异构代码融合的编译和生成方法
CN107239315A (zh) * 2017-04-11 2017-10-10 北京深鉴智能科技有限公司 面向神经网络异构计算平台的编程模型

Also Published As

Publication number Publication date
CN107239315B (zh) 2019-11-15
US20180293057A1 (en) 2018-10-11
US10732943B2 (en) 2020-08-04
CN107239315A (zh) 2017-10-10

Similar Documents

Publication Publication Date Title
CN107239315B (zh) 面向神经网络异构计算平台的编程模型
CN113031966B (zh) 一种智能选择编译加速库的深度学习编译优化方法
KR101137126B1 (ko) 확장성 컴파일러 및 툴의 기반 구조에 대한 기술 언어
EP3702908B1 (en) Hardware acceleration method and the corresponding program
US8336035B2 (en) Customizable dynamic language expression interpreter
CN109933327B (zh) 基于代码融合编译框架的OpenCL编译器设计方法和系统
CN107341010B (zh) 一种C/C++与COStream混合编程方法和可视化编译系统
JP2013533533A (ja) コンピューティング・プラットフォーム内のワークロードの分配及び並列化
JP6598981B2 (ja) データ処理グラフのコンパイル
CN104932905A (zh) 一种aadl到c语言的代码自动生成方法
US20180004495A1 (en) Verification of a dataflow representation of a program through static type-checking
US10564947B2 (en) Computer system and method for multi-processor communication
CN104423994B (zh) 一种代码生成方法、编译器、调度方法、装置及调度系统
CN115600664B (zh) 算子处理方法、电子设备及存储介质
CN118409758B (zh) 用于编译核函数的方法、装置、介质和程序产品
KR20210120937A (ko) 딥 러닝 프레임워크 중의 모드 전환 방법, 장치, 전자 기기, 컴퓨터 저장 매체 및 컴퓨터 프로그램 제품
CN111190585A (zh) 基于字节码增强技术实现java模式匹配特性的方法
CN113031954A (zh) 代码编译方法、装置、电子设备、存储介质及异构系统
JP2006163686A (ja) コンパイル方法、コンパイルプログラム、コンパイル装置およびコンパイル用の記録媒体
TWI776338B (zh) 應用於圖形處理器的編譯器及非暫態電腦可讀式儲存媒體
Andrade et al. ParallelME: A parallel mobile engine to explore heterogeneity in mobile computing architectures
CN114169525A (zh) 推理框架的调用方法和调用装置
CN104598288B (zh) Java程序执行方法和装置
CN115048184A (zh) 一种解释器指令的优化执行方法和装置
CN117331541B (zh) 面向动态图框架和异构芯片的编译与运行方法和装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18784639

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18784639

Country of ref document: EP

Kind code of ref document: A1