WO2022095413A1 - 神经网络的编译方法、系统、计算机存储介质及编译设备 - Google Patents

神经网络的编译方法、系统、计算机存储介质及编译设备 Download PDF

Info

Publication number
WO2022095413A1
WO2022095413A1 PCT/CN2021/095209 CN2021095209W WO2022095413A1 WO 2022095413 A1 WO2022095413 A1 WO 2022095413A1 CN 2021095209 W CN2021095209 W CN 2021095209W WO 2022095413 A1 WO2022095413 A1 WO 2022095413A1
Authority
WO
WIPO (PCT)
Prior art keywords
operator
file
compiling
neural network
intermediate expression
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2021/095209
Other languages
English (en)
French (fr)
Inventor
刘子汉
冷静文
陆冠东
陈�全
李超
过敏意
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Jiao Tong University
Original Assignee
Shanghai Jiao Tong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Jiao Tong University filed Critical Shanghai Jiao Tong University
Priority to US18/846,282 priority Critical patent/US20250348296A1/en
Publication of WO2022095413A1 publication Critical patent/WO2022095413A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/40—Transformation of program code
    • G06F8/41—Compilation
    • G06F8/44—Encoding
    • G06F8/443—Optimisation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks

Definitions

  • the invention belongs to the technical field of neural networks, and relates to a compiling method, in particular to a compiling method, system, computer storage medium and compiling device of a neural network.
  • neural networks Recently promoted the development of machine learning and artificial intelligence and related industries, such as face recognition, speech recognition, online translation, autonomous driving and other technologies.
  • face recognition speech recognition
  • online translation online translation
  • autonomous driving autonomous driving
  • the large delay is the main obstacle affecting their large-scale industrial production. Therefore, how to reduce the operation delay and improve the calculation speed of the neural network is an important issue in the development of the neural network.
  • the existing tools have a high degree of encapsulation, and the interfaces open to users are limited, which causes inconvenience in debugging and parameter adjustment.
  • the optimization process and detailed algorithm are invisible to the user, and cannot support the user for further optimization.
  • the flexibility of the existing optimization algorithms is poor, the rule-based method leads to the loss of a large optimization space in the front-end optimization, and the back-end optimization for different hardware has poor transferability, which requires the intervention of more human experts.
  • the purpose of the present invention is to provide a method, system, computer storage medium and compiling device for compiling a neural network, which are used to solve the problem that the prior art has a high encapsulation degree and limited interfaces open to users.
  • the optimization process and detailed algorithm are invisible to the user, unable to support the user for further optimization, the optimization algorithm is less flexible, and the rule-based method leads to the loss of a large optimization space in the front-end optimization, and later
  • the problem that the port is optimized for different hardware has poor portability and requires the intervention of more human experts.
  • one aspect of the present invention provides a method for compiling a neural network, including: translating a network file into an intermediate expression file; performing optimization; generating a network template file based on a hardware interface from the optimized intermediate expression file; and compiling the network template file into an executable inference application.
  • the network file includes structure and parameters;
  • the intermediate expression file includes an abstraction layer, a description of the abstraction layer, and main fields;
  • the abstraction layer includes a model, an operator set, a fusion block, a basic layer and operator;
  • the description of the model includes the description of the complete model execution flow;
  • the description of the operator set includes the version of the specified operator set;
  • the description of the fusion block includes a block formed by fusion of multiple base layers;
  • the description of the base layer includes a base layer representing an operator in the network;
  • the description of the operator includes a detailed description of the operator;
  • the main domain of the model includes a set of fusion graphs, the middle of which represents a version;
  • the operator The main domain of the subset includes the version and the included operator list;
  • the main domain of the fusion block includes a set of layers, and the input and output of the layer;
  • the main domain of the base layer includes operators, input, output, parallelism ;
  • the main fields of the operator include operator
  • the step of optimizing the intermediate expression file from the perspective of performance analysis includes: using a performance-based method to characterize performance, generating a series of measurement performances with different parameters, and obtaining parameters that affect operator performance. Influence parameters, with which mathematical models are constructed to characterize performance.
  • the step of optimizing the intermediate expression file from the perspective of a single node includes: characterizing the model parallelism and operator fusion, selecting the optimal model parallelism for the operator, and characterizing the fusion block. Size, redundant computation, and performance rules.
  • the steps of optimizing the intermediate expression file from the perspective of multi-node coordination include: reading the next base layer; judging whether the next base layer can be fused with the current fusion block; , continue to judge whether the next base layer is the fully connected layer or convolution layer of the neural network; if so, count the calculation amount of the base layer, add the current total calculation amount, and add the base layer to the current fusion block , go to the next step; if not, directly add the base layer to the current fusion block, and go to the next step; if not, open a new fusion block; judge the total calculation in the current fusion block Whether the amount exceeds the calculation amount threshold, if yes, then go to the step of opening a new fusion block; if not, go to the step of reading the next base layer.
  • the step of generating the network template file from the optimized intermediate expression file further includes using the abstraction layer to hide redundant operations and expose optimized nodes at the same time.
  • the network template file is compiled into an executable reasoning application by a G++ compiler.
  • a neural network compiling system comprising: a translation module for translating a network file into an intermediate expression file; an optimization module for analyzing the intermediate expression from the perspective of performance analysis, single-node and multi-node coordination The expression file is optimized; the file generation module is used for generating the network template file based on the hardware interface from the optimized intermediate expression file; the compiling module is used for compiling the network template file into an executable reasoning application.
  • Another aspect of the present invention provides a computer storage medium on which a computer program is stored, and when the computer program is executed by a processor, the method for compiling the neural network is implemented.
  • a final aspect of the present invention provides a compiling apparatus, including: a processor and a memory; the memory is used for storing a computer program, and the processor is used for executing the computer program stored in the memory, so that the compiling apparatus executes the Compilation methods for neural networks.
  • the neural network compilation method, system, computer storage medium and compilation device of the present invention have the following beneficial effects:
  • the neural network compiling method, system, computer storage medium and compiling device of the present invention aim to design and implement a compiling toolchain framework, intermediate representation and corresponding optimization algorithm that can automatically adjust parameters and generate codes according to software and hardware information, so that the When computing on the target chip, higher computing speed and smaller computing delay can be obtained in a shorter optimization time without changing the network output result. And it is convenient for users to debug and adjust parameters by themselves.
  • FIG. 1 is a schematic flowchart of a method for compiling a neural network according to an embodiment of the present invention.
  • FIG. 2 shows a schematic diagram of an optimization flow of the present invention for optimizing the intermediate expression file from the perspective of multi-node coordination.
  • FIG. 3 is a schematic diagram showing the principle structure of the method for compiling a neural network according to an embodiment of the present invention.
  • the present invention provides a method for compiling a neural network, comprising:
  • the method for compiling a neural network provided by this embodiment will be described in detail below with reference to the drawings.
  • the neural network compiling method described in this embodiment provides end-to-end reasoning services for users, and generates a template file based on a target hardware interface from an existing, packaged network file, thereby generating an executable reasoning application.
  • the optimization process can optimize the execution efficiency of the generated code.
  • FIG. 1 shows a schematic flowchart of a method for compiling a neural network in one embodiment.
  • the method for compiling the neural network specifically includes the following steps:
  • the specific steps include using the API in the Python.ONNX library to read the neural network file in the onnx format into structured data, which includes the network structure (computation graph), operator details (nodes of the computational graph), etc. information, and use TVM to extract the weight information required by the operator contained in the onnx file, and store it as a text file for later use.
  • the network file including structure and parameters is translated into an intermediate representation file containing part of hardware information.
  • the intermediate expression file includes an abstraction layer, a description of the abstraction layer, and main fields;
  • the abstraction layer includes a model, an operator set, a fusion block, a base layer and an operator;
  • the description of the model includes a description of the complete model execution flow; the description of the operator set includes a specified version of the operator set; the description of the fusion block includes a block formed by fusion of multiple base layers; the description of the base layer includes represents the base layer of an operator in the network; the description of the operator includes a detailed description of the operator;
  • the main domain of the model consists of a set of fusion fasts, the intermediate representation of which is a version;
  • the main fields of the operator set include a version and a list of included operators
  • the main domain of the fusion block includes a set of layers, and the inputs and outputs of the layers;
  • the main domains of the base layer include operator, input, output, parallelism;
  • the main fields of the operator include operator type and operator attribute.
  • the steps of optimizing the intermediate expression file from the perspective of performance analysis include:
  • the performance is characterized by a method based on performance testing, a series of measurement performances with different parameters are generated, the influencing parameters affecting the performance of the operator are obtained, and a mathematical model is constructed by using the influencing parameters to characterize the performance.
  • the intermediate expression file is optimized from the perspective of performance analysis.
  • the influence parameters that influence the performance of the operator can be calculated by the PCA algorithm.
  • Cambricon MLU-100 the amount of operator calculation and the number of channels are the main parameters that affect the performance.
  • the steps of optimizing the intermediate expression file from the perspective of a single node include:
  • the optimization nodes are optimized one by one or the performance variation law is described.
  • Cambricon MLU-100 the model parallelism and operator fusion are described, the optimal model parallelism is selected for the operator, and the rules of fusion block size, redundant computation and performance are described.
  • the intermediate expression file is optimized from the perspective of multi-node coordination.
  • the optimization principle is as follows:
  • the performance model will be constructed based on the amount of computation as a guide for optimization.
  • the interface provided by MLU-100 mainly supports the optimization of model parallelism and fusion mode. Therefore, the single-node optimization part mainly optimizes these two optimized nodes and describes the performance change law.
  • the chip is a multi-core architecture, which can allocate several cores to each operator for its calculation.
  • allocating too many cores to an operator will result in a small amount of computation per core, which cannot saturate core performance and increase the communication overhead between cores. Therefore, guided by the most significant impact of computation on operator performance, the relationship between the most model parallelism and computation is constructed based on performance tests, and the model parallelism at the base layer is determined based on it.
  • each fusion block can only set a uniform degree of parallelism, and the optimal model parallelism of different layers is different, in order to make the fusion block satisfy the optimal model parallelism of all layers as much as possible, this step adopts The parallelism of the layer model, and the method of re-aggregating the layers with similar model parallelism and merging them.
  • the size of each fusion block is controlled so that the ratio of its total computation to the degree of parallelism is close to but smaller than the single-core saturated computation.
  • FIG. 2 shows a schematic diagram of an optimization flow for optimizing the intermediate expression file from the perspective of multi-node coordination.
  • the specific steps for optimizing the intermediate expression file from the perspective of multi-node coordination include:
  • next base layer can be fused with the current fusion block; if so, continue to judge whether the next base layer is the fully connected layer or convolution layer of the neural network; if so, count the calculation amount of the base layer and add The current total calculation amount, add the base layer to the current fusion block, and go to the next step; if not, directly add the base layer to the current fusion block and go to the next step; if not If yes, open a new fusion block;
  • the specific steps include traversing the intermediate expression file and processing it layer by layer. Since each unit of the intermediate expression file contains the information of each operator (layer), the text that conforms to the hardware interface syntax will be generated according to the operator information during traversal. file, which is the network template file.
  • the network template file is a network template file of a software development kit.
  • the step S13 further includes using the abstraction layer to hide redundant operations (for example, operations such as initialization, memory allocation, etc.), while exposing optimization nodes.
  • redundant operations for example, operations such as initialization, memory allocation, etc.
  • the S13 can list the interfaces provided by the Cambricon MLU-100 and the optimized nodes supported by the middle layer.
  • the user can easily adjust the network structure, hyperparameters, etc. through the network template file, and can support the adjustment of some hyperparameters at runtime.
  • the network template file is compiled into an executable inference application by a G++ compiler.
  • This embodiment also provides a computer storage medium (also referred to as a computer-readable storage medium), on which a computer program is stored, and when the computer program is executed by a processor, the method for compiling the neural network is implemented.
  • a computer storage medium also referred to as a computer-readable storage medium
  • a person of ordinary skill in the art can understand a computer-readable storage medium: all or part of the steps of implementing the above method embodiments can be completed by hardware related to a computer program.
  • the aforementioned computer program may be stored in a computer-readable storage medium.
  • the steps including the above method embodiments are executed; and the foregoing storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.
  • the neural network compilation method described in this embodiment aims to design and implement a compilation toolchain framework, intermediate representation and corresponding optimization algorithm that can automatically adjust parameters and generate codes according to software and hardware information, so that when it is calculated on the target chip, While not changing the network output results, a higher calculation rate and a smaller calculation delay can be obtained in a shorter optimization time. And it is convenient for users to debug and adjust parameters by themselves.
  • This embodiment provides a compiling system for a neural network, including:
  • a translation module for translating network files into intermediate expression files
  • an optimization module for optimizing the intermediate expression file from the perspective of performance analysis, single-node and multi-node collaboration
  • the file generation module is used to generate the network template file based on the hardware interface from the optimized intermediate expression file;
  • the compilation module is used for compiling the network template file into an executable reasoning application.
  • FIG. 3 is a schematic diagram showing the principle structure of a compiling system of a neural network in an embodiment.
  • the compiling system 3 of the neural network includes a translation module 31, an optimization module 32, a file generation module 33 and a compilation module 34.
  • the translation module 31 is used to translate the network file into an intermediate expression file.
  • the translation module 31 translates the network file including the structure and parameters into an intermediate expression file including some hardware information.
  • the translation module 31 uses the API in the Python.ONNX library to read the neural network file in the onnx format as structured data, and the structured data includes the network structure (computation graph), operator detailed information ( Calculate the nodes of the graph) and other information, and use TVM to extract the weight information required by the operators contained in the onnx file, and store it as a text file for later use.
  • the structured data includes the network structure (computation graph), operator detailed information ( Calculate the nodes of the graph) and other information, and use TVM to extract the weight information required by the operators contained in the onnx file, and store it as a text file for later use.
  • the intermediate expression file includes an abstraction layer, a description of the abstraction layer, and main fields;
  • the abstraction layer includes a model, an operator set, a fusion block, a base layer and an operator;
  • the description of the model includes a description of the complete model execution flow; the description of the operator set includes a specified version of the operator set; the description of the fusion block includes a block formed by merging multiple base layers; the description of the base layer includes represents the base layer of an operator in the network; the description of the operator includes a detailed description of the operator;
  • the main domain of the model consists of a set of fusion fasts, the intermediate representation of which is a version;
  • the main fields of the operator set include a version and a list of included operators
  • the main domain of the fusion block includes a set of layers, and the inputs and outputs of the layers;
  • the main domains of the base layer include operator, input, output, parallelism;
  • the main fields of the operator include operator type and operator attribute.
  • the optimization module 32 is used to optimize the intermediate expression file from the perspective of performance analysis, single-node and multi-node coordination. Continuing to refer to FIG. 3 , the optimization module 32 includes a performance analysis unit 321 , a single-node optimization unit 322 and a collaborative optimization unit 323 .
  • the performance analysis unit 321 is configured to optimize the intermediate expression file from the perspective of performance analysis.
  • the performance analysis unit 321 uses a performance test-based approach to characterize performance, generates a series of measurement performances with different parameters, obtains influence parameters that affect operator performance, and uses the influence parameters to build a mathematical model to characterize performance.
  • the intermediate expression file is optimized from the perspective of performance analysis.
  • the influence parameters that influence the performance of the operator can be calculated by the PCA algorithm.
  • the single-node optimization unit 322 is configured to optimize the intermediate expression file from a single-node perspective.
  • the single node optimization unit 322 optimizes the optimized nodes one by one or describes the performance variation law according to the optimization result of optimizing the intermediate expression file from the perspective of performance analysis and the interface supported by the target hardware.
  • the collaborative optimization unit 323 is configured to optimize the intermediate expression file from the perspective of multi-node collaboration.
  • the collaborative optimization unit 323 reads the next base layer; judges whether the next base layer can be fused with the current fusion block; if so, continues to judge whether the next base layer is the fully connected layer or volume of the neural network If it is, then count the calculation amount of the base layer, add the current total calculation amount, add the base layer to the current fusion block, and transfer to judge whether the total calculation amount in the current fusion block exceeds the calculation amount Threshold; if not, directly add the base layer to the current fusion block, and transfer to judge whether the total calculation amount in the current fusion block exceeds the calculation amount threshold; if not, open a new fusion block; if it exceeds the calculation amount If the amount threshold is exceeded, then go to open a new fusion block; if the calculation amount threshold is not exceeded, go to read the next basic layer.
  • the file generation module 33 is configured to generate a network template file based on the hardware interface from the optimized intermediate expression file.
  • the network template file is a network template file of a software development kit.
  • the file generation module 33 is to traverse the intermediate expression file and process it layer by layer. Since each unit of the intermediate expression file contains the information of each operator (layer), it will generate the corresponding hardware according to the operator information during traversal.
  • the text file of the interface syntax that is, the network template file.
  • the network template file is a network template file of a software development kit.
  • the file generation module 33 is further configured to use the abstraction layer to hide redundant operations (for example, operations such as initialization, memory allocation, etc.), while exposing optimization nodes.
  • redundant operations for example, operations such as initialization, memory allocation, etc.
  • the user can easily adjust the network structure, hyperparameters, etc. through the network template file, and can support the adjustment of some hyperparameters at runtime.
  • the compiling module 34 is used for compiling the network template file into an executable reasoning application.
  • the compiling module 34 compiles the network template file into an executable reasoning application through a G++ compiler.
  • each module of the above system is only a division of logical functions, and may be fully or partially integrated into a physical entity in actual implementation, or may be physically separated.
  • these modules can all be implemented in the form of software calling through processing elements, or all can be implemented in hardware, and some modules can be implemented in the form of calling software through processing elements, and some modules can be implemented in hardware.
  • the x module may be a separately established processing element, or may be integrated in a certain chip of the above-mentioned system to be implemented.
  • the x module can also be stored in the memory of the above-mentioned system in the form of program code, and is called by a certain processing element of the above-mentioned system to execute the function of the above x-module.
  • the implementation of other modules is similar. All or part of these modules can be integrated together or implemented independently.
  • the processing element described here may be an integrated circuit with signal processing capability.
  • each step of the above-mentioned method or each of the above-mentioned modules can be completed by an integrated logic circuit of hardware in the processor element or an instruction in the form of software.
  • the above modules may be one or more integrated circuits configured to implement the above methods, such as: one or more specific integrated circuits (Application Specific Integrated Circuit, ASIC for short), one or more microprocessors (Digital Singnal Processor, DSP for short), one or more Field Programmable Gate Arrays (FPGA for short), etc.
  • ASIC Application Specific Integrated Circuit
  • DSP Digital Singnal Processor
  • FPGA Field Programmable Gate Arrays
  • the processing element may be a general-purpose processor, such as a central processing unit (Central Processing Unit, CPU for short) or other processors that can call program codes.
  • CPU Central Processing Unit
  • These modules can be integrated together and implemented in the form of a System-on-a-chip (SOC for short).
  • SOC System-on-a-chip
  • This embodiment provides a compiling device, including: a processor, a memory, a transceiver, a communication interface or/and a system bus; the memory and the communication interface are connected to the processor and the transceiver through the system bus and complete mutual communication, and the memory is used for The computer program is stored, the communication interface is used to communicate with other devices, and the processor and the transceiver are used to run the computer program, so that the compiling device executes each step of the above neural network compiling method.
  • the system bus mentioned above may be a Peripheral Component Interconnect (PCI for short) bus or an Extended Industry Standard Architecture (EISA for short) bus or the like.
  • PCI Peripheral Component Interconnect
  • EISA Extended Industry Standard Architecture
  • the system bus can be divided into address bus, data bus, control bus and so on. For ease of presentation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
  • the communication interface is used to realize the communication between the database access device and other devices (such as client, read-write library and read-only library).
  • the memory may include random access memory (Random Access Memory, RAM for short), and may also include non-volatile memory (non-volatile memory), such as at least one disk storage.
  • the above-mentioned processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, referred to as CPU), a network processor (Network Processor, referred to as NP), etc.; may also be a digital signal processor (Digital Signal Processing, referred to as DSP) , Application Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
  • CPU Central Processing Unit
  • NP Network Processor
  • DSP Digital Signal Processing
  • ASIC Application Specific Integrated Circuit
  • FPGA Field Programmable Gate Array
  • the protection scope of the neural network compiling method of the present invention is not limited to the execution order of the steps listed in this embodiment, and any solutions implemented by adding or subtracting steps and replacing steps in the prior art based on the principles of the present invention are included in the within the protection scope of the present invention.
  • the present invention also provides a neural network compiling system, the neural network compiling system can implement the neural network compiling method of the present invention, but the implementation device of the neural network compiling method of the present invention includes but is not limited to The structure of the neural network compiling system enumerated in this embodiment, all structural modifications and replacements of the prior art made according to the principles of the present invention are included in the protection scope of the present invention.
  • the neural network compiling method, system, computer storage medium and compiling device of the present invention aim to design and implement a compiling toolchain framework, intermediate representation, and corresponding compiling tool chain framework that can automatically adjust parameters and generate codes according to software and hardware information. Optimize the algorithm so that when it is calculated on the target chip, it can obtain a higher calculation rate and a smaller calculation delay in a shorter optimization time without changing the network output result. And it is convenient for users to debug and adjust parameters by themselves.
  • the invention effectively overcomes various shortcomings in the prior art and has high industrial utilization value.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Stored Programmes (AREA)

Abstract

一种神经网络的编译方法、系统、计算机存储介质及编译设备，所述神经网络的编译方法包括：将网络文件翻译为中间表达文件（S11）；从性能分析、单节点以及多节点协同角度对所述中间表达文件进行优化（S12）；将优化后的中间表达文件生成基于硬件接口的网络模版文件（S13）；将所述网络模版文件编译为可执行推理应用（S14）。设计并实现了一个能够自动根据软硬件信息调节参数、生成代码的编译工具链框架、中间表示以及相应优化算法，使之在目标芯片上计算时，在不改变网络输出结果的同时，在较短的优化时间内获得更高的计算速率、更小的计算时延。且方便用户自行调试、调参。

Description

神经网络的编译方法、系统、计算机存储介质及编译设备 技术领域
本发明属于神经网络技术领域，涉及一种编译方法，特别是涉及一种神经网络的编译方法、系统、计算机存储介质及编译设备。
背景技术
如今神经网络的发展极大的推动了机器学习和人工智能及其相关产业的发展，例如人脸识别、语音识别、在线翻译、自动驾驶等等技术。然而由于神经网络拥有庞大的网络结构和计算量，延迟大是影响其大规模投入工业生产的主要障碍。因此，如何减小运算时延、提高神经网络的计算速度，是发展神经网络的一个重要问题。
现有的大多数神经网络编译、优化工具在编译时，采用接收用户提供的网络文件，直接生成可执行的推理会话的方式，供Python，C++等语言调用。在进行优化时，主要根据预先制定的针对不同目标硬件以及不同算子的规则，进行前端(算子级优化，包括算子融合，公共子表达式替换等)与后端(硬件相关优化，如循环展开，向量化等)优化。
现有工具封装度较高，开放给用户的接口有限，造成调试、调参的不便。且优化过程、详细算法对用户不可见，无法支持用户做进一步优化。其次，现有优化算法灵活性较差，基于规则的方法导致在前端优化中丢失较大优化空间，且后端针对不同硬件优化可迁移性较差，需要较多人类专家的介入。
因此，如何提供一种神经网络的编译方法、系统、计算机存储介质及编译设备，以解决现有技术封装度较高，开放给用户的接口有限，造成调试、调参的不便、优化过程、详细算法对用户不可见，无法支持用户做进一步优化，优化算法灵活性较差，基于规则的方法导致在前端优化中丢失较大优化空间，且后端针对不同硬件优化可迁移性较差，需要较多人类专家的介入等缺陷，实已成为本领域技术人员亟待解决的技术问题。
发明内容
鉴于以上所述现有技术的缺点，本发明的目的在于提供一种神经网络的编译方法、系统、计算机存储介质及编译设备，用于解决现有技术封装度较高，开放给用户的接口有限，造成调试、调参的不便、优化过程、详细算法对用户不可见，无法支持用户做进一步优化，优化算法灵活性较差，基于规则的方法导致在前端优化中丢失较大优化空间，且后端针对不同硬件优化可迁移性较差，需要较多人类专家的介入的问题。
为实现上述目的及其他相关目的，本发明一方面提供一种神经网络的编译方法，包括：将网络文件翻译为中间表达文件；从性能分析、单节点以及多节点协同角度对所述中间表达文件进行优化；将优化后的中间表达文件生成基于硬件接口的网络模版文件；将所述网络模版文件编译为可执行推理应用。
于本发明的一实施例中，所述网络文件包括结构和参数；所述中间表达文件包括抽象层、抽象层的描述及主要域；所述抽象层包括模型、算子集、融合块、基本层及操作算子；所述模型的描述包括描述完整模型执行流；所述算子集的描述包括指定算子集版本；所述融合块的描述包括多个基本层融合而成的块；所述基本层的描述包括代表网络中一个算子的基本层；所述操作算子的描述包括算子的详细描述；所述模型的主要域包括一组融合快，其中间表示版本；所述算子集的主要域包括版本及包含的算子列表；所述融合块的主要域包括一组层，及层的输入及输出；所述基本层的主要域包括算子、输入、输出、并行度；所述算子的主要域包括算子类型及算子属性。
于本发明的一实施例中，从性能分析角度对所述中间表达文件进行优化的步骤包括：采用基于性能测试的方式刻画性能，生成系列参数各异的测量性能，并获取影响算子性能的影响参数，利用所述影响参数构建数学模型以刻画性能。
于本发明的一实施例中，从单节点角度对所述中间表达文件进行优化的步骤包括：对模型并行度以及算子融合进行刻画，为算子挑选最优模型并行度，并刻画融合块大小、冗余计算量以及性能的规律。
于本发明的一实施例中，从多节点协同角度对所述中间表达文件进行优化的步骤包括：读取下一基本层；判断下一基本层是否能与当前的融合块进行融合；若能够，继续判断下一基本层是否为神经网络的全连接层或卷积层；若是，则统计该基本层的计算量，并加入当前的总计算量，并将该基本层加入到当前的融合块中，转入下一步骤；若否，则直接将该基本层加入到当前的融合块中，转入下一步骤；若不能够，则开启新的融合块；判断当前的融合块中总计算量是否超过计算量阈值，若是，则转入开启新的融合块的步骤；若否，则转入读取下一基本层的步骤。
于本发明的一实施例中，将优化后的中间表达文件生成网络模版文件的步骤还包括利用所述抽象层，将冗余操作隐藏，同时暴露优化节点。
于本发明的一实施例中，通过G++编译器将所述网络模版文件编译为可执行推理应用。
本发明另一方面提供一种神经网络的编译系统，包括：翻译模块，用于将网络文件翻译为中间表达文件；优化模块，用于从性能分析、单节点以及多节点协同角度对所述中间表达 文件进行优化；文件生成模块，用于将优化后的中间表达文件生成基于硬件接口的网络模版文件；编译模块，用于将所述网络模版文件编译为可执行推理应用。
本发明又一方面提供一种计算机存储介质，其上存储有计算机程序，该计算机程序被处理器执行时实现所述神经网络的编译方法。
本发明最后一方面提供一种编译设备，包括：处理器及存储器；所述存储器用于存储计算机程序，所述处理器用于执行所述存储器存储的计算机程序，以使所述编译设备执行所述神经网络的编译方法。
如上所述，本发明所述的神经网络的编译方法、系统、计算机存储介质及编译设备，具有以下有益效果：
本发明所述神经网络的编译方法、系统、计算机存储介质及编译设备旨在设计并实现一个能够自动根据软硬件信息调节参数、生成代码的编译工具链框架、中间表示以及相应优化算法，使之在目标芯片上计算时，在不改变网络输出结果的同时，在较短的优化时间内获得更高的计算速率、更小的计算时延。且方便用户自行调试、调参。
附图说明
图1显示为本发明的神经网络的编译方法于一实施例中的流程示意图。
图2显示为本发明的从多节点协同角度对所述中间表达文件进行优化的优化流程示意图。
图3显示为本发明的神经网络的编译方法于一实施例中的原理结构示意图。
元件标号说明
3         神经网络的编译系统
31        翻译模块
32        优化模块
33        文件生成模块
34        编译模块
321       性能分析单元
322       单节点优化单元
323       协同优化单元
S11～S14  步骤
具体实施方式
以下通过特定的具体实例说明本发明的实施方式，本领域技术人员可由本说明书所揭露的内容轻易地了解本发明的其他优点与功效。本发明还可以通过另外不同的具体实施方式加以实施或应用，本说明书中的各项细节也可以基于不同观点与应用，在没有背离本发明的精神下进行各种修饰或改变。需说明的是，在不冲突的情况下，以下实施例及实施例中的特征可以相互组合。
需要说明的是，以下实施例中所提供的图示仅以示意方式说明本发明的基本构想，遂图式中仅显示与本发明中有关的组件而非按照实际实施时的组件数目、形状及尺寸绘制，其实际实施时各组件的型态、数量及比例可为一种随意的改变，且其组件布局型态也可能更为复杂。
实施例一
本发明提供一种神经网络的编译方法，包括：
将网络文件翻译为中间表达文件；
从性能分析、单节点以及多节点协同角度对所述中间表达文件进行优化；
将优化后的中间表达文件生成基于硬件接口的网络模版文件；
将所述网络模版文件编译为可执行推理应用。
以下将结合图示对本实施例所提供的神经网络的编译方法进行详细描述。本实施例所述神经网络的编译方法为用户提供端到端的推理服务，由已有的、打包好的网络文件生成基于目标硬件接口的模板文件，进而生成可执行的推理应用。而优化过程能够优化生成代码的执行效率。
请参阅图1，显示为神经网络的编译方法于一实施例中的流程示意图。如图1所示，所述神经网络的编译方法具体包括以下步骤：
S11，将网络文件翻译为中间表达文件。
具体步骤包括使用Python.ONNX库中的API将onnx格式的神经网络文件读取为结构化的数据，该结构化数据中包含网络结构(计算图)、算子详细信息(计算图的节点)等信息，同时使用TVM提取onnx文件中包含的算子所需的权重信息，并存储为文本文件供之后的运行使用。
具体地，将包括结构和参数的网络文件翻译为包含部分硬件信息的中间表达文件。
在本实施例中，所述中间表达文件包括抽象层、抽象层的描述及主要域；
所述抽象层包括模型、算子集、融合块、基本层及操作算子；
所述模型的描述包括描述完整模型执行流；所述算子集的描述包括指定算子集版本；所述融合块的描述包括多个基本层融合而成的块；所述基本层的描述包括代表网络中一个算子的基本层；所述操作算子的描述包括算子的详细描述；
所述模型的主要域包括一组融合快，其中间表示版本；
所述算子集的主要域包括版本及包含的算子列表；
所述融合块的主要域包括一组层，及层的输入及输出；
所述基本层的主要域包括算子、输入、输出、并行度；
所述算子的主要域包括算子类型及算子属性。
中间表达文件的具体内容如表1所示：
表1：中间表达文件的具体内容
抽象层 描述 主要域
Model模型 描述完整模型执行流 一组融合快，中间表示版本
Op Set算子集 指定算子集版本 版本，包含的算子列表
F-Block融合块 多个基本层融合而成的块 一组层，输入，输出
Layer基本层 代表网络中一个算子的层 算子、输入、输出、并行度
Operator算子 算子的详细描述 算子类型、属性
S12，从性能分析、单节点以及多节点协同角度对所述中间表达文件进行优化。
具体地，从性能分析角度对所述中间表达文件进行优化的步骤包括：
采用基于性能测试的方式刻画性能，生成系列参数各异的测量性能，并获取影响算子性能的影响参数，利用所述影响参数构建数学模型以刻画性能。在本实施例中，由于在开发过程中发现实际网络中的算子性能与理论模型相差较大，所以从性能分析角度对所述中间表达文件进行优化。
在本实施例中，可通过PCA算法计算影响算子性能的影响参数。
以Cambricon MLU-100为例，在进行卷积操作时，算子计算量以及通道数为影响性能的主要参数。
从单节点角度对所述中间表达文件进行优化的步骤包括：
根据从性能分析角度对所述中间表达文件进行优化的优化结果，以及目标硬件所支持的接口对优化节点进行逐一优化或性能变化规律的刻画。
以Cambricon MLU-100为例，对模型并行度以及算子融合进行刻画，为算子挑选最优模型并行度，并刻画融合块大小、冗余计算量以及性能的规律。
从多节点协同角度对所述中间表达文件进行优化，优化原理如下：
由于优化节点众多且每个节点选择众多，导致总体优化空间极大，无法采用朴素搜索的 方式，故需要利用启发信息进行搜索。在利用启发信息进行搜索时，需要评估某种参数选择的优劣。然而，在观测到已有的针对硬件的性能模型与算子实际运行性能差距较大，现有性能模型无法对算子的运行准确地刻画。故采用基于性能测试的方式，生成一组参数各异的算子测量其实际运行性能。并使用主成分分析方法提取对算子性能影响最为显著的参数，利用这些参数进行建模。以MLU-100为例，通过主成分分析发现算子的计算量对性能影响最为显著。故在之后的单节点、协同优化过程中，将以计算量构建性能模型作为优化的指导。
MLU-100提供的接口主要支持模型并行度以及融合模式的优化，故单节点优化部分主要针对这两个优化节点进行优化以及性能变化规律的刻画。
a.模型并行度方面，该芯片为多核架构，能够为每个算子分配若干个核心供其计算。然而为一个算子分配过多的核心会导致每个核心计算量较小，无法使核心性能达到饱和，且核间通信开销增大。故以计算量对算子性能影响最为显著为指导，基于性能测试构建了最有模型并行度与计算量之间的关系，并根据它确定基本层的模型并行度。
b.算子融合方面，将若干算子融合为一个融合算子能够以流水线化的运行增加其并行度。然而由于卷积计算的光环效应，融合块越大、并行度越高，被引入的冗余计算越多，故需要控制融合块的大小以及并行度。根据对于具有不同计算量大小的融合块的研究发现，当融合块计算量与并行度比值接近每个核心的饱和计算量时，该融合块能较好地平衡并行化带来的性能提升以及冗余计算带来的开销。
在多节点协同优化步骤中，需要为模型挑选合适的融合模式，并为每个融合块设置合适的并行度。由于每个融合块只能设置统一的并行度，而不同层的最优模型并行度各不相同，为了使融合块尽可能满足其中所有层的最优模型并行度，本步骤采用先确定每个层模型并行度，再聚集模型并行度相近的层加以融合的方式。而在融合时，控制每个融合块的大小，使得其总计算量与并行度的比值接近但小于单核心饱和计算量。
请参阅图2，显示为从多节点协同角度对所述中间表达文件进行优化的优化流程示意图。如图2所示，所述从多节点协同角度对所述中间表达文件进行优化的具体步骤包括：
读取下一基本层；
判断下一基本层是否能与当前的融合块进行融合；若能够，继续判断下一基本层是否为神经网络的全连接层或卷积层；若是，则统计该基本层的计算量，并加入当前的总计算量，并将该基本层加入到当前的融合块中，转入下一步骤；若否，则直接将该基本层加入到当前的融合块中，转入下一步骤；若不能够，则开启新的融合块；
判断当前的融合块中总计算量是否超过计算量阈值，若是，则转入开启新的融合块的步 骤；若否，则转入读取下一基本层的步骤。
S13，将优化后的中间表达文件生成基于硬件接口的网络模版文件，
具体步骤包括为遍历中间表达文件并逐层处理，由于中间表达文件的每个单元包含了每个算子(层)的信息，故在遍历时将会按照算子信息生成符合硬件接口语法的文本文件，即网络模板文件。所述网络模版文件为软件开发工具包的网络模板文件。
于本实施例中，所述S13还包括利用所述抽象层，将冗余操作隐藏(例如，初始化、内存分配等操作)，同时暴露优化节点。
例如，所述S13可列出Cambricon MLU-100提供的接口以及中间层所支持的优化节点。
在本实施例中，通过网络模版文件用户能够方便地对网络结构、超参数等进行调整，并且能够支持运行时调整部分超参数。
S14，将所述网络模版文件编译为可执行推理应用。
于本实施例中，通过G++编译器将所述网络模版文件编译为可执行推理应用。
本实施例还提供一种计算机存储介质(亦称计算机可读存储介质)，其上存储有计算机程序，该计算机程序被处理器执行时实现所述神经网络的编译方法。
本领域普通技术人员可以理解计算机可读存储介质：实现上述各方法实施例的全部或部分步骤可以通过计算机程序相关的硬件来完成。前述的计算机程序可以存储于一计算机可读存储介质中。该程序在执行时，执行包括上述各方法实施例的步骤；而前述的存储介质包括：ROM、RAM、磁碟或者光盘等各种可以存储程序代码的介质。
本实施例所述神经网络的编译方法旨在设计并实现一个能够自动根据软硬件信息调节参数、生成代码的编译工具链框架、中间表示以及相应优化算法，使之在目标芯片上计算时，在不改变网络输出结果的同时，在较短的优化时间内获得更高的计算速率、更小的计算时延。且方便用户自行调试、调参。
实施例二
本实施例提供一种神经网络的编译系统，包括：
翻译模块，用于将网络文件翻译为中间表达文件；
优化模块，用于从性能分析、单节点以及多节点协同角度对所述中间表达文件进行优化；
文件生成模块，用于将优化后的中间表达文件生成基于硬件接口的网络模版文件；
编译模块，用于将所述网络模版文件编译为可执行推理应用。
以下将结合图示对本实施例所提供的神经网络的编译系统进行详细描述。请参与图3，显示为神经网络的编译系统于一实施例中的原理结构示意图。如图3所示，所述神经网络的 编译系统3包括翻译模块31、优化模块32、文件生成模块33及编译模块34。
所述翻译模块31用于将网络文件翻译为中间表达文件。
具体地，所述翻译模块31将包括结构和参数的网络文件翻译为包含部分硬件信息的中间表达文件。
更具体地，所述翻译模块31使用Python.ONNX库中的API将onnx格式的神经网络文件读取为结构化的数据，该结构化数据中包含网络结构(计算图)、算子详细信息(计算图的节点)等信息，同时使用TVM提取onnx文件中包含的算子所需的权重信息，并存储为文本文件供之后的运行使用。
在本实施例中，所述中间表达文件包括抽象层、抽象层的描述及主要域；
所述抽象层包括模型、算子集、融合块、基本层及操作算子；
所述模型的描述包括描述完整模型执行流；所述算子集的描述包括指定算子集版本；所述融合块的描述包括多个基本层融合而成的块；所述基本层的描述包括代表网络中一个算子的基本层；所述操作算子的描述包括算子的详细描述；
所述模型的主要域包括一组融合快，其中间表示版本；
所述算子集的主要域包括版本及包含的算子列表；
所述融合块的主要域包括一组层，及层的输入及输出；
所述基本层的主要域包括算子、输入、输出、并行度；
所述算子的主要域包括算子类型及算子属性。
所述优化模块32用于从性能分析、单节点以及多节点协同角度对所述中间表达文件进行优化。继续参阅图3，所述优化模块32包括性能分析单元321、单节点优化单元322及协同优化单元323。
所述性能分析单元321用于从性能分析角度对所述中间表达文件进行优化。
具体地，所述性能分析单元321采用基于性能测试的方式刻画性能，生成系列参数各异的测量性能，并获取影响算子性能的影响参数，利用所述影响参数构建数学模型以刻画性能。在本实施例中，由于在开发过程中发现实际网络中的算子性能与理论模型相差较大，所以从性能分析角度对所述中间表达文件进行优化。
在本实施例中，可通过PCA算法计算影响算子性能的影响参数。
所述单节点优化单元322用于从单节点角度对所述中间表达文件进行优化。
具体地，所述单节点优化单元322根据从性能分析角度对所述中间表达文件进行优化的优化结果，以及目标硬件所支持的接口对优化节点进行逐一优化或性能变化规律的刻画。
所述协同优化单元323用于从多节点协同角度对所述中间表达文件进行优化。
具体地，所述协同优化单元323读取下一基本层；判断下一基本层是否能与当前的融合块进行融合；若能够，继续判断下一基本层是否为神经网络的全连接层或卷积层；若是，则统计该基本层的计算量，并加入当前的总计算量，并将该基本层加入到当前的融合块中，转入判断当前的融合块中总计算量是否超过计算量阈值；若否，则直接将该基本层加入到当前的融合块中，转入判断当前的融合块中总计算量是否超过计算量阈值；若不能够，则开启新的融合块；若超过计算量阈值，则转入开启新的融合块；若未超过计算量阈值，则转入读取下一基本层。
所述文件生成模块33用于将优化后的中间表达文件生成基于硬件接口的网络模版文件。所述网络模版文件为软件开发工具包的网络模板文件。
具体所述文件生成模块33为遍历中间表达文件并逐层处理，由于中间表达文件的每个单元包含了每个算子(层)的信息，故在遍历时将会按照算子信息生成符合硬件接口语法的文本文件，即网络模板文件。所述网络模版文件为软件开发工具包的网络模板文件。
于本实施例中，所述文件生成模块33还用于利用所述抽象层，将冗余操作隐藏(例如，初始化、内存分配等操作)，同时暴露优化节点。
在本实施例中，通过网络模版文件用户能够方便地对网络结构、超参数等进行调整，并且能够支持运行时调整部分超参数。
所述编译模块34用于将所述网络模版文件编译为可执行推理应用。
于本实施例中，所述编译模块34通过G++编译器将所述网络模版文件编译为可执行推理应用。
需要说明的是，应理解以上系统的各个模块的划分仅仅是一种逻辑功能的划分，实际实现时可以全部或部分集成到一个物理实体上，也可以物理上分开。且这些模块可以全部以软件通过处理元件调用的形式实现，也可以全部以硬件的形式实现，还可以部分模块通过处理元件调用软件的形式实现，部分模块通过硬件的形式实现。例如：x模块可以为单独设立的处理元件，也可以集成在上述系统的某一个芯片中实现。此外，x模块也可以以程序代码的形式存储于上述系统的存储器中，由上述系统的某一个处理元件调用并执行以上x模块的功能。其它模块的实现与之类似。这些模块全部或部分可以集成在一起，也可以独立实现。这里所述的处理元件可以是一种集成电路，具有信号的处理能力。在实现过程中，上述方法的各步骤或以上各个模块可以通过处理器元件中的硬件的集成逻辑电路或者软件形式的指令完成。以上这些模块可以是被配置成实施以上方法的一个或多个集成电路，例如：一个或多个 特定集成电路(Application Specific Integrated Circuit，简称ASIC)，一个或多个微处理器(Digital Singnal Processor，简称DSP)，一个或者多个现场可编程门阵列(Field Programmable Gate Array，简称FPGA)等。当以上某个模块通过处理元件调度程序代码的形式实现时，该处理元件可以是通用处理器，如中央处理器(Central Processing Unit，简称CPU)或其它可以调用程序代码的处理器。这些模块可以集成在一起，以片上系统(System-on-a-chip，简称SOC)的形式实现。
实施例三
本实施例提供一种编译设备，包括：处理器、存储器、收发器、通信接口或/和系统总线；存储器和通信接口通过系统总线与处理器和收发器连接并完成相互间的通信，存储器用于存储计算机程序，通信接口用于和其他设备进行通信，处理器和收发器用于运行计算机程序，使编译设备执行如上神经网络的编译方法的各个步骤。
上述提到的系统总线可以是外设部件互连标准(Peripheral Component Interconnect，简称PCI)总线或扩展工业标准结构(Extended Industry Standard Architecture，简称EISA)总线等。该系统总线可以分为地址总线、数据总线、控制总线等。为便于表示，图中仅用一条粗线表示，但并不表示仅有一根总线或一种类型的总线。通信接口用于实现数据库访问装置与其他设备(如客户端、读写库和只读库)之间的通信。存储器可能包含随机存取存储器(Random Access Memory，简称RAM)，也可能还包括非易失性存储器(non-volatile memory)，例如至少一个磁盘存储器。
上述的处理器可以是通用处理器，包括中央处理器(Central Processing Unit，简称CPU)、网络处理器(Network Processor，简称NP)等；还可以是数字信号处理器(Digital Signal Processing，简称DSP)、专用集成电路(Application Specific Integrated Circuit，简称ASIC)、现场可编程门阵列(Field Programmable Gate Array，简称FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。
本发明所述的神经网络的编译方法的保护范围不限于本实施例列举的步骤执行顺序，凡是根据本发明的原理所做的现有技术的步骤增减、步骤替换所实现的方案都包括在本发明的保护范围内。
本发明还提供一种神经网络的编译系统，所述神经网络的编译系统可以实现本发明所述的神经网络的编译方法，但本发明所述的神经网络的编译方法的实现装置包括但不限于本实施例列举的神经网络的编译系统的结构，凡是根据本发明的原理所做的现有技术的结构变形和替换，都包括在本发明的保护范围内。
综上所述，本发明所述神经网络的编译方法、系统、计算机存储介质及编译设备旨在设计并实现一个能够自动根据软硬件信息调节参数、生成代码的编译工具链框架、中间表示以及相应优化算法，使之在目标芯片上计算时，在不改变网络输出结果的同时，在较短的优化时间内获得更高的计算速率、更小的计算时延。且方便用户自行调试、调参。本发明有效克服了现有技术中的种种缺点而具高度产业利用价值。
上述实施例仅例示性说明本发明的原理及其功效，而非用于限制本发明。任何熟悉此技术的人士皆可在不违背本发明的精神及范畴下，对上述实施例进行修饰或改变。因此，举凡所属技术领域中具有通常知识者在未脱离本发明所揭示的精神与技术思想下所完成的一切等效修饰或改变，仍应由本发明的权利要求所涵盖。

Claims (10)

  1. 一种神经网络的编译方法，其特征在于，包括：
    将网络文件翻译为中间表达文件；
    从性能分析、单节点以及多节点协同角度对所述中间表达文件进行优化；
    将优化后的中间表达文件生成基于硬件接口的网络模版文件；
    将所述网络模版文件编译为可执行推理应用。
  2. 根据权利要求1所述的神经网络的编译方法，其特征在于，
    所述网络文件包括结构和参数；
    所述中间表达文件包括抽象层、抽象层的描述及主要域；
    所述抽象层包括模型、算子集、融合块、基本层及操作算子；
    所述模型的描述包括描述完整模型执行流；所述算子集的描述包括指定算子集版本；
    所述融合块的描述包括多个基本层融合而成的块；所述基本层的描述包括代表网络中一个算子的基本层；所述操作算子的描述包括算子的详细描述；
    所述模型的主要域包括一组融合快，其中间表示版本；
    所述算子集的主要域包括版本及包含的算子列表；
    所述融合块的主要域包括一组层，及层的输入及输出；
    所述基本层的主要域包括算子、输入、输出、并行度；
    所述算子的主要域包括算子类型及算子属性。
  3. 根据权利要求2所述的神经网络的编译方法，其特征在于，从性能分析角度对所述中间表达文件进行优化的步骤包括：
    采用基于性能测试的方式刻画性能，生成系列参数各异的测量性能，并获取影响算子性能的影响参数，利用所述影响参数构建数学模型以刻画性能。
  4. 根据权利要求3所述的神经网络的编译方法，其特征在于，从单节点角度对所述中间表达文件进行优化的步骤包括：
    对模型并行度以及算子融合进行刻画，为算子挑选最优模型并行度，并刻画融合块大小、冗余计算量以及性能的规律。
  5. 根据权利要求3所述的神经网络的编译方法，其特征在于，从多节点协同角度对所述中间表达文件进行优化的步骤包括：
    读取下一基本层；
    判断下一基本层是否能与当前的融合块进行融合；若能够，继续判断下一基本层是否为神经网络的全连接层或卷积层；若是，则统计该基本层的计算量，并加入当前的总计算量，并将该基本层加入到当前的融合块中，转入下一步骤；若否，则直接将该基本层加入到当前的融合块中，转入下一步骤；若不能够，则开启新的融合块；
    判断当前的融合块中总计算量是否超过计算量阈值，若是，则转入开启新的融合块的步骤；若否，则转入读取下一基本层的步骤。
  6. 根据权利要求3所述的神经网络的编译方法，其特征在于，将优化后的中间表达文件生成网络模版文件的步骤还包括利用所述抽象层，将冗余操作隐藏，同时暴露优化节点。
  7. 根据权利要求3所述的神经网络的编译方法，其特征在于，通过G++编译器将所述网络模版文件编译为可执行推理应用。
  8. 一种神经网络的编译系统，其特征在于，包括：
    翻译模块，用于将网络文件翻译为中间表达文件；
    优化模块，用于从性能分析、单节点以及多节点协同角度对所述中间表达文件进行优化；
    文件生成模块，用于将优化后的中间表达文件生成基于硬件接口的网络模版文件；
    编译模块，用于将所述网络模版文件编译为可执行推理应用。
  9. 一种计算机存储介质，其上存储有计算机程序，其特征在于，该计算机程序被处理器执行时实现权利要求1至7中任一项所述神经网络的编译方法。
  10. 一种编译设备，其特征在于，包括：处理器及存储器；
    所述存储器用于存储计算机程序，所述处理器用于执行所述存储器存储的计算机程序，以使所述编译设备执行如权利要求1至7中任一项所述神经网络的编译方法。
PCT/CN2021/095209 2020-11-05 2021-05-21 神经网络的编译方法、系统、计算机存储介质及编译设备 Ceased WO2022095413A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/846,282 US20250348296A1 (en) 2020-11-05 2021-05-21 Method and system for compiling neural network, computer storage medium, and compilation device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011224090.4A CN112529175B (zh) 2020-11-05 2020-11-05 神经网络的编译方法、系统、计算机存储介质及编译设备
CN202011224090.4 2020-11-05

Publications (1)

Publication Number Publication Date
WO2022095413A1 true WO2022095413A1 (zh) 2022-05-12

Family

ID=74980667

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/095209 Ceased WO2022095413A1 (zh) 2020-11-05 2021-05-21 神经网络的编译方法、系统、计算机存储介质及编译设备

Country Status (3)

Country Link
US (1) US20250348296A1 (zh)
CN (1) CN112529175B (zh)
WO (1) WO2022095413A1 (zh)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115034402A (zh) * 2022-06-20 2022-09-09 寒武纪行歌(南京)科技有限公司 模型推理性能的优化方法、装置及相关产品
CN115238876A (zh) * 2022-07-19 2022-10-25 北京苹芯科技有限公司 一种基于异构存储的存内神经网络计算的装置及方法
CN115423082A (zh) * 2022-09-26 2022-12-02 北京航空航天大学 一种硬件特性相关的深度模型计算图自动优化方法
CN115423101A (zh) * 2022-08-19 2022-12-02 贝式计算(天津)信息技术有限公司 基于编译器的张量数据计算推理方法、装置及存储介质
CN115904394A (zh) * 2023-03-02 2023-04-04 之江实验室 一种面向众核架构的神经网络增量编译方法和装置
CN117236426A (zh) * 2022-12-20 2023-12-15 北京九章云极科技有限公司 一种数据处理方法及系统
WO2024040844A1 (zh) * 2022-08-24 2024-02-29 北京百度网讯科技有限公司 模型调试方法、装置、电子设备及存储介质
CN120144195A (zh) * 2025-03-04 2025-06-13 电子科技大学 一种面向非易失性存算一体架构的缓存容量求解器

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112328227B (zh) 2020-11-03 2022-02-25 清华大学 编译方法、装置、计算设备和介质
CN112529175B (zh) * 2020-11-05 2022-03-18 上海交通大学 神经网络的编译方法、系统、计算机存储介质及编译设备
US11775317B2 (en) * 2021-04-30 2023-10-03 International Business Machines Corporation Locate neural network performance hot spots
CN114895920A (zh) * 2022-05-31 2022-08-12 苏州浪潮智能科技有限公司 Transformer网络的算子融合方法及装置
CN117709400A (zh) * 2022-09-02 2024-03-15 深圳忆海原识科技有限公司 层次化系统、运算方法、运算装置、电子设备及存储介质
CN116306918A (zh) * 2023-02-13 2023-06-23 深存科技(无锡)有限公司 基于mlir的fpga神经网络模型部署方法
CN116149797B (zh) * 2023-04-04 2023-07-07 上海燧原科技有限公司 面向异构场景的ai统一计算方法、装置、设备及介质
CN117077161B (zh) * 2023-07-31 2024-05-03 上海交通大学 基于动态规划求解的隐私保护深度模型构建方法与系统
CN121614407A (zh) * 2026-01-30 2026-03-06 中昊芯英(杭州)科技有限公司 软件测试方法、相关装置及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180136912A1 (en) * 2016-11-17 2018-05-17 The Mathworks, Inc. Systems and methods for automatically generating code for deep learning systems
CN110321999A (zh) * 2018-03-30 2019-10-11 北京深鉴智能科技有限公司 神经网络计算图优化方法
CN110717584A (zh) * 2019-09-30 2020-01-21 上海寒武纪信息科技有限公司 神经网络编译方法、编译器、计算机设备及可读存储介质
CN110766147A (zh) * 2018-07-25 2020-02-07 赛灵思公司 神经网络编译器架构及编译方法
CN111880807A (zh) * 2020-07-31 2020-11-03 Oppo广东移动通信有限公司 深度学习编译方法、装置、设备及存储介质
CN112529175A (zh) * 2020-11-05 2021-03-19 上海交通大学 神经网络的编译方法、系统、计算机存储介质及编译设备

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9684544B1 (en) * 2016-02-05 2017-06-20 Sas Institute Inc. Distributed data set storage and analysis reproducibility
CN107239315B (zh) * 2017-04-11 2019-11-15 赛灵思公司 面向神经网络异构计算平台的编程模型
CN110377288A (zh) * 2018-04-13 2019-10-25 赛灵思公司 神经网络压缩编译器及其编译压缩方法
CN110764744B (zh) * 2018-07-25 2023-12-08 赛灵思公司 用于神经网络计算的中间表示生成方法和装置
GB2586198A (en) * 2019-03-08 2021-02-17 Tescap Ltd A monitoring and recording system
US10872275B2 (en) * 2019-03-22 2020-12-22 Nokia Technologies Oy Semantic segmentation based on a hierarchy of neural networks
US10789402B1 (en) * 2019-05-01 2020-09-29 Xilinx, Inc. Compiler and hardware abstraction layer architecture for a neural network accelerator
CN111443917B (zh) * 2020-03-26 2023-12-22 上海寒武纪信息科技有限公司 神经网络运行优化方法、装置及相关产品
CN111753973B (zh) * 2020-06-22 2024-11-26 深圳鲲云信息科技有限公司 一种神经网络芯片的优化方法、系统、设备和存储介质
CN111860816A (zh) * 2020-07-08 2020-10-30 Oppo广东移动通信有限公司 神经网络模型的编译方法、装置、设备及存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180136912A1 (en) * 2016-11-17 2018-05-17 The Mathworks, Inc. Systems and methods for automatically generating code for deep learning systems
CN110321999A (zh) * 2018-03-30 2019-10-11 北京深鉴智能科技有限公司 神经网络计算图优化方法
CN110766147A (zh) * 2018-07-25 2020-02-07 赛灵思公司 神经网络编译器架构及编译方法
CN110717584A (zh) * 2019-09-30 2020-01-21 上海寒武纪信息科技有限公司 神经网络编译方法、编译器、计算机设备及可读存储介质
CN111880807A (zh) * 2020-07-31 2020-11-03 Oppo广东移动通信有限公司 深度学习编译方法、装置、设备及存储介质
CN112529175A (zh) * 2020-11-05 2021-03-19 上海交通大学 神经网络的编译方法、系统、计算机存储介质及编译设备

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115034402A (zh) * 2022-06-20 2022-09-09 寒武纪行歌(南京)科技有限公司 模型推理性能的优化方法、装置及相关产品
CN115238876A (zh) * 2022-07-19 2022-10-25 北京苹芯科技有限公司 一种基于异构存储的存内神经网络计算的装置及方法
CN115423101A (zh) * 2022-08-19 2022-12-02 贝式计算(天津)信息技术有限公司 基于编译器的张量数据计算推理方法、装置及存储介质
WO2024040844A1 (zh) * 2022-08-24 2024-02-29 北京百度网讯科技有限公司 模型调试方法、装置、电子设备及存储介质
CN115423082A (zh) * 2022-09-26 2022-12-02 北京航空航天大学 一种硬件特性相关的深度模型计算图自动优化方法
CN117236426A (zh) * 2022-12-20 2023-12-15 北京九章云极科技有限公司 一种数据处理方法及系统
CN115904394A (zh) * 2023-03-02 2023-04-04 之江实验室 一种面向众核架构的神经网络增量编译方法和装置
CN120144195A (zh) * 2025-03-04 2025-06-13 电子科技大学 一种面向非易失性存算一体架构的缓存容量求解器

Also Published As

Publication number Publication date
US20250348296A1 (en) 2025-11-13
CN112529175A (zh) 2021-03-19
CN112529175B (zh) 2022-03-18

Similar Documents

Publication Publication Date Title
WO2022095413A1 (zh) 神经网络的编译方法、系统、计算机存储介质及编译设备
Shi et al. Welder: Scheduling deep learning memory access via tile-graph
CN110187885A (zh) 一种量子程序编译的中间代码生成方法及装置
CN105335412A (zh) 用于数据转换、数据迁移的方法和装置
Roesch et al. Relay: A high-level compiler for deep learning
CN116368494A (zh) 一种神经网络编译优化方法和相关装置
Zhang et al. Cocktailer: Analyzing and optimizing dynamic control flow in deep learning
CN115481718A (zh) 一种基于精简算子集的深度学习图算一体优化器
Yamato Study and evaluation of automatic offloading method in mixed offloading destination environment
CN120371318B (zh) 面向risc-v向量扩展平台的向量优化算法翻译方法及装置
Ahmad et al. Leveraging parallel data processing frameworks with verified lifting
Larsen et al. Tensorflow graph optimizations
EP4113284A1 (en) Cross-platform code conversion method and device
CN118245181A (zh) 面向大规模超算系统的自适应程序并行划分与调度方法
Zhang et al. Python meets JIT compilers: A simple implementation and a comparative evaluation
CN110673877B (zh) 一种基于手动向量化的并行计算方法
US20250217122A1 (en) Compiler for machine learning programs
Ali et al. Parallelizing user-defined functions in the ETL workflow using orchestration style sheets
US20110246972A1 (en) Method of selecting an expression evaluation technique for domain-specific language compilation
Papavasileiou et al. Ariadne: Online provenance for big graph analytics
Wu et al. Hector: An efficient programming and compilation framework for implementing relational graph neural networks in GPU architectures
Zhao et al. Code refactoring from OpenMP to MapReduce model for big data processing
CN117519709A (zh) 计算图编译方法、编译装置、计算设备及存储介质
Liu et al. Survey and design of paleozoic: a high-performance compiler tool chain for deep learning inference accelerator
Farkas et al. Improving productivity in large scale testing at the compiler level by changing the intermediate language from C++ to Java

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21888116

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21888116

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 110124)

122 Ep: pct application non-entry in european phase

Ref document number: 21888116

Country of ref document: EP

Kind code of ref document: A1

WWP Wipo information: published in national office

Ref document number: 18846282

Country of ref document: US