WO2018171715A1

WO2018171715A1 - Automated design method and system applicable for neural network processor

Info

Publication number: WO2018171715A1
Application number: PCT/CN2018/080200
Authority: WO
Inventors: 韩银和; 许浩博; 王颖
Original assignee: 中国科学院计算技术研究所
Priority date: 2017-03-23
Filing date: 2018-03-23
Publication date: 2018-09-27
Also published as: CN107016175A; CN107016175B

Abstract

Disclosed are an automated design method and system applicable for a neural network processor. The method comprises: acquiring a topological structure configuration file of a neural network model and a hardware resource constraint file of a target hardware circuit; constructing, according to the topological structure configuration file of the neural network model and the hardware resource constraint file, a hardware architecture and descriptor file thereof, of a neural network processor, corresponding to the neural network model, and a control descriptor file for controlling data scheduling, storage and computing of the neural network processor, and then based on the hardware architecture descriptor file and the control descriptor file, generating a hardware descriptor code with the neural network processor, so as to realize a hardware circuit of the neural network processor on the target hardware circuit. The system and method realize an automated design of a neural network processor, shorten the designing cycle of the neural network processor, and adapt to the characteristics of the rapid update of network models in neural network technology and the requirement of high operating speed.

Description

Automatic design method and system for neural network processor

Technical field

The present invention relates to the technical field of neural network processor architecture, and in particular to an automatic design method and system for a neural network processor.

Background technique

With the rapid development of related technologies in the field of artificial intelligence, deep learning, as an interdisciplinary product of computer science and life science, has outstanding performance in solving high-level abstract cognitive problems, and thus has become a research hotspot in academia and industry. In order to improve the computational performance of neural networks and to adapt to more complex application problems, the scale of neural networks is expanding, and the amount of computation, data volume, and computing energy consumption are also increasing. Finding high-performance and low-energy neural network computing methods and equipment has become a hot spot for researchers

At present, real-time task analysis using deep neural networks mostly relies on large-scale high-performance processors or general-purpose graphics processors. These devices are high in cost and high in power consumption. When applied to portable smart device applications, there are large circuit scales, high energy consumption, and expensive products. A series of problems, therefore, for applications of high-efficiency real-time processing in embedded devices and small-scale low-cost data centers, the use of specialized neural network processor acceleration rather than software for neural network model calculation becomes a more effective Solution, however, the topology and parameter design of the neural network model will change according to different application scenarios. In addition, the development of the neural network model is very fast, providing a kind of neural network model that can cover various application scenarios and cover various neural network models. The general-purpose high-efficiency neural network processor is very difficult, which brings the high-end application developers to design hardware acceleration solutions for different application requirements.

At present, the existing neural network hardware acceleration technology includes an Application Specific Integrated Circuit (ASIC) chip and a Field Programmable Gate Array (FPGA). Under the same process conditions, the ASIC chip runs at a speed. Fast and low power consumption, but the design process is complex, the filming period is long, the development cost is high, and it can not adapt to the characteristics of rapid update of the neural network model; FPGA has the characteristics of flexible circuit configuration and short development cycle, but the running speed is relatively low, hardware overhead And the power consumption is relatively large. Regardless of which hardware acceleration technology is used, the neural network model and algorithm developers need to master the hardware development technology while understanding the network topology and data flow mode, including processor architecture design, hardware code writing, Simulation verification and layout and other aspects, these technologies are more difficult for high-level application developers who focus on researching neural network models and structural design without hardware design capabilities.

Summary of the invention

In view of the deficiencies of the prior art, the present invention provides an automatic design method and system for a neural network processor, so that high-level developers can efficiently perform neural network technology application development.

In one aspect, the present invention provides an automated design method for a neural network processor, the method comprising:

Step A: acquiring a neural network model topology configuration file and a hardware resource constraint file of the target hardware circuit for the neural network model to be implemented in a hardware circuit manner;

Step B: construct a neural network processor hardware architecture corresponding to the neural network model according to the neural network model topology configuration file and the hardware resource constraint file, and generate a hardware architecture description file;

Step C: Generate a control description file for controlling data scheduling, storage, and calculation of the neural network processor according to the neural network model topology, the hardware resource constraint file, and the hardware architecture description file;

Step D: Generate a hardware circuit description language corresponding to the neural network processor according to the hardware architecture description file and the control description file, so as to implement a hardware circuit of the neural network processor on the target hardware circuit.

In the above method, the step B may include:

Obtaining, according to the neural network model topology configuration file and the hardware resource constraint parameter, a component of a neural model component and a specific hardware structure thereof according to a pre-established cell library, wherein the cell library is reusable by a neural network Various types of unit components, each unit including a hardware description file and a configuration script for describing its hardware structure;

And setting a configuration script of each unit acquired from the unit library according to the description file of the neural network model and the hardware resource constraint parameter to obtain a description file of a hardware structure corresponding to each unit, thereby obtaining the neural network. The hardware architecture description file for the processor.

In the above method, the hardware resource constraint file may include one or more of the following: an operating frequency of the target hardware circuit, a target circuit area overhead, a target circuit power consumption overhead, a supported data precision, and a target circuit memory size.

In the above method, the neural network model topology configuration file may include the number of neural network layers and each layer network size, data bit width, weight bit width, current layer function attribute, current layer input layer number, current layer output layer Number, current layer convolution kernel size, current layer step size, next layer connection property.

In the above method, the step C may further include determining a convolution kernel splitting and data sharing manner by the following steps:

(1) For a given neural network layer, if the convolution kernel size k is consistent with the step value s, the weight sharing mode is adopted, and the convolution kernel performs a convolution operation in the single layer data graph;

(2) If the number of data layers is smaller than the calculation unit width, the convolution kernel k is divided into a plurality of convolution kernels k _s ; if the number of data layers is greater than the calculation unit width, a data sharing manner is adopted;

(3), judging the calculation mode of the next neural network layer, and storing the calculation result of the current layer according to the convolution operation mode of the next neural network layer.

In the above method, the control description file may include an instruction stream for controlling data scheduling, storage, and calculation of the neural network processor, wherein the types of instructions include load/store instructions and operation instructions.

In the above method, the load/store instructions may include data exchange instructions between an external memory and an internal memory of the neural network processor, data for the internal memory, and weight loading to the computing unit. And an instruction for storing the calculation result of the calculation unit into the memory; and the operation instruction includes a convolution operation instruction, a pooling operation instruction, a local corresponding normalization instruction, a clear instruction, and an excitation function operation instruction.

In the above method, the format of the convolutional instruction may include the following fields: an opcode for marking the type of instruction, a number of computation cores for marking the number of computational cores participating in the operation, and a transmission interval for marking each operation of the instruction. The transmission interval; the mode of operation for marking the mode of convolution and cross-layer convolution in the layer, and the destination register for marking the storage location of the calculation result.

In yet another aspect, the present invention provides an automated design method for a neural network processor, comprising:

1) acquiring a neural network model topology configuration file and a hardware resource constraint file, where the hardware resource constraint file includes a target circuit area overhead, a target circuit power consumption overhead, and a target circuit operating frequency;

2) generating a neural network processor hardware architecture according to the neural network model topology configuration file and the hardware resource constraint file, and generating a hardware architecture description file;

3) generating, according to the neural network model topology, the hardware resource constraint file, and the hardware architecture description file, a data control, storage, and calculation manner, and generating a corresponding control description file;

4), according to the hardware architecture description file, the control description file from the constructed neural network reusable cell library to find a cell library that meets the design requirements, generate corresponding control logic, and generate a corresponding hardware circuit description language, Converting the hardware circuit description language into a hardware circuit.

The neural network model topology configuration file may include the number of neural network layers and each layer network size, data bit width, weight bit width, current layer function attribute, current layer input layer number, current layer output layer number, current Layer convolution kernel size, current layer step size, next layer connection property. In one embodiment, the method further includes generating a stream of control instructions while generating a neural network circuit model, the types of instructions including load/store instructions and arithmetic instructions. In an embodiment, the step 3) may include: performing convolution kernel partitioning, data partitioning according to the neural network model topology configuration file, and generating a control state machine; generating a control instruction according to the control state machine flow. In some embodiments, the hardware architecture description file includes input data memory capacity, input memory bit width, weight memory capacity, weight memory bit width, offset memory capacity, offset memory bit width, output data memory capacity, output data Memory bit width, data bit width, calculation unit width, calculation unit depth, data sharing flag bit, and weight sharing flag bit.

In yet another aspect, the present invention also provides an automated design system for a neural network processor, comprising:

Obtaining a data module, configured to acquire a neural network model topology configuration file and a hardware resource constraint file, where the hardware resource constraint file includes a target circuit area overhead, a target circuit power consumption overhead, and a target circuit operating frequency;

Generating a hardware architecture description file module, configured to generate a neural network processor hardware architecture according to the neural network model topology configuration file and the hardware resource constraint file, and generate a hardware architecture description file;

Generating a control description file module, configured to optimize a data scheduling, storage, and calculation manner according to the neural network model topology, the hardware resource constraint file, and the hardware architecture description file, to generate a corresponding control description file;

Generating a hardware circuit module, configured to search, according to the hardware architecture description file, the control description file, a cell library that meets a design requirement from a constructed neural network reusable cell library, and generate a corresponding hardware circuit description language, where The hardware circuit description language is converted into a hardware circuit.

In yet another aspect, the present invention also provides an optimization method based on an automated design method for a neural network processor as described, comprising:

Step (1), for a given neural network layer, if the convolution kernel size k is consistent with the step value s, the weight sharing mode is adopted, and the convolution kernel performs a convolution operation in the single layer data graph;

Step (2), if the number of data layers is smaller than the calculation unit width, the convolution kernel is divided into multiple convolution kernels k _s by a method of convolution kernel division; if the number of data layers is greater than the calculation unit width, data is used. Sharing method

In step (3), the calculation mode of the next neural network layer is determined, and the calculation result of the current layer is stored according to the convolution operation mode of the next neural network layer.

As can be seen from the above scheme, the advantages of the present invention are:

The neural network model is mapped to the hardware description language code for designing the hardware circuit, and the designed hardware circuit structure and data storage mode are automatically optimized according to hardware resource constraints and network characteristics, and the corresponding control instruction stream is generated at the same time, and the neural network is realized. The hardware and software automation of the hardware accelerator is designed to shorten the design cycle of the neural network processor, improve the performance of the neural network processor, and meet the neural network operation requirements of the upper application developers.

DRAWINGS

The embodiments of the present invention are further described below with reference to the accompanying drawings, in which:

Figure 1 shows a schematic diagram of a topology common to neural networks;

Figure 2 shows a schematic block diagram of a neural network convolution operation;

Figure 3 shows a schematic block diagram of a common structure according to a neural network processor;

4 is a schematic diagram of an automated design flow of a neural network processor in accordance with one embodiment of the present invention;

FIG. 5 is a schematic diagram of a compiler workflow according to an embodiment of the present invention; FIG.

6 is a flow chart of a control state machine for performing a convolution operation by a neural network processor in accordance with one embodiment of the present invention;

7 is a schematic diagram of the operation of a convolution kernel in a weight sharing mode according to an embodiment of the present invention;

FIG. 8 is a schematic diagram of a convolution kernel split according to an embodiment of the present invention; FIG.

Figure 9 is a diagram of an instruction format of a load/store instruction;

Fig. 10 is a diagram showing an instruction format of an arithmetic instruction.

detailed description

The present invention will be further described in detail below with reference to the accompanying drawings. It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

The neural network is a mathematical model for modeling human brain structure and behavioral activities. It is usually divided into input layer, hidden layer and output layer. Each layer is composed of multiple neuron nodes. The neuron nodes in this layer are composed. The output value is passed as input to the next level of neuron nodes, connected layer by layer. The neural network itself has bionic characteristics, and its multi-layer abstract iterative process has similar information processing methods as the human brain and other sensory organs. Figure 1 shows a common topology diagram of a neural network. The first layer input value of the neural network multilayer structure is the original image (the "original image" in the present invention refers to the original data to be processed, not only the image obtained by taking a photograph in a narrow sense), typically, for Each layer of the neural network can be calculated by calculating the neuron node values (also referred to herein as data) and its corresponding weight values for the layer to obtain the node values of the next layer. For example, suppose x=x ₁ , x ₂ , x ₃ ,..., x _n represent several neuron nodes of a layer in the neural network, which are connected to the node y of the next layer.

Indicates the weight of the corresponding connection, then the value of y is defined as: y=x×w. Therefore, there are a large number of convolution operations based on multiply and add operations for each layer of the neural network. The convolution operation process in the neural network is generally as shown in Fig. 2: a K*K-sized two-dimensional weight convolution check feature map is scanned, and the inner weight of the corresponding feature elements in the feature map is obtained during the scanning process. And sum all the inner product values to get an output layer feature element. When each convolutional layer has N feature layers, N convolution kernels of K*K size are convoluted with the feature maps in the convolutional layer, and N inner product values are summed to obtain an output layer. Feature element. In addition to the above-described convolution calculation based on vector multiplication and addition, the neural network operation may also include pooling, normalization calculation, and the like. Considering the complexity of neural network operations, hardware acceleration techniques are usually used to construct a dedicated neural network processor to implement neural network computing. At present, common hardware acceleration technologies include ASIC or FPGA. For convenience of description, the following takes FPGA as an example to illustrate. FPGAs are more flexible from a design perspective. For example, Verilog HDL (Hard Description Language), VHDL, or other hardware description language can be used to define internal logic structures to implement custom hardware circuits.

Common neural network processors are based on storage-control-calculation logic structures. The storage structure is configured to store data participating in the calculation, the weight of the neural network, and the operation instruction of the processor; the control structure includes a decoding circuit and a control logic circuit for parsing the operation instruction, generating a control signal to control scheduling of the data in the processor, and Storage and computational processes of neural networks; computational structures are responsible for neural network computational operations. The storage unit may store data transmitted from outside the neural network processor (for example, original feature map data), trained neural network weights, processing results or intermediate results generated in the calculation process, instruction information participating in the calculation, and the like. 3 is a schematic diagram showing a conventional neural network processor system 101 including an input data storage unit 102, a control unit 103, an output data storage unit 104, a weight storage unit 105, and an instruction storage unit 106. , calculation unit 107. The input data storage unit 102 is configured to store data participating in the calculation, the data includes original feature map data and data participating in the intermediate layer calculation; the output data storage unit 104 stores the calculated neuron response value; and the instruction storage unit 106 stores the participating calculations. The instruction information is interpreted as a control flow to schedule neural network calculations; the weight storage unit 105 is configured to store the trained neural network weights. The control unit 103 is connected to the output data storage unit 104, the weight storage unit 105, the instruction storage unit 106, and the calculation unit 107, respectively. The control unit 103 is responsible for instruction decoding, data scheduling, process control, etc., for example, is obtained and stored in the instruction storage unit 106. The instruction in the instruction parses the instruction, schedules the data according to the parsed control signal, and controls the computing unit to perform a correlation operation of the neural network. The calculation unit 107 is configured to perform a corresponding neural network calculation according to a control signal generated by the control unit 103. The computing unit 107 is associated with one or more storage units, which may obtain data from a data storage component in its associated input data storage unit 102 for calculation and may output to the output data storage unit 104 associated therewith. data input. The computing unit 107 performs most of the operations in the neural network algorithm, including vector multiply and add operations, pooling, normalization calculations, and the like.

However, as mentioned in the background section, the topology and parameter design of the neural network model will change according to different application scenarios or application requirements, and the development of the neural network model is fast, which is a neural network model and algorithm. High-level application developers bring great development challenges, not only need to quickly design or adjust related hardware acceleration solutions for different application requirements, but also require high-level developers to understand hardware development such as FPGA while mastering neural network models and algorithms. Technology, so development is very difficult.

In one embodiment of the present invention, an automated design method system or apparatus suitable for a neural network processor is provided, the system or apparatus comprising a hardware generator and a compiler; wherein the hardware generator is based on a neural network model and hardware The resource constraint automatically generates the hardware description language code of the neural network processor for subsequent hardware designers to generate the hardware circuit of the neural network processor through the hardware description language by using the existing hardware circuit design method; and the compiler can be used to generate the neural network processor Control of the circuit structure and flow of data dispatch instructions. In some embodiments, the hardware generator may construct a neural network processor hardware architecture according to a topology of the neural network model, a hardware resource constraint file, and a constructed neural network reusable cell library according to the processor hardware architecture. And the control state machine generated by the compiler generates the hardware description language code.

In some embodiments, to avoid repetitive design and to accommodate hardware implementations of various neural network models, the system may also include a pre-built neural network reusable cell library, which may include various recoverable neural network models. The basic units used include, for example but are not limited to: a neuron unit, an accumulator unit, a pooling unit, a classifier unit, a local response normalization unit, a lookup table unit, an address generation unit, a control unit, and the like. The specific hardware structure of each unit in the cell library is defined by the hardware description file associated with it. The hardware description file for each unit can be described in Verilog HDL or other hardware description language. Preferably, each unit also has a configuration script associated therewith, by which the hardware structure of each unit can be appropriately adjusted; for example, configuring the bit width of the register in the neuron unit, and adding the addition included in the configuration addition tree unit The number of devices, the number of comparators in the configuration pooling unit, and so on.

FIG. 4 illustrates a workflow applicable to a neural network processor automation design system according to an embodiment of the present invention, which may mainly include:

In step S1, the neural network model topology configuration file is read. The neural network model topology configuration file is mainly used to describe a neural network model designed according to specific application requirements, including a network topology of the neural network model and various operational layer definitions. For example, the neural network model topology configuration file may include a number of neural network layers, a network size and structure of each layer, a data bit width, a weight bit width, a current layer function attribute, a current layer input layer number, a current layer output layer number, Current layer convolution kernel size, current layer step size, next layer connection properties, and so on. Each layer of the neural network includes one or more units, and the unit types are usually basic neuron units, convolution units, pool units, normalization units, loop units, and the like. In short, the neural network model description file may include three parts: a basic attribute, a parameter description, and a connection information, wherein the basic attribute may include a layer name, a layer type, a layer structure, and the like; the parameter description may include an output layer number, a convolution kernel size, Step size, etc.; connection information may include connection name, connection direction, connection type, and the like.

Step S2, reading a hardware resource constraint file, the hardware resource constraint file including some parameters describing the available hardware resources of the target hardware circuit of the neural network processor to be implemented, for example, may include implementing the neural network processing The operating frequency of the target hardware circuit, the target circuit area overhead, the target circuit power consumption overhead, the supported data precision, the target circuit memory size, and so on. These hardware resource constraint parameters can be loaded into the system together in a constraint file.

Step S3, the hardware generator of the system constructs a neural network processor hardware architecture according to the neural network model topology configuration file and the hardware resource constraint file, and generates a corresponding hardware architecture description file. The hardware architecture description file may include hardware circuit structure, input data memory capacity, input memory bit width, weight memory capacity, weight memory bit width, offset memory capacity, offset memory bit width, output data memory capacity, output data memory bit. Width, data bit width, calculation unit width, calculation unit depth, data sharing flag, weight sharing flag, and so on.

In step S4, the compiler of the system generates a control of the neural network processor circuit structure and a data dispatch instruction stream, and these processes can be described by controlling the state machine. For example, according to the neural network model topology, hardware resource constraints, and hardware architecture description files, data scheduling, storage, and calculation methods are optimized, and corresponding control description files are generated. FIG. 5 is a flowchart showing a workflow of a compiler for generating a control instruction stream according to a neural network topology, a constructed hardware architecture, and a hardware resource constraint file to perform real-time on a neural network processor according to an embodiment of the present invention. control. Specifically, the method may include: step a1, reading the neural network model topology configuration file network topology structure configuration file, the hardware architecture description file, and the hardware resource constraint file. In step a2, the compiler performs scheduling optimization such as convolution kernel partitioning and data partitioning according to the above file, and generates a control state machine. The control state machine can be used to schedule the operational state of the neural network processor hardware circuitry to be implemented. In step a3, a control instruction stream for the neural network processor is generated based on the control state machine. Figure 6 depicts a partial control state machine flow diagram with a neural network processor performing a convolution operation as an example. For example, the control neural network related unit reads the neural network data and the weight data from the external memory into the internal memory, and then loads the relevant neural network data, the offset data, and the weight data to be subjected to the convolution operation into the calculation unit, and then controls the calculation unit to perform multiplication. Add operation and accumulate operation, and repeat the above loading and calculation operations until the corresponding data is calculated.

With continued reference to FIG. 4, in step S5, the hardware generator indexes the cell library that meets the design requirements from the constructed neural network reusable cell library according to the hardware architecture description file and the control description file, generates corresponding control logic, and generates and generates The hardware circuit description language of the neural network processor corresponding to the neural network model. Then in step S6, the generated hardware circuit description language can be converted into a specific hardware circuit implementing the neural network processor by an existing hardware design method.

In some embodiments, due to the limitation of resource constraints of the target hardware circuit, the neural network model is often unable to fully expand according to its model description when mapped to a hardware circuit, whereby the compiler can analyze the computational throughput of the neural network processor and On-chip memory size, which divides neural network feature data and weight data into appropriate data block storage and access. The computational data of the neural network includes input feature data and trained weight data. Through good data segmentation and storage layout, the internal data bandwidth of the processor can be reduced and the storage space utilization efficiency can be improved. The optimization method of the compiler based on convolution kernel partitioning and data sharing is described below with reference to Figs. 7 and 8, and mainly includes the following steps:

Step (1), for a given neural network layer, if the convolution kernel size k and the step value s are the same, the weight sharing mode is adopted, and the convolution kernel performs a convolution operation in the single layer data graph, as shown in FIG. ;

Step (2), if the number of data layers is smaller than the calculation unit width, the convolution kernel segmentation method is used to divide the large convolution kernel k into small convolution kernels k _s , as shown in FIG. 8; if the data layer number is greater than Calculate the unit width and use data sharing.

In some embodiments, the instruction streams generated by the compiler of the system for control and data scheduling of neural network processor circuit structures may be collectively referred to as instruction streams. These instruction streams are used to control the operation and operation of the designed neural network processor. Instruction types can include types such as load/store instructions and arithmetic instructions. For example, where the load/store instruction can include the following instructions:

* External and internal memory data transfer instructions for data exchange between external memory and internal memory, including data participating in neural network calculations, trained weights and offset data, etc.;

* input data memory and computing unit transfer instructions for loading data in the on-chip memory into the computing unit in a compiled optimized scheduling manner;

* a weight memory and a calculation unit transfer instruction for loading the weight data in the on-chip memory into the calculation unit in a compiling-optimized scheduling manner;

* A calculation unit and an output data memory transfer instruction for storing the calculation result of the calculation unit in the memory.

Taking the input data memory and the calculation unit transfer instruction as an example, the instruction format of the load/store instruction is introduced. The instruction format is as shown in FIG. 9, wherein the operation code is used to mark the instruction type; the transmission interval is used to mark the emission of each operation of the instruction. Interval; data first address is used to mark the data first address; operation mode is used to describe the working state of the circuit, including large convolution kernel operation, small convolution kernel operation, pooling operation, full connection operation, etc.; convolution kernel size The convolution kernel value is marked; the output image size is used to mark the output image size; the number of input layers is used to mark the number of input layers; the number of output layers is used to mark the number of output layers; and the clear signal is used to clear the data values.

For another example, the operation instructions may include a convolution operation instruction for controlling a convolution operation; a pooled operation instruction for controlling a pooling operation; a local corresponding normalization instruction for controlling a local response normalization operation; A clear instruction for clearing data loaded in the calculation unit; an excitation function operation instruction for controlling the operation of the excitation function and configuring the function mode, and the like. Taking the convolution instruction as an example, the instruction format of the operation instruction is introduced. The instruction format is shown in Figure 10, where the operation code is used to mark the instruction type; the calculation core number is used to mark the number of calculation cores participating in the operation; the transmission interval is used to mark the instruction The transmission interval of each operation; the operation mode is used to include modes such as intra-layer convolution and cross-layer convolution; the target register is used to mark the storage location of the calculation result, including the output data memory, the excitation function register, and the lookup table register.

In one embodiment, the compiler may, for example, take the following steps to generate the above described instruction stream:

Step b1, reading the name of the neural network layer;

Step b2, reading in the neural network layer type;

Step b3, parsing neural network layer parameters;

Step b4, determining a hardware circuit structure and parameters;

Step b5, performing scheduling optimization based on the convolution kernel segmentation and data sharing optimization manner described above in connection with FIGS. 7 and 8;

In step b6, the instruction parameters are determined and the control flow instruction is generated according to the neural network working mode and the scheduling mode. The command parameters may include, for example, a neural network layer serial number, an input layer number, an output layer number, a data size per layer, a data width, a weight width, a convolution kernel size, and the like.

In yet another embodiment, an automated design method for a neural network processor is provided, the method comprising: step A, acquiring a neural network model topology configuration file for a neural network model to be implemented in a hardware circuit manner, and a hardware resource constraint file of the target hardware circuit; step B, constructing a hardware structure of the neural network processor corresponding to the neural network model according to the neural network model topology configuration file and the hardware resource constraint file, and generating a hardware architecture description a file; step C, generating a control description file for controlling data scheduling, storage, and calculation of the neural network processor according to the neural network model topology, the hardware resource constraint file, and the hardware architecture description file Step D, generating a hardware circuit description language corresponding to the neural network processor according to the hardware architecture description file and the control description file, so as to implement the hardware circuit of the neural network processor on the target hardware circuit. The hardware resource constraint file may include one or more of the following: an operating frequency of the target hardware circuit, a target circuit area overhead, a target circuit power consumption overhead, a supported data precision, and a target circuit memory size. The control description file may include an instruction stream for controlling data scheduling, storage, and calculation of the neural network processor, wherein the types of instructions include load/store instructions and operation instructions.

In some embodiments, step B may include: acquiring the neural model components and their associated specific hardware structures according to the neural network model topology configuration file and the hardware resource constraint parameters and based on the pre-established cell library, wherein The unit library is composed of various types of units reusable in a neural network, each unit includes a hardware description file and a configuration script for describing a hardware structure thereof; and a description file according to the neural network model and the description The hardware resource constraint parameter sets a configuration script of each unit acquired from the unit library to obtain a description file of a hardware structure corresponding to each unit, thereby obtaining a hardware architecture description file of the neural network processor.

In some embodiments, the step C may further comprise determining a convolution kernel partitioning and data sharing manner by the following steps:

In still another embodiment, an automated design method for a neural network processor is provided, including: step 1), acquiring a neural network model topology configuration file and a hardware resource constraint file, wherein the hardware resource constraint file includes Target circuit area overhead, target circuit power consumption overhead, and target circuit operating frequency; step 2), generating a neural network processor hardware architecture according to the neural network model topology configuration file and the hardware resource constraint file, and generating a hardware architecture description a file; step 3), optimizing a data scheduling, storing, and calculating manner according to the neural network model topology, the hardware resource constraint file, and the hardware architecture description file, and generating a corresponding control description file; step 4) The hardware architecture description file, the control description file, the cell library that meets the design requirements are searched from the constructed neural network reusable cell library, the corresponding control logic is generated, and a corresponding hardware circuit description language is generated, and the hardware circuit is generated. The description language is converted to a hardware circuit.

In some embodiments, the neural network model topology configuration file may include a number of neural network layers and each layer network size, a data bit width, a weight bit width, a current layer function attribute, a current layer input layer number, and a current layer output. The number of layers, the current layer convolution kernel size, the current layer step size, and the next layer of connection properties. In some embodiments, the hardware architecture description file may include input data memory capacity, input memory bit width, weight memory capacity, weight memory bit width, offset memory capacity, offset memory bit width, output data memory capacity, output Data memory bit width, data bit width, calculation unit width, calculation unit depth, data sharing flag bit, and weight sharing flag bit.

In some embodiments, the method can also include generating a stream of control instructions while generating a neural network circuit model, the types of instructions including load/store instructions and arithmetic instructions. In some embodiments, step 3) may comprise: convolution kernel partitioning, data partitioning according to the neural network model topology configuration file, and generating a control state machine; generating a control instruction stream according to the control state machine.

In some embodiments, an automated design apparatus for a neural network processor is also provided, comprising:

In some embodiments, the neural network reusable unit library includes: a neuron unit, an accumulator unit, a pooling unit, a classifier unit, a local response normalization unit, a lookup table unit, an address generation unit, and a control unit . In some embodiments, the generating the control description file includes: performing convolution kernel partitioning, data partitioning according to the neural network model topology configuration file, and generating a control state machine; generating a control instruction according to the control state machine flow.

It can be seen from the above embodiment that the automatic design system applicable to the neural network processor according to the present invention can map the neural network model to the hardware description language of the neural network dedicated processor, and optimize the data calculation and scheduling according to the processor structure. The method and the corresponding control flow instruction are generated, which realizes the automatic design of the neural network processor, reduces the design cycle of the neural network processor, and adapts to the neural network technology network model update fast, the operation speed requirement block, and the energy efficiency requirement. Application characteristics.

References in the specification to "individual embodiments," "some embodiments," "one embodiment," or "an embodiment" or "an" In at least one embodiment. Thus, appearances of the phrases "in the various embodiments", "in some embodiments", "in one embodiment", or "in an embodiment" example. Furthermore, the particular features, structures, or properties may be combined in any suitable manner in one or more embodiments. Thus, the particular features, structures, or properties shown or described in connection with one embodiment may be combined, in whole or in part, with the features, structures, or properties of one or more other embodiments without limitation, as long as the combination is not Logical or not working. In addition, the various elements in the drawings of the present application are only for the purpose of illustration and illustration.

Having thus described several aspects of at least one embodiment of the present invention, it is understood that various changes, modifications and improvements can be readily made by those skilled in the art. Such changes, modifications, and improvements are intended to be within the spirit and scope of the invention.

Claims

An automated design method for a neural network processor, the method comprising:

Step A: acquiring a neural network model topology configuration file and a hardware resource constraint file of the target hardware circuit for the neural network model to be implemented in a hardware circuit manner;

Step B: construct a neural network processor hardware architecture corresponding to the neural network model according to the neural network model topology configuration file and the hardware resource constraint file, and generate a hardware architecture description file;

Step C: Generate a control description file for controlling data scheduling, storage, and calculation of the neural network processor according to the neural network model topology, the hardware resource constraint file, and the hardware architecture description file;

Step D: Generate a hardware circuit description language corresponding to the neural network processor according to the hardware architecture description file and the control description file, so as to implement a hardware circuit of the neural network processor on the target hardware circuit.
The method of claim 1 wherein said step B comprises:

Obtaining, according to the neural network model topology configuration file and the hardware resource constraint parameter, a component of a neural model component and a specific hardware structure thereof according to a pre-established cell library, wherein the cell library is reusable by a neural network Various types of unit components, each unit including a hardware description file and a configuration script for describing its hardware structure;

And setting a configuration script of each unit acquired from the unit library according to the description file of the neural network model and the hardware resource constraint parameter to obtain a description file of a hardware structure corresponding to each unit, thereby obtaining the neural network. The hardware architecture description file for the processor.
The method of claim 1, wherein the hardware resource constraint file comprises one or more of the following: operating frequency of the target hardware circuit, target circuit area overhead, target circuit power consumption overhead, supported data accuracy, target Circuit memory size.
The method according to claim 1, wherein the neural network model topology configuration file comprises a number of neural network layers and a network size of each layer, a data bit width, a weight bit width, a current layer function attribute, a current layer input layer number, The current layer output layer number, the current layer convolution kernel size, the current layer step size, and the next layer connection property.
The method of claim 1, the step C further comprising determining, by the following steps, a convolution kernel partitioning and data sharing manner of the neural network processor:

(1) For a given neural network layer, if the convolution kernel size k is consistent with the step value s, the weight sharing mode is adopted, and the convolution kernel performs a convolution operation in the single layer data graph;

(2) If the number of data layers is smaller than the calculation unit width, the convolution kernel k is divided into a plurality of convolution kernels k s ; if the number of data layers is greater than the calculation unit width, a data sharing manner is adopted;

(3), judging the calculation mode of the next neural network layer, and storing the calculation result of the current layer according to the convolution operation mode of the next neural network layer.
The method of claim 1, the control description file comprising an instruction stream for controlling data scheduling, storage, and calculation of the neural network processor, wherein the type of the instruction includes a load/store instruction and an operation instruction .
The method of claim 6 wherein said load/store instructions include data exchange instructions between an external memory and an internal memory of a neural network processor, data for said internal memory, and weight loading to An instruction of the calculation unit, an instruction for storing the calculation result of the calculation unit into the memory; and the operation instruction includes a convolution operation instruction, a pooled operation instruction, a local corresponding normalization instruction, a clear instruction, an excitation function operation instruction .
The method of claim 7, the format of the convolution instruction comprising the following fields: an opcode for marking the type of instruction, a number of computation cores for marking the number of computational cores participating in the operation, for marking the instruction each time The transmission interval of the transmission interval; the operation mode for marking the mode of convolution and cross-layer convolution in the layer, and the destination register for marking the storage location of the calculation result.
An automated design method for a neural network processor, comprising:

Step 1) acquiring a neural network model topology configuration file and a hardware resource constraint file, where the hardware resource constraint file includes a target circuit area overhead, a target circuit power consumption overhead, and a target circuit operating frequency;

Step 2) generating a neural network processor hardware architecture according to the neural network model topology configuration file and the hardware resource constraint file, and generating a hardware architecture description file;

Step 3) Optimize data scheduling, storage, and calculation manner according to the neural network model topology, the hardware resource constraint file, and the hardware architecture description file, and generate a corresponding control description file;

Step 4) Searching, according to the hardware architecture description file, the control description file, a cell library that meets the design requirements from the constructed neural network reusable cell library, generating corresponding control logic, and generating a corresponding hardware circuit description language, In order to convert the hardware circuit description language into a hardware circuit.
The method of claim 9, wherein the hardware architecture description file comprises an input data memory capacity, an input memory bit width, a weight memory capacity, a weight memory bit width, an offset memory capacity, an offset memory bit width, Output data memory capacity, output data memory bit width, data bit width, calculation unit width, calculation unit depth, data sharing flag bits, and weight sharing flag bits.
An automated design system for a neural network processor, comprising:

Obtaining a data module, configured to acquire a neural network model topology configuration file and a hardware resource constraint file, where the hardware resource constraint file includes a target circuit area overhead, a target circuit power consumption overhead, and a target circuit operating frequency;

Generating a hardware architecture description file module, configured to generate a neural network processor hardware architecture according to the neural network model topology configuration file and the hardware resource constraint file, and generate a hardware architecture description file;

Generating a control description file module, configured to optimize a data scheduling, storage, and calculation manner according to the neural network model topology, the hardware resource constraint file, and the hardware architecture description file, to generate a corresponding control description file;

Generating a hardware circuit module, configured to search, according to the hardware architecture description file, the control description file, a cell library that meets a design requirement from a constructed neural network reusable cell library, and generate a corresponding hardware circuit description language, so as to The hardware circuit description language is converted into a hardware circuit.