CN111738423A - Compiling method, device, storage medium and electronic device for neural network model - Google Patents

Abstract

The embodiment of the invention provides a compiling method and device of a neural network model, a storage medium and electronic equipment, and relates to the field of deep learning. The method comprises the following steps: acquiring an original characteristic diagram parameter of each convolution layer in the neural network model; splitting the original characteristic diagram parameters according to the input-output parameter relational expression and the memory capacity to obtain a feasible characteristic diagram parameter set of each convolution layer; determining a corresponding target characteristic map parameter with the highest data transfer efficiency for each convolution layer from the feasible characteristic map parameter set of each convolution layer; and generating an executable file for the neural network model according to the target characteristic diagram parameters corresponding to each convolution layer. By finding a corresponding parameter with the highest data carrying efficiency for each convolution layer, the overall data reuse rate of the neural network model is improved, the operation amount during operation is reduced, the operation amount of an executable file corresponding to the neural network model during operation can be reduced, and the operation efficiency is improved.

Description

Compiling method, device, storage medium and electronic device for neural network model

technical field

The present application relates to the field of deep learning, and in particular, to a method, apparatus, storage medium and electronic device for compiling a neural network model.

Background technique

Deep learning can make machines imitate human activities such as audio-visual and thinking, and solve many complex pattern recognition problems. Progress.

How to construct a neural network model and compile the constructed neural network model is the core part of deep learning. At present, when compiling the constructed neural network model into an executable file, due to the large amount of parameters in the neural network model, the compiled executable file has a large amount of computation and low operating efficiency at runtime. Therefore, the compilation process of the neural network model needs to be optimized in order to improve the execution efficiency of the compiled executable file. However, the current optimization method is relatively simple, and the running efficiency of the compiled executable file is still very low.

SUMMARY OF THE INVENTION

The purpose of the present application includes, for example, to provide a method, apparatus, storage medium and electronic device for compiling a neural network model, which can compile a neural network model into an executable file and reduce the operation of the executable file at runtime to improve the running efficiency of the executable file.

The embodiments of the present application can be implemented as follows:

In a first aspect, the embodiments provide a method for compiling a neural network model, the method comprising: acquiring the original feature map parameters of each convolutional layer in the neural network model; The feature map parameters are split to obtain a feasible feature map parameter set of each convolution layer; a corresponding target feature map parameter is determined for each convolution layer from the feasible feature map parameter set of each convolution layer; The data handling efficiency corresponding to the target feature map parameters is the highest; an executable file is generated for the neural network model according to the target feature map parameters corresponding to each convolution layer.

In an optional implementation manner, the original feature map parameters include original output feature map parameters; the original feature map parameters are split according to the relationship between input and output parameters and memory capacity to obtain the parameters of each convolutional layer. The step of the feasible feature map parameter set includes: splitting the original output feature map parameters according to the relationship between input and output parameters and memory capacity to obtain a feasible output feature map parameter set for each convolution layer; according to each convolution layer The feasible output feature map parameter set and the input and output parameter relational expression, determine the feasible input feature map parameter set of each convolution layer; according to the feasible output feature map parameter set of each convolution layer and the feasible input feature Graph parameter set, which determines the set of feasible feature map parameters for each convolutional layer.

In an optional embodiment, the step of determining a corresponding target feature map parameter for each convolutional layer from the feasible feature map parameter set of each convolutional layer includes: according to the feasible feature map parameter of each convolutional layer The value of each feasible feature map parameter in the feature map parameter set determines the data repetition loading amount corresponding to each feasible feature map parameter; The feasible feature map parameters are used as the target feature map parameters corresponding to each convolutional layer.

In an optional embodiment, the neural network model includes a plurality of convolution layers, and a corresponding target feature map parameter is determined for each convolution layer from a set of feasible feature map parameters of each convolution layer. The steps include: according to the value of each feasible feature map parameter in the feasible feature map parameter set of the target convolution layer, determine the data repetition loading amount corresponding to each of the feasible feature map parameters; according to the preset number of processing cores and the repeated loading of the data, an optimal feature map parameter is determined from the feasible feature map parameter set of the target convolutional layer; the target convolutional layer is any one of the multiple convolutional layers, so The optimal feature map parameter is the feasible feature map parameter with the highest data handling efficiency in the feasible feature map parameter set of the target convolution layer; the optimal feature map parameter is used as the target feature map corresponding to the target convolution layer. parameter.

In an optional implementation manner, the step of generating an executable file for the neural network model according to the target feature map parameters corresponding to each convolutional layer includes: generating a three-dimensional Direct memory access to DMA data transfer instructions, and generate an executable file of the neural network model according to the three-dimensional DMA data transfer instructions.

In an optional implementation manner, the method further includes: executing the executable file to implement the data processing function of the neural network model.

In a second aspect, the embodiments provide an apparatus for compiling a neural network model, including: an acquisition module for acquiring the original feature map parameters of each convolutional layer in the neural network model; a splitting module for according to the relationship between input and output parameters The original feature map parameters are split according to the formula and memory capacity to obtain a feasible feature map parameter set of each convolution layer; the splitting module is also used to obtain a feasible feature map parameter set of each convolution layer. A corresponding target feature map parameter is determined for each convolutional layer; the data handling efficiency corresponding to the target feature map parameter is the highest; the generation module is used to generate the neural network according to the target feature map parameter corresponding to each convolutional layer. The network model generates an executable.

In an optional implementation manner, the original feature map parameters include original output feature map parameters; the splitting module is configured to split the original output feature map parameters according to the input and output parameter relational expressions and memory capacity, to obtain A feasible output feature map parameter set of each convolutional layer; the splitting module is further configured to determine each convolutional layer according to the feasible output feature map parameter set of each convolutional layer and the input and output parameter relationship The set of feasible input feature map parameters; the splitting module is also used to determine the feasible feature map parameters of each convolution layer according to the set of feasible output feature map parameters of each convolution layer and the set of feasible input feature map parameters gather.

In an optional embodiment, the splitting module is configured to determine the data corresponding to each feasible feature map parameter according to the value of each feasible feature map parameter in the feasible feature map parameter set of each convolutional layer Repeated loading; the splitting module is further configured to use the feasible feature map parameter with the smallest data repeated loading in the feasible feature map parameter set of each convolutional layer as the target feature map parameter corresponding to each convolutional layer.

In an optional embodiment, the neural network model includes a plurality of convolutional layers, and the splitting module is configured to determine the value of each feasible feature map parameter in the feasible feature map parameter set of the target convolutional layer to determine the data repetition loading amount corresponding to each of the feasible feature map parameters; the splitting module is further configured to, according to the preset number of processing cores and the data repetition loading amount, from the feasible feature map parameter set of the target convolution layer An optimal feature map parameter is determined from ; the target convolutional layer is any one of the multiple convolutional layers, and the optimal feature map parameter is the set of feasible feature map parameters of the target convolutional layer The feasible feature map parameter with the highest data handling efficiency in the data processing module; the splitting module is further configured to use the optimal feature map parameter as the target feature map parameter corresponding to the target convolutional layer.

In an optional embodiment, the generation module is configured to generate a 3D direct memory access DMA data handling instruction according to the target feature map parameter corresponding to each convolutional layer, and generate the neural network according to the 3D DMA data handling instruction The executable for the network model.

In an optional implementation manner, the apparatus further includes a running module configured to execute the executable file to implement the data processing function of the neural network model.

In a third aspect, an embodiment provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the method for compiling a neural network model according to any one of the foregoing embodiments.

In a fourth aspect, an embodiment provides an electronic device, including a processor and a memory, where the memory stores machine-readable instructions, and the processor is configured to execute the machine-readable instructions to implement any one of the foregoing embodiments The compiling method of the neural network model described in item.

The beneficial effects of the embodiments of the present application include, for example, through the method for compiling a neural network model provided by the present application, the original feature map parameters of each convolutional layer in the neural network model can be split, and the parameters of each convolutional layer can be found for each convolutional layer. A corresponding target feature map parameter with the highest data handling efficiency, thereby improving the overall data reuse rate of the neural network model, reducing the amount of computation of the executable file corresponding to the neural network model at runtime, and improving the computational efficiency of the executable file. That is, the present application can compile the neural network model into an executable file, reduce the amount of computation of the executable file when it is running, and improve the running efficiency of the executable file.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present application more clearly, the following drawings will briefly introduce the drawings that need to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore do not It should be regarded as a limitation of the scope, and for those of ordinary skill in the art, other related drawings can also be obtained according to these drawings without any creative effort.

FIG. 1 is a structural block diagram of an electronic device provided by an embodiment of the application;

2 is a flowchart of a method for compiling a neural network model provided by an embodiment of the present application;

3 is a flowchart of S110 in the method for compiling a neural network model provided by an embodiment of the present application;

4 is a flowchart of S120 in the method for compiling a neural network model provided by an embodiment of the present application;

5 is another flowchart of S120 in the method for compiling a neural network model provided by an embodiment of the present application;

6 is another flowchart of a method for compiling a neural network model provided by an embodiment of the present application;

FIG. 7 is a functional block diagram of an apparatus for compiling a neural network model provided by an embodiment of the present application.

Icon: 100-electronic equipment; 110-memory; 120-processor; 130-bus; 140-communication interface; 200-compiling device for neural network model; 210-acquiring module; 220-splitting module; 230-generating module; 240 - Run the module.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments It is a part of the embodiments of the present application, but not all of the embodiments. The components of the embodiments of the present application generally described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations.

Thus, the following detailed description of the embodiments of the application provided in the accompanying drawings is not intended to limit the scope of the application as claimed, but is merely representative of selected embodiments of the application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

It should be noted that like numerals and letters refer to like items in the following figures, so once an item is defined in one figure, it does not require further definition and explanation in subsequent figures.

It should be noted that the features in the embodiments of the present application may be combined with each other under the condition of no conflict.

In the process of realizing the technical solutions of the embodiments of the present application, the inventors of the present application found that:

At present, the constructed neural network model is usually compiled into an executable file by a compiler. When compiling the constructed neural network model into an executable file, due to the large amount of parameters of the convolutional layer in the neural network model, if the Compile directly after optimization, which will result in a large amount of computation and low operation efficiency of the compiled executable file at runtime. Therefore, the compiler often needs to split the feature map (split one operation in the convolution layer into multiple sub-operations, and these multiple sub-operations are independently calculated by the processor), and reuse the weights or feature maps ( Data multiplexing method), and then realize the optimization of convolution operation.

However, the current feature map splitting method is usually only implemented at the hardware level, and the hardware splitting and data multiplexing methods are fixed. For a neural network model including multiple convolutional layers, the prior art can only reuse the weights of the multiple convolutional layers of the neural network model, or only reuse the multiple convolutional layers of the neural network model. Perform feature map reuse. However, for the convolution operations of different convolutional layers, different feature map splitting methods and different data multiplexing methods have different influences on the computational complexity of the convolutional layers. When the amount of feature map data is large and the amount of weight data is small, the method of multiplexing feature maps and loading weights repeatedly can reduce the overall data load and the convolution operation of the convolution layer; when the feature map is small and the weight data is large When the value is larger, the method of reusing weights and loading feature maps repeatedly reduces the overall data loading and reduces the convolution operation of the convolution layer.

In a common neural network model, the feature map will gradually become smaller with the execution of the convolutional layer. Therefore, both cases will exist. If only weights are applied to multiple convolutional layers of this neural network model The multiplexing of multiple convolutional layers of this neural network model, or only the multiplexing of feature maps for multiple convolutional layers of this neural network model, cannot minimize the overall data loading of multiple convolutional layers in the neural network model. That is to say, the current optimization method is relatively simple, and the running efficiency of the compiled executable file is still very low.

Therefore, in order to improve the above-mentioned defects, the embodiments of the present application propose a method, apparatus, storage medium and electronic device for compiling a neural network model, which can compile the neural network model into an executable file and reduce the runtime of the executable file. The amount of computation to improve the running efficiency of the executable file. It should be noted that the defects existing in the above solutions in the prior art are the results obtained by the inventor after practice and careful research. Therefore, the discovery process of the above problems and the following examples of the present application are aimed at the above problems. The proposed solutions should all be contributions made by the inventor to this application during the process of this application.

Please refer to FIG. 1 , which is a structural block diagram of an electronic device 100 provided by an embodiment of the present application. The electronic device 100 may include a memory 110 , a processor 120 , a bus 130 and a communication interface 140 , and the memory 110 , the processor 120 and the communication interface 140 are directly or indirectly electrically connected to each other to realize data transmission or interaction. For example, these elements may be electrically connected to each other through one or more buses 130 or signal lines. The processor 120 may process information and/or data related to the compilation of the neural network model to perform one or more functions described herein. For example, the processor 120 may acquire the original feature map parameters of each convolutional layer in the neural network model, and compile the neural network model according to the above data, thereby implementing the method for compiling the neural network model provided by the present application.

Wherein, the memory 110 may be, but not limited to, random access memory (Random Access Memory, RAM), read only memory (Read Only Memory, ROM), programmable read only memory (Programmable Read-Only Memory, PROM), erasable memory Read-only memory (Erasable Programmable Read-Only Memory, EPROM), Electric Erasable Programmable Read-Only Memory (Electric Erasable Programmable Read-Only Memory, EEPROM), etc.

The processor 120 may be an integrated circuit chip with signal processing capability. The processor 120 may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; it may also be a digital signal processor (Digital Signal Processing, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

It can be understood that the structure shown in FIG. 1 is only for illustration, and the electronic device 100 may further include more or less components than those shown in FIG. 1 , or have different configurations from those shown in FIG. 1 . Each component shown in FIG. 1 may be implemented in hardware, software, or a combination thereof. For example, the above-mentioned electronic device 100 may be a neural network accelerator, server, computer, mobile phone, tablet, cloud platform, etc. Therefore, the present application does not limit the specific type of the electronic device 100 .

Hereinafter, for ease of understanding, the following embodiments of the present application will take the electronic device 100 shown in FIG. 1 as an example, and in conjunction with the accompanying drawings, to specifically describe the method for compiling the neural network model provided by the embodiments of the present application. Please refer to FIG. 2. FIG. 2 shows a flowchart of a method for compiling a neural network model provided by an embodiment of the present application. The method for compiling the neural network model can be applied to the electronic device 100 described above, and the method for compiling the neural network model can include the following steps:

S100, obtain the original feature map parameters of each convolutional layer in the neural network model.

In some possible embodiments, the electronic device 100 may obtain the pre-established neural network model from the storage medium of other devices (for example, a background server, a cloud server, etc.), or may also obtain it from its own storage medium The pre-established neural network model, therefore, the present application does not limit the acquisition method of the neural network model.

After acquiring the neural network model, the electronic device 100 can calculate each convolution in the neural network model according to the global input feature map size of the neural network model and operator calculation parameters (eg, stride parameter, kernel size parameter, etc.). The original feature map parameters of the layer (e.g. input feature map parameters and output feature map parameters of convolutional layers).

In addition, the original feature map parameters of each convolutional layer in the neural network model may also be pre-stored in the storage medium of the electronic device 100. When “obtaining the original feature map parameters of each convolutional layer in the neural network model” , it is only necessary to obtain the original feature map parameters of each convolutional layer in the pre-stored neural network model from the storage medium. Therefore, the present application provides information on how to obtain the original feature map parameters of each convolutional layer in the neural network model. method is not limited.

S110: Split the original feature map parameters according to the relationship between the input and output parameters and the memory capacity to obtain a feasible feature map parameter set for each convolutional layer.

In some possible embodiments, when the neural network model is compiled into an executable file, the input, output, and weight sub-blocks of the executable file all need to be loaded into SRAM (Static Random-Access Memory, static random-access memory). ) space, so the above-mentioned memory capacity may be the storage capacity of the SRAM of the electronic device 100 . Of course, in some other possible embodiments, the above-mentioned memory capacity may also be a preset capacity or a partial capacity in the SRAM storage capacity, which is not limited in this application.

According to the basic principle of convolution calculation, there is a fixed correspondence between the input feature map parameters and the output feature map parameters of each convolutional layer of the neural network model. Therefore, the above-mentioned input and output parameter relationship can be related to the fixed relationship. The corresponding relationship is the same, or in other words, the fixed corresponding relationship may be the above-mentioned input-output parameter relationship. Furthermore, after obtaining the original feature map parameters of each convolutional layer in the neural network model, the electronic device 100 can split the original feature map parameters according to the input-output parameter relationship and memory capacity, and obtain each convolutional layer. A set of feasible feature map parameters for .

Wherein, the above-mentioned original feature map parameters may include original input feature map parameters and original output feature map parameters. When "splitting the original feature map parameters according to the relationship between input and output parameters and memory capacity to obtain a feasible feature map parameter set for each convolutional layer", the original feature map parameters can be split based on the original input feature map parameters , to obtain the feasible feature map parameter set of each convolution layer; the original feature map parameters can also be split based on the original output feature map parameters to obtain the feasible feature map parameter set of each convolution layer; it can also be based on the original input at the same time. The feature map parameters and the original output feature map parameters are used to split the original feature map parameters to obtain a feasible feature map parameter set for each convolutional layer. Therefore, the present application does not limit the specific manner of splitting.

In order to further improve the operation efficiency of the method provided by the present application, in some possible embodiments, the original feature map parameters in the present application may include the original output feature map parameters. The feature map parameters are split to obtain a feasible feature map parameter set for each convolutional layer", on the basis of Figure 2, please refer to Figure 3, S110 may include:

S110A, split the original output feature map parameters according to the relationship between the input and output parameters and the memory capacity to obtain a set of feasible output feature map parameters for each convolutional layer.

In some possible embodiments, the original input feature map parameters included in the original feature map parameters may include: channel dimension parameters inf_c, height dimension parameters inf_h, and width dimension parameters inf_w of the input feature map; The output feature map parameters may include: channel dimension parameter ouf_c, height dimension parameter ouf_h, and width dimension parameter ouf_w of the output feature map, and the original feature map parameters may also include weight parameters: weight channel parameter wt_c, weight dimension parameter wt_n, weight height parameter wt_h and the weight width parameter wt_w.

In the basic principle of convolution calculation, the channel dimension parameter ouf_c of the output feature map and the weight dimension parameter wt_n of the weight parameter have a corresponding fixed relationship, and the height dimension parameter ouf_h and width dimension parameter ouf_w of the output feature map are related to the input feature map. The height dimension parameter inf_h and the width dimension parameter inf_w have corresponding fixed relationships respectively. According to these fixed relations, the following formula 1 can be obtained:

Among them, pad is the number of zero-padded convolution operations, s is the convolution step parameter, pad and s are parameter values parsed according to the neural network model, that is to say, pad and s can be considered as preset parameter values ;

In addition, since the input, output, and weight sub-blocks of the compiled executable file need to be loaded into the SRAM space for storage, the maximum size of each sub-block must be smaller than the maximum capacity of the SRAM, assuming the accelerator's input feature map, weight, output The SRAM capacity sizes of the feature map are inf_L, wt_L, and ouf_L, respectively, and the above dimensions must satisfy the following formula 2:

In particular, when the input feature map, weight, and output feature map share the accelerator SRAM capacity, assuming that the accelerator SRAM capacity is L, the above formula 2 can be:

inf_c*inf_h*inf_w+ouf_c*ouf_h*ouf_w+wt_n*wt_c*wt_h*wt_w<L;

For the convolution calculation, the width and height of the convolution input feature map must be larger than the weight width and height, and the output feature map sub-block size must be smaller than the original size. Therefore, assuming that the original size of the output feature map is s_ouf_c*s_ouf_h*s_ouf_w (that is, the original size of the output feature map is s_ouf_c, s_ouf_h, s_ouf_w respectively), the following formula 3 can be obtained:

Then, by combining the above formula 1, formula 2 and formula 3, the relationship between the input and output parameters can be obtained, and the relationship between the input and output parameters is as follows:

In order to ensure the accumulation integrity of calculating an output point, avoid splitting the accumulation process of calculating an output point into several calculations, and increase the intermediate data cache, the unknown variables inf_c, wt_h, and wt_w in the above formula can all use the original values (ie inf_c, wt_h, wt_w are known quantities), it can be seen that in the above formula, only ouf_c, ouf_h, ouf_w are unknown quantities, and their upper and lower limits are fixed. According to the above input and output parameter relationship, the size of the memory capacity L, and by setting ouf_c, ouf_h, and ouf_w as unknowns, and traversing the possible values of these unknowns, various split size combinations of the output feature map can be obtained (ie , to get a feasible feature map parameter set for each convolutional layer).

It should be understood that S110A can be understood as the first step of "splitting the original feature map parameters based on the original output feature map parameters to obtain a feasible feature map parameter set for each convolutional layer", since the original output feature map parameters are based on the original output feature map parameters. The splitting of feature map parameters can avoid splitting the accumulation process of calculating one output point into several calculations, and increase the intermediate data cache (that is, SS110A can effectively avoid caching intermediate results), thereby improving the operation efficiency of the method provided by this application. .

S110B: Determine a feasible input feature map parameter set of each convolutional layer according to the feasible output feature map parameter set of each convolutional layer and the input-output parameter relational expression.

After obtaining the feasible output feature map parameter set of each convolution layer, according to the feasible output feature map parameter set of each convolution layer and combining the above formula 1, the corresponding feasible input under each split size combination can be calculated Feature map parameters, feasible weight parameters (that is, to determine the set of feasible input feature map parameters for each convolutional layer).

S110C: Determine a feasible feature map parameter set of each convolution layer according to the feasible output feature map parameter set and the feasible input feature map parameter set of each convolution layer.

After determining the feasible output feature map parameter set and the feasible input feature map parameter set of each convolutional layer, the feasible output feature map parameter set and the feasible input feature map parameter set of each convolutional layer are respectively merged, and then Determine the set of feasible feature map parameters for each convolutional layer.

Referring to FIG. 2 again, in S120, a corresponding target feature map parameter is determined for each convolutional layer from the feasible feature map parameter set of each convolutional layer; the data handling efficiency corresponding to the target feature map parameter is the highest.

In this embodiment, after S100 to S110 are performed, each convolutional layer corresponds to a feasible feature map parameter set, and each feasible feature map parameter set may include multiple feasible feature map parameters. In order to reduce the amount of operation of the executable file compiled by the neural network model and improve the running efficiency of the executable file, for a certain feasible feature map parameter set, it can calculate the multiple feasible features included in it. The feasible feature map parameter with the highest data handling efficiency among the map parameters is used as the target feature map parameter corresponding to the feasible feature map parameter set. By analogy, the above operations can be performed on each feasible feature map parameter set, and finally a corresponding target feature map parameter can be determined for each convolutional layer from the feasible feature map parameter set of each convolutional layer.

It should be understood that, since the present application can find a corresponding target feature map parameter with the highest data handling efficiency for each convolutional layer, and then for a neural network model including a plurality of convolutional layers, the present application can realize this neural network model. Find a weight reuse or feature map reuse method with the highest data handling efficiency (ie, data reuse rate) for each convolutional layer. That is to say, the present application can find a best data multiplexing method for each convolutional layer of the neural network model to improve the overall data reuse rate of the neural network model and reduce the running of executable files corresponding to the neural network model. It can increase the computational efficiency of executable files (that is, convolution calculations of different sizes can find the best splitting combination and data multiplexing method, improve the overall data reuse rate of the network, and improve the inference efficiency). That is, by finding the optimal split size for convolution calculations of any size, flexible splitting of different types of convolution calculations in the same network can be achieved, thereby improving the overall data reuse rate of the network and indirectly improving the efficiency of network reasoning.

Therefore, the method provided by the present application can reduce the amount of operation of the executable file compiled by the neural network model and improve the running efficiency of the executable file. In addition, the method provided by the present application is flexible in implementation, the supported convolution calculation size is not restricted by the hardware support size, and can support large-sized convolution operations.

In some possible embodiments, for how to "determine a corresponding target feature map parameter for each convolutional layer from the feasible feature map parameter set of each convolutional layer", based on FIG. Figure 4, S120 may include:

S120A, according to the value of each feasible feature map parameter in the feasible feature map parameter set of each convolutional layer, determine the data repetition loading amount corresponding to each feasible feature map parameter.

In some possible embodiments, after obtaining a feasible output feature map parameter set, a feasible input feature map parameter set, and various split size combinations of feasible weight parameters (that is, after obtaining a feasible feature map parameter set for each convolution layer) ), when no splitting is done, the input feature map, weight, and output feature map will only undergo a data transfer process from DDR (Double DataRate SDRAM, double-rate SDRAM) to SRAM (or SRAM to DDR), if the data If the number of loading of the whole part is more than one time, it will be included in the statistics. When splitting, depending on the combination of split sizes, there are different ways of calculating the amount of repeated data loading, as follows:

Calculation method 1, when only the output channel is split (that is, the weight channel parameter wt_c, the weight dimension parameter wt_n, the weight height parameter wt_h and the weight width parameter wt_w are split, and the channel dimension parameter of the input feature map is split. inf_c, height dimension parameter inf_h and width dimension parameter inf_w, the channel dimension parameter ouf_c, height dimension parameter ouf_h and width dimension parameter ouf_w of the output feature map remain unchanged), and perform feature map analysis on multiple convolutional layers of this neural network model. Better reuse. At this time, although the weights need to be loaded multiple times, the weights loaded multiple times are independent of each other and there is no duplication. Furthermore, in the case of multiplexing feature maps, all data only needs to be loaded once. Therefore, when only the output channel is split, the data duplication load _F1 is zero.

Calculation method 2, when the output width and height are split (that is, the channel dimension parameter inf_c, height dimension parameter inf_h and width dimension parameter inf_w of the input feature map, channel dimension parameter ouf_c, height of the output feature map The dimension parameter ouf_h and the width dimension parameter ouf_w are split, while the weight channel parameter wt_c, weight dimension parameter wt_n, weight height parameter wt_h and weight width parameter wt_w remain unchanged) to weight the multiple convolutional layers of this neural network model. better reuse. Similar to calculation method 1, the feature maps need to be loaded multiple times, and the weights only need to be loaded once. However, due to the division of width and height, according to the scanning characteristics of convolution, the input feature map will introduce fixed repeated data, and in this case, the amount of repeated data F ₂ can be calculated using the following formula:

Among them, s is the convolution step parameter, h_num is the number of sub-blocks divided in the height (H) dimension of the input feature map, and w_num is the number of sub-blocks divided in the width (W) dimension of the input feature map.

Calculation method 3, when the output width, height and channel are split at the same time (that is, the channel dimension parameter inf_c, height dimension parameter inf_h and width dimension parameter inf_w of the input feature map, and the channel dimension parameter ouf_c of the output feature map. , height dimension parameter ouf_h and width dimension parameter ouf_w, weight channel parameter wt_c, weight dimension parameter wt_n, weight height parameter wt_h and weight width parameter wt_w), a sub-feature map needs to be convolved with multiple weight sub-blocks, A weight sub-block also needs to be convolved with multiple sub-feature maps. At this time, if the feature map is reused, the feature map is loaded only once, and the repeated loading amount of the feature map due to splitting needs to be calculated, and the weight needs to be repeated h_num*w_num-1 times as a whole, so the calculation formula of the data repetition amount is: :

F ₃ =(h_num*w_num-1)*ouf_c*inf_c*wt_h*wt_w+F ₂ ;

If the weight is reused, the weight is loaded only once, and the feature map needs to be loaded repeatedly (wt_num-1) times. At this time, the calculation formula of the data repetition amount is:

F′ ₃ =(wt_num-1)*inf_c*inf_h*inf_w+wt_num*F ₂ ;

Among them, wt_num is the number of sub-blocks split by the weight in the N dimension, and is the repeated data generated by splitting the input feature map W dimension and H dimension.

S120B: Use the feasible feature map parameter with the smallest data repetition load in the feasible feature map parameter set of each convolution layer as the target feature map parameter corresponding to each convolution layer.

In some other possible embodiments, in order to further improve the operation efficiency of the method provided by the present application, the neural network model includes multiple convolutional layers, and for how to "select each convolutional layer from the feasible feature map parameter set for each The convolution layer determines a corresponding target feature map parameter", on the basis of Figure 2, please refer to Figure 5, S120 may include:

S120a, according to the value of each feasible feature map parameter in the feasible feature map parameter set of the target convolutional layer, determine the data repetition loading amount corresponding to each feasible feature map parameter.

In this embodiment, for how to "determine the data repetition loading amount corresponding to each feasible feature map parameter according to the value of each feasible feature map parameter in the feasible feature map parameter set of the target convolutional layer", reference can be made to the above S120A, It is not repeated here.

S120b, according to the preset number of processing cores and the amount of repeated data loading, determine an optimal feature map parameter from the feasible feature map parameter set of the target convolutional layer; the target convolutional layer is any one of the multiple convolutional layers. One layer, the optimal feature map parameter is the feasible feature map parameter with the highest data handling efficiency in the feasible feature map parameter set of the target convolution layer.

In some possible embodiments, after the data repetition loading amount F of various split size combinations is counted, the data repetition loading amount and the MAC resource utilization rate can be comprehensively considered to determine the feasible feature map parameters with the highest data handling efficiency, that is, Find the best split combination based on less data repetitive loading and higher MAC resource utilization.

The hardware MAC resource utilization is related to the specific hardware layout. In the hardware design, it is assumed that the hardware MAC resources are divided into _Npe groups, each group is called a PE (logical core), and each PE calculates an output channel Therefore, when the output channel is a multiple of _Npe , the efficiency is the highest. At this time, the data handling efficiency of any two feasible feature map parameters in the feasible feature map parameter set of the target convolution layer can be compared by the following formula:

分别为向下、向上取整。当公式的计算结果w大于零时，表示gourp2更优；反之，则gourp1更优。Among them, it is assumed that any two feasible feature map parameters in the feasible feature map parameter set of the target convolution layer are gourp1 and group2, respectively, T _s in the formula is the time spent on transporting unit data, and T _p is the calculation of a single channel-specific size convolution cost When , T _s and T _p can be obtained by testing in advance; s_ouf_c is the number of original output channels; ouf_c _group1 and ouf_c _group1 are the maximum number of output channels of gourp1 and group2 respectively; ouf_clast _group1 and ouf_clast _group2 are the final output channels generated by division respectively The number of output channels of a sub-block. operation

They are rounded down and up, respectively. When the calculation result w of the formula is greater than zero, it means that gourp2 is better; otherwise, gourp1 is better.

Furthermore, by traversing and comparing any two feasible feature map parameters in the feasible feature map parameter set of the target convolution layer according to the above formula, an optimal feature map parameter can be determined from the feasible feature map parameter set of the target convolution layer.

S120c, taking the optimal feature map parameter as the target feature map parameter corresponding to the target convolution layer.

Since the target convolutional layer is any one of multiple convolutional layers, S120a, S120b, and S120c are processed for the feasible feature map parameter sets of each convolutional layer, so that one convolutional layer can be determined for each convolutional layer. The corresponding target feature map parameters with the highest data handling efficiency.

Please refer to FIG. 2 again, S130 , generate an executable file for the neural network model according to the target feature map parameters corresponding to each convolutional layer.

In some possible embodiments, for how to "generate an executable file for the neural network model according to the target feature map parameters corresponding to each convolutional layer", S130 may include: generating according to the target feature map parameters corresponding to each convolutional layer Three-dimensional direct memory access DMA (Direct Memory Access, direct memory access, that is, direct memory access) data handling instructions, and generate an executable file of the neural network model according to the three-dimensional DMA data handling instructions. Among them, the three-dimensional direct memory access DMA data transfer instruction can be understood as a three-dimensional DMA data transfer instruction.

It can be understood that by using 3D DMA to carry out feature map handling, the seamless splicing of each sub-block in the 3D logical size can be realized directly in the process of outputting the sub-feature map, without the need for additional slice/concate operations.

Further, in order to realize the data processing function of the neural network model, on the basis of FIG. 2, please refer to FIG. 6, after S130, the method may further include:

S140, execute the executable file to realize the data processing function of the neural network model.

In this embodiment, the data processing function of the neural network model may be functions such as natural language processing, image recognition, speech recognition, data mining, and personalized recommendation. Furthermore, the present application can perform offline feature map splitting in the compiler, which can reduce processing logic at runtime, reduce hardware logic complexity, and reduce implementation costs.

It should be understood that, through the compiling method of the neural network model provided by this application, the original feature map parameters of each convolutional layer in the neural network model can be split, and a corresponding data handling efficiency is found for each convolutional layer. The target feature map parameters, thereby improving the overall data reuse rate of the neural network model, reducing the amount of computation of the executable file corresponding to the neural network model during runtime, and improving the computational efficiency of the executable file. The model is compiled into an executable file, and the operation amount of the executable file is reduced, and the running efficiency of the executable file is improved.

In order to perform the corresponding steps in the above-mentioned embodiment and each possible manner, an implementation manner of an apparatus for compiling a neural network model is given below. Please refer to FIG. 7 , which shows the neural network model provided by the embodiment of the present application. A functional block diagram of a compilation device. It should be noted that the basic principles and technical effects of the neural network model compiling apparatus 200 provided in this embodiment are the same as those in the above-mentioned embodiments. The corresponding content in the examples. The apparatus 200 for compiling the neural network model includes: an acquiring module 210 , a splitting module 220 , a generating module 230 , and an operating module 240 .

Optionally, the above-mentioned modules may be stored in a memory in the form of software or firmware (Firmware) or solidified in an operating system (Operating System, OS) of the electronic device 100 provided by the present application, and can be executed by a processor in the electronic device 100. . Meanwhile, data required to execute the above-mentioned modules, codes of programs, and the like may be stored in the memory.

The obtaining module 210 can be used to obtain the original feature map parameters of each convolutional layer in the neural network model.

It can be understood that the obtaining module 210 may be used to support the electronic device 100 to perform the above-mentioned S100, etc., and/or other processes for the techniques described herein.

The splitting module 220 may be configured to split the original feature map parameters according to the relationship between input and output parameters and memory capacity, to obtain a feasible feature map parameter set for each convolutional layer.

It can be understood that the splitting module 220 can be used to support the electronic device 100 to perform the above-mentioned S110, etc., and/or other processes for the techniques described herein.

The splitting module 220 may also be configured to determine a corresponding target feature map parameter for each convolutional layer from the feasible feature map parameter set of each convolutional layer; the target feature map parameter corresponds to the highest data handling efficiency.

It can be understood that the splitting module 220 can be used to support the electronic device 100 to perform the above S120 etc., and/or other processes for the techniques described herein.

The generating module 230 may be configured to generate an executable file for the neural network model according to the target feature map parameters corresponding to each convolutional layer.

It can be understood that the generation module 230 may be used to support the electronic device 100 to perform the above-mentioned S130, etc., and/or other processes for the techniques described herein.

The running module 240 can be used to execute the executable file to realize the data processing function of the neural network model.

It can be understood that the operation module 240 may be used to support the electronic device 100 to perform the above-mentioned S140, etc., and/or other processes for the techniques described herein.

In some possible embodiments, the original feature map parameters in this application may include the original output feature map parameters. The splitting module 220 can be used to split the original output feature map parameters according to the relationship between input and output parameters and memory capacity to obtain a set of feasible output feature map parameters for each convolution layer; The sub-module 220 can also be used to determine the feasible input feature map parameter set of each convolution layer according to the feasible output feature map parameter set of each convolutional layer and the input-output parameter relational expression; the splitting module 220 can also be used for According to the feasible output feature map parameter set and the feasible input feature map parameter set of each convolution layer, the feasible feature map parameter set of each convolution layer is determined.

It will be appreciated that the splitting module 220 may be used to support the electronic device 100 to perform the above-described S110A, S110B, S110C, etc., and/or other processes for the techniques described herein.

In some possible embodiments, regarding how to "determine a corresponding target feature map parameter for each convolutional layer from the set of feasible feature map parameters of each convolutional layer", the splitting module 220 can be used to determine the corresponding target feature map parameter according to each convolutional layer The value of each feasible feature map parameter in the feasible feature map parameter set of the convolutional layers determines the data repetition load corresponding to each of the feasible feature map parameters; the splitting module 220 can also be used to separate each convolutional layer. The feasible feature map parameter with the smallest data repetition load in the feasible feature map parameter set of the layer is used as the target feature map parameter corresponding to each convolutional layer.

It will be appreciated that the splitting module 220 may be used to support the electronic device 100 to perform the above-described S120A, S120B, etc., and/or other processes for the techniques described herein.

In some other possible embodiments, in order to further improve the operation efficiency of the method provided by the present application, the neural network model includes multiple convolutional layers, and for how to "select each convolutional layer from the feasible feature map parameter set for each The convolution layer determines a corresponding target feature map parameter", and the splitting module 220 can be used to determine each feasible feature map parameter according to the value of each feasible feature map parameter in the feasible feature map parameter set of the target convolution layer. Corresponding data repetition loading amount; the splitting module 220 can also be used to determine an optimal feature map parameter from the feasible feature map parameter set of the target convolution layer according to the preset number of processing cores and the data repetition loading amount; The target convolutional layer is any one of the multiple convolutional layers, and the optimal feature map parameter is the feasible feature map parameter with the highest data handling efficiency in the feasible feature map parameter set of the target convolutional layer; the splitting module 220 can also use The optimal feature map parameters are used as the target feature map parameters corresponding to the target convolutional layer.

It will be appreciated that the splitting module 220 may be used to support the electronic device 100 to perform the above-described S120a, S120b, S120c, etc., and/or other processes for the techniques described herein.

Based on the foregoing method embodiments, embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and the computer program is executed by a processor when the method for compiling the neural network model is executed. step.

Specifically, the storage medium can be a general storage medium, such as a removable disk, a hard disk, etc., when the computer program on the storage medium is run, it can execute the above-mentioned method for compiling the neural network model, so that the current optimization method is relatively simple. The problem that the running efficiency of the compiled executable file is still very low, to achieve the purpose of compiling the neural network model into an executable file, reducing the amount of operation of the executable file at runtime, and improving the running efficiency of the executable file .

In summary, the embodiments of the present application provide a method, device, storage medium, and electronic device for compiling a neural network model. The method includes: acquiring the original feature map parameters of each convolutional layer in the neural network model; The original feature map parameters are split by relational expression and memory capacity to obtain a set of feasible feature map parameters for each convolution layer; a corresponding set of parameters for each convolution layer is determined from the set of feasible feature map parameters for each convolution layer. The target feature map parameters of the target feature map; the data handling efficiency corresponding to the target feature map parameters is the highest; according to the target feature map parameters corresponding to each convolutional layer, an executable file is generated for the neural network model. Through the method for compiling a neural network model provided in this application, the original feature map parameters of each convolutional layer in the neural network model can be split, and a corresponding target feature map with the highest data handling efficiency can be found for each convolutional layer. parameters, thereby improving the overall data reuse rate of the neural network model, reducing the amount of computation of the executable file corresponding to the neural network model at runtime, and improving the computational efficiency of the executable file, that is, the present application can compile the neural network model into The executable file is reduced, and the amount of operation of the executable file is reduced, and the running efficiency of the executable file is improved.

The above are only specific embodiments of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, All should be covered within the scope of protection of this application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims (10)

1. A method of compiling a neural network model, the method comprising:

acquiring an original characteristic diagram parameter of each convolution layer in the neural network model;

splitting the original characteristic diagram parameters according to the input-output parameter relational expression and the memory capacity to obtain a feasible characteristic diagram parameter set of each convolution layer;

determining a corresponding target characteristic map parameter for each convolution layer from the feasible characteristic map parameter set of each convolution layer; the data handling efficiency corresponding to the target characteristic diagram parameters is highest;

and generating an executable file for the neural network model according to the target characteristic diagram parameters corresponding to each convolution layer.

2. The method of claim 1, wherein the raw feature map parameters comprise raw output feature map parameters;

the step of splitting the original characteristic diagram parameters according to the input-output parameter relational expression and the memory capacity to obtain a feasible characteristic diagram parameter set of each convolution layer comprises the following steps:

splitting the original output characteristic diagram parameters according to the input-output parameter relational expression and the memory capacity to obtain a feasible output characteristic diagram parameter set of each convolution layer;

determining a feasible input characteristic diagram parameter set of each convolution layer according to the feasible output characteristic diagram parameter set of each convolution layer and the input-output parameter relational expression;

and determining a feasible feature map parameter set of each convolutional layer according to the feasible output feature map parameter set and the feasible input feature map parameter set of each convolutional layer.

3. The method of claim 2, wherein the step of determining a corresponding target profile parameter for each convolutional layer from the set of feasible profile parameters for each convolutional layer comprises:

determining the data repeated loading capacity corresponding to each feasible feature map parameter according to the value of each feasible feature map parameter in the feasible feature map parameter set of each convolutional layer;

and taking the feasible characteristic diagram parameter with the minimum data repeated loading amount in the feasible characteristic diagram parameter set of each convolution layer as the target characteristic diagram parameter corresponding to each convolution layer.

4. The method of claim 2, wherein the neural network model comprises a plurality of convolutional layers, and wherein the step of determining a corresponding target feature map parameter for each convolutional layer from the set of feasible feature map parameters for each convolutional layer comprises:

determining the data repeated loading capacity corresponding to each feasible feature map parameter according to the value of each feasible feature map parameter in the feasible feature map parameter set of the target convolutional layer;

determining an optimal characteristic map parameter from a feasible characteristic map parameter set of the target convolutional layer according to the number of preset processing cores and the data repetition loading amount; the target convolutional layer is any one of the plurality of convolutional layers, and the optimal characteristic map parameter is a feasible characteristic map parameter with the highest data transfer efficiency in a feasible characteristic map parameter set of the target convolutional layer;

and taking the optimal characteristic map parameters as target characteristic map parameters corresponding to the target convolutional layer.

5. The method of claim 1, wherein the step of generating an executable file for the neural network model based on the target profile parameters corresponding to each convolutional layer comprises:

and generating a three-dimensional Direct Memory Access (DMA) data carrying instruction according to the target characteristic diagram parameters corresponding to each convolution layer, and generating an executable file of the neural network model according to the three-dimensional DMA data carrying instruction.

6. The method of claim 1, further comprising:

executing the executable file to implement the data processing function of the neural network model.

7. An apparatus for compiling a neural network model, comprising:

the acquisition module is used for acquiring the original characteristic map parameters of each convolution layer in the neural network model;

the splitting module is used for splitting the original characteristic diagram parameters according to the input-output parameter relational expression and the memory capacity to obtain a feasible characteristic diagram parameter set of each convolution layer;

the splitting module is further configured to determine a corresponding target feature map parameter for each convolution layer from the feasible feature map parameter set of each convolution layer; the data handling efficiency corresponding to the target characteristic diagram parameters is highest;

and the generating module is used for generating an executable file for the neural network model according to the target characteristic diagram parameters corresponding to each convolution layer.

8. The apparatus of claim 7, wherein the raw feature map parameters comprise raw output feature map parameters;

the splitting module is used for splitting the original output characteristic diagram parameters according to the input-output parameter relational expression and the memory capacity to obtain a feasible output characteristic diagram parameter set of each convolution layer;

the splitting module is further used for determining a feasible input characteristic diagram parameter set of each convolution layer according to the feasible output characteristic diagram parameter set of each convolution layer and the input-output parameter relational expression;

the splitting module is further used for determining a feasible feature map parameter set of each convolutional layer according to the feasible output feature map parameter set and the feasible input feature map parameter set of each convolutional layer.

9. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out a method of compiling a neural network model according to any one of claims 1 to 6.

10. An electronic device comprising a processor and a memory, the memory storing machine readable instructions, the processor being configured to execute the machine readable instructions to implement the method of compiling a neural network model of any one of claims 1-6.

Images