CN110378413A - Neural network model processing method, device and electronic device - Google Patents

Abstract

The embodiment of the application discloses a neural network model processing method and device and electronic equipment. The method comprises the following steps: acquiring a neural network model to be configured; mapping the neural network model into a graph structure based on the dependency relationship of operators in the neural network model, wherein one node in the graph structure represents one operator in the neural network model; traversing the graph structure to obtain the execution sequence of the nodes; configuring an execution order of operators characterized by the nodes based on the execution order of the nodes. According to the method, the neural network model is converted into the graph structure, and the execution sequence of the operator represented by each node is determined by traversing the nodes in the graph structure, so that the execution sequences of all operators can be rapidly determined, and the operation efficiency of the whole model is improved.

Description

Neural network model processing method, device and electronic device

technical field

The present application relates to the field of computer technology, and more particularly, to a method, apparatus and electronic device for processing a neural network model.

Background technique

Usually neural network models are trained on computers and other equipment. In order to facilitate the trained neural network model to run on electronic devices such as mobile phones or tablet computers, the trained neural network model can be optimized. However, during the optimization process, the execution sequence of the operators of the neural network model may be disordered.

SUMMARY OF THE INVENTION

In view of the above problems, the present application proposes a neural network model processing method, apparatus, and electronic device to improve the above problems.

In a first aspect, the present application provides a method for processing a neural network model, which is applied to an electronic device. The method includes: acquiring a neural network model to be configured; The neural network model is mapped to a graph structure, wherein a node in the graph structure represents an operator in the neural network model; the graph structure is traversed to obtain the execution order of the nodes; based on the node The execution order of the node configures the execution order of the operators represented by the node.

In a second aspect, the present application provides a neural network model processing apparatus, which runs on an electronic device, and the method includes: a model obtaining unit, configured to obtain a neural network model to be configured; and a model processing unit, configured based on the neural network model The dependency relationship of operators in the network model, the neural network model is mapped to a graph structure, wherein a node in the graph structure represents an operator in the neural network model; a traversal unit is used for the The graph structure is traversed to obtain the execution order of the nodes; the order determination unit is configured to configure the execution order of the operators represented by the nodes based on the execution order of the nodes.

In a fourth aspect, the present application provides an electronic device, comprising a multi-core processor, a boot controller, and a memory, where the memory is used to store data to be loaded; one or more programs are stored in the boot controller and is configured to be executed by the startup controller to implement the method described above.

In a fifth aspect, the present application provides a computer-readable storage medium, where a program code is stored in the computer-readable storage medium, wherein the above-mentioned method is executed when the program code is run by a startup controller.

In a neural network model processing method, device and electronic device provided by the present application, after acquiring a neural network model to be configured, the neural network model is mapped to a graph structure based on the dependencies of operators in the neural network model , wherein a node in the graph structure represents an operator in the neural network model. Then, the graph structure is traversed to obtain the execution order of the nodes, and the execution order of the operators represented by the nodes is configured based on the execution order of the nodes. Therefore, by converting the neural network model into a graph structure, and then by traversing the nodes in the graph structure to determine the execution order of the operators represented by each node, it is possible to quickly determine the execution order of all operators and improve the overall performance. The computational efficiency of the model.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the drawings that are used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can also be obtained from these drawings without creative effort.

1 shows a flowchart of a method for processing a neural network model proposed by an embodiment of the present application;

2 shows a schematic diagram corresponding to a neural network model in a neural network model processing method proposed by an embodiment of the present application;

3 shows a schematic diagram of a mapping relationship between processing resource consumption degree and time in a neural network model processing method proposed by an embodiment of the present application;

FIG. 4 shows a flowchart of a method for processing a neural network model proposed by another embodiment of the present application;

FIG. 5 shows a flowchart of a method for processing a neural network model proposed by yet another embodiment of the present application;

FIG. 6 shows a structural block diagram of an apparatus for processing a neural network model proposed by an embodiment of the present application;

FIG. 7 shows a structural block diagram of an apparatus for processing a neural network model proposed by another embodiment of the present application;

FIG. 8 shows a structural block diagram of a neural network model processing apparatus proposed by still another embodiment of the present application;

FIG. 9 shows a structural block diagram of an electronic device of the present application for executing the neural network model processing method according to an embodiment of the present application;

FIG. 10 is a storage unit for storing or carrying a program code for implementing the neural network model processing method according to the embodiment of the present application according to the embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

Neural network (Neural Networks, NN) is a complex network system formed by extensive interconnection of a large number of simple processing units (called neurons). Neural networks have massive parallelism, distributed storage and processing, self-organization, self-adaptation and self-learning capabilities. A large number of operators are usually included in the neural network model. Among them, it can be understood that an operator can be regarded as a part of the algorithm process in a neural network model, and an operator can map a function into a function, or map a function into a number.

However, the inventor found in the research that when the neural network has more operators or higher complexity, the overall operator execution order cannot be determined relatively quickly.

Furthermore, the operator sequence of the neural network mostly adopts the sequence of operators trained by the neural network framework on the server side. For example, similar to the operator sequence of the neural network model trained by the frameworks such as Tensorflow, when it is necessary to increase the neural network model obtained by training. When the operator is used, it can only be added between specific operators in the original network diagram. As a result, the neural network diagram needs to be regenerated when the operator is optimized, resulting in a much slower model conversion speed. The reason is that each operator in the neural network graph is dependent, that is, if an operator is to be executed correctly, the operator it depends on must be executed, so the operator has an execution order. Models solidified by PC-side frameworks like Tensorflow are generally arranged for operators. For example, operator B depends on operator A. In the file (the file storing the neural network model), operator A is ranked in front of B. If operator C needs to be added between A and B, the general The method is to regenerate the network, copy it in order until operator A, then add operator C, then copy operator B, and then copy subsequent operators. That is to say, C must be added between A and B in order to ensure the correct execution of the operator. This causes adding operators to become a very complex operation, which will make the neural network model conversion take a long time, which in turn will cause the model conversion speed to be much slower.

In addition, the deployment of the neural network model on the mobile terminal is generally by solidifying the trained model into a file on a computer such as a PC, and then the neural network framework on the mobile terminal parses the file, reads it into memory, and executes the neural network model in sequence. the operator. However, due to the limited resources on the mobile terminal, only small models can often be run. Large models need to undergo a series of optimizations, such as operator fusion, network pruning, model quantization, network cutting, etc. the purpose of running the terminal. These optimizations often lead to changes in the execution order of the original operators, so it is necessary to reorder the operators in an effective way, so that the neural network model can be executed normally.

Therefore, the inventor proposes a neural network model processing method, apparatus and electronic device that can improve the above problems in the present application.

The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

Referring to FIG. 1, a method for processing a neural network model provided by an embodiment of the present application is applied to an electronic device, and the method includes:

Step S110: Obtain the neural network model to be configured.

As a method, the neural network model to be configured may be a neural network model obtained directly from the network end. In addition, the neural network model to be configured may also be a neural network model obtained by optimizing the neural network model obtained from the network end. Among them, the optimization of the neural network model can be understood as operator fusion, network pruning, model quantization, network cutting and other operations on the neural network model.

Step S120: Map the neural network model into a graph structure based on the operator dependency in the neural network model, wherein a node in the graph structure represents an operator in the neural network model.

A graph structure is a mathematical object that represents the relationship between objects. If a direction is specified for each edge of a graph structure, the resulting graph structure is called a directed graph. In a directed graph, there are outgoing and incoming edges associated with a node. Conversely, a graph whose edges have no direction is called an undirected graph.

As a way, the graph structure in this embodiment is a Directed AcyclicGraph (Directed AcyclicGraph). Generally speaking, an acyclic directed graph is called a directed acyclic graph. A directed acyclic graph is actually a data structure like an array, an array, and a blockchain. But unlike the blockchain, the directed acyclic graph changes the longest chain consensus to the heaviest chain consensus. On the traditional blockchain, the newly released block will be added to the original longest chain, and the chain that is considered to be the longest by all nodes shall prevail, and the chain will spread indefinitely. In the directed acyclic graph, each newly added unit is not only added to one block in the long chain, but also added to all previous blocks.

It can be understood that the neural network model is composed of many operators. And some operators have mutual dependencies. The dependencies here can be understood as data dependencies. For example, there are operator A, operator B, operator C, operator D, operator E, operator F, operator G and operator H in the neural network model. Among them, the neural network model defines that the input data of operator C is the output data of operator B, and the input data of operator B is the output data of operator A, and operator A has only data output. Then it can be concluded from this that the operation of operator D requires the output data of operator B, then operator D depends on operator B. Similarly, the operation of operator B requires the output of operator A. data, then operator B is dependent on operator A.

Furthermore, the output data of operator C is the output data of operator F, the input data of operator E is the output data of operator A, and the input data of operator F is not only from operator C, but also from operator C. Sub B. The input data of operator G comes from operator B, operator C and operator E, and the input data of operator H comes from operator D and operator F. Then based on the above explanation of the dependency relationship, operator F depends on operator B and operator C at the same time, operator E depends on operator A, and operator G depends on operator B, operator C and operator E at the same time, Operator H depends on both operator D and operator F.

Furthermore, the phased acyclic graph shown in Fig. 2 is obtained based on the above dependencies. There are node 0, node 1, node 2, node 3, node 4, node 5, node 6 and node 7 in the figure. Among them, node 0 represents operator A, node 1 represents operator B, node 2 represents operator C, node 3 represents operator D, node 4 represents operator E, node 5 represents operator F, and node 6 represents operator G , node 7 represents operator H. Among them, the node pointed by the arrow depends on the starting node of the arrow.

Then in this way, the electronic device may firstly traverse the graph structure to obtain the execution order of the nodes, including: performing depth-first traversal on the directed acyclic graph to obtain the The execution order of the nodes.

Step S130: Traverse the graph structure to obtain the execution sequence of the nodes.

Among them, as shown in Figure 2, although the execution order between each adjacent two nodes is relatively clear, if all nodes that do not have adjacent relationships are considered, then for the overall The order of execution may be wrong. For example, for node 1, node 2, and node 5 in FIG. 2, it is relatively clear that node 1 is executed first, node 5 is executed, and node 2 is executed first, and then node 5 is executed. However, if node 5 needs to be executed after node 1 is executed, it will cause node 5 to run before node 2, but the operator corresponding to node 5 will have no effect on the operator corresponding to node 2 during the running process. The output data may have certain dependencies, which will cause data errors. Then, as a method, the electronic device can traverse each node of the graph structure to obtain the overall execution order of each node.

As a method, the depth-first traversal of the directed acyclic graph is performed to obtain the execution order of the nodes.

It should be noted that the node traversal method of the graph structure includes depth-first search and breadth-first search. Depth-first search belongs to a kind of graph algorithm, the English abbreviation is DFS, which is Depth First Search.

In the specific traversal process, because there are two vertices that depend on node 0, node 1 and node 3, one of the vertices is selected here (whichever one is selected will not matter), and node 1 is selected here as the node to continue the search. After finding node 1, you need to find the vertices that depend on node 1. From the figure, you can see that node 3, node 5, and node 6 all depend on node 1. Here node 3 is selected as the next searched node. Then continue to find the next node that depends on node 3. Since only node 7 depends on node 3, node 7 is selected. Since no node depends on node 7, node 7 is taken as the first target node. Because the node that depends on node 7 cannot be found, the next step is to go back to the previous node 3. Because the only node that depends on node 3 is node 7, and node 7 has been traversed, the untraversed node that depends on node 3 cannot be found, so node 3 is also used as the target node. The target node at this time includes node 7 and node 3 in turn.

Continue to select the previous node of node 3, that is, node 1, and find the untraversed nodes that depend on node 1. There are still nodes 5 and 6, and node 5 is selected. After finding node 5, continue to look for nodes that depend on node 5. There is only node 7 here, and it has been traversed, so node 5 is marked as the target node. The target node at this time includes node 7 , node 3 and node 5 in sequence.

Go back to the previous node 1 of node 5, and continue to look for nodes that depend on node 1. Only node 6 is left here. Since no node depends on node 6, node 6 is marked as the target node. At this time, the target node includes node 7 , node 3 , node 5 and node 6 in sequence. At this time, go back to the previous node of node 6, that is, node 6. At this time, because no node depends on it and has not been traversed, node 1 is marked as the target node. At this time, the target node includes node 7 , node 3 , node 5 , node 6 and node 1 in sequence. Go back to the previous node of node 1, that is, node 0, and find node 4 that depends on node 0. Since no untraversed vertices depend on node 4, node 4 is marked as the target node. At this time, the target node includes node 7 , node 3 , node 5 , node 6 , node 1 and node 4 in sequence. Go back to the previous node of node 4, that is, node 0. Since there is no node that depends on node 0, node 0 is marked as the target node. At this time, the target node includes node 7, node 3, node 5, node 6, node 1, node 4, and node 0 in sequence. Since we have returned to the input node, namely node 0, we need to start the traversal from another input node, namely node 2. Because the nodes that depend on node 2 have all been traversed, node 2 is directly marked as the target node.

Then, the execution sequence of each node can be obtained based on the marking sequence of the above target nodes, that is, the execution sequence is node 2, node 0, node 4, node 1, node 6, node 5, node 3 and node 7. Wherein, in order to facilitate the electronic device to quickly identify the execution sequence of the marked target nodes, the storage space of the stack structure may be used to store the marked target nodes. Then, in this way, during the traversal process, the electronic device performs depth-first traversal on the directed acyclic graph, and stores the target nodes obtained during the traversal process into the storage space of the stack structure in sequence, and the target nodes are A node without a subordinate node or a node to which all subordinate nodes are traversed; the popping order of the nodes in the storage space is taken as the execution order of the nodes. Based on the example shown in FIG. 2 , the order of the target nodes stored in the storage space of the stack structure is node 7 , node 3 , node 5 , node 6 , node 1 , node 4 , node 0 , and node 2 . Then the corresponding popping order is node 2, node 0, node 4, node 1, node 6, node 5, node 3 and node 7.

It can be understood that for nodes that do not depend on other nodes, electronic devices can be identified as input vertices. Then, when the electronic device recognizes that there are multiple input vertices, the electronic device can perform depth-first traversal on the directed acyclic graph based on the specified initial input vertex to obtain the execution order of the nodes.

It can be understood that in the case of depth-first traversal, the input vertices traversed first will be executed later. For example, in the foregoing example, the traversal is started from the input vertex 0 (node 0) first, and then the traversal is started from the input vertex 2 (node 2), then the first traversed node 0 is executed later. Similarly, for nodes that are depended on by multiple nodes, the nodes that are traversed first will also be executed later. For example, node 3, node 5, and node 6 in Figure 2 all depend on node 1. If the traversal starts from node 3 first, then node 3 will be executed after node 5 or node 6 accordingly. However, the processing resources consumed by the operators represented by different nodes may be different when they are executed. In addition, the total processing resources of the electronic device are limited. If a large amount of processing resources are consumed during the execution of the operator corresponding to a node, and the electronic device always has other programs that also consume a large amount of processing resources, This will cause the electronic equipment to freeze or crash.

Then, as a way to improve the above problem, the electronic device may perform a traversal of all branch situations once during the node traversal process, thereby obtaining various execution orders of all nodes. For example, in the case of node 0, node 1, node 2, node 3, node 4, node 5, node 6 and node 7, an execution order can be obtained as node 2, node 0, node 4, node 1, Node 6, node 5, node 3, and node 7, another execution order can be obtained: node 2, node 0, node 4, node 1, node 6, node 3, node 5, and node 7, and another execution sequence can be obtained. The execution order is node 2, node 0, node 4, node 1, node 3, node 5, node 6, and node 7. In the case of different processing resources consumed during the execution of the operators corresponding to different nodes, the electronic device can already obtain the neural network model after determining the execution order of all nodes. The relative amount of processing resources that each stage of the process needs to consume. One of the phases represents the execution of an operator.

Correspondingly, the electronic device can also estimate the consumption status of processing resources by other programs during the entire execution process of the neural network model, and then compare the consumption status of processing resources by other programs with the resources corresponding to the aforementioned various execution sequences. The consumption status is matched, and the one with the least conflict with the consumption status of processing resources of other programs among the multiple execution sequences is used as the final execution sequence.

In one way, the electronic device may associate processing resource consumption states of various execution sequences with processing time to establish a two-dimensional mapping relationship between processing resource consumption and time. As shown in Figure 3, the abscissa indicates the time, the ordinate indicates the degree of resource consumption, and each column represents the execution process of a node. For example, t1 to t2 represent the execution process of the operator corresponding to a node. It can be understood that the execution time of the operator represented by each node may be different, and the figure is only an exemplary display. Moreover, for the convenience of calculation, the resource consumption degree corresponding to each node in the figure is not the average resource consumption degree during the operation of the operator corresponding to the node.

Then, based on the method shown in FIG. 3 , the mapping relationship between the resource consumption degree and the time corresponding to various execution orders can be obtained. Then, the mapping relationship between the resource consumption programs and time of other programs of the electronic device in the corresponding time period is obtained, and then the mapping relationship between the resource consumption programs and time of other programs and the resource consumption degree corresponding to various execution sequences are obtained. By segment matching with the mapping relationship between times, an execution order with the least resource consumption conflict can be obtained.

It can be understood that each segment in the segment matching represents the execution process of an operator represented by a node. For example, t1 to t2 shown in FIG. 3 is a segment, and t2 to t3 are also a segment. Then in the matching process, the electronic device can calculate the processing resource consumption degree of other programs of the electronic device in the time period from t1 to t2, and then compare the processing resource consumption degree of other programs in the time period from t1 to t2 with that from t1 to t2. The processing resources required for the execution of the operators corresponding to the nodes in this time period are matched, and then the matching is completed for each stage in turn, so as to obtain the overall matching degree.

In this embodiment, as a method, the matching can be completed by scoring. Optionally, the electronic device can establish a mapping relationship between the difference in the resource consumption degree and the score, and can obtain a score when matching the resource consumption degree of each stage. For example, the smaller the difference, the higher the score. , then the electronic device can obtain the score of each stage, so as to obtain the total score of each execution order, and take the execution order with the lowest score as the selected execution order. Specifically, taking the stages shown in Fig. 3 as an example, the execution stage corresponding to node a is the time period from t1 to t2, and the corresponding processing resource consumption level a1, then it is obtained that the electronic device is in the time period from t1 to t2. The processing resource consumption degree of other applications in the system is a2, then the score associated with the absolute value of the difference between a1 minus a2 is taken as the score at this stage. Similarly, the scores of several time periods from t2 to t3, t3 to t4, t4 to t5, t5 to t6, t6 to t7, and t7 to t8 can be obtained, and then the overall total score can be obtained.

It should be noted that the resource consumption degree of other applications in the electronic device in each time period may also be an average consumption degree. Furthermore, it can be understood that the aforementioned selection process of various execution sequences is performed before the actual neural network model is executed. Then, the time consumed by the operation of the operator corresponding to the node in each stage and the occupancy degree of processing resources are pre-calculated estimated values. Similarly, the processing resource occupancy level of other applications of the electronic device in each stage is also an estimated value.

As a method, the time consumed by the operation of the operator corresponding to the node in each stage and the degree of processing resource occupancy can be pre-calculated during the training of the neural network model. Then, in this way, the electronic device receives the file storing the neural network model from the network, and also receives the file identifying the computation time and resource occupancy level of each node in the neural network model. Furthermore, the processing resource occupation degree of other applications of the electronic device in each stage can be estimated according to the historical running records of the applications of the electronic device. It can be understood that, the electronic device can record what application program is running in what time period of each day, and then the corresponding consumption degree of processing resources during the running process can also be recorded. Then, the electronic device can obtain a certain rule through statistics of the consumption degree of processing resources in a period of time, and then estimate the degree of consumption of processing resources in each subsequent period of time.

It should be noted that each stage shown above is a time period in a day, for example, a time period such as 12:10 to 12:12. For another example, the time period from 12:1:30 to 12:2, or even a shorter or longer time period. The processing resources described therein can be understood as the occupancy rate of the processor.

Step S140: Configure the execution sequence of the operators represented by the node based on the execution sequence of the node.

In a method for processing a neural network model provided by the present application, after acquiring a neural network model to be configured, the neural network model is mapped to a graph structure based on the dependencies of operators in the neural network model, wherein the A node in the graph structure represents an operator in the neural network model. Then, the graph structure is traversed to obtain the execution order of the nodes, and the execution order of the operators represented by the nodes is configured based on the execution order of the nodes. Therefore, by converting the neural network model into a graph structure, and then by traversing the nodes in the graph structure to determine the execution order of the operators represented by each node, it is possible to quickly determine the execution order of all operators and improve the overall performance. The computational efficiency of the model.

Referring to FIG. 4 , a method for processing a neural network model provided by an embodiment of the present application is applied to an electronic device, and the method includes:

Step S210: Receive the neural network model obtained by training.

Step S220: Optimizing the neural network model obtained by the training to obtain a neural network model to be configured; wherein the neural network model to be configured is a neural network model adapted to the data computing capability of the electronic device.

The step of optimizing the neural network model obtained by training at least includes at least one of the following steps: performing operator fusion on the neural network model obtained by training; performing network trimming on the neural network model obtained by training. performing model quantization on the neural network model obtained by the training; and performing network cutting on the neural network model obtained by the training.

Among them, operator fusion can be understood as merging some operators to reduce calculations or reduce memory copies (such as the merger of Conv2D and BatchNorm). Network pruning can be understood as removing some unnecessary operators to simplify the network (such as removing some redundant operators that are only used during training). Furthermore, the internal calculation of the general neural network model uses floating-point calculation, and the calculation of floating-point number will consume relatively large computing resources (space and cpu/gpu time), if the accuracy of the neural network model is not affected. , if other simple numerical types can be used for calculation inside the neural network model, the calculation speed will be greatly improved, and the consumption of computing resources will be greatly reduced, especially for mobile devices, this is especially important. Quantization is the compression of the original neural network model by reducing the number of bits required to represent each weight.

Step S230: mapping the neural network model to a graph structure based on the operator dependency in the neural network model, wherein a node in the graph structure represents an operator in the neural network model;

Step S240: Traverse the graph structure to obtain the execution sequence of the nodes.

Step S250: Configure the execution order of the operators represented by the node based on the execution order of the node.

In a method for processing a neural network model provided by the present application, after acquiring a neural network model to be configured, the neural network model is mapped to a graph structure based on the dependencies of operators in the neural network model, wherein the A node in the graph structure represents an operator in the neural network model. Then, the graph structure is traversed to obtain the execution order of the nodes, and the execution order of the operators represented by the nodes is configured based on the execution order of the nodes. Therefore, by converting the neural network model into a graph structure, and then by traversing the nodes in the graph structure to determine the execution order of the operators represented by each node, it is possible to quickly determine the execution order of all operators and improve the overall performance. The computational efficiency of the model. In addition, the method provided in this embodiment may calculate the optimized neural network model, thereby realizing that the accurate operator execution order can be quickly obtained after the neural network model is optimized.

Referring to FIG. 5 , a method for processing a neural network model provided by an embodiment of the present application is applied to an electronic device, and the method includes:

Step S310: Receive the neural network model obtained by training.

Step S320: adding a target operator after specifying the execution order of the operators in the neural network model obtained by training to obtain the neural network model to be configured, where the target operator is used to execute the specified operator to obtain the storage format of the data Convert to the target format.

It should be noted that, after the storage format conversion, the electronic device can perform data calculation faster. For example, in electronic devices such as smart phones and tablet computers, there are generally two memory layouts for operators that are calculated by GPU: Buffer and Image. The former does not need to process the model, but the calculation speed is not as fast as the latter, so it is generally used. Image method, but the weights in the network model need to be converted from plain memory to Image format, so operators need to be added to the network graph for memory conversion.

Step S330: Map the neural network model into a graph structure based on the operator dependency in the neural network model, wherein a node in the graph structure represents an operator in the neural network model;

Step S340: Traverse the graph structure to obtain the execution sequence of the nodes.

Step S350: Configure the execution order of the operators represented by the node based on the execution order of the node.

In a method for processing a neural network model provided by the present application, after acquiring a neural network model to be configured, the neural network model is mapped to a graph structure based on the dependencies of operators in the neural network model, wherein the A node in the graph structure represents an operator in the neural network model. Then, the graph structure is traversed to obtain the execution order of the nodes, and the execution order of the operators represented by the nodes is configured based on the execution order of the nodes. Therefore, by converting the neural network model into a graph structure, and then by traversing the nodes in the graph structure to determine the execution order of the operators represented by each node, it is possible to quickly determine the execution order of all operators and improve the overall performance. The computational efficiency of the model. In addition, the method provided in this embodiment may be to calculate the neural network model after inserting a new operator, so as to realize that after the neural network model inserts a new operator, the accurate execution order of the operators can be quickly obtained. .

Referring to FIG. 6 , an apparatus 400 for processing a neural network model provided by an embodiment of the present application operates on an electronic device, and the apparatus 400 includes:

The model obtaining unit 410 is configured to obtain the neural network model to be configured.

As one way, as shown in FIG. 7 , the model obtaining unit 410 includes:

The model receiving subunit 411 is used for receiving the neural network model obtained by training;

The model optimization subunit 412 is used to optimize the neural network model obtained by the training, and obtain the neural network model to be configured; wherein, the neural network model to be configured is adapted to the data computing capability of the electronic device. Neural network model.

As one way, as shown in FIG. 8 , the model obtaining unit 410 includes:

The model receiving subunit 413 is configured to receive the neural network model obtained by training.

The operator adding subunit 414 adds a target operator after the execution order of the specified operator in the neural network model obtained by training, to obtain the neural network model to be configured, and the target operator is used to execute the specified operator to obtain The storage format of the data is converted to the target format.

Wherein, optionally, the step of optimizing the neural network model obtained by training includes at least one of the following steps: performing operator fusion on the neural network model obtained by training; Network pruning is performed on the neural network model; model quantization is performed on the neural network model obtained by the training; and network cutting is performed on the neural network model obtained by the training.

A model processing unit 420, configured to map the neural network model to a graph structure based on the dependencies of operators in the neural network model, wherein a node in the graph structure represents one of the neural network models operator.

The traversing unit 430 is configured to traverse the graph structure to obtain the execution order of the nodes.

The sequence determination unit 440 is configured to configure the execution sequence of the operators represented by the node based on the execution sequence of the node.

As an approach, the graph structure is a directed acyclic graph. In this manner, the model processing unit 420 is specifically configured to perform depth-first traversal on the directed acyclic graph to obtain the execution order of the nodes. The traversal unit 430 is specifically configured to perform depth-first traversal on the directed acyclic graph, and store the target nodes obtained in the traversal process into the storage space of the stack structure in turn, and the target nodes are nodes without subordinate nodes or subordinate nodes. A node whose nodes are all traversed; the pop-out order of the nodes in the storage space is taken as the execution order of the nodes.

In one way, the traversal unit 430 is specifically configured to perform depth-first traversal on the directed acyclic graph based on a specified initial input vertex to obtain the execution order of the nodes.

It should be noted that the apparatus embodiments in the present application correspond to the foregoing method embodiments, and the specific principles in the apparatus embodiments may refer to the content in the foregoing method embodiments, which will not be repeated here.

An electronic device provided by the present application will be described below with reference to FIG. 9 .

Referring to FIG. 9 , based on the above-mentioned method and apparatus for processing a neural network model, an embodiment of the present application further provides another electronic device 200 that can execute the aforementioned method for processing a neural network model. The electronic device 200 includes one or more (only one shown in the figure) a processor 102, a memory 104, and a network module 106 that are coupled to each other. Wherein, the memory 104 stores a program that can execute the content in the foregoing embodiments, and the processor 102 can execute the program stored in the memory 104 .

The processor 102 may include one or more cores for processing data. The processor 102 uses various interfaces and lines to connect various parts of the entire electronic device 200, and executes by running or executing the instructions, programs, code sets or instruction sets stored in the memory 104, and calling the data stored in the memory 104. Various functions of the electronic device 200 and processing data. Optionally, the processor 102 may employ at least one of a digital signal processing (Digital Signal Processing, DSP), a Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA), and a Programmable Logic Array (Programmable Logic Array, PLA). implemented in hardware. The processor 102 may integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU), a modem, and the like. Among them, the CPU mainly handles the operating system, user interface and application programs, etc.; the GPU is used for rendering and drawing of the display content; the modem is used to handle wireless communication. It can be understood that, the above-mentioned modem may not be integrated into the processor 102, and is implemented by a communication chip alone.

The memory 104 may include random access memory (Random Access Memory, RAM), or may include read-only memory (Read-Only Memory). Memory 104 may be used to store instructions, programs, codes, sets of codes, or sets of instructions. The memory 104 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing the operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.) , instructions for implementing the following method embodiments, and the like. The storage data area may also store data created by the terminal 100 during use (such as phone book, audio and video data, chat record data) and the like.

The network module 106 is used for receiving and sending electromagnetic waves, realizing mutual conversion between electromagnetic waves and electrical signals, so as to communicate with a communication network or other devices, for example, communicate with an audio playback device. The network module 106 may include various existing circuit elements for performing these functions, eg, antennas, radio frequency transceivers, digital signal processors, encryption/decryption chips, subscriber identity module (SIM) cards, memory, etc. . The network module 106 can communicate with various networks such as the Internet, an intranet, a wireless network, or communicate with other devices through a wireless network. The aforementioned wireless network may include a cellular telephone network, a wireless local area network, or a metropolitan area network. For example, the network module 106 may interact with the base station for information.

Please refer to FIG. 10 , which shows a structural block diagram of a computer-readable storage medium provided by an embodiment of the present application. The computer-readable medium 1100 stores program codes, and the program codes can be invoked by the processor to execute the methods described in the above method embodiments.

The computer-readable storage medium 1100 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 1100 includes a non-transitory computer-readable storage medium. Computer readable storage medium 1100 has storage space for program code 810 to perform any of the method steps in the above-described methods. These program codes can be read from or written to one or more computer program products. Program code 1110 may be compressed, for example, in a suitable form.

In a neural network model processing method, device and electronic device provided by the present application, after acquiring a neural network model to be configured, the neural network model is mapped to a graph structure based on the dependencies of operators in the neural network model , wherein a node in the graph structure represents an operator in the neural network model. Then, the graph structure is traversed to obtain the execution order of the nodes, and the execution order of the operators represented by the nodes is configured based on the execution order of the nodes. Therefore, by converting the neural network model into a graph structure, and then by traversing the nodes in the graph structure to determine the execution order of the operators represented by each node, it is possible to quickly determine the execution order of all operators and improve the overall performance. The computational efficiency of the model.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand: it can still be Modifications are made to the technical solutions described in the foregoing embodiments, or some technical features thereof are equivalently replaced; and these modifications or replacements do not drive the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims (10)

1. a neural network model processing method, is characterized in that, is applied to electronic equipment, and described method comprises: Get the neural network model to be configured; Based on the dependencies of the operators in the neural network model, the neural network model is mapped into a graph structure, wherein a node in the graph structure represents an operator in the neural network model; Traversing the graph structure to obtain the execution order of the nodes; The execution order of the operators represented by the node is configured based on the execution order of the node. 2. The method according to claim 1, wherein the graph structure is a directed acyclic graph; the step of traversing the graph structure to obtain the execution order of the nodes comprises: A depth-first traversal is performed on the directed acyclic graph to obtain the execution order of the nodes. 3. The method according to claim 2, wherein the step of performing depth-first traversal on the directed acyclic graph to obtain the execution order of the nodes comprises: Depth-first traversal is performed on the directed acyclic graph, and the target nodes obtained in the traversal process are sequentially stored in the storage space of the stack structure, where the target nodes are nodes without subordinate nodes or nodes whose subordinate nodes are all traversed; The popping order of the nodes in the storage space is taken as the execution order of the nodes. 4 . The method according to claim 1 , wherein the directed acyclic graph has a plurality of input vertices, and the depth-first traversal of the directed acyclic graph is performed to obtain the execution order of the nodes. 5 . The steps include: Based on the specified initial input vertex, depth-first traversal is performed on the directed acyclic graph to obtain the execution order of the nodes. 5. The method according to claim 1, wherein the step of acquiring the neural network model to be configured comprises: Receive the trained neural network model; The neural network model obtained by the training is optimized to obtain a neural network model to be configured; wherein, the neural network model to be configured is a neural network model adapted to the data computing capability of the electronic device. 6. The method according to claim 5, wherein the step of optimizing the neural network model obtained by the training at least comprises at least one of the following steps: Perform operator fusion on the neural network model obtained by the training; Perform network pruning on the neural network model obtained by the training; performing model quantization on the trained neural network model; and Perform network cutting on the neural network model obtained by the training. 7. The method according to claim 5, wherein the step of optimizing the neural network model obtained by the training to obtain the neural network model to be configured comprises: A target operator is added after specifying the execution order of the operators in the neural network model obtained by training to obtain the neural network model to be configured. The target operator is used to convert the storage format of the data obtained by executing the specified operator into a target. Format. 8. A neural network model processing device, characterized in that, running on an electronic device, the device comprising: a model obtaining unit, used to obtain the neural network model to be configured; A model processing unit, configured to map the neural network model to a graph structure based on the operator dependency in the neural network model, wherein a node in the graph structure represents an operator in the neural network model. son; a traversal unit, used for traversing the graph structure to obtain the execution order of the nodes; An order determination unit, configured to configure the execution order of the operators represented by the node based on the execution order of the node. 9. An electronic device, comprising a processor and a memory; One or more programs are stored in the memory and configured to be executed by the processor to implement the method of any of claims 1-7. 10. A computer-readable storage medium, wherein a program code is stored in the computer-readable storage medium, wherein when the program code is executed by a processor, any one of claims 1-7 is executed method.