CN117521737B - Network model conversion method, device, terminal and computer readable storage medium - Google Patents
Network model conversion method, device, terminal and computer readable storage medium Download PDFInfo
- Publication number
- CN117521737B CN117521737B CN202410012640.8A CN202410012640A CN117521737B CN 117521737 B CN117521737 B CN 117521737B CN 202410012640 A CN202410012640 A CN 202410012640A CN 117521737 B CN117521737 B CN 117521737B
- Authority
- CN
- China
- Prior art keywords
- time consumption
- operation node
- layer structure
- network model
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000006243 chemical reaction Methods 0.000 title claims abstract description 69
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000012545 processing Methods 0.000 claims abstract description 75
- 238000001514 detection method Methods 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 5
- 238000004458 analytical method Methods 0.000 claims description 4
- 230000007704 transition Effects 0.000 claims description 4
- 230000001186 cumulative effect Effects 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 7
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Abstract
The application provides a method, a device, a terminal and a computer readable storage medium for converting a network model, wherein the method for converting the network model comprises the following steps: acquiring a graph network corresponding to the network model; the graph network includes a plurality of connected layer structures; the layer structure comprises a plurality of parallel operation nodes, and the operation nodes respectively contained in the layer structure which are connected with each other; the operation nodes have operation time consumption, and two operation nodes connected with each other have corresponding conversion time consumption; determining the processing time consumption of the current operation node based on the operation time consumption of the current operation node, the historical accumulated time consumption and the conversion time consumption between the current operation node and the operation node in the adjacent subsequent layer structure; and determining a connection structure corresponding to the network model based on the time consumption of processing of each operation node in the last layer structure. The application can determine the connection structure of the operation node with the shortest time consumption and improve the performance of the network model.
Description
Technical Field
The present invention relates to the field of computer science and technology, and in particular, to a method, an apparatus, a terminal, and a computer readable storage medium for converting a network model.
Background
With the continuous development of artificial intelligence technology, deep learning neural networks are increasingly widely used. The neural network model application is generally divided into a training stage and a deployment stage, wherein the training stage uses a training framework (such as Caffe, tensorFlow) to learn and update weight data on sample data, the deployment stage generally converts an original model into a model which can be identified by a target platform, after the target platform reads information of the model, a data structure which is required to be operated by reasoning is constructed, and then network reasoning is carried out and a reasoning result is obtained for further processing.
In the deployment stage, because the deployed platforms are different and the output parameters are different, the whole network performance is poor when the network model is deployed on the target platform.
Disclosure of Invention
The invention mainly solves the technical problem of providing a method, a device, a terminal and a computer readable storage medium for converting a network model, and solves the problem of poor performance of the network model in the prior art.
In order to solve the technical problems, the first technical scheme adopted by the invention is as follows: the utility model provides a conversion method of a network model, wherein the network model is deployed on a target platform, and the conversion method comprises the following steps:
Acquiring a graph network corresponding to the network model; the graph network includes a plurality of connected layer structures; the layer structure comprises a plurality of parallel operation nodes, and the operation nodes respectively contained in the layer structure which are connected with each other; the operation nodes have operation time consumption, and two operation nodes connected with each other have corresponding conversion time consumption;
Determining the processing time consumption of the current operation node based on the operation time consumption of the current operation node, the historical accumulated time consumption and the conversion time consumption between the current operation node and the operation node in the adjacent subsequent layer structure; the historical accumulated time consumption is the processing time consumption of the operation node in the previous layer structure which is adjacently connected with the current operation node;
and determining a connection structure corresponding to the network model based on the time consumption of processing of each operation node in the last layer structure.
The historical accumulated time consumption is the processing time consumption with the minimum value in each operation node in the previous layer structure which is adjacently connected with the current operation node.
The method for determining the processing time consumption of the current operation node based on the operation time consumption of the current operation node, the historical accumulated time consumption and the conversion time consumption between the current operation node and the operation node in the adjacent subsequent layer structure comprises the following steps:
Determining initial accumulated time consumption of the current operation node based on the historical accumulated time consumption of the current operation node and the operation time consumption of the current operation node;
and determining the processing time consumption of the current operation node based on the initial accumulated time consumption of the current operation node and the conversion time consumption between the current operation node and each operation node in the adjacent subsequent layer structure.
Wherein determining the processing time consumption of the current operation node based on the initial accumulated time consumption of the current operation node and the conversion time consumption between the current operation node and each operation node in the adjacent subsequent layer structure comprises:
Adding the conversion time consumption between each operation node and the current operation node in the later layer structure adjacent to the current operation node with the initial accumulated time consumption of the current operation node respectively to obtain a plurality of candidate accumulated time consumption corresponding to the current operation node;
And selecting the candidate accumulated time consumption with the minimum value from the plurality of candidate accumulated time consumption as the processing time consumption of the current operation node.
The graph network comprises an input layer, wherein the input layer is connected with a plurality of layer structures which are connected in sequence; the input layer is connected with each operation node in the first layer structure; the historical cumulative time consumption of each operation node in the first layer structure is the conversion time consumption between the operation node and the input layer.
The graph network comprises an output layer, and a plurality of layer structures which are sequentially connected with the output layer; each operation node in the last layer structure in the plurality of layer structures connected in sequence is connected with the output layer respectively;
the time-consuming calculation method for processing each operation node in the last layer structure comprises the following steps:
Determining initial accumulated time consumption of each operation node in the last layer structure based on the summation of the operation time consumption corresponding to each operation node in the last layer structure and the historical accumulated time consumption;
and determining the processing time consumption of each operation node in the last layer structure based on the addition of the initial accumulated time consumption of each operation node in the last layer structure and the conversion time consumption between the corresponding operation node and the output layer.
The determining a connection structure corresponding to the network model based on time consumption of processing of each operation node in the last layer structure comprises the following steps:
Comparing the processing time consumption of each operation node in the last layer structure;
selecting an operation node corresponding to the processing time consumption with the minimum value as an optimal operation node of the last layer structure;
Traversing each layer of structure, and taking the operation node corresponding to the historical accumulated time consumption as the optimal operation node of the previous layer of structure based on the historical accumulated time consumption corresponding to the optimal operation node of the current layer of structure;
and connecting the optimal operation nodes corresponding to the structures of each layer respectively to obtain a connection structure of the network model.
In order to solve the technical problems, a second technical scheme adopted by the invention is as follows: there is provided a conversion device of a network model, the network model being deployed on a target platform, the conversion device of the network model comprising:
the acquisition module is used for acquiring a graph network corresponding to the network model; the graph network includes a plurality of connected layer structures; the layer structure comprises a plurality of parallel operation nodes, and the operation nodes respectively contained in the layer structure which are connected with each other; the operation nodes have operation time consumption, and two operation nodes connected with each other have corresponding conversion time consumption;
the analysis module is used for determining the processing time consumption of the current operation node based on the operation time consumption of the current operation node, the historical accumulated time consumption and the conversion time consumption between the current operation node and the operation node in the adjacent subsequent layer structure; the historical accumulated time consumption is the processing time consumption of the operation node in the previous layer structure which is adjacently connected with the current operation node;
And the determining module is used for determining the connection structure corresponding to the network model based on the time consumption of processing of each operation node in the last layer structure.
In order to solve the technical problems, a third technical scheme adopted by the invention is as follows: there is provided a terminal comprising a memory, a processor and a computer program stored in the memory and running on the processor, the processor being adapted to execute program data to carry out the steps in a method of converting a network model as described above.
In order to solve the technical problems, a fourth technical scheme adopted by the invention is as follows: there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps in a method of converting a network model as described above.
The beneficial effects of the application are as follows: different from the prior art, the provided method, device, terminal and computer readable storage medium for converting the network model are provided, the network model is deployed on a target platform, and the method for converting the network model comprises the following steps: acquiring a graph network corresponding to the network model; the graph network includes a plurality of connected layer structures; the layer structure comprises a plurality of parallel operation nodes, and the operation nodes respectively contained in the layer structure which are connected with each other; the operation nodes have operation time consumption, and two operation nodes connected with each other have corresponding conversion time consumption; determining the processing time consumption of the current operation node based on the operation time consumption of the current operation node, the historical accumulated time consumption and the conversion time consumption between the current operation node and the operation node in the adjacent subsequent layer structure; the historical accumulated time consumption is the processing time consumption of the operation node in the previous layer structure which is adjacently connected with the current operation node; and determining a connection structure corresponding to the network model based on the time consumption of processing of each operation node in the last layer structure. According to the application, the processing time consumption of the current operation node is determined through the operation time consumption of the operation node, the historical accumulated time consumption and the conversion time consumption between the operation node and the operation node in the next layer structure, the optimal operation node in each layer structure in the network model is selected based on the processing time consumption of the operation node between each layer structure, so that the connection structure corresponding to the network model is obtained, the connection structure of the operation node with the shortest time consumption is determined, and the performance of the network model is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of a method for converting a network model according to the present invention;
FIG. 2 is a pictorial network of a network model provided by the present invention;
FIG. 3 is a flowchart illustrating a step S2 in the method for converting the network model shown in FIG. 1 according to an embodiment;
FIG. 4 is a flowchart illustrating a step S3 in the method for converting the network model shown in FIG. 1 according to an embodiment;
FIG. 5 is a schematic diagram of a network model conversion device according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of a frame of an embodiment of a terminal provided by the present invention;
FIG. 7 is a schematic diagram of a computer-readable storage medium according to an embodiment of the present invention.
Detailed Description
The following describes embodiments of the present application in detail with reference to the drawings.
In the following description, for purposes of explanation and not limitation, specific details are set forth such as the particular system architecture, interfaces, techniques, etc., in order to provide a thorough understanding of the present application.
The term "and/or" is herein merely an association relationship describing an associated object, meaning that there may be three relationships, e.g., a and/or B, may represent: a exists alone, A and B exist together, and B exists alone. In addition, the character "/" herein generally indicates that the front and rear associated objects are an "or" relationship. Further, "a plurality" herein means two or more than two.
In order to enable those skilled in the art to better understand the technical solution of the present invention, the following describes in further detail a network model conversion method provided by the present invention with reference to the accompanying drawings and the detailed description.
Referring to fig. 1, fig. 1 is a flow chart of a method for converting a network model according to the present invention.
In this embodiment, a method for converting a network model is provided, where the method embodiment provided in the embodiment of the present application may be executed in a mobile terminal, a computer terminal, or a similar computing device, and the network model is deployed on a target platform. The conversion method of the network model comprises the following steps.
S1: acquiring a graph network corresponding to the network model; the graph network includes a plurality of connected layer structures; the layer structure comprises a plurality of parallel operation nodes, and the operation nodes respectively contained in the layer structure which are connected with each other; the operation nodes have operation time consumption, and two operation nodes connected with each other have corresponding conversion time consumption.
S2: determining the processing time consumption of the current operation node based on the operation time consumption of the current operation node, the historical accumulated time consumption and the conversion time consumption between the current operation node and the operation node in the adjacent subsequent layer structure; the historical accumulated time consumption is the processing time consumption of the operation node in the previous layer structure connected with the current operation node adjacently.
S3: and determining a connection structure corresponding to the network model based on the time consumption of processing of each operation node in the last layer structure.
Specifically, the specific embodiment of the graph network corresponding to the network model obtained in step S1 is as follows.
In this embodiment, the network model may be a text recognition model, a speech recognition model, an image detection model, or a video detection model.
Referring to fig. 2, fig. 2 is a diagram of a network model according to the present invention.
The network model comprises an input layer, M network layers and an output layer which are sequentially connected in a cascading manner. Each network layer comprises N operators in parallel, and the N operators refer to N processing modes which can be performed on input data in the network layer. Operators within the same network layer are not connected. The operators in two interconnected network layers are interconnected, i.e. the operators between two different network layers having a connection relationship are interconnected. Since the data type and arrangement of the output information of the operators in the previous network layer may be different from the data type and arrangement of the input information of the operators in the other network layer connected, a conversion layer is arranged between the two operators connected with each other. The conversion layer converts the data type and arrangement of the output information of the operator in the previous network layer into the data type and arrangement of the input information of the operator in the other network layer connected with the conversion layer, specifically, the time consumed for converting the data type and arrangement of the output information of the operator in the previous network layer into the data type and arrangement of the input information of the operator in the other network layer connected with the conversion layer is called conversion time consumption. The time taken for each operator to process the input information is referred to as the operational time consuming. The value of M is a positive integer greater than 2, and the value of N is a positive integer greater than 1; or the value of M is a positive integer greater than 1, and the value of N is a positive integer greater than 2.
The network layer and operators in each network layer in the network model are abstracted, the network layer is abstracted into a layer structure, the operators are abstracted into operation nodes, and the conversion layer is abstracted into conversion nodes. M sequentially connected layer structures, N operation nodes in each layer structure, an input layer and an output layer form a graph network of the network model.
Specifically, the input layer is connected with each operation node in a first layer structure in the M layer structures which are sequentially connected, and conversion nodes are respectively arranged between the input layer and each operation node in the first layer structure. Each operation node in the first layer structure is connected with each operation node in the second layer structure respectively, and a conversion node is arranged between the two operation nodes which are connected with each other. The operation nodes in the two layer structures with the connection relationship are sequentially connected. Each operation node in the last layer structure in the M sequentially connected layer structures is respectively connected with the output layer, and a conversion node is respectively arranged between the output layer and each operation node in the last layer structure. As shown in fig. 2, rectangular frames in the layer structure corresponding to the network model are operation nodes, and connection lines between the operation nodes are conversion nodes.
Specifically, a connection structure corresponding to the network model is determined through a Viterbi algorithm.
Referring to fig. 3, fig. 3 is a flowchart illustrating an embodiment of step S2 in the method for converting a network model provided in fig. 1.
Specifically, the specific embodiment of determining that the processing of the current operation node is time-consuming in step S2 is as follows.
In one embodiment, the historical cumulative time consumption of each operation node in a first one of the M sequentially connected layer structures is the transition time consumption between the operation node and the input layer.
The historical accumulated time consumption of each operation node in other layers except the first layer is the minimum processing time consumption in each operation node in the previous layer adjacent to the current operation node. For example, if the processing time consumption of the ith operation node in the first layer structure is the smallest, the historical accumulated time consumption of each operation node in the second layer structure is the processing time consumption of the ith operation node.
S21: and determining the initial accumulated time consumption of the current operation node based on the historical accumulated time consumption of the current operation node and the operation time consumption of the current operation node.
Specifically, an initial accumulated time consumption of the current operational node is determined based on a summation of the historical accumulated time consumption of the current operational node and the operational time consumption of the current operational node.
S22: and determining the processing time consumption of the current operation node based on the initial accumulated time consumption of the current operation node and the conversion time consumption between the current operation node and each operation node in the adjacent subsequent layer structure.
Specifically, adding the conversion time consumption between each operation node and the current operation node in the later layer structure adjacent to the current operation node with the initial accumulated time consumption of the current operation node respectively to obtain a plurality of candidate accumulated time consumption corresponding to the current operation node; and selecting the candidate accumulated time consumption with the minimum value from the plurality of candidate accumulated time consumption as the processing time consumption of the current operation node.
For example, the initial accumulated time consumption of the ith operation node in the first layer structure is summed with the corresponding conversion time consumption between each operation node in the connected second layer structure, so as to obtain a plurality of candidate accumulated time consumption corresponding to the ith operation node. And selecting the candidate accumulated time consumption with the shortest time from the plurality of candidate accumulated time consumption as the processing time consumption of the ith operation node. The processing time consumption of each operation node in the first layer structure is determined through the method.
Comparing the processing time consumption of each operation node in the first layer structure, and selecting the shortest processing time consumption as the historical accumulated time consumption of the second layer structure. The historical accumulated time consumption of the second layer structure has a corresponding relation with the operation node in the first layer structure. That is, each operation node in the second layer structure is associated with the operation node corresponding to the shortest time of processing time in the first layer structure.
Referring to fig. 4, fig. 4 is a flowchart illustrating an embodiment of step S3 in the method for converting a network model provided in fig. 1.
Specifically, the specific embodiment of determining the connection structure corresponding to the network model in step S3 is as follows.
S31: the processing of each operational node in the last layer structure is time consuming.
Specifically, determining initial accumulated time consumption of each operation node in the last layer structure based on the sum of the operation time consumption corresponding to each operation node in the last layer structure and the historical accumulated time consumption; and determining the processing time consumption of each operation node in the last layer structure based on the addition of the initial accumulated time consumption of each operation node in the last layer structure and the conversion time consumption between the corresponding operation node and the output layer.
An optimal operation node is selected from each layer structure according to the connection relation among the layer structures by the following method.
S32: the processing time of each operational node in the last layer structure is compared.
S33: and selecting the operation node corresponding to the processing time consumption with the minimum value as the optimal operation node of the last layer structure.
Specifically, in order to optimize the network model so that the network model has the lowest time consumption, an operation node corresponding to the processing time consumption with the smallest numerical value is selected as the optimal operation node of the last layer structure.
S34: traversing each layer of structure, and taking the operation node corresponding to the historical accumulated time consumption as the optimal operation node of the previous layer of structure based on the historical accumulated time consumption corresponding to the optimal operation node of the current layer of structure.
S35: and connecting the optimal operation nodes corresponding to the structures of each layer respectively to obtain a connection structure of the network model.
Specifically, the optimal operation nodes corresponding to each layer structure are connected through the corresponding conversion nodes to serve as optimal connection structures corresponding to the M layer structures, the input layer is connected with the first optimal operation node in the optimal connection structures through the corresponding conversion nodes, and the output layer is connected with the last operation node in the optimal connection structures through the corresponding conversion nodes.
The method optimizes to obtain the optimal operation nodes in each layer structure and the conversion nodes among the operation nodes, writes the optimal operation nodes and the conversion nodes among the operation nodes into file packets or parameters, stores the file packets or parameters to a target platform, and is convenient for calling the file packets or parameters to process input information when the network model is operated, so that the processing time is reduced, and the performance of the model is improved.
The conversion method of the network model provided by the embodiment comprises the following steps: acquiring a graph network corresponding to the network model; the graph network includes a plurality of connected layer structures; the layer structure comprises a plurality of parallel operation nodes, and the operation nodes respectively contained in the layer structure which are connected with each other; the operation nodes have operation time consumption, and two operation nodes connected with each other have corresponding conversion time consumption; determining the processing time consumption of the current operation node based on the operation time consumption of the current operation node, the historical accumulated time consumption and the conversion time consumption between the current operation node and the operation node in the adjacent subsequent layer structure; the historical accumulated time consumption is the processing time consumption of the operation node in the previous layer structure which is adjacently connected with the current operation node; and determining a connection structure corresponding to the network model based on the time consumption of processing of each operation node in the last layer structure. According to the application, the processing time consumption of the current operation node is determined through the operation time consumption of the operation node, the historical accumulated time consumption and the conversion time consumption between the operation node and the operation node in the next layer structure, the optimal operation node in each layer structure in the network model is selected based on the processing time consumption of the operation node between each layer structure, so that the connection structure corresponding to the network model is obtained, the connection structure of the operation node with the shortest time consumption is determined, and the performance of the network model is improved.
Referring to fig. 5, fig. 5 is a schematic diagram of a frame of a conversion device of a network model according to an embodiment of the invention.
The present embodiment provides a conversion device 60 of a network model, where the conversion device 60 of a network model includes an acquisition module 61, an analysis module 62, and a determination module 63.
The obtaining module 61 is configured to obtain a graph network corresponding to the network model; the graph network includes a plurality of connected layer structures; the layer structure comprises a plurality of parallel operation nodes, and the operation nodes respectively contained in the layer structure which are connected with each other; the operation nodes have operation time consumption, and two operation nodes connected with each other have corresponding conversion time consumption.
The analysis module 62 is configured to determine a processing time consumption of the current operation node based on the operation time consumption of the current operation node, the historical accumulated time consumption, and the transition time consumption between the current operation node and the operation node in the adjacent subsequent layer structure; the historical accumulated time consumption is the processing time consumption of the operation node in the previous layer structure connected with the current operation node adjacently.
The determining module 63 is configured to determine a connection structure corresponding to the network model based on time consumed in processing each operation node in the last layer structure.
According to the conversion device of the network model, the processing time consumption of the current operation node is determined through the operation time consumption and the historical accumulated time consumption of the operation node and the conversion time consumption between the operation node and the operation node in the next layer structure, the optimal operation node in each layer structure in the network model is selected based on the processing time consumption of the operation node between each layer structure, the connection structure corresponding to the network model is further obtained, the connection structure of the operation node with the shortest time consumption is determined, and the performance of the network model is improved.
Referring to fig. 6, fig. 6 is a schematic diagram of a frame of an embodiment of a terminal according to the present invention.
The terminal 80 comprises a memory 81 and a processor 82 coupled to each other, the processor 82 being adapted to execute program instructions stored in the memory 81 for implementing the steps of the method embodiment of the conversion method of any of the network models described above. In one particular implementation scenario, terminal 80 may include, but is not limited to: the microcomputer, server, and the terminal 80 may also include mobile devices such as a notebook computer and a tablet computer, which are not limited herein.
In particular, the processor 82 is configured to control itself and the memory 81 to implement the steps of the conversion method embodiment of any of the network models described above. The processor 82 may also be referred to as a CPU (Central Processing Unit ). The processor 82 may be an integrated circuit chip having signal processing capabilities. The Processor 82 may also be a general purpose Processor, a digital signal Processor (DIGITAL SIGNAL Processor, DSP), an Application SPECIFIC INTEGRATED Circuit (ASIC), a Field-Programmable gate array (Field-Programmable GATE ARRAY, FPGA) or other Programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. In addition, the processor 82 may be commonly implemented by an integrated circuit chip.
Referring to fig. 7, fig. 7 is a schematic diagram of a frame of an embodiment of a computer readable storage medium according to the present invention.
The computer readable storage medium 90 stores program instructions 901 executable by a processor, the program instructions 901 for implementing the steps of the conversion method embodiment of any one of the network models described above.
In some embodiments, functions or modules included in an apparatus provided by the embodiments of the present disclosure may be used to perform a method described in the foregoing method embodiments, and specific implementations thereof may refer to descriptions of the foregoing method embodiments, which are not repeated herein for brevity.
The foregoing description of various embodiments is intended to highlight differences between the various embodiments, which may be the same or similar to each other by reference, and is not repeated herein for the sake of brevity.
In the several embodiments provided in the present application, it should be understood that the disclosed method and apparatus may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of modules or units is merely a logical functional division, and there may be additional divisions of actual implementation, e.g., units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical, or other forms.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a storage medium, including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) to execute all or part of the steps of the methods of the embodiments of the present application. And the aforementioned storage medium includes: a usb disk, a removable hard disk, a read-only memory (ROM), a random access memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The foregoing is only the embodiments of the present invention, and therefore, the patent protection scope of the present invention is not limited thereto, and all equivalent structures or equivalent flow changes made by the content of the present specification and the accompanying drawings, or direct or indirect application in other related technical fields, are included in the patent protection scope of the present invention.
Claims (8)
1. A method for converting a network model, wherein the network model is deployed on a target platform, the network model including any one of a text recognition model, a speech recognition model, an image detection model, and a video detection model, the method comprising:
Acquiring a graph network corresponding to the network model; the graph network includes a plurality of connected layer structures; the layer structure comprises a plurality of parallel operation nodes, and the operation nodes respectively contained in the layer structure which are connected with each other; the operation nodes have operation time consumption, and two operation nodes connected with each other have corresponding conversion time consumption;
Determining processing time consumption of a current operation node based on operation time consumption, historical accumulated time consumption of the current operation node and the conversion time consumption between the current operation node and the operation node in the adjacent subsequent layer structure; the historical accumulated time consumption of the current operation node is the processing time consumption with the minimum value in each operation node in the previous layer structure which is adjacently connected with the current operation node;
Determining a connection structure corresponding to the network model based on the time consumption of processing of each operation node in the last layer structure;
the determining the connection structure corresponding to the network model based on the time consumption of processing each operation node in the last layer structure includes:
comparing the processing time consumption of each operation node in the last layer structure;
selecting the operation node corresponding to the processing time consumption with the smallest value as the optimal operation node of the last layer structure;
Traversing each layer structure, and taking the operation node corresponding to the historical accumulated time consumption as the optimal operation node of the previous layer structure of the current layer structure based on the historical accumulated time consumption corresponding to the optimal operation node of the current layer structure;
and connecting the optimal operation nodes corresponding to the layer structures respectively to obtain a connection structure of the network model.
2. The method for converting network model according to claim 1, wherein,
The determining the processing time of the current operation node based on the operation time of the current operation node, the historical accumulated time and the conversion time between the current operation node and the operation node in the layer structure of the next and subsequent operation nodes comprises:
determining initial accumulated time consumption of the current operation node based on the historical accumulated time consumption of the current operation node and the operation time consumption of the current operation node;
determining the processing time consumption of the current operation node based on the initial accumulated time consumption of the current operation node and the conversion time consumption between the current operation node and each operation node in the adjacent subsequent layer structure.
3. The method for converting network model according to claim 2, wherein,
The determining the processing time consumption of the current operation node based on the initial accumulated time consumption of the current operation node and the transition time consumption between the current operation node and each operation node in the layer structure next to the current operation node comprises:
Adding the conversion time consumption between each operation node and the current operation node in the later layer structure adjacent to the current operation node with the initial accumulated time consumption of the current operation node respectively to obtain a plurality of candidate accumulated time consumption corresponding to the current operation node;
selecting the candidate accumulated time consumption with the minimum value from the plurality of candidate accumulated time consumption as the processing time consumption of the current operation node.
4. The method for converting a network model according to claim 1, wherein the graph network comprises an input layer, the input layer being connected to the plurality of sequentially connected layer structures; the input layer is connected with each operation node in the first layer structure; the historical cumulative time consumption of each of the operation nodes in the first one of the layer structures is the transition time consumption between the operation node and the input layer.
5. The method for converting a network model according to claim 1 or 4, wherein the graph network includes an output layer, and the plurality of sequentially connected layer structures are sequentially connected to the output layer; each operation node in the last layer structure in the plurality of layer structures connected in sequence is connected with the output layer respectively;
the calculation method for determining the processing time consumption of each operation node in the last layer structure comprises the following steps:
Determining initial accumulated time consumption of each operation node in the last layer structure based on the sum of the operation time consumption corresponding to each operation node in the last layer structure and the historical accumulated time consumption;
determining the processing time consumption of each operation node in the last layer structure based on the initial accumulated time consumption of each operation node in the last layer structure and the summation of the conversion time consumption between the corresponding operation node and the output layer.
6. A conversion device of a network model, wherein the network model is deployed on a target platform, the network model includes any one of a text recognition model, a voice recognition model, an image detection model, and a video detection model, and the conversion device of the network model includes:
The acquisition module is used for acquiring a graph network corresponding to the network model; the graph network includes a plurality of connected layer structures; the layer structure comprises a plurality of parallel operation nodes, and the operation nodes respectively contained in the layer structure which are connected with each other; the operation nodes have operation time consumption, and two operation nodes connected with each other have corresponding conversion time consumption;
The analysis module is used for determining the processing time consumption of the current operation node based on the operation time consumption of the current operation node, the historical accumulated time consumption and the conversion time consumption between the current operation node and the operation node in the adjacent subsequent layer structure; the historical accumulated time consumption of the current operation node is the processing time consumption with the minimum value in each operation node in the previous layer structure which is adjacently connected with the current operation node;
the determining module is used for determining a connection structure corresponding to the network model based on the time consumption of processing of each operation node in the last layer structure; comparing the processing time consumption of each operation node in the last layer structure; selecting the operation node corresponding to the processing time consumption with the smallest value as the optimal operation node of the last layer structure; traversing each layer structure, and taking the operation node corresponding to the historical accumulated time consumption as the optimal operation node of the previous layer structure of the current layer structure based on the historical accumulated time consumption corresponding to the optimal operation node of the current layer structure; and connecting the optimal operation nodes corresponding to the layer structures respectively to obtain a connection structure of the network model.
7. A terminal comprising a memory, a processor and a computer program stored in the memory and running on the processor, the processor being configured to execute program data to implement the steps in the method for converting a network model according to any one of claims 1 to 5.
8. A computer readable storage medium, wherein a computer program is stored on the computer readable storage medium, and the computer program when executed by a processor implements the steps in the method for converting a network model according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410012640.8A CN117521737B (en) | 2024-01-04 | Network model conversion method, device, terminal and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410012640.8A CN117521737B (en) | 2024-01-04 | Network model conversion method, device, terminal and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117521737A CN117521737A (en) | 2024-02-06 |
| CN117521737B true CN117521737B (en) | 2024-04-19 |
Family
ID=
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109491784A (en) * | 2018-10-18 | 2019-03-19 | 北京旷视科技有限公司 | Reduce method, apparatus, the electronic equipment, readable storage medium storing program for executing of EMS memory occupation amount |
| CN114896950A (en) * | 2022-07-11 | 2022-08-12 | 浙江大华技术股份有限公司 | Model conversion method, model conversion device, and storage medium |
| CN115049057A (en) * | 2022-08-11 | 2022-09-13 | 浙江大华技术股份有限公司 | Model deployment method and device, electronic equipment and storage medium |
| CN115640851A (en) * | 2022-09-23 | 2023-01-24 | 中国电子科技集团公司第四十一研究所 | Neural network efficient reasoning method suitable for test instrument |
| WO2023082644A1 (en) * | 2021-11-15 | 2023-05-19 | 上海商汤智能科技有限公司 | Network model processing method and apparatus, and device, storage medium and computer program product |
| CN116266273A (en) * | 2021-12-15 | 2023-06-20 | 北京嘀嘀无限科技发展有限公司 | Neural network generation method, neural network generation device, neural network image processing device and storage medium |
| CN116306821A (en) * | 2023-03-28 | 2023-06-23 | 深圳开鸿数字产业发展有限公司 | Model performance prediction method, device and storage medium |
| CN116306866A (en) * | 2023-02-24 | 2023-06-23 | 浙江大华技术股份有限公司 | Deployment method of network model, computer equipment and storage medium |
| CN117217280A (en) * | 2022-05-31 | 2023-12-12 | 华为技术有限公司 | Neural network model optimization method and device and computing equipment |
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109491784A (en) * | 2018-10-18 | 2019-03-19 | 北京旷视科技有限公司 | Reduce method, apparatus, the electronic equipment, readable storage medium storing program for executing of EMS memory occupation amount |
| WO2023082644A1 (en) * | 2021-11-15 | 2023-05-19 | 上海商汤智能科技有限公司 | Network model processing method and apparatus, and device, storage medium and computer program product |
| CN116266273A (en) * | 2021-12-15 | 2023-06-20 | 北京嘀嘀无限科技发展有限公司 | Neural network generation method, neural network generation device, neural network image processing device and storage medium |
| CN117217280A (en) * | 2022-05-31 | 2023-12-12 | 华为技术有限公司 | Neural network model optimization method and device and computing equipment |
| CN114896950A (en) * | 2022-07-11 | 2022-08-12 | 浙江大华技术股份有限公司 | Model conversion method, model conversion device, and storage medium |
| CN115049057A (en) * | 2022-08-11 | 2022-09-13 | 浙江大华技术股份有限公司 | Model deployment method and device, electronic equipment and storage medium |
| CN115640851A (en) * | 2022-09-23 | 2023-01-24 | 中国电子科技集团公司第四十一研究所 | Neural network efficient reasoning method suitable for test instrument |
| CN116306866A (en) * | 2023-02-24 | 2023-06-23 | 浙江大华技术股份有限公司 | Deployment method of network model, computer equipment and storage medium |
| CN116306821A (en) * | 2023-03-28 | 2023-06-23 | 深圳开鸿数字产业发展有限公司 | Model performance prediction method, device and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| An Empirical Study of Challenges in Converting Deep Learning Models;Moses Openja et al.;《2022 IEEE International Conference on Software Maintenance and Evolution》;20221231;第13-23页 * |
| 面向深度神经网络模型的性能优化技术研究;马秀;《中国博士学位论文全文数据库 信息科技辑》;20231215;I140-8 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11575502B2 (en) | Homomorphic encryption processing device, system including the same and method of performing homomorphic encryption processing | |
| US20230401833A1 (en) | Method, computer device, and storage medium, for feature fusion model training and sample retrieval | |
| CN113435509A (en) | Small sample scene classification and identification method and system based on meta-learning | |
| KR20190125029A (en) | Methods and apparatuses for generating text to video based on time series adversarial neural network | |
| CN113988464A (en) | Network link attribute relation prediction method and equipment based on graph neural network | |
| CN115345301A (en) | Model training method and device based on mixed disturbance, storage medium and server | |
| CN111325340A (en) | Information network relation prediction method and system | |
| CN108470251B (en) | Community division quality evaluation method and system based on average mutual information | |
| JP7063274B2 (en) | Information processing equipment, neural network design method and program | |
| CN112418256A (en) | Classification, model training and information searching method, system and equipment | |
| CN108234195B (en) | Method, apparatus, device, medium for predicting network performance | |
| CN111767985B (en) | Neural network training method, video identification method and device | |
| CN117521737B (en) | Network model conversion method, device, terminal and computer readable storage medium | |
| CN112396166A (en) | Graph convolution neural network training method and device based on mixed granularity aggregator | |
| CN115001937B (en) | Smart city Internet of things-oriented fault prediction method and device | |
| CN109697511B (en) | Data reasoning method and device and computer equipment | |
| CN115984742A (en) | Training method of video frame selection model, video processing method and device | |
| CN115809688B (en) | Model debugging method and device, electronic equipment and storage medium | |
| CN116049691A (en) | Model conversion method, device, electronic equipment and storage medium | |
| CN117521737A (en) | Network model conversion method, device, terminal and computer readable storage medium | |
| CN113128696A (en) | Distributed machine learning communication optimization method and device, server and terminal equipment | |
| CN113905066B (en) | Networking method of Internet of things, networking device of Internet of things and electronic equipment | |
| CN115983372A (en) | Neural network training method and device, computing equipment and storage medium | |
| CN113301673B (en) | Distributed filtering method, device, equipment and storage medium for wireless sensor network | |
| CN114021705A (en) | Model accuracy determination method, related device and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |