CN112598117A - Neural network model design method, deployment method, electronic device and storage medium - Google Patents
Neural network model design method, deployment method, electronic device and storage medium Download PDFInfo
- Publication number
- CN112598117A CN112598117A CN202011598884.7A CN202011598884A CN112598117A CN 112598117 A CN112598117 A CN 112598117A CN 202011598884 A CN202011598884 A CN 202011598884A CN 112598117 A CN112598117 A CN 112598117A
- Authority
- CN
- China
- Prior art keywords
- neural network
- target
- candidate
- network model
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003062 neural network model Methods 0.000 title claims abstract description 177
- 238000000034 method Methods 0.000 title claims abstract description 86
- 238000013461 design Methods 0.000 title claims abstract description 54
- 238000013528 artificial neural network Methods 0.000 claims abstract description 305
- 230000006870 function Effects 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 8
- 238000010276 construction Methods 0.000 claims description 7
- 230000008569 process Effects 0.000 description 12
- 238000011161 development Methods 0.000 description 7
- 230000018109 developmental process Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 7
- 230000011218 segmentation Effects 0.000 description 7
- 238000004891 communication Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 5
- 238000012360 testing method Methods 0.000 description 5
- 238000001514 detection method Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 239000000306 component Substances 0.000 description 3
- 238000012938 design process Methods 0.000 description 3
- 238000004821 distillation Methods 0.000 description 3
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 description 2
- 230000001537 neural effect Effects 0.000 description 2
- 238000013138 pruning Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000012827 research and development Methods 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 239000008358 core component Substances 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011002 quantification Methods 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 230000011664 signaling Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a neural network model design method, a deployment method, electronic equipment and a storage medium, wherein the neural network model design method comprises the following steps: acquiring a neural network module library; the neural network module library comprises a plurality of neural network modules; determining a plurality of target neural network modules matched with the scene tasks from a neural network module library according to the scene tasks; and then constructing a target neural network model matched with the scene task according to the plurality of target neural network modules. The invention greatly reduces the time cost of network model design and improves the efficiency of model design.
Description
Technical Field
The invention relates to the technical field of deep learning, in particular to a neural network model design method, a deployment method, electronic equipment and a storage medium.
Background
The deep neural network has achieved great success in many fields, and at present, the deep neural network has complex and various structures and large design space, and high labor cost is required for manual design, so that how to find the optimal neural network structure is a technical difficulty.
At present, the most appropriate model structure is searched out by adopting a scheme based on neural network search in the related technology, and the method generally needs to reinitialize the neural network search space for each new scene requirement for searching again, so that the time consumption is huge, and the problem of low search efficiency exists.
Disclosure of Invention
An objective of the present invention is to provide a neural network model design method, a neural network model deployment method, an electronic device, and a storage medium, so as to reduce the time consumption for designing a neural network model and improve the efficiency of obtaining an optimal neural network model.
The technical scheme of the invention can be realized as follows:
in a first aspect, the present invention provides a neural network model design method, including: acquiring a neural network module library; the neural network module library comprises a plurality of neural network modules; determining a plurality of target neural network modules matched with the scene tasks from the neural network module library according to the scene tasks; and constructing a target neural network model matched with the scene task according to the plurality of target neural network modules.
In a second aspect, the present invention provides a neural network model deployment method, including: acquiring a target neural network model; and when the performance index of the target neural network model on a target hardware platform is within a set performance index range, deploying the target neural network model on the target hardware platform.
In a third aspect, the present invention provides a neural network model designing apparatus, including: the acquisition module is used for acquiring a neural network module library; the neural network module library comprises a plurality of neural network modules; the determining module is used for determining a plurality of target neural network modules matched with the scene tasks from the neural network module library according to the scene tasks; and the construction module is used for constructing a target neural network model matched with the scene task according to the plurality of target neural network modules.
In a fourth aspect, the present invention provides a neural network model deployment apparatus, including: the acquisition module is used for acquiring a target neural network model; and the deployment module is used for deploying the target neural network model on the target hardware platform when the performance index of the target neural network model on the target hardware platform is within the set performance index range.
In a fifth aspect, the present invention provides an electronic device, including a processor and a memory, where the memory stores a computer program executable by the processor, and the processor can execute the machine executable instructions to implement the neural network model design method of the first aspect and/or the neural network model deployment method of the second aspect.
In a sixth aspect, the present invention provides a storage medium having a computer program stored thereon, where the computer program is executed by a processor to implement the neural network model design method of the first aspect and/or the neural network model deployment method of the second aspect.
The invention provides a neural network model design method, a deployment method, electronic equipment and a storage medium, wherein the neural network model design method comprises the following steps: obtaining a neural network module library; the neural network module library comprises a plurality of neural network modules; determining a plurality of target neural network modules matched with the scene tasks from a neural network module library according to the scene tasks; and then constructing a target neural network model matched with the scene task according to the plurality of target neural network modules. The difference from the prior art is that: in the prior art, an optimal neural network model needs to be obtained through a neural network structure searching mode, the time consumption is long, and the searching efficiency is low.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
FIG. 1 is a schematic flow chart of a neural network model design method provided by an embodiment of the present invention;
fig. 2 is a schematic flow chart of an implementation manner of step S13 provided by the embodiment of the present invention;
fig. 3 is a schematic flow chart of an implementation manner of step S132 provided by the embodiment of the present invention;
fig. 4 is a schematic flowchart of an implementation manner of step S131 provided by the embodiment of the present invention;
FIG. 5 is a schematic flow chart of another neural network model design method provided by an embodiment of the present invention;
FIG. 6 is a schematic flow chart of one implementation of step S14 provided by an embodiment of the present invention;
FIG. 7 is a second schematic flowchart of another neural network model design method according to an embodiment of the present invention;
FIG. 8 is a third schematic flowchart of another neural network model design method provided in an embodiment of the present invention;
FIG. 9 is a fourth schematic flowchart of another neural network model design method provided in the embodiments of the present invention;
FIG. 10 is a schematic flow chart of a neural network model deployment method provided by an embodiment of the present invention;
FIG. 11 is a functional block diagram of a neural network model designing apparatus according to an embodiment of the present invention;
fig. 12 is a functional block diagram of a neural network model deployment apparatus according to an embodiment of the present invention;
fig. 13 is a block diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
Before describing the embodiments of the present invention, terms involved in the embodiments of the present invention are explained:
pruning: cutting the number of feature maps of the neural network;
and (3) quantification: representing the neural network operation with a fewer number of bits;
and (3) distillation: a small model with better performance is obtained in a mode that a teacher model (a big model) guides a student model (a small model) to train.
Deep neural networks have been used with great success in many areas, such as natural language processing, speech recognition and computer vision. The trained neural network model can be generally deployed on a selected hardware platform, such as cloud equipment, edge equipment and the like, and the performance of the neural network model can be tested and actual scene tasks can be executed on the hardware platform, such as image recognition, target detection, scene segmentation and the like, and the implementation can be realized on a mobile phone or an embedded terminal.
In order to obtain an optimal neural network model supporting operation on a hardware platform, in the related art, the most appropriate model structure is searched for a specific application or specific hardware through automatic neural network structure search (NAS for short), or a super network is trained first, and then an optimal sub network is searched for each task.
The inventor finds that the method for obtaining the optimal neural network model needs to reinitialize the neural network search space for each new scene task in the research process, and has the problems of huge time consumption and low search efficiency.
In order to solve the above technical problems, the inventor provides a neural network model design method, which combines the existing neural network knowledge base to obtain all neural network modules, constructs a series of candidate neural networks matched with scene functions based on actual scene tasks, and then screens out candidate neural network models with optimal operating speed and accuracy on a hardware platform as target neural network models based on the neural network modules capable of operating on the hardware platform, the whole process does not need to search models, so as to improve the efficiency of model design, and the neural network model design process in the embodiment of the invention can be performed in parallel with the hardware platform development, that is, the neural network model design method in the embodiment of the invention can be performed alone to obtain the optimal neural network model which can be deployed on the hardware platform, in the process of deployment, the neural network model design method can be synchronously executed, so that decoupling of a model design flow and a hardware platform development flow is realized, research and development time is reduced, the purpose of decoupling of the model design and the model deployment flow is realized, and the research and development time is reduced.
First, an implementation process of the neural network model design method provided by the embodiment of the present invention is described in detail below, please refer to fig. 1, where fig. 1 is a schematic flow chart of the neural network model design method provided by the embodiment of the present invention, and the method includes:
and S11, acquiring a neural network module library.
It is understood that a plurality of neural network modules may be included in the neural network module library. These neural network modules are all from key modules in the advanced neural network structure in the neural network knowledge base, the key modules can be core components of each neural network structure different from other neural network structures, and can also be from published research papers, modules adjusted according to effects in practice and custom modules.
For example, the neural network module library may include an inverse residual block (inversed residual block) (denoted as B1), a Fire module (denoted as B2), a PSP module (denoted as B3), an adjusted PSP module (denoted as B4), a custom upsampling module (denoted as B5), and the like.
In some possible embodiments, each neural network module in the neural network module library may further correspond to a precision index to characterize a reference index of the neural network module in the instantiated neural network. For example, in the ImageNet classification, an example network is obtained by combining modules of a single type, such as MobileNet v3, SqueezeNet and the like, the precision index of each neural network module under the example network is recorded and used as a reference index, for example, the classification accuracy of the SqueezeNet consisting of Fire modules on the ImageNet is recorded.
And S12, determining a plurality of target neural network modules matched with the scene tasks from the neural network module library according to the scene tasks.
In some possible embodiments, a plurality of target neural network modules matched with the scene task are selected from the neural network module library, a plurality of neural network models can be obtained by performing combined design according to the selected target neural network modules, the neural network modules can meet the requirements of the scene task, and the scene task can be segmentation, classification, detection and identification and the like. When the user determines that the classification task is required currently, the current scene task can be selected as the classification task. Currently, a user may also set a combination sequence of multiple scene tasks in advance, for example, the combination sequence is a classification task, a detection and identification task, and a segmentation task, and then multiple target neural networks matched with the task scenes may be sequentially selected according to the combination sequence of the multiple scene tasks.
And S13, constructing a target neural network model matched with the scene task according to the plurality of target neural network modules.
According to the neural network model design method provided by the embodiment of the invention, the neural network module library is obtained, the target neural network module capable of solving the scene task can be selected according to the actual scene task, and then the neural network model matched with the scene task is constructed based on the target neural network module. The difference from the prior art is that: in the prior art, an optimal neural network model needs to be obtained by a neural network structure searching mode, the time consumption is long, and the searching efficiency is low.
Optionally, in some possible scenarios, because the number of the selected target neural network modules is large, a large number of neural network models may be designed in combination, and meanwhile, in order to enable the designed target neural model to run on a target hardware platform, an implementation manner for obtaining the target neural network model is given below, referring to fig. 2, fig. 2 is a schematic flow chart of an implementation manner of step S13 provided by the embodiment of the present invention, that is, step S13 may include:
s131, constructing a candidate neural network module set and a first candidate neural network set according to the plurality of target neural network modules.
In some possible embodiments, each first candidate neural network in the first set of candidate neural networks is formed by a combination of at least two target neural network modules; each first candidate neural network is matched with a scene task, namely, the designed first candidate neural network can solve the current scene task, such as segmentation, classification, detection and identification; the function index of each first candidate neural network is in a set function index range, the function index can be a precision index, and when the precision index of the network is greater than or equal to 80%, the network corresponding to the precision index can be used as the first candidate neural network. Each candidate neural network module in the set of candidate neural network modules is one of the plurality of target neural network modules, and each candidate neural network module is supported to run on the target hardware platform.
And S132, determining a target neural network model according to the first candidate neural network set and the candidate neural network module set.
Optionally, in order to select a possible target neural network from the first candidate neural network set, referring to fig. 3, fig. 3 is a schematic flow chart of an implementation manner of step S13 provided in the embodiment of the present invention, that is, a possible implementation manner of step S132 is:
s132-1, determining the first candidate neural networks which meet the set conditions in the first candidate neural network set as the target neural network model.
In some possible embodiments, the setting conditions may be: the first candidate neural network includes each target neural network module included in the set of candidate neural network modules and the first candidate neural network has a minimum runtime on the target hardware platform.
For example, the target neural network modules may be B1, B2, B3, B4 and B5, and the first candidate neural network designed in combination according to the scene task is a model M1 (including B1, B4 and B5), a model M2 (including B2, B3 and B5) and a model M3 (including B4 and B5). The set of candidate neural network modules includes B2, B3, B4, B5. It can be seen that B3 does not support running on the target hardware platform, therefore, model M2 can be eliminated, where the running time of model M3 on the target hardware platform is less than that of model M1 on the target hardware platform, and the running speed of model M3 is optimal, and at this time, M3 can be used as the target neural network model.
It is understood that if the runtime of model M3 at the target hardware platform is the same as the runtime of model M1 at the target hardware platform, then model M3 or model M1 may be selected as the target neural network model.
Optionally, in order to facilitate understanding of the implementation of obtaining the candidate neural network and the candidate neural network module according to the embodiment of the present invention, a possible implementation is given below, referring to fig. 4, where fig. 4 is a schematic flowchart of an implementation of S131 provided by the embodiment of the present invention, and step S131 may include:
s131-1, performing weighted combination on at least two target neural network modules in the plurality of target neural network modules according to the scene task and the set hyper-parameters to obtain a plurality of first candidate neural networks to form a first candidate neural network set.
In some possible embodiments, the feature structure of the candidate neural network to be designed may be determined by analyzing the implementation characteristics of the current scene task, so that the designed first candidate neural network model can realize the function of matching with the current scene task, for example, the segmentation task is characterized by a large scene but small key objects, and multi-layer semantic features are considered during design. Meanwhile, in order to enable the designed neural network to run on a hardware platform quickly, a lightweight neural network module can be selected for designing the module.
For example, if the current scene task is a segmentation task, and the above design factors are combined, the designed neural network model may include: model M1: b4 × n1+ B1 × n2+ B5 × n3, where n1, n2, and n3 are hyper-parameters, which can be determined according to the size of the segmented object and the actual experimental effect. Because the hardware platform supports different modules differently, more alternative model structures can be designed, such as model M2: b3 × n4+ B2 × n5+ B5 × n6, n4, n5, n6 are hyper-parameters.
In a possible implementation manner, for the first candidate neural network that has been designed, a richer structure may also be added, such as modifying the structure of the model M1, and using another structure instead of B1 to implement the cascade connection between B4 and B4, so as to obtain the composition M3.
The models are trained and tested to obtain a scene segmentation result, and a first candidate neural network set is formed on the premise that the accuracy rate meets the service requirement.
S131-2, cascading each target neural network module to obtain a network corresponding to the target neural network module; and taking target neural network modules corresponding to the network supported by the hardware platform to operate as candidate neural network modules to form a candidate neural network module set.
In some possible implementation manners, each target neural network module may be cascaded to obtain a network with an n-layer structure, specifically, n individual target neural network modules are cascaded, for example, a module B1, where n is 5, forms a B1 × 5 network, the networks formed by combining each target neural network module have the same scale, and then the network formed by combining each target neural network module is converted into a form supported by the same target hardware platform, a neural network module corresponding to a network that does not support conversion is excluded, and a candidate neural network module library is further obtained. For example, the target hardware platform does not support the running of the target neural network module B3, and B1, B2, B4, and B5 all support the running, then the candidate neural network modules collectively include B1, B2, B4, and B5.
In one possible implementation, after obtaining the set of candidate neural network modules, the test runtime of each candidate neural network module may be recorded, and then all candidate neural network modules may be ranked based on the test runtime, for example, the ranking result of each module in the library of candidate neural network modules is B2> B1> B4> B5 based on the test runtime.
Optionally, in a scenario, in the process of determining the target neural network model, when there is no first candidate neural network model that satisfies the set condition, for example, the hardware platform does not support B1, B3, and B4, then M1 (including B1, B4, and B5), M2 (including B2, B3, and B5), and M3 (including B4 and B5) do not satisfy the set condition, a second candidate neural network set may be further constructed, and a possible implementation manner is given below on the basis of fig. 1, referring to fig. 5, where fig. 5 is one of schematic flow charts of another neural network model design method provided by an embodiment of the present invention, and the method further includes:
and S14, when the first candidate neural network which meets the set conditions does not exist, constructing a second candidate neural network set based on the candidate neural network module set.
It is to be appreciated that each second candidate neural network in the second set of candidate neural networks matches the scenario task; and the function index of each second candidate neural network is in a set function index range, the scale of the constructed second candidate neural network is consistent with that of the first candidate neural network, and then a target neural network model matched with the hardware platform is screened out based on the second candidate neural network.
Optionally, to facilitate understanding of the above implementation process for constructing the second candidate neural network set, a possible implementation is given below, referring to fig. 6, where fig. 6 is a schematic flow chart of an implementation of step S14, and step S14 may include:
s141, modifying the feature layer of at least one neural network module of each first candidate neural network to obtain a modified first candidate neural network, and taking the modified first candidate neural network as a second candidate neural network set.
In some possible embodiments, in the first candidate neural network set, if there is a first candidate neural network that includes a target neural network module that does not support operation on a target hardware platform but can be deployed on the hardware platform after a small amount of structural modification to the target neural network module, the target neural network module may be modified, and the first candidate neural network corresponding to the modified target neural network module may be used as a second candidate neural network;
for example, the model M1 includes modules B1, B4, and B5, and the tested target hardware platform does not support run B1, but may be deployed after modifying the feature layer in B1, for example, modifying B1 to B1 'after modifying the feature layer ReLU6 in B1 to a ReLU layer, to obtain a modified model M1', which includes B4 n1+ B1 'n 2+ B5 n3, and similarly, M3' may also perform similar processing, and make the modified first candidate neural network into the second candidate neural network set.
S142, sequencing all candidate neural network modules based on the running time corresponding to each candidate neural network module; and constructing a second candidate neural network set based on the ordered candidate neural network module sets and the speed selection strategy.
In other possible embodiments, a second candidate neural network matching the scenario task may be reconstructed according to candidate neural network modules in the candidate neural network module set, for example, the candidate neural network module library includes B1, B2, B4 and B5, and the result of the ranking of each module in the candidate neural network module library is B2> B1> B4> B5 based on the test runtime, based on which the second candidate neural network may be reconstructed, for example, such as that the reconstructed model M4 is B2 n7+ B4 n8+ B5 n9, where n7, n8 and n9 are hyper-parameters.
It should be noted that, the difference between the construction of the second candidate neural network and the construction of the first candidate neural network is that, in the process of constructing the first candidate neural network, the speed index between each of the neural network modules designed in combination is unknown, so the process of constructing the first candidate neural network belongs to blind construction, and aims to obtain all candidate neural network modules capable of being matched with the scene task, while in the process of constructing the second candidate neural network, the construction is performed based on the candidate neural network module library, the candidate neural network modules in the candidate neural network set have corresponding running time, and the modules for constructing the second candidate neural network can be selected based on the running speed left-right principle.
It should be further noted that there is no execution sequence between step S141 and step S142, and in one scenario, the second candidate neural network may be obtained only through step S141; in another scenario, when the candidate neural modules cannot be deployed even after being structurally modified, the neural network modules in the candidate neural network module library may be recombined to obtain a second candidate neural network set in step S142.
Optionally, the neural network model design process in the embodiment of the present invention may be performed in parallel with hardware platform development, so as to achieve decoupling of the model design process and the hardware platform development process, that is, after obtaining the target neural network model, the target neural network model may be taken to the target hardware platform for deployment to test performance of the target neural network model, an implementation manner is given below, that is, on the basis of fig. 1, referring to fig. 7, fig. 7 is a second schematic flow chart of another neural network model design method provided in the embodiment of the present invention, and the method may further include:
and S15, when the performance index of the target neural network model on the target hardware platform is in the set performance index range, deploying the target neural network model on the target hardware platform.
Optionally, if the screened target neural network model is deployed on the target hardware platform, and both the precision and the speed cannot reach the set conditions, the target neural network model may be continuously adjusted, so that the target neural network model can finally meet the set conditions, an implementation manner for adjusting the target neural network model is provided below, referring to fig. 8, where fig. 8 is a third schematic flow chart of another neural network model design method provided in the embodiment of the present invention, and the method may further include:
and S16, when the speed index of the target neural network model on the target hardware platform is not in the speed index range and the precision index is in the precision index range, compressing the target neural network model so that the performance index of the compressed target neural network model is in the performance index range.
And S17, distilling the target neural network model when the speed index of the target neural network model on the target hardware platform is in the speed index range and the precision index is not in the precision index range, so that the performance index of the compressed target neural network model is in the performance index range.
For example, if the target neural network model is the model M3 in the above embodiment, if the speed index is not in the speed index range and the accuracy index is in the accuracy index range after M3 deployment, the acceleration network is further compressed by pruning and quantization to obtain a model meeting the requirements, and deployment is completed; the target neural network model is the model M4 in the above embodiment, and if the speed index is in the speed index range after M4 deployment and the accuracy index is not in the accuracy index range, the accuracy of M4 is improved in a distillation manner, wherein in the process of modifying the model by applying the distillation manner, the teacher model selects the model with the highest accuracy in the candidate neural network set, the student model selects the model which can be directly deployed on hardware or the model compressed in step S16, the student model is retrained to obtain the model finally used for deployment, and finally deployment is completed.
It should be noted that there is no execution sequence between step S16 and step S17, and in one scenario, when the speed of the target neural network model does not meet the requirement, step S16 may be executed, and if the accuracy of the target neural network model does not meet the requirement, step S17 may be executed.
Alternatively, in the prior art, another solution based on neural network search is to train a super network first and then search for the optimal sub-network for each task, which is efficient in searching, but since the super network is determined in advance, it cannot be updated with the domain knowledge. In order to solve the above problem, a possible implementation is given below, referring to fig. 9, where fig. 9 is a fourth schematic flowchart of another neural network model design method provided by an embodiment of the present invention, and the method may further include:
and S18, updating the neural network module library according to the neural network knowledge library.
Based on the above embodiments, the neural network model design method in the embodiments of the present invention can obtain a target neural network model, and the target neural network model can be deployed on a hardware platform, but in the deployment process, the neural network model design method can be synchronously executed, thereby achieving decoupling of a model design flow and a hardware platform development flow, reducing development time, achieving the purpose of decoupling of the model design and the model deployment flow, and reducing development time. To this end, an embodiment of the present invention provides a neural network model deployment method, and referring to fig. 10, fig. 10 is a schematic flowchart of a neural network model deployment method provided in an embodiment of the present invention, where the method includes:
and S21, acquiring the target neural network model.
It is understood that the target neural network model can be obtained by various embodiments of the neural network model design method provided in the above embodiments, and the details are not repeated herein.
And S22, when the performance index of the target neural network model on the target hardware platform is in the set performance index range, deploying the target neural network model on the target hardware platform.
It can be understood that when the performance index of the target neural network model on the target hardware platform is within the set performance index range, the target neural network model is deployed on the target hardware platform. If the performance index of the target neural network model on the target hardware platform is not within the set performance index range, the target neural network model may be modified through steps S16 and S17 in the above embodiment, so that the modified target neural network model meets the deployment condition.
In order to implement the above steps S11 to S18 to achieve the corresponding technical effects, an implementation manner of a neural network model designing apparatus is given below, referring to fig. 11, fig. 11 is a functional block diagram of a neural network model designing apparatus according to an embodiment of the present invention, wherein the neural network model designing apparatus 30 includes: an acquisition module 301, a construction module 302 and a determination module 303.
An obtaining module 301, configured to obtain a neural network module library; the neural network module library comprises a plurality of neural network modules.
And the determining module 303 is configured to determine, according to the scenario task, a plurality of target neural network modules that match the scenario task from the neural network module library.
A building module 302, configured to build a target neural network model matched with the scene task according to the plurality of target neural network modules.
It is understood that the obtaining module 301, the constructing module 302 and the determining module 303 may be used to perform the steps S11 to S13 to achieve the corresponding technical effect.
Optionally, the neural network model designing apparatus 30 may further include a deployment module, and the deployment module is configured to deploy the target neural network model on the target hardware platform when the performance index of the target neural network model on the target hardware platform is within the set performance index range. The deployment module is also used for compressing the target neural network model when the speed index of the target neural network model on the target hardware platform is not in the speed index range and the precision index is in the precision index range so as to enable the performance index of the compressed target neural network model to be in the performance index range; or distilling the target neural network model when the speed index of the target neural network model on the target hardware platform is in the speed index range and the precision index is not in the precision index range so as to enable the performance index of the compressed target neural network model to be in the performance index range.
Optionally, the neural network model designing apparatus 30 may further include an updating module, and the updating module is configured to update the neural network module library according to the neural network knowledge library.
Optionally, the building module 302 is specifically configured to: constructing a candidate neural network module set and a first candidate neural network set according to the plurality of target neural network modules; each first candidate neural network in the first candidate neural network set is formed by combining at least two target neural network modules; each first candidate neural network is matched with a scene task; the function index of each first candidate neural network is in a set function index range; each candidate neural network module in the candidate neural network module set is one of a plurality of target neural network modules and each candidate neural network module is supported to run on a target hardware platform; and determining a target neural network model according to the first candidate neural network set and the candidate neural network module set.
Optionally, the determining module 303 is specifically configured to determine a first candidate neural network, in which the first candidate neural network set meets the set condition, as the target neural network model; wherein, the setting conditions are as follows: the first candidate neural network includes each target neural network module included in the set of candidate neural network modules, and the first candidate neural network has a minimum runtime on the target hardware platform.
Optionally, the building module 302 is specifically configured to: combining at least two target neural network modules in a target neural network module library according to the scene task and the set hyper-parameters to obtain a plurality of first candidate neural networks to form a candidate neural network set; cascading each target neural network module to obtain a network corresponding to the target neural network module; and taking target neural network modules corresponding to the network supported by the hardware platform to operate as candidate neural network modules to form a candidate neural network module set. The method is further specifically configured to modify the structure of the first candidate neural network to obtain a modified first candidate neural network, and add the modified first candidate neural network to the candidate neural network set.
Optionally, the building module 302 is further configured to: when the first candidate neural network meeting the set conditions does not exist, constructing a second candidate neural network set based on the candidate neural network module set; each second candidate neural network in the second set of candidate neural networks is matched with the scene task; the function index of each second candidate neural network is within a set function index range, and is further specifically configured to: modifying the characteristic layer of at least one target neural network module of each first candidate neural network to obtain a modified first candidate neural network, and taking the modified first candidate neural network as a second candidate neural network set; and/or ranking all candidate neural network modules based on the corresponding running time of each candidate neural network module; and constructing a second candidate neural network set based on the ranked candidate neural network module targets and the speed selection strategy.
In order to implement the above steps S21 to S22 to achieve the corresponding technical effect, an implementation manner of a neural network model deployment apparatus is given below, referring to fig. 12, fig. 12 is a functional block diagram of a neural network model deployment apparatus according to an embodiment of the present invention, where the neural network model deployment apparatus 40 includes: an acquisition module 401 and a deployment module 402.
An obtaining module 401, configured to obtain a target neural network model.
The deployment module 402 is configured to deploy the target neural network model on the target hardware platform when the performance index of the target neural network model on the target hardware platform is within the set performance index range.
Fig. 13 shows a block diagram of an electronic device according to an embodiment of the present invention, where fig. 13 is a block diagram of the electronic device according to the embodiment of the present invention. The electronic device 50 comprises a communication interface 501, a processor 502 and a memory 503. The processor 502, the memory 503, and the communication interface 501 are electrically connected to each other, directly or indirectly, to enable transmission or interaction of data. For example, the components may be electrically connected to each other via one or more communication buses or signal lines. The memory 503 can be used for storing software programs and modules, such as program instructions/modules corresponding to the neural network model deployment method and/or the neural network model design method provided by the embodiment of the present invention, and the processor 502 executes various functional applications and data processing by executing the software programs and modules stored in the memory 503. The communication interface 501 may be used for communicating signaling or data with other node devices. The electronic device 500 may have a plurality of communication interfaces 501 in the present invention.
The memory 503 may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a programmable read-only memory (PROM), an erasable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), and the like.
The processor 502 may be an integrated circuit chip having signal processing capabilities. The processor may be a general-purpose processor including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, etc.
It is understood that the respective modules of the neural network model deploying device 40 or the neural network model designing device 30 may be stored in the form of software or Firmware (Firmware) in the memory 503 of the electronic device 50 and executed by the processor 502, and at the same time, data, codes of programs, and the like required for executing the modules may be stored in the memory 503.
Embodiments of the present invention provide a storage medium on which a computer program is stored, where the computer program, when executed by a processor, implements a neural network model deployment method and/or a neural network model design method as in any one of the foregoing embodiments. The computer readable storage medium may be, but is not limited to, various media that can store program codes, such as a U disk, a removable hard disk, a ROM, a RAM, a PROM, an EPROM, an EEPROM, a magnetic or optical disk, etc.
The above description is only for the specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.
In the description of the present invention, it should be noted that if the terms "upper", "lower", "inside", "outside", etc. indicate an orientation or a positional relationship based on that shown in the drawings or that the product of the present invention is used as it is, this is only for convenience of description and simplification of the description, and it does not indicate or imply that the device or the element referred to must have a specific orientation, be constructed in a specific orientation, and be operated, and thus should not be construed as limiting the present invention.
Furthermore, the appearances of the terms "first," "second," and the like, if any, are used solely to distinguish one from another and are not to be construed as indicating or implying relative importance.
It should be noted that the features of the embodiments of the present invention may be combined with each other without conflict.
Claims (16)
1. A method of neural network model design, the method comprising:
acquiring a neural network module library; the neural network module library comprises a plurality of neural network modules;
determining a plurality of target neural network modules matched with the scene tasks from the neural network module library according to the scene tasks;
and constructing a target neural network model matched with the scene task according to the plurality of target neural network modules.
2. The neural network model design method of claim 1, wherein constructing a target neural network model matching the scene task according to the target neural network module comprises:
constructing a candidate neural network module set and a first candidate neural network set according to the plurality of target neural network modules;
wherein each first candidate neural network in the first set of candidate neural networks is formed by combining at least two target neural network modules; each first candidate neural network is matched with a scene task; the function index of each first candidate neural network is in a set function index range; each candidate neural network module in the set of candidate neural network modules is one of the plurality of target neural network modules and each of the candidate neural network modules is supported to run on a target hardware platform;
determining the target neural network model from the first set of candidate neural networks and the set of candidate neural network modules.
3. The neural network model design method of claim 2, wherein determining the target neural network model from the first set of candidate neural networks and the set of candidate neural network modules comprises:
determining the first candidate neural networks in the first candidate neural network set meeting set conditions as target neural network models;
wherein the setting conditions are as follows: each of the target neural network modules included in the first candidate neural network is included in the set of candidate neural network modules, and the first candidate neural network has a minimum runtime on the target hardware platform.
4. The neural network model designing method according to claim 3, further comprising, after determining the first candidate neural networks satisfying a set condition in the first set of candidate neural networks as a target neural network model:
and when the performance index of the target neural network model on the target hardware platform is within a set performance index range, deploying the target neural network model on the target hardware platform.
5. The neural network model design method of claim 1, further comprising, after constructing a target neural network model matching the scenario task from the plurality of target neural network modules:
and updating the neural network module library according to the neural network knowledge library.
6. The neural network model design method of claim 2, wherein constructing a set of candidate neural network modules and a first set of candidate neural network modules from the plurality of target neural network modules comprises:
combining at least two target neural network modules in the plurality of target neural network modules according to the scene task and the set hyper-parameters to obtain a plurality of first candidate neural networks to form a first candidate neural network set;
cascading each target neural network module to obtain a network corresponding to the target neural network module; and taking the target neural network module corresponding to the network supported by the hardware platform to operate as the candidate neural network module to form the candidate neural network module set.
7. The neural network model design method of claim 6, further comprising:
and modifying the structure of the first candidate neural network to obtain a modified first candidate neural network, and adding the modified first candidate neural network to the first candidate neural network set.
8. The neural network model design method of claim 3, further comprising:
when the first candidate neural network meeting the set condition does not exist, constructing a second candidate neural network set based on the candidate neural network module set;
wherein each second candidate neural network in the second set of candidate neural networks is matched to the scenario task; the function index of each second candidate neural network is within the set function index range.
9. The neural network model design method of claim 8, wherein constructing a second set of candidate neural networks based on the set of candidate neural network modules comprises:
modifying the feature layer of at least one target neural network module of each first candidate neural network to obtain a modified first candidate neural network, and taking the modified first candidate neural network as the second candidate neural network set; and/or the presence of a gas in the gas,
ranking all the candidate neural network modules based on the corresponding running time of each candidate neural network module; and constructing the second candidate neural network set based on the ordered candidate neural network module sets and the speed selection strategy.
10. The neural network model design method of claim 4, wherein the performance indicators include a speed indicator and a precision indicator; the performance index range comprises a speed index range and a precision index range; further comprising:
when the speed index of the target neural network model on the target hardware platform is not in the speed index range and the precision index is in the precision index range, compressing the target neural network model so that the performance index of the compressed target neural network model is in the performance index range; alternatively, the first and second electrodes may be,
when the speed index of the target neural network model on the target hardware platform is in the speed index range and the precision index is not in the precision index range, distilling the target neural network model so as to enable the performance index of the compressed target neural network model to be in the performance index range.
11. A neural network model deployment method, the method comprising:
acquiring a target neural network model;
and when the performance index of the target neural network model on a target hardware platform is within a set performance index range, deploying the target neural network model on the target hardware platform.
12. The neural network model deployment method of claim 11, wherein the method comprises: the target neural network model is obtained by the neural network model designing method according to any one of claims 1 to 10.
13. An apparatus for designing a neural network model, comprising:
the acquisition module is used for acquiring a neural network module library; the neural network module library comprises a plurality of neural network modules;
the determining module is used for determining a plurality of target neural network modules matched with the scene tasks from the neural network module library according to the scene tasks;
and the construction module is used for constructing a target neural network model matched with the scene task according to the plurality of target neural network modules.
14. A neural network model deployment device, comprising:
the acquisition module is used for acquiring a target neural network model;
and the deployment module is used for deploying the target neural network model on the target hardware platform when the performance index of the target neural network model on the target hardware platform is within the set performance index range.
15. An electronic device comprising a processor and a memory, the memory storing a computer program executable by the processor, the computer program executable by the processor to implement the neural network model design method of any one of claims 1-10 and/or to implement the neural network model deployment method of any one of claims 11-12.
16. A storage medium having stored thereon a computer program for implementing a neural network model design method according to any one of claims 1 to 10 and/or for implementing a neural network model deployment method according to any one of claims 11 to 12 when executed by a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011598884.7A CN112598117A (en) | 2020-12-29 | 2020-12-29 | Neural network model design method, deployment method, electronic device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011598884.7A CN112598117A (en) | 2020-12-29 | 2020-12-29 | Neural network model design method, deployment method, electronic device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112598117A true CN112598117A (en) | 2021-04-02 |
Family
ID=75203775
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011598884.7A Pending CN112598117A (en) | 2020-12-29 | 2020-12-29 | Neural network model design method, deployment method, electronic device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112598117A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113408634A (en) * | 2021-06-29 | 2021-09-17 | 深圳市商汤科技有限公司 | Model recommendation method and device, equipment and computer storage medium |
| JP7090817B1 (en) * | 2021-06-14 | 2022-06-24 | 三菱電機株式会社 | Neural network construction device, neural network construction method, image processing device, and image processing method |
| CN114708270A (en) * | 2021-12-15 | 2022-07-05 | 华东师范大学 | Semantic segmentation model compression system and method based on knowledge aggregation and decoupling distillation |
| CN114861890A (en) * | 2022-07-05 | 2022-08-05 | 深圳比特微电子科技有限公司 | Method and device for constructing neural network, computing equipment and storage medium |
| WO2023116416A1 (en) * | 2021-12-24 | 2023-06-29 | 中兴通讯股份有限公司 | Neural network generation method, indication information sending method, communication node, and medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180018555A1 (en) * | 2016-07-15 | 2018-01-18 | Alexander Sheung Lai Wong | System and method for building artificial neural network architectures |
| CN107977706A (en) * | 2017-08-09 | 2018-05-01 | 小蚁科技（香港）有限公司 | Modularized distribution type artificial neural network |
| WO2020092810A1 (en) * | 2018-10-31 | 2020-05-07 | Movidius Ltd. | Automated generation of neural networks |
| WO2020099606A1 (en) * | 2018-11-15 | 2020-05-22 | Camlin Technologies Limited | Apparatus and method for creating and training artificial neural networks |
| US10748057B1 (en) * | 2016-09-21 | 2020-08-18 | X Development Llc | Neural network modules |
-
2020
- 2020-12-29 CN CN202011598884.7A patent/CN112598117A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180018555A1 (en) * | 2016-07-15 | 2018-01-18 | Alexander Sheung Lai Wong | System and method for building artificial neural network architectures |
| US10748057B1 (en) * | 2016-09-21 | 2020-08-18 | X Development Llc | Neural network modules |
| CN107977706A (en) * | 2017-08-09 | 2018-05-01 | 小蚁科技（香港）有限公司 | Modularized distribution type artificial neural network |
| WO2020092810A1 (en) * | 2018-10-31 | 2020-05-07 | Movidius Ltd. | Automated generation of neural networks |
| WO2020099606A1 (en) * | 2018-11-15 | 2020-05-22 | Camlin Technologies Limited | Apparatus and method for creating and training artificial neural networks |
Non-Patent Citations (2)
| Title |
|---|
| BAOLIANG LU 等: ""Task decomposition based on class relations: A modular neural network architecture for pattern classification"", 《INTERNATIONAL WORK-CONFERENCE ON ARTIFICAL NEURAL NETWORKS》 * |
| 蒙西: ""模块化神经网络优化设计及应用研究"", 《中国优秀博士学位论文全文数据库信息科技辑》 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP7090817B1 (en) * | 2021-06-14 | 2022-06-24 | 三菱電機株式会社 | Neural network construction device, neural network construction method, image processing device, and image processing method |
| WO2022264186A1 (en) * | 2021-06-14 | 2022-12-22 | 三菱電機株式会社 | Neural network construction device, neural network construction method, image processing device, and image processing method |
| CN113408634A (en) * | 2021-06-29 | 2021-09-17 | 深圳市商汤科技有限公司 | Model recommendation method and device, equipment and computer storage medium |
| CN113408634B (en) * | 2021-06-29 | 2022-07-05 | 深圳市商汤科技有限公司 | Model recommendation method and device, equipment and computer storage medium |
| WO2023272987A1 (en) * | 2021-06-29 | 2023-01-05 | 深圳市商汤科技有限公司 | Model recommendation method and apparatus, and device and computer storage medium |
| CN114708270A (en) * | 2021-12-15 | 2022-07-05 | 华东师范大学 | Semantic segmentation model compression system and method based on knowledge aggregation and decoupling distillation |
| CN114708270B (en) * | 2021-12-15 | 2023-08-08 | 华东师范大学 | Application of compression method based on knowledge aggregation and decoupling distillation in semantic segmentation |
| WO2023116416A1 (en) * | 2021-12-24 | 2023-06-29 | 中兴通讯股份有限公司 | Neural network generation method, indication information sending method, communication node, and medium |
| CN114861890A (en) * | 2022-07-05 | 2022-08-05 | 深圳比特微电子科技有限公司 | Method and device for constructing neural network, computing equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112598117A (en) | Neural network model design method, deployment method, electronic device and storage medium | |
| CN110807515A (en) | Model generation method and device | |
| CN110515986B (en) | Processing method and device of social network diagram and storage medium | |
| CN109993298A (en) | Method and apparatus for compressing neural network | |
| CN109523022A (en) | Terminal data processing method, apparatus and terminal | |
| CN112466334B (en) | Audio identification method, equipment and medium | |
| CN110706015A (en) | Advertisement click rate prediction oriented feature selection method | |
| CN110929867A (en) | Method, device and storage medium for evaluating and determining neural network structure | |
| KR20210157302A (en) | Method and Apparatus for Automatic Predictive Modeling Based on Workflow | |
| CN116932384A (en) | Software defect prediction method based on feature fusion and feature selection | |
| CN110069997B (en) | Scene classification method and device and electronic equipment | |
| CN115237804A (en) | Performance bottleneck assessment method, performance bottleneck assessment device, electronic equipment, medium and program product | |
| CN111309946A (en) | Established file optimization method and device | |
| CN112348188B (en) | Model generation method and device, electronic device and storage medium | |
| CN112784025A (en) | Method and device for determining target event | |
| CN112733724A (en) | Relativity relationship verification method and device based on discrimination sample meta-digger | |
| CN114168446B (en) | Simulation evaluation method and device for mobile terminal operation algorithm model | |
| CN114417982A (en) | Model training method, terminal device and computer readable storage medium | |
| CN113157582A (en) | Method and device for determining execution sequence of test script | |
| CN112433950A (en) | Method for automatically building test environment, electronic equipment and storage medium | |
| CN111414538A (en) | Text recommendation method and device based on artificial intelligence and electronic equipment | |
| CN116543796B (en) | Audio processing method and device, computer equipment and storage medium | |
| CN112766558A (en) | Modeling sample generation method, device, equipment and computer readable storage medium | |
| CN117236387A (en) | Super network training method, device, equipment, medium and program product | |
| CN113342648A (en) | Test result analysis method and device based on machine learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 510000 Block C, 115 Gaopu Road, Tianhe District, Guangzhou City, Guangdong Province Applicant after: XAG Co., Ltd. Address before: 510000 Block C, 115 Gaopu Road, Tianhe District, Guangzhou City, Guangdong Province Applicant before: Guangzhou Xaircraft Technology Co.,Ltd. |
|
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210402 |