CN110276443A - Model conversion consistency checking and analysis method, device and storage medium - Google Patents
Model conversion consistency checking and analysis method, device and storage medium Download PDFInfo
- Publication number
- CN110276443A CN110276443A CN201910456809.8A CN201910456809A CN110276443A CN 110276443 A CN110276443 A CN 110276443A CN 201910456809 A CN201910456809 A CN 201910456809A CN 110276443 A CN110276443 A CN 110276443A
- Authority
- CN
- China
- Prior art keywords
- neural network
- network model
- data
- tensor data
- modified
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000006243 chemical reaction Methods 0.000 title claims abstract description 53
- 238000004458 analytical method Methods 0.000 title claims abstract description 34
- 238000003062 neural network model Methods 0.000 claims abstract description 193
- 238000012795 verification Methods 0.000 claims abstract description 97
- 238000012986 modification Methods 0.000 claims abstract description 41
- 230000004048 modification Effects 0.000 claims abstract description 41
- 238000000034 method Methods 0.000 claims abstract description 38
- 230000009466 transformation Effects 0.000 claims description 33
- 238000009795 derivation Methods 0.000 claims description 9
- 238000013528 artificial neural network Methods 0.000 abstract description 9
- 230000008569 process Effects 0.000 description 16
- 238000012360 testing method Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 238000004590 computer program Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 210000003746 feather Anatomy 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The present invention relates to field of neural networks, it is proposed a kind of model conversion consistency checking and analysis method, device and storage medium, method therein is applied to electronic device, method comprises determining that neural network model to be modified, data export interface is set in the key node position of the neural network model to be modified, and records tensor data derived from the data export interface institute；Analysis modification is carried out to the layer for needing to modify in the neural network model, and consistency verification method is carried out to the modification of the neural network model by corresponding tensor data.The present invention by the way that data export interface is arranged in the key node position of neural network model, verified, and can accelerate the speed of orientation problem node by the related data of acquisition key node position.
Description
Technical Field
The invention relates to the technical field of neural network model conversion processing, in particular to a method and a device for verifying and analyzing model conversion consistency and a computer readable storage medium.
Background
Due to the endless variety of deep learning frameworks, different frameworks are often needed in the process from algorithm development to algorithm deployment. For example, it is very likely that we use tensorflow, pytorch, etc. to develop and debug the algorithm, and by the mobile deployment stage, frames such as NCNN or face of feather cnnihui may be used, which causes the problem of model transformation.
At present, when model deployment is performed in the deep learning field, a problem that a deployment tool does not support some layers in a model is usually encountered, the layers have to be modified to complete model deployment, the model is converted, consistency of model conversion is rapidly verified, and positioning of a node position where a problem exists is a difficult problem at present, and the industry has no mature solution.
Disclosure of Invention
The invention provides a model conversion consistency verification and analysis method, an electronic device and a computer readable storage medium, and mainly aims to realize integral or segmented verification of a neural network model by arranging a data export interface at a key position node of the neural network model and accelerate the positioning speed of a problem node.
In order to achieve the above object, the present invention provides a method for verifying and analyzing consistency of model transformation, which is applied to an electronic device, and the method comprises:
determining a neural network model to be modified;
setting a data export interface at the key node position of the neural network model to be modified, and recording tensor data exported by the data export interface;
and analyzing and modifying the layer needing to be modified in the neural network model, and carrying out consistency verification on the modification of the neural network model through corresponding tensor data.
Preferably, the step of recording tensor data derived by the corresponding data derivation interface includes:
recording tensor data of a Forward path exported by the data export interface through a data recording device, and storing each tensor data as a standard data file; wherein,
the tensor data comprise input end tensor data and output end tensor data of the whole neural network model, and input end tensor data and output end tensor data of a layer to be modified in the neural network model.
Preferably, the step of consistency-verifying the modification of the neural network model by the corresponding tensor data comprises:
inputting the tensor data of the input end recorded by the input end of the neural network model to be modified from the input end of the modified neural network model;
recording output end tensor data at the output end of the modified neural network model;
comparing the output end tensor data of the modified neural network model with the output end tensor data of the to-be-modified neural network model;
when the output end tensor data of the modified neural network model is the same as the output end tensor data of the to-be-modified neural network model, indicating that the conversion consistency of the neural network model is good; otherwise, the consistency of the neural network model conversion is shown to be in problem.
Preferably, when the consistency of the neural network model transformation is in question:
and narrowing the verification range of the tensor data, and performing segmented verification by using the tensor data of the intermediate layer position nodes of the neural network model until the position of the problem node is positioned.
Preferably, the method further comprises:
and analyzing the problem nodes and modifying codes, and performing consistency verification on the modified neural network model until the consistency of the conversion of the neural network model is met.
In addition, to achieve the above object, the present invention also provides an electronic device including: the system comprises a memory, a processor and a data recording device, wherein the memory comprises a model conversion consistency verification and analysis program, and the model conversion consistency verification and analysis program realizes the following steps when being executed by the processor:
determining a neural network model to be modified;
setting a data export interface at the key node position of the neural network model to be modified, and recording tensor data exported by the data export interface;
and analyzing and modifying the layer needing to be modified in the neural network model, and carrying out consistency verification on the modification of the neural network model through corresponding tensor data.
Preferably, the step of recording tensor data derived by the corresponding data derivation interface includes:
recording tensor data of a Forward path exported by the data export interface through a data recording device, and storing each tensor data as a standard data file; wherein,
the tensor data comprise input end tensor data and output end tensor data of the whole neural network model, and input end tensor data and output end tensor data of a layer to be modified in the neural network model.
Preferably, the step of consistency-verifying the modification of the neural network model by the corresponding tensor data comprises:
inputting the tensor data of the input end recorded by the input end of the neural network model to be modified from the input end of the modified neural network model;
recording output end tensor data at the output end of the modified neural network model;
comparing the output end tensor data of the modified neural network model with the output end tensor data of the to-be-modified neural network model;
when the output end tensor data of the modified neural network model is the same as the output end tensor data of the to-be-modified neural network model, indicating that the conversion consistency of the neural network model is good; otherwise, the consistency of the neural network model conversion is shown to be in problem.
Preferably, when the consistency of the neural network model transformation is in question:
and narrowing the verification range of the tensor data, and performing segmented verification by using the tensor data of the intermediate layer position nodes of the neural network model until the position of the problem node is positioned.
In addition, to achieve the above object, the present invention further provides a computer-readable storage medium, which includes a model transformation consistency verification and analysis program, and when the model transformation consistency verification and analysis program is executed by a processor, the steps of the model transformation consistency verification and analysis method are implemented.
According to the model conversion consistency verification and analysis method, the electronic device and the computer readable storage medium, the data export interfaces are arranged at the key node positions of the neural network model, tensor data at the data export interfaces are recorded, integral or segmented verification of the neural network model is achieved through the tensor data, and the speed of positioning problem nodes can be increased; in addition, the consistency verification of model conversion can be completed by collecting representative data (tensor data), so that the learning cost of developers can be greatly saved, and engineers in model deployment can work by separating from principles and knowledge hidden in the model.
Drawings
FIG. 1 is a schematic diagram of an application environment in accordance with an embodiment of a model transformation consistency verification and analysis method of the present invention;
FIG. 2 is a block diagram of one embodiment of a model transformation consistency verification and analysis process of FIG. 1;
FIG. 3 is a schematic diagram of a partial structure of a neural network model before modification;
FIG. 4 is a schematic diagram of a partial structure of a modified neural network model;
FIG. 5 is a flow chart of an embodiment of a model transformation consistency verification and analysis method of the present invention;
FIG. 6 is a flow chart of the consistency verification of the present invention for modifications to the neural network model by corresponding tensor data.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The invention provides a model conversion consistency verification and analysis method, which is applied to an electronic device 1. Fig. 1 is a schematic diagram of an application environment of a model transformation consistency verification and analysis method according to an embodiment of the present invention.
In the present embodiment, the electronic device 1 may be a terminal device having an arithmetic function, such as a server, a smart phone, a tablet computer, a portable computer, or a desktop computer.
The electronic device 1 includes: processor 12, memory 11, data recording device 13, network interface 14, and communication bus 15.
The memory 11 includes at least one type of readable storage medium. The at least one type of readable storage medium may be a non-volatile storage medium such as a flash memory, a hard disk, a multimedia card, a card-type memory 11, and the like. In some embodiments, the readable storage medium may be an internal storage unit of the electronic apparatus 1, such as a hard disk of the electronic apparatus 1. In other embodiments, the readable storage medium may also be an external memory 11 of the electronic device 1, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the electronic device 1.
In the present embodiment, the readable storage medium of the memory 11 is generally used for storing the model transformation consistency verification and analysis program 10 installed in the electronic device 1. The memory 11 may also be used to temporarily store data that has been output or is to be output.
The processor 12 may be a Central Processing Unit (CPU), microprocessor or other data Processing chip in some embodiments, and is used for running program codes stored in the memory 11 or Processing data, such as the model transformation consistency verification and analysis program 10.
The data recording device 13 may be a part of the electronic device 1 or may be independent of the electronic device 1. In some embodiments, the electronic device 1 is a terminal device having a data recording unit, such as a smart phone, a tablet computer, a portable computer, and the like, and then the data recording device 13 is a data acquisition unit of the electronic device 1. In other embodiments, the electronic device 1 may be a server, and the data acquisition device 13 is independent from the electronic device 1 and connected to the electronic device 1 through a network.
The network interface 14 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface), and is typically used to establish a communication link between the electronic apparatus 1 and other electronic devices.
The communication bus 15 is used to realize connection communication between these components.
Fig. 1 only shows the electronic device 1 with components 11-15, but it is to be understood that not all of the shown components are required to be implemented, and that more or fewer components may alternatively be implemented.
Optionally, the electronic device 1 may further include a user interface, the user interface may include an input unit such as a Keyboard (Keyboard), a voice input device such as a microphone (microphone) or other equipment with a voice recognition function, a voice output device such as a sound box, a headset, etc., and optionally the user interface may further include a standard wired interface, a wireless interface.
Optionally, the electronic device 1 may further comprise a display, which may also be referred to as a display screen or a display unit. In some embodiments, the display device may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an Organic Light-Emitting Diode (OLED) touch device, or the like. The display is used for displaying information processed in the electronic apparatus 1 and for displaying a visualized user interface.
Optionally, the electronic device 1 further comprises a touch sensor. The area provided by the touch sensor for the user to perform touch operation is called a touch area. Further, the touch sensor described herein may be a resistive touch sensor, a capacitive touch sensor, or the like. The touch sensor may include not only a contact type touch sensor but also a proximity type touch sensor. Further, the touch sensor may be a single sensor, or may be a plurality of sensors arranged in an array, for example.
The area of the display of the electronic device 1 may be the same as or different from the area of the touch sensor. Optionally, a display is stacked with the touch sensor to form a touch display screen. The device detects touch operation triggered by a user based on the touch display screen.
Optionally, the electronic device 1 may further include a Radio Frequency (RF) circuit, a sensor, an audio circuit, and the like, which are not described herein again.
In the embodiment of the apparatus shown in fig. 1, the memory 11, which is a kind of computer storage medium, may include therein an operating system, and a model conversion consistency verification and analysis program 10; the processor 12, when executing the model transformation consistency verification and analysis program 10 stored in the memory 11, implements the following steps:
determining a neural network model to be modified;
setting a data export interface at the key node position of the neural network model to be modified, and recording tensor data exported by the data export interface;
and analyzing and modifying the layer needing to be modified in the neural network model, and carrying out consistency verification on the modification of the neural network model through corresponding tensor data.
The selection of the key node positions is generally an input position and an output position of the whole neural network model, and an input position and an output position of an internal layer to be modified. The layer to be modified mainly refers to some layers which are not supported by the deployment tool and have to be modified in the neural network model conversion process.
The setting principle of the data export interface is as follows: the data export interface only exists as a bypass and does not influence the data content in the Forward path of the neural network model.
Tensor data recorded by each data derivation interface comprise input end tensor data and output end tensor data of the whole neural network, and input end tensor data and output end tensor data of a layer to be modified inside; and the collected tensor data need to be marked with corresponding positions for selective use according to the position information in subsequent verification.
Further, the step of recording tensor data derived by the corresponding data derivation interface includes: and recording tensor data of the Forward path exported by the data exporting interface through a data recording device, and saving each tensor data as a standard data file. In other words, tensor data of the forwarded path derived by the corresponding data deriving interface is recorded by the data recording device, and the data is saved as a standard data file to be used as Golden data in the subsequent verification stage.
Preferably, the step of consistency-verifying the modification of the neural network model by the corresponding tensor data further comprises:
and inputting the tensor data of the input end recorded by the integral input end of the neural network model to be modified (the original neural network model or the neural network model before being modified, the same below) from the input end of the modified neural network model.
Output tensor data is then recorded at the output of the modified neural network model.
And secondly, comparing the output end tensor data of the modified neural network model with the output end tensor data of the to-be-modified neural network model.
When the output end tensor data of the modified neural network model is the same as the output end tensor data of the to-be-modified neural network model, indicating that the conversion consistency of the neural network model is good; otherwise, when the output end tensor data of the modified neural network model is different from the output end tensor data of the to-be-modified neural network model, indicating that the conversion consistency of the neural network model has a problem.
Further, when consistency of neural network model conversion is problematic: and narrowing the verification range of the tensor data, and performing segmented verification by using the tensor data of the intermediate layer position nodes of the neural network model until the position of the problem node is positioned.
For example, tensor data recorded by a pre-modified layer of the original neural network model is input from an input end of the modified layer, tensor data is recorded at an output end of the modified neural network model, and then comparison test is carried out on the newly recorded output end tensor data and the tensor data recorded at the output end of the original neural network model. If the two data are not consistent, the difference exists, the consistency problem of the model conversion is shown, and the problem layer is between the modification layer and the output end of the neural network model. At the moment, the verification range is reduced again, and segmented verification is carried out by using data of intermediate position nodes between the modification layer and the output end of the whole neural network model until the position of the problem node is positioned.
And finally, carrying out comparison analysis and code modification on the discovered problem nodes, and carrying out verification test by using the model verification and analysis method again, and repeating iteration until the test consistency of the input end and the output end of the neural network model is met.
The process of model transformation consistency verification and analysis provided by the present invention will be described in detail with reference to specific examples.
FIG. 3 shows a partial schematic structure of a neural network model before modification; fig. 4 is a partial schematic structure of a modified neural network model.
As shown in fig. 3, the neural network model is composed of an input layer, n hidden layers, and an output layer. In the process of model conversion, it is found that the deployment platform does not support the hidden layer 2 and the hidden layer 4, which means that the hidden layer 2 and the hidden layer 4 need to be modified, thereby realizing model deployment.
In order to enable fast verification and consistency of analysis model conversion, the following steps need to be performed:
firstly, the method comprises the following steps: under the structure of an original neural network model, selecting an input end and an output end of the whole neural network model, an input end and an output end of the hidden layer 2 and an input end and an output end of the hidden layer 4 as key nodes, arranging corresponding data export interfaces at 6 positions, and recording tensor data collected by the data export interfaces to obtain tensor data A-F.
Secondly, the method comprises the following steps: according to the deployment requirement, the hidden layer 2 and the hidden layer 4 are modified, and a new modified neural network model is obtained, as shown in fig. 4. In order to verify the consistency of model conversion, the data record A collected before is injected again from the input end of a new neural network, data records are carried out on other key nodes, and 5 new tensor data are obtained and recorded as tensor data B 'to F'.
In the verification process, firstly, the comparison between the data record F 'and the data record F is completed at the output port of the new neural network model, and if the data record F' and the data record F are consistent, the model modification consistency is good; otherwise, the verification range needs to be further narrowed, and the verification steps are repeated until the positions of the inconsistent layers are found out.
Specifically, when the data record F ' at the output end of the new neural network model is inconsistent with the data record F, the output end data C ' of the hidden layer 2 at the intermediate position is further compared with the data C, and if the data C ' is consistent with the data C, it is indicated that the consistency of the modified hidden layer 2 is good, and then it can be known that the problem of poor model conversion consistency is that the hidden layer 4 is involved, and then the hidden layer 4 is analyzed and code modified. Otherwise, if the data C' is inconsistent with the data C, it indicates that there is a problem with the modification of the hidden layer 2, and it is still unknown whether there is a problem with the modification of the hidden layer 4.
In the above case, the correctness of the modification of the hidden layer 4 can be verified in two ways.
One is as follows: firstly, analyzing and modifying codes of a hidden layer 2 with problems, then verifying the neural network model (the data of each corresponding data export interface are B '-F respectively) formed after the neural network model is modified again, and if the data record F' is consistent with the data record F, indicating that the model modification is consistent; at the same time, it also shows that the modification of the hidden layer 4 is not problematic.
The second step is as follows: inputting the data D recorded for the first time from the input end of the hidden layer 4, acquiring corresponding recorded data F 'or E' at the output end of the whole neural network model or the output end of the hidden layer 4, and completing the verification of the modification of the hidden layer 4 by comparing whether the data record F 'is consistent with the data record F or whether the record E' is consistent with the data record E or not until the position of the problem node is found.
In other embodiments, the model transformation consistency verification and analysis program 10 may also be divided into one or more modules, which are stored in the memory 11 and executed by the processor 12 to accomplish the present invention. The modules referred to herein are referred to as a series of computer program instruction segments capable of performing specified functions. Referring to FIG. 2, a block diagram of a preferred embodiment of the model transformation consistency verification and analysis process 10 of FIG. 1 is shown.
As shown in fig. 2, the model transformation consistency verification and analysis program 10 may be divided into: a node position determining unit 210, a data recording unit 220, a model converting unit 230, and a verification analyzing unit 240. The functions or operation steps implemented by the module 210-240 are similar to those described above, and are not detailed here, for example, where:
and a node position determining unit 210, configured to determine a key node position of the neural network model, and set a data export interface at the key node position of the neural network model.
The data recording unit 220 is configured to record tensor data derived by the corresponding data deriving interface.
And the model conversion unit 230 is used for analyzing and modifying the layer needing to be modified in the neural network model.
The verification analysis unit 240 is configured to perform consistency verification on the modifications of the neural network model through the corresponding tensor data.
Wherein, the verification analysis unit 240 further comprises:
and the data input module 241 is configured to input tensor data recorded at the overall input end of the original neural network model from the input end of the modified neural network model.
And a data output module 242, configured to record new tensor data at an output end of the modified neural network.
The comparing module 243 compares the newly recorded tensor data with the tensor data recorded at the output end of the original neural network for analysis.
The judging module 244 shows that the consistency of the model conversion is good when the newly recorded tensor data is consistent with the tensor data recorded at the output end of the original neural network. Otherwise, the verification range is narrowed, and the data of the intermediate position node is utilized to carry out segmented verification until the position of the problem node is positioned.
In addition, the invention also provides a model conversion consistency verification and analysis method. Referring to FIG. 5, a flow chart of a preferred embodiment of the model transformation consistency verification and analysis method of the present invention is shown. The method may be performed by an apparatus, which may be implemented by software and/or hardware.
In this embodiment, the method for verifying and analyzing consistency of model transformation includes:
s110: a neural network model to be modified is determined.
S120: and setting a data export interface at the key node position of the neural network model to be modified, and recording tensor data exported by the data export interface.
The selection of the key node positions is generally an input position and an output position of the whole neural network model, and an input position and an output position of an internal layer to be modified. The layer to be modified mainly refers to some layers which are not supported by the deployment tool and have to be modified in the neural network model conversion process.
The setting principle of the data export interface is as follows: the data export interface only exists as a bypass and does not influence the data content in the Forward path of the neural network model.
Tensor data recorded by each data derivation interface comprise input end tensor data and output end tensor data of the whole neural network, and input end tensor data and output end tensor data of a layer to be modified inside; and the collected tensor data need to be marked with corresponding positions for selective use according to the position information in subsequent verification.
Further, the step of recording tensor data derived by the corresponding data derivation interface includes: and recording tensor data of the Forward path exported by the data exporting interface through a data recording device, and saving each tensor data as a standard data file. In other words, tensor data of the forwarded path derived by the corresponding data deriving interface is recorded by the data recording device, and the data is saved as a standard data file to be used as Golden data in the subsequent verification stage.
S130: and analyzing and modifying the layer needing to be modified in the neural network model, and carrying out consistency verification on the modification of the neural network model through corresponding tensor data.
As shown in fig. 6, the step of validating consistency of the modifications of the neural network model by the corresponding tensor data further comprises:
s121: and inputting the tensor data of the input end recorded by the integral input end of the neural network model to be modified (the original neural network model or the neural network model before being modified, the same below) from the input end of the modified neural network model.
S122: and recording output end tensor data at the output end of the modified neural network model.
S123: and comparing the output end tensor data of the modified neural network model with the output end tensor data of the to-be-modified neural network model.
S124: when the output end tensor data of the modified neural network model is the same as the output end tensor data of the to-be-modified neural network model, indicating that the conversion consistency of the neural network model is good; otherwise, when the output end tensor data of the modified neural network model is different from the output end tensor data of the to-be-modified neural network model, indicating that the conversion consistency of the neural network model has a problem.
Further, when consistency of neural network model conversion is problematic: and narrowing the verification range of the tensor data, and performing segmented verification by using the tensor data of the intermediate layer position nodes of the neural network model until the position of the problem node is positioned.
For example, tensor data recorded by a pre-modified layer of the original neural network model is input from an input end of the modified layer, tensor data is recorded at an output end of the modified neural network model, and then comparison test is carried out on the newly recorded output end tensor data and the tensor data recorded at the output end of the original neural network model. If the two data are not consistent, the difference exists, the consistency problem of the model conversion is shown, and the problem layer is between the modification layer and the output end of the neural network model. At the moment, the verification range is reduced again, and segmented verification is carried out by using data of intermediate position nodes between the modification layer and the output end of the whole neural network model until the position of the problem node is positioned.
And finally, carrying out comparison analysis and code modification on the discovered problem nodes, and carrying out verification test by using the model verification and analysis method again, and repeating iteration until the test consistency of the input end and the output end of the neural network model is met.
The process of model transformation consistency verification and analysis provided by the present invention will be described in detail with reference to specific examples.
As shown in fig. 3, the neural network model is composed of an input layer, n hidden layers, and an output layer. In the process of model conversion, it is found that the deployment platform does not support the hidden layer 2 and the hidden layer 4, which means that the hidden layer 2 and the hidden layer 4 need to be modified, thereby realizing model deployment.
In order to enable fast verification and consistency of analysis model conversion, the following steps need to be performed:
firstly, the method comprises the following steps: under the structure of an original neural network model, selecting an input end and an output end of the whole neural network model, an input end and an output end of the hidden layer 2 and an input end and an output end of the hidden layer 4 as key nodes, arranging corresponding data export interfaces at 6 positions, and recording tensor data collected by the data export interfaces to obtain tensor data A-F.
Secondly, the method comprises the following steps: according to the deployment requirement, the hidden layer 2 and the hidden layer 4 are modified, and a new modified neural network model is obtained, as shown in fig. 4. In order to verify the consistency of model conversion, the data record A collected before is injected again from the input end of a new neural network, data records are carried out on other key nodes, and 5 new tensor data are obtained and recorded as tensor data B 'to F'.
In the verification process, firstly, the comparison between the data record F 'and the data record F is completed at the output port of the new neural network model, and if the data record F' and the data record F are consistent, the model modification consistency is good; otherwise, the verification range needs to be further narrowed, and the verification steps are repeated until the positions of the inconsistent layers are found out.
Specifically, when the data record F ' at the output end of the new neural network model is inconsistent with the data record F, the output end data C ' of the hidden layer 2 at the intermediate position is further compared with the data C, and if the data C ' is consistent with the data C, it is indicated that the consistency of the modified hidden layer 2 is good, and then it can be known that the problem of poor model conversion consistency is that the hidden layer 4 is involved, and then the hidden layer 4 is analyzed and code modified. Otherwise, if the data C' is inconsistent with the data C, it indicates that there is a problem with the modification of the hidden layer 2, and it is still unknown whether there is a problem with the modification of the hidden layer 4.
In the above case, the correctness of the modification of the hidden layer 4 can be verified in two ways.
One is as follows: firstly, analyzing and modifying codes of a hidden layer 2 with problems, then verifying the neural network model (the data of each corresponding data export interface are B '-F respectively) formed after the neural network model is modified again, and if the data record F' is consistent with the data record F, indicating that the model modification is consistent; at the same time, it also shows that the modification of the hidden layer 4 is not problematic.
The second step is as follows: inputting the data D recorded for the first time from the input end of the hidden layer 4, acquiring corresponding recorded data F 'or E' at the output end of the whole neural network model or the output end of the hidden layer 4, and completing the verification of the modification of the hidden layer 4 by comparing whether the data record F 'is consistent with the data record F or whether the record E' is consistent with the data record E or not until the position of the problem node is found.
The model conversion consistency verification and analysis method according to the present invention is an objective model conversion consistency verification and analysis method for testing a driving verification scheme; by adopting a data-driven verification scheme, the learning cost of developers can be greatly saved, and engineers in model deployment can work by separating from principles and knowledge hidden in the model; in addition, segmented verification can be realized, and the position of the problem node can be positioned more quickly; the method has universality and can be applied to model conversion processes of all artificial neural networks.
In addition, an embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium includes a model transformation consistency verification and analysis program, and when executed by a processor, the model transformation consistency verification and analysis program implements the following operations:
determining a neural network model to be modified;
setting a data export interface at the key node position of the neural network model to be modified, and recording tensor data exported by the data export interface;
and analyzing and modifying the layer needing to be modified in the neural network model, and carrying out consistency verification on the modification of the neural network model through corresponding tensor data.
Preferably, the step of recording tensor data derived by the corresponding data derivation interface includes:
recording tensor data of a Forward path exported by the data export interface through a data recording device, and storing each tensor data as a standard data file; wherein,
the tensor data comprise input end tensor data and output end tensor data of the whole neural network model, and input end tensor data and output end tensor data of a layer to be modified in the neural network model.
Preferably, the step of consistency-verifying the modification of the neural network model by the corresponding tensor data comprises:
inputting the tensor data of the input end recorded by the input end of the neural network model to be modified from the input end of the modified neural network model;
recording output end tensor data at the output end of the modified neural network model;
comparing the output end tensor data of the modified neural network model with the output end tensor data of the to-be-modified neural network model;
when the output end tensor data of the modified neural network model is the same as the output end tensor data of the to-be-modified neural network model, indicating that the conversion consistency of the neural network model is good; otherwise, the consistency of the neural network model conversion is shown to be in problem.
Preferably, when the consistency of the neural network model transformation is in question:
and narrowing the verification range of the tensor data, and performing segmented verification by using the tensor data of the intermediate layer position nodes of the neural network model until the position of the problem node is positioned.
Preferably, the method further comprises:
and analyzing the problem nodes and modifying codes, and performing consistency verification on the modified neural network model until the consistency of the conversion of the neural network model is met.
The embodiment of the computer-readable storage medium of the present invention is substantially the same as the embodiment of the model transformation consistency verification and analysis method and the electronic device, and will not be described herein again.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, apparatus, article, or method. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, apparatus, article, or method that includes the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments. Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) as described above and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A method for verifying and analyzing consistency of model conversion is applied to an electronic device, and is characterized by comprising the following steps:
determining a neural network model to be modified;
setting a data export interface at the key node position of the neural network model to be modified, and recording tensor data exported by the data export interface;
and analyzing and modifying the layer needing to be modified in the neural network model, and carrying out consistency verification on the modification of the neural network model through corresponding tensor data.
2. The model transformation consistency verification and analysis method of claim 1, wherein the step of recording tensor data derived by corresponding data derivation interfaces comprises:
recording tensor data of a Forward path exported by the data export interface through a data recording device, and storing each tensor data as a standard data file; wherein,
the tensor data comprise input end tensor data and output end tensor data of the whole neural network model, and input end tensor data and output end tensor data of a layer to be modified in the neural network model.
3. The model transformation consistency verification and analysis method of claim 1, wherein the step of consistency verifying the modifications of the neural network model by the corresponding tensor data comprises:
inputting the tensor data of the input end recorded by the input end of the neural network model to be modified from the input end of the modified neural network model;
recording output end tensor data at the output end of the modified neural network model;
comparing the output end tensor data of the modified neural network model with the output end tensor data of the to-be-modified neural network model;
when the output end tensor data of the modified neural network model is the same as the output end tensor data of the to-be-modified neural network model, indicating that the conversion consistency of the neural network model is good; otherwise, the consistency of the neural network model conversion is shown to be in problem.
4. The model transformation consistency verification and analysis method according to claim 3, wherein when consistency of the neural network model transformation is in question:
and narrowing the verification range of the tensor data, and performing segmented verification by using the tensor data of the intermediate layer position nodes of the neural network model until the position of the problem node is positioned.
5. The model transformation consistency verification and analysis method of claim 4, wherein the method further comprises:
and analyzing the problem nodes and modifying codes, and performing consistency verification on the modified neural network model until the consistency of the conversion of the neural network model is met.
6. An electronic device, comprising: the system comprises a memory, a processor and a data recording device, wherein the memory comprises a model conversion consistency verification and analysis program, and the model conversion consistency verification and analysis program realizes the following steps when being executed by the processor:
determining a neural network model to be modified;
setting a data export interface at the key node position of the neural network model to be modified, and recording tensor data exported by the data export interface;
and analyzing and modifying the layer needing to be modified in the neural network model, and carrying out consistency verification on the modification of the neural network model through corresponding tensor data.
7. The electronic device of claim 6, wherein the step of recording tensor data derived by the corresponding data derivation interface comprises:
recording tensor data of a Forward path exported by the data export interface through a data recording device, and storing each tensor data as a standard data file; wherein,
the tensor data comprise input end tensor data and output end tensor data of the whole neural network model, and input end tensor data and output end tensor data of a layer to be modified in the neural network model.
8. The electronic device of claim 6, wherein the step of consistency-verifying the modifications to the neural network model by the corresponding tensor data comprises:
inputting the tensor data of the input end recorded by the input end of the neural network model to be modified from the input end of the modified neural network model;
recording output end tensor data at the output end of the modified neural network model;
comparing the output end tensor data of the modified neural network model with the output end tensor data of the to-be-modified neural network model;
when the output end tensor data of the modified neural network model is the same as the output end tensor data of the to-be-modified neural network model, indicating that the conversion consistency of the neural network model is good; otherwise, the consistency of the neural network model conversion is shown to be in problem.
9. The electronic device of claim 8, wherein when there is a problem with consistency of the neural network model transformation:
and narrowing the verification range of the tensor data, and performing segmented verification by using the tensor data of the intermediate layer position nodes of the neural network model until the position of the problem node is positioned.
10. A computer-readable storage medium, comprising a model transformation consistency verification and analysis program, which when executed by a processor, performs the steps of the model transformation consistency verification and analysis method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910456809.8A CN110276443B (en) | 2019-05-29 | 2019-05-29 | Model conversion consistency verification and analysis method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910456809.8A CN110276443B (en) | 2019-05-29 | 2019-05-29 | Model conversion consistency verification and analysis method, device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110276443A true CN110276443A (en) | 2019-09-24 |
| CN110276443B CN110276443B (en) | 2024-07-05 |
Family
ID=67959130
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910456809.8A Active CN110276443B (en) | 2019-05-29 | 2019-05-29 | Model conversion consistency verification and analysis method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110276443B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113127058A (en) * | 2021-04-28 | 2021-07-16 | 北京百度网讯科技有限公司 | Data annotation method, related device and computer program product |
| CN115187821A (en) * | 2022-07-05 | 2022-10-14 | 阿波罗智能技术(北京)有限公司 | Method for verifying correctness before and after model conversion, related device and program product |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101673198A (en) * | 2009-03-06 | 2010-03-17 | 华东师范大学 | Method for verifying consistency of dynamic behavior in UML model and time-sequence contract |
| CN105446883A (en) * | 2015-12-08 | 2016-03-30 | 中国航空工业集团公司西安航空计算技术研究所 | Model verification analysis method based on data configuration tool |
| CN108229651A (en) * | 2017-11-28 | 2018-06-29 | 北京市商汤科技开发有限公司 | Neural network model moving method and system, electronic equipment, program and medium |
| CN109389136A (en) * | 2017-08-08 | 2019-02-26 | 上海为森车载传感技术有限公司 | Classifier training method |
| US20190139179A1 (en) * | 2017-11-03 | 2019-05-09 | Baidu Usa Llc | Systems and methods for unsupervised learning of geometry from images using depth-normal consistency |
-
2019
- 2019-05-29 CN CN201910456809.8A patent/CN110276443B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101673198A (en) * | 2009-03-06 | 2010-03-17 | 华东师范大学 | Method for verifying consistency of dynamic behavior in UML model and time-sequence contract |
| CN105446883A (en) * | 2015-12-08 | 2016-03-30 | 中国航空工业集团公司西安航空计算技术研究所 | Model verification analysis method based on data configuration tool |
| CN109389136A (en) * | 2017-08-08 | 2019-02-26 | 上海为森车载传感技术有限公司 | Classifier training method |
| US20190139179A1 (en) * | 2017-11-03 | 2019-05-09 | Baidu Usa Llc | Systems and methods for unsupervised learning of geometry from images using depth-normal consistency |
| CN108229651A (en) * | 2017-11-28 | 2018-06-29 | 北京市商汤科技开发有限公司 | Neural network model moving method and system, electronic equipment, program and medium |
Non-Patent Citations (2)
| Title |
|---|
| JASONAIDM: "深度学习模型转换", pages 1 - 22, Retrieved from the Internet <URL:https://blog.csdn.net/jasonaidm/article/details/90522615?spm=1001.2014.3001.5502> * |
| 张志超 等: "一种基于神经网络的SQL注入漏洞的检测模型", 计算机与现代化, no. 10, pages 67 - 71 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113127058A (en) * | 2021-04-28 | 2021-07-16 | 北京百度网讯科技有限公司 | Data annotation method, related device and computer program product |
| CN113127058B (en) * | 2021-04-28 | 2024-01-16 | 北京百度网讯科技有限公司 | Data labeling method, related device and computer program product |
| CN115187821A (en) * | 2022-07-05 | 2022-10-14 | 阿波罗智能技术(北京)有限公司 | Method for verifying correctness before and after model conversion, related device and program product |
| CN115187821B (en) * | 2022-07-05 | 2024-03-22 | 阿波罗智能技术(北京)有限公司 | Method, related device and program product for verifying correctness of model before and after conversion |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110276443B (en) | 2024-07-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2019085087A1 (en) | Interface testing and test data generation method, device, terminal, and storage medium | |
| CN109509056B (en) | Commodity recommendation method based on countermeasure network, electronic device and storage medium | |
| US9983984B2 (en) | Automated modularization of graphical user interface test cases | |
| CN108491326B (en) | Test behavior a recombination process apparatus and storage medium | |
| US7340475B2 (en) | Evaluating dynamic expressions in a modeling application | |
| CN110825619A (en) | Automatic generation method and device of interface test case and storage medium | |
| CN110389896A (en) | Code automated analysis and test method, device and computer readable storage medium | |
| CN108415826B (en) | Application testing method, terminal device and computer readable storage medium | |
| WO2019169760A1 (en) | Test case range determining method, device, and storage medium | |
| CN111178512B (en) | Device operation neural network test method and device | |
| WO2016026328A1 (en) | Information processing method and device and computer storage medium | |
| KR102151318B1 (en) | Method and apparatus for malicious detection based on heterogeneous information network | |
| CN113435608B (en) | Method and device for generating federal learning model, computer equipment and storage medium | |
| CN113254354A (en) | Test case recommendation method and device, readable storage medium and electronic equipment | |
| CN112597718B (en) | Verification method, verification device and storage medium for integrated circuit design | |
| CN110276443B (en) | Model conversion consistency verification and analysis method, device and storage medium | |
| US12001823B2 (en) | Systems and methods for building and deploying machine learning applications | |
| CN104182347A (en) | Test requirement automatic analysis method based on automatic test platform | |
| CN116245074A (en) | Chip verification method, device and storage medium | |
| CN107153608A (en) | Code detection method and code detecting apparatus | |
| US10915427B2 (en) | Equivalence verification apparatus and computer readable medium | |
| CN113505895A (en) | Machine learning engine service system, model training method and configuration method | |
| CN112416784A (en) | Interface checking method, system and device based on configuration center and storage medium | |
| CN107305522A (en) | The apparatus and method detected for the repetition collapse to application program | |
| CN112464636B (en) | Constraint file comparison method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |