CN114997397A - Model conversion method, device, terminal equipment and storage medium - Google Patents
Model conversion method, device, terminal equipment and storage medium Download PDFInfo
- Publication number
- CN114997397A CN114997397A CN202210913699.5A CN202210913699A CN114997397A CN 114997397 A CN114997397 A CN 114997397A CN 202210913699 A CN202210913699 A CN 202210913699A CN 114997397 A CN114997397 A CN 114997397A
- Authority
- CN
- China
- Prior art keywords
- model
- operator
- tensorflow
- basic
- basic operator
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000006243 chemical reaction Methods 0.000 title claims abstract description 36
- 238000000034 method Methods 0.000 title claims abstract description 30
- 239000011159 matrix material Substances 0.000 claims abstract description 29
- 238000012549 training Methods 0.000 claims abstract description 11
- 238000011426 transformation method Methods 0.000 claims description 12
- 230000005012 migration Effects 0.000 claims description 5
- 238000013508 migration Methods 0.000 claims description 5
- 238000010276 construction Methods 0.000 claims description 4
- 238000013519 translation Methods 0.000 claims description 2
- 238000013473 artificial intelligence Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 11
- 230000009466 transformation Effects 0.000 description 8
- 230000008569 process Effects 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/10—Interfaces, programming languages or software development kits, e.g. for simulating neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/40—Transformation of program code
- G06F8/52—Binary to binary
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/94—Hardware or software architectures specially adapted for image or video understanding
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Multimedia (AREA)
- Life Sciences & Earth Sciences (AREA)
- Molecular Biology (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Biophysics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Biomedical Technology (AREA)
- Image Processing (AREA)
Abstract
The invention relates to the technical field of artificial intelligence, and provides a model conversion method, a device, terminal equipment and a storage medium, wherein the method comprises the steps of acquiring a first basic operator of a Pythrch model obtained by pre-training, wherein the first basic operator comprises a weight matrix; constructing a Tensorflow model corresponding to the Pythrch model according to the first basic operator, and modifying a second basic operator of the Tensorflow model; and migrating the weight matrix to a Tensorflow model according to the matching relationship between the first basic operator and the second basic operator to obtain the Tensorflow model. By adopting the model conversion method, the lossless conversion of the Pythrch model and the Tensorflow model can be realized without using middleware, and the conversion efficiency is improved.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to a model conversion method, a model conversion device, terminal equipment and a storage medium.
Background
As a deep learning frame, the Pythrch frame has the advantages of low learning cost, strong readability, convenient training and the like compared with a Tensorflow frame, but the Tensorflow frame has a wider application range. Therefore, for the convenience of training and model building, a pytorech framework needs to be used, and for the deployment of models on more devices, a tensoflow framework needs to be used.
For example, chinese patent application with publication number CN114327525A proposes a deep learning model conversion deployment method, which uses a deconvolution function to perform an upsampling operation on a model file to be processed, uses a normalization process to obtain an intermediate file, and uses an activation function to correct the intermediate file, thereby solving the problem of converting from a Pytorch model to a TensorRT model.
At present, in the related art, middleware such as Open Neural Network Exchange (ONNX) is used for converting a pitoch framework and a tensoflow framework, so that the operation is complex, the conversion failure is easily caused by mismatch between versions, and the performance loss is increased.
Disclosure of Invention
In view of the above-mentioned drawbacks and deficiencies of the related art, it is desirable to provide a model conversion method, apparatus, terminal device and storage medium, which can perform model lossless conversion directly without using middleware, thereby improving the conversion efficiency.
In a first aspect, the present invention provides a model transformation method, comprising:
acquiring a first basic operator of a Pythrch model obtained by pre-training, wherein the first basic operator comprises a weight matrix;
constructing a Tensorflow model corresponding to the Pythrch model according to the first basic operator, and modifying a second basic operator of the Tensorflow model;
and migrating the weight matrix to a Tensorflow model according to the matching relation between the first basic operator and the second basic operator to obtain the Tensorflow model.
Optionally, in some embodiments of the present invention, the first basic operator and the second basic operator are Conv operators, and modifying the second basic operator of the tensflow version model includes:
calculating a Padding value corresponding to the Padding operation of the first basic operator;
if the Padding value is a non-integer, adding a Padding operator before the second base operator, and canceling the Padding operation of the second base operator.
Optionally, in some embodiments of the present invention, the Padding value corresponding to the Padding operation of the first base operator is calculated as follows:
wherein,
the value of the padding is represented by,
which is indicative of the size of the output image,
which represents the size of the input image,
which represents the size of the convolution kernel and,
representing the step size of the convolution kernel.
Optionally, in some embodiments of the present invention, modifying the second basic operator of the tensrflow model further includes:
detecting whether the groups parameter of the first basic operator is equal to the input _ channel parameter or not;
if the groups parameter is equal to the input _ channel parameter, the second base operator is replaced with the DepthwiseConv2d operator in the Pytorch model.
Optionally, in some embodiments of the present invention, the first basic operator and the second basic operator are bilinear upsampling operators, and the modifying the second basic operator of the tensflow model further includes:
acquiring the corresponding position of each pixel point in the input image in the pixel grid of the output image;
and respectively calculating the pixel values of the pixel points to be filled in the pixel grid according to the corresponding positions, and outputting the up-sampled image.
Optionally, in some embodiments of the present invention, before migrating the weight matrix to the tensoflow version model, the model conversion method further includes performing dimension conversion on the weight matrix.
Optionally, in some embodiments of the present invention, constructing a tensoflow model corresponding to the Pytorch model according to the first basic operator includes:
generating a basic operator list based on the position sequence of the first basic operator in the Pythrch model;
and sequentially acquiring the category and the attribute of each first basic operator in the basic operator list, and arranging second basic operators corresponding to each first basic operator in the Tensorflow framework to obtain the Tensorflow model.
In a second aspect, the present invention provides a model transformation apparatus, comprising:
the acquisition module is configured to acquire a first basic operator of a Pythrch model obtained by pre-training, wherein the first basic operator comprises a weight matrix;
the construction module is configured to construct a Tensorflow model corresponding to the Pythrch model according to the first basic operator, and modify a second basic operator of the Tensorflow model;
and the migration module is configured and used for migrating the weight matrix to the Tensorflow model according to the matching relation between the first basic operator and the second basic operator to obtain the Tensorflow model.
In a third aspect, the present invention provides a terminal device comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a set of codes, or a set of instructions, which is loaded and executed by the processor to implement the steps of the model translation method described in the first aspect.
In a fourth aspect, the present invention provides a computer-readable storage medium storing one or more programs, the one or more programs being executable by one or more processors to implement the steps of the model transformation method described in the first aspect.
The technical scheme of the invention has the following advantages that:
the invention provides a model conversion method, a device, a terminal device and a storage medium, wherein a first basic operator of a Pythroch model obtained by pre-training is used as a reference to construct a Tensorflow model corresponding to the Pythroch model, and a second basic operator of the Tensorflow model is modified, so that a weight matrix is transferred to the Tensorflow model according to a matching relation between the two basic operators to obtain the Tensorflow model.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:
fig. 1 is a schematic flowchart of a model transformation method according to an embodiment of the present invention;
FIG. 2 is a schematic structural diagram of a Pythrch model according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a convolution operation according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a padding operation according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of another padding operation according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of grid alignment according to an embodiment of the present invention;
fig. 7 is a schematic diagram of bilinear upsampling according to an embodiment of the present invention;
FIG. 8 is a schematic structural diagram of a model transformation apparatus according to an embodiment of the present invention;
FIG. 9 is a schematic structural diagram of another model transformation apparatus according to an embodiment of the present invention;
FIG. 10 is a schematic structural diagram of another model transformation apparatus according to an embodiment of the present invention;
fig. 11 is a schematic structural diagram of another model transformation apparatus according to an embodiment of the present invention.
Detailed Description
In order to make those skilled in the art better understand the technical solutions of the present invention, the technical solutions in the embodiments will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments described are capable of operation in sequences other than those illustrated or described herein.
Moreover, the terms "comprises," "comprising," and any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or modules is not necessarily limited to those steps or modules explicitly listed, but may include other steps or modules not expressly listed or inherent to such process, method, article, or apparatus.
It should be noted that the features of the embodiments and the examples may be combined with each other without conflict. The present invention will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
For convenience of understanding and explanation, the model conversion method, the model conversion apparatus, the terminal device, and the storage medium according to the embodiments of the present invention are described in detail below with reference to fig. 1 to 11.
Please refer to fig. 1, which is a flowchart illustrating a model transformation method according to an embodiment of the present invention, the model transformation method includes the following steps:
s101, acquiring a first basic operator of a Pythrch model obtained through pre-training, wherein the first basic operator comprises a weight matrix.
It should be noted that the Pytorch model is composed of basic operators in a Pytorch frame, for example, as shown in fig. 2, the Pytorch model includes a module 1 and a module 2, the module 1 includes a basic operator 1, a basic operator 2, and the like, and the module 2 includes a basic operator 1, a basic operator 2, and the like.
For example, the first base operator of the pytorech model can be obtained through the self-contained interface of the pytorech framework, for example, the first base operator includes base operator 1 and base operator 2 of module 1, and base operator 1 and base operator 2 of module 2.
S102, constructing a Tensorflow model corresponding to the Pythrch model according to the first basic operator, and modifying a second basic operator of the Tensorflow model.
For example, in the embodiment of the present invention, a basic operator list may be generated based on a position sequence of a first basic operator in a Pytorch model, a category and an attribute of each first basic operator in the basic operator list may be sequentially obtained, and a second basic operator corresponding to each first basic operator is arranged in a tensoflow frame, so as to obtain a tensoflow version model, where the second basic operator is provided by the tensoflow frame.
Preferably, in some embodiments of the present invention, the first basic operator and the second basic operator are Conv (convolution) operators, and since the convolution operation uses an n × n matrix (i.e., convolution kernel) as a sliding window, please refer to fig. 3 to move on a pixel grid of a picture, and use the calculation result as a pixel of a new image, and the size of the output image changes due to the size and the step size of the convolution kernel, at this time, the calculation formula of the output image is as shown in equation (1), that is:
wherein,
which represents the size of the output image,
which represents the size of the input image,
which represents the size of the convolution kernel,
representing the step size of the convolution kernel.
For example, the input image is [ [1,1,1,1], [1,1,1,1], [1,1,1,1], [1,1,1,1] ], and the output image is [0.37,0.37,0.37,0.37 ].
In order to ensure that the size of the output image is consistent with that of the input image, the image needs to be padded, namely Padding operation, and a calculation formula after Padding operation is added is as shown in formula (2), namely:
wherein,
and represents the Padding value corresponding to the Padding operation, namely the pixel value required to pad the input image.
Further, when
When they are integers, e.g. as shown in FIG. 4
Namely, the periphery of the input image needs to be filled with one circle;
for example, the input images before Padding operation are:
[[1,1,1,1],[1,1,1,1],[1,1,1,1],[1,1,1,1]]；
the input image after Padding operation is:
[[0,0,0,0,0,0],[0,1,1,1,1,0],[0,1,1,1,1,0],[0,1,1,1,1,0],[0,1,1,1,1,0],[0,0,0,0,0,0]]。
but when
Is a non-integer number, i.e.
The Pytorch model and the tensoflow model may perform different Padding operations, for example, the Pytorch model shown in fig. 5 may fill half a circle above and to the left of the image, and the tensoflow model may fill a half circle to the right and below of the image;
for example, the pytorech model corresponds to:
[[0,0,0,0,0],[0,1,1,1,1],[0,1,1,1,1],[0,1,1,1,1],[0,1,1,1,1]]；
the Tensorflow model corresponds to:
[[1,1,1,1,0],[1,1,1,1,0],[1,1,1,1,0],[1,1,1,1,0],[0,0,0,0,0]]。
therefore, in order to ensure that the input and output of the Tensorflow model and the Pythrch model are consistent, the embodiment of the invention judges the Conv operator of the Pythrch model, namely calculates the Padding value corresponding to the Padding operation of the first base operator
If the padding value is not satisfied
If the value is not an integer, adding a Padding operator before the second basic operator to simulate Padding operation of the Pythrch model and canceling the Padding operation of the second basic operator, and if the value is filled in
For integers, no change is made to the second base operator.
Wherein Padding values corresponding to Padding operations of the first base operator
This is obtained by modifying the formula (2), i.e., the formula (3):
preferably, some embodiments of the invention are such that the DwConv (depth separable convolution) operator and Conv operator of the Pytorch model are the same operator, while the DwConv operator and Conv operator of the tenserflow model are two separate operators. Therefore, in the embodiment of the present invention, the DwConv operator and the Conv operator are distinguished by detecting whether the groups parameter of the first basic operator is equal to the input _ channel parameter, and in the case that the groups parameter is equal to the input _ channel parameter, the second basic operator is replaced by the DepthwiseConv2d operator in the Pytorch model, where the second basic operator is the Conv operator.
Preferably, in some embodiments of the present invention, the first and second base operators are upsampling bilinear2d (bilinear upsampling) operators, which are used to enlarge the image. Manual calculation is required because bilinear upsampling of the pytorech model defaults to on-corner alignment (align _ horns), while the tensoflow model defaults to off-corner alignment, and the corner alignment operation is not available on some terminal devices.
Therefore, in the embodiment of the present invention, firstly, grid alignment is performed on an input image, for example, as shown in fig. 6, each pixel point in the input image is placed in a pixel grid of an output image by using an align _ kernel = True method, and a corresponding position is obtained, and then according to the corresponding position, pixel values of pixel points to be filled in the pixel grid are respectively calculated, so as to output an up-sampled image. For example, fig. 7 is a schematic diagram of bilinear upsampling provided by an embodiment of the present invention, wherein Q is 11 、Q 12 、Q 21 And Q 22 For a known pixel value, R 1 Has the coordinates of (x，y 1 ），R 2 Has the coordinates of (x，y 2 ) To is aThe pixel value of the P point is obtained and can be calculated by equations (4) to (6), that is:
for example, the input image is [ [1,2,3], [4,5,6], [7,8,9] ], and the output image is [ [1.0, 1.4, 1.8, 2.2, 2.6, 3.0], [2.2, 2.6, 3.0, 3.4, 3.8, 4.2], [3.4, 3.8, 4.2, 4.6, 5.0, 5.4], [4.6, 5.0, 5.4, 5.8, 6.2, 6.6], [5.8, 6.2, 6.6, 7.0, 7.4, 7.8], [7.0, 7.4, 7.8, 8.2, 8.6, 9.0] ].
S103, migrating the weight matrix to a Tensorflow model according to the matching relation of the first basic operator and the second basic operator to obtain the Tensorflow model.
For example, the embodiment of the present invention may perform matching according to the order, the weight type, and the weight matrix size of the basic operators in the two models, generate an operator matching index list, and then migrate the weight matrix to the tensoflow version model according to the operator matching index list.
Preferably, some embodiments of the present invention perform dimension conversion on the weight matrix before migrating the weight matrix to the tensrflow model, because the format of the picture input model in the Pytorch frame is [ N, C, H, W ], that is [ number of pictures, number of picture channels, height of pictures, width of pictures ], and the format of the picture input in the tensrflow frame is [ N, H, W, C ], considering that the input formats are different. For example, the input NCHW is [ [ [ [ [1,2], [3,4] ] ] ], and the output NHWC is [ [ [ [1], [2], [3], [4] ] ]. In addition, the weight (weight) and the offset (bias) of the Conv operator in the Pythroch frame are stored separately, and the weight and the offset are placed in a list in the Tensorflow frame, so that when a variable is migrated, the weight and the offset of the Conv operator in the Pythroch model are firstly obtained, and then the weight and the offset are merged and placed in the Tensorflow model through dimension conversion.
It should be noted that embodiments of the present invention may store the tensoflow model in the format of h 5. And then reading the stored Tensorflow model, and testing the Tensorflow model by using false data with the same value, wherein the output deviation of the model is less than 1e-4, which means that lossless conversion is realized.
The embodiment of the invention provides a model conversion method, which comprises the steps of constructing a Tensorflow model corresponding to a Pythrch model by taking a first basic operator of the Pythrch model obtained by pre-training as a reference, modifying a second basic operator of the Tensorflow model, transferring a weight matrix to the Tensorflow model according to a matching relation between the two basic operators to obtain the Tensorflow model, realizing the lossless conversion of the Pythrch model and the Tensorflow model without using a middleware, simultaneously avoiding adding more operators when constructing the model, not influencing inference speed and improving conversion efficiency.
Based on the foregoing embodiments, an embodiment of the present invention provides a model conversion apparatus. The model transformation apparatus 100 can be applied to the model transformation methods according to the embodiments of fig. 1 to 7. Referring to fig. 8, the model transformation apparatus 100 includes:
an obtaining module 101, configured to obtain a first basic operator of a pretrained Pytorch model, where the first basic operator includes a weight matrix;
the building module 102 is configured to build a Tensorflow model corresponding to the Pythrch model according to the first basic operator, and modify a second basic operator of the Tensorflow model;
and the migration module 103 is configured to migrate the weight matrix to the Tensorflow model according to the matching relationship between the first basic operator and the second basic operator to obtain the Tensorflow model.
Preferably, as shown in fig. 9, in some embodiments of the present invention, the first basic operator and the second basic operator are Conv operators, and the building module 102 includes:
a calculating unit 1021, configured to calculate a Padding value corresponding to a Padding operation of the first base operator;
an adding unit 1022, configured to add a Padding operator before the second basic operator and cancel the Padding operation of the second basic operator if the Padding value is a non-integer.
Preferably, in some embodiments of the present invention, the Padding value corresponding to the Padding operation of the first base operator is calculated as follows:
wherein,
the value of the padding is represented by,
which is indicative of the size of the output image,
which represents the size of the input image,
which represents the size of the convolution kernel,
representing the step size of the convolution kernel.
Preferably, as shown in fig. 10, the building module 102 in some embodiments of the present invention further includes:
a detecting unit 1023, configured to detect whether the groups parameter of the first base operator is equal to the input _ channel parameter;
a replacing unit 1024, configured to replace the second base operator with a DepthwiseConv2d operator in the Pytorch model if the groups parameter is equal to the input _ channel parameter.
Preferably, as shown in fig. 11, in some embodiments of the present invention, the first basic operator and the second basic operator are bilinear upsampling operators, and the building module 102 further includes:
an obtaining unit 1025 configured to obtain a corresponding position of each pixel point in the input image in a pixel grid of the output image;
and an output unit 1026, configured to calculate pixel values of the pixels to be filled in the pixel grid according to the corresponding positions, and output the upsampled image.
Preferably, in some embodiments of the present invention, the migration module 103 is further configured to perform dimension transformation on the weight matrix before migrating the weight matrix to the tensrflow model.
Preferably, in some embodiments of the present invention, the constructing module 102 is further configured to generate a list of basic operators based on a position order of the first basic operator in the Pytorch model; and the number of the first and second groups,
and sequentially acquiring the category and the attribute of each first basic operator in the basic operator list, and arranging second basic operators corresponding to each first basic operator in the Tensorflow framework to obtain the Tensorflow model.
The embodiment of the invention provides a model conversion device, wherein an acquisition module in the model conversion device can acquire a first basic operator of a Pyorch model obtained by pre-training, the first basic operator comprises a weight matrix, a construction module can construct a Tensorflow model corresponding to the Pyorch model by taking the first basic operator as a reference, and modify a second basic operator of the Tensorflow model, so that a migration module can migrate the weight matrix to the Tensorflow model according to the matching relation between the two basic operators to obtain the Tensorflow model, thereby realizing the lossless conversion of the Pyorch model and the Tensorflow model without using an intermediate piece, simultaneously, no extra operator exists during model construction, the inference speed is not influenced, and the conversion efficiency is improved.
Based on the foregoing embodiments, an embodiment of the present invention provides a terminal device, where the terminal device includes a processor and a memory. The memory has stored therein at least one instruction, at least one program, set of codes, or set of instructions that is loaded and executed by the processor to implement the steps of the model transformation method of the corresponding embodiment of fig. 1-7.
As another aspect, an embodiment of the present invention provides a computer-readable storage medium for storing program code for implementing any one of the foregoing model conversion methods according to the corresponding embodiments in fig. 1 to 7.
It can be clearly understood by those skilled in the art that, for convenience and simplicity of description, the specific working processes of the system, the apparatus, and the module described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the embodiments provided in the present invention, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is merely a logical division, and in actual implementation, there may be other divisions, for example, multiple modules or components may be combined or integrated into another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or modules, and may be in an electrical, mechanical or other form. Modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one position, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing unit, or each module may exist alone physically, or two or more units are integrated into one module. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit. The integrated unit, if implemented as a software functional unit and sold or used as a separate product, may be stored in a computer readable storage medium.
Based on such understanding, the technical solution of the present invention may be substantially or partially implemented in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the model transformation method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, an optical disk, or other various media capable of storing program codes.
It should be noted that the above embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A model transformation method, characterized in that the model transformation method comprises:
acquiring a first basic operator of a Pythrch model obtained by pre-training, wherein the first basic operator comprises a weight matrix;
constructing a Tensorflow model corresponding to the Pyorch model according to the first basic operator, and modifying a second basic operator of the Tensorflow model;
and transferring the weight matrix to the Tensorflow model according to the matching relation between the first basic operator and the second basic operator to obtain the Tensorflow model.
2. The model conversion method of claim 1, wherein said first and second base operators are Conv operators, and said modifying said second base operator of said Tensorflow version of said model comprises:
calculating a Padding value corresponding to the Padding operation of the first basic operator;
if the Padding value is a non-integer, adding a Padding operator before the second basic operator, and canceling the Padding operation of the second basic operator.
3. The model transformation method of claim 2, wherein the Padding value corresponding to Padding operation of the first base operator is calculated as follows:
wherein,
the value of the padding is represented by,
which is indicative of the size of the output image,
which represents the size of the input image,
which represents the size of the convolution kernel,
representing the step size of the convolution kernel.
4. The model conversion method according to claim 2, wherein said modifying the second base operator of the Tensorflow version of the model further comprises:
detecting whether the groups parameter of the first basic operator is equal to the input _ channel parameter or not;
and if the groups parameter is equal to the input _ channel parameter, replacing the second basic operator with a DepthWisseConv 2d operator in the Pythrch model.
5. The method for model conversion according to claim 1, wherein the first base operator and the second base operator are bilinear upsampling operators, and the modifying the second base operator of the Tensorflow version model further comprises:
acquiring the corresponding position of each pixel point in the input image in the pixel grid of the output image;
and respectively calculating the pixel values of the pixel points to be filled in the pixel grids according to the corresponding positions, and outputting the up-sampled image.
6. The model conversion method according to claim 1, further comprising dimension converting the weight matrix before migrating the weight matrix to the Tensorflow version model.
7. The model conversion method according to any one of claims 1 to 6, wherein the constructing a Tensorflow version model corresponding to the Pythrch model according to the first base operator comprises:
generating a base operator list based on the position sequence of the first base operator in the Pythrch model;
and sequentially acquiring the category and the attribute of each first basic operator in the basic operator list, and arranging the second basic operators corresponding to each first basic operator in a Tensorflow frame to obtain the Tensorflow model.
8. A model conversion apparatus, characterized by comprising:
the acquisition module is configured to acquire a first basic operator of a Pythrch model obtained through pre-training, wherein the first basic operator comprises a weight matrix;
the construction module is configured to construct a Tensorflow model corresponding to the Pythrch model according to the first basic operator, and modify a second basic operator of the Tensorflow model;
and the migration module is configured to migrate the weight matrix to the Tensorflow model according to the matching relationship between the first basic operator and the second basic operator to obtain the Tensorflow model.
9. A terminal device, characterized in that it comprises a processor and a memory, in which at least one instruction, at least one program, set of codes or set of instructions is stored, which is loaded and executed by the processor to implement the steps of the model translation method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores one or more programs which are executable by one or more processors to implement the steps of the model transformation method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210913699.5A CN114997397B (en) | 2022-08-01 | 2022-08-01 | Model conversion method and device, terminal equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210913699.5A CN114997397B (en) | 2022-08-01 | 2022-08-01 | Model conversion method and device, terminal equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114997397A true CN114997397A (en) | 2022-09-02 |
| CN114997397B CN114997397B (en) | 2022-10-21 |
Family
ID=83022380
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210913699.5A Active CN114997397B (en) | 2022-08-01 | 2022-08-01 | Model conversion method and device, terminal equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114997397B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101551903A (en) * | 2009-05-11 | 2009-10-07 | 天津大学 | Super-resolution image restoration method in gait recognition |
| CN107808664A (en) * | 2016-08-30 | 2018-03-16 | 富士通株式会社 | Audio recognition method, speech recognition equipment and electronic equipment based on sparse neural network |
| CN108764487A (en) * | 2018-05-29 | 2018-11-06 | 北京百度网讯科技有限公司 | For generating the method and apparatus of model, the method and apparatus of information for identification |
| CN111291882A (en) * | 2018-12-06 | 2020-06-16 | 北京百度网讯科技有限公司 | Model conversion method, device, equipment and computer storage medium |
| CN111753948A (en) * | 2020-06-23 | 2020-10-09 | 展讯通信（上海）有限公司 | Model processing method and related equipment |
| CN112541159A (en) * | 2020-09-30 | 2021-03-23 | 华为技术有限公司 | Model training method and related equipment |
| CN114491399A (en) * | 2021-12-30 | 2022-05-13 | 深圳云天励飞技术股份有限公司 | Data processing method and device, terminal equipment and computer readable storage medium |
| WO2022141513A1 (en) * | 2020-12-31 | 2022-07-07 | 华为技术有限公司 | Processing method and apparatus for model |
-
2022
- 2022-08-01 CN CN202210913699.5A patent/CN114997397B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101551903A (en) * | 2009-05-11 | 2009-10-07 | 天津大学 | Super-resolution image restoration method in gait recognition |
| CN107808664A (en) * | 2016-08-30 | 2018-03-16 | 富士通株式会社 | Audio recognition method, speech recognition equipment and electronic equipment based on sparse neural network |
| CN108764487A (en) * | 2018-05-29 | 2018-11-06 | 北京百度网讯科技有限公司 | For generating the method and apparatus of model, the method and apparatus of information for identification |
| CN111291882A (en) * | 2018-12-06 | 2020-06-16 | 北京百度网讯科技有限公司 | Model conversion method, device, equipment and computer storage medium |
| CN111753948A (en) * | 2020-06-23 | 2020-10-09 | 展讯通信（上海）有限公司 | Model processing method and related equipment |
| CN112541159A (en) * | 2020-09-30 | 2021-03-23 | 华为技术有限公司 | Model training method and related equipment |
| WO2022141513A1 (en) * | 2020-12-31 | 2022-07-07 | 华为技术有限公司 | Processing method and apparatus for model |
| CN114491399A (en) * | 2021-12-30 | 2022-05-13 | 深圳云天励飞技术股份有限公司 | Data processing method and device, terminal equipment and computer readable storage medium |
Non-Patent Citations (1)
| Title |
|---|
| HO YOUNG JHOO等: "A Static Analyzer for Detecting Tensor Shape Errors in Deep Neural Network Training Code", 《 2022 IEEE/ACM 44TH INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING: COMPANION PROCEEDINGS (ICSE-COMPANION)》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114997397B (en) | 2022-10-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11030782B2 (en) | Accurately generating virtual try-on images utilizing a unified neural network framework | |
| CN111898701B (en) | Model training, frame image generation and frame insertion methods, devices, equipment and media | |
| CN111369440B (en) | Model training and image super-resolution processing method, device, terminal and storage medium | |
| EP4064189A1 (en) | Image processing method, apparatus and device, and storage medium | |
| JP6961139B2 (en) | An image processing system for reducing an image using a perceptual reduction method | |
| Wohlberg et al. | An iteratively reweighted norm algorithm for minimization of total variation functionals | |
| JP2021100247A (en) | Distorted document image correction method and device | |
| CN109146788A (en) | Super-resolution image reconstruction method and device based on deep learning | |
| CN109871841A (en) | Image processing method, device, terminal and storage medium | |
| CN114387317B (en) | CT image and MRI three-dimensional image registration method and device | |
| CN110335330A (en) | Image simulation generation method and its system, deep learning algorithm training method and electronic equipment | |
| CN115170418A (en) | Degradation-compliant low-rank high-dimensional image filling model and filling method and system thereof | |
| KR102537207B1 (en) | Method for processing image based on machine learning and apparatus therefof | |
| CN111068314B (en) | NGUI resource rendering processing method and device based on Unity | |
| CN108701360A (en) | Image processing system and method | |
| CN109829963B (en) | Image drawing method and device, computing equipment and storage medium | |
| CN114997397B (en) | Model conversion method and device, terminal equipment and storage medium | |
| WO2023246403A1 (en) | Model training method, watermark restoration method, and related device | |
| CN110975284A (en) | Unity-based NGUI resource rendering processing method and device | |
| US20230060988A1 (en) | Image processing device and method | |
| CA3177593A1 (en) | Transformer-based shape models | |
| CN115272527A (en) | Image coloring method based on color disc countermeasure network | |
| CN115112134A (en) | Map construction method, device, equipment and storage medium | |
| CN114140340A (en) | Image data artifact processing method and device, storage medium and terminal | |
| CN113706415A (en) | Training data generation method, countermeasure sample generation method, image color correction method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |