CN116266117A - Model conversion method, computer program product, storage medium, and electronic device - Google Patents
Model conversion method, computer program product, storage medium, and electronic device Download PDFInfo
- Publication number
- CN116266117A CN116266117A CN202210239286.3A CN202210239286A CN116266117A CN 116266117 A CN116266117 A CN 116266117A CN 202210239286 A CN202210239286 A CN 202210239286A CN 116266117 A CN116266117 A CN 116266117A
- Authority
- CN
- China
- Prior art keywords
- conversion
- model
- file
- environment
- model file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000006243 chemical reaction Methods 0.000 title claims abstract description 737
- 238000000034 method Methods 0.000 title claims abstract description 131
- 238000003860 storage Methods 0.000 title claims abstract description 20
- 238000004590 computer program Methods 0.000 title claims abstract description 17
- 230000008569 process Effects 0.000 claims description 74
- 230000009466 transformation Effects 0.000 claims description 35
- 238000013515 script Methods 0.000 claims description 34
- 238000012360 testing method Methods 0.000 claims description 34
- 238000010276 construction Methods 0.000 claims description 15
- 230000000007 visual effect Effects 0.000 claims description 13
- 238000011426 transformation method Methods 0.000 claims description 4
- 238000011022 operating instruction Methods 0.000 claims description 3
- 238000012545 processing Methods 0.000 abstract description 9
- 238000011161 development Methods 0.000 description 17
- 238000001514 detection method Methods 0.000 description 11
- 230000006870 function Effects 0.000 description 10
- 238000005516 engineering process Methods 0.000 description 8
- 238000013139 quantization Methods 0.000 description 8
- 238000013519 translation Methods 0.000 description 8
- 238000012795 verification Methods 0.000 description 8
- 238000013473 artificial intelligence Methods 0.000 description 7
- 230000008901 benefit Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 5
- 238000010801 machine learning Methods 0.000 description 4
- 238000003062 neural network model Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 230000003796 beauty Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 230000010076 replication Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000009529 body temperature measurement Methods 0.000 description 1
- 238000003066 decision tree Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000011990 functional testing Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000012706 support-vector machine Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/70—Software maintenance or management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/60—Software deployment
- G06F8/65—Updates
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/70—Software maintenance or management
- G06F8/71—Version control; Configuration management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Biomedical Technology (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Computer Security & Cryptography (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Stored Programmes (AREA)
Abstract
The application relates to the technical field of image processing, and provides a model conversion method, a computer program product, a storage medium and electronic equipment. The model conversion method comprises the following steps: acquiring an original model file to be converted and converting configuration parameters; constructing a conversion environment according to the conversion configuration parameters, wherein the conversion environment is an environment for the operation of a conversion tool; and converting the original model file by using a conversion tool under a conversion environment to obtain a converted target model file. According to the method, a conversion environment is automatically built according to the conversion configuration parameters, and the model conversion is automatically completed, so that the model conversion efficiency is remarkably improved, the artificial risk is avoided, and the labor cost is reduced.
Description
Technical Field
The present invention relates to the field of machine learning technology, and in particular, to a model conversion method, a computer program product, a storage medium, and an electronic device.
Background
The photographing quality of modern mobile phones is continuously improved, more complex scenes and problems can be dealt with, and the remarkable changes benefit from the application of the neural network model at the mobile terminal. However, there is a large difference on the deployment side from the different chips on which different handset devices depend: the neural network model obtained under the same training frame is different in the inference frames applied when deployed on mobile phones of different manufacturers, so that after the model is trained, the model can be really put into use only by the processes of conversion, adaptation, halving, performance evaluation and the like from the training model to the inference model.
Currently, for model conversion, a common practice is to manually deploy one or more conversion environments on a server for manual conversion, which is a long time, easy to introduce human errors and unfavorable for quick deployment of models.
Disclosure of Invention
An object of the embodiments of the present application is to provide a model conversion method, a computer program product, a storage medium and an electronic device, so as to improve the above technical problems.
In order to achieve the above purpose, the present application provides the following technical solutions:
in a first aspect, an embodiment of the present application provides a model conversion method, including: acquiring an original model file to be converted and converting configuration parameters; constructing a conversion environment according to the conversion configuration parameters, wherein the conversion environment is an environment for a conversion tool to operate; and converting the original model file by using the conversion tool under the conversion environment to obtain a converted target model file.
According to the method, the conversion environment is automatically built according to the conversion configuration parameters, the model conversion is automatically completed, the model conversion efficiency is remarkably improved, the artificial risk is avoided, and the labor cost is reduced.
In addition, the method can be realized as an online model conversion system, and a user uniformly performs model conversion through the system, namely a set of standardized model conversion steps are formed, and the configuration and resource management of a conversion environment are not performed by means of personal experience of the user, so that the problems that error reproduction is difficult in the conversion process, the conversion process is difficult to scale, and the system is difficult to continue iteration when human resources are changed (for example, experienced developers leave the office) are solved. And when a new device is introduced, all users can benefit by only carrying out corresponding update on the system (supporting a new conversion flow), thereby being beneficial to the rapid introduction and verification of the new device.
Note that the model in the above method may be a neural network model, but may be other models.
In an implementation manner of the first aspect, the building a conversion environment according to the conversion configuration parameter includes: obtaining a basic mirror image universal for model conversion, and constructing a container according to the basic mirror image; acquiring the conversion tool and resources on which the conversion tool runs according to the conversion configuration parameters; the converting the original model file by using the converting tool in the converting environment to obtain a converted target model file, including: and converting the original model file in the container by using the conversion tool to obtain the target model file.
Different model transformation tasks depend on different transformation environments, and the base image can be considered as a common part in the transformation environments, for example, an operating system, a Python development kit, a c++ development kit and the like, so that the content of the part cannot be easily modified, and the correctness of the content can be ensured, so that a 'clean' container can be created based on the base image.
While the containers may be considered as a relatively independent workspace, personalized parts for different model transformation tasks, such as different transformation tools and the resources on which these transformation tools depend, may be added to each container. The container is used for executing the conversion tasks, so that each conversion task can independently run to a certain extent and is not interfered with each other, even if a certain conversion task is in error during running, the internal environment of the container where the conversion task is located is only destroyed at most, and the running of other conversion tasks is not influenced, thereby effectively guaranteeing the stability of the model conversion process and providing powerful support for batch conversion of the model. And, as the conversion task is executed, the container is destroyed, and the unresolved error is not left.
In an implementation manner of the first aspect, the conversion environment includes resources on which the conversion tool runs, and the building the conversion environment according to the conversion configuration parameters includes: and acquiring the resources from a resource management server according to the conversion configuration parameters.
The resource management server can uniformly manage the resources, and pull the resources to the server for executing the model conversion task when the conversion environment needs to be constructed, so that all users can be ensured to use the same resource set, and the problems of resource maintenance, scattered resource distribution and disordered resource version management of each user are avoided.
In an implementation manner of the first aspect, the converting, in the converting environment, the original model file with the converting tool to obtain the target model file includes: determining a conversion flow according to the conversion configuration parameters, wherein the conversion flow comprises at least one conversion module involved in model conversion; executing the conversion flow under the conversion environment to convert the original model file to obtain the target model file; the conversion module in the conversion flow calls the conversion tool when executing.
In the above implementation manner, each conversion link may be implemented as one conversion module (the conversion module may include some scripts or conversion tools), and then a complete conversion flow is formed by connecting the conversion modules in series, so that modularization of the conversion process is implemented. The conversion modules can be flexibly combined and reused in different conversion flows, and each conversion module can be independently maintained.
In an implementation manner of the first aspect, the resource is obtained from a resource management server, and the original model file is converted by executing a conversion flow in the container, where the conversion flow includes at least one conversion module involved in model conversion; the model transformation is performed in an in-line model transformation system, the updating of which comprises at least one of the following three: updating the base image: after a new basic image is tested on line, uploading a construction script corresponding to the new basic image to the line, and constructing the new basic image on the line based on the construction script; updating the resource: after testing a new resource package on line, uploading the new resource package to the resource management server; updating the conversion flow: and uploading the new conversion module to the line after testing the new conversion module on the line, and/or uploading the description parameters of the new conversion flow to the line after testing the new conversion flow on the line.
The above implementation provides an update process of the model conversion system, where the update process includes three layers, that is, an update of a base image, an update of a resource, and an update of a conversion process, and if there are all the three layers, the update may be performed sequentially. And the updated content is uniform for all system users, so that all users can benefit, and the problem of management confusion caused by independently updating and maintaining the method by each user is avoided.
In an implementation manner of the first aspect, the converting, in the converting environment, the original model file with the converting tool to obtain a converted target model file includes: performing multistage conversion on the original model file by using the conversion tool in the conversion environment to obtain the target model file; and each stage of conversion is used for obtaining an intermediate model file, and the intermediate model file obtained by the last stage of conversion is the target model file.
In the implementation manner, the original model file is not converted into the target model file once, but is converted into the target model file after intermediate conversion for several times, so that on one hand, the conversion difficulty can be reduced (the existing conversion tool may not support direct one-time completion of conversion), and in addition, the conversion flow can be split into a plurality of conversion modules, so that modular management is performed.
In an implementation manner of the first aspect, the obtaining an original model file to be converted and a conversion configuration parameter includes: acquiring the original model file submitted by a user from a visual interface of terminal equipment and the conversion configuration parameters; or receiving a model conversion notice sent by a code warehouse, responding to the model conversion notice, acquiring a conversion configuration file submitted by a user from the code warehouse, and acquiring the original model file and the conversion configuration parameters according to a storage position indicated by the conversion configuration file.
In the above implementation, the input data (the original model file and the conversion configuration parameters) required for the conversion may be obtained in two ways: firstly, input data is obtained from a visual interface (for example, a front-end page) of the terminal equipment, for example, a user can directly upload an original model file on the visual interface of the terminal equipment and select or input conversion configuration parameters on the visual interface; the second mode is to acquire input data by using a conversion configuration file in a code warehouse, the technical threshold of the mode is higher (a user is required to use the code warehouse such as GitLab), but the flexibility is better, and batch conversion is facilitated.
In one implementation manner of the first aspect, the model transformation is performed in a model transformation system on-line, and the method further includes: storing and constructing the conversion environment and operating instructions executed when converting the original model file to form a record file; the record file is used for reproducing the model conversion process on line and debugging the model conversion system.
Because the on-line system cannot be directly debugged, the conversion process needs to be reproduced on line, and a developer can position the problem. However, if the developer builds the conversion environment on line, and re-executes the model conversion, the time and the effort are wasted, and it is difficult to ensure that the off-line conversion process is a complete reproduction of the on-line conversion process.
The above implementation manner provides a debugging process of a model conversion system, in the debugging process, when the model is converted online, the executed conversion operation instruction is recorded to form a record file (for example, an executable script), when the model is debugged online, the conversion process can be quickly and accurately reproduced by directly using the record file, steps of building a conversion environment and the like by oneself are simplified, and the model can be quickly executed to a position where a problem occurs in the conversion process, so that a developer can conveniently and quickly locate an error cause.
In a second aspect, an embodiment of the present application provides a model conversion device, including: the conversion data acquisition unit is used for acquiring an original model file to be converted and conversion configuration parameters; the conversion environment construction unit is used for constructing a conversion environment according to the conversion configuration parameters, wherein the conversion environment is an environment for the conversion tool to operate; and the model conversion unit is used for converting the original model file by using the conversion tool under the conversion environment to obtain a converted target model file.
In a third aspect, embodiments of the present application provide a computer program product comprising computer program instructions which, when read and executed by a processor, perform the method provided by the first aspect or any one of the possible implementations of the first aspect.
In a fourth aspect, embodiments of the present application provide a computer readable storage medium having stored thereon computer program instructions which, when read and executed by a processor, perform the method provided by the first aspect or any one of the possible implementations of the first aspect.
In a fifth aspect, embodiments of the present application provide an electronic device, including: a memory and a processor, the memory having stored therein computer program instructions which, when read and executed by the processor, perform the method of the first aspect or any one of the possible implementations of the first aspect.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 illustrates a system architecture that may perform the model transformation method provided by embodiments of the present application;
FIG. 2 illustrates steps of a model transformation method provided in an embodiment of the present application;
FIG. 3 illustrates a transformation process of a model transformation system provided in an embodiment of the present application;
FIG. 4 illustrates a debugging process of a model conversion system provided by an embodiment of the present application;
FIG. 5 illustrates an update process of a model conversion system provided by an embodiment of the present application;
FIG. 6 illustrates a system framework of a model conversion system provided by an embodiment of the present application;
fig. 7 shows functional modules included in a model conversion device provided in an embodiment of the present application;
fig. 8 shows a structure of an electronic device provided in an embodiment of the present application.
Detailed Description
In recent years, technology research such as computer vision, deep learning, machine learning, image processing, image recognition and the like based on artificial intelligence has been advanced significantly. Artificial intelligence (Artificial Intelligence, AI for short) is an emerging scientific technology for studying and developing theories, methods, techniques and application systems for simulating and extending human intelligence. The artificial intelligence discipline is a comprehensive discipline and relates to various technical categories such as chips, big data, cloud computing, internet of things, distributed storage, deep learning, machine learning, neural networks and the like. Computer vision is an important branch of artificial intelligence, and particularly, machine recognition is used in the world, and computer vision technologies generally include technologies such as face recognition, living body detection, fingerprint recognition and anti-counterfeiting verification, biological feature recognition, face detection, pedestrian detection, object detection, pedestrian recognition, image processing, image recognition, image semantic understanding, image retrieval, word recognition, video processing, video content recognition, behavior recognition, three-dimensional reconstruction, virtual reality, augmented reality, synchronous positioning and map construction, computational photography, robot navigation and positioning, and the like. With research and progress of artificial intelligence technology, the technology expands application in various fields, such as security protection, city management, traffic management, building management, park management, face passing, face attendance, logistics management, warehouse management, robots, intelligent marketing, computed photography, mobile phone images, cloud services, intelligent home, wearing equipment, unmanned driving, automatic driving, intelligent medical treatment, face payment, face unlocking, fingerprint unlocking, personnel verification, intelligent screen, intelligent television, camera, mobile internet, network living broadcast, beauty, make-up, medical beauty, intelligent temperature measurement and the like. The model conversion method in the embodiment of the application also belongs to the category of artificial intelligence technology.
The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further definition or explanation thereof is necessary in the following figures.
The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The terms "first," "second," and the like, are used merely to distinguish one entity or action from another entity or action, and are not to be construed as indicating or implying any actual such relationship or order between such entities or actions.
Fig. 1 illustrates a system architecture that may perform the model conversion method provided in the embodiments of the present application. Referring to fig. 1, the architecture includes: terminal equipment 110, resource management server 120, conversion background server 130, code repository 140, and development device 150, the connection lines between the components representing possible data interactions between them.
The terminal device 110 may be a mobile phone, a PC, or other devices used by a user, where the terminal device 110 is configured to trigger a model conversion task (to convert one model into another model to be regarded as a model conversion task), and directly or indirectly submit input data (a model file to be converted, etc.) required for model conversion to the conversion back-end server 130, and the conversion back-end server 130 performs model conversion according to the received input data to obtain a conversion result (a converted model file, etc.), and may feed back the conversion result to the terminal device 110.
By directly submitting the input data to the translation background server 130, it may be meant that the user performs certain operations on a visual interface, where the "visual interface" may be a front page accessed by a browser running on the terminal device 110 or an interface within client software running on the terminal device 110, and submitting the input data to the background server 130.
By indirectly submitting input data to the translation background server 130, it may be meant that the user submits the input data to the background server 130 through the code repository 140 (the specific manner of submission will be described later). The code repository 140 may be understood herein as a background server of a code management platform, the specific type of which is not limited, and may be, for example, a GitLab.
It should be noted that the possibility of terminal device 110 automatically submitting data to translation background server 130 is not excluded.
The resource management server 120 is configured to uniformly manage resources related to conversion, and when performing model conversion, the conversion background server 130 may pull resources required for the conversion from the resource management server 120.
In fig. 1, the terminal device 110, the resource management server 120, the conversion backend server 130, and the code repository 140 all belong to an on-line environment (hereinafter, often abbreviated as on-line), meaning an environment that formally provides a model conversion service to a user, while the development device 150 belongs to an off-line environment (hereinafter, often abbreviated as off-line), meaning an environment for development, testing, and debugging by a developer, and is not open to external general users. In software development, software products are typically deployed to run on-line after testing is completed on-line.
Alternatively, a set of model conversion systems may be deployed on the conversion background server 130, where the model conversion method provided in the embodiments of the present application is integrated, and is used to provide a model conversion service to a user. In some implementations, the resource management server 120, code repository 140, and even the portion of the terminal device 110 that is relevant to the model conversion may also be considered part of the model conversion system.
It should be understood that the system architecture in fig. 1 is only an example, and some components are not necessary to implement the model conversion method provided in the embodiment of the present application, and other components may be used to implement the model conversion method provided in the embodiment of the present application: for example, code repository 140 may be omitted if the user submits input data only through the visual interface of terminal device 110; for another example, if the resources are not centrally managed or are placed on the conversion backend server 130, the resource management server 120 may be omitted; for another example, before the model conversion system is brought online, a developer may first deploy the model conversion system on the development device 150 for testing, and during the testing, the model conversion method may also be performed, where one or more components on the line may be omitted.
However, for simplicity, the model conversion method provided in the embodiment of the present application will be mainly described below in conjunction with the system architecture in fig. 1.
Fig. 2 shows steps of a model conversion method provided in an embodiment of the present application. The method may be, but is not limited to being, performed by a model conversion system deployed on the conversion backend server 130 in fig. 1, and the conversion backend server 130 may be, but is not limited to, employing the architecture of the electronic device in fig. 8. Referring to fig. 2, the model conversion method includes:
step S210: and obtaining an original model file to be converted and converting configuration parameters.
The model file is a file describing the model, and may include the structure, parameters, and the like of the model. The model here may be a neural network model, but is not excluded from other machine learning models, such as decision tree models, support vector machine models, etc.
The model file to be converted is called an original model file, the converted model file is called a target model file (see step S230), and in different occasions, the original model/target model may be referred to, or the original model file/target model file may be referred to, and it should be understood that there is no essential difference between the two, and the model file is just a specific expression mode of the model.
In addition, the solution of the present application does not limit the source of the original model file and the purpose of the target model file:
for example, the original model file may be a model file under a training framework, and the target model file may be a model file under an inference framework, for example, a model trained under a Caffe framework, which is expected to be deployed on a mobile phone (SNPE framework) using a high-pass platform, so as to complete conversion from a bin format file (a model file under the Caffe framework) to a dlc format file (a model file under the SNPE framework). Of course, it is also possible that the original model file is a model file under inference framework a, the object model file is a model file under inference framework B, i.e. one model is migrated from framework a to use under framework B, etc.
The conversion configuration parameters refer to parameters related to conversion, and the values of the parameters can be configured by a user. For example, the conversion configuration parameters include, but are not limited to, one or more of the following:
(1) Parameters related to the conversion target, such as target platform (e.g., high pass, MTK), target device (e.g., 6983 device, 6984 device, where the numbers represent chip model numbers), target frame (e.g., SNPE frame), version of target frame (e.g., 1.0, 2.0), etc.; note that this parameter may be omitted if the conversion target is fixed;
(2) Parameters related to the conversion source, like (1), are not described in detail; note that this parameter may be omitted if the conversion source is fixed;
(3) The quantization parameters of the model are specific to the situation that the original model file and the target model file are both quantization models; note that the model conversion itself may not include the operation of model quantization, but the expression of quantization parameters may also be converted when the model is converted;
(4) Converting command line parameters used by the tool (see step S220);
(5) Parameters for describing the constitution of the conversion flow, concepts concerning the conversion flow are explained later.
The different ways of obtaining the original model files and the transformation configuration parameters are described in the description of fig. 1, and are described in more detail herein:
mode 1: the model conversion system obtains the original model file submitted by the user and the conversion configuration parameters from the visual interface of the terminal device 110.
For example, buttons for uploading the original model file and the quantized parameter file can be provided on the front-end page for users to upload the file, controls such as an input box and a drop-down box can be provided on the page for users to input or select other parameters, buttons such as "submit" or "start conversion" can be provided on the page, after the user clicks, the file or parameter information is transferred from the terminal device 110 to the model conversion system, and the process can be realized by means of a jenkins service management tool.
Mode 2: the model conversion system receives the model conversion notification sent by the code repository 140, obtains the conversion configuration file submitted by the user from the code repository 140 in response to the model conversion notification, and obtains the original model file and the conversion configuration parameters according to the storage location indicated by the conversion configuration file.
Of course, the object of version management by code repository 140 is not limited to code, but may be other types of files, such as the conversion configuration file herein. The conversion profile has recorded therein the storage locations (e.g., may be stored in a database) of the original model file and conversion profile parameters. The user may fill in the contents of the conversion profile for a particular model conversion task and submit the file from terminal device 110 to code repository 140.
Some implementations of code repository 140 allow a user to attach description information to the data when submitting the data, so that the user may add certain keywords to the description information, and after receiving the conversion profile submitted by the user and parsing the keywords from the description information, code repository 140 may confirm that the user wants to initiate a new model conversion task.
The code repository 140 may then send a model conversion notification to the model conversion system, the notification message serving to inform the model conversion system that a new model conversion task needs to be performed, and the model conversion system, upon receiving the notification, may pull the conversion configuration file in the code repository 140 to the conversion backend server 130, then obtain the storage location of the original model file and the conversion configuration parameters by parsing the content of the conversion configuration file, and further obtain the original model file and the conversion configuration parameters from the location.
Compared with the mode 1 and the mode 2, the mode 1 is friendly to the user because of the visual interface for the user to operate, and has lower technical level requirements for the user (the user can perform interface operation), but has poorer flexibility (for example, only one of several parameters provided on the interface can be selected), and is inconvenient for batch conversion (for example, the batch uploading of the original model file on the visual interface is difficult); the original model files and the conversion configuration parameters in the mode 2 are not submitted from the visual interface, but are stored in a position indicated by the conversion configuration files in advance, so that flexibility is better (for example, a user can freely specify the values of the conversion configuration parameters in one file), batch conversion is also convenient to support (for example, a batch of original model files are easy to store in advance), but the mode may be more complicated to develop (for example, the code repository 140 needs to be built or configured in advance), and the technical level requirements on the user are relatively high (for example, the code repository 140 needs to be used by the user).
Obviously, there are other ways to obtain the original model file and the conversion configuration parameters besides the ways 1 and 2, for example, the program may also be directly run on the terminal device 110, the original model file and the conversion configuration parameters specified in the program may be submitted to the model conversion system, and the like, which are not illustrated one by one.
Optionally, after the conversion configuration parameters are obtained, whether the conversion configuration parameters meet a preset constraint relation or not may be checked, if yes, the subsequent steps may be continuously performed, otherwise, the model conversion process may be ended. For example, the original model file is a quantized model file, but the conversion configuration parameters do not include any quantization parameters (for example, quantization parameters are not uploaded), which is the case when the constraint relationship is not satisfied.
Step S220: and constructing a conversion environment according to the conversion configuration parameters.
Step S230: and converting the original model file by using a conversion tool under a conversion environment to obtain a converted target model file.
Steps S220 and S230 are illustrated together. Model transformations require corresponding transformation tools to be completed, which may include either first party tools (developed by developers of model transformation systems) or third party tools (developed by other vendors). The conversion tools of the third party can be integrated in software development kits (Software Development Kit, abbreviated as SDKs) of corresponding deep learning frameworks (e.g., caffe, TFLite frameworks), and these SDKs often include not only conversion tools but also simulation and reasoning related kits.
The conversion environment may be understood as an environment in which the conversion tool is operable because the conversion environment is prepared in step S220, and the model conversion can be normally performed in step S230. For example, the conversion environment may include an operating system suitable for the conversion tool to run, library files, databases, resource files, etc., upon which the conversion tool is dependent to run. The required conversion tools and conversion environments are different according to the conversion sources and/or the conversion targets, so that the conversion environments need to be constructed according to conversion configuration parameters including parameters (1) (2) related to the conversion sources and/or the conversion targets. For example, if the target platform is designated as the MTK platform in the conversion configuration parameters, the target device is 6983 device, and the quantization parameter file is uploaded, the model conversion system will construct a "MTK-6983-quantized" conversion environment from the conversion configuration parameters.
For the resources on which the conversion tool depends, a special server may be set to perform unified management, for example, the resource management server 120 in fig. 1, and when the conversion environment needs to be built by the model conversion system, the corresponding resources are pulled from the resource management server 120 to the conversion background server 130 according to the conversion configuration parameters. The resource management server 120 may be an FTP server, and the resources may be organized in the form of a wheell package, a Zip package, or the like. Of course, the case where the resource management server 120 and the conversion backend server 130 are the same server is not excluded.
It should be mentioned that the conversion tools of the third party may be included in SDKs of the third party, and these SDKs may be regarded as a resource, and are uniformly managed by the resource management server 120, and are pulled to the conversion back-end server 130 for use when model conversion is required, and for those conversion tools of the first party, they may be stored in the conversion back-end server 130, and are directly used when model conversion is required.
The unified management of the resources can ensure that all users use the same resource set when performing model conversion, and the problems of scattered resource distribution and disordered resource version management caused by the maintenance of the resources by each user are avoided. Of course, unified management of resources is not mandatory, and for example, the location of the resources may be specified by the user in the conversion configuration parameters, or the resources may be searched by the model conversion system itself, or the like.
In some implementations of step S220, the operating system in which the model conversion system is located may be built as one conversion environment, but such implementations are difficult to support simultaneous execution of multiple conversion tasks (because the conversion environments required for different conversion tasks are likely to be different), and also have poor security.
In other implementations of step S220, a container (e.g., a dock container) may be built for each conversion task, with the container being the conversion environment in which the conversion tool runs. This allows each conversion task to run independently within different containers, thereby enabling multiple conversion tasks to be performed simultaneously. In addition, as the containers are not interfered with each other, even if a certain conversion task is in error during operation, the internal environment of the container where the conversion task is positioned is only destroyed at most, and the operation of other conversion tasks is not influenced, so that the stability of the model conversion process is effectively ensured, and powerful support is provided for batch conversion of the models. In addition, these implementations are safer because the container exterior is not easily accessible to the container interior. Finally, the life cycle of the container is limited, and the container is destroyed after the conversion task is executed, so that unresolved errors are not left.
How the container is constructed is described in more detail below. It should be noted that in the alternative, the container may be replaced by a virtual machine, but the container is a more lightweight way of virtualizing, with far less resources being consumed than the virtual machine.
The model conversion in step S230 may be completed at one time, for example, the original model file under the frame a may be directly converted into the target model file under the frame B using the conversion tool X. However, in some cases, the model conversion cannot be completed at one time, for example, the target model file is under the frame C, but the conversion tool X does not support the conversion from the frame a to the frame C, and at this time, it may be necessary to perform multi-stage conversion on the original model file to obtain the target model file. Wherein each stage of conversion outputs an intermediate model file which is to be used as an input of the next stage of conversion, and the intermediate model file output by the last stage of conversion is the target model file. For example, the original model file under the frame a can be converted into the intermediate model file under the frame B by using the conversion tool X, and then converted into the target model file under the frame C by using the conversion tool Y.
On one hand, the multistage conversion can reduce the conversion difficulty, and in addition, the conversion flow can be split into a plurality of conversion modules, so that the modular management of the model conversion is facilitated. The concept of the conversion module is described immediately below.
The transformation tool is sporadic and needs to be organized in order to accomplish the model transformation task. In some implementations, step S230 may be implemented as follows:
step a1: and determining a conversion flow according to the conversion configuration parameters. The conversion flow is formed by connecting at least one conversion module related to the model conversion in series (only one conversion module is not required to be connected in series), each conversion module can correspond to one link in the model conversion, for example, each stage of intermediate conversion mentioned above can be implemented as a single conversion module, but the conversion modules and the intermediate conversion are not necessarily in one-to-one correspondence. Each conversion module may contain one or more executable scripts (e.g., python scripts, shell scripts) within it that, when executed, may call the conversion tool, although it is possible that the conversion module may have some conversion tools within it.
The conversion module may be developed by a developer of the model conversion system and stored in advance on the conversion backend server 130. As for the conversion process, it may be set up before step S230 is performed, or may be set up temporarily when step S230 is performed:
for example, there are 10 conversion modules, each named module 1-module 10. Several available conversion processes may be preset inside the model conversion system and the conversion target corresponding to each conversion process may be indicated, for example:
Frame a→frame B: module 1-Module 3-Module 5-Module 7"
Frame a→frame C: module 2-Module 4-Module 6-Module 8"
Frame a→frame D: "Module 9-Module 10"
And then selecting the conversion flow to be secondarily used from the preset conversion flows to execute according to the parameters related to the conversion source and/or the conversion target in the conversion configuration parameters.
For another example, it is also possible that no conversion process is preset, but the user is required to specify which conversion modules are to be used to form the conversion process (i.e., the parameter of item (5) described above) in the conversion configuration parameter, for example, if the conversion configuration parameter includes the parameter "1-3-5-7", this indicates that the conversion process is to be formed by the series of modules 1, 3, 5, and 7.
However, since the user can arbitrarily designate the conversion modules and the serial connection sequence thereof, a conversion flow which is not supported by the actual system may be established, so that the model conversion system can add a verification step in this way to verify whether the conversion flow determined according to the conversion configuration parameters is legal. For example, a table may be preset in the model conversion system, in which description parameters of legal conversion flows are recorded, and the validity determination may be performed by comparing the conversion flows specified in the conversion configuration parameters with the conversion flows recorded in the table. The description parameters of the conversion process may include which conversion modules the conversion process is composed of and the serial connection sequence between the conversion modules, for example, the description parameters may also take the form of "1-3-5-7".
Hereinafter, description will be mainly given taking a case where a user designates a conversion flow in conversion configuration parameters as an example.
Step b1: and converting the original model file by executing a conversion flow under a conversion environment to obtain a corresponding target model file.
After the conversion flow is determined in the step a1, the model conversion task can be completed only by executing each conversion module in the conversion flow in turn.
Steps a1 and b1 enable the modularity of the conversion process. Each conversion module can be flexibly combined and reused in different conversion flows, and each conversion module can be independently maintained, so that development difficulty and maintenance difficulty are reduced. It should be appreciated that modularity is not the only way to implement model conversion, for example, the entire conversion flow may be contained in one script without distinguishing between the conversion modules.
In addition, in step S230, conversion configuration parameters, such as the parameters of item (3) (4) described above, may be used unless the conversion tool uses default parameters for conversion.
Optionally, after the execution of step S230, a preliminary test may be performed on the obtained target model to verify the usability thereof. For example, the same data may be input to the original model and the target model respectively, and it is determined whether the outputs of the two are the same, if so, it indicates that the model conversion is correct, otherwise, it indicates that the model conversion has an error.
In summary, the model conversion method provided by the embodiment of the application automatically constructs the conversion environment and automatically completes the model conversion according to the conversion configuration parameters, thereby remarkably improving the model conversion efficiency, avoiding the artificial risk and reducing the labor cost.
In addition, as described above, the method can be integrated in an online model conversion system, and a user performs model conversion through the system uniformly, that is, a set of standardized model conversion steps is formed, and the configuration and resource management of the conversion environment are not performed by means of personal experience of the user, so that the problems that error reproduction in the conversion process is difficult, the conversion process is difficult to scale, and the system is difficult to continue iteration when human resources are changed (for example, experienced developers leave) are solved. And when a new device is introduced, all users can benefit by only carrying out corresponding update on the system (supporting a new conversion flow), thereby being beneficial to the rapid introduction and verification of the new device.
The implementation of steps S220 and S230 when a container is used as the conversion environment will be described further on the basis of the above embodiments:
at this time, step S220 may further include:
Step a2: and obtaining a general basic mirror image for model conversion, and constructing a container according to the basic mirror image.
The different model conversion tasks of the conversion source and/or conversion target may depend on different conversion environments, and the base image may be considered as a common part of these conversion environments, for example, an operating system, a Python development kit, a c++ development kit, a database, a conversion tool kit (a conversion tool that is partially universal), etc., and this part of the content may not be easily modified, and its correctness may be guaranteed by sufficient verification, so that a "clean" container may be created based on the base image.
The base image may be built and stored on the translation background server 130 before step a2 is performed, and the use of the image is directly read when step a2 is performed; alternatively, the base image may be temporarily constructed when step a2 is performed. For example, if a docker container is adopted, the base image may be built on the conversion background server 130 according to a build script (dockerfile), and the dockerfile may be uploaded to the conversion background server 130 in advance by a developer.
In particular, since the base image has versatility, it is likely that for some model conversion tasks that share one base image, the base image needs to be built only once (unless the base image needs to be updated), and then the container building is performed based on the base image each time. Of course, it is not necessary to say that all the model conversion tasks share the same base image, and it is also possible that one base image is only applicable to part of the model conversion tasks, and at this time, an appropriate base image needs to be determined according to the conversion configuration parameters.
After the base image is obtained, the base image can be used for constructing the container, for example, a docker container can be constructed according to the docker image. Unlike the base image, a dedicated container can be built for each model conversion task. The containers can be considered as a relatively independent workspace, which allows the individual model transformation tasks to run relatively independently in different containers, without interfering with each other.
Step b2: and acquiring resources on which the conversion tool runs according to the conversion configuration parameters.
The container constructed in step b1 is an "empty" container, and not a complete conversion environment, but a complete environment supporting the operation of the conversion tool is formed by adding resources on which the conversion tool depends, and in addition, the conversion tool naturally needs to be acquired since model conversion is performed.
As described above, if the resources are uniformly managed on the resource management server 120, the conversion tool and the resources may be acquired from the resource management server 120 according to the conversion configuration parameters. For example, if the target frame of the conversion is a high-pass SNPE frame, the SDK of the SNPE (including the conversion tool) may be pulled to Caffe frame's resources. It should be appreciated that if portions of the conversion tool and resources are stored on the conversion backend server 130, they may also be obtained directly from the local site.
The acquired conversion tools and resources may be added to the container or stored in a location accessible in the container, such as to a storage space shared by the operating system of the conversion backend server 130 and the container, so that the conversion tools and resources are used in the container when performing model conversion.
The relationship of step b1 and step b2 can be understood as follows: since the mirror image in the step b1 has a certain commonality, the container constructed in the step b1 also has a certain commonality, or only a common part in the conversion environment is obtained, and the step b2 is equivalent to adding personalized parts for different model conversion tasks into the container.
After step b2 is performed, the original model file may be converted in the container by using a conversion tool, resulting in a converted target model file (step S230). The specific way to perform model conversion is as described above, for example, the conversion flow may be constructed and executed, and the description will not be repeated. As for the conversion modules required for constructing the conversion flow, they can be added to the container after the container is constructed, or stored in a place accessible in the container. The operations of step b2, constructing the conversion flow, and executing the conversion flow may be integrated in an upper layer script (higher layer than the script in the conversion module) that is executed when the upper layer script is run in the container.
Based on the above embodiments, how to debug the model conversion system is described further below:
the model conversion system may have errors in the process of performing model conversion, so that conversion failure is caused, for example, the error may be caused by input data (for example, original model file error, conversion configuration parameter error) or code logic of the model conversion system itself, so that in order to ensure continuous and stable operation of the system, a corresponding debugging process needs to be set.
Further, because the online system is a closed environment (i.e., a user can input data into the system and obtain output, but is hard to learn the logic implemented therein), it is generally difficult to give detailed reasons for error in conversion due to performance requirements, and thus the online system cannot be directly debugged, and the online (e.g., on the development device 150) model conversion process needs to be repeated, so that the developer can locate the problem.
One possibility is that the developer manually builds up the conversion environment (such as manually downloading resources, building images and containers) on line, re-executes the model conversion process of an error once, and searches the error cause in cooperation with the code and log information (log) of the conversion system. However, this method is not only time-consuming and laborious, but also it is difficult to ensure that the off-line conversion process is a complete reproduction of the on-line conversion process, and the result is likely to be that the reproduction is not problematic or that the reproduction is inconsistent with the on-line problem.
Thus, as an alternative, while the conversion environment is built and the model is converted on-line, the executed conversion operation instructions can be recorded to form a record file (for example, an executable script), and the operation instructions in the record file are directly executed when the on-line debugging is performed, so that the conversion environment can be built automatically, the conversion process can be quickly and accurately reproduced, and the conversion process can be directly executed to the error position (because the operation instructions can be recorded at least to the error position). Therefore, the method not only simplifies the steps of self-building the conversion environment and the like, but also can quickly execute the steps to the position where the problem occurs in the conversion process, and is convenient for developers to quickly position error reasons according to codes, log information and the like.
Based on the above embodiments, how to update the model conversion system is described further:
the model conversion system may be updated when a problem is found by the debug and the problem has been solved or the system requires a functional upgrade.
In order to fully present the update procedure, the following preconditions may be considered: the resource management server 120 performs unified management on the resources, converts the environment into a container, and the specific mode of model conversion is as follows: firstly, a conversion flow is formed by connecting conversion modules in series according to conversion configuration parameters, and then the conversion flow is executed to complete model conversion. At this time, the update of the model conversion system includes at least one of the following aspects (the update is required to be updated, and not necessarily all of the three aspects are updated at the same time):
(1) Updating of base images
After a developer tests a new base image on-line (e.g., the development device 150), a build script (e.g., dockerfile) corresponding to the new base image may be uploaded to the on-line (e.g., the conversion background server 130), where the new base image may be built based on the build script, without excluding the solution that the developer directly uploads the new base image to the on-line.
As for the original base image in the online environment, the original base image may be replaced by a new base image, or may be kept as required.
(2) Updating of resources
After the developer tests a new resource package on line, the new resource package is uploaded to the resource management server, and the new resource package may be a replacement (such as a new version) of the original resource package or may be a brand new resource.
(3) Updating of conversion flow
Since the conversion flow is constituted by conversion modules, (3) there are various possibilities, which are described by the following examples:
for example, there are 10 conversion modules, each named module 1-module 10. "Module 1-Module 3-Module 5-Module 7" is an existing conversion flow in the model conversion system, abbreviated as flow p, and the description parameters of flow p are stored on the line for use in verifying the validity of the flow (see description above). There are three possible ways of updating the flow p:
First, the module 1 is changed, and the essential content of the flow p is changed (the operation performed by the module 1 is changed), but the description parameters of the flow p are not changed (or the series order of the modules 1, 3, 5 and 7 is not changed), so that only the module 1 is required to be updated on line, and the description parameters of the flow p are not required to be updated.
Second, the conversion modules included in the flow p are not changed, but the serial sequence among the modules is changed, for example, the conversion modules are changed into 'modules 1-5-3-7', and the conversion modules do not need to be updated at this time, and only the description parameters of the flow p need to be updated.
Thirdly, assume that a new module 11 is developed, and that module 11 and modules 1, 4, 6 constitute a new conversion flow q: "Module 1-Module 4-Module 6-Module 11", at this time, the module 11 is updated on-line, and the description parameters of the new flow q are added on-line.
Note that the above omits the case of adding a new conversion module, but the conversion module is not included in any conversion flow, because adding an isolated conversion module is not significant.
Summarizing the above example, one can get a concrete meaning of the conversion flow update: after the developer tests the new conversion module on line, the new conversion module is uploaded to the line, and/or after the developer tests the new conversion flow on line, the description parameters of the new conversion flow are uploaded to the line.
The basic idea of the update of the above three aspects is to test the update content on-line and then to synchronize the update content on-line, and for specific tests to be performed, reference is made to the examples hereinafter, which are not described herein.
If in a certain update, all the above three updates exist, the updates may be performed in the order of (1) - (2) - (3), because the later-ordered update content may have a dependency on the earlier-ordered update content.
The updating process updates the model conversion system, so that the updating content is uniform for all system users, all users can benefit, and the problem of management confusion caused by independently updating and maintaining the method by each user is avoided.
Next, a possible conversion process, a debugging process and an updating process of the model conversion system are described by fig. 3, 4 and 5, respectively, on the basis of the above embodiments.
Referring to fig. 3, after the conversion task is triggered, the original model file and the conversion configuration parameters are first acquired, and at least two possible acquisition manners are described above.
Then, the base image is obtained, and a container is constructed according to the base image, and the subsequent steps can be performed in the container, which is not particularly pointed out.
Then, the conversion configuration parameters are analyzed and preprocessed: for example, in the manner of submitting input data by using a data warehouse, the conversion configuration file acquired from the data warehouse needs to be parsed to obtain the original model file and the storage position of the conversion configuration parameter, and the task can be performed after the container is built; as another example, it may be desirable to convert certain conversion configuration parameters into a form suitable for use by a conversion tool, and so forth. If the parameter analysis and the preprocessing are successful, the subsequent steps are continuously executed, otherwise, the conversion task is ended.
And then, checking whether the conversion configuration parameters meet a preset constraint relation, if so, continuing to execute the subsequent steps, otherwise, ending the conversion task. For example, the original model file is a quantized model file, but the conversion configuration parameters do not include any quantization parameters, which is the case when the constraint relationship is not satisfied.
The conversion target is then determined from the conversion configuration parameters (here, for simplicity of illustration, the conversion source is considered fixed, i.e., no configuration of parameters related to the conversion source is required), and if the target is present, the subsequent steps are performed, otherwise the conversion task is ended. For example, there is no chip with model number 6984 under the MTK platform, but the target platform is designated as the MTK platform, and the target device is 6984 devices, which belongs to the situation that the conversion target does not exist. Legal conversion targets may be preset in the conversion system to support this determination.
Then, a conversion flow is determined according to the conversion configuration parameters. For example, the conversion modules 1, 3, 5, 7 are connected in series according to the parameter "1-3-5-7" into a conversion flow.
And then, checking the conversion flow determined in the last step according to the preset description parameters of the legal conversion flow, if the conversion flow is legal, executing the conversion flow, otherwise, ending the conversion task. For example, for a particular conversion target, the conversion flow "Module 1-Module 3-Module 5-Module 7" is legal, but the conversion flow "Module 1-Module 3-Module 5-Module 8" is illegal.
To execute the conversion process, a working space is first prepared, where the working space refers to an independent space where the current conversion task runs, and may be considered as a previously constructed container, or may perform necessary preparation work on the basis of the container.
The converted input data is then copied into the workspace. For example, the original model file, the transformation configuration parameters may be added to the container, or a location in the container that may be accessed. Note that the conversion configuration parameter herein may refer to a configuration after parsing and preprocessing, which is not explicitly shown in the figure.
Then, necessary conversion tools and the resources on which the necessary conversion tools depend are acquired according to the conversion targets. For example, the conversion environment can be obtained from a resource management server, and the conversion environment is constructed after the step is completed.
Then, a process of generating a model, that is, a process of executing a conversion flow in a conversion environment and generating a target model file based on the original model file, is entered. The process uses previously obtained conversion tools and possibly conversion configuration parameters, not explicitly shown in the figure. As previously described, the model conversion may not be one-step in place, and may require multiple levels of conversion, three levels of conversion being experienced during the model generation process shown in fig. 3: original model file- & gt first intermediate model file- & gt second intermediate model file- & gt target model file, and performing next-stage conversion when each-stage conversion is successful, otherwise ending the conversion task.
Finally, if the target model file is obtained through conversion, the target model file can be packaged, and the subsequent use is convenient.
It should be appreciated that the conversion process of fig. 3 is but one specific implementation of the conversion method described above, but is not the only implementation.
Referring to fig. 4, when the online model conversion system performs model conversion (not only the model generation in fig. 3, but also the steps of container construction, resource downloading and the like can be included), the executed conversion operation instruction is recorded to form a record file (such as an executable script), and when the conversion fails, a developer acquires the record file from the online, executes the operation instruction in the file online, and automatically reproduces the model conversion process. Note that when model conversion is performed again, a debug switch in the code may be turned on to output log information for assisting debugging.
For each functional unit (which may refer to a conversion module or a primary conversion process) in the model conversion process, whether the module operates successfully, whether the output result is correct, and whether the input model is correct are tested, and unit testing tools can be used for testing.
If the problem of the model is found after the test, the model file submitted by the user is incorrect, the problem can be fed back to the developer for further analysis, and if the problem is found to be running or output after the test, the problem that the conversion logic of the model conversion system may have errors (problem of codes) is described, and the problem can be further positioned by combining the codes and log information through the developer.
If the positioning problem is successful, the developer can make code modifications and make system updates. If the problem positioning fails, the operation instruction in the record file can be referred by the developer to manually reproduce the model conversion process once, and the developer can combine the code and the log information to perform the problem positioning. If the positioning problem is successful, the code can be modified and the system can be updated, and if the positioning problem is failed, the manual reproduction of the model conversion process can be repeated until the problem is found.
It should be appreciated that since manual reproduction is time consuming and labor intensive and the positioning problem is also relatively slow, it is only a back-up means in fig. 4, which prioritizes after automatic reproduction, but manual reproduction can fine-check each step in the model conversion process, so that it is possible to solve more problems.
In performing manual replication, since each functional unit in the model conversion process is replicated one by one, it may be necessary to manually prepare an intermediate model file as an input, as shown in fig. 4.
It should be appreciated that the debugging process in fig. 4 is only one, but not the only, way to debug the model conversion system described above.
Referring to fig. 5, it is first determined whether the base image needs to be updated (which may be determined by a developer, and the later determination is similar), where, taking a dock image as an example, if the base image does not need to be updated, whether the resource package needs to be updated is continuously determined, and if the resource package needs to be updated, the dock image is updated according to the content included in the dock image. The dock image at least includes an operating system (e.g., ubuntu system in the figure), and may further include general components such as an environment management tool (e.g., conda, node), a development related library (e.g., python, c++ library), and a database.
After updating the dockerfile, a new docker mirror image can be constructed by utilizing the new docker mirror image, then a docker container is further constructed based on the new docker mirror image, if the mirror image construction fails, the container construction fails or the container cannot normally run, whether the constitution of the docker mirror image is correct or not can be checked again, and if the version detection is successful, the version detection is executed.
Version detection detects whether the constructed dock image or a component thereof is a desired version. For example, a mirror image of ubuntu18.04 is required, but in practice a mirror image of ubuntu16.04 is used, in which case the version detection results in failure. If the version detection is successful, the dockerfile is uploaded to the line (all steps of updating the basic image before can be executed on line), and if the detection fails, the constitution of the docker image can be rechecked.
In addition, if the version detection is successful, which also indicates that the phase of the base image update is over, a resource update, here exemplified by a resource in the form of an update Wheel package, may be performed. If the conversion flow is not required to be updated, the conversion flow is continuously updated, and if the conversion flow is required to be updated, the Wheel package is tested, which can comprise testing the functions and versions of the Wheel package itself and testing the functions of conversion modules depending on the Wheel package, and the testing can be performed by using a unit testing tool.
If the above tests are successful, the content of the Wheel package is packaged, and the Wheel package is uploaded to an FTP server (resource management server) (each step of updating the resource can be performed on line), and if one or more of the tests fail, the content adjustment and the test of the resource package can be performed again until the test is successful.
The completion of the packaging also indicates that the resource update phase is complete and the conversion flow update phase can be entered. Firstly, whether the conversion module needs to be updated is judged, if the conversion module does not need to be updated, whether the script needs to be updated is judged, if the conversion module needs to be updated, the function of the conversion module is redesigned, and the design may depend on the update content of the base image and the update content of the resource (shown by a dotted line in the figure).
After the conversion module is designed, whether the script needs to be updated is further judged, and Shell and Python scripts are taken as examples, wherein the scripts can refer to the scripts forming the conversion module, the function of the conversion module is updated, but each script contained in the conversion module does not need to be updated, and in addition, even if the function of the conversion module is kept unchanged, some scripts may need to be updated.
If the script needs to be updated, the Shell and Python scripts are redesigned and the function test is performed, so that the functions of the script itself, the functions of the whole conversion module and even the functions of the conversion flow using the conversion module can be tested. If the script does not need to be updated, judging whether the conversion target is newly added. If the functional test of the script is successful, the method also judges whether a conversion target is newly added.
In fig. 3, there is a step of determining a conversion target, which determines whether the conversion target exists, so that legal conversion targets may be preset in the conversion system, and a new conversion target may be added (or new device verification may be performed) through an update process. If the conversion target needs to be newly added, the conversion target needs to be designed, whether the description parameters of the conversion flow need to be updated or not is judged (the new conversion target often corresponds to the new conversion flow), and if the conversion target does not need to be newly added, whether the description parameters of the conversion flow need to be updated is directly judged.
If the description parameters of the conversion process need to be updated, the conversion process is designed and tested, and after the test is successful, the data needing to be updated (including the description parameters of the new conversion process and other data needing to be updated, such as a new conversion target and a new conversion module) are uploaded onto the line (all the steps related to the update of the conversion process can be executed on line), and if the description parameters of the conversion process do not need to be updated, but other data related to the conversion process need to be updated (such as a new conversion target and a new conversion module), the data also need to be uploaded onto the line.
It should be appreciated that the update process in fig. 5 is but one way of updating the model conversion system described above, but is not the only way.
Fig. 6 shows a system framework of a model conversion system provided in an embodiment of the present application. It should be appreciated that the framework in FIG. 6 is not limited to the model conversion system itself, but may include content related to the model conversion system.
At the bottom of fig. 6 are two different versions of the base image, based on which different workspaces may be created on the conversion backend server 130 for different conversion tasks, each workspace containing a separate directory, conversion tool, script, and conversion tool dependent resources. The resources can be uniformly managed on an FTP server, and resources belonging to different platforms and frames (for example, SNPE frames under a high-pass platform) can be separately managed on the FTP server.
Optionally, the FTP server also supports resource verification, which may be implemented by calculating a corresponding MD5 value (or other types of hash values) for each resource packet, and before the background server 130 downloads the resource, requesting the FTP server to compare the MD5 value of the resource packet to be downloaded with the MD5 value of the existing resource packet, if the MD5 value of the resource packet is the same, not downloading, if the MD5 value of the resource packet is not the same, it indicates that the resource packet on the FTP server is updated, and may be downloaded.
The debugging space in the upper right corner may be located on the development device 150 for use in reproducing the conversion process during debugging, in a similar sense as the workspace. However, if the developer is debugging on the translation background server 130 (but not directly debugging the on-line system), the translation tools and resources can be multiplexed into the same model translation task's workspace without data replication, and the debug space can be considered to contain only grey workspace directories and scripts.
Fig. 7 shows a possible structure of a model conversion device 300 provided in an embodiment of the present application. Referring to fig. 7, the model conversion apparatus 300 includes:
a conversion data obtaining unit 310, configured to obtain an original model file to be converted and conversion configuration parameters;
a conversion environment construction unit 320, configured to construct a conversion environment according to the conversion configuration parameter, where the conversion environment is an environment in which a conversion tool can operate;
the model conversion unit 330 is configured to convert the original model file by using the conversion tool in the conversion environment, so as to obtain a converted target model file.
In one implementation of the model conversion apparatus 300, the conversion environment construction unit 320 constructs a conversion environment according to the conversion configuration parameters, including: obtaining a basic mirror image universal for model conversion, and constructing a container according to the basic mirror image; acquiring the conversion tool and resources on which the conversion tool runs according to the conversion configuration parameters; the model conversion unit 330 converts the original model file by using the conversion tool under the conversion environment to obtain a converted target model file, which includes: and converting the original model file in the container by using the conversion tool to obtain the target model file.
In one implementation of the model conversion apparatus 300, the conversion environment includes resources on which the conversion tool runs, and the conversion environment construction unit 320 constructs a conversion environment according to the conversion configuration parameters, including: and acquiring the resources from a resource management server according to the conversion configuration parameters.
In one implementation of the model conversion apparatus 300, the converting unit 330 converts the original model file by using the conversion tool in the conversion environment to obtain the target model file, including: determining a conversion flow according to the conversion configuration parameters, wherein the conversion flow comprises at least one conversion module involved in model conversion; executing the conversion flow under the conversion environment to convert the original model file to obtain the target model file; the conversion module in the conversion flow calls the conversion tool when executing.
In one implementation of the model conversion apparatus 300, the conversion environment construction unit 320 acquires the resource from the resource management server, and the model conversion unit 330 converts the original model file by performing a conversion procedure in the container, the conversion procedure including at least one conversion module involved in model conversion; the model transformation is performed in an in-line model transformation system, the updating of which comprises at least one of the following three: updating the base image: after a new basic image is tested on line, uploading a construction script corresponding to the new basic image to the line, and constructing the new basic image on the line based on the construction script; updating the resource: after testing a new resource package on line, uploading the new resource package to a resource management server; updating the conversion flow: and uploading the new conversion module to the line after testing the new conversion module on the line, and/or uploading the description parameters of the new conversion flow to the line after testing the new conversion flow on the line.
In one implementation of the model conversion apparatus 300, the model conversion unit 330 converts the original model file with the conversion tool under the conversion environment to obtain a converted target model file, including: performing multistage conversion on the original model file by using the conversion tool in the conversion environment to obtain the target model file; and each stage of conversion is used for obtaining an intermediate model file, and the intermediate model file obtained by the last stage of conversion is the target model file.
In one implementation of the model conversion apparatus 300, the conversion data obtaining unit 310 obtains an original model file to be converted and conversion configuration parameters, including: acquiring the original model file submitted by a user from a visual interface of terminal equipment and the conversion configuration parameters; or receiving a model conversion notice sent by a code warehouse, responding to the model conversion notice, acquiring a conversion configuration file submitted by a user from the code warehouse, and acquiring the original model file and the conversion configuration parameters according to a storage position indicated by the conversion configuration file.
In one implementation of the model transformation device 300, the model transformation is performed in an in-line model transformation system, the method further comprising: storing and constructing the conversion environment and operating instructions executed when converting the original model file to form a record file; the record file is used for reproducing the model conversion process on line and debugging the model conversion system.
The model conversion device 300 according to the embodiment of the present application has been described in the foregoing method embodiment, and for brevity, reference may be made to the corresponding contents of the method embodiment where the device embodiment is not mentioned.
Fig. 8 shows one possible structure of an electronic device 400 provided in an embodiment of the present application. Referring to fig. 8, an electronic device 400 includes: processor 410, memory 420, and communication interface 430, which are interconnected and communicate with each other by a communication bus 440 and/or other forms of connection mechanisms (not shown).
Wherein the processor 410 includes one or more (only one shown), which may be an integrated circuit chip, having signal processing capabilities. The processor 410 may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a micro control unit (Micro Controller Unit, MCU), a network processor (Network Processor, NP), or other conventional processor; but may also be a special purpose processor including a graphics processor (Graphics Processing Unit, GPU), a Neural network processor (Neural-network Processing Unit, NPU for short), a digital signal processor (Digital Signal Processor, DSP for short), an application specific integrated circuit (Application Specific Integrated Circuits, ASIC for short), a field programmable gate array (Field Programmable Gate Array, FPGA for short) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. Also, when the processor 410 is plural, some of them may be general-purpose processors, and another may be special-purpose processors.
The Memory 420 includes one or more (Only one shown in the drawings), which may be, but is not limited to, a random access Memory (Random Access Memory, RAM), a Read Only Memory (ROM), a programmable Read Only Memory (Programmable Read-Only Memory, PROM), an erasable programmable Read Only Memory (Erasable Programmable Read-Only Memory, EPROM), an electrically erasable programmable Read Only Memory (Electric Erasable Programmable Read-Only Memory, EEPROM), and the like.
The processor 410, as well as other possible components, may access, read, and/or write data from, the memory 420. In particular, one or more computer program instructions may be stored in memory 420 that may be read and executed by processor 410 to implement the model conversion methods provided by embodiments of the present application.
It is to be understood that the configuration shown in fig. 8 is merely illustrative, and that electronic device 400 may also include more or fewer components than those shown in fig. 8, or have a different configuration than that shown in fig. 8. The components shown in fig. 8 may be implemented in hardware, software, or a combination thereof. The electronic device 400 may be a physical device such as a server, a PC, a notebook, a tablet, a cell phone, a robot, etc., or may be a virtual device such as a virtual machine, a container, etc. The electronic device 400 is not limited to a single device, and may be a combination of a plurality of devices or a cluster of a large number of devices. For example, the translation background server 130 of fig. 1 may employ the structure of the electronic device 400.
The embodiment of the application also provides a computer readable storage medium, and the computer readable storage medium stores computer program instructions which, when read and executed by a processor, perform the model conversion method provided by the embodiment of the application. For example, a computer-readable storage medium may be implemented as memory 420 in electronic device 400 in FIG. 8.
The present embodiments also provide a computer program product comprising computer program instructions which, when read and executed by a processor, perform the model conversion method provided by the embodiments of the present application.
The foregoing is merely exemplary embodiments of the present application and is not intended to limit the scope of the present application, and various modifications and variations may be suggested to one skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.
Claims (11)
1. A model conversion method, comprising:
acquiring an original model file to be converted and converting configuration parameters;
constructing a conversion environment according to the conversion configuration parameters, wherein the conversion environment is an environment for a conversion tool to operate;
And converting the original model file by using the conversion tool under the conversion environment to obtain a converted target model file.
2. The model transformation method according to claim 1, wherein said constructing a transformation environment according to said transformation configuration parameters comprises:
obtaining a basic mirror image universal for model conversion, and constructing a container according to the basic mirror image;
acquiring the conversion tool and resources on which the conversion tool runs according to the conversion configuration parameters;
the converting the original model file by using the converting tool in the converting environment to obtain a converted target model file, including:
and converting the original model file in the container by using the conversion tool to obtain the target model file.
3. The model transformation method according to claim 1 or 2, wherein the transformation environment comprises resources on which the transformation tool is running, and the building a transformation environment according to the transformation configuration parameters comprises:
and acquiring the resources from a resource management server according to the conversion configuration parameters.
4. A model conversion method according to any one of claims 1-3, wherein said converting the original model file with the conversion tool in the conversion environment to obtain the target model file includes:
Determining a conversion flow according to the conversion configuration parameters, wherein the conversion flow comprises at least one conversion module involved in model conversion;
executing the conversion flow under the conversion environment to convert the original model file to obtain the target model file; the conversion module in the conversion flow calls the conversion tool when executing.
5. The model conversion method according to claim 2, wherein the resource is obtained from a resource management server, the original model file is converted by executing a conversion flow in the container, the conversion flow including at least one conversion module involved in model conversion;
the model transformation is performed in an in-line model transformation system, the updating of which comprises at least one of the following three:
updating the base image: after a new basic image is tested on line, uploading a construction script corresponding to the new basic image to the line, and constructing the new basic image on the line based on the construction script;
updating the resource: after testing a new resource package on line, uploading the new resource package to the resource management server;
Updating the conversion flow: and uploading the new conversion module to the line after testing the new conversion module on the line, and/or uploading the description parameters of the new conversion flow to the line after testing the new conversion flow on the line.
6. The model conversion method according to any one of claims 1 to 5, wherein converting the original model file with the conversion tool in the conversion environment to obtain a converted target model file includes:
performing multistage conversion on the original model file by using the conversion tool in the conversion environment to obtain the target model file; and each stage of conversion is used for obtaining an intermediate model file, and the intermediate model file obtained by the last stage of conversion is the target model file.
7. The method for model transformation according to any one of claims 1 to 6, wherein the obtaining the original model file to be transformed and the transformation configuration parameters includes:
acquiring the original model file submitted by a user from a visual interface of terminal equipment and the conversion configuration parameters; or,
and receiving a model conversion notice sent by a code warehouse, responding to the model conversion notice, acquiring a conversion configuration file submitted by a user from the code warehouse, and acquiring the original model file and the conversion configuration parameters according to a storage position indicated by the conversion configuration file.
8. The model conversion method according to any one of claims 1 to 7, characterized in that the model conversion is performed in a model conversion system on a line, the method further comprising:
storing and constructing the conversion environment and operating instructions executed when converting the original model file to form a record file; the record file is used for reproducing the model conversion process on line and debugging the model conversion system.
9. A computer program product comprising computer program instructions which, when read and executed by a processor, perform the method of any of claims 1-7.
10. A computer readable storage medium, having stored thereon computer program instructions which, when read and executed by a processor, perform the method of any of claims 1-8.
11. An electronic device, comprising: a memory and a processor, the memory having stored therein computer program instructions that, when read and executed by the processor, perform the method of any of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210239286.3A CN116266117A (en) | 2022-03-11 | 2022-03-11 | Model conversion method, computer program product, storage medium, and electronic device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210239286.3A CN116266117A (en) | 2022-03-11 | 2022-03-11 | Model conversion method, computer program product, storage medium, and electronic device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116266117A true CN116266117A (en) | 2023-06-20 |
Family
ID=86744085
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210239286.3A Pending CN116266117A (en) | 2022-03-11 | 2022-03-11 | Model conversion method, computer program product, storage medium, and electronic device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116266117A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117873910A (en) * | 2024-03-13 | 2024-04-12 | 磐基技术有限公司 | Rapid prototyping system, method, apparatus, storage medium, and product for intelligent algorithms |
| CN118012468A (en) * | 2024-04-08 | 2024-05-10 | 浙江深象智能科技有限公司 | Model processing method, system and equipment |
-
2022
- 2022-03-11 CN CN202210239286.3A patent/CN116266117A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117873910A (en) * | 2024-03-13 | 2024-04-12 | 磐基技术有限公司 | Rapid prototyping system, method, apparatus, storage medium, and product for intelligent algorithms |
| CN118012468A (en) * | 2024-04-08 | 2024-05-10 | 浙江深象智能科技有限公司 | Model processing method, system and equipment |
| CN118012468B (en) * | 2024-04-08 | 2024-07-09 | 浙江深象智能科技有限公司 | Model processing method, system and equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107766126B (en) | Container mirror image construction method, system and device and storage medium | |
| CN112199086B (en) | Automatic programming control system, method, device, electronic equipment and storage medium | |
| CN110851167B (en) | Container environment updating method, device, equipment and storage medium | |
| CN116266117A (en) | Model conversion method, computer program product, storage medium, and electronic device | |
| CN109408800B (en) | Dialogue robot system and related skill configuration method | |
| CN112036577B (en) | Method and device for applying machine learning based on data form and electronic equipment | |
| CN113449858A (en) | Processing method of neural network model and related equipment | |
| CN112256321A (en) | Static library packaging method and device, computer equipment and storage medium | |
| CN110109681B (en) | Method and system for converting codes between different platforms | |
| CN108595342A (en) | Unit test method and device | |
| US20190303115A1 (en) | Automated source code sample adaptation | |
| CN111142884B (en) | Version deployment method and device of applet, electronic equipment and storage medium | |
| CN111459621B (en) | Cloud simulation integration and scheduling method and device, computer equipment and storage medium | |
| CN112633501A (en) | Development method and system of machine learning model framework based on containerization technology | |
| CN114297056A (en) | Automatic testing method and system | |
| CN113505082A (en) | Application program testing method and device | |
| Del Sole | Microsoft computer vision APIs distilled: Getting started with cognitive services | |
| CN117076335B (en) | Model test method, system, medium and electronic equipment | |
| CN112559343B (en) | Test path generation method and related equipment | |
| CN117520205A (en) | AI software stack testing method and device, computer readable storage medium and terminal | |
| CN117035065A (en) | Model evaluation method and related device | |
| CN112199896A (en) | Chip logic comprehensive optimization acceleration method based on machine learning | |
| CN116643814A (en) | Model library construction method, model calling method based on model library and related equipment | |
| CN113326113A (en) | Task processing method and device, electronic equipment and storage medium | |
| CN113935100B (en) | Cloud modeling method, cloud modeling device and cloud modeling system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |