CN117149270A - Method, system and related equipment for generating model file crossing hardware platform - Google Patents
Method, system and related equipment for generating model file crossing hardware platform Download PDFInfo
- Publication number
- CN117149270A CN117149270A CN202311417570.6A CN202311417570A CN117149270A CN 117149270 A CN117149270 A CN 117149270A CN 202311417570 A CN202311417570 A CN 202311417570A CN 117149270 A CN117149270 A CN 117149270A
- Authority
- CN
- China
- Prior art keywords
- model
- file
- data set
- platform
- indication information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 81
- 238000006243 chemical reaction Methods 0.000 claims abstract description 91
- 238000004806 packaging method and process Methods 0.000 claims abstract description 30
- 238000009434 installation Methods 0.000 claims abstract description 21
- 238000012360 testing method Methods 0.000 claims description 53
- 238000011156 evaluation Methods 0.000 claims description 50
- 238000012545 processing Methods 0.000 claims description 22
- 238000002054 transplantation Methods 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 5
- 238000010586 diagram Methods 0.000 description 12
- 238000013508 migration Methods 0.000 description 10
- 230000005012 migration Effects 0.000 description 10
- 238000013528 artificial neural network Methods 0.000 description 6
- 238000011161 development Methods 0.000 description 6
- 238000013139 quantization Methods 0.000 description 6
- 230000005540 biological transmission Effects 0.000 description 5
- 238000013527 convolutional neural network Methods 0.000 description 5
- 238000001514 detection method Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000012546 transfer Methods 0.000 description 4
- 230000010354 integration Effects 0.000 description 3
- 230000001537 neural effect Effects 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- LLQPHQFNMLZJMP-UHFFFAOYSA-N Fentrazamide Chemical compound N1=NN(C=2C(=CC=CC=2)Cl)C(=O)N1C(=O)N(CC)C1CCCCC1 LLQPHQFNMLZJMP-UHFFFAOYSA-N 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000004064 recycling Methods 0.000 description 1
- 238000007711 solidification Methods 0.000 description 1
- 230000008023 solidification Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/70—Software maintenance or management
- G06F8/76—Adapting program code to run in a different environment; Porting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/60—Software deployment
- G06F8/61—Installation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/34—Network arrangements or protocols for supporting network services or applications involving the movement of software or configuration parameters
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention provides a method, a system and related equipment for generating a model file crossing a hardware platform, relates to the technical field of computers, and aims to solve the problem of low operation convenience of conventional algorithm transplanting. The method comprises the steps of receiving a model file of a source model, target indication information and a calibration data set, wherein the model file is sent by cloud equipment, the target indication information is used for indicating model conversion parameters for migrating the source model of a first platform to a second platform, and the calibration data set is used for calibrating conversion data in a model conversion process; converting the source model according to the target indication information and the calibration data set to obtain a model file of a back-end model; packaging the model file of the back-end model to obtain a packaged file, wherein the packaged file is an installation package file of the back-end model; and sending the packaged file to the cloud device. The method can improve the convenience of algorithm transplanting operation and reduce the difficulty of algorithm transplanting.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method, a system, and a related device for generating a model file across hardware platforms.
Background
With the rapid development of digital transformation in various industries, edge computing is taken as an important component part in an industrial Internet 'end-side-cloud' architecture, and various current edge intelligent business scenes require that various types of AI algorithm models can run on artificial intelligence (Artificial Intelligence, AI) computing chips with different models of edge intelligent gateways. Therefore, for different types of AI algorithm models trained by the cloud, different AI algorithm chips need to be transplanted.
However, the existing AI algorithm transplanting technology based on the edge intelligent gateway has the problems of strong specificity and great transplanting difficulty, and a professional technical person who needs to master the model development, a model conversion tool provided by an AI chip manufacturer and the skills of an inference software development kit (Software Development Kit, SDK) simultaneously finishes algorithm transplanting under the operation of a full command line, so that the operation convenience of algorithm transplanting is low.
Disclosure of Invention
The embodiment of the invention provides a method, a system and related equipment for generating a model file crossing a hardware platform, which are used for solving the problem of low operation convenience of the conventional algorithm transplanting.
In order to solve the technical problems, the invention is realized as follows:
in a first aspect, an embodiment of the present invention provides a method for generating a model file across hardware platforms, which is applied to an edge gateway device, and includes:
receiving a model file of a source model, target indication information and a calibration data set, wherein the model file is sent by cloud equipment, the target indication information is used for indicating model conversion parameters for migrating the source model of a first platform to a second platform, and the calibration data set is used for calibrating conversion data in a model conversion process;
converting the source model according to the target indication information and the calibration data set to obtain a model file of a back-end model;
packaging the model file of the back-end model to obtain a packaged file, wherein the packaged file is an installation package file of the back-end model;
and sending the packaged file to the cloud device.
In a second aspect, an embodiment of the present invention provides a method for generating a model file across hardware platforms, which is applied to cloud devices, and includes:
obtaining a model file of a source model, a calibration data set and target indication information, wherein the target indication information is used for indicating model conversion parameters for migrating the source model of a first platform to a second platform, and the calibration data set is used for calibrating conversion data in a model conversion process;
Transmitting the model file of the source model, the calibration data set and the target indication information to edge gateway equipment of the second platform;
and receiving a packaging file sent by the edge gateway equipment of the second platform, wherein the packaging file is an installation package file of a back-end model obtained by performing model conversion on the edge gateway equipment of the second platform according to the model file of the source model, the calibration data set and the target indication information.
In a third aspect, an embodiment of the present invention provides a system for generating a model file across hardware platforms, including:
the model transplanting module is used for converting the source model according to the target indication information and the calibration data set to obtain a model file of the back-end model, and packaging the model file of the back-end model to obtain a model packaging file, wherein the model packaging file is an installation package file of the back-end model;
the model evaluation module is used for testing a back-end model of the edge gateway equipment deployed on the second platform by adopting a test data set to obtain an evaluation report, wherein the evaluation report is used for indicating the algorithm accuracy and the processing performance of the back-end model;
An asset management module for storing a model file of the source model, a model file of the back-end model, a packaged file of the back-end model, and the calibration data set and the test data set;
the cloud edge coordination module is used for controlling the flow of a cloud edge tool chain for the source model transplantation, and the cloud edge tool chain is used for carrying out model transplantation and model evaluation on the back end model corresponding to the source model based on an edge end gateway device.
In a fourth aspect, an embodiment of the present invention provides a device for generating a model file across hardware platforms, which is applied to an edge gateway device, including:
the system comprises a receiving module, a storage module and a storage module, wherein the receiving module is used for receiving a model file of a source model, target indication information and a calibration data set, the model file is sent by cloud equipment, the target indication information is used for indicating model conversion parameters for migrating the source model of a first platform to a second platform, and the calibration data set is used for calibrating conversion data in a model conversion process;
the conversion module is used for converting the source model according to the target indication information and the calibration data set to obtain a model file of a back-end model;
the packaging module is used for packaging the model file of the back-end model to obtain a packaged file, wherein the packaged file is an installation package file of the back-end model;
And the sending module is used for sending the packaged file to the cloud device.
In a fifth aspect, an embodiment of the present invention provides a device for generating a model file across hardware platforms, which is applied to cloud devices, and includes:
the system comprises an acquisition module, a calibration data set and target indication information, wherein the acquisition module is used for acquiring a model file of a source model, the calibration data set and the target indication information, the target indication information is used for indicating model conversion parameters for transferring the source model of a first platform to a second platform, and the calibration data set is used for calibrating conversion data in a model conversion process;
the sending module is used for sending the model file of the source model, the calibration data set and the target indication information to edge gateway equipment of the second platform;
the receiving module is used for receiving a packaging file sent by the edge gateway equipment of the second platform, wherein the packaging file is an installation package file of a back-end model obtained by performing model conversion on the edge gateway equipment of the second platform according to the model file of the source model, the calibration data set and the target indication information.
In a sixth aspect, an embodiment of the present invention provides a device for generating a model file across hardware platforms, where the device includes: a transceiver, a memory, a processor, and a program stored on the memory and executable on the processor;
The processor is configured to read a program in the memory to implement the steps in the method according to the first aspect; alternatively, the processor is configured to read a program in a memory to implement the steps in the method according to the second aspect.
In a seventh aspect, embodiments of the present invention provide a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method as described in the first aspect above; alternatively, the computer program is executed by a processor to implement the steps of the method as described in the second aspect above.
According to the method for generating the model file across the hardware platform, the source model can be converted according to the target indication information and the calibration data set sent by the receiving cloud device to obtain the model file of the back-end model, the model file of the back-end model is packaged to obtain the packaged file, the packaged file is sent to the cloud device, so that a user can determine the source model, the target indication information and the calibration data set in the cloud device, further the packaged file generated by the edge gateway device according to the target indication information can be directly downloaded in the cloud device, and the installation and the use are carried out, so that the algorithm model can be transplanted from the first platform to the second platform, the convenience of algorithm transplanting operation can be improved, and the algorithm transplanting difficulty is reduced.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments of the present invention will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort to a person of ordinary skill in the art.
FIG. 1 is one of the flowcharts of a method for generating a model file across hardware platforms provided by an embodiment of the present invention;
FIG. 2 is a second flowchart of a method for generating a model file across hardware platforms according to an embodiment of the present invention;
FIG. 3 is a third flowchart of a method for generating a model file across hardware platforms according to an embodiment of the present invention;
FIG. 4 is a flowchart of a method for generating a model file across hardware platforms according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a system for generating a model file across hardware platforms according to an embodiment of the present invention;
FIG. 6 is a second schematic diagram of a system for generating a model file across hardware platforms according to an embodiment of the present invention;
FIG. 7 is a schematic structural diagram of a device for generating a model file across hardware platforms according to an embodiment of the present invention;
FIG. 8 is a second schematic structural diagram of a device for generating a model file across hardware platforms according to an embodiment of the present invention;
FIG. 9 is a schematic diagram of a device for generating a model file across hardware platforms according to an embodiment of the present invention;
fig. 10 is a schematic diagram of a second embodiment of a device for generating a model file across hardware platforms.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
For ease of understanding, some of the following descriptions are directed to embodiments of the present invention:
with the rapid development of digital transformation in various industries, edge computing is used as an important component in an industrial Internet 'end-side-cloud' architecture, and by providing computing capacity locally, the number of data uploading is reduced, network bandwidth is saved, service delay is reduced, privacy and safety are enhanced, and the closed loop of local business capacity is realized. The edge intelligent gateway is a hardware device in an edge computing environment and is used for connecting and managing a camera or an internet of things device, and analyzing and processing heterogeneous data of texts, pictures and videos from the device through AI. The edge intelligent gateway mainly takes a System On Chip (SOC) as a main control Chip and provides AI computing power in a mode of an image processor (Graphics Processing Unit, GPU) or a neural network processor (Neural network Processing Unit, NPU).
The current diversified edge intelligent service scenes require that various AI algorithm models can run on AI algorithm power chips of different models of the edge intelligent gateway. Therefore, for different types of AI algorithm models trained by the cloud, different AI algorithm chips need to be transplanted. Mainly comprises the following steps:
different kinds of algorithm models, such as target detection models like once only (You Only Look Once, YOLO), single-shot multi-frame detector (Single Shot MultiBox Detector, SSD), focus loss for dense target detection (Focal Loss for Dense Object Detection, retinaNet), image classification models like deep residual learning for image recognition (Deep Residual Learning for Image Recognition, resNet), efficient convolutional neural network for mobile vision applications (Efficient Convolutional Neural Networks for Mobile Vision Applications, mobileNet); different algorithmic model frameworks such as Tensorflow, pyTorch, open neural network exchange (Open Neural Network Exchange, ONNX), fast feature embedded convolution structure (Convolutional Architecture for Fast Feature Embedding, caffe), etc.; different computing chips, such as NVIDIA Jetson NX, rayleigh micro RK3588/3568/3399Pro, katana 220, etc.
The main process of algorithm transplanting comprises the following steps:
firstly, preparing environment, installing a model conversion tool chain matched with an AI computing chip and relying on a three-party library file;
secondly, model conversion and debugging are carried out, quantization parameters are configured, and then the model is converted and a back-end model is derived;
then, reasoning development, and developing and compiling reasoning engineering based on the SDK provided by the AI computing chip;
finally, the model is deployed and packaged into algorithm services or applications which can be executed by the edge intelligent gateway.
However, the existing AI algorithm transplanting technology based on the edge intelligent gateway has the following disadvantages:
first, the specialization is too strong, and a person who needs to master the development of an algorithm model, a model conversion tool provided by an AI chip manufacturer and a professional technology for reasoning SDK skills at the same time finishes the algorithm transplanting under the operation of a full command line.
Secondly, the transplanting difficulty is high, heterogeneous hardware platforms, middleware dependence, running environments and algorithm deployment of different edge intelligent gateways are greatly different, and the algorithm transplanting workload is high and the period is long.
Thirdly, the repeatability of the transplanting work is high, a plurality of algorithm models are respectively matched for AI chips of a plurality of models, and a large number of repeated works which can be processed in batch exist in the processes of environment and parameter configuration, reasoning development and model deployment.
Fourth, repeated tuning and later version iterative updating in the algorithm transplanting process all require repeated use of the source model, quantization deviation correction and evaluation dataset, and derivation of different versions of the back-end model, while the existing transplanting method is difficult to repeatedly use the model and data resources.
In the embodiment of the invention, a method, a system and related equipment for generating a model file crossing a hardware platform are provided, so that the problem of low operation convenience of the conventional algorithm transplanting is solved.
Referring to fig. 1, fig. 1 is one of flowcharts of a method for generating a model file across hardware platforms, which is provided by an embodiment of the present invention, and is applied to an edge gateway device, as shown in fig. 1, and the method includes the following steps:
step 101, receiving a model file of a source model, target indication information and a calibration data set, wherein the model file is sent by cloud equipment, the target indication information is used for indicating model conversion parameters for migrating the source model of a first platform to a second platform, and the calibration data set is used for calibrating conversion data in a model conversion process.
Specifically, the cloud device may be a device for performing algorithm transplanting operation by a user, and may specifically be an electronic device such as a mobile phone, a computer, and the like. The source model is an algorithm model which can be operated on a first platform, the first platform and the second platform are different hardware platforms, and the second platform is a target hardware platform for algorithm transplantation of a user. The target indication information may be a model conversion parameter designated by a user when the cloud device performs algorithm migration, and is used for indicating migration of the source model of the first platform to the second platform, and may specifically include a quantization parameter of model conversion and second platform information.
It should be noted that the target indication information may indicate a plurality of the second platforms.
And 102, converting the source model according to the target indication information and the calibration data set to obtain a model file of a back-end model.
It should be noted that, the tool corresponding to the algorithm migration work chain of the edge gateway device integration algorithm may specifically include a model conversion, an evaluation tool and an inference SDK, which may be preset by the edge gateway device or may be an algorithm migration tool corresponding to an authority that receives the cloud device transmission.
And 103, packaging the model file of the back-end model to obtain a packaged file, wherein the packaged file is an installation package file of the back-end model.
Specifically, the package file may include a model file of the back-end model, a runtime middleware, an inference engine, and the like. An application programming interface (Application Programming Interface, API) for providing reasoning services externally in a Docker containerized manner by integrating the runtime middleware, the reasoning engine.
And 104, sending the packaged file to the cloud device.
Specifically, the packaged file is sent to the cloud device, so that a user can directly download the packaged file at the cloud device for installation and use, and the transplanting of the algorithm model is completed.
According to the method for generating the model file across the hardware platform, the source model can be converted according to the target indication information and the calibration data set sent by the receiving cloud device to obtain the model file of the back-end model, the model file of the back-end model is packaged to obtain the packaged file, the packaged file is sent to the cloud device, so that a user can determine the source model, the target indication information and the calibration data set in the cloud device, further the packaged file generated by the edge gateway device according to the target indication information can be directly downloaded in the cloud device, and the installation and the use are carried out, so that the algorithm model can be transplanted from the first platform to the second platform, the convenience of algorithm transplanting operation can be improved, and the algorithm transplanting difficulty is reduced.
Optionally, the method further comprises:
deploying the packaged file to the edge gateway device;
receiving a test data set sent by the cloud device, wherein the test data set is used for testing the performance and the precision of the back-end model deployed on the second platform;
testing the packaged file by adopting the test data set to obtain an evaluation report, wherein the evaluation report is used for indicating the algorithm accuracy and the processing performance of the back-end model;
And sending the evaluation report to the cloud device.
In the embodiment of the invention, the method for generating the model file across the hardware platform can be used for deploying the package file in the edge gateway device, receiving the test data set sent by the cloud device, testing the package file by adopting the test data set to obtain the evaluation report, wherein the evaluation report can be used for indicating the algorithm accuracy and the processing performance of the back-end model, further can be used for judging the effect of model conversion, and sending the evaluation report to the cloud device, so that the cloud device can synchronize the performance parameters of the algorithm model conversion, and is convenient for users to refer to and the administrator of the cloud device further perfects the algorithm conversion process.
Optionally, before the converting the source model according to the target indication information and the calibration data set to obtain a model file of a back-end model, the method further includes:
determining whether the source model is an intermediate representation IR model;
under the condition that the source model is not an IR model, carrying out IR conversion on the source model to obtain a target source model;
The converting the source model according to the target indication information and the calibration data set to obtain a model file of a back-end model, including:
and converting the target source model according to the target indication information and the calibration data set to obtain a model file of a back-end model.
Specifically, in the case where the source model is not an IR model, the source model is converted to a serial number intermediate representation IR model (such as ONNX or TorchScript), and then quantized and converted to an inferable back-end model by a tool chain corresponding to the hardware platform.
Fig. 2 is a second flowchart of a method for generating a model file across hardware platforms according to an embodiment of the present invention, where, as shown in fig. 2, before performing model conversion, it is preferentially determined whether the source model is an intermediate representation (Intermediate Representation, IR) model, if the source model is not an IR model, IR conversion is performed on the source model, and the obtained IR model is quantized and converted into an inferable back-end model through a tool chain corresponding to the hardware platform. For example, for a central processing unit (Central Processing Unit, CPU) can use ONNXRuntime for model reasoning; for a rayleigh micro-neural processor (RockChip Neural Processing Unit, rockChip NPU), after the conversion using a rayleigh micro-neural network conversion tool (RockChip Neural Network Conversion Tool, RKNN cover) tool, reasoning is performed by a neural processor transmission tool (Neural Processing Unit Transfer Tool, NPU transfer); for an Innova graphics processor (NVIDIA Graphics Processing Unit, NVIDIA GPU) TensorRT conversion tools (TensorRT Conversion Tool, tensorRT converter) can be used to convert and then infer by TensorRT.
In the embodiment of the invention, the generation method of the model file crossing the hardware platform can determine the source model to be converted as the IR model before the model conversion is carried out, so that the algorithm model can be deployed and applied to more hardware platforms, and the universality of the algorithm model is improved.
Referring to fig. 3, fig. 3 is a third flowchart of a method for generating a model file across hardware platforms, which is applied to a cloud device and shown in fig. 3, and the method includes the following steps:
step 301, obtaining a model file of a source model, a calibration data set and target indication information, wherein the target indication information is used for indicating model conversion parameters for migrating the source model of a first platform to a second platform, and the calibration data set is used for calibrating conversion data in a model conversion process.
Specifically, the target indication information may be indication information generated when the user performs algorithm migration on the cloud device, and is specifically used for indicating a model conversion parameter for migrating the source model of the first platform to the second platform, where the model conversion parameter includes a quantization parameter and second platform information.
Step 302, sending the model file of the source model, the calibration data set and the target indication information to an edge gateway device of the second platform.
Step 303, receiving a package file sent by the edge gateway device of the second platform, where the package file is an installation package file of a back-end model obtained by performing model conversion on the edge gateway device of the second platform according to the model file of the source model, the calibration data set and the target indication information.
Optionally, after receiving the package file sent by the edge gateway device of the second platform, the method further includes:
transmitting a test data set to the edge gateway device, wherein the test data set is used for testing performance and precision of the back-end model deployed on the second platform;
and receiving an evaluation report sent by the edge gateway equipment, wherein the evaluation report is obtained by testing the back-end model of the second platform based on the test data set, and the evaluation report is used for indicating the algorithm accuracy and processing performance of the back-end model.
It should be noted that, in this embodiment, as an implementation manner of the cloud device corresponding to the embodiment shown in fig. 1, a specific implementation manner of the embodiment may refer to a related description of the embodiment shown in fig. 1, so that in order to avoid repeated description, the embodiment is not described again, and the same beneficial effects may be achieved.
Fig. 4 is a flowchart of a method for generating a model file across a hardware platform according to an embodiment of the present invention, as shown in fig. 4, the flow of the method for generating a model file across a hardware platform is as follows:
firstly, uploading a source model and a quantized data set to be converted to cloud equipment, storing the source model and the quantized data set into an algorithm resource management module of the cloud equipment, and if the source model exists in the cloud equipment, directly selecting the source model for conversion by a user without repeating the source model uploading;
secondly, configuring a converted target hardware platform and quantization parameters, uploading a calibration data set, and if the cloud device already has the calibration data set, directly selecting the calibration data set by a user without repeating the uploading of the calibration data set;
thirdly, the source model, the configuration parameters and the calibration data set are sent to edge gateway equipment corresponding to the hardware platform, and a matched tool corresponding to an AI computing chip in the edge gateway equipment is driven to complete back-end model conversion and is exported to an algorithm resource management module of the cloud equipment;
then, the back-end model is operated to be deployed on the edge gateway equipment corresponding to the hardware platform;
and finally, driving a matched tool corresponding to the AI computing chip in the edge gateway equipment to complete model performance and precision evaluation, and synchronizing evaluation data to a cloud evaluation report.
The embodiment of the invention also provides a system for generating the model file crossing the hardware platform, which comprises the following steps:
the model transplanting module is used for converting the source model according to the target indication information and the calibration data set to obtain a model file of the back-end model, and packaging the model file of the back-end model to obtain a model packaging file, wherein the model packaging file is an installation package file of the back-end model;
the model evaluation module is used for testing a back-end model of the edge gateway equipment deployed on the second platform by adopting a test data set to obtain an evaluation report, wherein the evaluation report is used for indicating the algorithm accuracy and the processing performance of the back-end model;
an asset management module for storing a model file of the source model, a model file of the back-end model, a packaged file of the back-end model, and the calibration data set and the test data set;
the cloud edge coordination module is used for controlling the flow of a cloud edge tool chain for the source model transplantation, and the cloud edge tool chain is used for carrying out model transplantation and model evaluation on the back end model corresponding to the source model based on an edge end gateway device.
Fig. 5 is a schematic structural diagram of a system for generating a model file across hardware platforms, which is provided by an embodiment of the present invention, and as shown in fig. 5, the system for generating a model file across hardware platforms uses an operation guide and pipeline mode and a cloud edge collaboration scheme to perform flow solidification and tool chain integration on an algorithm model migration process, so as to mask differences of different heterogeneous hardware platforms (CPU, GPU, NPU and other computing chips), different algorithm model types (YOLO, convolutional neural network feature-based region detection algorithms (Regions with Convolutional Neural Network features, R-CNN), SSD, resNet and the like), and different algorithm model frames (TensorFlow, pyTorch, caffe and the like).
The system provides an algorithm transplanting tool with a Browser/Server (B/S) structure, and a user can operate the whole algorithm transplanting process in a visual mode.
Specifically, as shown in fig. 5, the system sets an algorithm model conversion guide, manages the process of integrating algorithm model migration through tasks of a pipeline, and provides a user-friendly visual WEB interface to simplify configuration and operation processes, which may include operation options such as model selection, quantization configuration, model conversion, model deployment, and model evaluation.
The system comprises an asset management module, in particular an algorithm asset warehouse, which carries out version and weight division domain management on input and output data of algorithm migration, and comprises the following steps: an input source algorithm model, a calibration data set and an evaluation data set, and an output back-end algorithm model.
The cloud edge coordination system comprises cloud edge coordination modules, particularly tool chain cloud edge coordination, model conversion of partial AI computing power chips and deployment and evaluation of a back-end model are required to be completed on corresponding hardware platforms, and cloud edge tool chain integration is required to be supported through a cloud edge control and data synchronization mechanism.
Specifically, fig. 6 is a second schematic structural diagram of a system for generating a model file across hardware platforms, where, as shown in fig. 6, the system integrates algorithm migration tool chains of different hardware platforms and uses a cloud edge collaboration module to implement unified management, and provides an algorithm model migration service for a user, where the main contents include:
the method comprises the steps that a Client (Client) is deployed at an edge gateway device to shield differences of different hardware platforms in model conversion, reasoning and evaluation, an integrated environment of an algorithm transplanting tool chain is provided, and linkage of cloud algorithm transplanting operation guidance is realized through message queue telemetry transmission (Message Queuing Telemetry Transport, MQTT); and implementing resource synchronization of the cloud device and edge gateway device algorithm model and the data set through a secure hypertext transfer protocol (Hypertext Transfer Protocol Secure, HTTPS) distributed file storage service.
The system can also set a back-end algorithm model running environment, integrates a running middleware and an inference engine, externally provides an API of an inference service in a dock container mode, and completes the packaging operation of the back-end model so as to be convenient for a user to directly download and use.
In the embodiment of the invention, the generation system of the model file crossing the hardware platform can complete one-key conversion of the back-end algorithm model and generate an evaluation report under the condition of little or no manual intervention by providing a guide type user operation interface; the asset management module is arranged, so that the system can carry out version-based weight and domain division management on model files, databases and the like in the process of carrying out algorithm model transplanting, and the recycling of data resources is realized; meanwhile, when the model conversion is carried out on the same source model, a plurality of target hardware platforms can be selected to carry out conversion at the same time, so that a large number of repeated work which can be processed in batch can be reduced.
Referring to fig. 7, fig. 7 is one of schematic structural diagrams of a generating apparatus of a model file across hardware platforms, which is provided in an embodiment of the present invention, and is applied to an edge gateway device, as shown in fig. 7, the generating apparatus 700 of a model file across hardware platforms includes:
the first receiving module 701 is configured to receive a model file of a source model, target indication information, and a calibration data set, where the model file is sent by the cloud device, the target indication information is used to indicate model conversion parameters for migrating the source model of the first platform to the second platform, and the calibration data set is used to calibrate conversion data in a model conversion process;
A first conversion module 702, configured to convert the source model according to the target indication information and the calibration data set, to obtain a model file of a back-end model;
the packaging module 703 is configured to package the model file of the back-end model to obtain a packaged file, where the packaged file is an installation package file of the back-end model;
and a first sending module 704, configured to send the packaged file to the cloud device.
Optionally, the apparatus further includes:
the deployment module is used for deploying the packaged file to the edge gateway equipment;
the second receiving module is used for receiving a test data set sent by the cloud device, and the test data set is used for testing the performance and the precision of the back-end model deployed on the second platform;
the test module is used for testing the packaged file by adopting the test data set to obtain an evaluation report, and the evaluation report is used for indicating the algorithm accuracy and the processing performance of the back-end model;
and the second sending module is used for sending the evaluation report to the cloud device.
Optionally, the apparatus further includes:
a determination module for determining whether the source model is an intermediate representation IR model;
The second conversion module is used for carrying out IR conversion on the source model under the condition that the source model is not an IR model to obtain a target source model;
the first conversion module includes:
and the conversion unit is used for converting the target source model according to the target indication information and the calibration data set to obtain a model file of the back-end model.
Referring to fig. 8, fig. 8 is a second schematic structural diagram of a device for generating a model file across hardware platforms, which is provided in an embodiment of the present invention, and is applied to a cloud device, as shown in fig. 8, the device 800 for generating a model file across hardware platforms includes:
an obtaining module 801, configured to obtain a model file of a source model, a calibration data set, and target indication information, where the target indication information is used to indicate model conversion parameters for migrating the source model of a first platform to a second platform, and the calibration data set is used to calibrate conversion data in a model conversion process;
a first sending module 802, configured to send the model file of the source model, the calibration data set, and the target indication information to an edge gateway device of the second platform;
and a receiving module 803, configured to receive a package file sent by the edge gateway device of the second platform, where the package file is an installation package file of a back-end model obtained by performing model conversion on the edge gateway device of the second platform according to the model file of the source model, the calibration data set, and the target indication information.
Optionally, the apparatus further includes:
the second sending module is used for sending a test data set to the edge gateway equipment, wherein the test data set is used for testing the performance and the precision of the back-end model deployed on the second platform;
the second receiving module is configured to receive an evaluation report sent by the edge gateway device, where the evaluation report is an evaluation report obtained by testing the back-end model of the second platform based on the test data set, and the evaluation report is used to indicate algorithm accuracy and processing performance of the back-end model.
Fig. 9 is one of schematic diagrams of a device for generating a model file across hardware platforms according to an embodiment of the present invention, where, as shown in fig. 9, the device includes: a transceiver 901, a memory 902, a processor 900, and a program stored on the memory and executable on the processor;
the transceiver 901 is configured to receive a model file of a source model sent by a cloud device, target indication information, and a calibration data set, where the target indication information is used to indicate a model conversion parameter for migrating the source model of a first platform to a second platform, and the calibration data set is used to calibrate conversion data in a model conversion process;
The processor 900 is configured to convert the source model according to the target indication information and the calibration data set to obtain a model file of a back-end model;
packaging the model file of the back-end model to obtain a packaged file, wherein the packaged file is an installation package file of the back-end model;
the transceiver 901 is also for: and sending the packaged file to the cloud device.
Where in FIG. 9, a bus architecture may comprise any number of interconnected buses and bridges, with one or more processors, represented in particular by processor 900, and various circuits of memory, represented by memory 902, linked together. The bus architecture may also link together various other circuits such as peripheral devices, voltage regulators, power management circuits, etc., which are well known in the art and, therefore, will not be described further herein. The bus interface provides an interface. The transceiver 901 may be a number of elements, including a transmitter and a transceiver, providing a means for communicating with various other apparatus over a transmission medium. The processor 900 is responsible for managing the bus architecture and general processing, and the memory 902 may store data used by the processor 900 in performing operations.
Optionally, the processor 900 is further configured to:
deploying the packaged file to the edge gateway device;
the transceiver 901 is also for:
receiving a test data set sent by the cloud device, wherein the test data set is used for testing the performance and the precision of the back-end model deployed on the second platform;
the processor 900 is also configured to:
testing the packaged file by adopting the test data set to obtain an evaluation report, wherein the evaluation report is used for indicating the algorithm accuracy and the processing performance of the back-end model;
and sending the evaluation report to the cloud device.
Optionally, the processor 900 is further configured to:
determining whether the source model is an intermediate representation IR model;
under the condition that the source model is not an IR model, carrying out IR conversion on the source model to obtain a target source model;
the processor 900 is also configured to: and converting the target source model according to the target indication information and the calibration data set to obtain a model file of a back-end model.
Fig. 10 is a second schematic diagram of a device for generating a model file across hardware platforms according to an embodiment of the present invention, as shown in fig. 10, where the device includes: a transceiver 1001, a memory 1002, a processor 1000, and a program stored on the memory and executable on the processor;
The processor 1000 is configured to: obtaining a model file of a source model, a calibration data set and target indication information, wherein the target indication information is used for indicating model conversion parameters for migrating the source model of a first platform to a second platform, and the calibration data set is used for calibrating conversion data in a model conversion process;
the transceiver 1001 is configured to: transmitting the model file of the source model, the calibration data set and the target indication information to edge gateway equipment of the second platform;
and receiving a packaging file sent by the edge gateway equipment of the second platform, wherein the packaging file is an installation package file of a back-end model obtained by performing model conversion on the edge gateway equipment of the second platform according to the model file of the source model, the calibration data set and the target indication information.
Wherein in fig. 10, a bus architecture may comprise any number of interconnected buses and bridges, and in particular, one or more processors represented by the processor 1000 and various circuits of memory represented by the memory 1002, linked together. The bus architecture may also link together various other circuits such as peripheral devices, voltage regulators, power management circuits, etc., which are well known in the art and, therefore, will not be described further herein. The bus interface provides an interface. The transceiver 1001 may be a number of elements, including a transmitter and a transceiver, providing a means for communicating with various other apparatus over a transmission medium. The processor 1000 is responsible for managing the bus architecture and general processing, and the memory 1002 may store data used by the processor 1000 in performing operations.
Optionally, the transceiver 1001 is further configured to:
transmitting a test data set to the edge gateway device, wherein the test data set is used for testing performance and precision of the back-end model deployed on the second platform;
and receiving an evaluation report sent by the edge gateway equipment, wherein the evaluation report is obtained by testing the back-end model of the second platform based on the test data set, and the evaluation report is used for indicating the algorithm accuracy and processing performance of the back-end model.
The embodiment of the invention also provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements the processes of the above embodiment of the method for generating a model file across hardware platforms, and can achieve the same technical effects, so that repetition is avoided and no further description is given here. Wherein the computer readable storage medium is selected from Read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), magnetic disk or optical disk.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The embodiments of the present invention have been described above with reference to the accompanying drawings, but the present invention is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those having ordinary skill in the art without departing from the spirit of the present invention and the scope of the claims, which are to be protected by the present invention.
Claims (10)
1. The method for generating the model file of the cross-hardware platform is applied to the edge gateway equipment and is characterized by comprising the following steps:
receiving a model file of a source model, target indication information and a calibration data set, wherein the model file is sent by cloud equipment, the target indication information is used for indicating model conversion parameters for migrating the source model of a first platform to a second platform, and the calibration data set is used for calibrating conversion data in a model conversion process;
converting the source model according to the target indication information and the calibration data set to obtain a model file of a back-end model;
packaging the model file of the back-end model to obtain a packaged file, wherein the packaged file is an installation package file of the back-end model;
and sending the packaged file to the cloud device.
2. The method according to claim 1, wherein the method further comprises:
deploying the packaged file to the edge gateway device;
receiving a test data set sent by the cloud device, wherein the test data set is used for testing the performance and the precision of the back-end model deployed on the second platform;
testing the packaged file by adopting the test data set to obtain an evaluation report, wherein the evaluation report is used for indicating the algorithm accuracy and the processing performance of the back-end model;
And sending the evaluation report to the cloud device.
3. The method of claim 1, wherein prior to said converting the source model based on the target indication information and the calibration data set to obtain a model file for a back-end model, the method further comprises:
determining whether the source model is an intermediate representation IR model;
under the condition that the source model is not an IR model, carrying out IR conversion on the source model to obtain a target source model;
the converting the source model according to the target indication information and the calibration data set to obtain a model file of a back-end model, including:
and converting the target source model according to the target indication information and the calibration data set to obtain a model file of a back-end model.
4. The method for generating the model file crossing the hardware platform is applied to cloud equipment and is characterized by comprising the following steps of:
obtaining a model file of a source model, a calibration data set and target indication information, wherein the target indication information is used for indicating model conversion parameters for migrating the source model of a first platform to a second platform, and the calibration data set is used for calibrating conversion data in a model conversion process;
Transmitting the model file of the source model, the calibration data set and the target indication information to edge gateway equipment of the second platform;
and receiving a packaging file sent by the edge gateway equipment of the second platform, wherein the packaging file is an installation package file of a back-end model obtained by performing model conversion on the edge gateway equipment of the second platform according to the model file of the source model, the calibration data set and the target indication information.
5. The method of claim 4, wherein after receiving the packaged file sent by the edge gateway device of the second platform, the method further comprises:
transmitting a test data set to the edge gateway device, wherein the test data set is used for testing performance and precision of the back-end model deployed on the second platform;
and receiving an evaluation report sent by the edge gateway equipment, wherein the evaluation report is obtained by testing the back-end model of the second platform based on the test data set, and the evaluation report is used for indicating the algorithm accuracy and processing performance of the back-end model.
6. A system for generating a model file across a hardware platform, comprising:
The model transplanting module is used for converting the source model according to the target indication information and the calibration data set to obtain a model file of the back-end model, and packaging the model file of the back-end model to obtain a model packaging file, wherein the model packaging file is an installation package file of the back-end model;
the model evaluation module is used for testing a back-end model of the edge gateway equipment deployed on the second platform by adopting a test data set to obtain an evaluation report, wherein the evaluation report is used for indicating the algorithm accuracy and the processing performance of the back-end model;
an asset management module for storing a model file of the source model, a model file of the back-end model, a packaged file of the back-end model, and the calibration data set and the test data set;
the cloud edge coordination module is used for controlling the flow of a cloud edge tool chain for the source model transplantation, and the cloud edge tool chain is used for carrying out model transplantation and model evaluation on the back end model corresponding to the source model based on an edge end gateway device.
7. A device for generating a model file across hardware platforms, which is applied to an edge gateway device, and is characterized by comprising:
The first receiving module is used for receiving a model file of the source model, target indication information and a calibration data set, wherein the model file is sent by the cloud device, the target indication information is used for indicating model conversion parameters for migrating the source model of the first platform to the second platform, and the calibration data set is used for calibrating conversion data in a model conversion process;
the first conversion module is used for converting the source model according to the target indication information and the calibration data set to obtain a model file of a back-end model;
the packaging module is used for packaging the model file of the back-end model to obtain a packaged file, wherein the packaged file is an installation package file of the back-end model;
and the first sending module is used for sending the packaged file to the cloud device.
8. The device for generating the model file across the hardware platform is applied to cloud equipment and is characterized by comprising the following components:
the system comprises an acquisition module, a calibration data set and target indication information, wherein the acquisition module is used for acquiring a model file of a source model, the calibration data set and the target indication information, the target indication information is used for indicating model conversion parameters for transferring the source model of a first platform to a second platform, and the calibration data set is used for calibrating conversion data in a model conversion process;
The first sending module is used for sending the model file of the source model, the calibration data set and the target indication information to edge gateway equipment of the second platform;
the first receiving module is used for receiving a packaging file sent by the edge gateway equipment of the second platform, wherein the packaging file is an installation package file of a back-end model obtained by the edge gateway equipment of the second platform through model conversion according to the model file of the source model, the calibration data set and the target indication information.
9. A device for generating a model file across a hardware platform, the device comprising: a transceiver, a memory, a processor, and a program stored on the memory and executable on the processor; it is characterized in that the method comprises the steps of,
the processor for reading a program in a memory to implement the steps in the method of any one of claims 1 to 3; alternatively, the processor is configured to read a program in a memory to implement the steps in the method according to any one of claims 4 to 5.
10. A readable storage medium, characterized in that it has stored thereon a computer program which, when executed by a processor, implements the steps of the method according to any of claims 1 to 3; alternatively, the computer program, when executed by a processor, implements the steps of the method according to any of claims 4 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311417570.6A CN117149270A (en) | 2023-10-30 | 2023-10-30 | Method, system and related equipment for generating model file crossing hardware platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311417570.6A CN117149270A (en) | 2023-10-30 | 2023-10-30 | Method, system and related equipment for generating model file crossing hardware platform |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117149270A true CN117149270A (en) | 2023-12-01 |
Family
ID=88899089
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311417570.6A Pending CN117149270A (en) | 2023-10-30 | 2023-10-30 | Method, system and related equipment for generating model file crossing hardware platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117149270A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118012468A (en) * | 2024-04-08 | 2024-05-10 | 浙江深象智能科技有限公司 | Model processing method, system and equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103109271A (en) * | 2012-08-30 | 2013-05-15 | 华为技术有限公司 | Inter-platform application migration realization method and system |
| CN114298284A (en) * | 2021-12-29 | 2022-04-08 | 浙江大华技术股份有限公司 | Network model conversion method, device, system, storage medium and electronic device |
| EP4095713A1 (en) * | 2021-05-27 | 2022-11-30 | Ovh | Systems and methods for generating a target dataset having a target data format on a user device |
-
2023
- 2023-10-30 CN CN202311417570.6A patent/CN117149270A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103109271A (en) * | 2012-08-30 | 2013-05-15 | 华为技术有限公司 | Inter-platform application migration realization method and system |
| EP4095713A1 (en) * | 2021-05-27 | 2022-11-30 | Ovh | Systems and methods for generating a target dataset having a target data format on a user device |
| CN114298284A (en) * | 2021-12-29 | 2022-04-08 | 浙江大华技术股份有限公司 | Network model conversion method, device, system, storage medium and electronic device |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118012468A (en) * | 2024-04-08 | 2024-05-10 | 浙江深象智能科技有限公司 | Model processing method, system and equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110888722B (en) | Task processing method and device, electronic equipment and computer readable storage medium | |
| US11389960B2 (en) | Systems and methods for robotic process automation | |
| US11514367B2 (en) | Computer-based systems and methods configured to utilize automating deployment of predictive models for machine learning tasks | |
| CN111797969A (en) | Neural network model conversion method and related device | |
| Namiot et al. | On iot programming | |
| CN117149270A (en) | Method, system and related equipment for generating model file crossing hardware platform | |
| CN108306844B (en) | Method for API communication between server and client | |
| CN103701834A (en) | Proxy based mobile cloud service access system and method | |
| CN110955715A (en) | ERP system, data conversion method and device of target platform and electronic equipment | |
| Puttonen et al. | Planning-based semantic web service composition in factory automation | |
| CN114296933A (en) | Implementation method of lightweight container under terminal edge cloud architecture and data processing system | |
| CN117132890A (en) | Remote sensing image target detection method and system based on Kubernetes edge computing cluster | |
| CN115794106A (en) | Method and system for analyzing configuration of binary protocol data of rail transit | |
| Moin et al. | Supporting AI engineering on the IoT edge through model-driven TinyML | |
| US20240095529A1 (en) | Neural Network Optimization Method and Apparatus | |
| CN116578423B (en) | Task processing method, automatic question answering method and image generation method | |
| CN115567526B (en) | Data monitoring method, device, equipment and medium | |
| US10289462B1 (en) | Systems and methods for integration of websphere optimized local adapters with the spring framework | |
| CN115599396A (en) | Machine learning model deployment method and device, equipment, medium and product thereof | |
| CN113849287A (en) | Processing method and device of algorithm service, electronic equipment and storage medium | |
| CN110050291B (en) | Aggregation platform, demand owner and methods thereof | |
| CN116860353B (en) | Data processing method, device, electronic equipment and storage medium | |
| Pillajo et al. | Implementation of a network control system for a Robotic Manipulator as cloud service | |
| WO2023039757A1 (en) | Method and apparatus for automatically configuring and deploying containerized application | |
| CN115174703B (en) | Device driver processing method, device communication method, processing system and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |