CN117075915A - Tensor flow model conversion method, system, device and readable storage medium - Google Patents
Tensor flow model conversion method, system, device and readable storage medium Download PDFInfo
- Publication number
- CN117075915A CN117075915A CN202310944731.0A CN202310944731A CN117075915A CN 117075915 A CN117075915 A CN 117075915A CN 202310944731 A CN202310944731 A CN 202310944731A CN 117075915 A CN117075915 A CN 117075915A
- Authority
- CN
- China
- Prior art keywords
- model
- file
- flow model
- tensor flow
- tensor
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000006243 chemical reaction Methods 0.000 title claims abstract description 90
- 238000000034 method Methods 0.000 title claims abstract description 48
- 238000005457 optimization Methods 0.000 claims abstract description 12
- 230000001131 transforming effect Effects 0.000 claims description 18
- 238000001514 detection method Methods 0.000 claims description 13
- 230000009466 transformation Effects 0.000 claims description 11
- 230000000977 initiatory effect Effects 0.000 claims description 8
- 230000004044 response Effects 0.000 claims description 8
- 230000008676 import Effects 0.000 claims description 6
- 238000012545 processing Methods 0.000 description 14
- 230000008569 process Effects 0.000 description 10
- 238000004891 communication Methods 0.000 description 9
- 238000013473 artificial intelligence Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 5
- 238000012549 training Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000006978 adaptation Effects 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000003491 array Methods 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000009877 rendering Methods 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 238000012356 Product development Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000010079 rubber tapping Methods 0.000 description 1
- 238000011426 transformation method Methods 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
- GPRLSGONYQIRFK-MNYXATJNSA-N triton Chemical compound [3H+] GPRLSGONYQIRFK-MNYXATJNSA-N 0.000 description 1
- 239000013598 vector Substances 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/60—Software deployment
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/04—Inference or reasoning models
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a method, a system, a device and a readable storage medium for converting a tensor flow model, wherein the method comprises the following steps: acquiring directory information of the tensor flow model through an artificial intelligent platform, and sending a model conversion request; the method comprises the steps of deriving directory information of a tensor stream model, converting address information of the model into mounting point addresses in a container according to the directory information of the tensor stream model, generating a console statement for converting the tensor stream model into ONNX files, and requesting the container on a corresponding server; detecting whether the ONNX file exists or not by using directory information of the tensor flow model, and if so, converting the ONNX file into a corresponding reasoning model file through a model optimization tool; and confirming that the conversion of the reasoning model file is successful by using the directory information of the tensor flow model. According to the invention, the tensor flow model is converted into the high-performance reasoning model, so that the reasoning efficiency of deploying the reasoning service, and the use efficiency and experience of the user are effectively improved.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method, a system, an apparatus, and a readable storage medium for transforming a tensor flow model.
Background
With the wide use of related applications of artificial intelligence platforms in the internet industry, the demands of internet companies for artificial intelligence platforms are increasing. A large number of Internet companies build artificial intelligent platforms for external or internal use, and most of the platforms are based on a Kubernetes scheduling platform integrated model library and support training and reasoning of engines such as Tensorflow and Pytorch so as to meet the internal requirements of the companies.
In the existing artificial intelligent platform, the model mode supports the functions of model inquiry, model import, model export, model verification, model deletion and the like, and version deployment of a newly added frame model can be supported by expanding a model engine built in the artificial intelligent platform. When the model is deployed, a user can select a corresponding model frame to perform service deployment according to the model requirement, and the frame types supported by the current model deployment comprise pytorch, tensorflow, triton, onnx and tvm, and the supported functions are shown in figure 1.
However, such artificial intelligence platforms do not perform corresponding adaptation processing for deployment of tensor flow models (tensor flow models), which makes the tensor flow model framework difficult to perform its real function in the absence of corresponding plug-in integration, and is disadvantageous for efficiency improvement in an objective aspect. At present, a model with a Tensorflow SavedModel format is usually imported by using a Tensorflow Serving framework model version at the tensor flow model deployment level, and the model deployment with relatively stable and high compatibility can be provided in the use process, but the reasoning performance and efficiency of the model are improved relatively limited, and the effect of high-performance reasoning is difficult to achieve.
Disclosure of Invention
Aiming at the problems, the invention aims to provide a method, a system, a device and a readable storage medium for converting a tensor flow model, which effectively improve the reasoning efficiency of deploying a reasoning service and the use efficiency and experience of a user by converting the tensor flow model into a high-performance reasoning model.
The invention aims to achieve the aim, and the aim is achieved by the following technical scheme:
in a first aspect, the invention discloses a method for transforming a tensor flow model, which comprises the following steps:
acquiring directory information of the tensor flow model through an artificial intelligent platform, and sending a model conversion request;
in response to receiving the model conversion request, deriving directory information for the tensor flow model;
starting a model conversion flow;
converting the address information of the model into a mounting point address in the container according to the directory information of the tensor flow model; generating a console statement for converting the tensor flow model into an ONNX file, and requesting a container on a corresponding server;
detecting whether ONNX files exist or not by using directory information of a tensor flow model;
if not, directly exiting after reporting errors; if so, converting the ONNX file into a corresponding reasoning model file through a model optimization tool;
detecting whether a corresponding reasoning model file exists or not by using directory information of the tensor flow model;
if not, directly exiting after reporting errors; if present, the conversion is complete.
Further, the starting the model conversion process includes:
in the conversion mode selection list, a conversion mode is selected as the inference model conversion, and the conversion state is set to be in progress.
Further, the converting the address information of the model into the mount point address in the container according to the directory information of the tensor flow model includes:
according to the main key of the model conversion request, the catalog information of the corresponding tensor flow model is taken out from the database;
the import and export addresses of the tensor flow model are translated into mount point addresses in the container.
Further, the generating a console statement that converts the tensor flow model into an ONNX file includes: according to the address of the mounting point in the container, using notes to obtain the container information under the current server configuration, and injecting the container information into the corresponding attribute to generate a console statement;
the sheet flow model is converted into an ONNX file by a console statement.
Further, the detecting whether the ONNX file exists by using the directory information of the tensor flow model includes:
acquiring a corresponding mounting point address by using directory information of the tensor flow model;
detecting whether an ONNX file is associated with the mounting point address;
if yes, ONNX files exist; otherwise the ONNX file does not exist.
Further, the detecting whether the corresponding optimization model file exists by using the directory information of the tensor flow model includes:
acquiring a corresponding mounting point address by using directory information of the tensor flow model;
detecting whether an inference model file is associated with the mounting point address or not;
if yes, the reasoning model file exists; otherwise, the reasoning model file does not exist.
Further, the container information includes: the name of the container, the address of the server where it is located, the address of the network file system where it is mounted.
In a second aspect, the present invention also discloses a system for transforming a tensor flow model, including:
the request initiating unit is configured to acquire directory information of the tensor flow model through the artificial intelligent platform and send a model conversion request;
a catalog deriving unit configured to derive catalog information of the tensor flow model in response to receiving the model conversion request;
the transformation starting unit is configured to start a model transformation flow;
the address conversion unit is configured to convert the address information of the model into the mounting point address in the container according to the directory information of the tensor flow model;
the instruction generating unit is configured to generate a console statement for converting the tensor flow model into an ONNX file and request a container on a corresponding server;
a first file detection unit configured to detect whether an ONNX file exists using directory information of a tensor flow model;
the file conversion unit is configured to convert the ONNX file into a corresponding reasoning model file through the model optimization tool when the first file detection unit detects that the ONNX file exists;
a second file detection unit configured to detect whether a corresponding inference model file exists using directory information of the tensor flow model;
and the alarm unit is configured to directly exit after reporting errors when the first file detection unit does not detect the ONNX file or the second file detection unit does not detect the reasoning model file.
Further, the conversion start unit is specifically configured to: in the conversion mode selection list, a conversion mode is selected as the inference model conversion, and the conversion state is set to be in progress.
Further, the address translation unit is specifically configured to: according to the main key of the model conversion request, the catalog information of the corresponding tensor flow model is taken out from the database; the import and export addresses of the tensor flow model are translated into mount point addresses in the container.
Further, the file conversion unit is specifically configured to: according to the address of the mounting point in the container, using notes to obtain the container information under the current server configuration, and injecting the container information into the corresponding attribute to generate a console statement; the sheet flow model is converted into an ONNX file by a console statement.
Further, the first file detecting unit is specifically configured to: acquiring a corresponding mounting point address by using directory information of the tensor flow model; detecting whether an ONNX file is associated with the mounting point address; if yes, ONNX files exist; otherwise the ONNX file does not exist.
Further, the second file detecting unit is specifically configured to: acquiring a corresponding mounting point address by using directory information of the tensor flow model; detecting whether an inference model file is associated with the mounting point address or not; if yes, the reasoning model file exists; otherwise, the reasoning model file does not exist.
In a third aspect, the present invention also discloses a device for transforming a tensor flow model, including:
a memory for storing a conversion program of the tensor flow model;
a processor for implementing the steps of the method for transforming a tensor flow model according to any one of the preceding claims when executing the transformation procedure of the tensor flow model.
In a fourth aspect, the present invention also discloses a readable storage medium, on which a conversion program of a tensor flow model is stored, which when executed by a processor, implements the steps of the method for converting a tensor flow model as described in any one of the above.
Compared with the prior art, the invention has the beneficial effects that: the invention discloses a method, a system, a device and a readable storage medium for converting a tensor flow model, which are used for converting the tensor flow model into an ONNX file by generating a console statement through corresponding conversion configuration in a container, thereby finally completing the function of converting the tensor flow model into a high-performance reasoning model by using the mirror image. The user can use the converted high-performance reasoning model to train and effectively deploy the reasoning service, so that the effect of high-performance reasoning is achieved. According to the invention, the existing tensor flow model is converted into the high-performance reasoning model, so that the model training time of the user is effectively shortened, the reasoning efficiency of deploying the reasoning service is improved, and the user can easily use the existing model to perform high-efficiency reasoning work, thereby improving the use efficiency and experience of the user.
It can be seen that the present invention has outstanding substantial features and significant advances over the prior art, as well as the benefits of its implementation.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present invention, and that other drawings can be obtained according to the provided drawings without inventive effort for a person skilled in the art.
FIG. 1 is a functional schematic of an artificial intelligence platform in the background of the invention.
FIG. 2 is a method flow diagram of a method for transforming a tensor flow model in accordance with one embodiment of the present invention.
FIG. 3 is a system architecture diagram of a system for transforming a tensor flow model in accordance with one embodiment of the present invention.
Fig. 4 is a schematic structural diagram of a tensor flow model conversion device according to an embodiment of the present invention.
In the figure, 1, a request initiating unit; 2. a catalog export unit; 3. a conversion start unit; 4. an address conversion unit; 5. an instruction generation unit; 6. a first document detection unit; 7. a file conversion unit; 8. a second document detection unit; 9. an alarm unit; 101. a processor; 102. a memory; 103. an input interface; 104. an output interface; 105. a communication unit; 106. a keyboard; 107. a display; 108. and a mouse.
Detailed Description
The core of the invention is to provide a method for transforming a tensor flow model, in the related technology, a model with Tensorflow SavedModel format is usually imported by using a Tensorflow Serving framework model version at the tensor flow model deployment level for deployment, and the model deployment with relatively stable and high compatibility can be provided in the use process, but the improvement of the reasoning performance and efficiency of the model is relatively limited, and the effect of high-performance reasoning is difficult to achieve.
In the method for converting the tensor flow model, firstly, directory information of the tensor flow model is acquired through an artificial intelligent platform, and a model conversion request is sent out. And then, deriving directory information of the tensor stream model, converting address information of the model into mounting point addresses in the container according to the directory information of the tensor stream model, generating a console statement for converting the tensor stream model into ONNX files, and requesting the container on a corresponding server. At this time, whether the ONNX file exists is detected by using the directory information of the tensor flow model, and if so, the ONNX file is converted into a corresponding reasoning model file through a model optimization tool. And finally, confirming that the conversion of the reasoning model file is successful by using the directory information of the tensor flow model. Therefore, the invention can improve the reasoning efficiency of deploying the reasoning service, and the use efficiency and experience of the user by converting the tensor flow model into the high-performance reasoning model.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.
The following explains key terms appearing in the present invention.
The TensorFlow model, namely a Tensor Flow model, has the most important concepts of Tensor and Flow, the Tensor is Tensor, and the Flow is Flow, and together, the Tensor and the Flow intuitively express the process of mutual conversion through calculation. In the T TensorFlow model, all data is represented in the form of tensors, which are tools for TensorFlow to manage the data. Tensors can be simply understood as multidimensional arrays, with zero-order tensors representing scalar quantities, i.e., a number, first-order tensors representing vectors, i.e., one-dimensional arrays, and n-order tensors representing an n-dimensional array. Tensorflow has a multi-level structure, can be deployed on various servers, PC terminals and webpages, supports GPU and TPU high-performance numerical computation, and is widely applied to product development and scientific research in various fields.
TensorRT is a high-performance deep learning reasoning (information) optimization model, and can provide low-delay and high-throughput deployment reasoning for deep learning applications. The TensorRT can be used for reasoning and accelerating a very large-scale data center, an embedded platform or an automatic driving platform. TensorRT can now support almost all deep learning frameworks of TensorFlow, caffe, mxnet, pytorch, and by combining the GPUs of TensorRT and NVIDIA, quick and efficient deployment reasoning can be performed in almost all frameworks.
ONNX is an open format representing a deep learning model, and is a standard format independent of the environment and platform. Regardless of what training frameworks are used to train the models, the models of these frameworks can be unified into ONNX storage after training. The ONNX file stores not only the weight of the neural network model, but also the structural information of the model, input and output information of each layer in the network, and other information. The converted ONNX model is then converted to a type that we need to deploy using a different framework.
In order to better understand the aspects of the present invention, the present invention will be described in further detail with reference to the accompanying drawings and detailed description. It will be apparent that the described embodiments are only some, but not all, embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 2, the present embodiment provides a method for transforming a tensor flow model, including the following steps:
s1: and acquiring directory information of the Tensorflow model through the artificial intelligent platform, and sending a model conversion request.
In a specific embodiment, the TensorRT model conversion request can be initiated by a user importing an online catalog of Tensorflow models from the foreground of the artificial intelligence platform.
S2: and in response to receiving the model conversion request, exporting directory information of the Tensorflow model.
In a specific embodiment, after the rear end of the artificial intelligence platform receives the request, the directory information of the Tensorflow model is exported according to the request.
S3: and starting a model conversion flow.
Specifically, in the conversion mode selection list, the conversion mode is selected as a TensorRT model conversion, and the conversion state is set to be in progress.
It should be noted that, in theory, a plurality of transformation modes may be selected, such as the TensorRT model transformation, the OpenVINO model transformation. The method aims at realizing the transformation of the TensorRT model, so that other transformation modes can be ignored.
S4: and converting the address information of the model into the mounting point address in the container according to the directory information of the Tensorflow model.
Specifically, firstly, according to a primary key of a model conversion request, directory information of a corresponding Tensorflow model is taken out from a database; the import and export addresses of the Tensorflow model are then translated into mount point addresses in the container.
S5: and generating a console statement for converting the Tensorflow model into an ONNX file, and requesting a container on a corresponding server.
In the specific embodiment, firstly, according to the address of the mounting point in the container, the name of the tensorRT container under the current server configuration, the address of the server where the tensorRT container is located and the address of the mounted network file system are obtained by using notes, and the name, the address and the address of the mounted network file system are injected into corresponding attributes to generate a console statement; then, converting the Tensorflow model into an ONNX file through a console statement; finally, a request is made for the container on the corresponding server.
It should be noted that, since the container of the existing server does not have a software package for converting the Tensorflow into the ONNX file generation console statement, a corresponding conversion configuration needs to be additionally performed in the container. The specific implementation mode is as follows: and the name of the tensorRT container under the current server configuration, the server address of the tensorRT container and the address of the mounted network file system are obtained by using notes through the corresponding service at the back end according to the information in the configuration file in the container, and are injected into the corresponding attribute, so that a console statement for file conversion is generated.
S6: and detecting whether the ONNX file exists or not by using directory information of the Tensorflow model. If not, step S7 is performed. If so, step S8 is performed.
In the specific embodiment, firstly, acquiring a corresponding mounting point address by using directory information of a Tensorflow model; at the moment, whether an ONNX file is associated with the mounting point address is detected; if yes, indicating that the ONNX file exists; otherwise, the ONNX file is not existed.
As an example, when a container execution statement is requested, a console statement needs to be preceded by a list of "sh" and "-c", a User value is set to root, then a POST request is performed through a corresponding interface to obtain corresponding processing, an instruction code is generated, after that, another RESTful interface is accessed to enable a corresponding instruction, and "detail" is set to "false" to wait for completion of the instruction code. The process enables the back end to track the ending time of each process more completely, and can make correct judgment at the corresponding time point so as to realize the generation confirmation of the ONNX file.
S7: and directly exiting after reporting errors.
S8: the ONNX file is converted into a corresponding TensorRT.plan file by a trtexec tool.
In a specific embodiment, the ONNX file of the online model is converted into a tensorrt.plan file by a tretec tool provided by tensort authorities.
S9: and detecting whether the corresponding TensorRT.plan file exists or not by using directory information of the Tensorflow model. If not, step S10 is performed. If present, the conversion is complete.
In the specific embodiment, firstly, acquiring a corresponding mounting point address by using directory information of a Tensorflow model; then detecting whether a TensorRT.plan file is associated with the mounting point address; if yes, the TensorRT.plan file exists; otherwise, the TensorRT.plan file does not exist.
S10: and directly exiting after reporting errors.
The invention provides a conversion method of tensor flow model, which generates a console statement to convert a Tensorflow model into ONNX file by carrying out corresponding conversion configuration in a container, thereby finally completing the function of converting the Tensorflow model into a TensorRT high-performance reasoning model by using the mirror image. The user can use the converted TensorRT high-performance reasoning model to train and effectively deploy the reasoning service, so that the effect of high-performance reasoning is achieved.
Referring to fig. 3, the invention also discloses a conversion system of the tensor flow model, which comprises: a request initiating unit 1, a catalog deriving unit 2, a conversion initiating unit 3, an address converting unit 4, an instruction generating unit 5, a first file detecting unit 6, a file converting unit 7, a second file detecting unit 8 and an alarm unit 9.
The request initiating unit 1 is configured to acquire directory information of the Tensorflow model through the artificial intelligent platform and initiate a model conversion request.
In a specific embodiment, the request initiation unit 1 is specifically configured to: and initiating a TensorRT model conversion request by importing an online catalog of a Tensorflow model from a foreground of the artificial intelligent platform by a user.
And a catalog deriving unit 2 configured to derive catalog information of the Tensorflow model in response to receiving the model conversion request.
In a specific embodiment, the catalog derivation unit 2 is specifically configured to: and after the rear end of the artificial intelligent platform receives the request, exporting directory information of the Tensorflow model according to the request.
And a conversion starting unit 3 configured to start the model conversion flow.
In a specific embodiment, the conversion start-up unit 3 is specifically configured to: in the conversion pattern selection list, a conversion pattern is selected as a TensorRT model conversion, and a conversion state is set to be in progress.
And an address conversion unit 4 configured to convert the address information of the model into the mount point address in the container according to the directory information of the Tensorflow model.
In a specific embodiment, the address translation unit 4 is specifically configured to: according to the main key of the model conversion request, the catalog information of the corresponding Tensorflow model is taken out from the database; the import and export addresses of the Tensorflow model are translated into mount point addresses in the container.
An instruction generating unit 5 configured to generate a console statement for converting the Tensorflow model into an ONNX file, and to request a container on the corresponding server.
In a specific embodiment, the instruction generating unit 5 is specifically configured to: according to the address of the mounting point in the container, using notes to obtain the name of the tensor rRT container under the current server configuration, the address of the server where the tensor container is located and the address of the mounted network file system, and injecting the names, the address of the server address and the address of the mounted network file system into the corresponding attribute to generate a console statement; converting the Tensorflow model into an ONNX file through a console statement; finally, a request is made for the container on the corresponding server.
A first file detecting unit 6 configured to detect whether or not the ONNX file exists using directory information of the Tensorflow model.
In a specific embodiment, the first file detecting unit 6 is specifically configured to: acquiring a corresponding mounting point address by using directory information of the Tensorflow model; detecting whether an ONNX file is associated with the mounting point address; if yes, indicating that the ONNX file exists; otherwise, the ONNX file is not existed.
And a file conversion unit 7 configured to convert the ONNX file into a corresponding tensorrt. Plan file by a tretec tool when the first file detection unit detects the presence of the ONNX file.
In a specific embodiment, the file conversion unit 7 is specifically configured to: the ONNX file of the online model is converted into a tensorrt.plan file by a tretec tool provided by tensort authorities.
And a second file detecting unit 8 configured to detect whether the corresponding TensorRT.plan file exists using directory information of the Tensorflow model.
In a specific embodiment, the second file detecting unit 8 is specifically configured to: acquiring a corresponding mounting point address by using directory information of the Tensorflow model; detecting whether a TensorRT.plan file is associated with the mounting point address; if yes, the TensorRT.plan file exists; otherwise, the TensorRT.plan file does not exist.
And an alarm unit 9 configured to directly exit after reporting an error when the first file detecting unit does not detect the ONNX file or the second file detecting unit does not detect the tensort.
Therefore, the invention provides a conversion system of tensor flow model, which converts the existing tensor flow model into a tensor RT high-performance reasoning model, effectively reduces the training time of the model of the user, improves the reasoning efficiency of deploying the reasoning service, and enables the user to easily use the existing model to perform high-efficiency reasoning work so as to improve the use efficiency and experience of the user.
Referring to fig. 4, the invention also discloses a device for converting tensor flow model, which comprises a processor 101 and a memory 102; wherein the processor 101 implements the following steps when executing the conversion program of the tensor flow model stored in the memory:
1. directory information of the tensor flow model is obtained through the artificial intelligent platform, and a model conversion request is sent out.
2. In response to receiving the model conversion request, directory information for the tensor flow model is derived.
3. And starting a model conversion flow.
4. And converting the address information of the model into the mounting point address in the container according to the directory information of the tensor flow model.
5. And generating a console statement for converting the tensor flow model into an ONNX file, and requesting a container on a corresponding server.
6. Detecting whether ONNX files exist or not by using directory information of a tensor flow model; if not, directly exiting after reporting errors; if so, the ONNX file is converted into a corresponding reasoning model file through a model optimization tool.
7. Detecting whether a corresponding reasoning model file exists or not by using directory information of the tensor flow model; if not, directly exiting after reporting errors; if present, the conversion is complete.
The conversion device of the tensor flow model provided in this embodiment may include, but is not limited to, a smart phone, a tablet computer, a notebook computer, a desktop computer, or the like.
Processor 101 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 101 may be implemented in at least one hardware form of digital signal processing (Digital Signal Processor, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), programmable logic array (Programmable Logic Array, PLA). The processor 101 may also include a main processor and a coprocessor, the main processor being a processor for processing data in an awake state, also referred to as a central processor (Central Processing Unit, CPU); a coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 101 may be integrated with an image processor (Graphics Processing Unit, GPU) for use in connection with rendering and rendering of content to be displayed by the display screen. In some embodiments, the processor 101 may also include an artificial intelligence (Artificial Intelligence, AI) processor for processing computing operations related to machine learning.
Memory 102 may include one or more computer-readable storage media, which may be non-transitory. Memory 102 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In this embodiment, the memory 102 is at least configured to store a computer program, where the computer program, when loaded and executed by the processor 101, is capable of implementing the relevant steps of the tensor flow model conversion method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 102 may also include an operating system, data, and the like, and the storage manner may be transient storage or permanent storage. The operating system may include Windows, unix, linux, among others. The data may include, but is not limited to, data involved in the transformation method of the tensor flow model described above, and the like.
Further, the device for converting a tensor flow model in this embodiment may further include:
the input interface 103 is configured to obtain a conversion program of an externally imported tensor flow model, store the obtained conversion program of the tensor flow model in the memory 102, and further obtain various instructions and parameters transmitted by an external terminal device, and transmit the various instructions and parameters to the processor 101, so that the processor 101 uses the various instructions and parameters to develop corresponding processing. In this embodiment, the input interface 103 may specifically include, but is not limited to, a USB interface, a serial interface, a voice input interface, a fingerprint input interface, a hard disk reading interface, and the like.
And an output interface 104 for outputting various data generated by the processor 101 to a terminal device connected thereto, so that other terminal devices connected to the output interface can acquire various data generated by the processor 101. In this embodiment, the output interface 104 may specifically include, but is not limited to, a USB interface, a serial interface, and the like.
And the communication unit 105 is used for establishing a remote communication connection between the server operation business optimization configuration device and an external server so that the conversion device of the tensor flow model can mount the image file to the external server. In this embodiment, the communication unit 105 may specifically include, but is not limited to, a remote communication unit based on a wireless communication technology or a wired communication technology.
A keyboard 106 for acquiring various parameter data or instructions inputted by a user by tapping the key cap in real time.
A display 107 for displaying information related to the conversion process of the running tensor flow model in real time.
The mouse 108 may be used to assist the user in inputting data and to simplify the user's operation.
The invention also discloses a readable storage medium, which includes Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. A readable storage medium stores therein a conversion program of a tensor flow model, which when executed by a processor, performs the steps of:
1. directory information of the tensor flow model is obtained through the artificial intelligent platform, and a model conversion request is sent out.
2. In response to receiving the model conversion request, directory information for the tensor flow model is derived.
3. And starting a model conversion flow.
4. And converting the address information of the model into the mounting point address in the container according to the directory information of the tensor flow model.
5. And generating a console statement for converting the tensor flow model into an ONNX file, and requesting a container on a corresponding server.
6. Detecting whether ONNX files exist or not by using directory information of a tensor flow model; if not, directly exiting after reporting errors; if so, the ONNX file is converted into a corresponding reasoning model file through a model optimization tool.
7. Detecting whether a corresponding reasoning model file exists or not by using directory information of the tensor flow model; if not, directly exiting after reporting errors; if present, the conversion is complete.
In summary, the tensor flow model is converted into the high-performance reasoning model, so that the reasoning efficiency of deploying the reasoning service, and the use efficiency and experience of the user are effectively improved.
In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, so that the same or similar parts between the embodiments are referred to each other. For the method disclosed in the embodiment, since it corresponds to the system disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the several embodiments provided by the present invention, it should be understood that the disclosed systems, and methods may be implemented in other ways. For example, the system embodiments described above are merely illustrative, e.g., the division of the elements is merely a logical functional division, and there may be additional divisions when actually implemented, e.g., multiple elements or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some interface, system or unit indirect coupling or communication connection, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional module in the embodiments of the present invention may be integrated in one processing unit, or each module may exist alone physically, or two or more modules may be integrated in one unit.
Similarly, each processing unit in the embodiments of the present invention may be integrated in one functional module, or each processing unit may exist physically, or two or more processing units may be integrated in one functional module.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software modules may be disposed in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
Finally, it is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The method, system, device and readable storage medium for transforming tensor flow model provided by the invention are described above in detail. The principles and embodiments of the present invention have been described herein with reference to specific examples, the description of which is intended only to facilitate an understanding of the method of the present invention and its core ideas. It should be noted that it will be apparent to those skilled in the art that various modifications and adaptations of the invention can be made without departing from the principles of the invention and these modifications and adaptations are intended to be within the scope of the invention as defined in the following claims.
Claims (10)
1. A method of transforming a tensor flow model, comprising:
acquiring directory information of the tensor flow model through an artificial intelligent platform, and sending a model conversion request;
in response to receiving the model conversion request, deriving directory information for the tensor flow model;
starting a model conversion flow;
converting the address information of the model into a mounting point address in the container according to the directory information of the tensor flow model; generating a console statement for converting the tensor flow model into an ONNX file, and requesting a container on a corresponding server;
detecting whether ONNX files exist or not by using directory information of a tensor flow model;
if not, directly exiting after reporting errors; if so, converting the ONNX file into a corresponding reasoning model file through a model optimization tool;
detecting whether a corresponding reasoning model file exists or not by using directory information of the tensor flow model;
if not, directly exiting after reporting errors; if present, the conversion is complete.
2. The method for transforming a tensor flow model according to claim 1, wherein the starting the model transformation flow includes:
in the conversion mode selection list, a conversion mode is selected as the inference model conversion, and the conversion state is set to be in progress.
3. The method for converting a tensor flow model according to claim 1, wherein the converting address information of the model into the mount point address in the container according to the directory information of the tensor flow model comprises:
according to the main key of the model conversion request, the catalog information of the corresponding tensor flow model is taken out from the database;
the import and export addresses of the tensor flow model are translated into mount point addresses in the container.
4. A method of transforming a tensor flow model according to claim 3, wherein the generating a console statement to transform the tensor flow model into an ONNX file comprises:
according to the address of the mounting point in the container, using notes to obtain the container information under the current server configuration, and injecting the container information into the corresponding attribute to generate a console statement;
the sheet flow model is converted into an ONNX file by a console statement.
5. The method for converting a tensor flow model according to claim 1, wherein detecting whether the ONNX file exists using directory information of the tensor flow model comprises:
acquiring a corresponding mounting point address by using directory information of the tensor flow model;
detecting whether an ONNX file is associated with the mounting point address;
if yes, ONNX files exist; otherwise the ONNX file does not exist.
6. The method for transforming a tensor flow model according to claim 1, wherein the detecting whether the corresponding optimization model file exists using directory information of the tensor flow model comprises:
acquiring a corresponding mounting point address by using directory information of the tensor flow model;
detecting whether an inference model file is associated with the mounting point address or not;
if yes, the reasoning model file exists; otherwise, the reasoning model file does not exist.
7. The method of claim 4, wherein the container information comprises: the name of the container, the address of the server where it is located, the address of the network file system where it is mounted.
8. A system for transforming a tensor flow model, comprising:
the request initiating unit is configured to acquire directory information of the tensor flow model through the artificial intelligent platform and send a model conversion request;
a catalog deriving unit configured to derive catalog information of the tensor flow model in response to receiving the model conversion request;
the transformation starting unit is configured to start a model transformation flow;
the address conversion unit is configured to convert the address information of the model into the mounting point address in the container according to the directory information of the tensor flow model;
the instruction generating unit is configured to generate a console statement for converting the tensor flow model into an ONNX file and request a container on a corresponding server;
a first file detection unit configured to detect whether an ONNX file exists using directory information of a tensor flow model;
the file conversion unit is configured to convert the ONNX file into a corresponding reasoning model file through the model optimization tool when the first file detection unit detects that the ONNX file exists;
a second file detection unit configured to detect whether a corresponding inference model file exists using directory information of the tensor flow model;
and the alarm unit is configured to directly exit after reporting errors when the first file detection unit does not detect the ONNX file or the second file detection unit does not detect the reasoning model file.
9. A device for transforming a tensor flow model, comprising:
a memory for storing a conversion program of the tensor flow model;
processor for implementing the steps of the method of transforming a tensor flow model according to any one of claims 1 to 7 when executing the program of transforming the tensor flow model.
10. A readable storage medium, characterized by: the readable storage medium has stored thereon a conversion program of a tensor flow model, which when executed by a processor, implements the steps of the tensor flow model conversion method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310944731.0A CN117075915A (en) | 2023-07-28 | 2023-07-28 | Tensor flow model conversion method, system, device and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310944731.0A CN117075915A (en) | 2023-07-28 | 2023-07-28 | Tensor flow model conversion method, system, device and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117075915A true CN117075915A (en) | 2023-11-17 |
Family
ID=88710646
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310944731.0A Pending CN117075915A (en) | 2023-07-28 | 2023-07-28 | Tensor flow model conversion method, system, device and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117075915A (en) |
-
2023
- 2023-07-28 CN CN202310944731.0A patent/CN117075915A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109033068A (en) | It is used to read the method, apparatus understood and electronic equipment based on attention mechanism | |
| CN110781082A (en) | Method, device, medium and equipment for generating test case of interface | |
| CN112836064A (en) | Knowledge graph complementing method and device, storage medium and electronic equipment | |
| CN116775183A (en) | Task generation method, system, equipment and storage medium based on large language model | |
| CN109815448A (en) | Lantern slide generation method and device | |
| CN113468344B (en) | Entity relationship extraction method and device, electronic equipment and computer readable medium | |
| CN114491064A (en) | Internet of things platform construction method and device, storage medium and terminal | |
| CN111651989B (en) | Named entity recognition method and device, storage medium and electronic device | |
| CN110991279B (en) | Document Image Analysis and Recognition Method and System | |
| CN116932147A (en) | Streaming job processing method and device, electronic equipment and medium | |
| CN117075915A (en) | Tensor flow model conversion method, system, device and readable storage medium | |
| CN112612427B (en) | Vehicle stop data processing method and device, storage medium and terminal | |
| CN108304219A (en) | Secondary developing platform and method | |
| CN114650436B (en) | Remote control method, device, equipment and medium based on background service | |
| CN117235236B (en) | Dialogue method, dialogue device, computer equipment and storage medium | |
| CN109508183B (en) | REST code generation method and device in storage cluster | |
| CN117744920A (en) | Energy data management method and related device | |
| CN117931910A (en) | Data storage method, device, equipment and storage medium | |
| CN115858639A (en) | Method, system and storage medium for dynamically generating financial statement | |
| CN117744651A (en) | Method and device for extracting slot position information of language large model fusion NLU | |
| CN114610807A (en) | Data import template configuration method, device, equipment and storage medium | |
| CN117931176A (en) | Business application generation method, device, platform and medium | |
| CN117951723A (en) | Task data construction method and device, computing equipment and readable storage medium | |
| CN117311668A (en) | Method, device, equipment and storage medium for generating technical requirements of intelligent instrument | |
| CN116467035A (en) | Scene transmission method, device, medium and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |