CN115358400A - Application method of deep learning model interface - Google Patents
Application method of deep learning model interface Download PDFInfo
- Publication number
- CN115358400A CN115358400A CN202211019035.0A CN202211019035A CN115358400A CN 115358400 A CN115358400 A CN 115358400A CN 202211019035 A CN202211019035 A CN 202211019035A CN 115358400 A CN115358400 A CN 115358400A
- Authority
- CN
- China
- Prior art keywords
- interface
- model
- data
- training
- loss
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 78
- 238000013136 deep learning model Methods 0.000 title claims abstract description 46
- 238000009795 derivation Methods 0.000 claims abstract description 104
- 238000012545 processing Methods 0.000 claims abstract description 32
- 238000012549 training Methods 0.000 claims description 100
- 238000012360 testing method Methods 0.000 claims description 55
- 238000012795 verification Methods 0.000 claims description 31
- 230000006870 function Effects 0.000 claims description 27
- 230000008569 process Effects 0.000 claims description 22
- 238000004422 calculation algorithm Methods 0.000 claims description 20
- 238000013135 deep learning Methods 0.000 claims description 20
- 238000007781 pre-processing Methods 0.000 claims description 19
- 230000003044 adaptive effect Effects 0.000 claims description 10
- 238000012546 transfer Methods 0.000 claims description 9
- 238000004806 packaging method and process Methods 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000010606 normalization Methods 0.000 claims description 5
- 238000005457 optimization Methods 0.000 claims description 5
- 230000005540 biological transmission Effects 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 238000010200 validation analysis Methods 0.000 claims description 2
- 230000015654 memory Effects 0.000 description 12
- 238000011161 development Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 238000005538 encapsulation Methods 0.000 description 5
- 238000003062 neural network model Methods 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 5
- 230000008901 benefit Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 230000002093 peripheral effect Effects 0.000 description 3
- 230000000712 assembly Effects 0.000 description 2
- 238000000429 assembly Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/04—Inference or reasoning models
- G06N5/046—Forward inferencing; Production systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention provides a using method of a deep learning model interface, which comprises the following steps: providing an inference interface configured to provide an input interface of a pure tensor at model deployment to facilitate model derivation; providing a data processor, wherein the data processor is configured to convert input data into tensor after batch processing and transmit the tensor to a forward derivation interface so as to perform batch processing strategy and model decoupling; and providing a forward derivation interface configured to accept the data processor processed tensor, return loss or prediction results, to be able to be used directly for model deployment.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to a use method of a deep learning model interface.
Background
The deep learning algorithm library usually needs to define an interface for neural network model training; calculating loss in a training stage, calculating gradient based on the loss, and updating model parameters; calculating a prediction result or loss in a verification stage, wherein the prediction result can be further calculated to obtain an evaluation index, and the loss and the evaluation index can be used for adjusting the model hyperparameter; in the testing stage, a prediction result is calculated to measure the quality of model training; furthermore, the model usually needs to define an inference interface for inferring a single picture.
In order to be compatible with distributed training and model deployment, the deep learning algorithm library is used for additionally packaging the neural network model, the process is complex, and the learning and development cost is high.
Disclosure of Invention
The invention aims to provide a using method of a deep learning model interface, which aims to solve the problem that the development workload is large because the packaging of a neural network model is complicated by the introduction of a deep learning algorithm library.
In order to solve the above technical problem, the present invention provides a method for using a deep learning model interface, including:
providing an inference interface configured to provide an input interface of a pure tensor at model deployment to facilitate model derivation;
providing a data processor, wherein the data processor is configured to convert input data into tensor after batch processing and transmit the tensor to a forward derivation interface so as to perform batch processing strategy and model decoupling; and
a forward derivation interface is provided that is configured to accept the data processor processed tensor, return loss or prediction results, to enable optimization and validation of the model during training. The inference interface is a function of the forward inference interface, or a branch.
Optionally, in the method for using the deep learning model interface,
forming a model interface which is friendly to deploy and is adaptive to a plurality of deep learning back-end distributed trainings by appointing a forward derivation interface, dividing boundaries of a data processor and the forward derivation interface;
respectively extracting uniform and universal interfaces for training, reasoning and testing of the model;
the algorithm of any deep learning back end adopts the interface convention, and complex packaging is not needed for distributed training.
Optionally, in the method for using the deep learning model interface, the deep learning model performs a training process, an inference process and a testing process, and provides a uniform interface for the deep learning model;
wherein the training process comprises: performing data enhancement in a training stage on data input into the model by using a data processor, transmitting the enhanced data to a training stepping interface of the model, further calling a forward derivation interface of the model by the training stepping interface, returning to a loss dictionary, optimizing model weight based on the loss dictionary, and returning to the loss dictionary;
the verification process comprises the following steps: using a data processor to carry out data enhancement of a test stage on data input into the model, and transmitting the enhanced data to a verification stepping interface of the model; the verification stepping interface further calls a forward derivation interface of the model, and returns a loss dictionary or a prediction result according to the transmitted parameters;
the test process comprises the following steps: processing the data of the input model by using a data processor to enhance the data in the testing stage, and transmitting the enhanced data to a testing stepping interface of the model; and the test stepping interface further calls a forward derivation interface of the model and returns a prediction result.
Optionally, in the method for using the deep learning model interface,
forming an independent class by using a data processor as a component of a model, and preprocessing data input into the model;
the user reloads the customized data processor as required, so that the data processor has the following functions:
the data processor is used for building the model and can be freely combined with the model; and
a forward derivation interface of the model directly accepts tensor input and returns a reasoning result, so that deployment or direct calling is facilitated;
wherein preprocessing data input to the model comprises:
transporting the data from the CPU to a designated device;
carrying out batch processing, wherein the batch processing comprises normalization, scaling and confusion enhancement; and
and performing data format conversion to convert the input data into tensor.
Optionally, in the method for using the deep learning model interface, the interface of the data processor includes:
a transfer data interface cast _ data, the interface function being introduced to transfer data to a specified device; the interface parameter list comprises data of an input model; the return value is data, and the data transferred to the target equipment has the same structure and input;
a forward derivation interface forward, wherein the interface function is introduced into a format required by the data processing into a model; the interface parameter list comprises data processed by the transfer data interface and is used for judging whether data enhancement in a training stage or training enhancement in a testing stage is adopted; the return value is preprocessed data, and then the preprocessed data is analyzed and transmitted to a forward derivation interface of the model.
Optionally, in the method for using the deep learning model interface,
extracting a training stepping interface, a verification stepping interface and a test stepping interface for the model so as to be compatible with a plurality of single-card and distributed training logics at the training rear end and meet different requirements of the training, verification and test stages;
the forward derivation method is called by the training step, the verification step and the test step respectively, receives the result processed by the data processor, and returns the loss or reasoning result. The forward derivation is an abstract method, and subclasses must be implemented to carry the body logic of the model.
Optionally, in the method for using the deep learning model interface, the inference interface includes:
forward derivation interface forward, interface function is described as calculating loss or prediction result; the list of interface parameters includes: receiving a parameter args returned by forward derivation of the data processor, and returning a loss or prediction result mode according to the value of the mode;
training a stepping interface train _ step, wherein the interface function description comprises the flow of forward derivation, gradient calculation, parameter updating and gradient zero clearing sequentially performed by a model base class, and subclasses are reloaded as required to realize more complex logic; the list of interface parameters includes: inputting data of a model and an optimizer; the return values include: loss dictionary loss, the gradient calculation process is completed at this time, and the loss dictionary is only used for recording logs;
verifying a stepping interface val _ step, wherein the interface function description comprises forward derivation of a model base class, and return of a loss or prediction result; the list of interface parameters includes: data of the input model, return loss judgment result return _ loss: if the prediction result is True, returning the loss dictionary, otherwise, returning the prediction result; the return values include loss dictionaries or prediction results loss | predictions;
and testing the stepping interface test _ step, wherein the interface function description comprises a model base class to realize forward derivation and return a prediction result, the interface parameter list comprises data of an input model, and the return value comprises prediction results.
Optionally, in the method for using the deep learning model interface, the training steps are sequentially performed as follows:
calling a data processor to preprocess input data and returning data of forward derivation of the adaptive model;
performing forward derivation of the model and returning loss;
training stepping directly calls a forward derivation interface of distributed data parallel to utilize the efficient gradient synchronization function of the forward derivation interface;
the training of the model base class realizes forward derivation, gradient reverse transmission, parameter updating and gradient logic under the condition of a single optimizer step by step so as to meet the task requirement;
and generating a model requiring multiple optimizer tasks to overload training steps as required, and realizing self-defined parameter optimization logic.
Optionally, in the method for using the deep learning model interface, the verification steps are sequentially performed:
calling a data processor to preprocess input data and returning data of forward derivation of the adaptive model;
performing forward derivation of the model, and returning loss or predicting a result according to the parameter return _ loss;
as cross-process synchronization is not needed in the verification stage, single-card training and distributed training are not distinguished, and a unified flow is used;
the model base class implements common inference logic, i.e., after data preprocessing, the forward derivation interface is directly invoked.
Optionally, in the method for using the deep learning model interface,
the testing stepping is consistent with the verification stepping flow, and the difference is that the testing stepping has no requirement of return loss, and only a prediction result is returned;
the forward derivation is an abstract method, model parameters are received, and logic for calculating loss and a prediction result is executed according to the values of the model in the forward derivation stage.
The inventor of the invention discovers through research that in order to be compatible with distributed training and model deployment, a deep learning algorithm library secondarily encapsulates a neural network model, abstracts an interface which is the same name as a model interface, and introduces the following problems:
firstly, during model reasoning and deployment, tensor (Tensor) type input is generally required to be received, and during model training, verification and testing, non-Tensor data are generally received and preprocessed, and the model reasoning and the model are not unified in flow. The existing deep learning algorithm library usually solves the problem by performing additional packaging on the model when the model is derived, but the development workload is increased.
Secondly, in general, the model as the core of the deep learning algorithm should be responsible for forward derivation, gradient back propagation and parameter updating. However, under distributed training, the deep learning training backend usually has some conventions. Taking the Pytorch as an example, in order to use the high-performance gradient synchronization function, a forward derivation interface of the distributed model encapsulation must be called, and gradient back transmission is not allowed in the forward derivation process, so the deep learning algorithm library usually performs complex encapsulation on the model to adapt to the distributed training of the corresponding back end.
Finally, deep learning models are also generally responsible for batch processing of data, such as normalization, scaling, confusion enhancement (MixUp), and the like, and deep learning algorithm libraries generally abstract this part of the logic into one method of the model, but this results in coupling of algorithms and batch processing strategies, and thus any combination of batch processing strategies and algorithm models cannot be realized.
Based on the above insights, the invention provides a use method of a deep learning model interface, which realizes the interface of an appointed model and the boundary of the division of the model function by respectively arranging an inference interface, a data processor and a forward derivation interface, so as to provide a set of model interfaces which are friendly to deployment and are adaptive to multi-back-end distributed training.
The invention provides a set of model training, verifying, testing and reasoning interfaces, provides a reasoning interface with pure tensor input for model deployment, and is convenient for model derivation; lightweight model encapsulation is provided for adapting to distributed conventions of different training back-ends.
The unified deep learning model interface is a set of appointments which are friendly to deployment and distributed training and have flexible matching of batch processing strategies and models. The current deep learning algorithm warehouse often appears: only single card training is supported; distributed training is supported, and single-card training is not supported; the model structure is not suitable for deployment and is difficult to derive, and if the algorithm is developed, the problem can be well solved according to the interface convention provided by the patent, so that the efficiency of a researcher for reproducing and following the corresponding algorithm is improved; the model is more convenient to derive, the workload of model deployment is reduced, and the landing efficiency of the model is improved.
Drawings
FIG. 1 is a diagram illustrating an example interface convention for a deep learning model interface model according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a deep learning model interface single-card/distributed training stepping workflow according to an embodiment of the present invention;
FIG. 3 is a schematic diagram illustrating a step-by-step workflow for interface verification of a deep learning model according to an embodiment of the present invention;
FIG. 4 is a schematic diagram illustrating a step-by-step workflow for deep learning model interface testing according to an embodiment of the present invention;
FIG. 5 is a schematic diagram illustrating the forward derivation work flow of the deep learning model interface according to an embodiment of the present invention;
FIG. 6 is a computer system in accordance with the present systems and/or methods.
Detailed Description
The invention is further elucidated with reference to the following description, in conjunction with the detailed description, and with reference to the accompanying drawings.
It should be noted that the components in the figures may be exaggerated and not necessarily to scale for illustrative purposes. In the figures, identical or functionally identical components are provided with the same reference symbols.
In the present invention, "disposed on …", "disposed above …" and "disposed above …" do not exclude the presence of an intermediate therebetween, unless otherwise specified. Further, "disposed on or above …" merely indicates the relative positional relationship between two members, and may also be converted to "disposed under or below …" or vice versa in certain cases, such as after reversing the product direction.
In the present invention, the embodiments are only intended to illustrate the aspects of the present invention, and should not be construed as limiting.
In the present invention, the terms "a" and "an" do not exclude the presence of a plurality of elements, unless otherwise specified.
It is further noted herein that in embodiments of the present invention, only a portion of the components or assemblies may be shown for clarity and simplicity, but those of ordinary skill in the art will appreciate that, given the teachings of the present invention, required components or assemblies may be added as needed in a particular scenario. Furthermore, features from different embodiments of the invention may be combined with each other, unless otherwise indicated. For example, a feature of the second embodiment may be substituted for a corresponding or functionally equivalent or similar feature of the first embodiment, and the resulting embodiments are likewise within the scope of the disclosure or recitation of the present application.
It is also noted herein that, within the scope of the present invention, the terms "same", "equal", and the like do not mean that the two values are absolutely equal, but allow some reasonable error, that is, the terms also encompass "substantially the same", "substantially equal". By analogy, in the present invention, the terms "perpendicular", "parallel" and the like in the directions of the tables also cover the meanings of "substantially perpendicular", "substantially parallel".
The numbering of the steps of the methods of the present invention does not limit the order of execution of the steps of the methods. Unless specifically stated, the method steps may be performed in a different order.
The method for using the deep learning model interface provided by the invention is further described in detail below with reference to the accompanying drawings and specific embodiments. The advantages and features of the present invention will become more apparent from the following description. It is to be noted that the drawings are in a very simplified form and are not to precise scale, which is merely for the purpose of facilitating and distinctly claiming the embodiments of the present invention.
The invention aims to provide a using method of a deep learning model interface, which aims to solve the problem that the development workload is large because the encapsulation of a neural network model by the existing deep learning algorithm library is complex.
In order to achieve the above object, the present invention provides a method for using a deep learning model interface, comprising: providing an inference interface configured to provide an input interface of a pure tensor at model deployment to facilitate model derivation; providing a data processor, wherein the data processor is configured to convert input data after batch processing into tensor and transmit the tensor to a forward derivation interface so as to decouple batch processing strategies from a model; and providing a forward derivation interface configured to accept the data processor processed tensor, return loss or prediction results, to be directly usable for model deployment.
Fig. 1-6 provide a first embodiment of the present invention, which shows a deep learning model interface schematic.
The invention designs a set of model training, verifying, testing and reasoning interfaces, and has the following beneficial effects: an inference interface of pure tensor input is provided for model deployment, and model derivation is facilitated; adapting to different deep learning back ends; and decoupling the batch processing strategy and the model to realize free combination of the batch processing strategy and the model.
The invention provides a set of model interfaces which are friendly to deployment and are adaptive to multi-back-end distributed training by appointing the interfaces of the model and dividing the boundaries of the functions of the model; first, the boundary dividing the data pre-processing and model forward derivation includes: data preprocessing is abstracted into a data processor, and decoupling of processing, enhancing strategies and models is realized; the data processor is responsible for converting input data into tensor after batch processing, transmitting the tensor to the forward derivation interface, and the forward derivation interface receives the tensor processed by the data processor and returns a loss or prediction result. The forward derivation interface can be used directly for model deployment.
Secondly, a uniform and universal interface is abstracted for the training, reasoning and testing of the model. The algorithm of any deep learning back end can adopt the interface convention, and distributed training can be realized without complex packaging.
In addition, training stepping, verification stepping, test stepping and forward derivation methods are abstracted for the model and are used for calling in the training, verification and test stages. The forward derivation method is also used to derive the model during the deployment phase.
And finally, abstracting the concept of data preprocessing into a data processor, realizing the decoupling of preprocessing and model main body logic, facilitating model deployment and freely combining a model and a batch processing strategy.
The invention defines the deep learning model training, reasoning and testing processes and provides a uniform interface for the deep learning model training, reasoning and testing processes.
The training process comprises the following steps: using a data processor to perform data enhancement in a training stage on data input into the model, and transmitting the enhanced data to a training stepping interface of the model; the training step interface further calls a forward derivation interface of the model to return to the loss dictionary; model weights are optimized based on the loss dictionary and returned to the loss dictionary.
The verification process comprises the following steps: using a data processor to carry out data enhancement of a test stage on data input into the model, and transmitting the enhanced data to a verification stepping interface of the model; and the verification stepping interface further calls a forward derivation interface of the model and returns a loss dictionary or a prediction result according to the transmitted parameters.
The test flow comprises the following steps: processing the data of the input model by using a data processor to enhance the data in the testing stage, and transmitting the enhanced data to a testing stepping interface of the model; and the testing stepping interface further calls a forward derivation interface of the model and returns a prediction result.
In the data processor of this embodiment, the model may perform the following pre-processing on the input data: data is transported from the cpu to a designated facility, batch processing such as normalization, scaling, confusion enhancement (mixup), etc., data format conversion, and input data is converted to a tensor.
The existing deep learning algorithm library usually abstracts the steps into a method of a model and then calls the method in the forward derivation process; however, this introduces some problems: the model preprocessing mode is bound with the model, so that the model preprocessing mode is inconvenient to expand, and a user cannot customize the model preprocessing mode; it is generally undesirable for the forward derivation interface to involve preprocessing operations, such as normalization, in model derivation.
The invention provides a concept of a data processor, wherein the data processor is used as a component of a model, is an independent class and is responsible for preprocessing data input into the model: the user can reload the self-defined data processor as required; the use of a data processor has the following advantages: the data processor is used for building the model and can be freely combined and configured; a forward derivation interface (forward) of the model can directly accept tensor input and return an inference result, and is convenient to deploy or directly call.
The interface of the data processor comprises: a transfer data interface cast _ data, the interface function being introduced to transfer data to a specified device; the interface parameter list comprises data of an input model; the return value is data, and the data transferred to the target equipment has the same structure and input; and a forward derivation interface forward, the interface function is introduced into the format required by the data processing model; the interface parameter list comprises data processed by the transfer data interface and is used for judging whether data enhancement in a training stage or training enhancement in a testing stage is adopted; the return value is preprocessed data, and then the preprocessed data is analyzed and transmitted to a forward derivation interface of the model.
In the model training/reasoning/testing interface of the embodiment, in order to be compatible with single-card and distributed training logics of different training back ends, three interfaces of training stepping, verification stepping and testing stepping are abstracted for a model so as to meet different requirements of training, verification and testing stages. The forward derivation method is called by the training step, the verification step and the test step respectively, receives the result processed by the data processor, and returns the loss or reasoning result. The forward derivation is an abstract method, and subclasses must be implemented to carry the body logic of the model.
The model training/reasoning/testing interface of the present embodiment includes: forward derivation interface forward, interface function is described as calculating loss or predicted result; the list of interface parameters includes: receiving a parameter args returned by forward derivation of the data processor, and returning a loss or prediction result mode according to the value of the mode; training a stepping interface train _ step, wherein the interface function description comprises the flow of forward derivation, gradient calculation, parameter updating and gradient zero clearing of a model base class in sequence, and subclasses are overloaded as required to realize more complex logic; the list of interface parameters includes: inputting data of a model and an optimizer; the return values include: loss dictionary loss, the gradient calculation process is completed at this time, and the loss dictionary is only used for recording logs; verifying a stepping interface val _ step, wherein the interface function description comprises forward derivation realized by a model base class and return loss or a prediction result; the list of interface parameters includes: data of the input model, return loss judgment result return _ loss: if the prediction result is True, returning the loss dictionary, otherwise, returning the prediction result; the return values include loss dictionaries or prediction results loss | predictions; and testing a stepping interface test _ step, wherein the interface function description comprises a model base class to realize forward derivation and return a prediction result, the interface parameter list comprises data of an input model, and the return value comprises prediction results predictions.
As shown in fig. 2, the training steps are performed in sequence: calling a data processor to preprocess input data (data) and returning data of forward derivation of the adaptive model; performing forward derivation of the model and returning loss; some training back-ends need to encapsulate the model under the distributed condition of performing gradient back-propagation and parameter updating based on the loss obtained by forward derivation and the introduced optimizer. Taking Pythrch as an example, opening distributed training usually requires parallel packaging of the model by distributed data. In this case, the encapsulation class needs to be reloaded, provided with the train _ step interface, and implementing the logic shown in the above figure. Unlike single-card training, the training steps directly invoke the distributed data-parallel forward derivation interface to take advantage of its efficient gradient synchronization functionality.
The model base class training step realizes common forward derivation, gradient back transmission, parameter updating and gradient logic under the condition of a single optimizer so as to meet the requirements of most tasks. Models such as the countermeasure network and the like which need multiple optimizer tasks can be generated to carry out heavy training stepping as required, and custom parameter optimization logic is realized.
As shown in fig. 3, the verification steps are performed in sequence: calling a data processor to preprocess input data and returning data of forward derivation of the adaptive model; and performing forward derivation of the model, and returning loss or predicting a result according to the parameter return _ loss. As the cross-process synchronization is not needed in the verification stage, the single-card training and the distributed training are not distinguished, and a unified flow is used. The model base class realizes common reasoning logic, namely, after data preprocessing, a forward derivation interface is directly called.
As shown in fig. 4, the test _ step flow is substantially identical to val step, and only the prediction result is returned in the case of a demand having no return loss from the test _ step.
As shown in FIG. 5, forward is an abstract method, and requires accepting mode parameters, and the forward derivation stage performs logic for calculating loss and prediction results according to the mode values.
The invention provides a set of unified and universal model interfaces which are friendly to deployment and distributed training for a deep learning algorithm library; and the user derives the subclass of the self-defined implementation based on the model base class and opens the subclass according to the interface specification. The model realized in this way is more convenient to deploy and is compatible with distributed training of different back ends. The deep learning model interface of the invention can be based on different back-end and different types of deep learning algorithms.
The application method of the deep learning model interface obtained by the technical scheme of the invention can be used for realizing the following technical effects in the fields of neural networks, computers, communication and the like: the distributed training system is more convenient to deploy and compatible with different back ends, and development efficiency is improved. The principle is that a set of unified and universal model interfaces friendly to deployment and distributed training is provided; and the user derives the subclass of the self-defined implementation based on the model base class and opens the subclass according to the interface specification.
FIG. 6 illustrates a computer system 100 implementing systems and/or methods in accordance with the present invention. Unless specifically stated otherwise, the method and/or system according to the present invention may be implemented in the computer system 100 shown in FIG. 6 for the purposes of the present invention, or the invention may be implemented in a distributed manner over a network, such as a local area network or the Internet, among a plurality of computer systems 100 according to the present invention. Computer system 100 of the present invention may comprise various types of computer systems, such as hand-held devices, laptop computers, personal Digital Assistants (PDAs), multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, network servers, tablet computers, and the like.
As shown in FIG. 6, computer system 100 includes a processor 111, a system bus 101, a system memory 102, a video adapter 105, an audio adapter 107, a hard drive interface 109, an optical drive interface 113, a network interface 114, and a Universal Serial Bus (USB) interface 112. The system bus 101 may be any of several types of bus structures such as a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. The system bus 101 is used for communication between the respective bus devices. In addition to the bus devices or interfaces shown in fig. 6, other bus devices or interfaces are also contemplated. The system memory 102 includes a Read Only Memory (ROM) 103 and a Random Access Memory (RAM) 104, where the ROM 103 may store, for example, basic input/output system (BIOS) data of basic routines for implementing information transfer at start-up, and the RAM 104 is used to provide a fast-access operating memory for the system. The computer system 100 further includes a hard disk drive 109 for reading from and writing to a hard disk 110, an optical drive interface 113 for reading from or writing to optical media such as a CD-ROM, and the like. Hard disk 110 may store, for example, an operating system and application programs. The drives and their associated computer-readable media provide nonvolatile storage of computer-readable instructions, data structures, program modules and other data for the computer system 100. Computer system 100 may also include a video adapter 105 for image processing and/or image output for connecting an output device such as a display 106. Computer system 100 may also include an audio adapter 107 for audio processing and/or audio output, for connecting output devices such as speakers 108. In addition, the computer system 100 may also include a network interface 114 for network connections, where the network interface 114 may connect to the Internet 116 through a network device, such as a router 115, where the connection may be wired or wireless. In addition, computer system 100 may also include a universal serial bus interface (USB) 112 for connecting peripheral devices, including, for example, a keyboard 117, a mouse 118, and other peripheral devices, such as a microphone, a camera, and the like.
When the present invention is implemented on the computer system 100 shown in fig. 6, a set of model interfaces that are friendly to deployment and are adapted to multi-backend distributed training may be provided by agreeing on the interfaces of the model and dividing the boundaries of the model functions; first, the boundary dividing the data pre-processing and model forward derivation includes: data preprocessing is abstracted into a data processor, and decoupling of processing, enhancing strategies and models is realized; the data processor is responsible for converting input data into tensor after batch processing, transmitting the tensor to the forward derivation interface, and the forward derivation interface receives the tensor processed by the data processor and returns a loss or prediction result. The forward derivation interface can be used directly for model deployment.
Furthermore, embodiments may be provided as a computer program product that may include one or more machine-readable media having stored thereon machine-executable instructions that, when executed by one or more machines such as a computer, network of computers, or other electronic devices, may result in the one or more machines performing operations according to embodiments of the present invention. The machine-readable medium may include, but is not limited to, floppy diskettes, optical disks, CD-ROMs (compact disc read-only memories), and magneto-optical disks, ROMs (read-only memories), RAMs (random access memories), EPROMs (erasable programmable read-only memories), EEPROMs (electrically erasable programmable read-only memories), magnetic or optical cards, flash memory, or other type of media/machine-readable medium suitable for storing machine-executable instructions.
Moreover, embodiments may be downloaded as a computer program product, wherein the program may be transferred from a remote computer (e.g., a server) to a requesting computer (e.g., a client) by way of one or more data signals embodied in and/or modulated by a carrier wave or other propagation medium via a communication link (e.g., a modem and/or network connection). Thus, a machine-readable medium as used herein may include, but is not necessarily required to be, such a carrier wave.
In summary, the above embodiments describe in detail different configurations of the method for using the deep learning model interface, and it is needless to say that the present invention includes, but is not limited to, the configurations listed in the above embodiments, and any content that is transformed based on the configurations provided by the above embodiments falls within the scope of protection of the present invention. One skilled in the art can take the contents of the above embodiments to take a counter-measure.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the system disclosed by the embodiment, the description is relatively simple because the system corresponds to the method disclosed by the embodiment, and the relevant points can be referred to the method part for description.
The above description is only for the purpose of describing the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention, and any variations and modifications made by those skilled in the art based on the above disclosure are intended to fall within the scope of the appended claims.
Claims (10)
1. A method for using a deep learning model interface, comprising:
providing an inference interface configured to provide an input interface of a pure tensor at model deployment to facilitate model derivation;
providing a data processor, wherein the data processor is configured to convert input data into tensor after batch processing and transmit the tensor to a forward derivation interface so as to perform batch processing strategy and model decoupling; and
a forward derivation interface is provided that is configured to accept the data processor processed tensor, return loss or prediction results, to enable optimization and validation of the model during training.
2. The method of using a deep learning model interface of claim 1,
forming a model interface which is friendly to deploy and is adaptive to a plurality of deep learning back-end distributed trainings by appointing a forward derivation interface, dividing boundaries of a data processor and the forward derivation interface;
respectively extracting uniform and universal interfaces for the training, reasoning and testing of the model;
the algorithm of any deep learning back end adopts the interface convention, and complex packaging is not needed for distributed training.
3. The method of using a deep learning model interface of claim 2,
the deep learning model carries out a training process, an inference process and a testing process and provides a uniform interface for the deep learning model;
wherein the training process comprises: performing data enhancement in a training stage on data input into the model by using a data processor, transmitting the enhanced data to a training stepping interface of the model, further calling a forward derivation interface of the model by the training stepping interface, returning to a loss dictionary, optimizing model weight based on the loss dictionary, and returning to the loss dictionary;
the verification process comprises the following steps: performing data enhancement of a testing stage on data input into the model by using a data processor, and transmitting the enhanced data to a verification stepping interface of the model; the verification stepping interface further calls a forward derivation interface of the model, and returns a loss dictionary or a prediction result according to the transmitted parameters;
the test process comprises the following steps: processing the data of the input model by using a data processor to enhance the data in the testing stage, and transmitting the enhanced data to a testing stepping interface of the model; and the test stepping interface further calls a forward derivation interface of the model and returns a prediction result.
4. The method of using a deep learning model interface as claimed in claim 3,
forming an independent class by using a data processor as a component of a model, and preprocessing data input into the model;
the user reloads the custom data processor as required, so that the data processor has the following functions:
the data processor is used for building the model and can be freely combined with the model; and
a forward derivation interface of the model directly accepts tensor input and returns a reasoning result, so that deployment or direct calling is facilitated;
wherein preprocessing data input to the model comprises:
transporting the data from the CPU to a designated device;
carrying out batch processing, wherein the batch processing comprises normalization, scaling and confusion enhancement; and
and performing data format conversion to convert the input data into tensor.
5. The method of using a deep learning model interface as claimed in claim 4, wherein the interface of the data processor comprises:
transferring a data interface east _ data, wherein the interface function is introduced to transfer the data to a specified device; the interface parameter list comprises data of an input model; the return value is data, and the data transferred to the target equipment has the same structure and input;
a forward derivation interface forward, wherein the interface function is introduced into a format required by the data processing into a model; the interface parameter list comprises data processed by the transfer data interface and is used for judging whether data enhancement in a training stage or training enhancement training in a testing stage is adopted; the return value is preprocessed data, and then the preprocessed data is analyzed and transmitted to a forward derivation interface of the model.
6. The method of using a deep learning model interface of claim 5,
extracting a training stepping interface, a verification stepping interface and a test stepping interface for the model so as to be compatible with a plurality of single-card and distributed training logics at the training rear end and meet different requirements of the training, verification and test stages;
the forward derivation method is called by the training step, the verification step and the test step respectively, receives the result processed by the data processor, and returns the loss or reasoning result. The forward derivation is an abstract method, and subclasses must be implemented to carry the body logic of the model.
7. The method of using a deep learning model interface as claimed in claim 6, wherein the inference interface comprises:
forward derivation interface forward, interface function is described as calculating loss or prediction result; the list of interface parameters includes: receiving a parameter args returned by forward derivation of the data processor, and returning a loss or prediction result mode according to the value of the mode;
training a stepping interface train _ step, wherein the interface function description comprises the flow of forward derivation, gradient calculation, parameter updating and gradient zero clearing of a model base class in sequence, and subclasses are overloaded as required to realize more complex logic; the list of interface parameters includes: inputting data of a model and an optimizer; the return values include: loss dictionary loss, the gradient calculation process is completed at this time, and the loss dictionary is only used for recording logs;
verifying a stepping interface val _ step, wherein the interface function description comprises forward derivation realized by a model base class and return loss or a prediction result; the list of interface parameters includes: data of the input model, return loss judgment result return _ loss: if the prediction result is True, returning to the loss dictionary, otherwise, returning to the prediction result; the return values include loss dictionaries or prediction results loss | predictions;
and testing the stepping interface test _ step, wherein the interface function description comprises a model base class to realize forward derivation and return a prediction result, the interface parameter list comprises data of an input model, and the return value comprises prediction results.
8. The method of using a deep learning model interface as claimed in claim 7, wherein the training steps are performed in sequence by:
calling a data processor to preprocess input data and returning data of forward derivation of the adaptive model;
performing forward derivation of the model and returning loss;
training stepping directly calls a forward derivation interface of distributed data parallel to utilize the efficient gradient synchronization function of the forward derivation interface;
the training of the model base class realizes forward derivation, gradient reverse transmission, parameter updating and gradient logic under the condition of a single optimizer step by step so as to meet the task requirement;
and generating a model requiring multiple optimizer tasks to overload training steps as required, and realizing self-defined parameter optimization logic.
9. The method of using a deep learning model interface as claimed in claim 8, wherein the verification step is performed in sequence by:
calling a data processor to preprocess input data and returning data of forward derivation of the adaptive model;
performing forward derivation of the model, and returning loss or predicting a result according to the parameter return _ loss;
as cross-process synchronization is not needed in the verification stage, single-card training and distributed training are not distinguished, and a unified flow is used;
the model base class implements common inference logic, i.e., after data preprocessing, the forward derivation interface is directly invoked.
10. The method of using a deep learning model interface of claim 9,
the testing stepping is consistent with the verification stepping flow, and the difference is that the testing stepping has no requirement of return loss, and only a prediction result is returned;
the forward derivation is an abstract method, model parameters are received, and logic for calculating loss and a prediction result is executed according to the values of the model in the forward derivation stage.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211019035.0A CN115358400A (en) | 2022-08-24 | 2022-08-24 | Application method of deep learning model interface |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211019035.0A CN115358400A (en) | 2022-08-24 | 2022-08-24 | Application method of deep learning model interface |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115358400A true CN115358400A (en) | 2022-11-18 |
Family
ID=84003671
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211019035.0A Pending CN115358400A (en) | 2022-08-24 | 2022-08-24 | Application method of deep learning model interface |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115358400A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111461332A (en) * | 2020-03-24 | 2020-07-28 | 北京五八信息技术有限公司 | Deep learning model online reasoning method and device, electronic equipment and storage medium |
| CN112819153A (en) * | 2020-12-31 | 2021-05-18 | 杭州海康威视数字技术股份有限公司 | Model transformation method and device |
| WO2021184346A1 (en) * | 2020-03-20 | 2021-09-23 | 云图技术有限公司 | Private machine learning model generation and training methods, apparatus, and electronic device |
-
2022
- 2022-08-24 CN CN202211019035.0A patent/CN115358400A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021184346A1 (en) * | 2020-03-20 | 2021-09-23 | 云图技术有限公司 | Private machine learning model generation and training methods, apparatus, and electronic device |
| CN111461332A (en) * | 2020-03-24 | 2020-07-28 | 北京五八信息技术有限公司 | Deep learning model online reasoning method and device, electronic equipment and storage medium |
| CN112819153A (en) * | 2020-12-31 | 2021-05-18 | 杭州海康威视数字技术股份有限公司 | Model transformation method and device |
Non-Patent Citations (1)
| Title |
|---|
| 周立君 等: "使用TensorRT进行深度学习推理", 应用光学, no. 2, 15 March 2020 (2020-03-15), pages 111 - 115 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zhou et al. | Edge intelligence: Paving the last mile of artificial intelligence with edge computing | |
| US11740941B2 (en) | Method of accelerating execution of machine learning based application tasks in a computing device | |
| CN112883149B (en) | Natural language processing method and device | |
| CN111666416B (en) | Method and device for generating semantic matching model | |
| CA3129720A1 (en) | Method for serving parameter efficient nlp models through adaptive architectures | |
| KR20200068050A (en) | Apparatus and method for generating learning data for artificial intelligence performance | |
| KR102420661B1 (en) | Data processing method and apparatus for neural network | |
| CN112418427A (en) | Method, device, system and equipment for providing deep learning unified reasoning service | |
| Huang et al. | A lightweight collaborative recognition system with binary convolutional neural network for mobile web augmented reality | |
| CN112036954A (en) | Item recommendation method and device, computer-readable storage medium and electronic device | |
| Guo et al. | Automated exploration and implementation of distributed cnn inference at the edge | |
| WO2022022571A1 (en) | Resource allocation for tuning hyperparameters of large-scale deep learning workloads | |
| CN115358400A (en) | Application method of deep learning model interface | |
| CN114490922A (en) | Natural language understanding model training method and device | |
| CN116862263A (en) | Method, device and storage medium for planning fusion of yield and policies | |
| CN111667060B (en) | Deep learning algorithm compiling method and device and related products | |
| CN112230911B (en) | Model deployment method, device, computer equipment and storage medium | |
| Nimi et al. | Chimera: Context-aware splittable deep multitasking models for edge intelligence | |
| Ansari | Deep Learning in Object Detection | |
| US20220245458A1 (en) | Apparatus and method for converting neural network | |
| CN112395859B (en) | Text processing method and related device | |
| CN115599987A (en) | Service processing method, device, equipment and storage medium | |
| Rodríguez et al. | Hybrid analysis pipelines in the REANA reproducible analysis platform | |
| CN113805976A (en) | Data processing method and device, electronic equipment and computer readable storage medium | |
| Carlsson et al. | Interconnecting Multiple FPGAs for Distributed Machine Learning Inference |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |