CN115358400B - Application method of deep learning model interface - Google Patents
Application method of deep learning model interface Download PDFInfo
- Publication number
- CN115358400B CN115358400B CN202211019035.0A CN202211019035A CN115358400B CN 115358400 B CN115358400 B CN 115358400B CN 202211019035 A CN202211019035 A CN 202211019035A CN 115358400 B CN115358400 B CN 115358400B
- Authority
- CN
- China
- Prior art keywords
- interface
- model
- data
- training
- loss
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 72
- 238000013136 deep learning model Methods 0.000 title claims abstract description 42
- 238000009795 derivation Methods 0.000 claims abstract description 53
- 238000012545 processing Methods 0.000 claims abstract description 21
- 238000010923 batch production Methods 0.000 claims abstract description 4
- 238000012549 training Methods 0.000 claims description 97
- 238000012360 testing method Methods 0.000 claims description 49
- 230000006870 function Effects 0.000 claims description 29
- 238000012795 verification Methods 0.000 claims description 26
- 238000004422 calculation algorithm Methods 0.000 claims description 20
- 238000013135 deep learning Methods 0.000 claims description 20
- 230000008569 process Effects 0.000 claims description 20
- 238000007781 pre-processing Methods 0.000 claims description 18
- 238000005538 encapsulation Methods 0.000 claims description 8
- 230000006978 adaptation Effects 0.000 claims description 6
- 238000012546 transfer Methods 0.000 claims description 6
- 238000005457 optimization Methods 0.000 claims description 5
- 238000010606 normalization Methods 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 238000010200 validation analysis Methods 0.000 claims description 3
- 230000026676 system process Effects 0.000 claims 1
- 238000011161 development Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 238000003062 neural network model Methods 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 5
- 230000008901 benefit Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 238000004806 packaging method and process Methods 0.000 description 3
- 230000002093 peripheral effect Effects 0.000 description 3
- 230000000712 assembly Effects 0.000 description 2
- 238000000429 assembly Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/04—Inference or reasoning models
- G06N5/046—Forward inferencing; Production systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computational Linguistics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention provides a use method of a deep learning model interface, which comprises the following steps: providing an inference interface configured to provide a pure tensor input interface at model deployment to facilitate model derivation; providing a data processor configured to batch-process the input data, convert the input data into tensors, and pass the tensors to a forward derivation interface for batch policy and model decoupling; and providing a forward derivation interface configured to accept the tensor after processing by the data processor, return loss or prediction results to enable direct use for model deployment.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to a use method of a deep learning model interface.
Background
Deep learning algorithm libraries typically require interfaces that define neural network model training; the training stage calculates loss, calculates gradient based on the loss, and updates model parameters; the prediction result or loss is calculated in the verification stage, wherein the prediction result can be further calculated to obtain an evaluation index, and the loss and the evaluation index can be used for adjusting the model super-parameters; calculating a prediction result in a test stage to measure the quality of model training; in addition, models often require defining an inference interface for inferring individual pictures.
In order to be compatible with distributed training and model deployment, the deep learning algorithm library is used for additionally packaging the neural network model, the process is complex, and the learning and development cost is high.
Disclosure of Invention
The invention aims to provide a use method of a deep learning model interface, which aims to solve the problem that the existing deep learning algorithm library introduces complex encapsulation to encapsulation of a neural network model, so that the development workload is large.
In order to solve the above technical problems, the present invention provides a method for using a deep learning model interface, including:
providing an inference interface configured to provide a pure tensor input interface at model deployment to facilitate model derivation;
Providing a data processor configured to batch-process the input data, convert the input data into tensors, and pass the tensors to a forward derivation interface for batch policy and model decoupling; and
A forward derivation interface is provided that is configured to accept tensors processed by the data processor, return loss or prediction results, to enable optimization and validation of the model during training. The inference interface is a function of the forward derivation interface or a branch.
Optionally, in the method for using a deep learning model interface,
Forming a model interface which is friendly to deployment and adapts to a plurality of deep learning back-end distributed training by agreeing with a forward deducing interface and dividing the boundaries of a data processor and the forward deducing interface;
respectively extracting unified and universal interfaces for training, reasoning and testing of the model;
The algorithm of any deep learning back end adopts the interface convention, and does not need complex encapsulation to carry out distributed training.
Optionally, in the method for using the deep learning model interface, the deep learning model performs a training process, an reasoning process and a testing process, and provides a unified interface for the deep learning model;
The training process comprises the following steps: the data processor is used for carrying out data enhancement in a training stage on the data input into the model, the enhanced data is transmitted to a training stepping interface of the model, the training stepping interface further calls a forward deducing interface of the model, returns to a loss dictionary, optimizes model weights based on the loss dictionary, and returns to the loss dictionary;
The verification process comprises the following steps: using a data processor to carry out data enhancement in a test stage on the data input into the model, and transmitting the enhanced data to a verification stepping interface of the model; the step interface is verified to further call a forward derivation interface of the model, and a loss dictionary or a prediction result is returned according to the input parameters;
The test flow comprises the following steps: processing data input into the model by using a data processor to carry out data enhancement in a test stage, and transmitting the enhanced data to a test stepping interface of the model; the test stepping interface further calls a forward deduction interface of the model, and a prediction result is returned.
Optionally, in the method for using a deep learning model interface,
Forming an independent class by taking a data processor as a module of the model, and preprocessing data input into the model;
The user reloads the custom data processor as needed to make the data processor have the following functions:
the data processor is used for constructing a model and can be freely combined with the model; and
The forward deduction interface of the model directly receives tensor input, returns an reasoning result and is convenient to deploy or directly call;
Wherein preprocessing the data of the input model comprises:
carrying data from the CPU to a designated device;
performing batch processing, wherein the batch processing comprises normalization, scaling and confusion enhancement; and
Data format conversion is performed to convert the input data into tensors.
Optionally, in the method for using a deep learning model interface, the interface of the data processor includes:
Transferring data interface cast_data, the interface function is introduced as transferring data to a designated device; the interface parameter list comprises data of an input model; the return value is data, and the data transferred to the target equipment has the same structure and input;
Forward deriving interface forward, interface function introduced as format needed for processing data into model; the interface parameter list comprises data processed by the transfer data interface and training enhancement tracking used for judging whether data enhancement of a training stage or a testing stage is adopted; the return value is the preprocessed data, and then is parsed and transmitted to the forward derivation interface of the model.
Optionally, in the method for using a deep learning model interface,
Extracting a training step interface, a verification step interface and a test step interface for the model so as to be compatible with single cards and distributed training logic of a plurality of training rear ends and meet different requirements of training, verification and test stages;
The forward deducing method is respectively called by training step, verification step and test step, receives the result processed by the data processor, and returns loss or reasoning result. Forward derivation is an abstract method that subclasses must implement to carry the subject logic of the model.
Optionally, in the method for using a deep learning model interface, the inference interface includes:
Forward deriving an interface forward, the interface function being described as calculating a penalty or prediction; the interface parameter list includes: receiving parameters args returned by forward deduction of the data processor, and returning loss or predicted result mode according to the value of mode;
training a stepping interface train_step, wherein the interface function description comprises the processes of forward deducing, gradient calculating, parameter updating and gradient clearing of model base classes in sequence, and the sub-classes are reloaded as required to realize more complex logic; the interface parameter list includes: inputting data of the model and an optimizer optimizer; the return values include: a loss dictionary loss, which is used only for logging, when the gradient calculation process has been completed;
verifying a stepping interface val_step, wherein the interface function description comprises forward deduction of model base class realization, and returning loss or prediction result; the interface parameter list includes: inputting data of a model, and returning a loss judgment result return_loss: if True, returning to the loss dictionary, otherwise, returning to the prediction result; the return value includes a loss dictionary or prediction result loss predictions;
the test step interface test_step, the interface function description includes model base class implementing forward derivation, and returns a prediction result, the interface parameter list includes data of the input model, and the return value includes the prediction result predictions.
Optionally, in the method for using the deep learning model interface, the training steps are sequentially performed:
Invoking a data processor to preprocess input data and returning data derived forward from the adaptation model;
Performing model forward derivation and return loss;
the training step directly calls a distributed data parallel forward deducing interface to utilize the efficient gradient synchronization function of the forward deducing interface;
Training steps of the model base class realize forward derivation, gradient reverse transfer, parameter updating and gradient logic under the condition of a single optimizer so as to meet task requirements;
And generating a model requiring multiple optimizer tasks to reload training steps as required, and realizing self-defined parameter optimization logic.
Optionally, in the method for using a deep learning model interface, the verification step may sequentially perform:
Invoking a data processor to preprocess input data and returning data derived forward from the adaptation model;
performing model forward deduction, and returning loss or prediction results according to a parameter return_loss;
Because the verification stage does not need cross-process synchronization, single card training and distributed training are not distinguished, and a unified flow is used;
the model base class realizes common reasoning logic, namely, directly calls a forward deduction interface after data preprocessing.
Optionally, in the method for using a deep learning model interface,
The testing stepping is consistent with the verification stepping flow, and the difference is that the testing stepping has no requirement of returning loss and only returns a prediction result;
the forward derivation is an abstract method, model parameters are accepted, and the forward derivation stage executes logic for calculating loss and calculating a prediction result according to the values of the model.
Through researches, the inventor of the invention discovers that in order to be compatible with distributed training and model deployment, a deep learning algorithm library can carry out secondary encapsulation on a neural network model, abstract an interface with the same name as a model interface, and the following problems are introduced:
First, model reasoning deployment generally requires the acceptance of Tensor (Tensor) type input, while model training, validation, and testing generally accepts non-Tensor data and pre-processes the data, both of which are not unified in flow. Existing deep learning algorithm libraries typically address this problem by making additional packages for the model as it is exported, but this increases the development effort.
Second, the model should be responsible for forward derivation, gradient back-propagation, and parameter updating, as a core of the deep learning algorithm, in general. However, in distributed training, there are usually some conventions for deep learning training back-end. Taking Pytorch as an example, to use the gradient synchronization function with high performance, the forward derivation interface of the distributed model package must be called, and gradient back-propagation is not allowed in the forward derivation process, so the deep learning algorithm library usually encapsulates the model in a complex manner to adapt to the distributed training of the corresponding back end.
Finally, deep learning models are also typically responsible for data batching, such as normalization, scaling, confusion enhancement (MixUp), etc., and deep learning algorithm libraries typically abstract this part of the logic into one way of modeling, but this can result in coupling of algorithms and batch strategies, failing to implement any combination of batch strategies and algorithm models.
Based on the insight, the invention provides a use method of the deep learning model interface, and the interface of the contracted model and the boundary of the dividing model function are realized by respectively setting the reasoning interface, the data processor and the forward deducing interface, so that a set of model interfaces which are friendly to deployment and adapt to multi-back-end distributed training is provided.
The invention provides a set of model training, verifying, testing and reasoning interfaces, provides a pure tensor input reasoning interface for model deployment, and is convenient for model export; lightweight model packaging is provided for adapting to the distributed conventions of different training back ends.
The unified deep learning model interface is a set of conventions of interfaces which are friendly to deployment and distributed training, and flexibly match batch processing strategies and models. Current deep learning algorithm warehouses often appear: only single card training is supported; distributed training is supported, and single-card training is not supported; the model structure is not suitable for deployment and difficult to derive, and if the algorithm is developed, the interface convention proposed by the patent is complied with, so that the problem can be well solved, and the efficiency of the researchers for reproducing and following the corresponding algorithm is improved; the model is more convenient to derive, the workload of model deployment is reduced, and the landing efficiency of the model is improved.
Drawings
FIG. 1 is a schematic diagram of an example interface convention of a deep learning model interface model according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a single card/distributed training stepping workflow of a deep learning model interface according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a deep learning model interface verification stepping workflow according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a deep learning model interface test stepping workflow according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a forward derivation workflow of a deep learning model interface according to an embodiment of the present invention;
FIG. 6 is a computer system in accordance with the systems and/or methods of the present invention.
Detailed Description
The invention is further elucidated below in connection with the embodiments with reference to the drawings.
It should be noted that the components in the figures may be shown exaggerated for illustrative purposes and are not necessarily to scale. In the drawings, identical or functionally identical components are provided with the same reference numerals.
In the present invention, unless specifically indicated otherwise, "disposed on …", "disposed over …" and "disposed over …" do not preclude the presence of an intermediate therebetween. Furthermore, "disposed on or above" … merely indicates the relative positional relationship between the two components, but may also be converted to "disposed under or below" …, and vice versa, under certain circumstances, such as after reversing the product direction.
In the present invention, the embodiments are merely intended to illustrate the scheme of the present invention, and should not be construed as limiting.
In the present invention, the adjectives "a" and "an" do not exclude a scenario of a plurality of elements, unless specifically indicated.
It should also be noted herein that in embodiments of the present application, only a portion of the components or assemblies may be shown for clarity and simplicity, but those of ordinary skill in the art will appreciate that the components or assemblies may be added as needed for a particular scenario under the teachings of the present application. In addition, features of different embodiments of the application may be combined with each other, unless otherwise specified. For example, a feature of the second embodiment may be substituted for a corresponding feature of the first embodiment, or may have the same or similar function, and the resulting embodiment may fall within the scope of disclosure or description of the application.
It should also be noted herein that, within the scope of the present invention, the terms "identical", "equal" and the like do not mean that the two values are absolutely equal, but rather allow for some reasonable error, that is, the terms also encompass "substantially identical", "substantially equal". By analogy, in the present invention, the term "perpendicular", "parallel" and the like in the table direction also covers the meaning of "substantially perpendicular", "substantially parallel".
The numbers of the steps of the respective methods of the present invention are not limited to the order of execution of the steps of the methods. The method steps may be performed in a different order unless otherwise indicated.
The method for using the deep learning model interface provided by the invention is further described in detail below with reference to the accompanying drawings and specific embodiments. The advantages and features of the present invention will become more apparent from the following description. It should be noted that the drawings are in a very simplified form and are all to a non-precise scale, merely for convenience and clarity in aiding in the description of embodiments of the invention.
The invention aims to provide a use method of a deep learning model interface, which aims to solve the problem that the existing deep learning algorithm library introduces complex encapsulation to encapsulation of a neural network model, so that the development workload is large.
In order to achieve the above object, the present invention provides a method for using a deep learning model interface, including: providing an inference interface configured to provide a pure tensor input interface at model deployment to facilitate model derivation; providing a data processor configured to batch-process the input data, convert the input data into tensors, and pass the tensors to a forward derivation interface for batch policy and model decoupling; and providing a forward derivation interface configured to accept the tensor after processing by the data processor, return loss or prediction results to enable direct use for model deployment.
FIGS. 1-6 provide a first embodiment of the present invention showing a deep learning model interface schematic.
The invention designs a set of model training, verifying, testing and reasoning interfaces, which has the following beneficial effects: the reasoning interface of pure tensor input is provided for model deployment, so that the model is conveniently exported; adapting different deep learning back ends; the batch processing strategy and the model are decoupled, so that free combination of the batch processing strategy and the model is realized.
According to the invention, a set of model interfaces which are friendly to deployment and adapt to multi-back-end distributed training is provided through interfaces of a contract model and boundaries of dividing model functions; first, dividing the boundaries of data preprocessing and model forward derivation includes: abstracting the data preprocessing into a data processor to realize the decoupling of processing, enhancement strategies and models; the data processor is responsible for converting the input data after batch processing into tensors, transmitting the tensors to the forward derivation interface, and the forward derivation interface receiving the tensors processed by the data processor and returning loss or prediction results. The forward derivation interface can be used directly for model deployment.
And secondly, respectively abstracting a unified and universal interface for training, reasoning and testing of the model. Any deep learning back-end algorithm can adopt such interface convention, and distributed training can be realized without complex packaging.
In addition, a training step, a verification step, a test step and a forward deducing method are abstracted for the model and are used for calling in the training, verification and test stages. The forward derivation method is also used to derive the model during the deployment phase.
And finally, abstracting the concept of data preprocessing into a data processor, realizing the decoupling of preprocessing and model main body logic, facilitating the deployment of the model, and being capable of freely combining the model and the batch processing strategy.
The invention defines the training, reasoning and testing flow of the deep learning model and provides a uniform interface for the deep learning model.
The training process comprises the following steps: the data processor is used for carrying out data enhancement in a training stage on the data input into the model, and the enhanced data is transmitted to a training stepping interface of the model; the step interface is further used for calling a forward deduction interface of the model, and returning to the loss dictionary; model weights are optimized based on the loss dictionary and returned to the loss dictionary.
The verification process comprises the following steps: using a data processor to carry out data enhancement in a test stage on the data input into the model, and transmitting the enhanced data to a verification stepping interface of the model; and the step interface is verified to further call a forward derivation interface of the model, and a loss dictionary or a prediction result is returned according to the input parameters.
The test flow comprises the following steps: processing data input into the model by using a data processor to carry out data enhancement in a test stage, and transmitting the enhanced data to a test stepping interface of the model; the test stepping interface further calls a forward deduction interface of the model, and a prediction result is returned.
In the data processor of the present embodiment, the model may perform the following preprocessing on the input data: data is carried from the cpu to a designated device, batched, such as normalized, scaled, confusion enhanced (mixup), etc., data format conversion, converting the input data into tensors.
The existing deep learning algorithm library generally abstracts the steps into a method of a model, and then the method is called in the forward deduction process; however, this introduces some problems: the model preprocessing mode and the model binding are inconvenient to expand, and a user cannot self-define the model collocation and the model preprocessing mode; the forward derivation interface is generally not expected to involve preprocessing operations, such as normalization, when model derivation is performed.
The invention provides a concept of a data processor, wherein the data processor is used as a component of a model and is an independent class and is responsible for preprocessing data input into the model: the user can reload the custom data processor as required; the use of a data processor has the following advantages: the data processor can be freely combined and configured as a model; the forward derivation interface (forward) of the model can directly accept tensor input, return reasoning results, and facilitate deployment or direct calling.
The interface of the data processor comprises: transferring data interface cast_data, the interface function is introduced as transferring data to a designated device; the interface parameter list comprises data of an input model; the return value is data, and the data transferred to the target equipment has the same structure and input; and forward deriving an interface forward, the interface function introduced as a format required to process the data into a model; the interface parameter list comprises data processed by the transfer data interface and training enhancement tracking used for judging whether data enhancement of a training stage or a testing stage is adopted; the return value is the preprocessed data, and then is parsed and transmitted to the forward derivation interface of the model.
In the model training/reasoning/testing interface of the embodiment, in order to be compatible with single card and distributed training logic of different training back ends, three interfaces of training stepping, verification stepping and testing stepping are abstracted for the model, so as to meet different requirements of training, verification and testing stages. The forward deducing method is respectively called by training stepping, verification stepping and test stepping, receives the result processed by the data processor, and returns loss or reasoning result. Forward derivation is an abstract method that subclasses must implement to carry the subject logic of the model.
The model training/reasoning/testing interface of the present embodiment includes: forward deriving an interface forward, the interface function being described as calculating a penalty or prediction; the interface parameter list includes: receiving parameters args returned by forward deduction of the data processor, and returning loss or predicted result mode according to the value of mode; training a stepping interface train_step, wherein the interface function description comprises the processes of forward deducing, gradient calculating, parameter updating and gradient clearing of model base classes in sequence, and the sub-classes are reloaded as required to realize more complex logic; the interface parameter list includes: inputting data of the model and an optimizer optimizer; the return values include: a loss dictionary loss, which is used only for logging, when the gradient calculation process has been completed; verifying a stepping interface val_step, wherein the interface function description comprises forward deduction of model base class realization, and returning loss or prediction result; the interface parameter list includes: inputting data of a model, and returning a loss judgment result return_loss: if True, returning to the loss dictionary, otherwise, returning to the prediction result; the return value includes a loss dictionary or prediction result loss predictions; and testing the stepping interface test_step, wherein the interface function description comprises model base class forward derivation, and returns a prediction result, the interface parameter list comprises data of an input model, and the return value comprises a prediction result predictions.
As shown in fig. 2, the training steps are sequentially performed: invoking a data processor to preprocess input data (data), and returning data derived forward from the adaptation model; performing model forward derivation and return loss; under the condition that the loss and input optimizers based on forward deduction execute gradient back-transmission and parameter updating are distributed, some training back-ends need to encapsulate the model. Taking Pytorch as an example, open distributed training typically requires that the model be packaged in parallel with distributed data. In this case, a reloading encapsulation class is required, which is provided with a train_step interface and implements the logic shown in the above figures. Unlike single-card training, training steps directly invoke the distributed data-parallel forward derivation interface to take advantage of its efficient gradient synchronization function.
The training steps of the model base class realize common forward derivation, gradient back transmission, parameter updating and gradient logic under the condition of a single optimizer so as to meet the demands of most tasks. The generation of a model requiring multiple optimizer tasks such as an countermeasure network can reload training steps as required, and the custom parameter optimization logic is realized.
As shown in fig. 3, the verification steps are sequentially performed: invoking a data processor to preprocess input data and returning data derived forward from the adaptation model; model forward derivation is performed, and loss or prediction results are returned according to the parameter return_loss. Because the verification stage does not need cross-process synchronization, single card training and distributed training are not distinguished, and a unified flow is used. The model base class realizes the common reasoning logic, namely, after data preprocessing, the forward deduction interface is directly called.
As shown in FIG. 4, the test_step flow is substantially identical to the val step, except that only the prediction result is returned, unlike the test_step, which does not return the lost demand.
As shown in fig. 5, forward is an abstract method, and needs to accept a mode parameter, and the forward derivation stage performs logic of calculating a loss and calculating a prediction result according to a mode value.
The invention provides a set of unified and universal model interfaces friendly to deployment and distributed training for the deep learning algorithm library; the user derives subclasses of the custom implementation based on the model base class and opens according to the interface specification. The model realized in this way is more convenient to deploy and compatible with distributed training of different back ends. The deep learning model interface of the invention can be based on different back-ends and different types of deep learning algorithms.
The application method of the deep learning model interface obtained by the technical scheme of the invention can be used for realizing the following technical effects in the fields of neural networks, computers, communication and the like: the distributed training device is more convenient to deploy and compatible with different back ends, and development efficiency is improved. The principle is that a set of unified and universal model interfaces friendly to deployment and distributed training is provided; the user derives subclasses of the custom implementation based on the model base class and opens according to the interface specification.
FIG. 6 illustrates a computer system 100 that implements systems and/or methods in accordance with the present invention. The method and/or system according to the present invention may be implemented in the computer system 100 shown in fig. 6 to accomplish the objects of the present invention, or the present invention may be distributed among a plurality of computer systems 100 according to the present invention via a network, such as a local area network or the internet, unless specifically stated otherwise. The computer system 100 of the present invention may comprise various types of computer systems, such as hand-held devices, laptop computers, personal Digital Assistants (PDAs), multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, network servers, tablet computers, and the like.
As shown in FIG. 6, computer system 100 includes processor 111, system bus 101, system memory 102, video adapter 105, audio adapter 107, hard disk drive interface 109, optical drive interface 113, network interface 114, and Universal Serial Bus (USB) interface 112. The system bus 101 may be any of several types of bus structures such as a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. The system bus 101 is used for communication between the various bus devices. In addition to the bus devices or interfaces shown in fig. 6, other bus devices or interfaces are also contemplated. The system memory 102 includes a Read Only Memory (ROM) 103 and a Random Access Memory (RAM) 104, where the ROM 103 may store basic input/output system (BIOS) data for implementing basic routines for information transfer at start-up, for example, and the RAM 104 is used to provide a running memory for the system that has a relatively high access speed. The computer system 100 further includes a hard disk drive 109 for reading from and writing to a hard disk 110, an optical drive interface 113 for reading from or writing to optical media such as a CD-ROM, and the like. The hard disk 110 may store, for example, an operating system and application programs. The drives and their associated computer-readable media provide nonvolatile storage of computer-readable instructions, data structures, program modules and other data for the computer system 100. Computer system 100 may also include a video adapter 105 for image processing and/or image output for interfacing with an output device such as a display 106. Computer system 100 may also include an audio adapter 107 for audio processing and/or audio output for connection to output devices such as speakers 108. In addition, computer system 100 may also include a network interface 114 for network connection, where network interface 114 may connect to the Internet 116 through a network device such as router 115, where the connection may be wired or wireless. In addition, computer system 100 may also include a universal serial bus interface (USB) 112 for connecting peripheral devices, including, for example, a keyboard 117, a mouse 118, and other peripheral devices, such as a microphone, a camera, and the like.
When the invention is implemented on the computer system 100 shown in fig. 6, a set of model interfaces which are friendly to deployment and adapt to multi-back-end distributed training can be provided through the interfaces of the contracted model and the boundaries of the divided model functions; first, dividing the boundaries of data preprocessing and model forward derivation includes: abstracting the data preprocessing into a data processor to realize the decoupling of processing, enhancement strategies and models; the data processor is responsible for converting the input data after batch processing into tensors, transmitting the tensors to the forward derivation interface, and the forward derivation interface receiving the tensors processed by the data processor and returning loss or prediction results. The forward derivation interface can be used directly for model deployment.
Furthermore, embodiments may be provided as a computer program product that may include one or more machine-readable media having stored thereon machine-executable instructions that, when executed by one or more machines, such as a computer, computer network, or other electronic device, may result in the one or more machines performing operations in accordance with embodiments of the present invention. The machine-readable medium may include, but is not limited to, floppy diskettes, optical disks, CD-ROMs (compact disk read-only memory), and magneto-optical disks, ROMs (read-only memory), RAMs (random access memory), EPROMs (erasable programmable read-only memory), EEPROMs (electrically erasable programmable read-only memory), magnetic or optical cards, flash memory, or other type of media/machine-readable medium suitable for storing machine-executable instructions.
Furthermore, embodiments may be downloaded as a computer program product, wherein the program may be transferred from a remote computer (e.g., a server) to a requesting computer (e.g., a client) by way of one or more data signals embodied in and/or modulated by a carrier wave or other propagation medium via a communication link (e.g., a modem and/or network connection). Accordingly, a machine-readable medium as used herein may include such a carrier wave, but is not required.
In summary, the above embodiments describe different configurations of the use method of the deep learning model interface in detail, and of course, the present invention includes, but is not limited to, the configurations listed in the above embodiments, and any content that changes based on the configurations provided in the above embodiments falls within the scope of protection of the present invention. One skilled in the art can recognize that the above embodiments are illustrative.
In the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other. For the system disclosed in the embodiment, the description is relatively simple because of corresponding to the method disclosed in the embodiment, and the relevant points refer to the description of the method section.
The above description is only illustrative of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention, and any alterations and modifications made by those skilled in the art based on the above disclosure shall fall within the scope of the appended claims.
Claims (7)
1. A method for using a deep learning model interface, comprising:
providing an inference interface configured to provide a pure tensor input interface at model deployment to facilitate model derivation;
Providing a data processor which
Configured to batch-process the input data, convert the input data into tensors, and transmit the tensors to a forward derivation interface for batch strategy and model decoupling; and
Providing a forward derivation interface, which
Configured to accept tensors processed by the data processor, return loss or prediction results to enable optimization and validation of the model during training;
the deep learning model performs training flow, reasoning flow and testing flow and provides a unified interface for the deep learning model;
the training process comprises the following steps: the data processor is used for carrying out data enhancement in a training stage on the data input into the model, the enhanced data is transmitted to a training stepping interface of the model, the training stepping interface further calls a forward deducing interface of the model, returns to a loss dictionary, optimizes model weights based on the loss dictionary, and returns to the loss dictionary;
The verification process comprises the following steps: using a data processor to carry out data enhancement in a test stage on the data input into the model, and transmitting the enhanced data to a verification stepping interface of the model; the step interface is verified to further call a forward derivation interface of the model, and a loss dictionary or a prediction result is returned according to the input parameters;
the test flow comprises the following steps: processing data input into the model by using a data processor to carry out data enhancement in a test stage, and transmitting the enhanced data to a test stepping interface of the model; the test stepping interface further calls a forward deduction interface of the model, and a prediction result is returned;
The training step interface, the verification step interface and the test step interface are extracted for the model so as to be compatible with single-card and distributed training logic of a plurality of training rear ends, and different requirements of training, verification and test stages are met;
The forward deducing method is called by training step, verification step and test step respectively, receives the result processed by the data processor, returns loss or reasoning result, and the forward deducing is an abstract method, and subclasses must be realized to bear the main logic of the model;
Wherein the inference interface comprises:
Forward deriving an interface forward, the interface function being described as calculating a penalty or prediction; the interface parameter list includes: receiving parameters args returned by forward deduction of the data processor, and returning loss or predicted result mode according to the value of mode;
training a stepping interface train_step, wherein the interface function description comprises the processes of forward deducing, gradient calculating, parameter updating and gradient clearing of model base classes in sequence, and the sub-classes are reloaded as required to realize more complex logic; the interface parameter list includes: inputting data of the model and an optimizer optimizer; the return values include: a loss dictionary loss, which is used only for logging, when the gradient calculation process has been completed;
Verifying a stepping interface val_step, wherein the interface function description comprises forward deduction of model base class realization, and returning loss or prediction result; the interface parameter list includes: inputting data of a model, and returning a loss judgment result return_loss, if True, returning a loss dictionary, otherwise, returning a prediction result; the return value includes a loss dictionary or prediction result loss predictions;
the test step interface test_step, the interface function description includes model base class implementing forward derivation, and returns a prediction result, the interface parameter list includes data of the input model, and the return value includes the prediction result predictions.
2. The method for using a deep learning model interface of claim 1,
Forming a model interface which is friendly to deployment and adapts to a plurality of deep learning back-end distributed training by agreeing with a forward deducing interface and dividing the boundaries of a data processor and the forward deducing interface;
respectively extracting unified and universal interfaces for training, reasoning and testing of the model;
The algorithm of any deep learning back end adopts the interface convention, and does not need complex encapsulation to carry out distributed training.
3. The method for using a deep learning model interface of claim 1,
Forming an independent class by taking a data processor as a module of the model, and preprocessing data input into the model;
The user reloads the custom data processor as needed to make the data processor have the following functions:
the data processor is used for constructing a model and can be freely combined with the model; and
The forward deduction interface of the model directly receives tensor input, returns an reasoning result and is convenient to deploy or directly call;
Wherein preprocessing the data of the input model comprises:
carrying data from the CPU to a designated device;
performing batch processing, wherein the batch processing comprises normalization, scaling and confusion enhancement; and
Data format conversion is performed to convert the input data into tensors.
4. A method of using a deep learning model interface as claimed in claim 3, wherein the interface of the data processor comprises:
Transferring data interface cast_data, the interface function is introduced as transferring data to a designated device; the interface parameter list comprises data of an input model; the return value is data, and the data transferred to the target equipment has the same structure and input;
Forward deriving interface forward, interface function introduced as format needed for processing data into model; the interface parameter list comprises data processed by the transfer data interface and training enhancement tracking used for judging whether data enhancement of a training stage or a testing stage is adopted; the return value is the preprocessed data, and then is parsed and transmitted to the forward derivation interface of the model.
5. The method of using a deep learning model interface of claim 1, wherein the training steps are performed sequentially by:
Invoking a data processor to preprocess input data and returning data derived forward from the adaptation model;
Performing model forward derivation and return loss;
the training step directly calls a distributed data parallel forward deducing interface to utilize the efficient gradient synchronization function of the forward deducing interface;
Training steps of the model base class realize forward derivation, gradient reverse transfer, parameter updating and gradient logic under the condition of a single optimizer so as to meet task requirements;
And generating a model requiring multiple optimizer tasks to reload training steps as required, and realizing self-defined parameter optimization logic.
6. The method of claim 5, wherein the step of verifying sequentially performs:
Invoking a data processor to preprocess input data and returning data derived forward from the adaptation model;
performing model forward deduction, and returning loss or prediction results according to a parameter return_loss;
because the verification stage does not need cross-process synchronization, single card training and distributed training are not distinguished, and a system-process is used;
the model base class realizes common reasoning logic, namely, directly calls a forward deduction interface after data preprocessing.
7. The method for using a deep learning model interface of claim 6,
The testing stepping is consistent with the verification stepping flow, and the difference is that the testing stepping has no requirement of returning loss and only returns a prediction result;
the forward derivation is an abstract method, model parameters are accepted, and the forward derivation stage executes logic for calculating loss and calculating a prediction result according to the values of the model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211019035.0A CN115358400B (en) | 2022-08-24 | 2022-08-24 | Application method of deep learning model interface |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211019035.0A CN115358400B (en) | 2022-08-24 | 2022-08-24 | Application method of deep learning model interface |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115358400A CN115358400A (en) | 2022-11-18 |
| CN115358400B true CN115358400B (en) | 2024-08-02 |
Family
ID=84003671
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211019035.0A Active CN115358400B (en) | 2022-08-24 | 2022-08-24 | Application method of deep learning model interface |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115358400B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112819153A (en) * | 2020-12-31 | 2021-05-18 | 杭州海康威视数字技术股份有限公司 | Model transformation method and device |
| WO2021184346A1 (en) * | 2020-03-20 | 2021-09-23 | 云图技术有限公司 | Private machine learning model generation and training methods, apparatus, and electronic device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111461332B (en) * | 2020-03-24 | 2021-08-20 | 北京五八信息技术有限公司 | Deep learning model online reasoning method and device, electronic equipment and storage medium |
-
2022
- 2022-08-24 CN CN202211019035.0A patent/CN115358400B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021184346A1 (en) * | 2020-03-20 | 2021-09-23 | 云图技术有限公司 | Private machine learning model generation and training methods, apparatus, and electronic device |
| CN112819153A (en) * | 2020-12-31 | 2021-05-18 | 杭州海康威视数字技术股份有限公司 | Model transformation method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115358400A (en) | 2022-11-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3667496B1 (en) | Distributed computing system, data transmission method and device in distributed computing system | |
| WO2021190597A1 (en) | Processing method for neural network model, and related device | |
| US20160004578A1 (en) | Realtime processing of streaming data | |
| US20220413943A1 (en) | Apparatus, articles of manufacture, and methods for managing processing units | |
| US20220318202A1 (en) | Method and subsystem of a distributed log-analytics system that automatically determine the source of log/event messages | |
| CN106970802A (en) | The method and device of integrated programming script in the language-specific of field | |
| US20220158889A1 (en) | Efficient event-type-based log/event-message processing in a distributed log-analytics system | |
| CN111459610A (en) | Model deployment method and device | |
| CN112085166A (en) | Convolutional neural network model accelerated training method and device, electronic equipment and storage medium | |
| CN115203282A (en) | Intelligent enterprise user data processing method and system combined with deep learning | |
| CN115358400B (en) | Application method of deep learning model interface | |
| US20080082473A1 (en) | Peer based event conversion | |
| CN110018831A (en) | Program processing method, device and Related product | |
| CN102880669A (en) | Business processing method and business processing device based on file | |
| Ansari | Deep Learning in Object Detection | |
| Heye | Scaling deep learning without increasing batchsize | |
| CN111273901A (en) | File format and deployment method of machine learning model capable of being rapidly deployed online | |
| CN113805976B (en) | Data processing method and device, electronic equipment and computer readable storage medium | |
| NL2031965B1 (en) | Apparatus, articles of manufacture, and methods for managing processing units | |
| Anisimov et al. | Development and Prospects of the PARCS-WCF System | |
| CN117493530B (en) | Resource demand analysis method, device, electronic equipment and storage medium | |
| CN116522002B (en) | Container recommendation method and system of navigation service system based on machine learning | |
| CN112988551B (en) | Code coverage rate testing method and device | |
| CN117130591A (en) | Code generation method, system and related equipment | |
| WO2024139812A1 (en) | Resource type data transmission method and related apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |