CN112559124A - Model management system and target operation instruction processing method and device - Google Patents
Model management system and target operation instruction processing method and device Download PDFInfo
- Publication number
- CN112559124A CN112559124A CN202011412388.8A CN202011412388A CN112559124A CN 112559124 A CN112559124 A CN 112559124A CN 202011412388 A CN202011412388 A CN 202011412388A CN 112559124 A CN112559124 A CN 112559124A
- Authority
- CN
- China
- Prior art keywords
- target
- model
- operation instruction
- target operation
- loader
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title description 11
- 238000012545 processing Methods 0.000 claims abstract description 53
- 238000000034 method Methods 0.000 claims abstract description 41
- 238000012795 verification Methods 0.000 claims description 39
- 230000015654 memory Effects 0.000 claims description 20
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000001514 detection method Methods 0.000 claims description 5
- 238000004590 computer program Methods 0.000 claims description 4
- 238000007726 management method Methods 0.000 description 21
- 238000010586 diagram Methods 0.000 description 13
- 230000005540 biological transmission Effects 0.000 description 7
- 230000000694 effects Effects 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000005096 rolling process Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Stored Programmes (AREA)
Abstract
The application relates to a model management system and a method and a device for processing a target operation instruction, wherein the model management system comprises a model scheduler and a plurality of reference model loaders, and the model scheduler is used for forwarding the target operation instruction to a target model loader in the plurality of reference model loaders according to the load value of each model loader after the target operation instruction is detected; and the target model loader is connected with the model scheduler and used for receiving the target operation instruction forwarded by the model scheduler and executing corresponding target operation on the target model file according to the target operation instruction. The method and the device solve the technical problem that the processing efficiency of the operation instruction is low.
Description
Technical Field
The present application relates to the field of computers, and in particular, to a model management system and a method and an apparatus for processing a target operation instruction.
Background
In the field of artificial intelligence, a deep learning method solves more and more practical problems in different fields, such as image recognition, automatic driving, intelligent customer service and the like, model service operations such as model updating or model application and the like are required to be carried out on a model along with the change of a scene, and the existing model service technology widely adopts related container technology, such as k8s (Kubernetes, an open-source container arrangement engine) related technology, so that the model service becomes simple and controllable and can support various models, however, when the existing container technology is used for deploying the relevant model examples, the resource usage principle is that reallocation is applied first, the resources beyond the application cannot be used, therefore, in order to make the service operate normally, a little more resource needs to be applied, and if the model service instances are especially many, the resource waste and the load pressure of the container are caused.
In view of the above problems, no effective solution has been proposed.
Disclosure of Invention
The application provides a model management system, a target operation instruction processing method and a target operation instruction processing device, and aims to at least solve the technical problem that the processing efficiency of operation instructions in the related art is low.
According to an aspect of an embodiment of the present application, there is provided a model management system including a model scheduler and a plurality of reference model loaders: the model scheduler is used for forwarding a target operation instruction to a target model loader in the plurality of reference model loaders according to the load value of each reference model loader after the target operation instruction is detected; and the target model loader is connected with the model scheduler and used for receiving the target operation instruction forwarded by the model scheduler and executing corresponding target operation on a target model file according to the target operation instruction.
Optionally, the model management system further comprises a model translator, wherein: the model converter is used for carrying out target processing on the file format of the initial model file so as to obtain the target model file in the target format.
Optionally, the model management system further includes a model storage center, a target database, and a model management component, wherein: the model storage center is used for storing the target model file; the model management component is used for determining a storage path of the target model file in the model storage center, determining meta information of the target model file containing the storage path, and sending the meta information to the target database for storage.
Optionally, the target model loader is further configured to: acquiring the meta information based on the target operation instruction; retrieving the target model file based on the storage path included in the meta information; and verifying the target model file according to the initial verification information included in the meta information, and executing corresponding target operation on the target model file according to the target operation instruction when a verification result meets a set requirement.
According to another aspect of the embodiments of the present application, there is also provided a method for processing a target operation instruction, including: detecting a target operation instruction; forwarding the target operation instruction to a target model loader in the plurality of reference model loaders according to load values of the plurality of reference model loaders, wherein the target operation instruction is used for instructing the target model loader to execute a target operation corresponding to the target operation instruction.
Optionally, forwarding the target operation instruction to the target model loader of the plurality of reference model loaders according to the load values of the plurality of reference model loaders comprises: obtaining the load value of each reference model loader; comparing each load value to determine a minimum load value; determining the reference model loader corresponding to the minimum load value as the target model loader; and forwarding the target operation instruction to the target model loader.
According to another aspect of the embodiments of the present application, there is also provided a method for processing a target operation instruction, including: receiving a target operation instruction forwarded by a model scheduler, wherein the model scheduler is used for forwarding the target operation instruction to a target model loader in a plurality of reference model loaders according to load values of the plurality of reference model loaders; calling a corresponding target model file according to the target operation instruction; and under the condition that the target model file meets the set requirement, executing target operation corresponding to the target operation instruction on the target model file.
Optionally, invoking the corresponding target model file according to the target operation instruction includes: inquiring meta information corresponding to the target operation instruction according to the operation instruction, wherein the meta information is stored in a target database and comprises a storage path of the target model file; and calling the target model file according to the storage path of the target model file in the meta information.
Optionally, before the target operation corresponding to the operation instruction is executed on the target model file, the method further includes: performing target calculation on the target model file to obtain corresponding target verification information; verifying the target verification information according to initial verification information included in the meta information to obtain a verification result; and when the verification result indicates that the initial verification information is consistent with the target verification information, determining that the target model file meets the set requirement.
According to another aspect of the embodiments of the present application, there is also provided a device for processing a target operation instruction, including: the detection module is used for detecting a target operation instruction; and the forwarding module is used for forwarding the target operation instruction to a target model loader in the plurality of reference model loaders according to the load values of the plurality of reference model loaders, wherein the target operation instruction is used for instructing the target model loader to execute a target operation corresponding to the target operation instruction.
According to another aspect of the embodiments of the present application, there is also provided a device for processing a target operation instruction, including: the model scheduler is used for forwarding the target operation instruction to a target model loader in a plurality of reference model loaders according to load values of the plurality of reference model loaders; the calling module is used for calling a corresponding target model file according to the target operation instruction; and the processing module is used for executing target operation corresponding to the target operation instruction on the target model file under the condition that the target model file meets the set requirement.
According to another aspect of the embodiments of the present application, there is also provided a storage medium including a stored program which, when executed, performs the above-described method.
According to another aspect of the embodiments of the present application, there is also provided an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the above method through the computer program.
In the embodiment of the application, the model management system comprises a model scheduler and a plurality of reference model loaders, wherein the model scheduler is used for forwarding a target operation instruction to a target model loader in the plurality of reference model loaders according to the load value of each reference model loader after the target operation instruction is detected; the target model loader is connected with the model scheduler and used for receiving the target operation instruction forwarded by the model scheduler and setting a plurality of reference model loaders responsible for executing operation on the target model file according to the mode of executing corresponding target operation on the target model file by the target operation instruction, the reference model loaders are connected with the model scheduler and send the load condition of the reference model loaders to the model scheduler, after the model scheduler detects the target operation instruction, the target model loader of the forwarded target operation instruction is determined according to the load value sent by each reference model loader and is sent to the target model loader according to the target operation instruction method, the target model loader executes corresponding target operation on the target model file after receiving the target operation instruction sent by the model scheduler, and the purpose of forwarding the target operation instruction according to the load value of each reference model loader is achieved, the target operation instruction received by the target model loader is matched with the load value of the target operation instruction, so that the technical effect of improving the processing efficiency of the operation instruction is achieved, and the technical problem of low processing efficiency of the operation instruction is solved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.
FIG. 1 is a block diagram of an alternative model management system according to an embodiment of the present application;
FIG. 2 is a schematic diagram of a hardware environment for a method of processing a target operation instruction according to an embodiment of the present application;
FIG. 3 is a flow chart of an alternative method of processing a target operation instruction according to an embodiment of the present application;
FIG. 4 is a flow chart of another alternative method of processing a target operation instruction according to an embodiment of the application;
FIG. 5 is an alternative target model deployment architecture diagram according to an embodiment of the present application;
FIG. 6 is a schematic diagram of an alternative processing apparatus for target operation instructions according to an embodiment of the present application;
FIG. 7 is a schematic diagram of an alternative processing device for target operational instructions according to an embodiment of the present application;
fig. 8 is a block diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
According to an aspect of embodiments of the present application, there is provided embodiments of a model management system.
Fig. 1 is a block diagram of an alternative model management system according to an embodiment of the present application, and as shown in fig. 1, the system includes a model scheduler 12 and a plurality of reference model loaders, which are described in detail below:
the model scheduler 12 is configured to, after a target operation instruction is detected, forward the target operation instruction to a target model loader of the plurality of reference model loaders according to a load value of each reference model loader;
and the target model loader 14 is used for connecting the target model loader with the model scheduler, receiving the target operation instruction forwarded by the model scheduler, and executing corresponding target operation on a target model file according to the target operation instruction.
By adopting the scheme of the model management system provided by the embodiment of the application, a plurality of reference model loaders which are responsible for executing operation on the target model file are arranged, the reference model loaders are connected with the model scheduler and send the load condition of the model scheduler to the model scheduler, after the model scheduler detects the target operation instruction, the target model loader for determining the target model loader of the forwarded target operation instruction according to the load value sent by each reference model loader sends the target operation instruction to the target model loader, the target model loader executes corresponding target operation on the target model file after receiving the target operation instruction sent by the model scheduler, the purpose of forwarding the target operation instruction according to the load value of each reference model loader is achieved, and the target operation instruction received by the target model loader is matched with the load value of the target operation instruction, therefore, the technical effect of improving the processing efficiency of the operation instruction is achieved, and the technical problem that the processing efficiency of the operation instruction is low is solved.
Optionally, in this embodiment, the model scheduler may include, but is not limited to, a physical host with a scheduling function, a functional module with a scheduling function in a database, or other modules or devices with similar processing capabilities, and the model scheduler may correspond to a plurality of reference model loaders, and the model scheduler and the reference model loaders may be connected, but is not limited to, through a middleware with a load balancing function, such as a Nginx (Engine X) connection, and of course, the model scheduler and the reference model loaders may also be connected directly, or through other components, and the like, which is not limited in this embodiment.
Optionally, in this embodiment, the reference model loader may include, but is not limited to, a physical host, a function module in a database, and the like, which is not limited in this embodiment.
Optionally, in this embodiment, the target operation request may include, but is not limited to, requesting the target model loader to perform application deployment, version upgrade, and the like on the target model file.
Optionally, in this embodiment, the detecting the target operation instruction may be, but is not limited to, implemented by tracking a log file in a database, for example, the model scheduler tracks a binlog log file of the MySQL database.
As an alternative embodiment, the model management system further comprises a model translator, wherein: the model converter is used for carrying out target processing on the file format of the initial model file so as to obtain the target model file in the target format.
Optionally, in this embodiment, the initial model file is a model file developed by a model developer, the code formats used by different model developers are also different, and the model converter may be an ONNX (Open Neural Network Exchange) model converter, and the file format of the initial model file is converted into an ONNX format file by the ONNX model converter.
Optionally, in this embodiment, the model file in the target format converted by the model converter may be quickly loaded by an interface tool compiled by a programming language of the system, for example, the interface tool compiled by a programming language such as C + + language, JAVA language, or the like quickly loads the model file in the target format converted by the model converter, which is not limited in this embodiment.
Through the steps, the initial model files in various formats developed by the model developers are converted into the unified model file format by using the model converter, so that the model development file is not influenced by the code quality of the model development file, and the model service efficiency of the target loader is improved.
As an optional embodiment, the model management system further comprises a model storage center, a target database, and a model management component, wherein: the model storage center is used for storing the target model file; the model management component is used for determining a storage path of the target model file in the model storage center, determining meta information of the target model file containing the storage path, and sending the meta information to the target database for storage.
Alternatively, in this embodiment, the target database is a database with a storage function, and may be used to store meta information of the model, the target model file, and the like.
Optionally, in this embodiment, the model storage center may be a designated storage module in the target database, or may be a certain physical host dedicated to storing the target model file.
Optionally, in this embodiment, the meta information is used to store attribute information of the model, such as: version information of the model, model registration time, model developer information, initial model file format, storage path of the target model file, and the like.
Through the steps, the object model file is stored in the model storage center through the model management component, and the storage path of the object model file in the model storage center is recorded in the meta information, so that the object model file is stored orderly.
As an alternative embodiment, the target model loader is further configured to: acquiring the meta information based on the target operation instruction; retrieving the target model file based on the storage path included in the meta information; and verifying the target model file according to the initial verification information included in the meta information, and executing corresponding target operation on the target model file according to the target operation instruction when a verification result meets a set requirement.
Optionally, in this embodiment, the initial verification information is obtained by calculating the model file, and the completeness, whether the version is changed, whether there is file tampering, and the like of the target model file called by the target model loader are verified through the initial verification information, for example: the MD5(Message-Digest Algorithm 5) value of the model file is calculated to verify whether the target model file called by the target model scheduler is complete, whether there is file tampering, etc.
Through the steps, the target model file called by the target model loader is verified by using the initial verification information, so that the accuracy of the target model file called by the target model loader is ensured.
According to an aspect of the embodiments of the present application, an embodiment of a method for processing a target operation instruction is provided.
Alternatively, in the present embodiment, the processing method of the target operation instruction described above may be applied to a hardware environment configured in a hardware environment configured by the terminal 201 and the middleware server 203, the plurality of slave servers (205-1 to 205-n), and the master server 207 as shown in fig. 2. Fig. 2 is a schematic diagram of a hardware environment of a processing method of a target operation instruction according to an embodiment of the present application, as shown in fig. 2, a middleware server 203 is connected to a terminal 201, a plurality of slave servers (205-1 to 205-n) and a master server 207 through a network, respectively, the middleware server 203 may be configured to provide services (such as operation request forwarding and the like) for the terminal or a client installed on the terminal, a database may be provided on the server or independent of the server, and the plurality of slave servers (205-1 to 205-n) and the master server 207 are configured to provide services of data storage, query and the like for the terminal 201, where the network includes but is not limited to: the terminal 201 is not limited to a PC, a mobile phone, a tablet computer, etc. in a wide area network, a metropolitan area network, or a local area network. The processing method of the target operation instruction of the embodiment of the application may be executed by the middleware server 203, the master server 207, and the plurality of slave servers (205-1 to 205-n), or may be executed by the middleware server 203, the master server 207, and the plurality of slave servers (205-1 to 205-n) together.
According to an aspect of the embodiments of the present application, an embodiment of a method for processing a target operation instruction is provided, and fig. 3 is a flowchart of an optional method for processing a target operation instruction according to the embodiment of the present application, and as shown in fig. 3, the method may include the following steps:
step S302, a target operation instruction is detected;
step S304, forwarding the target operation instruction to a target model loader of the plurality of reference model loaders according to load values of the plurality of reference model loaders, where the target operation instruction is used to instruct the target model loader to execute a target operation corresponding to the target operation instruction.
Through the steps S302 to S304, by detecting the target operation instruction, determining the target model loader capable of executing the target operation instruction according to the load values of the plurality of acquired reference model loaders and the load values of the reference model loaders, and sending the target operation instruction to the target model loader, the purpose of forwarding the target operation instruction according to the load values of the reference model loaders is achieved, the technical problem of low processing efficiency of the operation instruction can be solved, and the technical effect of improving the processing efficiency of the operation instruction is achieved.
In the technical solution provided in step S302, the target operation instruction may be detected, but not limited to, by detecting a log file in a detection system or a database, and the corresponding target operation instruction is determined by detecting a related operation statement in the log file, for example: and tracking related operation sentences in a binlog log file of the MySQL database, and sending a start-stop command corresponding to the model version after finding the registration or offline information of the new model so as to obtain a target operation instruction.
Optionally, in this embodiment, the target operation instruction may include, but is not limited to, requesting the target model loader to perform application deployment, version upgrade, function change, and the like on the target model file.
In the technical solution provided in step S304, the load value of the reference model loader can be represented by, but not limited to, using number of queries TPS (Transactions Per Second, number of Transactions transmitted Per Second), current number of Transactions processed, number of concurrency, and so on, such as: the larger the current query times, the larger the current transaction number and the larger the concurrency number of the reference model loader, the larger the load value of the reference model loader is.
Optionally, in this embodiment, the reference model loader may monitor its own load value, and report the load value to the model scheduler according to a fixed time interval, for example: the reference model scheduler reports load values to the model scheduler every 2 seconds, 3 seconds, 10 seconds, 20 seconds, etc.
As an alternative embodiment, forwarding the target operation instruction to the target model loader of the plurality of reference model loaders according to the load values of the plurality of reference model loaders comprises:
s11, acquiring the load value of each reference model loader;
s12, comparing each load value to determine the minimum load value;
s13, determining the reference model loader corresponding to the minimum load value as the target model loader;
s14, forwarding the target operation instruction to the target model loader.
Optionally, in this embodiment, a relatively idle reference model loader is determined as a target model loader according to the load value of each reference database, and a smaller load value indicates that the current reference model loader is relatively idle, and it is necessary to send a limited target operation instruction to the relatively idle reference model loader, for example: there are three reference model loaders with the same operation performance, and the load value of each current reference model loader is 80, and 40, respectively, at this time, the reference model loader with the load value of 40 is preferentially determined as the target model loader, and the target operation instruction is forwarded to the target model loader.
Through the steps, the relatively idle target model loader is selected according to the load value of the reference model loader, and the target operation instruction is preferentially sent to the target model loader, so that the load values of the reference model loaders are close to each other as much as possible, the resource utilization rate of the model loader is improved, and the model deployment efficiency is improved.
According to an aspect of the embodiments of the present application, there is further provided another embodiment of a method for processing a target operation instruction, and fig. 4 is a flowchart of another optional method for processing a target operation instruction according to the embodiments of the present application, and as shown in fig. 4, the method may include the following steps:
step S402, receiving a target operation instruction forwarded by a model scheduler, wherein the model scheduler is used for forwarding the target operation instruction to a target model loader in a plurality of reference model loaders according to load values of the plurality of reference model loaders;
step S404, calling a corresponding target model file according to the target operation instruction;
step S406, when the target model file meets the setting requirement, executing a target operation corresponding to the target operation instruction on the target model file.
Through the steps S402 to S406, after the model scheduler selects the target model loader according to the load value of each model loader, the target operation instruction is forwarded, and after the target model loader receives the target operation instruction, the target model loader executes the target operation corresponding to the target operation instruction on the target model file, so that the purpose of forwarding the target operation instruction according to the load value of each reference model loader is achieved, the technical problem of low processing efficiency of the operation instruction can be solved, and the technical effect of improving the processing efficiency of the operation instruction is achieved.
Optionally, in this embodiment, the processing method of the target operation instruction is applied to a target model loader determined according to a load value of each reference model loader among a plurality of reference model loaders.
In the technical solution provided in step S406, the target model file meeting the setting requirement may include, but is not limited to, that the version information, the file content, and the like of the target model file called by the target model loader are the same as those of the stored target model file.
As an optional embodiment, invoking the corresponding target model file according to the target operation instruction includes:
s21, inquiring meta information corresponding to the target operation instruction according to the target operation instruction, wherein the meta information is stored in a target database and comprises a storage path of the target model file;
s22, calling the target model file according to the storage path of the target model file in the meta information.
Optionally, in this embodiment, the meta information is attribute information of the target model file obtained when the model is registered, and the meta information may include, but is not limited to, version information of the model, model registration time, model developer information, an initial model file format, a storage path of the target model file, and the like.
Through the steps, the accuracy of the target model file called by the target model loader is ensured by inquiring the meta information corresponding to the target operation instruction and determining the target model file to be operated according to the storage path in the meta information.
As an optional embodiment, before performing the target operation corresponding to the operation instruction on the target model file, the method further includes:
s31, performing target calculation on the target model file to obtain corresponding target verification information;
s32, verifying the target verification information according to the initial verification information included in the meta information to obtain a verification result;
s33, when the verification result indicates that the initial verification information is consistent with the target verification information, determining that the target model file meets the setting requirement.
Alternatively, the verification information in this embodiment may be implemented by, but not limited to, calculating MD5 values for model files, calculating MD5 values for stored model files when the model files are stored, calculating MD5 values for target model files before target operations are performed on the target model files, and when the two MD5 values are consistent, proving that the target model files are completed, the version information and the file content are the same as those of the previously stored model files.
FIG. 5 is an alternative target model deployment architecture diagram according to an embodiment of the application, as shown in FIG. 5:
s501, model developers develop and train deep learning models, and different developers can develop the models by using different operation languages.
S502, performing file conversion on the developed model files of various operation languages, and converting the initial model file into a model file in an ONNX format by using an ONNX converter.
S503, the converted model file in the ONNX format enters a model register and is registered to the model management component.
S504, the model management component stores the model file in the ONNX format in the model storage center, records the storage path of the model file in the ONNX format in the model storage center, and manages the model meta-information including the version, the registration time, the author, the input parameter format, the output format, the path in the model storage center, the MD5 of the model file and the like by relying on the MySQL database.
And S505, the model storage center stores the converted ONNX-format model file.
And S506, the MySQL database stores the model meta-information.
S507, the model scheduler tracks the log files of the MySQL database, sends out target operation instructions corresponding to model versions after finding out the registration or offline information of new models, observes the loads of the model loaders, the model loaders are multiple, each model loader can report the load value of the model loader at regular time, the model scheduler preferentially transfers the target operation instructions to the model loaders with low loads according to the load values of the model loaders, and if the versions are upgraded or rolled back, the model scheduler issues commands one by one host to realize rolling upgrade or rolling back.
S508, the model loader can realize load balancing by butting Nginx, and internally, by inquiring the model meta information stored in MySQL, the version, the file position and the file MD5 of the converted ONNX-format model file can be obtained, then the corresponding ONNX-format model file is pulled from the model storage center, and the called model file can be directly loaded by the realization of C + + language after being verified by MD 5.
It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present application is not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application.
Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling an electronic device (such as a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present application.
According to another aspect of the embodiments of the present application, there is also provided a processing apparatus for a target operation instruction, which is used for implementing the processing method for a target operation instruction. Fig. 6 is a schematic diagram of an alternative processing apparatus for target operation instructions according to an embodiment of the present application, and as shown in fig. 6, the apparatus may include:
a detection module 62 for detecting a target operation instruction;
a forwarding module 64, configured to forward the target operation instruction to a target model loader of the multiple reference model loaders according to load values of the multiple reference model loaders, where the target operation instruction is used to instruct the target model loader to execute a target operation corresponding to the target operation instruction.
It should be noted that the detection module 62 in this embodiment may be configured to execute step S302 in this embodiment, and the forwarding module 64 in this embodiment may be configured to execute step S304 in this embodiment.
It should be noted here that the modules described above are the same as the examples and application scenarios implemented by the corresponding steps, but are not limited to the disclosure of the above embodiments. It should be noted that the modules described above as a part of the apparatus may operate in a hardware environment as shown in fig. 1, and may be implemented by software or hardware.
Through the module, the technical problem of low processing efficiency of the operation instruction can be solved, and the technical effect of improving the processing efficiency of the operation instruction is achieved.
As an optional embodiment, the forwarding module includes: an obtaining unit configured to obtain the load value of each of the reference model loaders; the comparison unit is used for comparing each load value to determine a minimum load value; a determining unit, configured to determine the reference model loader corresponding to the minimum load value as the target model loader; and the forwarding unit is used for forwarding the target operation instruction to the target model loader.
According to another aspect of the embodiments of the present application, there is provided another processing apparatus for implementing the above target operation instruction. Fig. 7 is a schematic diagram of another alternative target operation instruction processing apparatus according to an embodiment of the present application, and as shown in fig. 7, the apparatus may include:
a receiving module 72, configured to receive a target operation instruction forwarded by a model scheduler, where the model scheduler is configured to forward the target operation instruction to a target model loader in a plurality of reference model loaders according to load values of the plurality of reference model loaders;
the calling module 74 is configured to call a corresponding target model file according to the target operation instruction;
and the processing module 76 is configured to execute a target operation corresponding to the target operation instruction on the target model file when the target model file meets a set requirement.
It should be noted that the receiving module 72 in this embodiment may be configured to execute the step S402 in this embodiment, the retrieving module 74 in this embodiment may be configured to execute the step S404 in this embodiment, and the processing module 76 in this embodiment may be configured to execute the step S406 in this embodiment.
It should be noted here that the modules described above are the same as the examples and application scenarios implemented by the corresponding steps, but are not limited to the disclosure of the above embodiments. It should be noted that the modules described above as a part of the apparatus may operate in a hardware environment as shown in fig. 1, and may be implemented by software or hardware.
Through the modules, after the model scheduler selects the target model loader according to the load value of each model loader, the target operation instruction is forwarded, and after the target model loader receives the target operation instruction, the target model loader executes the target operation corresponding to the target operation instruction on the target model file, so that the purpose of forwarding the target operation instruction according to the load value of each reference model loader is achieved, the technical problem of low processing efficiency of the operation instruction can be solved, and the technical effect of improving the processing efficiency of the operation instruction is achieved.
As an alternative embodiment, the retrieving module includes: the query unit is used for querying the meta information corresponding to the target operation instruction according to the operation instruction, wherein the meta information is stored in a target database, and the meta information comprises a storage path of the target model file; and the calling unit is used for calling the target model file according to the storage path of the target model file in the meta information.
As an alternative embodiment, the apparatus further comprises: the calculation module is used for performing target calculation on the target model file before executing target operation corresponding to the operation instruction on the target model file so as to obtain corresponding target verification information; the verification module is used for verifying the target verification information according to the initial verification information included in the meta information to obtain a verification result; and the determining module is used for determining that the target model file meets the set requirement when the verification result indicates that the initial verification information is consistent with the target verification information.
It should be noted here that the modules described above are the same as the examples and application scenarios implemented by the corresponding steps, but are not limited to the disclosure of the above embodiments. It should be noted that the modules described above as a part of the apparatus may be operated in a hardware environment as shown in fig. 1, and may be implemented by software, or may be implemented by hardware, where the hardware environment includes a network environment.
According to another aspect of the embodiments of the present application, there is also provided an electronic device for implementing the processing method of the target operation instruction.
Fig. 8 is a block diagram of an electronic device according to an embodiment of the present application, and as shown in fig. 8, the electronic device may include: one or more processors 801 (only one of which is shown), a memory 803, and a transmission device 805, as shown in fig. 8, the electronic device may further include an input/output device 807.
The memory 803 may be used to store software programs and modules, such as program instructions/modules corresponding to the method and apparatus for processing the target operation instruction in the embodiment of the present application, and the processor 801 executes various functional applications and data processing by running the software programs and modules stored in the memory 803, that is, implements the method for processing the target operation instruction. The memory 803 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 803 may further include memory located remotely from the processor 801, which may be connected to electronic devices via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The above-mentioned transmission device 805 is used for receiving or sending data via a network, and may also be used for data transmission between a processor and a memory. Examples of the network may include a wired network and a wireless network. In one example, the transmission device 805 includes a Network adapter (NIC) that can be connected to a router via a Network cable and other Network devices to communicate with the internet or a local area Network. In one example, the transmission device 805 is a Radio Frequency (RF) module, which is used for communicating with the internet in a wireless manner.
Among them, the memory 803 is used to store an application program, in particular.
The processor 801 may call an application stored in the memory 803 via the transmission means 805 to perform the following steps: detecting a target operation instruction; forwarding the target operation instruction to a target model loader in the plurality of reference model loaders according to load values of the plurality of reference model loaders, wherein the target operation instruction is used for instructing the target model loader to execute a target operation corresponding to the target operation instruction.
The processor 801 may also call an application program stored in the memory 803 via the transmission means 805 to perform the following steps: receiving a target operation instruction forwarded by a model scheduler, wherein the model scheduler is used for forwarding the target operation instruction to a target model loader in a plurality of reference model loaders according to load values of the plurality of reference model loaders; calling a corresponding target model file according to the target operation instruction; and under the condition that the target model file meets the set requirement, executing target operation corresponding to the target operation instruction on the target model file.
The embodiment of the application provides a model management system, a target operation instruction processing method and a target operation instruction processing device. The method comprises the steps of setting a plurality of reference model loaders which are responsible for executing operation on a target model file, connecting the reference model loaders with a model scheduler and sending the load condition of the reference model loaders to the model scheduler, determining the target model loader of a forwarded target operation instruction according to the load value sent by each reference model loader after the model scheduler detects the target operation instruction, sending the target operation instruction to the target model loader, executing corresponding target operation on the target model file after the target model loader receives the target operation instruction sent by the model scheduler, achieving the purpose of forwarding the target operation instruction according to the load value of each reference model loader, enabling the target operation instruction received by the target model loader to be matched with the load value of the target operation instruction, and further achieving the technical effect of improving the processing efficiency of the operation instruction, and the technical problem of low processing efficiency of the operation instruction is solved.
Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment is not described herein again.
It will be understood by those skilled in the art that the structure shown in fig. 8 is merely an illustration, and the electronic device may be a smart phone (e.g., an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, and a Mobile Internet Device (MID), a PAD, etc. Fig. 8 is a diagram illustrating a structure of the electronic device. For example, the electronic device may also include more or fewer components (e.g., network interfaces, display devices, etc.) than shown in FIG. 8, or have a different configuration than shown in FIG. 8.
Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by a program for instructing hardware associated with an electronic device, where the program may be stored in a computer-readable storage medium, and the storage medium may include: flash disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.
Embodiments of the present application also provide a storage medium. Alternatively, in this embodiment, the storage medium may be used to execute a program code of a processing method of the target operation instruction.
Optionally, in this embodiment, the storage medium may be located on at least one of a plurality of network devices in a network shown in the above embodiment.
Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: detecting a target operation instruction; forwarding the target operation instruction to a target model loader in the plurality of reference model loaders according to load values of the plurality of reference model loaders, wherein the target operation instruction is used for instructing the target model loader to execute a target operation corresponding to the target operation instruction.
Optionally, in this embodiment, the storage medium is further configured to store program code for performing the following steps: receiving a target operation instruction forwarded by a model scheduler, wherein the model scheduler is used for forwarding the target operation instruction to a target model loader in a plurality of reference model loaders according to load values of the plurality of reference model loaders; calling a corresponding target model file according to the target operation instruction; and under the condition that the target model file meets the set requirement, executing target operation corresponding to the target operation instruction on the target model file.
Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment is not described herein again.
Optionally, in this embodiment, the storage medium may include, but is not limited to: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.
The integrated unit in the above embodiments, if implemented in the form of a software functional unit and sold or used as a separate product, may be stored in the above computer-readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or a part of or all or part of the technical solution contributing to the prior art may be embodied in the form of a software product stored in a storage medium, and including instructions for causing one or more computer devices (which may be personal computers, servers, network devices, or the like) to execute all or part of the steps of the method described in the embodiments of the present application.
In the above embodiments of the present application, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the several embodiments provided in the present application, it should be understood that the disclosed client may be implemented in other manners. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one type of division of logical functions, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The foregoing is only a preferred embodiment of the present application and it should be noted that those skilled in the art can make several improvements and modifications without departing from the principle of the present application, and these improvements and modifications should also be considered as the protection scope of the present application.
Claims (13)
1. A model management system, comprising a model scheduler and a plurality of reference model loaders:
the model scheduler is used for forwarding a target operation instruction to a target model loader in the plurality of reference model loaders according to the load value of each reference model loader after the target operation instruction is detected;
and the target model loader is connected with the model scheduler and used for receiving the target operation instruction forwarded by the model scheduler and executing corresponding target operation on a target model file according to the target operation instruction.
2. The system of claim 1, wherein the model management system further comprises a model translator, wherein:
the model converter is used for carrying out target processing on the file format of the initial model file so as to obtain the target model file in the target format.
3. The system of claim 1, wherein the model management system further comprises a model storage center, a target database, and a model management component, wherein:
the model storage center is used for storing the target model file;
the model management component is used for determining a storage path of the target model file in the model storage center, determining meta information of the target model file containing the storage path, and sending the meta information to the target database for storage.
4. The system of claim 3, wherein the target model loader is further configured to:
acquiring the meta information based on the target operation instruction;
retrieving the target model file based on the storage path included in the meta information;
and verifying the target model file according to the initial verification information included in the meta information, and executing corresponding target operation on the target model file according to the target operation instruction when a verification result meets a set requirement.
5. A method for processing a target operation instruction is characterized by comprising the following steps:
detecting a target operation instruction;
forwarding the target operation instruction to a target model loader in the plurality of reference model loaders according to load values of the plurality of reference model loaders, wherein the target operation instruction is used for instructing the target model loader to execute a target operation corresponding to the target operation instruction.
6. The method of claim 5, wherein forwarding the target operation instruction to the target model loader of the plurality of reference model loaders based on the load values of the plurality of reference model loaders comprises:
obtaining the load value of each reference model loader;
comparing each load value to determine a minimum load value;
determining the reference model loader corresponding to the minimum load value as the target model loader;
and forwarding the target operation instruction to the target model loader.
7. A method for processing a target operation instruction is characterized by comprising the following steps:
receiving a target operation instruction forwarded by a model scheduler, wherein the model scheduler is used for forwarding the target operation instruction to a target model loader in a plurality of reference model loaders according to load values of the plurality of reference model loaders;
calling a corresponding target model file according to the target operation instruction;
and under the condition that the target model file meets the set requirement, executing target operation corresponding to the target operation instruction on the target model file.
8. The method of claim 7, wherein invoking the corresponding target model file according to the target operation instruction comprises:
inquiring meta information corresponding to the target operation instruction according to the target operation instruction, wherein the meta information is stored in a target database and comprises a storage path of the target model file;
and calling the target model file according to the storage path of the target model file in the meta information.
9. The method of claim 8, wherein prior to performing a target operation corresponding to the operation instruction on the target model file, the method further comprises:
performing target calculation on the target model file to obtain corresponding target verification information;
verifying the target verification information according to initial verification information included in the meta information to obtain a verification result;
and when the verification result indicates that the initial verification information is consistent with the target verification information, determining that the target model file meets the set requirement.
10. An apparatus for forwarding a target operation instruction, comprising:
the detection module is used for detecting a target operation instruction;
and the forwarding module is used for forwarding the target operation instruction to a target model loader in the plurality of reference model loaders according to the load values of the plurality of reference model loaders, wherein the target operation instruction is used for instructing the target model loader to execute a target operation corresponding to the target operation instruction.
11. An apparatus for processing a target operation instruction, comprising:
the model scheduler is used for forwarding the target operation instruction to a target model loader in a plurality of reference model loaders according to load values of the plurality of reference model loaders;
the calling module is used for calling a corresponding target model file according to the target operation instruction;
and the processing module is used for executing target operation corresponding to the target operation instruction on the target model file under the condition that the target model file meets the set requirement.
12. A storage medium, comprising a stored program, wherein the program when executed performs the method of any one of claims 5 to 9.
13. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the method of any of the preceding claims 5 to 9 by means of the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011412388.8A CN112559124A (en) | 2020-12-04 | 2020-12-04 | Model management system and target operation instruction processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011412388.8A CN112559124A (en) | 2020-12-04 | 2020-12-04 | Model management system and target operation instruction processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112559124A true CN112559124A (en) | 2021-03-26 |
Family
ID=75048846
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011412388.8A Pending CN112559124A (en) | 2020-12-04 | 2020-12-04 | Model management system and target operation instruction processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112559124A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113094331A (en) * | 2021-03-30 | 2021-07-09 | 联想(北京)有限公司 | Processing method and device |
| CN113176906A (en) * | 2021-04-25 | 2021-07-27 | 京东数字科技控股股份有限公司 | Model management method, model management device, computer equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180285494A1 (en) * | 2017-02-22 | 2018-10-04 | Woodside Energy Technologies Pty Ltd | Model management system |
| CN111324379A (en) * | 2020-01-15 | 2020-06-23 | 携程旅游网络技术（上海）有限公司 | Model deployment system based on general SOA service |
| CN111988429A (en) * | 2020-09-01 | 2020-11-24 | 深圳壹账通智能科技有限公司 | Algorithm scheduling method and system |
-
2020
- 2020-12-04 CN CN202011412388.8A patent/CN112559124A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180285494A1 (en) * | 2017-02-22 | 2018-10-04 | Woodside Energy Technologies Pty Ltd | Model management system |
| CN111324379A (en) * | 2020-01-15 | 2020-06-23 | 携程旅游网络技术（上海）有限公司 | Model deployment system based on general SOA service |
| CN111988429A (en) * | 2020-09-01 | 2020-11-24 | 深圳壹账通智能科技有限公司 | Algorithm scheduling method and system |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113094331A (en) * | 2021-03-30 | 2021-07-09 | 联想(北京)有限公司 | Processing method and device |
| CN113176906A (en) * | 2021-04-25 | 2021-07-27 | 京东数字科技控股股份有限公司 | Model management method, model management device, computer equipment and storage medium |
| CN113176906B (en) * | 2021-04-25 | 2024-05-17 | 京东科技控股股份有限公司 | Model management method, device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3333704A1 (en) | Method and apparatus for repairing kernel vulnerability | |
| CN104246767A (en) | Telemetry system for a cloud synchronization system | |
| CN109492181B (en) | Page jump method, device, computer equipment and storage medium | |
| CN108255708B (en) | Method, device, storage medium and equipment for accessing production file in test environment | |
| CN113505520A (en) | Method, device and system for supporting heterogeneous federated learning | |
| CN112559124A (en) | Model management system and target operation instruction processing method and device | |
| CN113691635B (en) | Method and device for calling microservice, electronic equipment and readable storage medium | |
| CN111988429A (en) | Algorithm scheduling method and system | |
| CN111651140A (en) | Service method and device based on workflow | |
| CN107526584B (en) | Data access method and device | |
| CN109816453B (en) | Detection method and device for promotion resource link | |
| CN108647139B (en) | System test method, device, storage medium and electronic device | |
| CN112711466B (en) | Hanging affair inspection method and device, electronic equipment and storage medium | |
| CN109814911A (en) | Method, apparatus, computer equipment and storage medium for Manage Scripts program | |
| CN112367205B (en) | Processing method and scheduling system for HTTP scheduling request | |
| CN111190637B (en) | Version file release management method, device and system | |
| CN114282940A (en) | Method and apparatus for intention recognition, storage medium, and electronic device | |
| CN114610446A (en) | Method, device and system for automatically injecting probe | |
| CN114003457A (en) | Data acquisition method and device, storage medium and electronic equipment | |
| CN113326052A (en) | Method and device for upgrading service component, computer equipment and storage medium | |
| CN113239297A (en) | Message pushing method, system and storage medium | |
| CN112965767A (en) | Job processing method and apparatus, electronic device, and storage medium | |
| CN114675872A (en) | Data processing method, device and equipment for application program and storage medium | |
| CN111026466A (en) | File processing method and device, computer readable storage medium and electronic equipment | |
| CN101452390A (en) | Network boot method and system, and system boot method and apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |