CN112214285A - Docker-based model service deployment system - Google Patents
Docker-based model service deployment system Download PDFInfo
- Publication number
- CN112214285A CN112214285A CN202011135150.5A CN202011135150A CN112214285A CN 112214285 A CN112214285 A CN 112214285A CN 202011135150 A CN202011135150 A CN 202011135150A CN 112214285 A CN112214285 A CN 112214285A
- Authority
- CN
- China
- Prior art keywords
- model
- service
- management module
- mirror image
- docker
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5077—Logical partitioning of resources; Management or configuration of virtualized resources
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45562—Creating, deleting, cloning virtual machine instances
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45595—Network integration; Enabling network access in virtual machine instances
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention relates to a model service deployment system based on Docker, which comprises a model warehouse management module, a mirror image library management module, a resource quota management module, a service management module and a service monitoring module, wherein the model warehouse management module is used for managing the resource quota; the model warehouse management module is used for uniformly managing the models; the mirror image library management module is used for providing basic environment mirror image support for model training or service release; the resource quota management module is used for managing the running resources of each model service and can uniformly schedule various types of cluster resources to support heterogeneous hardware resource scheduling; the service management module is used for deploying the model and managing the life cycle of the model according to the model management mode and the resource quota management mode output by the model warehouse management module and the resource quota management module; the service monitoring module is used for monitoring the operation condition and service record of the model service in real time. The invention can more conveniently deploy the model service and realize the high availability and multi-type model support of the model service.
Description
Technical Field
The invention relates to the field of artificial intelligence, in particular to a model service deployment system based on Docker.
Background
With the development of artificial intelligence, machine learning and deep learning related technologies are applied to many industries and fields. Model deployment artificial intelligence is a very important link in application, modeling is difficult, and deployment is more difficult. For model deployment, single-service manual deployment is common at present, and shutdown updating is needed for model service updating in the later period; the model service operation resources cannot be managed and controlled, and the server operation resources cannot be reasonably distributed; the access rights of various model services cannot be uniformly controlled, and the running state and the access condition of the services cannot be monitored. In order to solve the defects, some model service management systems appear in the industry, model service authentication, service state monitoring and the like can be uniformly managed, and the problem of model service deployment management is solved to a certain extent. However, these systems have some disadvantages, such as that the deployment resources cannot be controlled in fine granularity, flexible expansion is not supported, and gray scale distribution is not supported.
Disclosure of Invention
In order to solve the problems, the invention provides a model service deployment system based on Docker.
The specific scheme is as follows:
a model service deployment system based on Docker comprises a model warehouse management module, a mirror image library management module, a resource quota management module, a service management module and a service monitoring module;
the model warehouse management module is used for uniformly managing the models, and the models are acquired in one or more modes of uploading through local model files and transmitting the models through a butted training platform;
the mirror image library management module is used for providing basic environment mirror image support for model training or service release;
the resource quota management module is used for managing the running resources of each model service and can uniformly schedule various types of cluster resources to support heterogeneous hardware resource scheduling;
the service management module is used for deploying the model and managing the life cycle of the model according to the model management mode and the resource quota management mode output by the model warehouse management module and the resource quota management module;
the service monitoring module is used for monitoring the operation condition and service record of the model service in real time.
Further, the functions of the model warehouse management module further include: online model compression conversion, model version management and one-key service release.
Further, the basic environment image comprises a built-in image and a user-defined image, the built-in image comprises a dependent environment of the fixed frame, and the user-defined image comprises a user-defined environment image except the dependent environment of the fixed frame.
Further, the user-defined mirror image supports the construction in a mode of uploading Dockerfile files or the construction through the selection of online environment mirror image options.
Further, the resource quota management module has functions of resource application, resource approval, resource monitoring and flexible expansion capacity.
Further, the deployment of the model in the service management module supports the deployment of a built-in fixed framework and a user-defined framework.
Furthermore, the service monitoring module provides a visual interface and monitors the performance of the service management module after the service management module deploys each model service and the operation condition of each model service in real time.
Furthermore, the real-time monitored operation status of each model service includes deployment resource usage, request information data, service response data, and service authentication information statistical data.
By adopting the technical scheme, the model service can be more conveniently deployed, and high-availability and multi-type model support of the model service is realized.
Drawings
FIG. 1 is a block diagram of a system according to an embodiment of the present invention.
Detailed Description
The invention is further illustrated by the accompanying drawings. The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the embodiments. Those skilled in the art will appreciate still other possible embodiments and advantages of the present invention with reference to these figures.
The invention will now be further described with reference to the accompanying drawings and detailed description.
The first embodiment is as follows:
an embodiment of the present invention provides a model service deployment system based on a Docker, as shown in fig. 1, the system includes a model warehouse management module, a mirror library management module, a resource quota management module, a service management module, and a service monitoring module, where:
(1) the model warehouse management module is used for uniformly managing the models, and the models are acquired in one or more modes of uploading through local model files and transmitting the models through a butted training platform.
The model is generated by machine learning or deep learning training.
The model warehouse management module comprises an external model operation interface and is in butt joint with other training platforms through the external model operation interface.
The functions of the model warehouse management module further comprise: online model compression conversion, model version management, one-key service release, and the like.
(2) The mirror image library management module is used for providing basic environment mirror image support for model training or service release so as to realize flexible framework environment support.
The basic environment mirror image comprises a built-in mirror image and a user-defined mirror image, wherein the built-in mirror image comprises a dependent environment of a fixed frame, such as a common frame: sklearn, XGboost, LigthGBM, SparkML, TensorFlow, caffe, Theano, Keras, Pythroch, MXNet, and the like. And if the built-in mirror image does not meet the requirements of the user, the user can configure the user-defined mirror image on line. The user-customized image comprises a user-customized environment image except for the dependent environment of the fixed framework. The user-defined mirror image supports the construction in a mode of uploading Dockerfile files or the construction through the selection of online environment mirror image options.
(3) The resource quota management module is used for managing the running resources of each model service, and can uniformly schedule various types of cluster resources to support heterogeneous hardware resource scheduling. The functions of the resource quota management module in this embodiment include resource application, resource approval, resource monitoring, elastic expansion and contraction, and the like, and the operating resources of each model service can be flexibly controlled in a fine-grained manner.
(4) The service management module is used for deploying the model and managing the life cycle of the model according to the model management mode and the resource quota management mode output by the model warehouse management module and the resource quota management module.
The deployment of the model has high reliability and expandability, and supports built-in fixed frameworks such as PMML, Sklearn, XGboost, LigthGBM and SparkML; deployment of ONNX, TensorFlow, cafe, Theano, Keras, Pythroch, MXNet and user-defined frameworks.
The service management module can provide functions of one-stop model service release management, service authentication management, service version control, gray release, online real-time prediction, batch prediction, model evaluation and the like, wherein the service authentication management supports control of service interface access limitation according to days and access quantity.
(5) The service monitoring module is used for monitoring the operation condition and service record of the model service in real time.
The service monitoring module provides a visual interface and monitors the performance of each model service after the service management module deploys each model service and the running condition of each model service in real time. The real-time monitored operation condition of each model service comprises deployment resource use condition, request information data, service response data and service authentication information statistical data.
The embodiment of the invention provides a visual one-stop model version control system, which can more conveniently deploy model services by the purposes of model service version management, monitoring deployment performance, operation condition and web service indexes, and realizes high availability of the model services and multi-type model support.
While the invention has been particularly shown and described with reference to a preferred embodiment, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (8)
1. A model service deployment system based on Docker is characterized by comprising a model warehouse management module, a mirror image library management module, a resource quota management module, a service management module and a service monitoring module;
the model warehouse management module is used for uniformly managing the models, and the models are acquired in one or more modes of uploading through local model files and transmitting the models through a butted training platform;
the mirror image library management module is used for providing basic environment mirror image support for model training or service release;
the resource quota management module is used for managing the running resources of each model service and can uniformly schedule various types of cluster resources to support heterogeneous hardware resource scheduling;
the service management module is used for deploying the model and managing the life cycle of the model according to the model management mode and the resource quota management mode output by the model warehouse management module and the resource quota management module;
the service monitoring module is used for monitoring the operation condition and service record of the model service in real time.
2. The Docker-based model service deployment system of claim 1, wherein: the functions of the model warehouse management module further comprise: online model compression conversion, model version management and one-key service release.
3. The Docker-based model service deployment system of claim 1, wherein: the basic environment mirror image comprises a built-in mirror image and a user-defined mirror image, the built-in mirror image comprises a dependent environment of the fixed frame, and the user-defined mirror image comprises a user-defined environment mirror image except the dependent environment of the fixed frame.
4. The Docker-based model service deployment system of claim 3, wherein: the user-defined mirror image supports the construction in a mode of uploading Dockerfile files or the construction through the selection of online environment mirror image options.
5. The Docker-based model service deployment system of claim 1, wherein: the resource quota management module has the functions of resource application, resource approval, resource monitoring and elastic expansion and contraction capacity.
6. The Docker-based model service deployment system of claim 1, wherein: the deployment of the model in the service management module supports the deployment of a built-in fixed framework and a user-defined framework.
7. The Docker-based model service deployment system of claim 1, wherein: the service monitoring module provides a visual interface and monitors the performance of each model service after the service management module deploys each model service and the running condition of each model service in real time.
8. The Docker-based model service deployment system of claim 7, wherein: the real-time monitored operation condition of each model service comprises deployment resource use condition, request information data, service response data and service authentication information statistical data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011135150.5A CN112214285A (en) | 2020-10-22 | 2020-10-22 | Docker-based model service deployment system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011135150.5A CN112214285A (en) | 2020-10-22 | 2020-10-22 | Docker-based model service deployment system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112214285A true CN112214285A (en) | 2021-01-12 |
Family
ID=74056306
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011135150.5A Pending CN112214285A (en) | 2020-10-22 | 2020-10-22 | Docker-based model service deployment system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112214285A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113283855A (en) * | 2021-04-29 | 2021-08-20 | 成都商高智能科技有限公司 | Recruitment system based on containerized resource programming |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014058411A1 (en) * | 2012-10-08 | 2014-04-17 | Hewlett-Packard Development Company, L.P. | Hybrid cloud environment |
| WO2017045424A1 (en) * | 2015-09-18 | 2017-03-23 | 乐视控股（北京）有限公司 | Application program deployment system and deployment method |
| CN109508238A (en) * | 2019-01-05 | 2019-03-22 | 咪付（广西）网络技术有限公司 | A kind of resource management system and method for deep learning |
| CN109600269A (en) * | 2019-01-21 | 2019-04-09 | 云南电网有限责任公司信息中心 | A kind of cloud management platform based on DCOS |
| US20190171438A1 (en) * | 2017-12-05 | 2019-06-06 | Archemy, Inc. | Active adaptation of networked compute devices using vetted reusable software components |
| CN110413294A (en) * | 2019-08-06 | 2019-11-05 | 中国工商银行股份有限公司 | Service delivery system, method, apparatus and equipment |
| CN111414233A (en) * | 2020-03-20 | 2020-07-14 | 京东数字科技控股有限公司 | Online model reasoning system |
| CN111629061A (en) * | 2020-05-28 | 2020-09-04 | 苏州浪潮智能科技有限公司 | Inference service system based on Kubernetes |
-
2020
- 2020-10-22 CN CN202011135150.5A patent/CN112214285A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014058411A1 (en) * | 2012-10-08 | 2014-04-17 | Hewlett-Packard Development Company, L.P. | Hybrid cloud environment |
| WO2017045424A1 (en) * | 2015-09-18 | 2017-03-23 | 乐视控股（北京）有限公司 | Application program deployment system and deployment method |
| US20190171438A1 (en) * | 2017-12-05 | 2019-06-06 | Archemy, Inc. | Active adaptation of networked compute devices using vetted reusable software components |
| CN109508238A (en) * | 2019-01-05 | 2019-03-22 | 咪付（广西）网络技术有限公司 | A kind of resource management system and method for deep learning |
| CN109600269A (en) * | 2019-01-21 | 2019-04-09 | 云南电网有限责任公司信息中心 | A kind of cloud management platform based on DCOS |
| CN110413294A (en) * | 2019-08-06 | 2019-11-05 | 中国工商银行股份有限公司 | Service delivery system, method, apparatus and equipment |
| CN111414233A (en) * | 2020-03-20 | 2020-07-14 | 京东数字科技控股有限公司 | Online model reasoning system |
| CN111629061A (en) * | 2020-05-28 | 2020-09-04 | 苏州浪潮智能科技有限公司 | Inference service system based on Kubernetes |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113283855A (en) * | 2021-04-29 | 2021-08-20 | 成都商高智能科技有限公司 | Recruitment system based on containerized resource programming |
| CN113283855B (en) * | 2021-04-29 | 2024-01-30 | 成都商高智能科技有限公司 | Recruitment system based on containerized resource programming |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| RU2737566C2 (en) | Methods, systems and device for dynamic provision of unlimited system administration with operating configuration m:n of high availability | |
| JP4809772B2 (en) | Management based on computer system and distributed application model | |
| CN110109686B (en) | Application operation and maintenance method and system based on container management engine | |
| CN1407441B (en) | System and method for automatic management computer service and programmable device | |
| CN116170316A (en) | Network system, instance management and control method, device and storage medium | |
| CN104508627A (en) | Hybrid cloud environment | |
| CN102819478B (en) | A kind of data handling system monitoring and management method without agency | |
| CN106657167B (en) | Management server, server cluster, and management method | |
| CN111143044B (en) | Task scheduling management system, method, device and storage medium thereof | |
| CN111930706B (en) | Remote call-based distributed network file storage system and method | |
| CN111984270A (en) | Application deployment method and system | |
| CN112114939A (en) | Distributed system deployment equipment and method | |
| US20220308559A1 (en) | Controlling an Industrial Process Using Virtualized Instances of Control Software | |
| CN112214285A (en) | Docker-based model service deployment system | |
| Mendiratta | Reliability analysis of clustered computing systems | |
| CN114090192A (en) | Docker Swarm cluster resource cleaning optimization method and system | |
| CN114489989A (en) | Method and system for parallel scheduling based on proxy client | |
| Achtaich et al. | Designing a framework for smart IoT adaptations | |
| CN112306640A (en) | Container dispensing method, apparatus, device and medium therefor | |
| CN114489761B (en) | Service integration and application integration method based on container cluster | |
| CN115048158A (en) | Process arranging and calling method, system and computer equipment thereof | |
| CN113986662A (en) | Edge cluster monitoring method and system | |
| CN113010531A (en) | Block chain BAAS system task scheduling framework based on directed acyclic graph | |
| CN101364224A (en) | Information management system and method | |
| CN111291101A (en) | Cluster management method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |