CN111045697A - Automatic rapid deployment method and system - Google Patents
Automatic rapid deployment method and system Download PDFInfo
- Publication number
- CN111045697A CN111045697A CN201911298407.6A CN201911298407A CN111045697A CN 111045697 A CN111045697 A CN 111045697A CN 201911298407 A CN201911298407 A CN 201911298407A CN 111045697 A CN111045697 A CN 111045697A
- Authority
- CN
- China
- Prior art keywords
- distributed storage
- rapid deployment
- storage system
- deployment method
- deployment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000013515 script Methods 0.000 claims abstract description 18
- 238000009434 installation Methods 0.000 claims abstract description 8
- 238000012544 monitoring process Methods 0.000 claims description 18
- 238000012423 maintenance Methods 0.000 claims description 15
- 230000000007 visual effect Effects 0.000 claims description 6
- 230000008569 process Effects 0.000 description 20
- 230000008901 benefit Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 5
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005111 flow chemistry technique Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/60—Software deployment
- G06F8/61—Installation
- G06F8/63—Image based installation; Cloning; Build to order
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45504—Abstract machines for programme code execution, e.g. Java virtual machine [JVM], interpreters, emulators
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45562—Creating, deleting, cloning virtual machine instances
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Stored Programmes (AREA)
Abstract
The invention provides an automatic rapid deployment method and system. The method is operated on a deployment node of a distributed storage system and comprises the following steps: and deploying the component mirror image running in the container on each cluster node of the distributed storage system according to the installation script and the component starting script through an automatic configuration deployment suite. The invention provides an automatic rapid deployment method and system, which realize the automatic rapid deployment of a distributed storage system through component mirroring and distribution and deployment according to configured components.
Description
Technical Field
The invention relates to the technical field of distributed systems, in particular to an automatic rapid deployment method and system.
Background
Docker is a lightweight application container framework, which can deploy and manage any application quickly and conveniently through a container. The container image generation file is a script composed of a series of commands and parameters, the commands are applied to the basic image and finally a new image is created, and the commands simplify the flow from beginning to end and greatly simplify the deployment work. The container image generation file contains various methods, commands, and parameters, which yield a new image that can be used to create the container.
An automated configuration deployment suite provides a tool for publishing, managing, and orchestrating computer systems in the simplest way, and deployment can be accomplished in minutes. The automatic configuration deployment suite is a model-driven configuration manager and supports multi-node issuing and remote task execution. Extensions can be made using a variety of programming languages without the need to install additional software on the managed node.
As each service of the distributed storage system is distributed on different server nodes, the deployment, operation and maintenance of software modules and process systems become more complicated along with the increase of the nodes. In the aspect of service deployment and upgrade, operation and maintenance personnel need to be specially maintained, and the labor cost is huge. The distributed storage system has a lot of services, and monitoring and alarming of the distributed storage system are inconvenient. Therefore, there is a need for an automated deployment and maintenance system that manages the resources that monitor the various servers. Although the existing distributed system based on Docker has the advantage that the service environment is easy to be packaged and deployed, the system is lack of real-time monitoring and operation and maintenance.
Disclosure of Invention
The technical problem to be solved by the invention is to provide an automatic rapid deployment method and system, which realize the automatic rapid deployment of a distributed storage system through component mirroring and distribution and deployment according to configured components.
In order to solve the above technical problem, the present invention provides an automated rapid deployment method, where the method is executed on a deployment node of a distributed storage system, and includes: and deploying the component mirror image running in the container on each cluster node of the distributed storage system according to the installation script and the component starting script through an automatic configuration deployment suite.
In some embodiments, further comprising: after component images running in the container are deployed on each node of the distributed storage system, cluster state configuration information of the distributed storage system is generated.
In some embodiments, further comprising: and according to the installation script and the component starting script, deploying an declarative API arrangement system on each node of the distributed storage system.
In some embodiments, further comprising: and the declarative API arrangement system automatically monitors and operates and maintains the distributed storage system according to the cluster state configuration information.
In some embodiments, automated monitoring and automated operations enable second-level container status monitoring and status information synchronization.
In some embodiments, the declarative API orchestration system automatically monitors and automatically maintains the distributed storage system according to the cluster state configuration information, including: a file in the format of yaml is formed as reference configuration information of the current declarative API service.
In some embodiments, the component mirroring comprises: the system comprises a core monitoring component mirror image, a metadata management mirror image, a hard disk management mirror image, a block device management mirror image, a file system management mirror image and an object gateway management mirror image.
In some embodiments, further comprising: and performing visual monitoring and visual management on the distributed system through a web management terminal deployed on the cluster node.
In some embodiments, the container comprises: a docker vessel.
In addition, the invention also provides an automatic rapid deployment system, which is integrated on the deployment node of the distributed storage system, and comprises: one or more processors; a storage device for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement the automated rapid deployment method according to the foregoing.
After adopting such design, the invention has at least the following advantages:
1. the components of the distributed storage system are mirrored based on the Docker container, so that the deployment process of each component is simplified, errors in the deployment process of the components are avoided, the complexity of deployment, operation and maintenance is reduced, and the deployment and management of each component of the cluster system are facilitated;
2. an automatic configuration deployment suite is adopted to configure the overall architecture of the system, the starting of multiple components, the flow processing and the like, so that the rapid and automatic deployment of the distributed storage system is realized;
3. the life cycle of the system component is arranged and managed through the declarative API, the second-level container state monitoring and the state information synchronization are realized, the system component state change can be rapidly sensed and automatically responded, and meanwhile, manual intervention and adjustment of the cluster on a web interface are supported.
Drawings
The foregoing is only an overview of the technical solutions of the present invention, and in order to make the technical solutions of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and the detailed description.
FIG. 1 is a flow chart of an automated rapid deployment method provided by an embodiment of the invention;
FIG. 2 is a schematic diagram of a distributed storage system provided by an embodiment of the present invention;
fig. 3 is a structural diagram of an automated rapid deployment system according to an embodiment of the present invention.
Detailed Description
The preferred embodiments of the present invention will be described in conjunction with the accompanying drawings, and it will be understood that they are described herein for the purpose of illustration and explanation and not limitation.
Fig. 1 shows an execution flow of the automated rapid deployment method provided by the present invention. Referring to fig. 1, the automated rapid deployment method includes:
s110, the automatic configuration deployment suite distributes and deploys the Docker containerized storage components through predefined host inventory and playbook configurations.
And S120, generating cluster state configuration information after the storage cluster deployment is finished.
S130, the automatic configuration deployment suite deploys the declarative API orchestration system through predefined host inventory and playbook configurations.
S140, the declarative API arrangement system automatically reads the cluster state configuration information and starts the step S150 to start the automatic operation and maintenance for monitoring the distributed storage system cluster.
And S160 and S170 belong to the web management part and are divided into web visual monitoring and web visual management operation and maintenance, an administrator triggers S120 to automatically generate new cluster state configuration information through the configuration operation and maintenance of S170, and then the clusters S140 and S150 form a virtuous circle to realize the process of automatic operation and maintenance.
The main core of the invention comprises three parts, namely mirroring of the distributed storage components, distribution and deployment of mirroring, management scripts and management software, and arranging and managing the life cycle of each component of the distributed storage through a declarative API.
Mirroring of distributed storage components: the distributed storage process comprises a core monitoring process, a metadata management process, a hard disk management process, a block device management process, a file system management process, an object gateway management process and the like. All these processes require installation of a distributed storage component software package in the operating system, modification of the configuration file to which the distributed storage component belongs, and startup management of processes associated with the distributed storage component. The software packages required by each distributed storage process are all mirrored respectively through the container mirror image generation file, the container mirror image generation file of each mirror image is added into the installation script and the component starting script of the related component software package, the state management of the process in the operating system is replaced through the starting and stopping of the container, and the communication management among the components is coordinated through the mode of the appointed port mapping in the container mirror image generation file.
Mirroring and managing scripts, managing the distribution and deployment of software: an automatic configuration deployment kit is deployed on one node, a mirror image and a script are uploaded, and the architecture of the whole distributed storage cluster is described by compiling configuration files host inventory and playbook of the automatic configuration deployment kit, wherein the architecture comprises a designated core monitoring container, a metadata management container, a hard disk management container, a block device management container, a file system management container, an object gateway management container and the like which are respectively deployed on nodes of the cluster, the starting sequence of the containers, relevant error reporting processing flows and log records.
Managing the lifecycle of the components of the distributed storage by the declarative API orchestration software: the scheduling software is deployed at each node of the distributed cluster, can realize the monitoring of the second-level container state and the state information synchronization, forms a file in a yaml format as the reference configuration information of the current declarative API service, and when a certain container component of the distributed storage reports an error and exits or does not respond to the communication with the core monitoring component, the declarative API scheduling software can automatically restart or generate a new container component in the second level according to the configuration in the yaml file and is integrated with a web management interface of the distributed storage cluster, so that the manual intervention and adjustment of an administrator on the cluster are realized.
Deployment and operation and maintenance processes: the method comprises the steps of installing Docker software on all nodes, installing an automatic configuration deployment suite on deployment nodes, compiling configuration information of the automatic configuration deployment suite, adding cluster node information, cluster component architecture information and the like, uploading declarative API layout software and various distributed storage mirror image components, starting the automatic configuration deployment suite, automatically finishing the deployment process of the distributed storage cluster, and enabling the declarative API layout software to automatically manage the life cycle of various components of the distributed storage cluster and be manually controlled through a web management interface.
Compared with other containerized deployment and operation and maintenance distributed storage clusters, the invention has the advantages that:
the method has the following advantages: the automation configuration deployment suite, the storage cluster and the orchestration system are decoupled, for example, normal use of the storage cluster is not affected by faults of the orchestration system.
The advantages are as follows: the arrangement system adopts the declarative API, and compared with most command API arrangement system modes, the complexity of system construction and operation and maintenance is reduced.
Fig. 2 shows a network architecture of a network to which the automated rapid deployment method provided by the present invention is applied. Referring to fig. 2, the network includes at least one deployment node, and a number of cluster nodes. Docker software and declarative API orchestration software are installed on both deployment nodes and cluster nodes. By means of the Docker software, each mirrored component has a uniform running environment. By means of the declarative API arrangement software, the complexity of system construction and operation maintenance is greatly reduced.
In addition, an automation configuration deployment suite is also installed on the deployment node. The automatic configuration deployment suite deploys the mirrored components according to the installation script and the component starting script, namely host inventoryy and playbook. Therefore, the components are mirrored, automation of configuration and deployment operation of the mirrored components is realized, and the deployment process of the components is greatly simplified.
Fig. 3 shows the structure of an automated rapid deployment system. Referring to fig. 3, an automated rapid deployment system includes: a Central Processing Unit (CPU)301 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) or a program loaded from a storage section 308 into a Random Access Memory (RAM) 303. In the RAM 303, various programs and data necessary for system operation are also stored. The CPU 301, ROM 302, and RAM 303 are connected to each other via a bus 304. An input/output (I/O) interface 305 is also connected to bus 304.
The following components are connected to the I/O interface 305: an input portion 306 including a keyboard, a mouse, and the like; an output section 307 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 308 including a hard disk and the like; and a communication section 309 including a network interface card such as a LAN card, a modem, or the like. The communication section 309 performs communication processing via a network such as the internet. A drive 310 is also connected to the I/O interface 305 as needed. A removable medium 311 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 310 as necessary, so that a computer program read out therefrom is mounted into the storage section 308 as necessary.
The technical scheme of the invention has the following beneficial effects:
1. the components of the distributed storage system are mirrored based on the Docker container, so that the deployment process of each component is simplified, errors in the deployment process of the components are avoided, the complexity of deployment, operation and maintenance is reduced, and the deployment and management of each component of the cluster system are facilitated;
2. an automatic configuration deployment suite is adopted to configure the overall architecture of the system, the starting of multiple components, the flow processing and the like, so that the rapid and automatic deployment of the distributed storage system is realized;
3. the life cycle of the system component is arranged and managed through the declarative API, the second-level container state monitoring and the state information synchronization are realized, the system component state change can be rapidly sensed and automatically responded, and meanwhile, manual intervention and adjustment of the cluster on a web interface are supported.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the present invention in any way, and it will be apparent to those skilled in the art that the above description of the present invention can be applied to various modifications, equivalent variations or modifications without departing from the spirit and scope of the present invention.
Claims (10)
1. An automated rapid deployment method, which runs on deployment nodes of a distributed storage system, is characterized by comprising:
and deploying the component mirror image running in the container on each cluster node of the distributed storage system according to the installation script and the component starting script through an automatic configuration deployment suite.
2. The automated rapid deployment method of claim 1, further comprising:
after component images running in the container are deployed on each node of the distributed storage system, cluster state configuration information of the distributed storage system is generated.
3. The automated rapid deployment method of claim 1, further comprising:
and according to the installation script and the component starting script, deploying an declarative API arrangement system on each node of the distributed storage system.
4. The automated rapid deployment method of claim 2 or 3, further comprising:
and the declarative API arrangement system automatically monitors and operates and maintains the distributed storage system according to the cluster state configuration information.
5. The automated rapid deployment method of claim 4, wherein the automatic monitoring and the automatic operation and maintenance are capable of implementing second-level container status monitoring and status information synchronization.
6. The automated rapid deployment method of claim 5, wherein the declarative API orchestration system automatically monitors and automatically operates and maintains the distributed storage system according to the cluster state configuration information, comprising:
a file in the format of yaml is formed as reference configuration information of the current declarative API service.
7. The automated rapid deployment method of claim 1, wherein component mirroring comprises: the system comprises a core monitoring component mirror image, a metadata management mirror image, a hard disk management mirror image, a block device management mirror image, a file system management mirror image and an object gateway management mirror image.
8. The automated rapid deployment method of claim 1, further comprising:
and performing visual monitoring and visual management on the distributed system through a web management terminal deployed on the cluster node.
9. The automated rapid deployment method of claim 1, wherein the container comprises: a docker vessel.
10. An automated rapid deployment system integrated on deployment nodes of a distributed storage system, comprising:
one or more processors;
a storage device for storing one or more programs,
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the automated rapid deployment method of any of claims 1-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911298407.6A CN111045697A (en) | 2019-12-17 | 2019-12-17 | Automatic rapid deployment method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911298407.6A CN111045697A (en) | 2019-12-17 | 2019-12-17 | Automatic rapid deployment method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111045697A true CN111045697A (en) | 2020-04-21 |
Family
ID=70236925
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911298407.6A Pending CN111045697A (en) | 2019-12-17 | 2019-12-17 | Automatic rapid deployment method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111045697A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111966365A (en) * | 2020-08-17 | 2020-11-20 | 北京一览群智数据科技有限责任公司 | Module deployment method and system |
| CN112738206A (en) * | 2020-12-25 | 2021-04-30 | 北京浪潮数据技术有限公司 | Butt joint method and assembly of distributed cluster and cloud platform |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130198719A1 (en) * | 2012-01-31 | 2013-08-01 | Red Hat Inc. | Automated deployment of an application in a computing platform |
| CN105468362A (en) * | 2015-11-17 | 2016-04-06 | 广州杰赛科技股份有限公司 | Application deployment method and cloud computing system |
| CN106325975A (en) * | 2016-09-09 | 2017-01-11 | 浪潮软件股份有限公司 | Method for automatically deploying and managing big data clusters by using Docker container |
| CN108924217A (en) * | 2018-06-29 | 2018-11-30 | 中山大学 | A kind of distribution cloud system Automation arranging method |
| CN109597626A (en) * | 2017-09-30 | 2019-04-09 | 北京国双科技有限公司 | A kind of deployment of components method and apparatus |
-
2019
- 2019-12-17 CN CN201911298407.6A patent/CN111045697A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130198719A1 (en) * | 2012-01-31 | 2013-08-01 | Red Hat Inc. | Automated deployment of an application in a computing platform |
| CN105468362A (en) * | 2015-11-17 | 2016-04-06 | 广州杰赛科技股份有限公司 | Application deployment method and cloud computing system |
| CN106325975A (en) * | 2016-09-09 | 2017-01-11 | 浪潮软件股份有限公司 | Method for automatically deploying and managing big data clusters by using Docker container |
| CN109597626A (en) * | 2017-09-30 | 2019-04-09 | 北京国双科技有限公司 | A kind of deployment of components method and apparatus |
| CN108924217A (en) * | 2018-06-29 | 2018-11-30 | 中山大学 | A kind of distribution cloud system Automation arranging method |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111966365A (en) * | 2020-08-17 | 2020-11-20 | 北京一览群智数据科技有限责任公司 | Module deployment method and system |
| CN112738206A (en) * | 2020-12-25 | 2021-04-30 | 北京浪潮数据技术有限公司 | Butt joint method and assembly of distributed cluster and cloud platform |
| CN112738206B (en) * | 2020-12-25 | 2022-08-23 | 北京浪潮数据技术有限公司 | Butt joint method and assembly of distributed cluster and cloud platform |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20120271874A1 (en) | System and method for cloud computing | |
| CN113569987A (en) | Model training method and device | |
| US20150081878A1 (en) | Describing datacenter rack information in management system | |
| CN113778486B (en) | Method, device, medium and equipment for containerization processing of code pipeline | |
| CN115220874B (en) | Kubernetes cluster deployment method, device, equipment and storage medium | |
| CN112099919B (en) | Application service visualization rapid deployment method based on cloud computing API | |
| US11354150B1 (en) | Utilizing maintenance event windows to determine placement of instances | |
| CN111324599B (en) | Block chain experiment system and management method | |
| CN111045697A (en) | Automatic rapid deployment method and system | |
| US9935830B1 (en) | Cloud management system and method for disparate computing nodes in a cloud computing environment | |
| Yiran et al. | Design and implementation of continuous integration scheme based on Jenkins and Ansible | |
| WO2019027597A1 (en) | Workflows management framework | |
| WO2017113835A1 (en) | Installation tool for large database system | |
| CN112565416B (en) | Cloud-native-based large-scale edge android equipment nanotube system and nanotube method thereof | |
| CN113687927A (en) | Method, device, equipment and storage medium for scheduling and configuring flash tasks | |
| CN117608761A (en) | Kubernetes cluster deployment method, device, equipment and storage medium | |
| US11431571B2 (en) | Monitoring time-base policy domain architecture | |
| CN115904632A (en) | Method, system, equipment and storage medium for realizing unified management of kubernets clusters | |
| US20230032516A1 (en) | Common platform for implementing rpa services on customer premises | |
| CN114745409A (en) | Remote control method for intelligent Internet of things equipment | |
| CN113377620A (en) | Automatic system self-checking method and system based on Kubernetes | |
| CN113014416A (en) | Method, system, device and storage medium for automatically configuring and managing network device | |
| CN112468349B (en) | Main node suitable for FT2000+ platform to deploy Ceph | |
| US11405267B2 (en) | Policy-based temporal domain configuration architecture | |
| CN118227271A (en) | Terminal all-in-one machine operation and maintenance method and device based on container cloud platform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200421 |