CN107454140A - A kind of Ceph cluster automatically dispose method and system based on big data platform - Google Patents
A kind of Ceph cluster automatically dispose method and system based on big data platform Download PDFInfo
- Publication number
- CN107454140A CN107454140A CN201710503028.0A CN201710503028A CN107454140A CN 107454140 A CN107454140 A CN 107454140A CN 201710503028 A CN201710503028 A CN 201710503028A CN 107454140 A CN107454140 A CN 107454140A
- Authority
- CN
- China
- Prior art keywords
- module
- management
- node
- ceph
- ceph clusters
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
Abstract
The present embodiments relate to cloud storage field, and in particular to a kind of Ceph cluster automatically dispose system and methods based on big data platform, including：Management node module and calculate node module；Wherein, management node module includes：Dispose management module, parameter configuration module, monitoring management module, alarm management module；Calculate node module includes：Automatic installation module, parameter synchronization module, monitoring module；The present invention can quickly realize automatically dispose, and efficiency improves more than 2 times.
Description
Technical field
The present invention relates to cloud storage field, and in particular to a kind of Ceph cluster Automation management sides based on big data platform
Method and system.
Background technology
Data storage is the basis of big data and field of cloud calculation, typical storage mode have block storage, object storage and
File stores three kinds：Block storage is widely used in the storage system of early stage because read or write speed is fast；Current big data field
Generally use distributed file system is to meet the needs that increase of data dynamic；And object stores the storage side because having flattening concurrently
Formula and it is easy to shared advantage, storage and retrieval particularly suitable for extensive unstructured data in cloud and big data environment.
Object stores no bibliographic structure and there was only memory space, and data are stored with Key-Value form, and are extended easily, thus
Internet and field of cloud calculation are widely used.
Typical object storage system has Ceph and Swift.On the whole, Ceph is dominant in performance, and Swift is more suitable
Together in the scene across data center.But the two deployment and configuration are generally all more complicated, need special messenger in actual use
Disposed and safeguarded, efficiency is very low and cost is higher.
Therefore, the automatically dispose for solving the problems, such as Ceph clusters be promote the widely used important steps of Ceph, have compared with
High practical value.
Existing Ceph deployment schemes be substantially it is manual, it is necessary to preparation, configuration and installation by more than 10 steps,
Complex operation.Individual other automatically dispose method, the Ceph deployment schemes such as based on Ansible, it is also desirable to which more early stage is accurate
The operation such as standby and manual configuration, and it is usually applicable only to specific operating system.
The deployment scheme that Ceph officials are recommended is manually dispose, and complex operation, configuration is various, it is necessary to which professional and technical personnel grasps
Make, it is difficult to use and promote.And the Ceph automatically disposes scheme increased income can not can not directly exist with big data fusion of platforms
Use, and operate still more complicated in big data platform.
In a word, existing deployment scheme operation is more complicated, and can be suitable for big data with big data fusion of platforms
Resource unified management and monitoring under platform.
Therefore, a kind of Ceph cluster automatically dispose method and system based on big data platform are built, it is existing to solve
Problem above in technology is necessary.
The content of the invention
The embodiments of the invention provide a kind of Ceph cluster automatically dispose method and system based on big data platform, energy
Enough rapid automatized deployment.
A kind of Ceph cluster automatically dispose systems based on big data platform provided in an embodiment of the present invention, including：
Management node module and calculate node module；
Wherein, management node module includes：
Management module is disposed, for the automatic configuration of Ceph clusters, unified management, start and stop, service inspection, and according to money
Source service condition, plan each functional node of Ceph clusters；
Parameter configuration module, for the individual cultivation of Ceph clusters to be showed into user to graphically, and according to
Actual disposition record, the modification parameter value at family；
Monitoring management module, for collecting the operational factor of Ceph clusters, and big data management platform is transferred to be uniformly processed
And displaying；
Alarm management module, for abnormal information caused by the operation of Ceph clusters to be filtered, judges and handled, and root
Decide whether to produce alarm according to the preset value of user；If producing alarm, warning information is sent to the alarm of big data platform
Management interface, then keeper is notified to perform corresponding action by big data platform；
Wherein, calculate node module includes：
Automatic installation module, for before installation Ceph clusters correspond to version, planning the installation on each node automatically
Journey, the operational order of the monitoring management module is received, and installed in each respective nodes automatic start；
Parameter synchronization module, for the configuration file and cluster individual cultivation parameter for generating the deployment management module
Automatic synchronization corresponding each node into Ceph clusters；
Monitoring module, for monitoring the running status of each Ceph clusters operation node and reporting the monitoring management
Module.
Further, the deployment management module, for the automatic configuration of Ceph clusters, unified management, start and stop, service
Check, and according to resource service condition, plan each functional node of Ceph clusters, including：Pass through assembly management module, start and stop pipe
Module, service inspection module, service module are managed, realizes automatic configuration, unified management, start and stop, service inspection, and according to
Resource service condition, plan each functional node of Ceph clusters.
Further, the parameter synchronization module, for the configuration file and collection for generating the deployment management module point
Group's individual cultivation parameter automatic synchronization corresponding each node into Ceph clusters, including：
The configuration file and cluster individual cultivation parameter of the deployment management module point generation are corresponding with Ceph clusters
Each host node it is synchronous；If synchronous success, same to the end of the step, otherwise, continue synchronous.
The embodiment of the present invention provides a kind of Ceph cluster Automation arranging methods based on big data platform, including：
Management node sends control command road calculate node, including：
Deployment management, realizes the automatic configurations of Ceph clusters, unified management, start and stop, service inspection, and make according to resource
With situation, each functional node of planning Ceph clusters；
Configuration parameter, the individual cultivation of Ceph clusters is showed into user to graphically, and according to the reality of user
Configuration record, modification parameter value；
Monitoring management, collects the operational factor of Ceph clusters, and transfers to big data management platform to be uniformly processed and show；
Alarm management, for Ceph clusters operation caused by abnormal information filter, judge and handle, and according to
The preset value at family decides whether to produce alarm；If producing alarm, warning information is sent to the alarm management of big data platform
Interface, then keeper is notified to perform corresponding action by big data platform；
Calculate node sends heartbeat and reports management node, including：
Automatic installation, before installation Ceph clusters correspond to version, the installation process on each node is planned automatically, receives prison
The operational order of management module is controlled, and is installed in each respective nodes automatic start；
Parameter synchronization, the configuration file for disposing management module generation and cluster individual cultivation parameter automatic synchronization are arrived
Corresponding each node in Ceph clusters；
Condition monitoring, monitor the running status of each Ceph clusters operation node and report the management node.
Further, the deployment management, the automatic configurations of Ceph clusters, unified management, start and stop, service inspection are realized
Look into, and according to resource service condition, plan each functional node of Ceph clusters, including：Pass through assembly management module, start-stop supervision
Module, service inspection module, service module, realize automatic configuration, unified management, start and stop, service inspection, and according to money
Source service condition, plan each functional node of Ceph clusters.
Further, the parameter synchronization, for the configuration file of the deployment management module point generation and cluster is individual
Property configuration parameter automatic synchronization corresponding each node into Ceph clusters, including：The configuration file and cluster individual character of generation
It is synchronous with corresponding each host node in Ceph clusters to change configuration parameter；If synchronous success, same to the end of the step, otherwise, continue synchronous.
The present invention has following technique effect：
Rapid automatized it can dispose, efficiency improves more than 2 times；It is simple to operate, six step selection operations are only needed, it is easy-to-use
Property is very good；Quick modification global configuration, and automatic synchronization, are obviously improved O＆M efficiency；Visualized management, graphical integration
Service inspection, alarm, condition monitoring；The stable sexual satisfaction production environment of scheme needs.
Brief description of the drawings
Technical scheme in order to illustrate the embodiments of the present invention more clearly, make required in being described below to embodiment
Accompanying drawing is briefly introduced, it should be apparent that, drawings in the following description are only some embodiments of the present invention, for this
For the those of ordinary skill in field, without having to pay creative labor, it can also be obtained according to these accompanying drawings
His accompanying drawing.
Fig. 1 is the system logic architecture figure of the big data platform described in one embodiment of the invention
Fig. 2 is the management node logical architecture figure described in one embodiment of the invention
Fig. 3 is the Technical Architecture figure described in one embodiment of the invention
Fig. 4 is the microcosmic logic chart described in one embodiment of the invention
Fig. 5 is the deployment management module block diagram described in one embodiment of the invention
Fig. 6 is the parameter synchronization block flow diagram described in one embodiment of the invention
Fig. 7 is the implementing procedure figure described in one embodiment of the invention
Fig. 8 is the hardware configuration connection diagram of the electronic equipment of the method described in one embodiment of the invention
Embodiment
In order that the object, technical solutions and advantages of the present invention are clearer, the present invention is made below in conjunction with accompanying drawing into
One step it is described in detail, it is clear that described embodiment is only part of the embodiment of the present invention, rather than whole implementation
Example.Based on the embodiment in the present invention, what those of ordinary skill in the art were obtained under the premise of creative work is not made
All other embodiment, belongs to the scope of protection of the invention.
The preferred embodiment that the invention will now be described in detail with reference to the accompanying drawings.
Embodiment 1
The present invention proposes a kind of Ceph cluster Automation arranging methods based on big data platform, utilizes big data platform
Master/Slave frameworks and Custom component framework, with reference to Ceph manually dispose methods, realize the automatic of Ceph clusters
Change deployment, configuration, inspection, alarm and condition monitoring etc., greatly simplify and speed up the deployment of Ceph clusters.
It is specific as follows：
Macroscopical service logic introduction：
1) system logic architecture
The system logic architecture of big data platform involved by this method is as shown in figure 1, the logic function main points of core are retouched
State as follows：
A) system is Master/Slave frameworks；
B) wherein Master is responsible for the centralized management of whole platform, task distribution, while owns by Agent interface managements
Slave nodes；
C) wherein Slave is responsible for execution, the monitoring resource specifically acted, while by heartbeat by the information reporting of this node
To Master；
D) state relation, task scheduling and resource is maintained by heartbeat mechanism between Master and Slave to converge；
E) related resource and information are exposed to end user by system by external unified interface.
The present invention is based on big data Platform Designing, relies on the Master/Slave frameworks of big data management platform and makes by oneself
Adopted component framework, Ceph is included in big data management platform in the form of serviced component, so as to realize as the common big number of deployment
The deployment Ceph clusters automated like that according to service.
The system logic architecture of the present invention on the management node Master of big data platform as shown in Fig. 2 start Ceph
The installation operation of cluster, specific Ceph clusters installation, configuration, service inspection are performed on each Slave nodes of big data platform
The operation such as look into；, can the condition monitoring of startup Ceph clusters, alarm, operation on each node after Ceph clusters install
Information Statistics etc. operate, and are referred to by the dynamic operation information and key technology of the whole cluster of the unified graphical representation in Master ends
Mark.
The deployment of whole Ceph clusters only needs simple six steps selection to complete, not only easy to operate, and efficiency
It is very high and be easy to extend, the scene for the large scale deployment that is particularly suitable for use in.
2) system architecture
The present invention is designed based on big data platform, is relied on the open frame of big data management platform and self-defined is connect
Mouthful, Ceph is included to a component for turning into big data platform in big data management platform in the form of serviced component.Scheme exists
While completing Ceph cluster automatically disposes, it may have service inspection, condition monitoring, running status displaying, alarm notification etc.
Function；While automatically dispose is met, the thought of the integrated and automatic O＆M of big data resource service is also contemplated, will
Ceph clusters are completely merged with big data platform, finally provide great convenience for the storage and use of data.
The present invention is mainly by following six module composition, as shown in figure 3, being respectively：Dispose management module, parameter configuration mould
Block, module, parameter synchronization module, monitoring management module and alarm management module are installed automatically.
Wherein, Master nodes operation deployment management, parameter configuration, monitoring management and alarm management module；And in Slave
Node operational factor is installed synchronously, automatically and monitoring management module.
It is responsible for installation startup and the initial work of Ceph clusters by Master nodes；Opened automatically after cluster installation
Dynamic Ceph clusters；After cluster operation, collect and converge the operation information of each instant node, status information, statistical information etc.,
And shown by unified UI；
It is responsible for the establishment work of the automatic installations of Ceph installation kits, parameter synchronization and each example by Slave nodes；Pass through
The order that Master nodes issue is performed, the Ceph examples of this node are operated accordingly；And after cluster operation, it incite somebody to action this
The information reportings such as the operational factor of node give Master nodes.
Microcosmic Logic
1) logic flow of this method
This method is mainly by six module compositions：Dispose management module, parameter configuration module, module, parameter are installed automatically
Synchronization module, monitoring management module and alarm management module.Its execution logic and flow are as shown in Figure 4.
2) each module details of this method
Deployment management module is responsible for starting the automatic configuration of Ceph clusters, and according to resource service condition, appropriate planning
Each functional node of Ceph clusters, its functional block logic are as shown in Figure 5.After the processing of this module, the framework of Ceph clusters
Tentatively establish.In addition, the module is also responsible for the operation such as unified management, start and stop, service inspection of Ceph clusters.This process
Corresponding to the manual shutdown process of cluster programming process and services of manually dispose Ceph clusters.
Parameter configuration module is responsible for the individual cultivation of Ceph clusters showing user to graphically, and according to user
Actual disposition recording parameters value, main configuration item is as shown in the table.In addition, after the module also provides clustered deploy(ment) success
Parameter is changed and inquiry operation.After the processing of this module, the individual cultivation parameter of Ceph clusters determines substantially, other
Parameter uses system default value.This process corresponds to the parameter manual configuration procedures of manually dispose Ceph clusters.
| Parameter name (interface display) | Parameter declaration | Default value |
| public_network | The network environment that Ceph cluster internals use | 192.168.111.0/24 |
| cluster_network | Provide the user the public network environment of access | 192.168.111.0/24 |
| osd_pool_default_size | Pool number of copies in cluster | 1 |
| osd_pool_default_min_size | Pool minimum this number of Fumel in cluster | 1 |
| osd_crush_chooseleaf_type | Bucket types during CRUSH rule chooseleaf | 0 |
| osd_max_object_name_len | The length of object oriented | 256 |
| osd_journal_size | The size of daily record | 128M |
| osd_pool_default_pg_num | The placement groups numbers of cluster | 128 |
| osd_storage_paths | The path of company-data storage | Nothing.Form is：hostname：path. |
1 main configuration item of table and lexical or textual analysis
Automatic installation module is responsible for before installation Ceph corresponds to version, plans the installation process on each node automatically, connects
The operational order of installation management module is received, and is installed in each respective nodes automatic start.Installation kit is united by big data management platform
One assembling provides, and participates in, configures and adjusts without user.After the processing of this module, Ceph is in each plan node
Complete installation.This process corresponds to the installation source configuration of manually dispose Ceph clusters, the process of installation kit installation.
The configuration file and cluster individual cultivation parameter that parameter synchronization module is responsible for that management node generation will be disposed are automatic
It is as shown in Figure 6 to be synchronized to corresponding each node, synchronizing process in Ceph clusters.After this resume module is complete after ability, Ge Gejie
Configuration parameter on point is updated and synchronous, and is got ready for the startup of Ceph clusters.This process corresponds to hand
The process of the configuration parameter manual synchronization of dynamic deployment Ceph clusters.
Monitoring management module is responsible for informations such as the state of Ceph clusters, operational factors and transfers to big data management to put down
Platform is uniformly processed and shown.This module runs on the whole process of Ceph lifecycle of industry cluster, can find Ceph collection in time rapidly
The abnormal information of group, as clock skewing, service delay machine, running example number, insufficient space the problems such as, and notify alarm management in time
Module and big data platform, the safe and stable operation for Ceph clusters provide support.This is not included in manually dispose Ceph clusters
Module and corresponding operating.
Alarm management module is responsible for that the information such as abnormal caused by the operation of Ceph clusters is filtered, judges and handled, and
Decided whether to produce alarm according to the preset value of user.If producing alarm, warning information is started into the announcement to big data platform
Alert management interface, then keeper is notified to perform corresponding action by big data platform.Do not include in manually dispose Ceph clusters
This module and corresponding operating.
3) implementation process of this method
The implementation process of this method is as shown in Figure 7：
A) by the Master/Slave frameworks of big data management platform, in Master nodes triggering Ceph pre-installation nodes
Automatic selection and basic configuration, specific action realized by the Core Feature of this method；
B) by the framework of big data management platform, the action that automatically configures of Ceph clusters is triggered in Master nodes, and
Each node in follow-up automatic synchronization to pre-installation Ceph, specific action are realized by the Core Feature of this method；
C) by the Scheduling Framework of big data management platform, installation action is triggered on Agent nodes；
D) by the Scheduling Framework of big data management platform, the triggering configuration synchronization action on Agent nodes, specific action
Realized by the Core Feature of this method；
E) by the Scheduling Framework of big data management platform, triggered on Agent nodes and automatically configure action, specific action
Realized by the Core Feature of this method；
F) by the Scheduling Framework of big data management platform, environmental test action, specific action are triggered on Agent nodes
Realized by the Core Feature of this method；
G) by the Scheduling Framework of big data management platform, the triggering service starting operation on Agent nodes, specific action
Realized by the Core Feature of this method；
H) by the Scheduling Framework of big data management platform, service inspection action, specific action are triggered on Agent nodes
Realized by the Core Feature of this method；
I) by the Scheduling Framework of big data management platform, automatic triggering monitoring and alarm action, specifically act by we
The Core Feature of method is realized；
J) convergence is simultaneously done by the Scheduling Framework of big data management platform, automatic triggering state supervisory control action, specifically
Action is realized by the Core Feature of this method；
K) Scheduling Framework of big data management platform, the automatic graphical exhibition for triggering cluster state and operation information are passed through
Show, specific action is realized by the Core Feature of this method.
Compared with prior art, technical solution of the present invention has following obvious advantages：
Rapid automatized deployment, efficiency improve more than 2 times；Simple to operate, only six step selection operations of need, ease for use are non-
Chang Hao；Quick modification global configuration, and automatic synchronization, are obviously improved O＆M efficiency；Visualized management, the clothes of graphical integration
Business is checked, alerted, condition monitoring；The stable sexual satisfaction production environment of scheme needs.
Embodiment 2
A kind of electronic equipment, including：At least one processor；And be connected with least one processor communication
Memory；Wherein, have can be by the instruction of one computing device for the memory storage, and the instruction is by described at least one
Individual computing device, so that at least one processor is able to carry out following methods step：
Management node sends control command road calculate node, including：
Deployment management, realizes the automatic configurations of Ceph clusters, unified management, start and stop, service inspection, and make according to resource
With situation, each functional node of planning Ceph clusters；
Configuration parameter, the individual cultivation of Ceph clusters is showed into user to graphically, and according to the reality of user
Configuration record, modification parameter value；
Monitoring management, collects the operational factor of Ceph clusters, and transfers to big data management platform to be uniformly processed and show；
Alarm management, for Ceph clusters operation caused by abnormal information filter, judge and handle, and according to
The preset value at family decides whether to produce alarm；If producing alarm, warning information is sent to the alarm management of big data platform
Interface, then keeper is notified to perform corresponding action by big data platform；
Calculate node sends heartbeat and reports management node, including：
Automatic installation, before installation Ceph clusters correspond to version, the installation process on each node is planned automatically, receives prison
The operational order of management module is controlled, and is installed in each respective nodes automatic start；
Parameter synchronization, the configuration file for disposing management module generation and cluster individual cultivation parameter automatic synchronization are arrived
Corresponding each node in Ceph clusters；
Condition monitoring, monitor the running status of each Ceph clusters operation node and report the management node.
Further, the deployment management, the automatic configurations of Ceph clusters, unified management, start and stop, service inspection are realized
Look into, and according to resource service condition, plan each functional node of Ceph clusters, including：Pass through assembly management module, start-stop supervision
Module, service inspection module, service module, realize automatic configuration, unified management, start and stop, service inspection, and according to money
Source service condition, plan each functional node of Ceph clusters.
Further, the parameter synchronization, for the configuration file of the deployment management module point generation and cluster is individual
Property configuration parameter automatic synchronization corresponding each node into Ceph clusters, including：The configuration file and cluster individual character of generation
It is synchronous with corresponding each host node in Ceph clusters to change configuration parameter；If synchronous success, same to the end of the step, otherwise, continue synchronous.
Embodiment 3
The embodiment of the present application provides a kind of nonvolatile computer storage media, and the computer-readable storage medium is stored with
Computer executable instructions, the computer executable instructions can perform the method in above-mentioned any means embodiment.
Embodiment 4
Fig. 8 is the hardware architecture diagram of the electronic equipment for the methods described that the present embodiment provides, as shown in figure 8, this sets
It is standby to include：
One or more processors 810 and memory 820, in Fig. 8 by taking a processor 810 as an example.
The equipment of intelligent method can also include：Input unit 830 and output device 840.
Processor 810, memory 820, input unit 830 and output device 840 can pass through bus or other modes
Connect, in Fig. 8 exemplified by being connected by bus.
Memory 820 is used as a kind of non-volatile computer readable storage medium storing program for executing, available for storage non-volatile software journey
Sequence, non-volatile computer executable program and module, programmed instruction/mould as corresponding to the method in the embodiment of the present application
Block.Processor 810 is stored in non-volatile software program, instruction and module in memory 820 by operation, so as to perform
The various function application of server and data processing, that is, realize above method embodiment method.
Memory 820 can include storing program area and storage data field, wherein, storing program area can store operation system
Application program required for system, at least one function；Storage data field can be stored according to using created data etc. in method.
In addition, memory 820 can include high-speed random access memory, nonvolatile memory, for example, at least one can also be included
Individual disk memory, flush memory device or other non-volatile solid state memory parts.
Input unit 830 can receive the numeral or character information of input, and produce with the user of electronic equipment set with
And the key signals input that function control is relevant.Output device 840 may include the display devices such as display screen.
One or more of modules are stored in the memory 820, when by one or more of processors
During 810 execution, the method in above-mentioned any means embodiment is performed.
The Chinese and English full name of abbreviation
A kind of Ceph-object storage system
The physically or logically memory cell of OSD-Object Storage Device, Ceph systems
The said goods can perform the method that the embodiment of the present application is provided, and possesses the corresponding functional module of execution method and has
Beneficial effect.Not ins and outs of detailed description in the present embodiment, reference can be made to the method that the embodiment of the present application is provided.
The terminal of the embodiment of the present invention exists in a variety of forms, includes but is not limited to:
(1) mobile communication equipment:The characteristics of this kind equipment is that possess mobile communication function, and to provide speech, data
Communicate as main target.This Terminal Type includes:Smart mobile phone (such as iPhone), multimedia handset, feature mobile phone, and it is low
Hold mobile phone etc..
(2) super mobile personal computer equipment:This kind equipment belongs to the category of personal computer, there is calculating and processing work(
Can, typically also possess mobile Internet access characteristic.This Terminal Type includes:PDA, MID and UMPC equipment etc., such as iPad.
(3) portable entertainment device:This kind equipment can show and play content of multimedia.The kind equipment includes:Audio,
Video player (such as iPod), handheld device, e-book, and intelligent toy and portable car-mounted navigation equipment.
(4) server:The equipment for providing the service of calculating, the composition of server are total including processor, hard disk, internal memory, system
Line etc., server is similar with general computer architecture, but due to needing to provide highly reliable service, therefore in processing energy
Power, stability, reliability, security, scalability, manageability etc. require higher.
(5) other electronic installations with data interaction function, such as television set, vehicle-mounted large-size screen monitors etc..
Device embodiment described above is only schematical, wherein the unit illustrated as separating component can
To be or may not be physically separate, it can be as the part that unit is shown or may not be physics list
Member, you can with positioned at a place, or can also be distributed on multiple NEs.It can be selected according to the actual needs
In some or all of module realize the purpose of this embodiment scheme.
Through the above description of the embodiments, those skilled in the art can be understood that each embodiment can
Realized by the mode of software plus required general hardware platform, naturally it is also possible to pass through hardware.Based on such understanding, on
The part that technical scheme substantially in other words contributes to prior art is stated to embody in the form of software product, should
Computer software product can store in a computer-readable storage medium, such as ROM/RAM, magnetic disc, CD, including some fingers
Make to cause a computer equipment (can be personal computer, server, or network equipment etc.) to perform each implementation
Method described in some parts of example or embodiment.
Finally it should be noted that：The above embodiments are merely illustrative of the technical solutions of the present invention, rather than its limitations；Although
The present invention is described in detail with reference to the foregoing embodiments, it will be understood by those within the art that：It still may be used
To be modified to the technical scheme described in foregoing embodiments, or equivalent substitution is carried out to which part technical characteristic；
And these modification or replace, do not make appropriate technical solution essence depart from various embodiments of the present invention technical scheme spirit and
Scope.
Claims (6)
- A kind of 1. Ceph cluster automatically dispose systems based on big data platform, it is characterised in that including：Management node module and calculate node module；Wherein, management node module includes：Management module is disposed, is made for the automatic configuration of Ceph clusters, unified management, start and stop, service inspection, and according to resource With situation, each functional node of planning Ceph clusters；Parameter configuration module, for the individual cultivation of Ceph clusters to be showed into user to graphically, and according to user's Actual disposition record, modification parameter value；Monitoring management module, for collecting the operational factor of Ceph clusters, and big data management platform is transferred to be uniformly processed and open up Show；Alarm management module, for Ceph clusters operation caused by abnormal information filter, judge and handle, and according to The preset value at family decides whether to produce alarm；If producing alarm, warning information is sent to the alarm management of big data platform Interface, then keeper is notified to perform corresponding action by big data platform；Wherein, calculate node module includes：Automatic installation module, for before installation Ceph clusters correspond to version, planning the installation process on each node automatically, connecing The operational order of the monitoring management module is received, and is installed in each respective nodes automatic start；Parameter synchronization module, for the configuration file of the deployment management module generation and cluster individual cultivation parameter is automatic It is synchronized to corresponding each node in Ceph clusters；Monitoring module, for monitoring the running status of each Ceph clusters operation node and reporting the monitoring management mould Block.
- 2. the system as claimed in claim 1, it is characterised in that：The deployment management module, the automation for Ceph clusters Configuration, unified management, start and stop, service inspection, and according to resource service condition, each functional node of Ceph clusters is planned, including： By assembly management module, start-stop supervision module, service inspection module, service module, automatic configuration, unified pipe are realized Reason, start and stop, service inspection, and according to resource service condition, each functional node of planning Ceph clusters.
- 3. the system as claimed in claim 1, it is characterised in that：The parameter synchronization module, for the deployment to be managed into mould Configuration file and cluster individual cultivation parameter automatic synchronization corresponding each node into Ceph clusters of block point generation, bag Include：The configuration file and cluster individual cultivation parameter of the deployment management module point generation are corresponding with Ceph clusters each Host node is synchronous；If synchronous success, same to the end of the step, otherwise, continue synchronous.
- A kind of 4. Ceph cluster Automation arranging methods based on big data platform, it is characterised in that including：Management node sends control command road calculate node, including：Deployment management, realizes the automatic configurations of Ceph clusters, unified management, start and stop, service inspection, and use feelings according to resource Condition, plan each functional node of Ceph clusters；Configuration parameter, the individual cultivation of Ceph clusters is showed into user to graphically, and according to the actual disposition of user Record, modification parameter value；Monitoring management, collects the operational factor of Ceph clusters, and transfers to big data management platform to be uniformly processed and show；Alarm management, for abnormal information caused by the operation of Ceph clusters to be filtered, judges and handled, and according to user's Preset value decides whether to produce alarm；If producing alarm, warning information is sent to the alarm management interface of big data platform, Then keeper is notified to perform corresponding action by big data platform；Calculate node sends heartbeat and reports management node, including：Automatic installation, before installation Ceph clusters correspond to version, the installation process on each node is planned automatically, receives monitoring pipe The operational order of module is managed, and is installed in each respective nodes automatic start；Parameter synchronization, the configuration file and cluster individual cultivation parameter automatic synchronization that deployment management module is generated to Ceph collection Corresponding each node in group；Condition monitoring, monitor the running status of each Ceph clusters operation node and report the management node.
- 5. method as claimed in claim 4, it is characterised in that：It is described deployment management, realize Ceph clusters automatic configuration, Unified management, start and stop, service inspection, and according to resource service condition, each functional node of Ceph clusters is planned, including：Pass through Assembly management module, start-stop supervision module, service inspection module, service module, realize automatic configuration, unified management, Start and stop, service inspection, and according to resource service condition, each functional node of planning Ceph clusters.
- 6. method as claimed in claim 4, it is characterised in that：The parameter synchronization, for disposing management module point by described The configuration file and cluster individual cultivation parameter automatic synchronization of generation corresponding each node into Ceph clusters, including：The configuration file and cluster individual cultivation parameter of generation are synchronous with corresponding each host node in Ceph clusters；If it is synchronous into Work(, same to the end of the step, otherwise, continue synchronous.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710503028.0A CN107454140A (en) | 2017-06-27 | 2017-06-27 | A kind of Ceph cluster automatically dispose method and system based on big data platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710503028.0A CN107454140A (en) | 2017-06-27 | 2017-06-27 | A kind of Ceph cluster automatically dispose method and system based on big data platform |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107454140A true CN107454140A (en) | 2017-12-08 |
Family
ID=60487118
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710503028.0A Withdrawn CN107454140A (en) | 2017-06-27 | 2017-06-27 | A kind of Ceph cluster automatically dispose method and system based on big data platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107454140A (en) |
Cited By (26)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108170524A (en) * | 2017-12-29 | 2018-06-15 | 北京奇虎科技有限公司 | The task processing method and device of a kind of computing cluster |
| CN108388605A (en) * | 2018-02-06 | 2018-08-10 | 广东暨通信息发展有限公司 | Big data analysis platform based on Internet of Things |
| CN108958744A (en) * | 2018-06-21 | 2018-12-07 | 北京京东金融科技控股有限公司 | Dispositions method, device, medium and the electronic equipment of big data distributed type assemblies |
| CN109039743A (en) * | 2018-08-03 | 2018-12-18 | 西安东美信息科技有限公司 | The centralized management method of distributed storage ceph cluster networks |
| CN109218109A (en) * | 2018-10-17 | 2019-01-15 | 北京京航计算通讯研究所 | The monitoring of Ceph distributed storage and tuning management system towards big data platform |
| CN109298945A (en) * | 2018-10-17 | 2019-02-01 | 北京京航计算通讯研究所 | The monitoring of Ceph distributed storage and tuning management method towards big data platform |
| WO2019169765A1 (en) * | 2018-03-06 | 2019-09-12 | 平安科技（深圳）有限公司 | Electronic device, method for acquiring state information in cluster environment, system, and storage medium |
| CN110737444A (en) * | 2019-10-18 | 2020-01-31 | 中电科技（北京）有限公司 | Remote self-adaptive dynamic deployment method and system for operating system based on firmware |
| CN110781158A (en) * | 2019-10-25 | 2020-02-11 | 山东乾云启创信息科技股份有限公司 | Distributed storage method and system based on CEPH |
| CN111078302A (en) * | 2019-11-19 | 2020-04-28 | 许昌许继软件技术有限公司 | Automatic deployment method and terminal of distribution network monitoring platform system |
| CN111193622A (en) * | 2019-12-30 | 2020-05-22 | 浪潮电子信息产业股份有限公司 | Asymmetric configuration management method, device and equipment and readable storage medium |
| CN111225064A (en) * | 2020-02-24 | 2020-06-02 | 中科星图股份有限公司 | Ceph cluster deployment method, system, device and computer-readable storage medium |
| CN111277632A (en) * | 2020-01-13 | 2020-06-12 | 中国建设银行股份有限公司 | Method and device for managing applications in system cluster |
| CN111274191A (en) * | 2020-01-08 | 2020-06-12 | 山东汇贸电子口岸有限公司 | Method for managing ceph cluster and cloud local storage coordinator |
| CN111290909A (en) * | 2020-01-19 | 2020-06-16 | 山东汇贸电子口岸有限公司 | System and method for monitoring and alarming ceph cluster |
| CN111381835A (en) * | 2018-12-28 | 2020-07-07 | 中国移动通信集团江西有限公司 | Method, device, equipment and medium for registering and discovering technical service component |
| CN111813346A (en) * | 2020-07-23 | 2020-10-23 | 山东超越数控电子股份有限公司 | Method, system, device and medium for establishing Ceph distributed storage based on cloud platform |
| CN111857735A (en) * | 2020-07-23 | 2020-10-30 | 浪潮云信息技术股份公司 | Method and system for creating Crush based on hook deployment Ceph |
| CN112241269A (en) * | 2019-07-16 | 2021-01-19 | 深圳兆日科技股份有限公司 | Zookeeper cluster control system, equipment and storage medium |
| CN112468349A (en) * | 2021-01-26 | 2021-03-09 | 柏科数据技术（深圳）股份有限公司 | Main node suitable for FT2000+ platform to deploy Ceph |
| WO2021082581A1 (en) * | 2019-10-29 | 2021-05-06 | 牛文运 | File system |
| CN113656147A (en) * | 2021-08-20 | 2021-11-16 | 北京百度网讯科技有限公司 | Cluster deployment method, device, equipment and storage medium |
| CN114327490A (en) * | 2022-03-04 | 2022-04-12 | 麒麟软件有限公司 | Method for adapting domestic operating system to Ceph |
| CN115080532A (en) * | 2022-07-22 | 2022-09-20 | 平安银行股份有限公司 | CEPH system management method and terminal |
| CN115174369A (en) * | 2021-04-02 | 2022-10-11 | 深圳市锐速云计算有限公司 | Nginx-based network node cluster scheduling and automatic deployment method and system |
| WO2024012082A1 (en) * | 2022-07-15 | 2024-01-18 | 京东方科技集团股份有限公司 | Big data cluster deployment method and apparatus, device, and medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102169448A (en) * | 2011-03-18 | 2011-08-31 | 浪潮电子信息产业股份有限公司 | Deployment method of cluster parallel computing environment |
| CN103617062A (en) * | 2013-12-05 | 2014-03-05 | 山东大学 | Flexible dynamic deployment system and method for render cluster |
| CN104104721A (en) * | 2014-07-17 | 2014-10-15 | 浪潮电子信息产业股份有限公司 | Method for achieving WINDOWS rendering cluster management and improving deployment efficiency based on WINDOWS domain control |
| CN104219099A (en) * | 2014-09-25 | 2014-12-17 | 浪潮(北京)电子信息产业有限公司 | Automatic server cluster deploying method and device |
| CN104572269A (en) * | 2015-01-19 | 2015-04-29 | 浪潮电子信息产业股份有限公司 | Quick cluster deployment method based on Linux operation system |
| CN105487891A (en) * | 2015-11-26 | 2016-04-13 | 航天恒星科技有限公司 | Distributed cluster deploying method and system |
-
2017
- 2017-06-27 CN CN201710503028.0A patent/CN107454140A/en not_active Withdrawn
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102169448A (en) * | 2011-03-18 | 2011-08-31 | 浪潮电子信息产业股份有限公司 | Deployment method of cluster parallel computing environment |
| CN103617062A (en) * | 2013-12-05 | 2014-03-05 | 山东大学 | Flexible dynamic deployment system and method for render cluster |
| CN104104721A (en) * | 2014-07-17 | 2014-10-15 | 浪潮电子信息产业股份有限公司 | Method for achieving WINDOWS rendering cluster management and improving deployment efficiency based on WINDOWS domain control |
| CN104219099A (en) * | 2014-09-25 | 2014-12-17 | 浪潮(北京)电子信息产业有限公司 | Automatic server cluster deploying method and device |
| CN104572269A (en) * | 2015-01-19 | 2015-04-29 | 浪潮电子信息产业股份有限公司 | Quick cluster deployment method based on Linux operation system |
| CN105487891A (en) * | 2015-11-26 | 2016-04-13 | 航天恒星科技有限公司 | Distributed cluster deploying method and system |
Cited By (33)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108170524A (en) * | 2017-12-29 | 2018-06-15 | 北京奇虎科技有限公司 | The task processing method and device of a kind of computing cluster |
| CN108388605A (en) * | 2018-02-06 | 2018-08-10 | 广东暨通信息发展有限公司 | Big data analysis platform based on Internet of Things |
| WO2019169765A1 (en) * | 2018-03-06 | 2019-09-12 | 平安科技（深圳）有限公司 | Electronic device, method for acquiring state information in cluster environment, system, and storage medium |
| CN108958744A (en) * | 2018-06-21 | 2018-12-07 | 北京京东金融科技控股有限公司 | Dispositions method, device, medium and the electronic equipment of big data distributed type assemblies |
| CN108958744B (en) * | 2018-06-21 | 2022-08-09 | 京东科技控股股份有限公司 | Deployment method, device, medium and electronic equipment of big data distributed cluster |
| CN109039743A (en) * | 2018-08-03 | 2018-12-18 | 西安东美信息科技有限公司 | The centralized management method of distributed storage ceph cluster networks |
| CN109218109A (en) * | 2018-10-17 | 2019-01-15 | 北京京航计算通讯研究所 | The monitoring of Ceph distributed storage and tuning management system towards big data platform |
| CN109298945A (en) * | 2018-10-17 | 2019-02-01 | 北京京航计算通讯研究所 | The monitoring of Ceph distributed storage and tuning management method towards big data platform |
| CN111381835A (en) * | 2018-12-28 | 2020-07-07 | 中国移动通信集团江西有限公司 | Method, device, equipment and medium for registering and discovering technical service component |
| CN112241269A (en) * | 2019-07-16 | 2021-01-19 | 深圳兆日科技股份有限公司 | Zookeeper cluster control system, equipment and storage medium |
| CN110737444A (en) * | 2019-10-18 | 2020-01-31 | 中电科技（北京）有限公司 | Remote self-adaptive dynamic deployment method and system for operating system based on firmware |
| CN110781158B (en) * | 2019-10-25 | 2022-08-16 | 山东乾云启创信息科技股份有限公司 | Distributed storage method and system based on CEPH |
| CN110781158A (en) * | 2019-10-25 | 2020-02-11 | 山东乾云启创信息科技股份有限公司 | Distributed storage method and system based on CEPH |
| WO2021082581A1 (en) * | 2019-10-29 | 2021-05-06 | 牛文运 | File system |
| CN111078302A (en) * | 2019-11-19 | 2020-04-28 | 许昌许继软件技术有限公司 | Automatic deployment method and terminal of distribution network monitoring platform system |
| CN111078302B (en) * | 2019-11-19 | 2023-08-11 | 许昌许继软件技术有限公司 | Automatic deployment method and terminal of distribution network monitoring platform system |
| CN111193622A (en) * | 2019-12-30 | 2020-05-22 | 浪潮电子信息产业股份有限公司 | Asymmetric configuration management method, device and equipment and readable storage medium |
| WO2021135279A1 (en) * | 2019-12-30 | 2021-07-08 | 浪潮电子信息产业股份有限公司 | Asymmetric configuration management method, apparatus and device, and readable storage medium |
| CN111193622B (en) * | 2019-12-30 | 2021-11-09 | 浪潮电子信息产业股份有限公司 | Asymmetric configuration management method, device and equipment and readable storage medium |
| CN111274191A (en) * | 2020-01-08 | 2020-06-12 | 山东汇贸电子口岸有限公司 | Method for managing ceph cluster and cloud local storage coordinator |
| CN111277632A (en) * | 2020-01-13 | 2020-06-12 | 中国建设银行股份有限公司 | Method and device for managing applications in system cluster |
| CN111277632B (en) * | 2020-01-13 | 2022-05-06 | 中国建设银行股份有限公司 | Method and device for managing applications in system cluster |
| CN111290909A (en) * | 2020-01-19 | 2020-06-16 | 山东汇贸电子口岸有限公司 | System and method for monitoring and alarming ceph cluster |
| CN111225064A (en) * | 2020-02-24 | 2020-06-02 | 中科星图股份有限公司 | Ceph cluster deployment method, system, device and computer-readable storage medium |
| CN111813346A (en) * | 2020-07-23 | 2020-10-23 | 山东超越数控电子股份有限公司 | Method, system, device and medium for establishing Ceph distributed storage based on cloud platform |
| CN111857735A (en) * | 2020-07-23 | 2020-10-30 | 浪潮云信息技术股份公司 | Method and system for creating Crush based on hook deployment Ceph |
| CN112468349B (en) * | 2021-01-26 | 2021-07-20 | 柏科数据技术（深圳）股份有限公司 | Main node suitable for FT2000+ platform to deploy Ceph |
| CN112468349A (en) * | 2021-01-26 | 2021-03-09 | 柏科数据技术（深圳）股份有限公司 | Main node suitable for FT2000+ platform to deploy Ceph |
| CN115174369A (en) * | 2021-04-02 | 2022-10-11 | 深圳市锐速云计算有限公司 | Nginx-based network node cluster scheduling and automatic deployment method and system |
| CN113656147A (en) * | 2021-08-20 | 2021-11-16 | 北京百度网讯科技有限公司 | Cluster deployment method, device, equipment and storage medium |
| CN114327490A (en) * | 2022-03-04 | 2022-04-12 | 麒麟软件有限公司 | Method for adapting domestic operating system to Ceph |
| WO2024012082A1 (en) * | 2022-07-15 | 2024-01-18 | 京东方科技集团股份有限公司 | Big data cluster deployment method and apparatus, device, and medium |
| CN115080532A (en) * | 2022-07-22 | 2022-09-20 | 平安银行股份有限公司 | CEPH system management method and terminal |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107454140A (en) | A kind of Ceph cluster automatically dispose method and system based on big data platform | |
| WO2021017301A1 (en) | Management method and apparatus based on kubernetes cluster, and computer-readable storage medium | |
| US20210405629A1 (en) | System and method for interoperable communication of an automation system component with multiple information sources | |
| US10061371B2 (en) | System and method for monitoring and managing data center resources in real time incorporating manageability subsystem | |
| CN105653425B (en) | Monitoring system based on complex event processing engine | |
| EP2625614B1 (en) | System and method for monitoring and managing data center resources in real time incorporating manageability subsystem | |
| CN108595316B (en) | Lifecycle management method, manager, device, and medium for distributed application | |
| CN103399781B (en) | Cloud Server and virtual machine management method thereof | |
| CN103268226A (en) | Test script file generation method and device | |
| CN113377626B (en) | Visual unified alarm method, device, equipment and medium based on service tree | |
| CN103914307A (en) | Interactive-interface fast implementation method based on reusable library | |
| CN111625535B (en) | Method, device and storage medium for realizing business data association | |
| US20090254652A1 (en) | Resource correlation prediction | |
| US20220182851A1 (en) | Communication Method and Apparatus for Plurality of Administrative Domains | |
| CN112148578A (en) | IT fault defect prediction method based on machine learning | |
| CN106776136B (en) | Data base processing method and device | |
| CN113312145A (en) | Container scheduling method and device, electronic equipment and medium | |
| CN114756301B (en) | Log processing method, device and system | |
| CN109240916A (en) | Information output controlling method, device and computer readable storage medium | |
| CN111897643A (en) | Thread pool configuration system, method, device and storage medium | |
| CN113010385B (en) | Task state updating method, device, equipment and medium | |
| CN113821412A (en) | Equipment operation and maintenance management method and device | |
| CN112558972A (en) | Application management platform, system, method, storage medium and program product | |
| Wollenberg et al. | A brief history: The common information model [in my view] | |
| CN115150466B (en) | Method and device for realizing data distribution, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20171208 |
|
| WW01 | Invention patent application withdrawn after publication |