CN108958840A - A kind of cluster configuration dynamic instrumentation merging loading method - Google Patents
A kind of cluster configuration dynamic instrumentation merging loading method Download PDFInfo
- Publication number
- CN108958840A CN108958840A CN201810744414.3A CN201810744414A CN108958840A CN 108958840 A CN108958840 A CN 108958840A CN 201810744414 A CN201810744414 A CN 201810744414A CN 108958840 A CN108958840 A CN 108958840A
- Authority
- CN
- China
- Prior art keywords
- configuration
- cluster
- node
- working node
- state
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/445—Program loading or initiating
- G06F9/44521—Dynamic linking or loading; Link editing at or after load time, e.g. Java class loading
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/445—Program loading or initiating
- G06F9/44505—Configuring for program initiating, e.g. using registry, configuration files
- G06F9/4451—User profiles; Roaming
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Transfer Between Computers (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Stored Programmes (AREA)
Abstract
The invention discloses a kind of cluster configuration dynamic instrumentations to merge loading method, comprising: step 1, passes through configuration management client-side management cluster configuration；Step 2, configuration management server end receives the configuration modification request of configuration management client and configuration management center is written by specified format in configuration；Step 3, configuration detecting module monitors the cluster configuration node at configuration management center in real time；When configuring detecting module and detecting that the content of node changes, cluster synchronization newly be configured to local, and update the state of respective nodes.The present invention provides version management and log for all configurations, and the configuration of any version can be applied in maintenance process；While cluster configuration file dynamic instrumentation loads, guarantee not interrupting currently just executing for task；Unified configuration modification entrance is provided, validity checking is increased to the configuration of modification；Configuration management flow control and configuration merge application, guarantee that cluster will not frequently restart because of configuration modification.
Description
Technical field
The present invention relates to data processing cluster management domains, and in particular to a kind of cluster configuration dynamic instrumentation merging load side
Method.
Background technique
With the fast development and popularization and application of computer and information technology, the scale of industrial application system is expanded rapidly,
Data caused by industrial application are in explosive increase.There has been proposed the concepts of big data.With Google MapReduce and
The publication of GoogleFile System (GFS), big data are no longer only used to describe a large amount of data, also cover processing data
Speed.Big data processing is a kind of to provide the resource of the telescopic virtualization of dynamic in a manner of servicing Internet
Calculating mode.Cluster management is this essential a part of calculating mode.
There is defects some in this way in management for existing distributed type assemblies:
1, it during cluster management, for the modification of a certain configuration, needs to do each node of cluster identical
Modification.It is easy to appear not yet done, the case where only having modified part of nodes due to fault or to system, lead to system configuration state
It is inconsistent.
2, it modifies in configuration process, configuration take-effective can just be made by needing to restart cluster, partial task can be made due to node in this way
Restart, computed repeatedly, waste computing resource, reduces efficiency.
3, majority of case is all that configuration file is directly directly modified in a manner of text editing, further increases appearance
A possibility that mistake.
4, some since improperly-configured caused mistake is difficult to trace.
Summary of the invention
The present invention is in order to solve the above-mentioned technical problem, and it is an object of the present invention to provide a kind of cluster configuration dynamic instrumentation merges load side
Method.
The present invention is achieved through the following technical solutions:
A kind of cluster configuration dynamic instrumentation merging loading method, comprising:
Step 1, pass through configuration management client-side management cluster configuration；
Step 2, configuration management server end receives the configuration modification request of configuration management client and configuration by specified
Configuration management center is written in format；
Step 3, configuration detecting module monitors the cluster configuration node at configuration management center in real time；When configuration detecting module inspection
When measuring the content of node and changing, cluster synchronization it is new be configured to local, and update the state of respective nodes；
Step 4, host node cluster configuration state-maintenance module obtains matching for each working node from each working node of cluster
Set state；The configuration status of working node includes that application state, configuration are synchronous not using shape for the configuration version number of node, configuration
State；When each working node start or configure update when, each working node is by the state registration of oneself to host node；
Step 5, heavily loaded module is located on each node；When heavily loaded module is located on the host node of cluster, step 6 is executed；
When heavily loaded module is located on the working node of cluster, step 7 is executed；
Step 6, when heavily loaded module is in host node, heavily loaded module assignment task is corresponding to first checking before working node
The state of working node, if corresponding working node configuration updates and configuration has been applied, heavily loaded module assignment task is to phase
The working node answered；
Step 7, when heavily loaded module is in working node,
Heavily loaded module checks whether the configuration version number of corresponding working node is newest, if corresponding working node is matched
It sets version number not and is newest, then follow the steps 3；
If the configuration version number of working node is newest accordingly, heavily loaded module checks whether corresponding working node is located
In the synchronous non-application state of configuration, if corresponding working node is in the synchronous non-application state of configuration, heavily loaded module notice master
The node corresponding working node refusal receives new task；
If corresponding working node is in configuration application state, heavily loaded module assignment task gives the corresponding work
Node.
The present invention provides version management and log for all configurations, and the configuration of any version can be applied in maintenance process；
While cluster configuration file dynamic instrumentation loads, guarantee not interrupting currently just executing for task；Unified configuration modification is provided
Entrance increases validity checking to the configuration of modification；Configuration management flow control and configuration merge application, guarantee that cluster will not be because
Frequently restart for configuration modification.
Further, the configuration management client-side management cluster configuration in step 1 includes modification cluster configuration, is each configuration
Input increases validity checking and permission control, checks configuration application all old versions and relevant information, by any version
This configuration application is merged into any cluster state.
Further, the configuration management server end in step 2 has flow control function.
Further, flow control function includes abandoning unreasonable configuration management client request.
Further, the node in step 3 includes host node and working node, wherein configuration detecting module is located at cluster
On each node.
Further, the configuration detecting module in step 3 is using api interface to the cluster configuration node at configuration management center
It is monitored in real time.
Further, the host node cluster configuration state-maintenance module in step 4 is located in the host node of cluster.
Compared with prior art, the present invention having the following advantages and benefits:
1, the present invention utilizes configuration management client, submits relevant configured parameter to configuration management center.Client has
The function of legitimacy detection and permission control guarantees that configuration is correctly modified；
2, the present invention utilizes the monitoring mechanism of body in configuration management, and each node of cluster detects the change of configuration simultaneously in real time
It is synchronized to local.Each node determines to update according to each self-supported situation the opportunity of configuration, when update merge it is all exist update
Or the configuration item of conflict, and the completion status that configuration updates is submitted to host node.Host node according to configuration update state table into
Row task schedule and resource allocation；
3, configuration management client of the invention provides configuration land parcel change trace function, can check all of cluster arbitrary node
The date is applied in history configuration application and its relevant information, including configuration, configuration take-effective time and configuration application cluster state etc..It can
The configuration merging of any cluster state of any version is applied to current cluster state by management client.
Detailed description of the invention
Attached drawing described herein is used to provide to further understand the embodiment of the present invention, constitutes one of the application
Point, do not constitute the restriction to the embodiment of the present invention.In the accompanying drawings:
Fig. 1 is that working node of the present invention configures dynamic instrumentation load flow chart.
Specific embodiment
To make the objectives, technical solutions, and advantages of the present invention clearer, below with reference to embodiment and attached drawing, to this
Invention is described in further detail, and exemplary embodiment of the invention and its explanation for explaining only the invention, are not made
For limitation of the invention.
Embodiment
As shown in Figure 1, a kind of cluster configuration dynamic instrumentation merges loading method, comprising:
Step 1, pass through configuration management client-side management cluster configuration；Configuration management client-side management cluster in step 1
Configuration includes modification cluster configuration, inputs increase validity checking and permission control, the institute for checking configuration application for each configuration item
There are old version and relevant information, the configuration application of any version is merged into any cluster state.
By configuration management client-side management cluster configuration, cluster configuration can only be modified by configuration management client, together
When for each configuration item input increase validity checking and permission control, greatly ensure that the validity of configuration；It can look into simultaneously
All old versions and relevant information for seeing configuration application, can be merged into any cluster state for the configuration application of any version.
Step 2, configuration management server end receives the configuration modification request of configuration management client and configuration by specified
Configuration management center is written in format；Configuration management server end in step 2 has flow control function.
Flow control function includes abandoning unreasonable configuration management client request.
Configuration management server end receives the request of client configuration modification and configuration management is written by specified format in configuration
Center；Flow control function is increased simultaneously, abandons unreasonable client request, avoids cluster abnormal due to flow attacking etc.
It is frequent and restarts the case where even paralysing.
Step 3, configuration detecting module monitors the cluster configuration node at configuration management center in real time；When configuration detecting module inspection
When measuring the content of node and changing, cluster synchronization it is new be configured to local, and update the state of respective nodes；In step 3
Node include host node and working node, wherein configuration detecting module is located on each node of cluster.Configuration in step 3
Detecting module is monitored in real time using cluster configuration node of the api interface to configuration management center.
Step 4, host node cluster configuration state-maintenance module obtains matching for each working node from each working node of cluster
Set state；The configuration status of working node includes that application state, configuration are synchronous not using shape for the configuration version number of node, configuration
State, as shown in table 1, unique identification of the NodeId as each node；Configuration version number of the Version as each node, the version
Number must be it is newest, the configuration of the node is only effectively；Configuration application state of the State as each node, in State
Be also divided into Applied and Synced, wherein Applied: configuration has been applied, which can receive new task；Synced: the section
Point, which has synchronized, is configured to local, but does not apply also；When each working node start or configure update when, each working node by oneself
State registration to host node；Host node cluster configuration state-maintenance module in step 4 is located in the host node of cluster.
Table 1
Step 5, heavily loaded module is located on each node；When heavily loaded module is located on the host node of cluster, step 6 is executed；
When heavily loaded module is located on the working node of cluster, step 7 is executed；
Step 6, when heavily loaded module is in host node, heavily loaded module assignment task is corresponding to first checking before working node
The state of working node, if corresponding working node configuration updates and configuration has been applied, heavily loaded module assignment task is to phase
The working node answered；
Step 7, when heavily loaded module is in working node,
Heavily loaded module checks whether the configuration version number of corresponding working node newest, i.e. Version whether be it is newest,
If the configuration version number of working node is not newest accordingly, 3 are thened follow the steps；
If the configuration version number of working node is newest accordingly, heavily loaded module checks whether corresponding working node is located
In the synchronous non-application state of configuration, i.e. Synced, if corresponding working node is in the synchronous non-application state of configuration, heavily loaded mould
The block notice host node corresponding working node refusal receives new task；After the completion of corresponding all tasks of node, answer
With new configuration, using after the completion, modification state is Applied and notifies host node.
If working node is in configuration application state, i.e. Synced accordingly, then heavily loaded module assignment task, which is given, is somebody's turn to do
Corresponding working node.
The invention avoids as maintenance personnel cluster is unfamiliar with or the cluster configuration as caused by artificial fault not
Unanimous circumstances, client make validity checking to the parameter of configuration item, guarantee that all configurations are correct effective；Utilize configuration management
Centre management cluster configuration guarantees the consistency and real-time of configuration；Heavily loaded module based on node load state ensure that new
Configuration the normal execution of existing task will not be influenced while can correctly be applied to all new tasks.Guarantee correctness
While improve the resource utilization of cluster.
Above-described specific embodiment has carried out further the purpose of the present invention, technical scheme and beneficial effects
It is described in detail, it should be understood that being not intended to limit the present invention the foregoing is merely a specific embodiment of the invention
Protection scope, all within the spirits and principles of the present invention, any modification, equivalent substitution, improvement and etc. done should all include
Within protection scope of the present invention.
Claims (7)
1. a kind of cluster configuration dynamic instrumentation merges loading method characterized by comprising
Step 1, pass through configuration management client-side management cluster configuration；
Step 2, configuration management server end receives the configuration modification request of configuration management client and specified format is pressed in configuration
Configuration management center is written；
Step 3, configuration detecting module monitors the cluster configuration node at configuration management center in real time；When configuration detecting module detects
When the content of node changes, cluster synchronization it is new be configured to local, and update the state of respective nodes；
Step 4, host node cluster configuration state-maintenance module obtains the configuration shape of each working node from each working node of cluster
State；The configuration status of working node includes the configuration version number of node, configuration application state, the synchronous non-application state of configuration；
When each working node start or configure update when, each working node is by the state registration of oneself to host node；
Step 5, heavily loaded module is located on each node；When heavily loaded module is located on the host node of cluster, step 6 is executed；When weight
It carries module to be located on the working node of cluster, executes step 7；
Step 6, when heavily loaded module is in host node, heavily loaded module assignment task first checks corresponding work to working node before
The state of node, if corresponding working node configuration updates and configuration has been applied, heavily loaded module assignment task is to corresponding
Working node；
Step 7, when heavily loaded module is in working node,
Heavily loaded module checks whether the configuration version number of corresponding working node is newest, if the configuration version of corresponding working node
This number be not it is newest, then follow the steps 3；
If the configuration version number of working node is newest accordingly, heavily loaded module is checked whether corresponding working node is in and is matched
Synchronous non-application state is set, if corresponding working node is in the synchronous non-application state of configuration, heavily loaded module notifies host node
The corresponding working node refusal receives new task；
If corresponding working node is in configuration application state, heavily loaded module assignment task gives the corresponding work section
Point.
2. a kind of cluster configuration dynamic instrumentation according to claim 1 merges loading method, which is characterized in that in step 1
Configuration management client-side management cluster configuration include modification cluster configuration, for the input of each configuration item increase validity checking and power
The configuration application of any version is merged into Arbitrary Sets by limit control, all old versions for checking configuration application and relevant information
Group's state.
3. a kind of cluster configuration dynamic instrumentation according to claim 1 merges loading method, which is characterized in that in step 2
Configuration management server end have flow control function.
4. a kind of cluster configuration dynamic instrumentation according to claim 3 merges loading method, which is characterized in that flow control
Function includes abandoning unreasonable configuration management client request.
5. a kind of cluster configuration dynamic instrumentation according to claim 1 merges loading method, which is characterized in that in step 3
Node include host node and working node, wherein configuration detecting module is located on each node of cluster.
6. a kind of cluster configuration dynamic instrumentation according to claim 1 merges loading method, which is characterized in that in step 3
Configuration detecting module monitored in real time using cluster configuration node of the api interface to configuration management center.
7. a kind of cluster configuration dynamic instrumentation according to claim 1 merges loading method, which is characterized in that in step 4
Host node cluster configuration state-maintenance module be located in the host node of cluster.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810744414.3A CN108958840B (en) | 2018-07-09 | 2018-07-09 | Dynamic detection, merging and loading method for cluster configuration |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810744414.3A CN108958840B (en) | 2018-07-09 | 2018-07-09 | Dynamic detection, merging and loading method for cluster configuration |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108958840A true CN108958840A (en) | 2018-12-07 |
| CN108958840B CN108958840B (en) | 2021-01-22 |
Family
ID=64482572
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810744414.3A Active CN108958840B (en) | 2018-07-09 | 2018-07-09 | Dynamic detection, merging and loading method for cluster configuration |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108958840B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112612545A (en) * | 2020-12-23 | 2021-04-06 | 北京浪潮数据技术有限公司 | Configuration hot loading system, method, equipment and medium of server cluster |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100198955A1 (en) * | 2009-02-05 | 2010-08-05 | Fujitsu Limited | Computer-readable recording medium storing software update command program, software update command method, and information processing device |
| US20150106796A1 (en) * | 2013-10-15 | 2015-04-16 | Fujitsu Technology Solutions Intellectual Property Gmbh | Class loading device for a java runtime environment, cluster system and method of executing a function |
| CN105005487A (en) * | 2015-06-29 | 2015-10-28 | 清华大学 | Continuous-service high-performance cluster operating system online upgrading method |
| CN106603281A (en) * | 2016-12-07 | 2017-04-26 | 乐视控股（北京）有限公司 | Configuration file management method and system |
| CN106843930A (en) * | 2016-12-23 | 2017-06-13 | 江苏途致信息科技有限公司 | Streaming dynamic configuration more new architecture and method based on zookeeper |
| CN107025139A (en) * | 2017-03-21 | 2017-08-08 | 北京天云融创软件技术有限公司 | A kind of high-performance calculation Scheduling Framework based on cloud computing |
| CN107465714A (en) * | 2017-01-23 | 2017-12-12 | 北京思特奇信息技术股份有限公司 | A kind of configuration data dynamic update system and method based on application cluster |
| CN107506213A (en) * | 2017-08-30 | 2017-12-22 | 深圳市小牛在线互联网信息咨询有限公司 | Dynamic Configuration, device, storage medium and computer equipment |
| CN108255592A (en) * | 2017-12-19 | 2018-07-06 | 武汉市烽视威科技有限公司 | A kind of Quartz clusters timing task processing system and method |
-
2018
- 2018-07-09 CN CN201810744414.3A patent/CN108958840B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100198955A1 (en) * | 2009-02-05 | 2010-08-05 | Fujitsu Limited | Computer-readable recording medium storing software update command program, software update command method, and information processing device |
| US20150106796A1 (en) * | 2013-10-15 | 2015-04-16 | Fujitsu Technology Solutions Intellectual Property Gmbh | Class loading device for a java runtime environment, cluster system and method of executing a function |
| CN105005487A (en) * | 2015-06-29 | 2015-10-28 | 清华大学 | Continuous-service high-performance cluster operating system online upgrading method |
| CN106603281A (en) * | 2016-12-07 | 2017-04-26 | 乐视控股（北京）有限公司 | Configuration file management method and system |
| CN106843930A (en) * | 2016-12-23 | 2017-06-13 | 江苏途致信息科技有限公司 | Streaming dynamic configuration more new architecture and method based on zookeeper |
| CN107465714A (en) * | 2017-01-23 | 2017-12-12 | 北京思特奇信息技术股份有限公司 | A kind of configuration data dynamic update system and method based on application cluster |
| CN107025139A (en) * | 2017-03-21 | 2017-08-08 | 北京天云融创软件技术有限公司 | A kind of high-performance calculation Scheduling Framework based on cloud computing |
| CN107506213A (en) * | 2017-08-30 | 2017-12-22 | 深圳市小牛在线互联网信息咨询有限公司 | Dynamic Configuration, device, storage medium and computer equipment |
| CN108255592A (en) * | 2017-12-19 | 2018-07-06 | 武汉市烽视威科技有限公司 | A kind of Quartz clusters timing task processing system and method |
Non-Patent Citations (1)
| Title |
|---|
| CODINGBOYJACK: "Zookeeper分布式过程协同技术详解笔记（二）", 《HTTPS://WWW.JIANSHU.COM/P/034F1C6ECD6B》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112612545A (en) * | 2020-12-23 | 2021-04-06 | 北京浪潮数据技术有限公司 | Configuration hot loading system, method, equipment and medium of server cluster |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108958840B (en) | 2021-01-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107783975B (en) | Method and device for synchronous processing of distributed databases | |
| CN110750592B (en) | Data synchronization method, device and terminal equipment | |
| CN106302596A (en) | A kind of method and apparatus of service discovery | |
| CN113569987A (en) | Model training method and device | |
| CN107741876A (en) | A kind of virtual machine process monitoring system and method | |
| US20220171652A1 (en) | Distributed container image construction scheduling system and method | |
| WO2016082594A1 (en) | Data update processing method and apparatus | |
| CN110063042A (en) | A kind of response method and its terminal of database failure | |
| CN114691050B (en) | Cloud native storage method, device, equipment and medium based on kubernets | |
| CN105955838A (en) | System halt reason check method and device | |
| CN117608825A (en) | Resource management method based on multi-cloud management platform and related equipment | |
| CN114090113B (en) | Method, device, equipment and storage medium for dynamically loading data source processing plug-in | |
| CN108958840A (en) | A kind of cluster configuration dynamic instrumentation merging loading method | |
| CN112256989A (en) | Page loading method and device based on offline package, terminal equipment and storage medium | |
| CN115147031B (en) | Clearing workflow execution method, device, equipment and medium | |
| US9262151B2 (en) | Methods and systems for automatic configuration of algorithms in a system based on self aware algorithms | |
| CN109491841A (en) | A method of improving Spark on yarn real-time task reliability | |
| CN110618884A (en) | Fault monitoring method, virtualized network function module manager and storage medium | |
| CN104102583A (en) | High-availability cluster software distributed automated testing framework | |
| CN113901047A (en) | Simple cluster master-slave election method based on memory database | |
| CN112350837A (en) | Cloud platform-based power application cluster management method and device | |
| CN112000720A (en) | Management method and management system for database connection and database connection pool | |
| CN104298763A (en) | Web-based external access method of structured database system | |
| Qiu et al. | Rta: Real time actionable events detection as a service | |
| CN117763041A (en) | Data table structure synchronization method and device applied to data aggregation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |