CN110134372A - A kind of rule-based zookeeper session external management system - Google Patents
A kind of rule-based zookeeper session external management system Download PDFInfo
- Publication number
- CN110134372A CN110134372A CN201910198627.5A CN201910198627A CN110134372A CN 110134372 A CN110134372 A CN 110134372A CN 201910198627 A CN201910198627 A CN 201910198627A CN 110134372 A CN110134372 A CN 110134372A
- Authority
- CN
- China
- Prior art keywords
- session
- detector
- rule
- zookeeper
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/20—Software design
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention discloses a kind of rule-based zookeeper session external management systems, including communication module, session controller, rules administration module, detector management module, rule execution module and detector execution module, communication module output end connects session controller, rules administration module and detector management module, session controller output end concatenate rule execution module and detector execution module；Communication module is used to realize network communication, and by server and groups of clients at；Rules administration module is used to definition rule, and is made of condition and movement, and administrator modifies update according to actual needs.The session mechanism of zookeeper is changed into semi-automation by full-automation, greatly strengthens the adaptability to network environment by the present invention by introducing outer guard system；The model of rule-based driving realizes various dimensions tactical management；It realizes based on database custom rule/detector flexible management.
Description
Technical field
The present invention relates to field of cloud computer technology, specially a kind of rule-based zookeeper session exterior tube
Reason system.
Background technique
Zookeeper is the distributed application program coordination service an of open source code, can be used for various distributed systems,
By taking certain storage cluster as an example:
When memory node starting, connection zookeeper first, zookeeper server is the distribution of each client
One session, and start session tracker.Session tracker is a poll thread, periodically, is repeatedly examined
All session are looked into, after session expires, session tracker is responsible for cleaning up them.
The expiration time of session is responsible for refreshing by client, when client and zookeeper progress communication transaction,
Corresponding session is refreshed automatically.Connection for long idle, client can periodically send PING instruction, zookeeper
After receiving, refresh operation can be also executed.
Under proper network environment, the session mechanism of zookeeper runs very perfect, timeliness and automation
Degree is all very high, theoretically, the unmanned O&M of storage cluster may be implemented.It, may be exactly but when flash occurs in network
Cause PING instruction that can not reach, zookeeper thinks that the corresponding session of the memory node has expired, and then is cleared up,
At this point, the memory node is offline from the point of view of storage cluster visual angle；In next time cycle, network recovery is normal,
Zookeeper receives the PING instruction of memory node transmission, but can not find corresponding session, it is believed that this is one new
Connection, from the point of view of storage cluster visual angle, the memory node is online.
Above process may occur repeatedly, the change of invalid topology caused by this network flash, can greatly aggravate to deposit
The burden of accumulation, and various distributed system urgent problems all the time.Therefore we make improvement to this, mention
A kind of rule-based zookeeper session external management system out.
Summary of the invention
To solve defect of the existing technology, the present invention is provided outside a kind of rule-based zookeeper session
Portion's management system.
In order to solve the above-mentioned technical problems, the present invention provides the following technical solutions:
A kind of rule-based zookeeper session external management system of the present invention, including communication module,
Session controller, rules administration module, detector management module, rule execution module and detector execution module, it is described logical
Believe module output end connection session controller, rules administration module and detector management module, the session control
Device output end concatenate rule execution module and detector execution module；The communication module is used to realize network communication, and by
Server and groups of clients at；Rules administration module is used to definition rule, and is made of condition and movement, and administrator is according to reality
Demand modification in border updates；Detector management module is used to define detector, including built-in detector and the customized detector of user,
Administrator modifies update according to actual needs；Session controller is core component, defines the control to session start and stop
Logic can freely call other modules as needed；Rule execution module is responsible for resolution rules, provides rule execution up and down
Text simultaneously executes；Detector execution module is responsible for parsing detector, provides detector and executes context and execute.
As a preferred technical solution of the present invention, comprising the following steps:
A. session is matched, if not being matched to session, ignores and terminates process；If being matched to session, mention
Take out connection belonging to the session；
B. the list of rules of the matching connection, sequence executing rule are inquired；
C. determined whether open detector according to regular implementing result；Operation detector issues control instruction, terminates
Session external management process, and subsequent control power is given back into zookeeper server.
As a preferred technical solution of the present invention, the rule and the realization object of detector include but is not limited to object
Manage table, view and function.
As a preferred technical solution of the present invention, the rule refers to the offline rule of memory node.
As a preferred technical solution of the present invention, the session controller is responsible for session binding and closing
Service logic, comprising the following steps:
Step 1: when zookeeper creates node, outer guard system is notified to bind session and memory node:
A.zookeeper increases node；
B. according to node path naming method, judge whether the node belongs to storage cluster, if so, the outer piping of notice
System；
C. outer tube system analysis node path, obtains cluster and path, establishes binding relationship with session, is incorporated to
Library；
Step 2: when zookeeper detects that session is expired, meeting guard system outside first initiates session close
Request, after securing permission, can execute normal switching-off process；Otherwise, the time-out time for updating session remains lower whorl detection:
A. outer tube system queries database obtains binding information: cluster, path, and parses memory node title
node；
B. according to cluster, node rule searching list, and sequence executes strictly all rules；
C. if there is part rule executes failure, into retrying process；
D. if strictly all rules run succeeded, process is executed into detector；
E. detector executes process 1: inquiry detector list, and sequence executes, if there is segment detector executes failure,
Illustrate that memory node is really offline, notice zookeeper closes session；
F. detector executes process 2: if all detectors are carried out success, illustrating that memory node is still survived, enters
Retry process；
G. retry process: statistics number of retries judges whether to reach and retries the upper limit, retries, notify if it is, terminating
Zookeeper closes session；Otherwise number of retries adds one, and notice zookeeper does not allow to close session.
As a preferred technical solution of the present invention, the zookeeper customization, to be adapted to outer piping
System.
As a preferred technical solution of the present invention, the zookeeper the following steps are included:
A. communication module is constructed first, can use the HTTP+JSON of standard, communication process is packaged into Command tune
With；
B.zookeeper increases by three extended attributes: url, retryLimit and retryInteval；
When c.zookeeper creates node, BindSessionCommand is called, binds session；
D.zookeeper is cleared up before expired session, calls CloseSessionCommand, extensive after obtaining permission
Recovery has process, otherwise, refreshes expired time as after retryInteval.
The beneficial effects of the present invention are: this kind of rule-based zookeeper session external management system:
1, by the outer guard system of introducing, the session mechanism of zookeeper is changed into semi-automation by full-automation,
Greatly strengthen the adaptability to network environment；
2, the model of rule-based driving realizes various dimensions tactical management；
3, it realizes based on database custom rule/detector flexible management.
Detailed description of the invention
Attached drawing is used to provide further understanding of the present invention, and constitutes part of specification, with reality of the invention
It applies example to be used to explain the present invention together, not be construed as limiting the invention.In the accompanying drawings:
Fig. 1 is a kind of system structure diagram of rule-based zookeeper session external management system of the present invention；
Fig. 2 is a kind of session binding sequence of rule-based zookeeper session external management system of the present invention
Column figure；
Fig. 3 is a kind of method flow diagram of rule-based zookeeper session external management system of the present invention；
Fig. 4 is the present invention a kind of communication module, the rule of rule-based zookeeper session external management system
The composition schematic diagram of management module and detector management module.
In figure: 1, communication module；2, session controller；3, rules administration module；4, detector management module；5, it advises
Then execution module；6, detector execution module.
Specific embodiment
Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings, it should be understood that preferred reality described herein
Apply example only for the purpose of illustrating and explaining the present invention and is not intended to limit the present invention.
Embodiment: a kind of rule-based zookeeper session external management system of the present invention, including communication module
1, session controller 2, rules administration module 3, detector management module 4, rule execution module 5 and detector execution module
6,1 output end of communication module connects session controller 2, rules administration module 3 and detector management module 4, session
2 output end concatenate rule execution module 5 of controller and detector execution module 6；As shown in figure 4, communication module 1 is used to realize net
Network communication, and by server and groups of clients at；Rules administration module 3 is used to definition rule, and by condition and action group
At administrator modifies update according to actual needs；Detector management module 4 is used to define detector, including built-in detector with
The customized detector of user, administrator modify update according to actual needs；Session controller 2 is core component, is defined pair
The control logic of session start and stop can freely call other modules as needed；Rule execution module 5 is responsible for parsing rule
Then, rule is provided to execute context and execute；Detector execution module 6 is responsible for parsing detector, provides detector and executes up and down
Text simultaneously executes.
Wherein, comprising the following steps:
Step 1, matching session ignore if not being matched to session and terminate process；If being matched to session,
Then extract connection belonging to the session；
Step 2, the list of rules for inquiring the matching connection, sequence executing rule；
Step 3 determines whether open detector according to regular implementing result；Operation detector issues control instruction,
Terminate session external management process, and subsequent control power is given back into zookeeper server.
Wherein, the rule and the realization object of detector include but is not limited to physical table, view and function.
Firstly, introducing rule and two concepts of detector.Here rule refers in particular to the offline rule of memory node, can be with
Specified threshold also can specify frequency etc..That is, even if zookeeper discovery and the network of memory node have disconnected,
Will not assert at once offline, but a series of specified rules be first carried out, meet rule just allow it is offline.Meanwhile in rule
Except, a kind of active probe mechanism is introduced, the auxiliary tool whether offline as judgement memory node.
A kind of typical rule definition such as table 1:
Table 1
Wherein, quota indicates limit, and unit indicates chronomere, can be one day (d or D), be also possible to one hour (h
Or H), scope indicates scope of statistics, and 1 indicates all nodes in cluster, and 0 indicates individual node.
Detector definition such as table 2:
Table 2
Wherein, two detectors ping and telnet built in system, corresponding internal command are PING and TELNET.User
The address ip and end of destination node can also be respectively indicated using parameter-embedded { ip }, { port } with customized detector script
Slogan.
Session controller
Session controller is responsible for the service logic of session binding and closing.
When zookeeper creates node, outer guard system is notified to bind session and memory node, as shown in Figure 2:
1.zookeeper increases node；
2. judging whether the node belongs to storage cluster according to node path naming method, if so, the outer piping of notice
System；
3. outer tube system analysis node path, obtains cluster and path, binding relationship is established with session, is incorporated to
Library；
When zookeeper detects that session is expired, meeting guard system outside first initiates session close request, obtains
After must permitting, normal switching-off process can be executed；Otherwise, the time-out time for updating session remains lower whorl detection.Such as Fig. 3 institute
Show:
1. outer tube system queries database obtains binding information: cluster, path, and parses memory node title
node；
2. according to cluster, node rule searching list, and sequence executes strictly all rules；
3. if there is part rule executes failure, into retrying process；
4. if executing process into detector strictly all rules run succeeded；
5. detector executes process 1: inquiry detector list, and sequence executes, if there is segment detector executes failure,
Illustrate that memory node is really offline, notice zookeeper closes session；
6. detector executes process 2: if all detectors are carried out success, illustrating that memory node is still survived, enter
Retry process；
7. retrying process: statistics number of retries judges whether to reach and retries the upper limit, retry if it is, terminating, notifies
Zookeeper closes session；Otherwise number of retries adds one, and notice zookeeper does not allow to close session.
zookeeper
Finally, zookeeper is also required to customization, to be adapted to outer guard system.It first has to do, exactly communication module
Building can use the HTTP+JSON of standard, and communication process is packaged into Command and is called.It is as shown in table 3:
Table 3
Next, zookeeper increases by three extended attributes:
1.url: the url (defaulting to sky, expression is not turned on) of outer guard system；
2.retryLimit: maximum reattempt times (default to 10)；
3.retryInteval: retrying interval, and unit is millisecond (defaulting to 15000ms)；
When zookeeper creates node, BindSessionCommand is called, binds session；
Zookeeper is cleared up before expired session, calls CloseSessionCommand, after obtaining permission, is restored
Otherwise original process refreshes expired time as after retryInteval.
Finally, it should be noted that the foregoing is only a preferred embodiment of the present invention, it is not intended to restrict the invention,
Although the present invention is described in detail referring to the foregoing embodiments, for those skilled in the art, still may be used
To modify the technical solutions described in the foregoing embodiments or equivalent replacement of some of the technical features.
All within the spirits and principles of the present invention, any modification, equivalent replacement, improvement and so on should be included in of the invention
Within protection scope.
Claims (7)
1. a kind of rule-based zookeeper session external management system, which is characterized in that including communication module (1),
Session controller (2), rules administration module (3), detector management module (4), rule execution module (5) and detector are held
Row module (6), communication module (1) output end connection session controller (2), rules administration module (3) and detector
Management module (4), session controller (2) the output end concatenate rule execution module (5) and detector execution module (6)；
The communication module (1) is used to realize network communication, and by server and groups of clients at；Rules administration module (3) is used to
Definition rule, and be made of condition and movement, administrator modifies update according to actual needs；Detector management module (4) is used
Define detector, including built-in detector and the customized detector of user, built-in detector have ping detector and telnet
Detector, customized detector are that administrator modifies update according to actual needs；Session controller (2) is core component,
The control logic to session start and stop is defined, can freely call other modules as needed；Rule execution module (5)
It is responsible for resolution rules, rule execution context is provided and is executed；Detector execution module (6) is responsible for parsing detector, provides spy
Device is surveyed to execute context and execute.
2. a kind of rule-based zookeeper session external management system according to claim 1, feature exist
In, comprising the following steps:
A. session is matched, if not being matched to session, ignores and terminates process；If being matched to session, extract
Connection belonging to the session；
B. the list of rules of the matching connection, sequence executing rule are inquired；
C. determined whether open detector according to regular implementing result；Operation detector issues control instruction, terminates
Session external management process, and subsequent control power is given back into zookeeper server.
3. a kind of rule-based zookeeper session external management system according to claim 1, feature exist
In the rule and the realization object of detector include but is not limited to physical table, view and function.
4. a kind of rule-based zookeeper session external management system according to claim 3, feature exist
In the rule refers to the offline rule of memory node.
5. a kind of rule-based zookeeper session external management system according to claim 1, feature exist
In the session controller (2) is responsible for the service logic of session binding and closing, comprising the following steps:
Step 1: when zookeeper creates node, outer guard system is notified to bind session and memory node:
A.zookeeper increases node；
B. according to node path naming method, judge whether the node belongs to storage cluster, if so, the outer guard system of notice；
C. outer tube system analysis node path, obtains cluster and path, establishes binding relationship with session, and be put in storage；
Step 2: when zookeeper detects that session is expired, meeting guard system outside first initiates session close request,
After securing permission, normal switching-off process can be executed；Otherwise, the time-out time for updating session remains lower whorl detection:
A. outer tube system queries database obtains binding information: cluster, path, and parses memory node title node；
B. according to cluster, node rule searching list, and sequence executes strictly all rules；
C. if there is part rule executes failure, into retrying process；
D. if strictly all rules run succeeded, process is executed into detector；
E. detector executes process 1: inquiry detector list, and sequence executes, if there is segment detector executes failure, explanation
Memory node is really offline, and notice zookeeper closes session；
F. detector executes process 2: if all detectors are carried out success, illustrating that memory node is still survived, into retrying
Process；
G. retry process: statistics number of retries judges whether to reach and retries the upper limit, retries, notify if it is, terminating
Zookeeper closes session；Otherwise number of retries adds one, and notice zookeeper does not allow to close session.
6. a kind of rule-based zookeeper session external management system according to claim 5, feature exist
In the zookeeper customization, to be adapted to outer guard system.
7. a kind of rule-based zookeeper session external management system according to claim 6, feature exist
In, the zookeeper the following steps are included:
A. communication module is constructed first, can use the HTTP+JSON of standard, and communication process is packaged into Command and is called；
B.zookeeper increases by three extended attributes: url, retryLimit and retryInteval；
When c.zookeeper creates node, BindSessionCommand is called, binds session；
D.zookeeper is cleared up before expired session, calls CloseSessionCommand, after obtaining permission, restores former
There is process, otherwise, refreshes expired time as after retryInteval.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910198627.5A CN110134372A (en) | 2019-07-10 | 2019-07-10 | A kind of rule-based zookeeper session external management system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910198627.5A CN110134372A (en) | 2019-07-10 | 2019-07-10 | A kind of rule-based zookeeper session external management system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110134372A true CN110134372A (en) | 2019-08-16 |
Family
ID=67568794
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910198627.5A Pending CN110134372A (en) | 2019-07-10 | 2019-07-10 | A kind of rule-based zookeeper session external management system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110134372A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111597056A (en) * | 2020-05-18 | 2020-08-28 | 北京思特奇信息技术股份有限公司 | Distributed scheduling method, system, storage medium and device |
| CN114584489A (en) * | 2022-03-08 | 2022-06-03 | 浪潮云信息技术股份公司 | Ssh channel-based remote environment information and configuration detection method and system |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2000022793A1 (en) * | 1998-10-08 | 2000-04-20 | Nortel Networks Corporation | Service capable network |
| US20120271926A1 (en) * | 2010-06-23 | 2012-10-25 | Bulat Shakirzyanov | System and method for managing a computing cluster |
| CN103530726A (en) * | 2013-09-30 | 2014-01-22 | 张家港市速达电子商务有限公司 | Information management system |
| US20140280904A1 (en) * | 2013-03-14 | 2014-09-18 | Centurylink Intellectual Property Llc | Session initiation protocol testing control |
| CN105468727A (en) * | 2015-11-20 | 2016-04-06 | 国家电网公司 | Zookeeper based method for realizing MySQL strong-consistency copy |
| WO2016118876A1 (en) * | 2015-01-23 | 2016-07-28 | Ebay Inc. | Messaging and processing high volume data |
| CN106209943A (en) * | 2015-05-07 | 2016-12-07 | 中兴通讯股份有限公司 | The system of selection of communication node and device |
| CN106713391A (en) * | 2015-11-13 | 2017-05-24 | 北京新媒传信科技有限公司 | Session information sharing method and sharing system |
| CN109150936A (en) * | 2017-06-19 | 2019-01-04 | 北京宝兰德软件股份有限公司 | The sharing method and device of session object Session in a kind of distributed type assemblies |
| CN109885613A (en) * | 2019-01-03 | 2019-06-14 | 江苏智途科技股份有限公司 | Clustering data rule processing method |
-
2019
- 2019-07-10 CN CN201910198627.5A patent/CN110134372A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2000022793A1 (en) * | 1998-10-08 | 2000-04-20 | Nortel Networks Corporation | Service capable network |
| US20120271926A1 (en) * | 2010-06-23 | 2012-10-25 | Bulat Shakirzyanov | System and method for managing a computing cluster |
| US20140280904A1 (en) * | 2013-03-14 | 2014-09-18 | Centurylink Intellectual Property Llc | Session initiation protocol testing control |
| CN103530726A (en) * | 2013-09-30 | 2014-01-22 | 张家港市速达电子商务有限公司 | Information management system |
| WO2016118876A1 (en) * | 2015-01-23 | 2016-07-28 | Ebay Inc. | Messaging and processing high volume data |
| CN106209943A (en) * | 2015-05-07 | 2016-12-07 | 中兴通讯股份有限公司 | The system of selection of communication node and device |
| CN106713391A (en) * | 2015-11-13 | 2017-05-24 | 北京新媒传信科技有限公司 | Session information sharing method and sharing system |
| CN105468727A (en) * | 2015-11-20 | 2016-04-06 | 国家电网公司 | Zookeeper based method for realizing MySQL strong-consistency copy |
| CN109150936A (en) * | 2017-06-19 | 2019-01-04 | 北京宝兰德软件股份有限公司 | The sharing method and device of session object Session in a kind of distributed type assemblies |
| CN109885613A (en) * | 2019-01-03 | 2019-06-14 | 江苏智途科技股份有限公司 | Clustering data rule processing method |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111597056A (en) * | 2020-05-18 | 2020-08-28 | 北京思特奇信息技术股份有限公司 | Distributed scheduling method, system, storage medium and device |
| CN114584489A (en) * | 2022-03-08 | 2022-06-03 | 浪潮云信息技术股份公司 | Ssh channel-based remote environment information and configuration detection method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11411897B2 (en) | Communication method and communication apparatus for message queue telemetry transport | |
| CN109361532B (en) | High availability system and method for network data analysis and computer readable storage medium | |
| US7739367B2 (en) | Managing network-enabled devices | |
| US8730819B2 (en) | Flexible network measurement | |
| CN102197627B (en) | The improvement of flux of multicast convergence | |
| CN104753697B (en) | A kind of method, equipment and system controlling the automatic beginning of the network equipment | |
| US20020143755A1 (en) | System and methods for highly distributed wide-area data management of a network of data sources through a database interface | |
| US20080086573A1 (en) | Distributed Web Services Network Architecture | |
| CN101300779B (en) | Method and system for providing service to subscribers | |
| US10795744B2 (en) | Identifying failed customer experience in distributed computer systems | |
| CN107332719A (en) | A kind of method that daily record is analyzed in real time in CDN system | |
| EP2590113A1 (en) | On demand multi-objective network optimization | |
| US11210277B2 (en) | Distributing and processing streams over one or more networks for on-the-fly schema evolution | |
| US10887408B2 (en) | Remote monitoring of network communication devices | |
| Suh et al. | Toward highly available and scalable software defined networks for service providers | |
| CN110134372A (en) | A kind of rule-based zookeeper session external management system | |
| CN102955801B (en) | Data control method and system based on distributed data base system | |
| CN104270432B (en) | Based on drilling well industry Real-time Data Service system and data interactive method | |
| CN108681588A (en) | A kind of interface accesses real-time statistical method and system | |
| WO2019080386A1 (en) | Network service management method and system | |
| US20200244755A1 (en) | Distributed federation of endpoints with proxy synchronization | |
| Pu et al. | Practical implementation of an OPC UA multi-server aggregation and management architecture for IIoT | |
| CN104462235A (en) | Internet of Things general event service mechanism based on Restful Web Service | |
| CN104010008A (en) | Method for controlling mass invalid access to server | |
| CN111324655B (en) | Data subscription method based on differential data extraction in distributed simulation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20190816 |
|
| WD01 | Invention patent application deemed withdrawn after publication |