CN106603631A - Distributed message system and expansion method thereof - Google Patents
Distributed message system and expansion method thereof Download PDFInfo
- Publication number
- CN106603631A CN106603631A CN201611024161.XA CN201611024161A CN106603631A CN 106603631 A CN106603631 A CN 106603631A CN 201611024161 A CN201611024161 A CN 201611024161A CN 106603631 A CN106603631 A CN 106603631A
- Authority
- CN
- China
- Prior art keywords
- resource
- cluster
- dilatation
- agent side
- agent
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention provides a distributed message system and an expansion method thereof. The system comprises a cluster resource monitoring module used for monitoring hardware resource use information of an agent side in real time and summarizing resource use standard exceeding rate of the agent side, and sending an expansion request when the resource use standard exceeding rate of the agent side reaches a threshold value; a cluster resource management module used for receiving the expansion request sent by the cluster resource monitoring module, calculating expansion required resources according to the expansion request and sending a resource equalization request; and a load balance notification module used for receiving the resource equalization request sent by the cluster resource management module, initiating resource change operation to a public address monitored by a message sending side and a message receiving side, and recording the expansion required resources in the public address. The system can realize automatic expansion.
Description
Technical field
The present invention relates to field of computer technology, more particularly to a kind of distributed information system and expansion method.
Background technology
Distributed information system is serviced, and is a kind of while providing the interim storage and message of message on a different computer
Distribution function service.Referring to Fig. 1, it is distributed information system configuration diagram, whole system includes message sending end collection
Group 101, Agent cluster 102 and message sink end cluster 103, wherein, message sending end cluster 101 includes multiple message sending ends
1011st, 1012,1013 ..., Agent cluster 102 includes multiple agent sides 1021,1022 ..., and message sink end cluster 103 includes many
Individual message receiving terminal 1031,1032,1033 ....Message sending end (Producer or Producer) constructs disappearing for different themes
Breath, being deployed with being sent to LAN on the agent side (Broker) of message system service carries out interim storage, and message sink end (disappears
Expense person or Customer) according to itself subscribe to theme obtain from the message system cluster corresponding message carry out subsequently plus
Work.The characteristics of distributed information system is that messenger service is distributed in multiple stage computers, reaches High Availabitity by fault tolerant mechanism,
Height is supported to handle up by load balancing.
But under actual application environment, distributed information system service lacks the function of automatization's dilatation.With business
The growth of amount, message system service can cannot gradually support it is increasing handle up, now need operation maintenance personnel according to message
The load of system service, and estimate future load and carry out dilatation.At least there is following defect in this dilatation way：Firstly, it is necessary to
Operation maintenance personnel viewing system is loaded, after confirming that current distributed information system service may support load instantly, then
The dilatation of artificial message system cluster computer resource is carried out, artificial operation has the risk of maloperation；Secondly, future services amount
Growth be not the amount that accurately can be estimated, so as to cause this artificial dilatation operation need frequently to carry out；Finally, expand
Per the load balancing that a new demand servicing will occur a complete or collected works group is newly added during appearance, multiple load balancing is likely to cause clothes
Business is unstable.
The content of the invention
In order to improve the Stability and veracity that dilatation is carried out to distributed information system, the embodiment of the present invention provides a kind of
Distributed information system and expansion method.
A kind of one side according to embodiments of the present invention, there is provided distributed information system, including message sending end cluster,
Agent cluster and message sink end cluster, wherein, message sending end cluster includes multiple message sending ends, and Agent cluster includes many
Individual agent side, message sink end cluster include multiple message sink ends, and the system also includes：Cluster resource monitoring module, uses
In the hardware resource use information of agent side described in monitor in real time, and count the resource of the agent side and use exceeding standard rate, in institute
State and when the resource of agent side reaches threshold value using exceeding standard rate, send dilatation request；Cluster resource management module, it is described for receiving
The dilatation request that cluster resource monitoring module sends, calculates dilatation resource requirement according to dilatation request, and it is equal to send resource
Weighing apparatus request；Load balancing notification module, for receiving the resources balance request that the cluster resource management module sends, to
The public address monitored by message sending end and message sink end initiates resource change operation, and the dilatation resource requirement is recorded
On the public address.
Preferably, the cluster resource monitoring module includes cluster resource monitor client and cluster resource monitoring service
End, wherein, the cluster resource monitor client is deployed in the agent side, and the hardware for agent side described in monitor in real time is provided
Source use information, and the hardware resource use information for being monitored is reported into the cluster resource monitoring service end；The collection
Group resource monitor service end uses exceeding standard rate for the resource that the agent side is counted according to the hardware resource use information, and
The sent dilatation request when the resource of the agent side reaches threshold value using exceeding standard rate.
Preferably, the hardware resource use information of the agent side of the cluster resource monitor client monitoring includes：
CPU usage, physical memory utilization rate, disk space utilization rate, magnetic disc i/o handling capacity, and/or, network interface card I/O handling capacities；Institute
Cluster resource monitoring service end is stated with the time as dimension, the exceeded of every hardware resource use information is counted on the agent side
Rate, when the exceeding standard rate of any one hardware resource use information exceedes preset threshold value, that is, sends dilatation request.
Preferably, the cluster resource management module calculates agent side cluster dilatation on the basis of an agent side resource
The time interval of request, if time interval is less than predetermined standard value, judges cluster expansion frequency as height, it is determined that the dilatation
Resource requirement is the preset times of the given resource of preset dilatation.
Preferably, the cluster resource management module virtually dissolves the dilatation resource requirement according to Intel Virtualization Technology.
Preferably, the agent side after the resource change operation is listened to performs load balancing operation, and updates this
The cluster information caching on ground.
A kind of other side according to embodiments of the present invention, there is provided distributed information system expansion method, it is distributed to disappear
Breath system includes message sending end cluster, Agent cluster and message sink end cluster, wherein, message sending end cluster includes multiple
Message sending end, Agent cluster include multiple agent sides, and message sink end cluster includes multiple message sink ends, methods described bag
Include：The hardware resource use information of agent side described in monitor in real time, and count the resource of the agent side and use exceeding standard rate；Institute
When stating the resource of agent side and threshold value is reached using exceeding standard rate, calculate dilatation resource requirement；To message sending end and message sink end
The public address monitored initiates resource change operation, and the dilatation resource requirement is recorded on the public address.
Preferably, the hardware resource use information of the agent side includes：CPU usage, physical memory utilization rate, disk
Space utilization rate, magnetic disc i/o handling capacity, and/or, network interface card I/O handling capacities；Methods described also includes：Determine the agent side
Whether resource reaches threshold value using exceeding standard rate, including：With the time as dimension, counting every hardware resource on the agent side makes
With the exceeding standard rate of information, when the exceeding standard rate of any one hardware resource use information exceedes preset threshold value, that is, the generation is determined
The resource at reason end reaches threshold value using exceeding standard rate.
Preferably, the calculating dilatation resource requirement includes：On the basis of an agent side resource, agent side cluster is calculated
The time interval of dilatation request, if time interval is less than predetermined standard value, judges cluster expansion frequency as height, it is determined that described
Dilatation resource requirement is the preset times of the given resource of preset dilatation.
Preferably, methods described also includes：By Intel Virtualization Technology, the dilatation resource requirement is virtually dissolved.
Preferably, methods described also includes：The agent side performs load equal after the resource change operation is listened to
Weighing apparatus operation, and update local cluster information caching.
It can be seen that, distributed information system provided in an embodiment of the present invention, by the three module cooperative work for increasing newly, can be real
Now dilatation automatically.Specifically, by the hardware resource service condition at cluster resource monitoring module monitor agent end, so as to examine automatically
The load of distributed information system service is surveyed, and judges whether to need dilatation according to agent side hardware resource utilization rate, reached certainly
The dynamic purpose for judging dilatation opportunity, and without the need for manual monitoring；Dilatation is calculated by cluster resource management module in the dilatation stage
Resource requirement, and resource updates operation is initiated by load balancing resource informing module.By dividing that entity example of the present invention is provided
Cloth message system can be automatically performed dilatation, without the need for manual monitoring and operation, can be prevented effectively from it is artificial it is cumbersome, inaccurate,
The problems such as inefficiency.
Description of the drawings
Fig. 1 is distributed information system structural representation in prior art；
Fig. 2 is a kind of distributed information system structural representation that one embodiment of the invention is provided；
Fig. 3 is each module logic process schematic diagram in a kind of distributed information system that one embodiment of the invention is provided；
Fig. 4 is a kind of expansion method flow chart of distributed information system that one embodiment of the invention is provided.
Specific embodiment
It is understandable to enable the above objects, features and advantages of the present invention to become apparent from, it is below in conjunction with the accompanying drawings and concrete real
The present invention is further detailed explanation to apply mode.
Referring to Fig. 2, it is distributed information system configuration diagram.
Existing distributed information system includes message sending end cluster 201, Agent cluster 202 and message sink end cluster
203, Agent cluster 202 is responsible for the storage and transmission of message.Wherein, message sending end cluster 201 includes multiple message sending ends
2011st, 2012,2013 ..., Agent cluster 202 includes multiple agent sides 2021,2022 ..., and message sink end cluster 203 includes many
Individual message receiving terminal 2031,2032,2033 ....Message sending end (Producer or Producer) constructs disappearing for different themes
Breath, being deployed with being sent to LAN on the agent side (Broker) of message system service carries out interim storage, and message sink end (disappears
Expense person or Customer) according to itself subscribe to theme obtain from the message system cluster corresponding message carry out subsequently plus
Work.
Generally, distributed information system also includes Name Server cluster (not shown)s, Name
Server clusters or referred to as zookeeper, are made up of multiple stage computers, (such as public for providing the coordination informations such as name service
Common address).Message sending end and message sink end all can monitor the data on public address constantly, if the number on public address
According to changing, message sending end and message sink end can be known the very first time.For example, it is if having increased an agent side newly, public
Information on address can accordingly make change altogether, then with the very first time, message sending end and message sink end can find that this is new
Agent side on messenger service.
Compared to prior art, distributed information system provided in an embodiment of the present invention possesses automatic dilatation function.Relatively
In existing distributed information system, the newly-increased following three module of the distributed information system that the present invention is provided：Cluster resource is supervised
Control module, for the hardware resource use information of monitor in real time agent side, and the resource at statistics agent end uses exceeding standard rate, in generation
The resource at reason end sends dilatation request when reaching threshold value using exceeding standard rate；Cluster resource management module, for receiving cluster resource
The dilatation request that monitoring module sends, calculates dilatation resource requirement according to dilatation request, and sends resource balancing requests；Load is equal
Weighing apparatus notification module, for receiving the resources balance request of cluster resource management module transmission, to message sending end and message
The public address monitored by receiving terminal initiates resource change operation, and dilatation resource requirement is recorded on public address.
Fig. 2 is still referred to, cluster resource monitor client 204a and cluster resource monitoring service end 204b, cluster is shown
Resource management module 205 and load balancing notification module 206.Wherein, cluster resource monitor client 204a is deployed in each generation
On reason end, cluster resource monitor client 204a and cluster resource monitoring service end 204b collectively forms cluster resource monitoring mould
Block.
Preferably, above-mentioned three newly-increased modules are modules independent mutually, have independent function.Cluster resource monitoring module
It is made up of cluster resource monitor client 204a and cluster resource monitoring service end 204b, cluster resource monitor client 204a portions
On the agent side of providing message service, cluster resource monitoring service end 204b is deployed on a stand-alone machine for administration.Cluster is provided
Source control module 205 is deployed on a stand-alone machine.Load balancing notification module 206 is deployed on a stand-alone machine.
Three just newly-increased modules are discussed in detail as follows below.
1st, cluster resource monitoring module
In order to solve the problems, such as that dilatation timing is difficult, cluster resource monitoring module is introduced, and according to the money for monitoring
Source service condition is determining whether to reach dilatation threshold value.
For distributed information system, in practical application, the agent side of deployment services tends to exceed 1 or even reaches
To up to a hundred.For large-scale agent side cluster, cluster resource monitoring module needs to cover all agent sides, while as far as possible
The fast resource for collecting all agent sides, to ensure the data for counting on point at the same time.In order to ensure this point,
Cluster resource monitor client 204a (for example a, journey is disposed in the client of all of message system service place
Sequence), the hardware resource use information that cluster resource monitor client 204a can be in monitor in real time client, including but do not limit
In：CPU usage, physical memory utilization rate, disk space utilization rate, magnetic disc i/o handling capacity, network interface card I/O handling capacities, etc..
Cluster resource monitor client 204a can be arrived according to the above-mentioned agent side hardware resource use information of assigned frequency sampling, and report
On cluster resource monitoring service end 204b.
Cluster resource monitoring service end 204b is usually an independent computer.Receiving cluster resource monitor client
204a send agent side hardware resource use information on the premise of, cluster resource monitoring service end 204b with the time as dimension,
The exceeding standard rate of every hardware resource use information on all agent sides is counted, once there is single hardware resource use information
Exceeding standard rate exceedes defined threshold, that is, judge that the performance of distributed information system reaches bottleneck.Then to cluster resource management module
205 send dilatation request.
2nd, cluster resource management module 205
Cluster resource management module 205 is received and records the dilatation request of monitoring resource module, and is calculated needed for dilatation
Resource.For example, computational methods can be, minimum on the basis of an agent side resource, calculate the time of cluster dilatation request
Interval, if time interval be less than preset time interval threshold value, judge cluster expansion frequency it is higher, this given resource according to
The preset times (for example, 1.5 times or 2 times) of the given resource of last round of dilatation are calculated.
Above-mentioned resource capacity expansion request method can tackle the scene of portfolio rapid growth, reduce dilatation number of times.It is determined that
In the case of dilatation resource quantity, using Intel Virtualization Technology, the virtual machine resource for specifying number virtually is dissolved.For example, from
Installation procedure and deployment script are downloaded on the program file resource path (such as ftp) specified, deployment is performed, and is started service.
3rd, load balancing notification module 206
After the completion of service arrangement, cluster resource management module 205 initiates again balanced to load balancing notification module 206
Request.Load balancing notification module 206 can be to the public address of message sending end and the monitoring of message sink end (such as
Zookeeper resource change operation) is initiated, by the new message system service information record for adding cluster on the public address.
All agent sides after cluster information altering event is listened to, new load-balancing algorithm of seeking unity of action, and update local
Cluster information is cached.Thus, the new Service Source for adding can be recognized and be connected by message sending end and message sink end simultaneously.
It is above-mentioned this when the resource delay notification strategy that all deployment startup is finished, can effectively reduce load balancing and lead to
The number of times known so that the impact that dilatation behavior is caused to cluster service is minimized.Because traditional situation is, often increases one newly and disappear
Breath service, will be observed one-shot change by all message sending ends and message sink end, trigger a message sending end and disappear
The load balancing of breath receiving terminal.If newly-increased multiple stage messenger service, this process can be repeatedly executed.And it is above-mentioned " when resource
All deployment start the delay notification for finishing " it is exactly to avoid this situation, only all dispose in all newly-increased messenger services
Cheng Hou, then go to change the information of public address, one-shot change can be only observed at such message sending end and message sink end.
Referring to Fig. 3, in a kind of distributed information system provided for one embodiment of the invention, each module logic is processed and is shown
It is intended to.
The dilatation logic of distributed information system is as follows：
(1) the cluster resource monitor client disposed in agent side is monitored to the hardware resource of agent side, and will prison
The agent side hardware resource utilization rate controlled reports cluster resource monitoring service end；
(2) cluster resource monitoring service end judges whether to need to expand according to the agent side hardware resource utilization rate for receiving
Hold；In the case of dilatation is needed, dilatation request is sent to cluster resource management module；
(3) cluster resource management module calculates dilatation resource requirement according to dilatation request, and fictionalizes dilatation resource requirement,
Resources balance request is sent to load balancing notification module；
(4) load balancing resource informing module is connect to message sending end and message after resources balance request is received
The public address that receiving end is monitored jointly sends resource change operation, and dilatation resource requirement is recorded on public address.
(5) agent side after the resource change operation is listened to performs load balancing operation, and updates local cluster
Information cache, also, new resource is known by monitoring public address in message sending end and message sink end.
It can be seen that, distributed information system provided in an embodiment of the present invention, by the three module cooperative work for increasing newly, can be real
Now dilatation automatically.Specifically, by the hardware resource service condition at cluster resource monitoring module monitor agent end, so as to examine automatically
The load of distributed information system service is surveyed, and judges whether to need dilatation according to agent side hardware resource utilization rate, reached certainly
The dynamic purpose for judging dilatation opportunity, and without the need for manual monitoring；Dilatation is calculated by cluster resource management module in the dilatation stage
Resource requirement, and resource updates operation is initiated by load balancing resource informing module.By dividing that entity example of the present invention is provided
Cloth message system can be automatically performed dilatation, without the need for manual monitoring and operation, can be prevented effectively from it is artificial it is cumbersome, inaccurate,
The problems such as inefficiency.
In addition, the embodiment of the present invention also provides a kind of distributed information system expansion method.
The distributed information system such as front introduction, including message sending end cluster, Agent cluster and message sink end collection
Group, wherein, message sending end cluster includes multiple message sending ends, and Agent cluster includes multiple agent sides, message sink end collection
Group includes multiple message sink ends.
Referring to Fig. 2, distributed information system includes message sending end cluster 201, Agent cluster 202 and message sink end collection
Group 203, Agent cluster 202 is responsible for the storage and transmission of message.Wherein, message sending end cluster 201 includes that multiple message send
End 2011,2012,2013 ..., Agent cluster 202 includes multiple agent sides 2021,2022 ..., and message sink end cluster 203 includes
Multiple message receiving terminals 2031,2032,2033 ....Message sending end (Producer, Producer) constructs disappearing for different themes
Breath, being deployed with being sent to LAN on the agent side (Broker) of message system service carries out interim storage, and message sink end (disappears
Fei Zhe, Customer) corresponding message is obtained from the message system cluster and carry out following process according to the theme itself subscribed to.
Above-mentioned distributed information system also includes cluster resource monitor client 204a and cluster resource monitoring service end
204b, cluster resource management module 205 and load balancing notification module 206.Wherein, cluster resource monitor client 204a deployment
On each agent side, cluster resource monitor client 204a and cluster resource monitoring service end 204b collectively form cluster resource
Monitoring module.
Preferably, above-mentioned three newly-increased modules are modules independent mutually, have independent function.Cluster resource monitoring module
It is made up of cluster resource monitor client 204a and cluster resource monitoring service end 204b, cluster resource monitor client 204a portions
On the agent side of providing message service, cluster resource monitoring service end 204b is deployed on a stand-alone machine for administration.Cluster is provided
Source control module 205 is deployed on a stand-alone machine.Load balancing notification module 206 is deployed on a stand-alone machine.
Referring to Fig. 4, it is a kind of distributed information system expansion method flow chart provided in an embodiment of the present invention, the method bag
Include following steps S401-S403.
S401：The hardware resource use information of monitor in real time agent side, and the resource at statistics agent end uses exceeding standard rate.
Specifically, the hardware resource use information of agent side is included but is not limited to：CPU usage, physical memory utilization rate,
Disk space utilization rate, magnetic disc i/o handling capacity, network interface card I/O handling capacities.
Wherein, the resource of agent side can be as using the statistical of exceeding standard rate：With the time as dimension, agent side is counted
The exceeding standard rate of upper every hardware resource use information, exceedes preset threshold value in the exceeding standard rate of any one hardware resource use information
When, that is, determine that the resource of agent side reaches threshold value using exceeding standard rate.
S402：When the resource of agent side reaches threshold value using exceeding standard rate, dilatation resource requirement is calculated.
Specifically, calculate dilatation resource requirement and can take following manner：On the basis of an agent side resource, generation is calculated
The time interval of reason end cluster dilatation request, if time interval is less than predetermined standard value, judges cluster expansion frequency as height, then
Determine the preset times that dilatation resource requirement is the given resource of preset dilatation.After dilatation resource requirement is calculated, can pass through
Intel Virtualization Technology, virtually dissolves dilatation resource requirement.For example, download from specified program file resource path (such as ftp) and pacify
Dress program and deployment script, perform deployment, and start service.
S403：Resource change operation is initiated to the public address monitored by message sending end and message sink end, by dilatation
Resource requirement is recorded on public address.
Agent side after the resource change operation is listened to performs load balancing operation, and updates local cluster letter
Breath caching, also, new resource is known by monitoring public address in message sending end and message sink end.
In message distribution formula message system, message sending end and message sink end all can be monitored on public address constantly
Data, if the data on public address change, message sending end and message sink end can be known the very first time.For example,
If having increased an agent side newly, the information on public address can accordingly make change, then message sending end and message sink
End can find messenger service on this new agent side with the very first time.
The distributed information system expansion method provided by entity example of the present invention, is capable of achieving automatic dilatation, without the need for artificial
Monitoring and operate, the problems such as artificial cumbersome, inaccurate, inefficiency can be prevented effectively from.
It should be noted that for embodiment of the method, in order to be briefly described, therefore which to be all expressed as a series of action group
Close, but those skilled in the art should know, the embodiment of the present invention is not limited by described sequence of movement, because according to
According to the embodiment of the present invention, some steps can adopt other orders or while carry out.Secondly, those skilled in the art also should
Know, embodiment described in this description belongs to preferred embodiment, the involved action not necessarily present invention is implemented
Example is necessary.
Each embodiment in this specification is described by the way of progressive, what each embodiment was stressed be with
The difference of other embodiment, between each embodiment identical similar part mutually referring to.
Those skilled in the art are it should be appreciated that the embodiment of the embodiment of the present invention can be provided as method, device or calculate
Machine program product.Therefore, the embodiment of the present invention can using complete hardware embodiment, complete software embodiment or combine software and
The form of the embodiment of hardware aspect.And, the embodiment of the present invention can using wherein include computer at one or more can
With in the computer-usable storage medium (including but not limited to disk memory, CD-ROM, optical memory etc.) of program code
The form of the computer program of enforcement.
The embodiment of the present invention is with reference to method according to embodiments of the present invention, terminal unit (system), and computer program
The flow chart and/or block diagram of product is describing.It should be understood that can be by computer program instructions flowchart and/or block diagram
In each flow process and/or square frame and flow chart and/or flow process and/or square frame in block diagram combination.These can be provided
Computer program instructions are set to general purpose computer, special-purpose computer, Embedded Processor or other programmable data processing terminals
Standby processor is producing a machine so that held by the processor of computer or other programmable data processing terminal equipments
Capable instruction is produced for realizing in one flow process of flow chart or one square frame of multiple flow processs and/or block diagram or multiple square frames
The device of the function of specifying.
These computer program instructions may be alternatively stored in and can guide computer or other programmable data processing terminal equipments
In the computer-readable memory for working in a specific way so that the instruction being stored in the computer-readable memory produces bag
The manufacture of command device is included, the command device is realized in one side of one flow process of flow chart or multiple flow processs and/or block diagram
The function of specifying in frame or multiple square frames.
These computer program instructions can be also loaded in computer or other programmable data processing terminal equipments so that
On computer or other programmable terminal equipments perform series of operation steps to produce computer implemented process, so as to
The instruction performed on computer or other programmable terminal equipments is provided for realizing in one flow process of flow chart or multiple flow processs
And/or specify in one square frame of block diagram or multiple square frames function the step of.
Although having been described for the preferred embodiment of the embodiment of the present invention, those skilled in the art once know base
This creative concept, then can make other change and modification to these embodiments.So, claims are intended to be construed to
Including preferred embodiment and fall into the had altered of range of embodiment of the invention and change.
Finally, in addition it is also necessary to explanation, herein, such as first and second or the like relational terms be used merely to by
One entity or operation are made a distinction with another entity or operation, and are not necessarily required or implied these entities or operation
Between there is any this actual relation or order.And, term " including ", "comprising" or its any other variant are anticipated
Covering including for nonexcludability, so that a series of process, method, article or terminal unit including key elements is not only wrapped
Those key elements are included, but also including other key elements being not expressly set out, or also include for this process, method, article
Or the key element that terminal unit is intrinsic.In the absence of more restrictions, by wanting that sentence "including a ..." is limited
Element, it is not excluded that also there is other identical element in the process including the key element, method, article or terminal unit.
Above to a kind of distributed information system provided by the present invention and expansion method, it is described in detail, herein
In apply specific case the principle and embodiment of the present invention be set forth, the explanation of above example is only intended to side
Assistant solves the method for the present invention and its core concept；Simultaneously for one of ordinary skill in the art, according to the think of of the present invention
Think, will change in specific embodiments and applications, in sum, it is right that this specification content should not be construed as
The restriction of the present invention.
Claims (11)
1. a kind of distributed information system, including message sending end cluster, Agent cluster and message sink end cluster, wherein, institute
Stating message sending end cluster includes multiple message sending ends, and Agent cluster includes multiple agent sides, and message sink end cluster includes
Multiple message sink ends, it is characterised in that the system also includes：
Cluster resource monitoring module, for the hardware resource use information of agent side described in monitor in real time, and counts the agency
The resource at end uses exceeding standard rate, and dilatation request is sent when the resource of the agent side reaches threshold value using exceeding standard rate；
Cluster resource management module, for receiving the dilatation request that the cluster resource monitoring module sends, according to the dilatation
Request calculates dilatation resource requirement, and sends resource balancing requests；
Load balancing notification module, for receiving the resources balance request that the cluster resource management module sends, to disappearing
The public address monitored by breath transmitting terminal and message sink end initiates resource change operation, and the dilatation resource requirement is recorded
On the public address.
2. system according to claim 1, it is characterised in that the cluster resource monitoring module includes that cluster resource is monitored
Client and cluster resource monitoring service end, wherein,
The cluster resource monitor client is deployed in the agent side, and the hardware resource for agent side described in monitor in real time makes
With information, and the hardware resource use information for being monitored is reported into the cluster resource monitoring service end；
The cluster resource monitoring service end is made for the resource that the agent side is counted according to the hardware resource use information
With exceeding standard rate, and dilatation request is sent when the resource of the agent side reaches threshold value using exceeding standard rate.
3. system according to claim 2, it is characterised in that the agency of the cluster resource monitor client monitoring
The hardware resource use information at end includes：CPU usage, physical memory utilization rate, disk space utilization rate, magnetic disc i/o are handled up
Amount, and/or, network interface card I/O handling capacities；The cluster resource monitoring service end is counted on the agent side with the time as dimension
The exceeding standard rate of every hardware resource use information, exceedes preset threshold value in the exceeding standard rate of any one hardware resource use information
When, send dilatation request.
4. the system according to 1,2 or 3 any one of claim, it is characterised in that the cluster resource management module is with
On the basis of platform agent side resource, the time interval of agent side cluster dilatation request is calculated, if time interval is less than preassigned
Value, judges cluster expansion frequency as height, it is determined that the dilatation resource requirement is the preset times of the given resource of preset dilatation.
5. system according to claim 4, it is characterised in that the cluster resource management module utilizes Intel Virtualization Technology,
The dilatation resource requirement is dissolved virtually.
6. the system according to 1,2 or 3 any one of claim, it is characterised in that the agent side is listening to the money
After the change operation of source, load balancing operation is performed, and updates local cluster information caching.
7. a kind of expansion method of distributed information system, the distributed information system include message sending end cluster, agency
Cluster and message sink end cluster, wherein, the message sending end cluster includes multiple message sending ends, and Agent cluster includes many
Individual agent side, message sink end cluster include multiple message sink ends, it is characterised in that methods described includes：
The hardware resource use information of agent side described in monitor in real time, and count the resource of the agent side and use exceeding standard rate；
When the resource of the agent side reaches threshold value using exceeding standard rate, dilatation resource requirement is calculated；
Resource change operation is initiated to the public address monitored by message sending end and message sink end, will be provided needed for the dilatation
Source record is on the public address.
8. method according to claim 7, it is characterised in that
The hardware resource use information of the agent side includes：CPU usage, physical memory utilization rate, disk space utilization rate,
Magnetic disc i/o handling capacity, and/or, network interface card I/O handling capacities；
Methods described also includes：Determine whether the resource of the agent side reaches threshold value using exceeding standard rate, including：
With the time as dimension, the exceeding standard rate of every hardware resource use information on the agent side is counted, in any one use
When the exceeding standard rate of information exceedes preset threshold value, that is, determine that the resource of the agent side reaches threshold value using exceeding standard rate.
9. the method according to claim 7 or 8 any one, it is characterised in that the calculating dilatation resource requirement includes：With
On the basis of one agent side resource, the time interval of agent side cluster dilatation request is calculated, if time interval is less than pre- calibration
Quasi- value, judges cluster expansion frequency as height, it is determined that the dilatation resource requirement is the preset times of the given resource of preset dilatation.
10. method according to claim 9, it is characterised in that also include：By Intel Virtualization Technology, virtually dissolve described
Dilatation resource requirement.
11. methods according to claim 7 or 8 any one, it is characterised in that also include：The agent side is being listened to
After the resource change operation, load balancing operation is performed, and updates local cluster information caching.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611024161.XA CN106603631B (en) | 2016-11-17 | 2016-11-17 | Distributed message system and capacity expansion method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611024161.XA CN106603631B (en) | 2016-11-17 | 2016-11-17 | Distributed message system and capacity expansion method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106603631A true CN106603631A (en) | 2017-04-26 |
| CN106603631B CN106603631B (en) | 2020-07-03 |
Family
ID=58592352
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611024161.XA Active CN106603631B (en) | 2016-11-17 | 2016-11-17 | Distributed message system and capacity expansion method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106603631B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109040263A (en) * | 2018-08-10 | 2018-12-18 | 北京奇虎科技有限公司 | Method for processing business and device based on distributed system |
| CN110968414A (en) * | 2018-09-28 | 2020-04-07 | 阿里巴巴集团控股有限公司 | Resource scaling method and device |
| CN113037794A (en) * | 2019-12-25 | 2021-06-25 | 马上消费金融股份有限公司 | Computing resource allocation scheduling method, device and system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103428008A (en) * | 2013-08-28 | 2013-12-04 | 浙江大学 | Big data distribution strategy oriented to multiple user groups |
| CN103970907A (en) * | 2014-05-28 | 2014-08-06 | 浪潮电子信息产业股份有限公司 | Method for dynamically expanding database cluster |
| CN104579761A (en) * | 2014-12-24 | 2015-04-29 | 西安工程大学 | Automatic nosql cluster configuration system and method based on cloud computing |
| CN105159775A (en) * | 2015-08-05 | 2015-12-16 | 浪潮(北京)电子信息产业有限公司 | Load balancer based management system and management method for cloud computing data center |
-
2016
- 2016-11-17 CN CN201611024161.XA patent/CN106603631B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103428008A (en) * | 2013-08-28 | 2013-12-04 | 浙江大学 | Big data distribution strategy oriented to multiple user groups |
| CN103970907A (en) * | 2014-05-28 | 2014-08-06 | 浪潮电子信息产业股份有限公司 | Method for dynamically expanding database cluster |
| CN104579761A (en) * | 2014-12-24 | 2015-04-29 | 西安工程大学 | Automatic nosql cluster configuration system and method based on cloud computing |
| CN105159775A (en) * | 2015-08-05 | 2015-12-16 | 浪潮(北京)电子信息产业有限公司 | Load balancer based management system and management method for cloud computing data center |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109040263A (en) * | 2018-08-10 | 2018-12-18 | 北京奇虎科技有限公司 | Method for processing business and device based on distributed system |
| CN109040263B (en) * | 2018-08-10 | 2022-02-25 | 北京奇虎科技有限公司 | Service processing method and device based on distributed system |
| CN110968414A (en) * | 2018-09-28 | 2020-04-07 | 阿里巴巴集团控股有限公司 | Resource scaling method and device |
| CN113037794A (en) * | 2019-12-25 | 2021-06-25 | 马上消费金融股份有限公司 | Computing resource allocation scheduling method, device and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106603631B (en) | 2020-07-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105281981B (en) | The data traffic monitoring method and device of network service | |
| CN110233860B (en) | Load balancing method, device and system | |
| WO2016155360A1 (en) | Method, related apparatus and system for processing service request | |
| CN102694868A (en) | Cluster system implementation and task dynamic distribution method | |
| CN109800204B (en) | Data distribution method and related product | |
| CN105516347A (en) | Method and device for load balance allocation of streaming media server | |
| CN111045808B (en) | Distributed network task scheduling method and device | |
| CN109981702B (en) | File storage method and system | |
| CN108616429A (en) | A kind of reconnection method and equipment of Push Service | |
| CN111538563A (en) | Event analysis method and device for Kubernetes | |
| CN106603631A (en) | Distributed message system and expansion method thereof | |
| CN109597800B (en) | Log distribution method and device | |
| CN110310007A (en) | Risk Identification Method, device, equipment and storage medium | |
| CN107819632A (en) | A kind of dynamic load leveling group system based on performance monitoring system and Docker Swarm | |
| CN110071952A (en) | The control method and device of service call amount | |
| WO2023093194A1 (en) | Cloud monitoring method and cloud management platform | |
| CN106790610A (en) | A kind of cloud system message distributing method, device and system | |
| CN105095315B (en) | The methods, devices and systems of dynamic regulation Hash link points | |
| CN110515693A (en) | A kind of method and system that the virtual machine based on rack perception is extending transversely | |
| CN115599547A (en) | Automatic management and distribution method and system for database resource pool | |
| CN113396573A (en) | Migration of computing services | |
| CN110119300A (en) | The load-balancing method and device of dummy unit cluster | |
| CN114979250A (en) | Message pushing method, device and equipment | |
| CN114185681A (en) | Automatic current-limiting processing method and device | |
| CN108234658A (en) | A kind of method, apparatus and server of aware services device cluster health status |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |