CN101188575B - A traffic control and message buffering method oriented to distribution subscription system - Google Patents
A traffic control and message buffering method oriented to distribution subscription system Download PDFInfo
- Publication number
- CN101188575B CN101188575B CN2007101570981A CN200710157098A CN101188575B CN 101188575 B CN101188575 B CN 101188575B CN 2007101570981 A CN2007101570981 A CN 2007101570981A CN 200710157098 A CN200710157098 A CN 200710157098A CN 101188575 B CN101188575 B CN 101188575B
- Authority
- CN
- China
- Prior art keywords
- message
- server
- resource
- flow control
- internal memory
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Abstract
The invention relates to flow quantity control facing to an issuing subscribing system and the message cache method thereof. The invention comprises the following steps: the server parameter of the issuing subscribing system is established, message is classified through the condition of the message occupying resource; different strategies are adopted by the server through the using condition of the system resource of a thread route according to the using condition of the system resource; the flow quantity control is performed to a message sending client-side, the actual realization adopts the structure that a flow quantity controlling datagram is sent to the sending message client-side by the server, the client-side reduces the speed of sending the message to the server according the control information of the datagram; the flow quantity control is performed to the sending message client-side and the message is cached, the message in the internal memory is written into caching, and the internal memory is released. The invention has the advantages that message caching function are increased, the message is cached into additional memory though a configured parameter under a presetting condition, the internal memory occupied by the message is released, therefore the abandonment of the message is reduced, and the resources of the system is reduced.
Description
Technical field
The present invention relates to flow control and message buffering method field, mainly is a kind of flow control and message buffering method to publish/subscribe system.
Background technology
Internet broad application and mobile computing, the fast development of grid computing and general fit calculation platform, require distributed system can satisfy on a large scale, decentralized control and the requirement that dynamically changes. this just requires between each participant of system, adopt a kind of flexible communications pattern and interaction mechanism with dynamic and loose couplings characteristic. publish/subscribe (Pub/Sub) communicate by letter pattern and the traditional pattern of communicating by letter (message transmission, the RPC/RMI and the communal space) compare, have asynchronous, the characteristics of multi-point, the participant who makes communication is in the space, full decoupled on time and the control flows, can satisfy the demand of the loose communication of large-scale distributed system well.
Publish/subscribe system is a kind of each participant who makes in the distributed system, can carry out mutual middleware system in the mode of publish/subscribe. in publish/subscribe system, between the producers and consumers of information mutual information be called as incident. as shown in Figure 1, the producer sends to the publish/subscribe system server with incident; The consumer then sends a subscription condition to the publish/subscribe system server, represents interestedly in which incident in the system, if no longer interested, also can cancel subscriptions; The publish/subscribe system server then guarantees to send the incident of producer's issue to all in time, reliably to it interested consumer. and the producer of information is called publisher (publisher), the consumer of information is called subscriber (subscriber), and publisher and subscriber are referred to as client. and matching algorithm (matcher) is responsible for finding efficiently all subscription conditions that are complementary with given incident; Routing algorithm (router) then is responsible for selecting suitable path, and an incident is sent to the subscriber from the publisher.
The flow control of publish/subscribe system is a kind of load controlling mechanism, and purpose is to provide under the prerequisite of favorable service, keeps the robustness of system.Flow control mechanism is to the service quality of publish/subscribe system, and systematic function is vital.Current flow control mechanism has all only carried out flow control, when can't normally provide service, simply carries out message and abandons.
Summary of the invention
The present invention will solve the existing defective of above-mentioned technology, and a kind of flow control and message buffering method towards distribution subscription system is provided.
The technical solution adopted for the present invention to solve the technical problems: this flow control and message buffering method towards distribution subscription system may further comprise the steps:
1) publish/subscribe system server (hereinafter to be referred as server) parameter is provided with, the situation that takies resource by message is classified to message.
2) server is taked different strategies by the operating position of a thread acquisition system resource according to the operating position of system resource.If system resource is in the normal scope of setting, do not operate, if system resource is reduced to below the normal range value of setting, enter step 3,, enter step 4 if memory source is reduced to below the normal range value of setting, when resource is reduced to setting, enter step 5.
3) carry out flow control to sending information client side.Specific implementation is that server is reported to sending information client side transmitted traffic control data, and client reduces the speed that sends message toward server according to the control information of datagram, thereby reduces the pressure of server.
4) carry out flow control and buffered message to sending information client side, specific implementation is when adopting step 2, and the message in the internal memory is write buffer memory, discharges internal memory, thereby reduces the use amount of server end internal memory.
5) directly abandon message.Specific implementation is to abandon the message policies method by configuration server, can abandon up-to-date message or message the earliest, perhaps the message of particular type.
Wherein step 1 comprises following substep:
1.1, system parameters is provided with, set the frequency acquisition of thread, main resource and triggering that initialization system is concerned about enter step 3, the threshold values of the resource of 4,5 steps.
1.2, the situation that takies resource by message classifies to message, and sets up corresponding index and counting, sets up the client's who sends message index and counting.
For step 3, carry out different control according to the different situations of step 2, can send clients or the individual customer end carry out flow control to all that cause resource, and can be according to the degree set of resource shortage different flow control sub levels is other.Server can be taked different flow control ranks in the different stages, but only takes a kind of flow control of degree a time period.
Step 4 is the controlling mechanisms that increase at the internal memory operating position in the resource, for avoiding the collapse of server, when low memory, carry out flow control to sending information client side on the one hand, on the one hand the message in the server memory is write buffer memory, if only the reservation flag position and the relevant information of from buffer memory, fetching message in internal memory, the memory source that release message is shared.When the relevant internal memory use amount of the buffer memory of reservation message during, this message is not handled more than or equal to the internal memory use amount of message.In the time of low memory, as if CPU also wretched insufficiency, consider equally also can take CPU in various degree to message buffering, reduce the speed of buffer memory this moment.
Step 5 is when step 3 and step 4 all can not reduce the operating position of resource of server, the means of taking for the collapse of avoiding server.This step is abandoned certain amount of message, thereby has reduced the use amount of system resource.
In step 3, in 4,5, server all moves a time period of setting, has avoided switching between different steps.
The effect that the present invention is useful is: the present invention has increased the function of message buffering in flow control mechanism, message buffering can be deposited to auxilliary under the environment of setting by configuration parameter, the internal memory that release message takies, thereby reduced abandoning of message, and reduced the resource of system, improved the service ability of system.
Description of drawings
Fig. 1 is the concept structure schematic diagram of publish/subscribe system;
Fig. 2 is the process chart to the publish/subscribe system flow control;
Embodiment
The invention will be described further below in conjunction with drawings and Examples:
In actual applications, the publish/subscribe system applied scene have a very big difference, required message to be processed also will be varied, and server hardware configuration also can be different, thereby also can be different to the control strategy of the flow control of message and buffer memory.Therefore, in an embodiment of the present invention, at first need the parameter of publish/subscribe system is provided with.Set the frequency acquisition of thread, the operating position of expression system resource of acquisition system after a time interval; The main resource that initialization system is concerned about, as CPU, internal memory, the network bandwidth etc.In an embodiment, if the CPU and the network bandwidth are the bottlenecks of service, system will be the main resource that the operating position of the CPU and the network bandwidth is concerned about as system; Set triggering and enter step 3, the threshold values of the resource of 4,5 steps is less than at 50% o'clock as setting CPU, enters step 3, and internal memory is less than at 30% o'clock and enters step 4.In different embodiments, when the resource difference of server, and application scenarios is when similar, and the less server of resource may need to set higher threshold values.
In different embodiments, the classification of message also can be different, and the sorting technique of message is a lot, standard that can be different with selection of configuration according to the conditions of demand of reality.In an embodiment, can classify according to the method that message mainly takies resource, such as main committed memory resource be divided into internal memory message, be divided into database message to what database was operated, and the message of each class is carried out corresponding index and counting.The speed of the transmission client transmission message that the while is different also can be different, therefore sets up the client's who sends message index and also count, and keeps the monitoring to client.
When comparing, each resource need be compared with the threshold values of setting respectively,, then not operate if surpass threshold values to the operating position of thread collection resource and with corresponding threshold values.
When the consumption of some resources surpasses threshold values, need carry out flow control, need judge and need control by algorithm this moment which client.Different embodiments also can be different to the sample that message is analyzed.For example adopt the right all message of the type that belongs to analyze in an embodiment, transmission information client side to the message that wherein occupation proportion is bigger is controlled, and then the transmission information client side of the bigger message of occupation proportion in the past period is controlled in another embodiment.Different embodiments also can be different to the setting of the transmission information client side quantity of carrying out flow control.
In an embodiment of the present invention, a kind of realization of flow control is, when server judges that need send information client side and carry out flow control some the time, it sends information client sides to these and sends a flow control datagram, the rank that comprises flow control, the time of flow control.Client is provided with the flow control flag bit according to the control information of datagram, and before the time of control finishes, makes client send specific time of dormancy between the message, thereby reduce the speed that sends message toward server.Carry out flow control and do not carry out in the scope of message buffering when server end is in, server can be set the different flow control ranks and the time of flow control again according to the percentage of resources occupation rate, thereby control more accurately, improve the utilance of server.
When needs with message buffering during to auxilliary depositing, can be in the file of database or specific format with message buffering.When the resource of system is eased, when perhaps memory source is sufficient, can once more message be read in internal memory, and change message and whether be saved in the auxilliary flag bit of depositing, and the auxilliary message of depositing of deletion, thereby improve the speed of messenger service.Another situation is when having the recipient of message to need message, equally message read in to change message and whether be saved in the auxilliary flag bit of depositing, and the auxilliary message of depositing of deletion.In a preferable embodiment of the present invention, when the needs buffered message, with the content arrayization of message, write specific file, this message is set by the flag bit of serializing in internal memory, preserves the title of the residing file of this message, and position in this document and length.When needs were fetched message, title and position and the length of message in this document according to file read binary message, carry out unserializing, are converted to message.After all message in the file are retrieved, can reuse this document.
Abandon message only when step 3 and step 4 all can not reduce the operating position of resource of server, the means of taking for the collapse of avoiding server.Different application scenarioss can be abandoned message according to the strategy that message is abandoned.In an embodiment, server has been abandoned the message that arrives the earliest, and in other embodiments, server may be abandoned the message that receives recently, perhaps server is abandoned the message that priority is low or importance is low according to the priority level or the severity level of message, and the influence that message is abandoned system drops to minimum.
The foregoing description is used for the present invention that explains, rather than limits the invention, and in the protection range of spirit of the present invention and claim, any modification and change to the present invention makes all fall into protection scope of the present invention.
Claims (4)
1. flow control and message buffering method towards a distribution subscription system is characterized in that: may further comprise the steps:
1) the distribution subscription system server parameter is provided with, the situation that takies resource by message is classified to message;
2) server is by the operating position of a thread acquisition system resource, take different strategies according to the operating position of system resource, if system resource is in the normal scope of setting, do not operate,, enter step 3 if cpu resource is reduced to below the normal range value of setting, if memory source is reduced to below the normal range value of setting, enter step 4, when step 3 and step 4 all can not reduce the operating position of resource of server, enter step 5 for avoiding server failing;
3) carry out flow control to sending information client side, specific implementation is that server is reported to sending information client side transmitted traffic control data, and client reduces the speed that sends message toward server according to the control information of datagram;
4) carry out flow control and buffered message to sending information client side, specific implementation is when adopting step 3, and the message in the internal memory is write auxilliary depositing, and discharges internal memory;
5) directly abandon message, specific implementation is to abandon the message policies method by configuration server, can abandon up-to-date message or message the earliest, perhaps the message of particular type.
2. flow control and message buffering method towards distribution subscription system according to claim 1, it is characterized in that: wherein step 1 comprises following substep:
1.1, server parameter is provided with, set the frequency acquisition of thread, main resource and triggering that initialization system is concerned about enter step 3, the threshold values of 4,5 resource;
1.2, the situation that takies resource by message classifies to message, and sets up corresponding index and counting, sets up the client's who sends message index and counting.
3. flow control and message buffering method towards distribution subscription system according to claim 1, it is characterized in that: step 4 is the controlling mechanisms that increase at the internal memory operating position in the resource, when low memory, carry out flow control to sending information client side on the one hand, on the one hand the message in the server memory is write auxilliary depositing, if only the reservation flag position and the relevant information of from auxilliary depositing, fetching message in internal memory, the memory source that release message is shared; When the relevant internal memory use amount of the buffer memory of reservation message during, this message is not handled, in the time of low memory, if also wretched insufficiency of CPU reduces the speed of buffer memory this moment more than or equal to the internal memory use amount of message.
4. flow control and message buffering method towards distribution subscription system according to claim 1 is characterized in that: in step 3, in 4,5, server all moves a time period of setting.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2007101570981A CN101188575B (en) | 2007-12-04 | 2007-12-04 | A traffic control and message buffering method oriented to distribution subscription system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2007101570981A CN101188575B (en) | 2007-12-04 | 2007-12-04 | A traffic control and message buffering method oriented to distribution subscription system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101188575A CN101188575A (en) | 2008-05-28 |
| CN101188575B true CN101188575B (en) | 2010-06-09 |
Family
ID=39480770
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2007101570981A Expired - Fee Related CN101188575B (en) | 2007-12-04 | 2007-12-04 | A traffic control and message buffering method oriented to distribution subscription system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101188575B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102195985B (en) * | 2011-05-19 | 2013-08-21 | 北京邮电大学 | System and method for controlling network load of presence service |
| CN103873465B (en) * | 2014-03-04 | 2016-12-07 | 上海交通大学 | A kind of caching method of the distribution subscription system being applicable to mobile context |
| CN105991463B (en) * | 2015-02-13 | 2020-12-25 | 创新先进技术有限公司 | Method, message main node, token server and system for realizing flow control |
| CN107623642A (en) * | 2017-08-30 | 2018-01-23 | 山东中创软件商用中间件股份有限公司 | A kind of message traffic control method and device |
| CN108376153A (en) * | 2018-02-07 | 2018-08-07 | 厦门集微科技有限公司 | A kind of Webpage production method and device |
| CN108881061B (en) * | 2018-06-28 | 2022-03-11 | 迈普通信技术股份有限公司 | Communication apparatus and communication method |
| CN111258840B (en) * | 2018-11-30 | 2023-10-10 | 杭州海康威视数字技术股份有限公司 | Cluster node management method and device and cluster |
| CN110417646A (en) * | 2019-08-01 | 2019-11-05 | 北京吉旗咨询服务有限公司 | The method and device for subscribing to message are pushed based on RPC application program |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6212582B1 (en) * | 1996-04-19 | 2001-04-03 | Lucent Technologies Inc. | Method for multi-priority, multicast flow control in a packet switch |
| CN1352510A (en) * | 2000-11-06 | 2002-06-05 | 深圳市中兴通讯股份有限公司 | Method for overload control in CDMA cellular communication system |
| CN1478345A (en) * | 2000-12-22 | 2004-02-25 | ��˹��ŵ�� | Flow control in packet-switched communication network using leaky bucket algorithm |
| CN1708030A (en) * | 2004-06-11 | 2005-12-14 | 华为技术有限公司 | System for controlling packet service network flow and method thereof |
-
2007
- 2007-12-04 CN CN2007101570981A patent/CN101188575B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6212582B1 (en) * | 1996-04-19 | 2001-04-03 | Lucent Technologies Inc. | Method for multi-priority, multicast flow control in a packet switch |
| CN1352510A (en) * | 2000-11-06 | 2002-06-05 | 深圳市中兴通讯股份有限公司 | Method for overload control in CDMA cellular communication system |
| CN1478345A (en) * | 2000-12-22 | 2004-02-25 | ��˹��ŵ�� | Flow control in packet-switched communication network using leaky bucket algorithm |
| CN1708030A (en) * | 2004-06-11 | 2005-12-14 | 华为技术有限公司 | System for controlling packet service network flow and method thereof |
Non-Patent Citations (1)
| Title |
|---|
| 潘苹萍.分布式高性能Java消息服务系统JTangMQ的设计与实现.硕士学位论文.2006,32. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101188575A (en) | 2008-05-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101188575B (en) | A traffic control and message buffering method oriented to distribution subscription system | |
| CN108768826B (en) | Message routing method based on MQTT and Kafka high concurrency scene | |
| US8769025B2 (en) | Cluster server of an instant messaging system and messaging method between clusters | |
| US7508837B2 (en) | System and method for receive queue provisioning | |
| CN111124819B (en) | Method and device for full link monitoring | |
| Alkhawaja et al. | Message oriented middleware with QoS support for smart grids | |
| Bhimani et al. | Message delivery guarantee and status update of clients based on IOT-AMQP | |
| CN108540367B (en) | Message processing method and system | |
| CN102859961A (en) | Distributed video transcode system with adaptive file processing | |
| CN103873523A (en) | Client cluster access method and device | |
| CN114710571B (en) | Data packet processing system | |
| CN112882842A (en) | Data transmission method based on redis storage as message middleware | |
| CN111526188A (en) | System and method for ensuring zero data loss based on Spark Streaming in combination with Kafka | |
| US20210392202A1 (en) | Artificial intelligence log processing and content distribution network optimization | |
| US20090172122A1 (en) | Message Transmission Method, Message Transmission Device, and Storage Medium Recorded with Message Transmission Program | |
| CN113904994B (en) | Method for uniformly reporting big data of home gateway to platform | |
| CN111475315A (en) | Server and subscription notification push control and execution method | |
| CN111241101A (en) | Distributed water conservancy RTU data acquisition system and method | |
| CN112256454B (en) | Message delay processing method and system | |
| US20210092199A1 (en) | Cachability of single page applications | |
| CN109413101B (en) | Information transmission method between client and server | |
| CN113467969A (en) | Method for processing message accumulation | |
| CN113132258A (en) | Time-sensitive network data transmission system and time delay analysis method | |
| CN113259408A (en) | Data transmission method and system | |
| JP7477251B2 (en) | Method, system, and program for improving cacheability of single-page applications |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20100609 Termination date: 20171204 |