CN107911484B - Message processing method and device - Google Patents
Message processing method and device Download PDFInfo
- Publication number
- CN107911484B CN107911484B CN201711326101.8A CN201711326101A CN107911484B CN 107911484 B CN107911484 B CN 107911484B CN 201711326101 A CN201711326101 A CN 201711326101A CN 107911484 B CN107911484 B CN 107911484B
- Authority
- CN
- China
- Prior art keywords
- client connection
- current
- client
- message
- buffer pool
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title description 5
- 238000000034 method Methods 0.000 claims abstract description 32
- 238000004806 packaging method and process Methods 0.000 claims abstract description 20
- 238000012545 processing Methods 0.000 claims abstract description 20
- 238000007726 management method Methods 0.000 claims description 3
- 238000013468 resource allocation Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000012858 packaging process Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 230000003247 decreasing effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 239000002699 waste material Substances 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000012856 packing Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/55—Push-based network services
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/12—Avoiding congestion; Recovering from congestion
- H04L47/125—Avoiding congestion; Recovering from congestion by balancing the load, e.g. traffic engineering
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/50—Queue scheduling
- H04L47/62—Queue scheduling characterised by scheduling criteria
- H04L47/622—Queue service order
- H04L47/623—Weighted service order
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/70—Admission control; Resource allocation
Abstract
The invention provides a method and a device for processing messages, wherein the method comprises the following steps: determining at least one topic to which at least one customer subscribes; for each topic, determining a client connection corresponding to each client subscribing to the current topic and aiming at the current topic; setting a client connection buffer pool for each client connection; periodically and respectively acquiring the message of each theme; for each topic, performing: determining at least one client connection corresponding to the current theme; storing the acquired message of the current theme into at least one client connection buffer pool corresponding to at least one client connection; obtaining a message from at least one client connection buffer pool; respectively packaging the messages in the same client connection buffer pool to generate a data packet to be pushed corresponding to each client connection buffer pool; and pushing the data packets to be pushed corresponding to each client connection buffer pool to the corresponding client connection. The invention can reduce the consumption of resources.
Description
Technical Field
The present invention relates to the field of network technologies, and in particular, to a method and an apparatus for processing a message.
Background
With the rapid development of network technology, more and more ways for users to acquire messages are provided. It is a common way for users to obtain required messages through subscription.
In the prior art, a user subscribes to a required message, and after the message subscribed by the user is found, the message is pushed to the user one by one. Specifically, each message is packed and sent to the user one by one after packing.
As can be seen from the above description, in the prior art, each message needs to be packaged and sent, which consumes a lot of resources.
Disclosure of Invention
The embodiment of the invention provides a message processing method and device, which can reduce the consumption of resources.
In one aspect, an embodiment of the present invention provides a method for processing a message, including:
determining at least one topic to which at least one customer subscribes;
for each topic, determining a client connection corresponding to each client subscribing to the current topic and aiming at the current topic;
setting a client connection buffer pool for each client connection;
periodically and respectively acquiring the message of each theme;
for each of the topics, performing: determining at least one client connection corresponding to the current theme; storing the acquired message of the current theme into at least one client connection buffer pool corresponding to the at least one client connection;
obtaining a message from the at least one client connection buffer pool;
respectively packaging the messages in the same client connection buffer pool to generate a data packet to be pushed corresponding to each client connection buffer pool;
and pushing the data packets to be pushed corresponding to each client connection buffer pool to the corresponding client connection.
Further, the air conditioner is provided with a fan,
the method further comprises the following steps:
setting a push weight for each of the customer connections;
the storing the obtained message of the current theme to at least one client connection buffer pool corresponding to the at least one client connection comprises:
for each of the at least one customer connection, performing:
determining a push weight corresponding to the current client connection;
determining the message of the current theme needing to be pushed to the current client connection according to the pushing weight corresponding to the current client connection;
and storing the message of the current theme needing to be pushed to the current client connection into a corresponding client connection buffer pool.
Further, the air conditioner is provided with a fan,
the determining the push weight corresponding to the current client connection includes:
receiving an initial pushing weight input by a client corresponding to the current client connection;
determining the receiving rate of the client corresponding to the current client connection;
and adjusting the initial pushing weight according to the receiving rate to generate a pushing weight corresponding to the current customer connection.
Further, the air conditioner is provided with a fan,
the method further comprises the following steps:
presetting at least one task queue, and distributing a push thread for each task queue;
after the storing the obtained message of the current topic into at least one client connection buffer pool corresponding to the at least one client connection, further comprising:
generating a pending task for each of the at least one customer connection, wherein the pending task includes information of the pending customer connection;
allocating at least one of the tasks to be processed to the at least one task queue;
said retrieving messages from said at least one client connection buffer pool, comprising:
for each of the push threads, performing: acquiring the task to be processed from a task queue corresponding to the current push thread by using the current push thread, determining the client connection to be processed according to the information of the client connection to be processed in the task to be processed, and acquiring a message from a client connection buffer pool corresponding to the client connection to be processed;
the said separately group package the message in the same said customer connects the buffer pool, produce the data packet to be pushed that every said customer connects the buffer pool to correspond, including:
for each of the push threads, performing: packaging the messages in the client connection buffer pool corresponding to the client connection to be processed by using the current push thread to generate a current data packet to be pushed;
the step of pushing the data packets to be pushed corresponding to each client connection buffer pool to the corresponding client connection respectively comprises:
for each of the push threads, performing: and pushing the data packet to be pushed to the client connection to be processed by utilizing the current pushing thread.
Further, the air conditioner is provided with a fan,
the method further comprises the following steps: presetting at least one acquisition thread, wherein the at least one acquisition thread is used for executing the information for respectively acquiring each theme;
further comprising:
and judging whether the difference between the number of the messages acquired by the at least one acquisition thread at the current time and the number of the messages pushed by the at least one pushing thread at the current time is greater than or equal to a first preset value or not in real time, and if so, sleeping the second preset value of the at least one acquisition thread which is not in sleep.
In another aspect, an embodiment of the present invention provides an apparatus for processing a message, including:
the service management module is used for determining at least one topic subscribed by at least one client; for each topic, determining a client connection corresponding to each client subscribing to the current topic and aiming at the current topic; setting a client connection buffer pool for each client connection;
the message acquisition module is used for periodically and respectively acquiring the message of each theme; for each of the topics, performing: determining at least one client connection corresponding to the current theme; storing the acquired message of the current theme into at least one client connection buffer pool corresponding to the at least one client connection;
a message push module for obtaining messages from the at least one client connection buffer pool; respectively packaging the messages in the same client connection buffer pool to generate a data packet to be pushed corresponding to each client connection buffer pool; and pushing the data packets to be pushed corresponding to each client connection buffer pool to the corresponding client connection.
Further, the air conditioner is provided with a fan,
the apparatus further comprises: a frequency setting module for setting a push weight for each of the customer connections;
the message obtaining module, when executing the storing of the obtained message of the current topic into the at least one client connection buffer pool corresponding to the at least one client connection, is specifically configured to:
for each of the at least one customer connection, performing:
determining a push weight corresponding to the current client connection;
determining the message of the current theme needing to be pushed to the current client connection according to the pushing weight corresponding to the current client connection;
and storing the message of the current theme needing to be pushed to the current client connection into a corresponding client connection buffer pool.
Further, the air conditioner is provided with a fan,
the message obtaining module, when executing the push weight corresponding to the determined current client connection, is specifically configured to:
receiving an initial pushing weight input by a client corresponding to the current client connection;
determining the receiving rate of the client corresponding to the current client connection;
and adjusting the initial pushing weight according to the receiving rate to generate a pushing weight corresponding to the current customer connection.
Further, the air conditioner is provided with a fan,
the apparatus further comprises:
the queue setting module is used for setting at least one task queue and distributing a pushing thread for each task queue;
the message obtaining module is configured to, after storing the obtained message of the current topic in at least one client connection buffer pool corresponding to the at least one client connection, perform:
generating a pending task for each of the at least one customer connection, wherein the pending task includes information of the pending customer connection; allocating at least one of the tasks to be processed to the at least one task queue;
the message pushing module is used for executing:
for each of the push threads, performing:
acquiring the task to be processed from a task queue corresponding to the current push thread by using the current push thread, determining the client connection to be processed according to the information of the client connection to be processed in the task to be processed, and acquiring a message from a client connection buffer pool corresponding to the client connection to be processed;
packaging the messages in the client connection buffer pool corresponding to the client connection to be processed by using the current push thread to generate a current data packet to be pushed;
and pushing the data packet to be pushed to the client connection to be processed by utilizing the current pushing thread.
Further, the air conditioner is provided with a fan,
the apparatus further comprises:
a thread setting module, configured to set at least one obtaining thread, where the at least one obtaining thread is used to execute the obtaining of each message of the theme;
further comprising:
and the resource allocation module is used for judging whether the difference between the number of the messages acquired by the at least one acquisition thread at the current time and the number of the messages pushed by the at least one pushing thread at the current time is greater than or equal to a first preset value or not in real time, and if so, sleeping the acquisition threads with a second preset value in the at least one acquisition thread which is not in sleep.
In the embodiment of the invention, a client connection buffer pool is set for each client connection, a message corresponding to each client connection is stored in the corresponding client connection buffer pool, the messages in the client connection buffer pool are uniformly packaged to generate a data packet to be pushed, and the whole data packet to be pushed is pushed to the corresponding client connection, that is, the messages in the client connection buffer pool can be integrally packaged without packaging one message, so that the times of packaging are reduced, the resources consumed in the packaging process are saved, the messages in the client connection buffer pool can be integrally pushed without pushing one message, the times of pushing are reduced, and the resources consumed in the pushing process are saved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a flowchart of a method for processing a message according to an embodiment of the present invention;
FIG. 2 is a flow diagram of another method for message processing according to an embodiment of the invention;
fig. 3 is a schematic diagram of a message processing apparatus according to an embodiment of the present invention;
fig. 4 is a schematic diagram of another message processing apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer and more complete, the technical solutions in the embodiments of the present invention will be described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention, and based on the embodiments of the present invention, all other embodiments obtained by a person of ordinary skill in the art without creative efforts belong to the scope of the present invention.
As shown in fig. 1, an embodiment of the present invention provides a method for processing a message, where the method may include the following steps:
step 101: determining at least one topic to which at least one customer subscribes;
step 102: for each topic, determining a client connection corresponding to each client subscribing to the current topic and aiming at the current topic;
step 103: setting a client connection buffer pool for each client connection;
step 104: periodically and respectively acquiring the message of each theme;
step 105: for each of the topics, performing: determining at least one client connection corresponding to the current theme; storing the acquired message of the current theme into at least one client connection buffer pool corresponding to the at least one client connection;
step 106: obtaining a message from the at least one client connection buffer pool;
step 107: respectively packaging the messages in the same client connection buffer pool to generate a data packet to be pushed corresponding to each client connection buffer pool;
step 108: and pushing the data packets to be pushed corresponding to each client connection buffer pool to the corresponding client connection.
In the embodiment of the invention, a client connection buffer pool is set for each client connection, a message corresponding to each client connection is stored in the corresponding client connection buffer pool, the messages in the client connection buffer pool are uniformly packaged to generate a data packet to be pushed, and the whole data packet to be pushed is pushed to the corresponding client connection, that is, the messages in the client connection buffer pool can be integrally packaged without packaging one message, so that the times of packaging are reduced, the resources consumed in the packaging process are saved, the messages in the client connection buffer pool can be integrally pushed without pushing one message, the times of pushing are reduced, and the resources consumed in the pushing process are saved.
In the embodiment of the invention, the same client can subscribe to a plurality of topics, and the same topic can be subscribed by a plurality of clients. For each topic, there may be multiple client connections, specifically, there is one client connection for each client subscribing to the topic. For each client, there may be multiple client connections, specifically, one client connection for the client exists for each topic to which the client subscribes.
When the message of each topic is obtained, the search can be carried out from the database or the network. For example, a keyword for searching may be determined by the name of the topic, and the search may be performed according to the keyword.
In an embodiment of the invention, the method further comprises:
setting a push weight for each of the customer connections;
the storing the obtained message of the current theme to at least one client connection buffer pool corresponding to the at least one client connection comprises:
for each of the at least one customer connection, performing:
determining a push weight corresponding to the current client connection;
determining the message of the current theme needing to be pushed to the current client connection according to the pushing weight corresponding to the current client connection;
and storing the message of the current theme needing to be pushed to the current client connection into a corresponding client connection buffer pool.
Because the push weights of the messages corresponding to different users are different, for example, the client a subscribes to the topic a and the topic B, and the client a prefers the topic a and wants to frequently receive the messages related to the topic a, at this time, the push weight of the client connection corresponding to the topic a may be set to be higher than the push weight of the client connection corresponding to the topic B for the client a.
In this embodiment of the present invention, when determining, according to the pushing weight corresponding to the current client connection, the message of the current topic that needs to be pushed to the current client connection, the following specific implementation may be performed: a certain number of messages can be selected from all the messages for obtaining the current theme according to the pushing weight as the messages needing to be pushed. For example: when the pushing weight is 1, each message needs to be pushed; when the push weight is 2, only one of every two messages needs to be pushed; when the push weight is n, only one of the messages needs to be pushed every n messages. Wherein, the push weight is a positive integer.
In an embodiment of the present invention, the determining the push weight corresponding to the current client connection includes:
receiving an initial pushing weight input by a client corresponding to the current client connection;
determining the receiving rate of the client corresponding to the current client connection;
and adjusting the initial pushing weight according to the receiving rate to generate a pushing weight corresponding to the current customer connection.
In the embodiment of the invention, the push weight is adjusted according to the receiving rate, so that the load balance is realized. Specifically, when the receiving rate of the client is high, the pushing weight is increased based on the initial pushing weight, and when the receiving rate of the client is low, the pushing weight is decreased based on the initial pushing weight, so that the reasonable distribution of the bandwidth is achieved.
In an embodiment of the invention, the method further comprises:
presetting at least one task queue, and distributing a push thread for each task queue;
after the storing the obtained message of the current topic into at least one client connection buffer pool corresponding to the at least one client connection, further comprising:
generating a pending task for each of the at least one customer connection, wherein the pending task includes information of the pending customer connection;
allocating at least one of the tasks to be processed to the at least one task queue;
said retrieving messages from said at least one client connection buffer pool, comprising:
for each of the push threads, performing: acquiring the task to be processed from a task queue corresponding to the current push thread by using the current push thread, determining the client connection to be processed according to the information of the client connection to be processed in the task to be processed, and acquiring a message from a client connection buffer pool corresponding to the client connection to be processed;
the said separately group package the message in the same said customer connects the buffer pool, produce the data packet to be pushed that every said customer connects the buffer pool to correspond, including:
for each of the push threads, performing: packaging the messages in the client connection buffer pool corresponding to the client connection to be processed by using the current push thread to generate a current data packet to be pushed;
the step of pushing the data packets to be pushed corresponding to each client connection buffer pool to the corresponding client connection respectively comprises:
for each of the push threads, performing: and pushing the data packet to be pushed to the client connection to be processed by utilizing the current pushing thread.
In the embodiment of the invention, the messages in the client connection buffer pool are distributed to each push thread in a task waiting mode, each push thread loads one task queue, and each push thread acquires the tasks to be processed one by one from the corresponding task queue and executes the acquired tasks to be processed. And information of the client connection to be processed is set in each task to be processed, and the push thread can find the corresponding client connection buffer pool through the information.
When the tasks to be processed are distributed, the tasks to be processed can be distributed according to the idle condition of each pushing thread, the tasks to be processed are distributed to the currently most idle pushing thread in a limited mode, and resource optimization is achieved.
In an embodiment of the invention, the method further comprises: presetting at least one acquisition thread, wherein the at least one acquisition thread is used for executing the information for respectively acquiring each theme;
further comprising:
and judging whether the difference between the number of the messages acquired by the at least one acquisition thread at the current time and the number of the messages pushed by the at least one pushing thread at the current time is greater than or equal to a first preset value or not in real time, and if so, sleeping the second preset value of the at least one acquisition thread which is not in sleep.
In the embodiment of the invention, when the number of the acquisition threads is more and the number of the messages pushed by the pushing threads is less, a large number of messages are accumulated in the client connection buffer pool, at this time, part of the acquisition threads are dormant, more resources are used for processing the pushing threads, the message pushing speed is accelerated, and the dynamic allocation of the resources is realized, so that the optimal utilization effect of the resources is achieved.
As shown in fig. 2, an embodiment of the present invention provides a method for processing a message, where the method may include the following steps:
step 201: at least one topic to which at least one customer subscribes is determined.
In particular, a polling list of topics may be provided to save the at least one topic.
In addition, a customer maintenance list may be provided to maintain information about the customer.
Specifically, each client saves at least one topic subscribed by the client into the topic polling list and saves the information of the client into the client maintenance list when registering.
For example, customer A subscribes to topic A and topic B. For example: topic a is sports and topic B is social news.
Step 202: and for each topic, determining the client connection corresponding to each client subscribing to the current topic and aiming at the current topic.
In particular, messages are pushed to the client of each client over the client connection. For example: and pushing the message to the client of the client A through the client connection corresponding to the client A and aiming at the theme A.
Specifically, a topic push list may be set to store the correspondence of topics to client connections.
Step 203: a client connection buffer pool is set for each client connection.
In particular, messages are buffered by the client connection buffer pool.
Step 204: at least one task queue is preset, and a push thread is distributed to each task queue.
Specifically, each task queue is responsible for one push thread.
Step 205: periodically, messages for each topic are acquired separately.
In particular, the polling list of topics may be polled to obtain messages for individual topics.
For example, the topic a is sports, and the required message can be searched periodically by using "sports" as the keyword, and of course, other keywords can also be extended based on sports, such as: basketball, football, sporting meetings, etc.
Step 206: for each topic, performing: determining at least one client connection corresponding to the current theme; storing the acquired message of the current theme into at least one client connection buffer pool corresponding to the at least one client connection, and generating a to-be-processed task aiming at each client connection in the at least one client connection, wherein the to-be-processed task comprises information of the to-be-processed client connection; at least one task to be processed is assigned to at least one task queue.
For example, for topic a, at least one client connection corresponding to topic a is determined (including client connection a of client a for topic a); storing the obtained message of the theme A into at least one client connection buffer pool corresponding to at least one client connection corresponding to the theme A (wherein, aiming at the client A, the message of the theme A is stored into the client connection A); generating a to-be-processed task for each client connection in at least one client connection corresponding to the topic A (wherein the to-be-processed task A corresponding to the client connection A is generated and includes information of the client connection A), and allocating the at least one to-be-processed task to at least one task queue (wherein the to-be-processed task A corresponding to the client connection A is allocated to any task queue A).
Step 207: for each push thread, performing: and acquiring a task to be processed from a task queue corresponding to the current push thread by using the current push thread, determining the connection of a client to be processed according to the information of the connection of the client to be processed in the task to be processed, and acquiring a message from a client connection buffer pool corresponding to the connection of the client to be processed.
For example, a pushing thread a is used for processing the task queue a, the pushing thread a obtains a to-be-processed task a from the task queue a, determines that a to-be-processed client connection is a client connection a according to information of the client connection a in the to-be-processed task a, and obtains a message of a topic a from a client connection buffer pool corresponding to the client connection a.
Step 208: for each push thread, performing: and packaging the messages in the client connection buffer pool corresponding to the client connection to be processed by using the current push thread to generate a current data packet to be pushed.
For example, a push thread a is used to package a message in a client connection buffer pool corresponding to a client connection a, so as to generate a data packet a to be currently pushed.
Step 209: for each push thread, performing: and pushing the data packet to be pushed to the client connection to be processed by utilizing the current pushing thread.
For example, a data packet a to be currently pushed is pushed to a client connection a by using a push thread a.
In the embodiment of the invention, the resources and the bandwidth are reasonably distributed among all tasks by adopting the modes of packet sending, load balancing and dynamic resource distribution, so that the distribution and the use of the network bandwidth and the resources are optimized, the network bandwidth and the system resources are fully utilized, and the message is efficiently pushed to the client in real time.
In the embodiment of the invention, different task queues can be adopted to complete different types of tasks, and each task queue only completes a single type of task, thereby reducing the complexity of multiple tasks and the waste of resources.
In the embodiment of the invention, the acquired message packets are packaged and sent, thereby reducing the system expense and bandwidth expense for frequently sending small data packets and improving the sending speed.
In the embodiment of the invention, the message is acquired and then sent to the client in real time without any stop, thereby having good real-time performance.
In the embodiment of the invention, the utilization rate of the network is improved, the additional expenditure of the system is reduced and the high-speed transmission of a large amount of data can be realized by the modes of message packet sending, load balancing, dynamic allocation of the system and network resources and the like.
In the embodiment of the invention, the system switching expenditure is reduced by dynamically distributing resources, acquiring grouped messages once, pushing weight distribution and the like, so that the resources are reasonably distributed among tasks, the resource utilization rate in a high concurrency environment is improved, the waste of system resources is reduced, and the service has good performance in the high concurrency environment.
As shown in fig. 3 and 4, an embodiment of the present invention provides a message processing apparatus. The device embodiments may be implemented by software, or by hardware, or by a combination of hardware and software. From a hardware level, as shown in fig. 3, a hardware structure diagram of a device in which a message processing apparatus according to an embodiment of the present invention is located is shown, where in addition to the processor, the memory, the network interface, and the nonvolatile memory shown in fig. 3, the device in which the message processing apparatus is located in the embodiment may also include other hardware, such as a forwarding chip responsible for processing a message, in general. Taking a software implementation as an example, as shown in fig. 4, as a logical apparatus, the apparatus is formed by reading a corresponding computer program instruction in a non-volatile memory into a memory by a CPU of a device in which the apparatus is located and running the computer program instruction. The message processing device provided by the embodiment comprises:
a service management module 401, configured to determine at least one topic subscribed by at least one client; for each topic, determining a client connection corresponding to each client subscribing to the current topic and aiming at the current topic; setting a client connection buffer pool for each client connection;
a message obtaining module 402, configured to periodically obtain messages of each topic respectively; for each of the topics, performing: determining at least one client connection corresponding to the current theme; storing the acquired message of the current theme into at least one client connection buffer pool corresponding to the at least one client connection;
a message pushing module 403, configured to obtain a message from the at least one client connection buffer pool; respectively packaging the messages in the same client connection buffer pool to generate a data packet to be pushed corresponding to each client connection buffer pool; and pushing the data packets to be pushed corresponding to each client connection buffer pool to the corresponding client connection.
In an embodiment of the present invention, the apparatus further comprises: a frequency setting module for setting a push weight for each of the customer connections;
the message obtaining module, when executing the storing of the obtained message of the current topic into the at least one client connection buffer pool corresponding to the at least one client connection, is specifically configured to:
for each of the at least one customer connection, performing:
determining a push weight corresponding to the current client connection;
determining the message of the current theme needing to be pushed to the current client connection according to the pushing weight corresponding to the current client connection;
and storing the message of the current theme needing to be pushed to the current client connection into a corresponding client connection buffer pool.
In an embodiment of the present invention, when the determining of the push weight corresponding to the current client connection is performed, the message obtaining module is specifically configured to:
receiving an initial pushing weight input by a client corresponding to the current client connection;
determining the receiving rate of the client corresponding to the current client connection;
and adjusting the initial pushing weight according to the receiving rate to generate a pushing weight corresponding to the current customer connection.
In an embodiment of the present invention, the apparatus further comprises:
the queue setting module is used for setting at least one task queue and distributing a pushing thread for each task queue;
the message obtaining module is configured to, after storing the obtained message of the current topic in at least one client connection buffer pool corresponding to the at least one client connection, perform:
generating a pending task for each of the at least one customer connection, wherein the pending task includes information of the pending customer connection; allocating at least one of the tasks to be processed to the at least one task queue;
the message pushing module is used for executing:
for each of the push threads, performing:
acquiring the task to be processed from a task queue corresponding to the current push thread by using the current push thread, determining the client connection to be processed according to the information of the client connection to be processed in the task to be processed, and acquiring a message from a client connection buffer pool corresponding to the client connection to be processed;
packaging the messages in the client connection buffer pool corresponding to the client connection to be processed by using the current push thread to generate a current data packet to be pushed;
and pushing the data packet to be pushed to the client connection to be processed by utilizing the current pushing thread.
In an embodiment of the present invention, the apparatus further comprises:
a thread setting module, configured to set at least one obtaining thread, where the at least one obtaining thread is used to execute the obtaining of each message of the theme;
further comprising:
and the resource allocation module is used for judging whether the difference between the number of the messages acquired by the at least one acquisition thread at the current time and the number of the messages pushed by the at least one pushing thread at the current time is greater than or equal to a first preset value or not in real time, and if so, sleeping the acquisition threads with a second preset value in the at least one acquisition thread which is not in sleep.
Because the information interaction, execution process, and other contents between the units in the device are based on the same concept as the method embodiment of the present invention, specific contents may refer to the description in the method embodiment of the present invention, and are not described herein again.
The embodiment of the invention provides a readable medium, which comprises an execution instruction, and when a processor of a storage controller executes the execution instruction, the storage controller executes any message processing method provided by the embodiment of the invention.
An embodiment of the present invention provides a storage controller, including: a processor, a memory, and a bus;
the memory is used for storing execution instructions, the processor is connected with the memory through the bus, and when the memory controller runs, the processor executes the execution instructions stored in the memory, so that the memory controller executes any message processing method provided by the embodiment of the invention.
The embodiments of the invention have at least the following beneficial effects:
1. in the embodiment of the invention, a client connection buffer pool is set for each client connection, a message corresponding to each client connection is stored in the corresponding client connection buffer pool, the messages in the client connection buffer pool are uniformly packaged to generate a data packet to be pushed, and the whole data packet to be pushed is pushed to the corresponding client connection, that is, the messages in the client connection buffer pool can be integrally packaged without packaging one message, so that the times of packaging are reduced, the resources consumed in the packaging process are saved, the messages in the client connection buffer pool can be integrally pushed without pushing one message, the times of pushing are reduced, and the resources consumed in the pushing process are saved.
2. In the embodiment of the invention, the push weight is adjusted according to the receiving rate, so that the load balance is realized. Specifically, when the receiving rate of the client is high, the pushing weight is increased based on the initial pushing weight, and when the receiving rate of the client is low, the pushing weight is decreased based on the initial pushing weight, so that the reasonable distribution of the bandwidth is achieved.
3. In the embodiment of the invention, when the number of the acquisition threads is more and the number of the messages pushed by the pushing threads is less, a large number of messages are accumulated in the client connection buffer pool, at this time, part of the acquisition threads are dormant, more resources are used for processing the pushing threads, the message pushing speed is accelerated, and the dynamic allocation of the resources is realized, so that the optimal utilization effect of the resources is achieved.
4. In the embodiment of the invention, the resources and the bandwidth are reasonably distributed among all tasks by adopting the modes of packet sending, load balancing and dynamic resource distribution, so that the distribution and the use of the network bandwidth and the resources are optimized, the network bandwidth and the system resources are fully utilized, and the message is efficiently pushed to the client in real time.
5. In the embodiment of the invention, the acquired message packets are packaged and sent, thereby reducing the system expense and bandwidth expense for frequently sending small data packets and improving the sending speed.
6. In the embodiment of the invention, the utilization rate of the network is improved, the additional expenditure of the system is reduced and the high-speed transmission of a large amount of data can be realized by the modes of message packet sending, load balancing, dynamic allocation of the system and network resources and the like.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising a" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
Those of ordinary skill in the art will understand that: all or part of the steps for realizing the method embodiments can be completed by hardware related to program instructions, the program can be stored in a computer readable storage medium, and the program executes the steps comprising the method embodiments when executed; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.
Finally, it is to be noted that: the above description is only a preferred embodiment of the present invention, and is only used to illustrate the technical solutions of the present invention, and not to limit the protection scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.
Claims (2)
1. A method of message processing, characterized in that,
determining at least one topic to which at least one customer subscribes;
for each topic, determining a client connection corresponding to each client subscribing to the current topic and aiming at the current topic;
setting a client connection buffer pool for each client connection;
the method comprises the following steps:
periodically and respectively acquiring the message of each theme;
for each of the topics, performing: determining at least one client connection corresponding to the current theme; storing the acquired message of the current theme into at least one client connection buffer pool corresponding to the at least one client connection;
obtaining a message from the at least one client connection buffer pool;
respectively packaging the messages in the same client connection buffer pool to generate a data packet to be pushed corresponding to each client connection buffer pool;
pushing the data packets to be pushed corresponding to each client connection buffer pool to the corresponding client connection respectively;
further comprising:
setting a push weight for each of the customer connections;
the storing the obtained message of the current theme to at least one client connection buffer pool corresponding to the at least one client connection comprises:
for each of the at least one customer connection, performing:
determining a push weight corresponding to the current client connection;
determining the message of the current theme needing to be pushed to the current client connection according to the pushing weight corresponding to the current client connection;
storing the message of the current theme needing to be pushed to the current client connection into a corresponding client connection buffer pool;
the determining the push weight corresponding to the current client connection includes:
receiving an initial pushing weight input by a client corresponding to the current client connection;
determining the receiving rate of the client corresponding to the current client connection;
adjusting the initial pushing weight according to the receiving rate to generate a pushing weight corresponding to the current customer connection;
further comprising:
presetting at least one task queue, and distributing a push thread for each task queue;
after the storing the obtained message of the current topic into at least one client connection buffer pool corresponding to the at least one client connection, further comprising:
generating a pending task for each of the at least one customer connection, wherein the pending task includes information of the pending customer connection;
allocating at least one of the tasks to be processed to the at least one task queue;
said retrieving messages from said at least one client connection buffer pool, comprising:
for each of the push threads, performing: acquiring the task to be processed from a task queue corresponding to the current push thread by using the current push thread, determining the client connection to be processed according to the information of the client connection to be processed in the task to be processed, and acquiring a message from a client connection buffer pool corresponding to the client connection to be processed;
the said separately group package the message in the same said customer connects the buffer pool, produce the data packet to be pushed that every said customer connects the buffer pool to correspond, including:
for each of the push threads, performing: packaging the messages in the client connection buffer pool corresponding to the client connection to be processed by using the current push thread to generate a current data packet to be pushed;
the step of pushing the data packets to be pushed corresponding to each client connection buffer pool to the corresponding client connection respectively comprises:
for each of the push threads, performing: pushing the data packet to be pushed to the client connection to be processed by using the current pushing thread;
further comprising: presetting at least one acquisition thread, wherein the at least one acquisition thread is used for executing the information for respectively acquiring each theme;
further comprising:
and judging whether the difference between the number of the messages acquired by the at least one acquisition thread at the current time and the number of the messages pushed by the at least one pushing thread at the current time is greater than or equal to a first preset value or not in real time, and if so, sleeping the second preset value of the at least one acquisition thread which is not in sleep.
2. An apparatus for message processing, comprising:
the service management module is used for determining at least one topic subscribed by at least one client; for each topic, determining a client connection corresponding to each client subscribing to the current topic and aiming at the current topic; setting a client connection buffer pool for each client connection;
the message acquisition module is used for periodically and respectively acquiring the message of each theme; for each of the topics, performing: determining at least one client connection corresponding to the current theme; storing the acquired message of the current theme into at least one client connection buffer pool corresponding to the at least one client connection;
a message push module for obtaining messages from the at least one client connection buffer pool; respectively packaging the messages in the same client connection buffer pool to generate a data packet to be pushed corresponding to each client connection buffer pool; pushing the data packets to be pushed corresponding to each client connection buffer pool to the corresponding client connection respectively;
further comprising: a frequency setting module for setting a push weight for each of the customer connections;
the message obtaining module, when executing the storing of the obtained message of the current topic into the at least one client connection buffer pool corresponding to the at least one client connection, is specifically configured to:
for each of the at least one customer connection, performing:
determining a push weight corresponding to the current client connection;
determining the message of the current theme needing to be pushed to the current client connection according to the pushing weight corresponding to the current client connection;
storing the message of the current theme needing to be pushed to the current client connection into a corresponding client connection buffer pool;
the message obtaining module, when executing the push weight corresponding to the determined current client connection, is specifically configured to:
receiving an initial pushing weight input by a client corresponding to the current client connection;
determining the receiving rate of the client corresponding to the current client connection;
adjusting the initial pushing weight according to the receiving rate to generate a pushing weight corresponding to the current customer connection;
further comprising:
the queue setting module is used for setting at least one task queue and distributing a pushing thread for each task queue;
the message obtaining module is configured to, after storing the obtained message of the current topic in at least one client connection buffer pool corresponding to the at least one client connection, perform:
generating a pending task for each of the at least one customer connection, wherein the pending task includes information of the pending customer connection; allocating at least one of the tasks to be processed to the at least one task queue;
the message pushing module is used for executing:
for each of the push threads, performing:
acquiring the task to be processed from a task queue corresponding to the current push thread by using the current push thread, determining the client connection to be processed according to the information of the client connection to be processed in the task to be processed, and acquiring a message from a client connection buffer pool corresponding to the client connection to be processed;
packaging the messages in the client connection buffer pool corresponding to the client connection to be processed by using the current push thread to generate a current data packet to be pushed;
pushing the data packet to be pushed to the client connection to be processed by using the current pushing thread;
further comprising:
a thread setting module, configured to set at least one obtaining thread, where the at least one obtaining thread is used to execute the obtaining of each message of the theme;
further comprising:
and the resource allocation module is used for judging whether the difference between the number of the messages acquired by the at least one acquisition thread at the current time and the number of the messages pushed by the at least one pushing thread at the current time is greater than or equal to a first preset value or not in real time, and if so, sleeping the acquisition threads with a second preset value in the at least one acquisition thread which is not in sleep.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711326101.8A CN107911484B (en) | 2017-12-13 | 2017-12-13 | Message processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711326101.8A CN107911484B (en) | 2017-12-13 | 2017-12-13 | Message processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107911484A CN107911484A (en) | 2018-04-13 |
| CN107911484B true CN107911484B (en) | 2020-07-07 |
Family
ID=61865329
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711326101.8A Active CN107911484B (en) | 2017-12-13 | 2017-12-13 | Message processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107911484B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109067643B (en) * | 2018-09-26 | 2022-07-29 | 中国平安财产保险股份有限公司 | Reply method and device based on keywords, computer equipment and storage medium |
| CN112202914B (en) * | 2020-10-13 | 2023-02-24 | 湖南快乐阳光互动娱乐传媒有限公司 | Message pushing method and device |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102186256A (en) * | 2011-05-27 | 2011-09-14 | 华南理工大学 | Method for determining user priority order in long term evolution (LTE) scheduling |
| CN102255966A (en) * | 2011-07-07 | 2011-11-23 | 苏州阔地网络科技有限公司 | Message pushing method and instant communication server |
| CN102591721A (en) * | 2011-12-30 | 2012-07-18 | 北京新媒传信科技有限公司 | Method and system for distributing thread execution task |
| CN103020056A (en) * | 2011-09-20 | 2013-04-03 | 佳都新太科技股份有限公司 | Subscription pushing engine for cross-open-platform social intercourse information optimizing computation |
| CN103428290A (en) * | 2013-08-19 | 2013-12-04 | 北京网秦天下科技有限公司 | Method and device for pushing data |
| CN104980898A (en) * | 2014-04-04 | 2015-10-14 | 中兴通讯股份有限公司 | Information pushing method, system and equipment |
| CN106790696A (en) * | 2017-02-21 | 2017-05-31 | 腾讯科技（深圳）有限公司 | A kind of method for message transmission and device |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR100739721B1 (en) * | 2005-08-17 | 2007-07-13 | 삼성전자주식회사 | A method for providing information service and push mode service |
| US10271345B2 (en) * | 2014-10-20 | 2019-04-23 | Telefonaktiebolaget Lm Ericsson (Publ) | Network node and method for handling a process of controlling a data transfer related to video data of a video streaming service |
| CN104754036A (en) * | 2015-03-06 | 2015-07-01 | 合一信息技术（北京）有限公司 | Message processing system and processing method based on kafka |
-
2017
- 2017-12-13 CN CN201711326101.8A patent/CN107911484B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102186256A (en) * | 2011-05-27 | 2011-09-14 | 华南理工大学 | Method for determining user priority order in long term evolution (LTE) scheduling |
| CN102255966A (en) * | 2011-07-07 | 2011-11-23 | 苏州阔地网络科技有限公司 | Message pushing method and instant communication server |
| CN103020056A (en) * | 2011-09-20 | 2013-04-03 | 佳都新太科技股份有限公司 | Subscription pushing engine for cross-open-platform social intercourse information optimizing computation |
| CN102591721A (en) * | 2011-12-30 | 2012-07-18 | 北京新媒传信科技有限公司 | Method and system for distributing thread execution task |
| CN103428290A (en) * | 2013-08-19 | 2013-12-04 | 北京网秦天下科技有限公司 | Method and device for pushing data |
| CN104980898A (en) * | 2014-04-04 | 2015-10-14 | 中兴通讯股份有限公司 | Information pushing method, system and equipment |
| CN106790696A (en) * | 2017-02-21 | 2017-05-31 | 腾讯科技（深圳）有限公司 | A kind of method for message transmission and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107911484A (en) | 2018-04-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Peng et al. | R-storm: Resource-aware scheduling in storm | |
| US11734271B2 (en) | Data query method, apparatus and device | |
| US10558498B2 (en) | Method for scheduling data flow task and apparatus | |
| CN104915407B (en) | A kind of resource regulating method based under Hadoop multi-job environment | |
| US5875329A (en) | Intelligent batching of distributed messages | |
| Zhang et al. | Toward transcoding as a service: energy-efficient offloading policy for green mobile cloud | |
| US9766960B2 (en) | Workload-driven techniques for providing biased service level guarantees | |
| US20170068574A1 (en) | Multiple pools in a multi-core system | |
| US20150295970A1 (en) | Method and device for augmenting and releasing capacity of computing resources in real-time stream computing system | |
| JP5744707B2 (en) | Computer-implemented method, computer program, and system for memory usage query governor (memory usage query governor) | |
| US8626955B2 (en) | Directing packets to a processor unit | |
| Liu et al. | An economical and SLO-guaranteed cloud storage service across multiple cloud service providers | |
| US10114866B2 (en) | Memory-constrained aggregation using intra-operator pipelining | |
| US20180349178A1 (en) | A method and system for scalable job processing | |
| CN103927225A (en) | Multi-core framework Internet information processing and optimizing method | |
| WO2021190087A1 (en) | Task execution method, device and system, and server | |
| CN110058940B (en) | Data processing method and device in multi-thread environment | |
| US9817698B2 (en) | Scheduling execution requests to allow partial results | |
| US20160352649A1 (en) | Methods for an automatic scaling of data consumers and apparatuses using the same | |
| CN107911484B (en) | Message processing method and device | |
| Komarasamy et al. | A novel approach for Dynamic Load Balancing with effective Bin Packing and VM Reconfiguration in cloud | |
| CN102981973A (en) | Method of executing requests in memory system | |
| CN108304236B (en) | User interface refreshing method based on message subscription under cloud platform | |
| CN109358964B (en) | Server cluster resource scheduling method | |
| CN114900525A (en) | Method and system for deflecting data stream |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |