CN116132367A - A message processing method, device and electronic equipment - Google Patents

Abstract

The invention provides a message processing method, a message processing device and electronic equipment, wherein the method comprises the following steps: sending a message to be consumed to at least one consumer node of the at least two consumer nodes; receiving feedback messages sent by each consumer node of the at least one consumer node for the message to be consumed respectively, wherein each feedback message carries pressure related information of the corresponding consumer node, and the pressure related information of the consumer node is used for representing the load pressure condition of the consumer node; and carrying out load balancing according to the pressure related information of part or all of the at least two consumer nodes. The message processing method provided by the invention can improve the effect of load balancing and reduce the network resource overhead of pressure related information feedback.

Description

A message processing method, device and electronic equipment

technical field

The present invention relates to the technical field of data processing, in particular to a message processing method, device and electronic equipment.

Background technique

With the expansion of the Internet business field, the amount of business carried in various business fields is increasing. The traditional point-to-point information interaction mode has long been unable to carry business needs. On this premise, a horizontally expanded cluster mode is introduced. Consumer clusters, message producer clusters, and message middleware clusters, etc.

In practical applications, the interaction between clusters, whether it is direct interaction between consumer nodes and producer nodes through Transmission Control Protocol (TCP, Transmission Control Protocol)/Hyper Text Transfer Protocol (Hyper Text Transfer Protocol, HTTP), or introducing message After the middleware, the interaction between the producer node and the consumer node through the message middleware is faced with the problem of enabling the instances in each consumer node of the consumer cluster to consume messages at a certain speed without causing accumulation. load balancing between nodes. At present, the amount of messages consumed by each consumer node is usually based on the sharding algorithm. This load balancing method fails to consider the actual situation of the consumer nodes, resulting in a poor load balancing effect.

Contents of the invention

Embodiments of the present invention provide a message processing method, device and electronic equipment to solve the problem of poor load balancing effect in the prior art.

In order to solve the problems of the technologies described above, the present invention is achieved in that:

In the first aspect, an embodiment of the present invention provides a message processing method, which is applied to a first producer node, and communication connections are established between the first producer node and at least two consumer nodes, and the method includes:

sending a message to be consumed to at least one of the at least two consumer nodes;

Respectively receive a feedback message sent by each consumer node in the at least one consumer node for the message to be consumed, wherein each of the feedback messages carries pressure-related information of the corresponding consumer node, and the consumer The pressure related information of the node is used to characterize the load pressure condition of the consumer node;

Load balancing is performed according to the pressure-related information of some or all of the consumer nodes in the at least two consumer nodes.

In the second aspect, the embodiment of the present invention also provides a message processing method, which is applied to the first consumer node, and the method includes:

Receive the message to be consumed sent by the first producer node;

Perform message consumption on the message to be consumed;

sending a feedback message for the message to be consumed to the first producer node, wherein the feedback message carries pressure-related information of the first consumer node, and the pressure-related information of the first consumer node It is used to characterize the load pressure condition of the first consumer node.

In the third aspect, the embodiment of the present invention also provides a message processing device, which is applied to a first producer node, and communication connections are established between the first producer node and at least two consumer nodes, and the device includes:

A first sending module, configured to send a message to be consumed to at least one consumer node among the at least two consumer nodes;

A first receiving module, configured to respectively receive a feedback message sent by each consumer node in the at least one consumer node for the message to be consumed, wherein each of the feedback messages carries the pressure of the corresponding consumer node Related information, the pressure related information of the consumer node is used to characterize the load pressure condition of the consumer node;

A load balancing module, configured to perform load balancing according to pressure-related information of some or all of the consumer nodes among the at least two consumer nodes.

In the fourth aspect, the embodiment of the present invention also provides a message processing device, which is applied to the first consumer node, and the device includes:

The first receiving module is configured to receive the message to be consumed sent by the first producer node;

A message consumption module, configured to consume the message to be consumed;

A first sending module, configured to send a feedback message for the message to be consumed to the first producer node, wherein the feedback message carries pressure-related information of the first consumer node, and the first The pressure related information of the consumer node is used to characterize the load pressure condition of the first consumer node.

In the fifth aspect, an embodiment of the present invention also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and operable on the processor, and the computer program is executed by the processor When implementing the steps of the message processing method provided by the above first aspect, or implementing the steps of the message processing method provided by the above second aspect.

In a sixth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the message processing method provided in the above-mentioned first aspect is implemented. or implement the steps of the message processing method provided in the second aspect above.

In the embodiment of the present invention, the first producer node sends a message to be consumed to at least one consumer node among at least two consumer nodes; respectively receives each consumer node in the at least one consumer node for the Feedback messages to be sent by messages to be consumed, wherein each of the feedback messages carries pressure-related information of a corresponding consumer node; load is performed according to the pressure-related information of some or all of the at least two consumer nodes Balanced, since the first producer node performs load balancing according to the actual pressure of each message consumer node, the effect of load balancing can be improved. In addition, since each consumer node carries the message consumer in the feedback message for the message to be consumed Pressure-related information of nodes, which can reduce network resource overhead for pressure-related information feedback.

Description of drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the accompanying drawings that need to be used in the description of the embodiments of the present invention. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those skilled in the art, other drawings can also be obtained based on these drawings without any creative effort.

FIG. 1 is a schematic diagram of a point-to-point interaction provided by an embodiment of the present invention;

FIG. 2 is one of the schematic diagrams of inter-cluster interaction provided by an embodiment of the present invention;

FIG. 3 is a flowchart of a message processing method provided by an embodiment of the present invention;

FIG. 4 is a second schematic diagram of inter-cluster interaction provided by an embodiment of the present invention;

FIG. 5 is a third schematic diagram of inter-cluster interaction provided by an embodiment of the present invention;

Fig. 6 is a flow chart of a message processing method provided by another embodiment of the present invention;

FIG. 7 is a structural diagram of a message processing device provided by an embodiment of the present invention;

Fig. 8 is a structural diagram of a message processing device provided by another embodiment of the present invention;

FIG. 9 is a structural diagram of an electronic device provided by an embodiment of the present invention.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some of the embodiments of the present invention, but not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

For ease of understanding, the following describes some of the content involved in the embodiments of the present invention:

Load Balance: It can be understood as being distributed to multiple operating units for execution, such as Web servers, File Transfer Protocol (File Transfer Protocol, FTP) servers, key enterprise application servers, and other mission-critical servers, so as to jointly Complete tasks.

Producer nodes, also known as message producer nodes, belong to the initiator of the business and are responsible for producing messages and providing them to consumer nodes.

Consumer (Consumer) node: It can also be called a message consumer node, which belongs to the business processor and is responsible for obtaining messages from the producer node and performing business logic processing. Among them, the consumer node can directly obtain from the producer node through the synchronous interface Messages can also be obtained from producer nodes through message middleware.

Message middleware: It can also be called message queue, which refers to the platform-independent data exchange with efficient and reliable message delivery mechanism, and the integration of distributed systems based on data communication. By providing a message passing and message queue model, the communication of processes can be extended in a distributed environment.

Input/Output (IO)-intensive: refers to the CPU performance of the system is much better than that of the hard disk and memory. At this time, the system is running, and most of the situation is that the CPU is waiting for the I/O (hard disk/memory) read/write For write operations, the CPU load (Loading) is not high at this time. In most cases, for distributed systems, the above-mentioned IO usually refers to the overhead of network IO.

Point-to-point interaction: It can also be called single-node interaction, that is, a producer node interacts with only one consumer node. For example, as shown in Figure 1, the producer node sends a message to the consumer node, and the consumer node responds to the Return a feedback message after the message is processed. It can be understood that in the single-node interactive mode, there is no load balancing problem.

Inter-cluster interaction: that is, the interaction between multiple producer nodes and multiple consumer nodes. Exemplarily, as shown in Figure 2, each producer node in the producer cluster can communicate with multiple consumer nodes in the consumer cluster Consumer interaction, the producer node sends a message to the consumer node, and the consumer node returns a feedback message after processing the message.

An embodiment of the present invention provides a message processing method, which is applied to a first producer node, and communication connections are established between the first producer node and at least two consumer nodes. Referring to FIG. 3, FIG. 3 is a flowchart of a message processing method provided by an embodiment of the present invention. As shown in FIG. 3, it includes the following steps:

Step 301, sending a message to be consumed to at least one consumer node among at least two consumer nodes.

In this embodiment, a communication connection is established between the first producer node and at least two consumer nodes, that is to say, each of the above-mentioned at least two consumer nodes can receive the above-mentioned first producer node For the message to be consumed sent by the node, for example, as shown in FIG. 4 , each producer node may deliver the message to be consumed to multiple consumer nodes. Wherein, the above-mentioned first producer node may be any producer node.

The at least one consumer node may include some or all of the at least two consumer nodes. In practical applications, the first consumer node may send messages to be consumed to all consumer nodes in one message delivery, or may only send messages to be consumed to some consumer nodes. In addition, it can be understood that the message to be consumed sent by the first consumer node to different consumer nodes is different. For example, the first consumer node sends the message A to be consumed to consumer node A, and the first consumer node Send message B to be consumed to consumer node B, and so on.

Step 302, respectively receiving feedback messages sent by each of the at least one consumer node for the message to be consumed, wherein each of the feedback messages carries pressure-related information of the corresponding consumer node, so The pressure related information of the consumer node is used to characterize the load pressure condition of the consumer node.

Exemplarily, as shown in FIG. 4, after each consumer node finishes processing the message to be consumed, it can send a feedback message for the message to be consumed to the first producer node, and the feedback message carries the pressure of the consumer node. Related Information. For example, if the first consumer node sends a message A to be consumed to the consumer node A, after the consumer node A completes the processing of the message A to be consumed, it can send a feedback message for the message A to be consumed to the first consumer node, The feedback message carries the pressure-related information of consumer node A; if the first consumer node sends message B to be consumed to consumer node B, after consumer node B completes the processing of message B to be consumed, it can report to the first consumer node The sender node sends a feedback message for the message B to be consumed, the feedback message carries the pressure-related information of the consumer node B, and so on.

The above-mentioned pressure-related information of the consumer node is used to characterize the load pressure condition of the consumer node, that is to say, the load pressure condition of the consumer node can be known through the pressure-related information of the consumer node, for example, the load pressure is high or Load pressure is small, etc. Exemplarily, the above-mentioned stress-related information of the consumer node may include but not limited to at least one of a stress index value, a stress evaluation value, and the like. Wherein, the above-mentioned pressure index value may include one or more of disk usage rate, CPU usage rate, memory usage rate, downstream call response time, disk writing speed, thread pool size, and the like. The above pressure assessment value may be calculated based on one or more pressure index values, for example, a pressure assessment score may be calculated based on CPU occupancy rate and memory occupancy rate, and the pressure assessment score value may be fed back to the producer node.

In some optional embodiments, when the first producer node receives the pressure-related information of each consumer node in the at least one consumer node, it may use the pressure-related information of each consumer node in the at least one consumer node Stress-related information is stored, for example, in a local cache or a shared cache, where the above-mentioned shared cache can also be called a distributed cache, which can be read and written by multiple producer nodes.

Step 303: Perform load balancing according to pressure-related information of some or all of the consumer nodes in the at least two consumer nodes.

For the above step 303, for example, the first producer node may perform load balancing directly based on the received pressure-related information of each consumer node in the at least one consumer node, or the first producer node may load balance from a cache (for example , local cache or shared cache, etc.) to read the pressure-related information of each of the above-mentioned at least two consumer nodes, and based on the read pressure-related information of each of the above-mentioned at least two consumer nodes information load balancing. Among them, the above load balancing is performed according to the pressure-related information of the consumer nodes. For example, less messages to be consumed are allocated to consumer nodes with higher load pressure, and more messages to be consumed are allocated to consumer nodes with lower load pressure. .

In the message processing method provided by the embodiment of the present invention, the first producer node sends a message to be consumed to at least one consumer node among at least two consumer nodes; respectively receives each consumer in the at least one consumer node A feedback message sent by a node for the message to be consumed, wherein each of the feedback messages carries pressure-related information of a corresponding consumer node; according to the pressure of some or all of the consumer nodes in the at least two consumer nodes Relevant information is used for load balancing. Because the first producer node performs load balancing according to the actual pressure of each message consumer node, the effect of load balancing can be improved. In addition, since each consumer node carries in the feedback message for the message to be consumed The pressure-related information of the message consumer node can reduce the network resource overhead of pressure-related information feedback. It should also be noted that since the consumer node directly reports the above pressure-related information to the producer node, that is, there is no interaction with other devices except the consumer node and the producer node, which can reduce the pressure-related information reported by the consumer node time delay, which can ensure that the pressure-related information of consumer nodes acquired by producer nodes can more accurately reflect the current pressure situation of consumer nodes, thereby improving the effect of load balancing based on the pressure-related information of consumer nodes.

Optionally, each of the at least two consumer nodes has a communication connection with at least two producer nodes, and the first producer node is the at least two producer nodes Any producer node in ;

After respectively receiving the feedback message sent by each consumer node in the at least one consumer node for the message to be consumed, the method further includes:

storing pressure-related information for each of the at least one consumer node in a shared cache, wherein the shared cache supports sharing between the at least two producer nodes;

The load balancing according to the pressure-related information of some or all of the consumer nodes in the at least two consumer nodes includes:

Acquiring pressure-related information of each of the at least two consumer nodes from the shared cache at intervals of a first preset duration;

Load balancing is performed according to the pressure-related information of each consumer node in the at least two consumer nodes.

The above-mentioned first preset duration can be reasonably set according to the actual situation, for example, 1 minute, 5 minutes or 10 minutes. In some optional embodiments, the above-mentioned first preset duration can be set according to the business type. For example, for a business with a large amount of messages, the above-mentioned first preset duration can be set smaller, and for a business with a small amount of messages , the above-mentioned first preset duration can be set larger.

Exemplarily, as shown in FIG. 5 , each producer node stores the pressure-related information of the consumer node in the shared cache after receiving the pressure-related information of the consumer node returned by the consumer node. In addition, each of the above-mentioned producer nodes can read the latest pressure-related information of each consumer node from the shared cache every first preset time period, and perform load balancing based on the latest pressure-related information of each consumer node.

In this embodiment, each producer node can store the pressure-related information received by the consumer node in the shared cache, so that the latest pressure-related information of each consumer node can be stored in the shared cache, and each producer node can be based on the shared Cache to obtain the latest pressure-related information of each consumer node for load balancing, which can improve the load balancing effect of each consumer node. In addition, in this embodiment, the producer nodes are equal to each other, and there is no central node or master node, so no There is a cluster election action, so there will be no cluster unavailability during the election process, and it also avoids the possibility of having a central node or a master node resulting in a "split brain" and degenerating into two different and irrelevant clusters, thereby avoiding It eliminates data inconsistency and improves the stability of the cluster.

In some optional embodiments, the method also includes:

In the case that no message to be consumed is sent to the first consumer node within the second preset time period, receiving the pressure-related information of the first consumer node sent by the first consumer node, wherein the first The consumer node is any consumer node in the at least two consumer nodes.

In this embodiment, the above-mentioned second preset duration can be reasonably set according to the actual situation, for example, 15 minutes, 30 minutes, 1 hour and so on.

In this embodiment, the consumer node can actively feed back its pressure-related information to the producer node when it has not received the message to be consumed for a long time, so that the producer node can actively feed back the pressure-related information based on the consumer node Updating the stored pressure-related information of the consumer node is convenient for the producer node to perform load balancing based on the latest pressure-related information of each consumer node, so as to improve the effect of load balancing.

In some optional embodiments, the pressure-related information of the consumer node includes at least one of the following: a pressure index value of the consumer node, and a stress evaluation value;

Wherein, the pressure evaluation value is determined according to the pressure index value of the consumer node.

In some optional embodiments, the pressure indicator value may include but not limited to at least one of the following: disk usage, CPU usage, memory usage, and downstream call response time.

The above-mentioned pressure evaluation value can be directly used to evaluate the pressure of the above-mentioned consumer node. For example, the above-mentioned pressure evaluation value can be a pressure score value calculated according to at least one pressure index value, wherein the greater the pressure score value of the consumer node , indicating that the greater the load pressure of the consumer node, the smaller the pressure score value of the consumer node, indicating that the lower the load pressure of the consumer node.

In one embodiment, each consumer node can report at least one pressure indicator value to the producer node, for example, disk occupancy rate, CPU occupancy rate, memory occupancy rate, downstream call response time, etc., so that the producer nodes can respectively according to At least one pressure index value reported by each consumer node calculates the load balancing weight value or load balancing score corresponding to each consumer node, and then load balancing can be performed based on the load balancing weight value or load balancing score corresponding to each consumer node . This method is beneficial to ensure the consistency of the calculation method of the load balancing weight value or the load balancing score corresponding to each consumer node, thereby improving the accuracy of the load balancing.

In another embodiment, each consumer node can report to the producer node the pressure evaluation value calculated based on at least one pressure index value, so that the producer node can directly perform load balancing based on the pressure evaluation value of each consumer node, No additional calculation is required, which not only reduces the resource overhead of the consumer node for reporting pressure-related information, but also saves the computing resources of the producer node.

In some optional embodiments, performing load balancing according to pressure-related information of some or all of the at least two consumer nodes includes:

Perform load balancing according to the pressure-related information of each of the at least two consumer nodes and the message response time corresponding to each of the consumer nodes, wherein the message response time is the first producer The node sends the message to be consumed to the time when the first producer node receives a feedback message for the message to be consumed.

In practical applications, there may be differences in the network status between the producer node and each consumer node. For a consumer node with a better network status, its message response time is often shorter. At this time, the producer node often hopes to send the consumer node Nodes send more messages to be consumed; and for consumer nodes with poor network status, their message response time is often longer, at this time, producer nodes often want to send fewer messages to be consumed to the consumer node, so that It is beneficial to improve the efficiency of message push.

In this embodiment, load balancing is carried out by synthesizing the pressure-related information and message response time of each consumer node. For example, the producer node can perform weighted calculation on the pressure evaluation value and message response time of the consumer node to obtain a load balancing score, and based on The load balancing score is used for load balancing, so that the efficiency of message push can be taken into account while ensuring the load balancing effect between consumer nodes.

Fig. 6 is a flowchart of a message processing method provided by another embodiment of the present invention, the message processing method is applied to the first consumer node, as shown in Fig. 6 , including the following steps:

Step 601. Receive a message to be consumed sent by a first producer node.

In this embodiment, the above-mentioned first producer node may be any producer node, wherein communication connections are established between the first producer node and at least two consumer nodes, that is, the above-mentioned at least two consumer nodes Each consumer node among the producer nodes can receive the to-be-consumed message sent by the above-mentioned first producer node. The above-mentioned first consumer node may be any consumer node among the above-mentioned at least two consumer nodes.

Step 602, perform message consumption on the message to be consumed.

It can be understood that the above-mentioned first consumer node can consume the message to be consumed, which means that the above-mentioned first consumer node performs related processing on the message to be consumed. For example, if the above-mentioned message to be consumed is an account query request message, the first consumer node can The account query request message queries the relevant information of the corresponding account.

Step 603: Send a feedback message for the message to be consumed to the first producer node, wherein the feedback message carries pressure-related information of the first consumer node, and the pressure-related information of the first consumer node The information is used to characterize the load pressure condition of the first consumer node.

The above-mentioned pressure-related information of the consumer node is used to characterize the load pressure condition of the consumer node, that is to say, the load pressure condition of the consumer node can be known through the pressure-related information of the consumer node, for example, the load pressure is high or Load pressure is small, etc. Exemplarily, the above-mentioned stress-related information of the consumer node may include but not limited to at least one of a stress index value, a stress evaluation value, and the like. Wherein, the above-mentioned pressure index value may include one or more of disk usage rate, CPU usage rate, memory usage rate, downstream call response time, disk writing speed, thread pool size, and the like. The above pressure assessment value may be calculated based on one or more pressure index values, for example, a pressure assessment score may be calculated based on CPU occupancy rate and memory occupancy rate, and the pressure assessment score value may be fed back to the producer node.

Exemplarily, if the message to be consumed is an account query request message, the feedback message of the message to be consumed may include relevant information of the corresponding account. In addition, the feedback message may also carry pressure-related information of the first consumer node.

In some optional embodiments, the first consumer node may carry pressure-related information of the first consumer node in the feedback message of the message to be consumed each time it receives a message to be consumed; or, the first consumer node A consumer node may carry the pressure-related information of the first consumer node in the feedback message of the latest received message to be consumed only when the pressure-related information changes.

In practical applications, the first producer node can send messages to be consumed to multiple consumer nodes. Exemplarily, as shown in FIG. A producer node sends a feedback message for the message to be consumed, and the feedback message carries pressure-related information of the consumer node. For example, after the consumer node A completes the processing of the message A to be consumed, it can send a feedback message for the message A to be consumed to the first consumer node, the feedback message carries the pressure-related information of the consumer node A; the consumer node B After completing the processing of the message B to be consumed, a feedback message for the message B to be consumed may be sent to the first consumer node, the feedback message carrying the pressure-related information of the consumer node B, and so on. In this way, the first producer node can perform load balancing based on the pressure-related information of each consumer node. For example, allocate less messages to be consumed to consumer nodes with high load pressure, and allocate more to consumer nodes with low load pressure. Messages to be consumed.

In the message processing method provided by the embodiment of the present invention, the first consumer node receives the message to be consumed sent by the first producer node, consumes the message to be consumed, and sends the message to the first producer node for The feedback message of the message to be consumed carries the pressure-related information of the first consumer node, since the consumer node carries the pressure-related information of the message consumer node in the feedback message for the message to be consumed, this can reduce the pressure-related information Feedback network resource overhead, in addition, because the consumer node directly reports the above pressure-related information to the producer node, that is, there is no interaction with other devices except the consumer node and the producer node, which can reduce the pressure on the consumer node The delay in reporting relevant information can ensure that the pressure-related information of consumer nodes obtained by producer nodes can more accurately reflect the current pressure situation of consumer nodes, which in turn can improve load balancing based on the pressure-related information of consumer nodes Effect.

In some optional embodiments, the method also includes:

Sending the pressure-related information of the first consumer node to the first producer node if the message to be consumed sent by the first producer node is not received within the second preset time period;

or

In the case that no message to be consumed sent by any producer node among the at least two producer nodes is received within the second preset time period, the first consumption message is sent to the producer node among the at least two producer nodes pressure-related information of a producer node, wherein a communication connection is established between the first consumer node and the at least two producer nodes, and the at least two producer nodes include the first producer node.

In this embodiment, the above-mentioned second preset duration can be reasonably set according to the actual situation, for example, 15 minutes, 30 minutes, 1 hour and so on.

In one embodiment, the first consumer node can actively report the pressure of the first consumer node to the producer node if it does not receive the message to be consumed sent by the producer node within the second preset time period In this way, each producer node can directly obtain the latest pressure-related information of the first consumer node.

In another embodiment, if the first consumer node does not receive any message to be consumed sent by any producer node within the second preset time period, it may send a message to any one of the at least two producer nodes or Multiple producer nodes actively report the pressure-related information of the first consumer node, or may actively report the pressure-related information of the first consumer node to a specific producer node among the above-mentioned at least two producer nodes, for example, may Actively report the pressure-related information of the first consumer node to the target producer node among the at least two producer nodes, wherein the message to be consumed last received by the first consumer node is the message to be consumed sent by the above-mentioned target producer node In this way, when the first consumer node interacts with at least two producer nodes, the network resource overhead of the first consumer node reporting pressure-related messages can be saved.

In some optional embodiments, the pressure-related information of the consumer node includes at least one of the following: a pressure index value of the consumer node, and a stress evaluation value;

Wherein, the pressure evaluation value is determined according to the pressure index value of the consumer node.

In some optional embodiments, the pressure indicator value includes at least one of the following: disk occupancy, CPU occupancy, memory occupancy, and downstream call response time.

The above-mentioned pressure evaluation value can be directly used to evaluate the pressure of the above-mentioned consumer node. For example, the above-mentioned pressure evaluation value can be a pressure score value calculated according to at least one pressure index value, wherein the greater the pressure score value of the consumer node , indicating that the greater the load pressure of the consumer node, the smaller the pressure score value of the consumer node, indicating that the lower the load pressure of the consumer node.

In one embodiment, each consumer node can report at least one pressure indicator value to the producer node, for example, disk occupancy rate, CPU occupancy rate, memory occupancy rate, downstream call response time, etc., so that the producer nodes can respectively according to At least one pressure index value reported by each consumer node calculates the load balancing weight value or load balancing score corresponding to each consumer node, and then load balancing can be performed based on the load balancing weight value or load balancing score corresponding to each consumer node . This method is beneficial to ensure the consistency of the calculation method of the load balancing weight value or the load balancing score corresponding to each consumer node, thereby improving the accuracy of the load balancing.

In another embodiment, each consumer node can report to the producer node the pressure evaluation value calculated based on at least one pressure index value, so that the producer node can directly perform load balancing based on the pressure evaluation value of each consumer node, No additional calculation is required, which not only reduces the resource overhead of the consumer node for reporting pressure-related information, but also saves the computing resources of the producer node.

In some optional embodiments, the stress evaluation value is calculated according to at least two stress index values of the consumer node and a target weight group, wherein the target weight group includes the at least two stress index values The weight of each pressure indicator value in .

In this embodiment, the aforementioned target weight group may be a preset weight group, or may be a weight group selected from multiple weight groups.

Exemplarily, the above-mentioned at least two stress index values include CPU occupancy rate and memory occupancy rate, and the above-mentioned target weight group includes a first weight and a second weight, wherein the first weight is the weight of the CPU occupancy rate, and the second weight is the memory occupancy rate. occupancy rate, the first consumer node can multiply the CPU occupancy rate by the first weight to obtain the first value, multiply the memory occupancy rate by the second weight to obtain the second value, and multiply the first value by the second weight The two values are added to obtain the above pressure estimate.

In some optional embodiments, the target weight group is the weight group corresponding to the target calculation period, and the target calculation period is the calculation period with the smallest variance value corresponding to the N calculation periods, where N is an integer greater than 1 , the N calculation cycles respectively correspond to N different weight groups, each of which includes the weight of each pressure index value in the at least two pressure index values;

Wherein, the variance value corresponding to the i-th calculation period in the N calculation periods is determined in the following manner:

In the i-th calculation period, calculate the i-th pressure evaluation value according to at least two pressure index values and the i-th weight group of the consumer node; the value range of i is [1, N];

sending the i-th pressure evaluation value to the first producer node, so that the first producer node performs load balancing according to the i-th pressure evaluation value;

Count the number of request responses corresponding to each time unit in the M time units in the i-th calculation cycle, where M is an integer greater than 1;

Calculate the ratio of the number of request responses corresponding to each time unit in the M time units and the i-th pressure evaluation value to obtain M ratios;

Calculate the variance values of the M ratios to obtain the variance values corresponding to the i-th calculation period.

In this embodiment, the length of the above calculation period may be reasonably set according to actual conditions, for example, 1 minute or 5 minutes. The above time unit can be reasonably set according to the actual situation, for example, 1 second, 10 seconds, 30 seconds or 1 minute. It should be noted that a calculation cycle includes M or more time units. The aforementioned N calculation cycles may be historical N calculation cycles. The number of request responses corresponding to the above time unit may include the number of messages to be consumed processed within the time unit, or the number of messages to be consumed received within the time unit, and the like.

The weight group corresponding to each calculation period in the above N calculation periods can be preset; it can also be determined according to the preset weight value range, for example, the weight value range corresponding to each pressure index value is set in advance , for each calculation cycle, the weight corresponding to each pressure index value is obtained from the weight value range corresponding to each pressure index value in sequence according to the preset step size, and the weight group corresponding to the calculation cycle is obtained.

Specifically, each consumer node calculates the pressure evaluation value corresponding to the calculation cycle based on the corresponding weight group in each calculation cycle and reports it to the first producer node. The first producer node can be based on the The pressure evaluation value is used for load balancing, so that each consumer node can count the ratio of the number of request responses corresponding to each time unit in each calculation cycle to the pressure evaluation value corresponding to the calculation cycle, and can calculate the corresponding The variance value of the ratio of is used as the variance value corresponding to the calculation period. After obtaining the variance value corresponding to each calculation period in the N calculation periods, the weight group corresponding to the calculation period with the smallest variance value can be used as the target weight reorganization.

In this embodiment, the first consumer node calculates the pressure evaluation value based on the corresponding weight group in each calculation cycle and sends it to the first producer node, so that the first producer node performs load calculation based on the reported pressure evaluation value. balance, so that the first consumer node can count the number of request responses corresponding to multiple time units in the corresponding calculation period after load balancing based on each pressure evaluation value, to calculate the variance value corresponding to each calculation period, and choose the method The weight group corresponding to the calculation period with the smallest difference is used as the target weight group, because the number of request responses corresponding to multiple time units in the corresponding calculation period after load balancing based on each pressure evaluation value can be more accurately reflected based on the calculation of each weight group Therefore, the selection of the target weight group based on the variance value calculated by the number of request responses corresponding to multiple time units in each calculation cycle can ensure that the selected target weight group is more reasonable, which in turn can improve The effect of load balancing based on the pressure evaluation value calculated by the target weight group.

In some optional embodiments, the first consumer node can also provide a reconstruction function, for example, an interface for changing the calculation of the pressure evaluation value can be provided, through which the calculation method of the pressure evaluation value can be changed, for example, changing The pressure index value, weight, etc. that participate in the calculation of the pressure assessment value allow the consumer node to realize the calculation of the pressure assessment value with its own characteristics, and provide the calculation of the pressure assessment value that meets its own needs.

To sum up, the message processing method provided by this embodiment can make the overall resource pressure of the consumer node relatively balanced, can refine the management and use of computing resources, can strengthen the robustness of the cluster, and avoid cluster failures caused by single node failures. "Avalanche"; no central coordination node is required; it can be implemented in more frequent network business interactions without additional network overhead for load balancing data, and can also be implemented when teaching less business, or because a node is temporarily under pressure Large, when the requested data is small, the frequency of active reporting. Compared with traditional load balancing, it can reduce the cost of load balancing equipment, reduce the cost of load balancing information transmission, and improve the effect of daily load balancing.

Referring to FIG. 7, FIG. 7 is a structural diagram of a message processing device provided by an embodiment of the present invention. The message processing device is applied to a first producer node, and a network is established between the first producer node and at least two consumer nodes. Communication connection, as shown in Figure 7, the message processing device 700 includes:

The first sending module 701 is configured to send a message to be consumed to at least one consumer node among the at least two consumer nodes;

The first receiving module 702 is configured to respectively receive a feedback message sent by each consumer node in the at least one consumer node for the message to be consumed, wherein each of the feedback messages carries the corresponding consumer node Pressure-related information, the pressure-related information of the consumer node is used to characterize the load pressure status of the consumer node;

A load balancing module 703, configured to perform load balancing according to pressure-related information of some or all of the consumer nodes in the at least two consumer nodes.

Optionally, each of the at least two consumer nodes has a communication connection with at least two producer nodes, and the first producer node is the at least two producer nodes Any producer node in ;

The device also includes:

The storage module is configured to, after respectively receiving the feedback message sent by each consumer node in the at least one consumer node for the message to be consumed, store the information of each consumer node in the at least one consumer node Stress-related information is stored in a shared cache, wherein the shared cache supports sharing between the at least two producer nodes;

The load balancing module is specifically used for:

Acquiring pressure-related information of each of the at least two consumer nodes from the shared cache at intervals of a first preset duration;

Load balancing is performed according to the pressure-related information of each consumer node in the at least two consumer nodes.

Optionally, the device also includes:

The second receiving module is configured to receive the pressure-related information of the first consumer node sent by the first consumer node when no message to be consumed is sent to the first consumer node within a second preset time period , wherein the first consumer node is any consumer node in the at least two consumer nodes.

Optionally, the pressure-related information of the consumer node includes at least one of the following: a pressure index value of the consumer node, and a pressure evaluation value;

Wherein, the pressure evaluation value is determined according to the pressure index value of the consumer node.

Optionally, the pressure indicator value includes at least one of the following: disk occupancy, CPU occupancy, memory occupancy, and downstream call response time.

Optionally, the load balancing module is specifically used for:

Perform load balancing according to the pressure-related information of each of the at least two consumer nodes and the message response time corresponding to each of the consumer nodes, wherein the message response time is the first producer The node sends the message to be consumed to the time when the first producer node receives a feedback message for the message to be consumed.

The message processing device 700 provided by the embodiment of the present invention can realize each process in the above-mentioned first producer node side method embodiment, and to avoid repetition, details are not repeated here.

Referring to FIG. 8 , FIG. 8 is a structural diagram of a message processing device provided by an embodiment of the present invention, and the message processing device is applied to a first consumer node. As shown in Figure 8, the message processing device 800 includes:

The first receiving module 801 is configured to receive the message to be consumed sent by the first producer node;

A message consumption module 802, configured to consume the message to be consumed;

The first sending module 803 is configured to send a feedback message for the message to be consumed to the first producer node, where the feedback message carries pressure-related information of the first consumer node, and the second The pressure related information of a consumer node is used to characterize the load pressure condition of the first consumer node.

Optionally, the device further includes a second sending module, and the second sending module is specifically configured to:

Sending the pressure-related information of the first consumer node to the first producer node if the message to be consumed sent by the first producer node is not received within the second preset time period;

or

In the case that no message to be consumed sent by any producer node among the at least two producer nodes is received within the second preset time period, the first consumption message is sent to the producer node among the at least two producer nodes pressure-related information of a producer node, wherein a communication connection is established between the first consumer node and the at least two producer nodes, and the at least two producer nodes include the first producer node.

Optionally, the pressure-related information of the consumer node includes at least one of the following: a pressure index value of the consumer node, and a pressure evaluation value;

Wherein, the pressure evaluation value is determined according to the pressure index value of the consumer node.

Optionally, the pressure indicator value includes at least one of the following: disk occupancy, CPU occupancy, memory occupancy, and downstream call response time.

Optionally, the pressure evaluation value is calculated according to at least two pressure index values and a target weight group of the consumer node, wherein the target weight group includes each pressure index in the at least two pressure index values The weight of the value.

Optionally, the target weight group is a weight group corresponding to a target calculation period, and the target calculation period is a calculation period corresponding to the smallest variance value among N calculation periods, N is an integer greater than 1, and the N The calculation periods respectively correspond to N different weight groups, each of which includes the weight of each pressure index value in the at least two pressure index values;

Wherein, the variance value corresponding to the i-th calculation period in the N calculation periods is determined in the following manner:

In the i-th calculation period, calculate the i-th pressure evaluation value according to at least two pressure index values and the i-th weight group of the consumer node; the value range of i is [1, N];

sending the i-th pressure evaluation value to the first producer node, so that the first producer node performs load balancing according to the i-th pressure evaluation value;

Count the number of request responses corresponding to each time unit in the M time units in the i-th calculation cycle, where M is an integer greater than 1;

Calculate the ratio of the number of request responses corresponding to each time unit in the M time units and the i-th pressure evaluation value to obtain M ratios;

Calculate the variance values of the M ratios to obtain the variance values corresponding to the i-th calculation period.

The message processing apparatus 800 provided by the embodiment of the present invention can realize each process in the above-mentioned first consumer node side method embodiment, and to avoid repetition, details are not repeated here.

Referring to FIG. 9, FIG. 9 is a structural diagram of a message processing device provided by another embodiment of the present invention. As shown in FIG. 9, the message processing device 900 includes: a processor 901, a memory 902, and a The computer program running on the processor, the various components in the message processing device 900 are coupled together through the bus interface 903, and the computer program is executed by the processor 901 to implement the following steps:

sending a message to be consumed to at least one of the at least two consumer nodes;

Respectively receive a feedback message sent by each consumer node in the at least one consumer node for the message to be consumed, wherein each of the feedback messages carries pressure-related information of the corresponding consumer node, and the consumer The pressure related information of the node is used to characterize the load pressure condition of the consumer node;

performing load balancing according to pressure-related information of some or all of the at least two consumer nodes;

or

Receive the message to be consumed sent by the first producer node;

Perform message consumption on the message to be consumed;

sending a feedback message for the message to be consumed to the first producer node, wherein the feedback message carries pressure-related information of the first consumer node, and the pressure-related information of the first consumer node It is used to characterize the load pressure condition of the first consumer node.

It should be understood that, in this embodiment of the present invention, when the computer program is executed by the processor 901, each process in the above embodiment of the message processing method can be implemented, and details are not repeated here to avoid repetition.

An embodiment of the present invention also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and operable on the processor. When the computer program is executed by the processor, the above message processing method embodiment is implemented. Each process can achieve the same technical effect, so in order to avoid repetition, it will not be repeated here.

The embodiment of the present invention also provides a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, each process of the above message processing method embodiment is realized, and the same technology can be achieved. Effect, in order to avoid repetition, will not repeat them here. Wherein, the computer-readable storage medium is, for example, a read-only memory (Read-Only Memory, ROM for short), a random access memory (Random Access Memory, RAM for short), a magnetic disk or an optical disk, and the like.

It should be noted that, in this document, the term "comprising", "comprising" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or apparatus comprising a set of elements includes not only those elements, It also includes other elements not expressly listed, or elements inherent in the process, method, article, or device. Without further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus comprising that element.

Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware, but in many cases the former is better implementation. Based on this understanding, the technical solution of the present invention can be embodied in the form of a software product in essence or the part that contributes to the prior art, and the computer software product is stored in a storage medium (such as ROM/RAM, disk, CD) contains several instructions to make a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the methods described in various embodiments of the present invention.

Embodiments of the present invention have been described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementations, and the above-mentioned specific implementations are only illustrative, rather than restrictive, and those of ordinary skill in the art will Under the enlightenment of the present invention, without departing from the gist of the present invention and the protection scope of the claims, many forms can also be made, all of which belong to the protection of the present invention.

Claims (16)

1. A message processing method, characterized in that it is applied to a first producer node, and a communication connection is established between the first producer node and at least two consumer nodes, the method comprising: sending a message to be consumed to at least one of the at least two consumer nodes; Respectively receive a feedback message sent by each consumer node in the at least one consumer node for the message to be consumed, wherein each of the feedback messages carries pressure-related information of the corresponding consumer node, and the consumer The pressure related information of the node is used to characterize the load pressure condition of the consumer node; Load balancing is performed according to the pressure-related information of some or all of the consumer nodes in the at least two consumer nodes. 2. The method according to claim 1, wherein a communication connection is established between each consumer node in the at least two consumer nodes and at least two producer nodes, and the first producer node The producer node is any producer node in the at least two producer nodes; After respectively receiving the feedback message sent by each consumer node in the at least one consumer node for the message to be consumed, the method further includes: storing pressure-related information for each of the at least one consumer node in a shared cache, wherein the shared cache supports sharing between the at least two producer nodes; The load balancing according to the pressure-related information of some or all of the consumer nodes in the at least two consumer nodes includes: Acquiring pressure-related information of each of the at least two consumer nodes from the shared cache at intervals of a first preset duration; Load balancing is performed according to the pressure-related information of each consumer node in the at least two consumer nodes. 3. The method according to claim 1, wherein the method further comprises: In the case that no message to be consumed is sent to the first consumer node within the second preset time period, receiving the pressure-related information of the first consumer node sent by the first consumer node, wherein the first The consumer node is any consumer node in the at least two consumer nodes. 4. The method according to claim 1, wherein the pressure-related information of the consumer node includes at least one of the following: a pressure index value of the consumer node, a pressure evaluation value; Wherein, the pressure evaluation value is determined according to the pressure index value of the consumer node. 5. The method according to claim 4, wherein the pressure indicator value includes at least one of the following: disk occupancy, CPU occupancy, memory occupancy, and downstream call response time. 6. The method according to claim 1, wherein the load balancing according to the pressure-related information of some or all of the consumer nodes in the at least two consumer nodes comprises: Perform load balancing according to the pressure-related information of each of the at least two consumer nodes and the message response time corresponding to each of the consumer nodes, wherein the message response time is the first producer The node sends the message to be consumed to the time when the first producer node receives a feedback message for the message to be consumed. 7. A message processing method, characterized in that it is applied to the first consumer node, and the method comprises: Receive the message to be consumed sent by the first producer node; Perform message consumption on the message to be consumed; sending a feedback message for the message to be consumed to the first producer node, wherein the feedback message carries pressure-related information of the first consumer node, and the pressure-related information of the first consumer node It is used to characterize the load pressure condition of the first consumer node. 8. The method according to claim 7, further comprising: If the message to be consumed sent by the first producer node is not received within the second preset time period, sending pressure-related information of the first consumer node to the first producer node; or In the case that no message to be consumed sent by any producer node among the at least two producer nodes is received within the second preset time period, the first consumption message is sent to the producer node among the at least two producer nodes pressure-related information of a producer node, wherein a communication connection is established between the first consumer node and the at least two producer nodes, and the at least two producer nodes include the first producer node. 9. The method according to claim 7, wherein the pressure-related information of the consumer node includes at least one of the following: a pressure index value of the consumer node, a pressure evaluation value; Wherein, the pressure evaluation value is determined according to the pressure index value of the consumer node. 10. The method according to claim 9, wherein the pressure indicator value includes at least one of the following: disk usage, CPU usage, memory usage, and downstream call response time. 11. The method according to claim 9, wherein the pressure assessment value is calculated according to at least two stress index values and a target weight group of the consumer node, wherein the target weight group includes the A weight for each of the at least two stress indicator values. 12. The method according to claim 11, wherein the target weight group is a weight group corresponding to a target calculation period, and the target calculation period is a calculation period corresponding to the smallest variance value among N calculation periods, N is an integer greater than 1, and the N calculation periods correspond to N different weight groups, each of which includes the weight of each pressure index value in the at least two pressure index values; Wherein, the variance value corresponding to the i-th calculation period in the N calculation periods is determined in the following manner: In the i-th calculation period, calculate the i-th pressure evaluation value according to at least two pressure index values and the i-th weight group of the consumer node; the value range of i is [1, N]; sending the i-th pressure evaluation value to the first producer node, so that the first producer node performs load balancing according to the i-th pressure evaluation value; Count the number of request responses corresponding to each time unit in the M time units in the i-th calculation cycle, where M is an integer greater than 1; Calculate the ratio of the number of request responses corresponding to each time unit in the M time units and the i-th pressure evaluation value to obtain M ratios; Calculate the variance values of the M ratios to obtain the variance values corresponding to the i-th calculation period. 13. A message processing device, characterized in that it is applied to a first producer node, and a communication connection is established between the first producer node and at least two consumer nodes, and the device comprises: A first sending module, configured to send a message to be consumed to at least one consumer node among the at least two consumer nodes; A first receiving module, configured to respectively receive a feedback message sent by each consumer node in the at least one consumer node for the message to be consumed, wherein each of the feedback messages carries the pressure of the corresponding consumer node Related Information; A load balancing module, configured to perform load balancing according to pressure-related information of some or all of the consumer nodes among the at least two consumer nodes. 14. A message processing device, characterized in that it is applied to a first consumer node, said device comprising: The first receiving module is configured to receive the message to be consumed sent by the first producer node; A message consumption module, configured to consume the message to be consumed; A first sending module, configured to send a feedback message for the message to be consumed to the first producer node, where the feedback message carries pressure-related information of the first consumer node. 15. An electronic device, characterized by comprising a processor, a memory, and a computer program stored on the memory and operable on the processor, when the computer program is executed by the processor, the The steps of the message processing method described in any one of claims 1 to 6, or the steps of the message processing method described in any one of claims 7 to 12. 16. A computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is realized. The steps of the message processing method, or the steps of implementing the message processing method according to any one of claims 7 to 12.

Images