CN116662022A - Distributed message processing method, system, device, communication equipment and storage medium - Google Patents
Distributed message processing method, system, device, communication equipment and storage medium Download PDFInfo
- Publication number
- CN116662022A CN116662022A CN202310961369.8A CN202310961369A CN116662022A CN 116662022 A CN116662022 A CN 116662022A CN 202310961369 A CN202310961369 A CN 202310961369A CN 116662022 A CN116662022 A CN 116662022A
- Authority
- CN
- China
- Prior art keywords
- message
- volume
- node
- list
- nodes
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003860 storage Methods 0.000 title claims abstract description 40
- 238000003672 processing method Methods 0.000 title claims abstract description 34
- 238000004891 communication Methods 0.000 title claims abstract description 26
- 238000012545 processing Methods 0.000 claims abstract description 65
- 238000004519 manufacturing process Methods 0.000 claims abstract description 64
- 230000004044 response Effects 0.000 claims abstract description 21
- 238000004364 calculation method Methods 0.000 claims description 89
- 230000006870 function Effects 0.000 claims description 41
- 238000000034 method Methods 0.000 claims description 22
- 238000005111 flow chemistry technique Methods 0.000 claims description 4
- 238000012544 monitoring process Methods 0.000 claims description 2
- 230000008602 contraction Effects 0.000 abstract description 17
- 238000009826 distribution Methods 0.000 abstract description 5
- 238000010586 diagram Methods 0.000 description 13
- 238000004590 computer program Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000013461 design Methods 0.000 description 4
- 238000012546 transfer Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000003319 supportive effect Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/547—Messaging middleware
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/548—Queue
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer And Data Communications (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The embodiment of the application provides a distributed message processing method, a system, a device, communication equipment and a storage medium, comprising the following steps: receiving a first message sent by a production end and an allocation request corresponding to the first message; in response to the allocation request, querying a registry for a list of message services and a list of message volumes in a non-operational state; and determining a message service node corresponding to the first message according to the message service list, so that the message service node receives the first message and sends the first message to all message volume nodes in the message volume list, so that all message volume nodes store the first message. According to the embodiment of the application, by introducing the independent message volume node, only the storage of the message is responsible, the transverse expansion or contraction can be independently carried out, and the response of the load balancing node to the distribution request of the production end can be used for forwarding the request of the user (the first message produced by the production end) to a message service node for processing in sequence.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a method, a system, an apparatus, a communication device, and a storage medium for processing distributed messages.
Background
Message middleware (message platform) is a supportive software system that provides synchronous or asynchronous, reliable message transmission for application systems in a network environment based on queue and message passing techniques. In short, the message middleware focuses on the sending and receiving of data, and utilizes an efficient and reliable information transfer mechanism to integrate all subsystems in the distributed system.
The current message middleware can meet the basic message transfer function and can also perform main-standby switching under the condition of abnormal main node, but the message service node heavily depends on a local file system for storing the message, and when the message service node receives the message, the message is additionally written on a local disk, so that the storage of the message cannot be independently and transversely expanded, and the storage capacity of the message is limited.
Disclosure of Invention
The embodiment of the application aims to provide a distributed message processing method, a system, a device, communication equipment and a storage medium, which are used for solving the technical problem that the storage of messages in the prior art cannot be independently and transversely expanded, and the storage capacity of the messages is limited. The specific technical scheme is as follows:
In a first aspect of the present application, there is provided a distributed message processing method applied to a load balancing node, the distributed message processing method including:
receiving an allocation request corresponding to a first message sent by a production end;
querying a message service list and a message volume list in a non-working state in a registry in response to the allocation request;
and determining a message service node corresponding to the first message according to the message service list so that the message service node receives the first message and sends the first message to all message volume nodes in the message volume list so that all the message volume nodes store the first message.
Optionally, the determining, according to the message service list, the message service node corresponding to the first message includes:
and determining a message service node corresponding to the first message in the message service list based on a polling access mode.
Optionally, the distributed message processing method further includes:
receiving a message reading request corresponding to a second message sent by a consumer;
responding to the message reading request, and inquiring a message service list and a message volume list in a non-working state in a registry;
And determining a message service node and a message volume node corresponding to the second message according to the message service list and the message volume list, wherein the message volume node comprises the second message corresponding to the message reading request, and the message service node is used for sending the second message to the consumption terminal.
In still another aspect of the present application, there is also provided a distributed message processing method applied to a message service node, the distributed message processing method including:
receiving a first message, wherein the first message is sent to the load balancing node by the production end, the load balancing node queries a message service list and a message volume list which are in a non-working state in a registry according to an allocation request sent by the production end, and determines a message service node corresponding to the first message according to the message service list;
and sending the first message to all message volume nodes in the message volume list so that all the message volume nodes store the first message.
Optionally, after the step of sending the first message to all message volume nodes in the message volume list, so that all the message volume nodes store the first message, the method includes:
Obtaining a target message, wherein the target message is generated by a data stream calculation module according to a function code uploaded by a user to generate a calculation function and carrying out data stream processing on the first message according to the calculation function;
and sending the target message to a consumer.
Optionally, the computation function includes computation rules corresponding to the first message, and the target message is the data flow computation module
And generating a calculation task according to the calculation rule, and starting a thread pool to execute the calculation task, wherein the calculation rule comprises a message service node corresponding to the first message, a message service node corresponding to the target message, and frequency and times corresponding to the calculation task.
In yet another aspect of the present application, there is also provided a distributed message processing system, the system comprising: a message flow cluster comprising a load balancing node, at least one message service node, and at least one message volume node;
the load balancing node is used for receiving an allocation request corresponding to a first message sent by a production end; querying a message service list and a message volume list in a non-working state in a registry in response to the allocation request; determining a message service node corresponding to the first message according to the message service list, so that the message service node receives the first message and sends the first message to all message volume nodes in the message volume list, so that all the message volume nodes store the first message;
The message service node is used for receiving a first message; and sending the first message to all message volume nodes in the message volume list so that all the message volume nodes store the first message.
Optionally, the system further comprises: a production end;
the production end is used for sending a first message to the load balancing nodes in the message flow cluster and an allocation request corresponding to the first message.
Optionally, the system further comprises: a consumer end;
the consumer is configured to send a message read request for a second message to the message flow cluster.
Optionally, the message volume node is configured to store the first message, and the second message.
Optionally, the message volume node includes a disk usage alarm, where the disk usage alarm is configured to monitor usage of a local disk on the node in real time, and report the usage to a registry.
Optionally, the message volume node is further configured to send alarm information to the registry when detecting that the usage rate of the local disk is greater than a preset threshold, so that the registry sends message volume node capacity expansion information to a user according to the alarm information.
Optionally, the message flow cluster further comprises a data flow calculation module;
the data flow calculation module is in communication connection with the message service node;
the data flow calculation module is used for generating a calculation function according to a function code uploaded by a user, acquiring a first message in the message service node, carrying out data flow processing on the first message according to the calculation function, generating a target message, and sending the target message to a consumer through the message service node.
Optionally, the message flow cluster further comprises a registry;
the registry is configured to maintain a message service list and a message volume list, where the message service list includes all message service nodes and states corresponding to the message service nodes, and the message volume list includes all message volume nodes, states corresponding to the message volume nodes, and disk usage.
Optionally, the state includes at least one of: a non-operating state, and an operating state.
Optionally, one of the message service nodes is communicatively connected to a plurality of the message volume nodes.
In still another aspect of the present application, there is also provided a distributed message processing apparatus applied to a load balancing node, the apparatus including:
The receiving module is used for receiving an allocation request corresponding to the first message sent by the production end;
the inquiring module is used for responding to the allocation request and inquiring a message service list and a message volume list which are in a non-working state in a registry;
and the determining module is used for determining the message service node corresponding to the first message according to the message service list so that the message service node receives the first message and sends the first message to all message volume nodes in the message volume list so that all the message volume nodes store the first message.
In yet another aspect of the present application, there is also provided a distributed message processing apparatus for use in a message service node, the apparatus comprising:
the receiving module is used for receiving a first message, wherein the first message is sent to the load balancing node by the production end, the load balancing node inquires a message service list and a message volume list which are in a non-working state in a registry according to an allocation request sent by the production end, and the message service node corresponding to the first message is determined according to the message service list;
And the sending module is used for sending the first message to all message volume nodes in the message volume list so as to enable all the message volume nodes to store the first message.
In yet another aspect of the present application, there is also provided a communication device including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory complete communication with each other through the communication bus;
a memory for storing a computer program;
and the processor is used for realizing any one of the distributed message processing methods when executing the programs stored in the memory.
In yet another aspect of the present application, there is also provided a computer readable storage medium having instructions stored therein which, when run on a computer, cause the computer to perform any of the above described distributed message processing methods.
The distributed message processing method provided by the embodiment of the application is characterized by receiving the distribution request corresponding to the first message sent by the production end; querying a message service list and a message volume list in a non-working state in a registry in response to the allocation request; and determining a message service node corresponding to the first message according to the message service list so that the message service node receives the first message and sends the first message to all message volume nodes in the message volume list so that all the message volume nodes store the first message. The embodiment of the application is only responsible for storing the messages by introducing the independent message volume nodes, can independently perform transverse expansion or contraction, and further separates the calculation and storage of the messages, namely the processing and storage of the messages are not bound in the same 1 node but are performed in 2 nodes (message service nodes and message volume nodes), the message service nodes are used for processing the messages, the message volume nodes are used for storing the messages and mutually independent, can independently perform transverse expansion or contraction without mutual influence, and can realize that the request of a user (the first message produced by the production end) is sequentially forwarded to one message service node for processing through the response of the load balancing node to the allocation request of the production end.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below.
FIG. 1 is a flowchart showing steps of a method for processing distributed messages according to an embodiment of the present application;
FIG. 2 is a flowchart showing a second step of the distributed message processing method according to the embodiment of the present application;
FIG. 3 shows a third flowchart of steps of a distributed message processing method provided by an embodiment of the present application;
FIG. 4 is a flowchart showing steps of a distributed message processing method according to an embodiment of the present application;
FIG. 5 is a schematic diagram of a distributed message processing system according to an embodiment of the present application;
FIG. 6 shows a block diagram of a distributed message processing apparatus according to an embodiment of the present application;
FIG. 7 is a block diagram of another distributed message processing apparatus according to an embodiment of the present application;
fig. 8 is a block diagram of a communication device according to an embodiment of the present application;
fig. 9 is a schematic diagram of a message middleware operation principle in the related art according to an embodiment of the present application;
fig. 10 shows a schematic diagram of message writing in a distributed message processing method according to an embodiment of the present application;
Fig. 11 shows a schematic diagram of data flow calculation in a distributed message processing method according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application.
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the following detailed description of the embodiments of the present application will be given with reference to the accompanying drawings. However, those of ordinary skill in the art will understand that in various embodiments of the present application, numerous technical details have been set forth in order to provide a better understanding of the present application. However, the claimed application may be practiced without these specific details and with various changes and modifications based on the following embodiments. The following embodiments are divided for convenience of description, and should not be construed as limiting the specific implementation of the present application, and the embodiments can be mutually combined and referred to without contradiction.
As shown in fig. 9, fig. 9 illustrates the principle of operation of a common message middleware. Message middleware is generally divided into 3 parts: producer (Producer) consumers (Consumer) and message services (Broker). The producer generates a message and sends it to the messaging service. The message service is generally deployed on a plurality of hosts in a master-slave cluster mode, writing and reading of the message are generally completed through a master node, a slave node is used as a standby node to switch when the master node is down, and the master node can synchronize to the slave node after writing data. Messages are stored on the local disk of the host where the message service resides, and the messages belong to a Topic (Topic) for categorizing the messages. The consumer reads the message from the message service for subsequent business processing. The general message middleware can only send and receive the message, cannot perform real-time streaming calculation when receiving the message, and can also independently deploy other systems if necessary. The current message middleware can meet the basic message transfer function and also can perform active-standby switching under the condition of abnormal master nodes, but has the following problems: the message service Broker is heavily dependent on the local file system for storing messages, and when the Broker receives a message, the Broker additionally writes the message to the local disk. The architecture determines that the storage of the message cannot be subjected to independent transverse expansion, and the storage capacity of the message can be increased only by adding a Broker node at the same time; the single-point bottleneck of reading and writing exists, the reading and writing are carried out through the master node, and the pressure of the master node is high; the real-time streaming calculation of the message cannot be performed in the Broker, and if necessary, additional systems are deployed, thereby increasing additional overhead.
Note that, the message middleware generally includes: a Producer Consumer (Consumer) and a message service (Broker), in which in the message flow cluster, the message service corresponds to a message service node, the Producer corresponds to a Producer side, and the Consumer corresponds to a Consumer side.
Referring to fig. 1, a first step flowchart of a distributed message processing method according to an embodiment of the present application is shown, where the method may include:
step 101, receiving an allocation request corresponding to a first message sent by a production end.
It should be noted that, the embodiment of the present application is applied to a load balancing node, where a load balancing software component is carried on the node, which may be represented as a load balancing device, and a production end is a server or a terminal where a producer is located in a message middleware, and a first message is a message sent by the production end to a message flow cluster in the present application.
The load balancing node is one of the modules in the message flow cluster, which may include a message service node, a message volume node, a load balancing node, a registry, and a data flow calculation module.
The load balancing node is a switching interface for external (producer and consumer), and after receiving a request for sending or obtaining a message, the load balancing node forwards the request to a certain 1 surviving Broker in the cluster for processing, so in the embodiment of the application, the load balancing node receives the allocation request corresponding to the first message sent by the production end, and the load balancing node determines the message service node of the received first message based on the allocation request.
And step 102, responding to the allocation request, and inquiring a message service list and a message volume list in a non-working state in a registry.
It should be noted that, the load balancing node is a switching interface for external (producer and consumer), and after receiving a request for sending or obtaining a message, the load balancing node forwards the request to a certain 1 surviving Broker in the cluster for processing, where the surviving Broker node is determined according to real-time heartbeat detection.
Specifically, two lists, namely a message volume list and a message service list, are recorded in the registry, the Broker list stores the currently surviving Broker node, and the Broker and the registry do real-time heartbeat detection and update the list. The message volume list records the current surviving message volume nodes and disk usage of the nodes, and the message volume nodes update the heartbeat (surviving) and usage in real time.
Step 103, determining a message service node corresponding to the first message according to the message service list, so that the message service node receives the first message, and sending the first message to all message volume nodes in the message volume list, so that all the message volume nodes store the first message.
Further, the determining, according to the message service list, the message service node corresponding to the first message includes: and determining a message service node corresponding to the first message in the message service list based on a polling access mode.
It should be noted that, in the embodiment of the present application, as shown in fig. 10, fig. 10 shows a schematic message writing diagram in a distributed message processing method provided in the embodiment of the present application, a producer sends a message to a message flow platform, a load balancer queries a surviving Broker list and a surviving message volume list from a registry after receiving a request, 1 is selected in a polling manner, the request is forwarded to the Broker, the Broker sends the message to all surviving message volume nodes under the default request, and the message is saved in a multi-copy redundancy manner.
Thus, the load balancing node, after determining the message service node for the received first message, informs the message service node that the first message is stored to all surviving message volume nodes after the message service node receives the first message.
It should be noted that, in the embodiment of the present application, the number of message service nodes and the number of message volume nodes are not limited, and the distributed message architecture in the present application adopts architecture design of separated computation and storage, and the Broker node is only responsible for receiving and sending the message, and is no longer responsible for storing the message, and the message volume node is introduced for storing the message, and the message volume node and the Broker node are independent from each other, and can be respectively and laterally expanded without influence.
The distributed message processing method provided by the embodiment of the application is characterized by receiving the distribution request corresponding to the first message sent by the production end; querying a message service list and a message volume list in a non-working state in a registry in response to the allocation request; and determining a message service node corresponding to the first message according to the message service list so that the message service node receives the first message and sends the first message to all message volume nodes in the message volume list so that all the message volume nodes store the first message. The embodiment of the application is only responsible for storing the messages by introducing the independent message volume nodes, can independently perform transverse expansion or contraction, and further separates the calculation and storage of the messages, namely the processing and storage of the messages are not bound in the same 1 node but are performed in 2 nodes (message service nodes and message volume nodes), the message service nodes are used for processing the messages, the message volume nodes are used for storing the messages and mutually independent, can independently perform transverse expansion or contraction without mutual influence, and can realize that the request of a user (the first message produced by the production end) is sequentially forwarded to one message service node for processing through the response of the load balancing node to the allocation request of the production end.
Referring to fig. 2, a second step flowchart of a distributed message processing method according to an embodiment of the present application is shown, where the method may include:
step 201, receiving an allocation request corresponding to a first message sent by a production end.
Step 202, in response to the allocation request, querying a registry for a list of message services and a list of message volumes in a non-operational state.
Step 203, determining a message service node corresponding to the first message according to the message service list, so that the message service node receives the first message, and sending the first message to all message volume nodes in the message volume list, so that all message volume nodes store the first message.
It should be noted that, in the embodiment of the present application, the steps 201 to 203 are discussed with reference to the foregoing, and are not repeated.
Step 204, receiving a message reading request corresponding to the second message sent by the consumer.
In response to the message reading request, the registry is queried for a list of message services and a list of message volumes in a non-operational state, step 205.
And step 206, determining a message service node and a message volume node corresponding to the second message according to the message service list and the message volume list, wherein the message volume node comprises the second message corresponding to the message reading request, and the message service node is used for sending the second message to the consumer.
It should be noted that, in the embodiment of the present application, the steps 204 to 206 are a process that the consumer obtains the second message from the message flow cluster, where the consumer end is a server or a terminal where the consumer is in the message middleware, and the second message is a message that the consumer end requests to obtain from the message flow cluster in the present application.
Specifically, the flow of the consumer reading the message is similar to the process of writing the message by the producer, the consumer sends a request to the load balancer, the load balancer queries the surviving Broker list and the surviving message volume list, 1 is selected in a polling mode, and forwards the request to the Broker, the Broker finds the 1 st volume node from the surviving message volume list and queries the message, and generally, the result can be queried, and if the result is not queried, the subsequent nodes are queried in turn.
The embodiment of the application only takes charge of storing the information by introducing the independent information volume node, can independently perform transverse expansion or contraction, and further separates the calculation and the storage of the information, namely the processing and the storage of the information are not bound in the same 1 node but are performed in 2 nodes (the information service node and the information volume node), the information service node is used for processing the information, the information volume nodes are used for storing the information, and the information volume nodes are mutually independent and can independently perform transverse expansion or contraction without mutual influence.
In addition, through the embodiment of the application, the producer can send and store the message to the message flow cluster, the consumer can acquire the message from the message flow cluster, and through the response of the load balancing node to the distribution request of the production end, the request of the user (the first message produced by the production end) can be forwarded to a message service node in sequence for processing.
Referring to fig. 3, a third step flowchart of a distributed message processing method according to an embodiment of the present application is shown, where the method may include:
step 301, receiving a first message, where the first message is sent by the production end to the load balancing node, and the load balancing node queries a message service list and a message volume list in a non-working state in a registry according to an allocation request sent by the production end, and determines a message service node corresponding to the first message according to the message service list.
Step 302, sending the first message to all message volume nodes in the message volume list, so that all the message volume nodes store the first message.
It should be noted that, the steps 301-302 are applied to the message service node, and specific reference may be made to the foregoing introduction, and detailed description is omitted herein.
The embodiment of the application only takes charge of storing the information by introducing the independent information volume node, can independently perform transverse expansion or contraction, and further separates the calculation and the storage of the information, namely the processing and the storage of the information are not bound in the same 1 node but are performed in 2 nodes (the information service node and the information volume node), the information service node is used for processing the information, the information volume nodes are used for storing the information and mutually independent, and can independently perform transverse expansion or contraction without mutual influence, and in the application, the response of the load balancing node to the distribution request of the production end can realize that the request of a user (the first information produced by the production end) is sequentially forwarded to one information service node for processing.
Referring to fig. 4, a fourth step flowchart of a distributed message processing method according to an embodiment of the present application is shown, where the method may include:
step 401, receiving a first message, where the first message is sent by the production end to the load balancing node, and the load balancing node queries a message service list and a message volume list in a non-working state in a registry according to an allocation request sent by the production end, and determines a message service node corresponding to the first message according to the message service list.
Step 402, sending the first message to all message volume nodes in the message volume list, so that all the message volume nodes store the first message.
It should be noted that, the steps 401-402 are discussed with reference to the foregoing, and are not repeated herein.
Step 403, obtaining a target message, where the target message is generated by a data flow computing module according to a function code uploaded by a user to generate a computing function, and performing data flow processing on the first message according to the computing function.
Further, the computing function includes a computing rule corresponding to the first message, the target message is generated by the data flow computing module according to the computing rule, and a thread pool is started to execute the computing task, the computing rule includes a message service node corresponding to the first message, a message service node corresponding to the target message, and a frequency and a number of times corresponding to the computing task.
Step 404, the target message is sent to the consumer.
It should be noted that, in the above steps 403-404, the message flow cluster includes a data flow calculation module, which may receive the function code uploaded by the user, generate a calculation task according to the logic set in the code, run the task in the thread pool to calculate the message of the designated Topic, and generate the message to the new Topic.
Wherein the message is stored on a local disk of a host where the message service is located, the message belongs to a Topic (Topic) for classifying the message.
The user self-defines a plurality of function codes, wherein the function codes comprise the calculation logic of the message (which Topic reads the message from which Topic the generated message belongs to, and the frequency and the times of the calculation tasks) and the message is uploaded to a message flow platform or is deployed along with the message flow platform, and after the function codes are received, a data flow calculator generates the calculation tasks according to the calculation logic and starts a thread pool to execute the calculation tasks.
Further, in the embodiment of the present application, in the form of calculation supporting streaming, as shown in fig. 11, fig. 11 shows a schematic diagram of data flow calculation in the distributed message processing method provided in the embodiment of the present application, that is, the last output message may be used as input to participate in the next calculation, 2 functions are defined by the user, 2 calculation tasks are generated, the task 1 reads the messages of the Topic1 and the Topic2, a new Topic3 message is generated after processing according to the logic calculation of the user, and the message and the original message of the Topic2 participate in the calculation of the task 2 again, so as to generate a new Topic4 message again.
The embodiment of the application is only responsible for storing the messages by introducing the independent message volume nodes, can independently perform transverse expansion or contraction, and further separates the calculation and storage of the messages, namely the processing and storage of the messages are not bound in the same 1 node but are performed in 2 nodes (message service nodes and message volume nodes), the message service nodes are used for processing the messages, the message volume nodes are used for storing the messages and mutually independent, can independently perform transverse expansion or contraction without mutual influence, and can realize that the request of a user (the first message produced by the production end) is sequentially forwarded to one message service node for processing through the response of the load balancing node to the allocation request of the production end.
In addition, by introducing a data flow calculator, the cluster not only has the function of transmitting the message, but also has the function of calculating the flow of the message, and a set of system is deployed, so that the transmission and calculation of the message can be completed at the same time.
Referring to fig. 5, fig. 5 shows a schematic diagram of a distributed message processing system according to an embodiment of the present application, where the system includes: a message flow cluster comprising a load balancing node, at least one message service node, and at least one message volume node;
The load balancing node is used for receiving an allocation request corresponding to a first message sent by a production end; querying a message service list and a message volume list in a non-working state in a registry in response to the allocation request; determining a message service node corresponding to the first message according to the message service list, so that the message service node receives the first message and sends the first message to all message volume nodes in the message volume list, so that all the message volume nodes store the first message;
the message service node is used for receiving a first message; and sending the first message to all message volume nodes in the message volume list so that all the message volume nodes store the first message.
It should be noted that, in the embodiment of the present application, as shown in fig. 5, the distributed message processing system includes a production end, a consumption end, and a message flow cluster, where the message flow cluster includes a load balancing node, at least one message service node, at least one message volume node, a data flow calculation module, and a registry.
Further, one of the message service nodes is communicatively coupled to a plurality of the message volume nodes.
The architecture design of separating calculation and storage is adopted, the Broker node is only responsible for receiving and sending the message, is not responsible for storing the message any more, and the message volume node is introduced for storing the message, and is mutually independent, can be respectively and transversely expanded and has no influence on each other, such as: the message read-write pressure is smaller, but the data volume is larger, the data volume node can be increased independently, and the Broker node is kept unchanged. The master-slave architecture design of a Broker is canceled, a centerless cluster design is changed, a load balancer is introduced, a producer sends a message to the load balancer, the balancer forwards the message to a certain Broker, the Broker sends the message to each message volume node for storage, a data stream calculator is introduced, the messages of certain Topic can be read according to a user-defined calculation function and rule, and new Topic messages are produced after calculation and stored in the message volume nodes.
Further, the system further comprises: a production end;
the production end is used for sending a first message to the load balancing nodes in the message flow cluster and an allocation request corresponding to the first message.
Further, the system further comprises: a consumer end;
The consumer is configured to send a message read request for a second message to the message flow cluster.
Further, the message volume node is configured to store the first message, and a second message.
The message volume node comprises a disk usage alarm which is used for monitoring the usage of the local disk on the node in real time and reporting the usage to a registry.
Further, the message volume node is further configured to send alarm information to the registry when it is detected that the usage rate of the local disk is greater than a preset threshold, so that the registry sends message volume node capacity expansion information to a user according to the alarm information.
It should be noted that, in the embodiment of the present application, the message volume node is a node only used for storing messages, and uses a local disk to store messages, which are independent from the Broker node. Each message volume node is internally provided with a disk utilization rate alarm, the utilization rate of a local disk can be monitored in real time and reported to a registry, when the utilization rate exceeds a threshold value, an alarm is sent to the registry, and a user can expand the volume of the message volume node and transfer data according to the alarm.
Specifically, when a disk with higher usage rate is detected, an alarm is given to the registry, and the user can check the usage rate of the disk in each node currently according to the registry, for example, when the writing information of the node 2 in the message volume nodes 1, 2 and 3 is too much, the user can expand the capacity of the node 2 after sensing, or increase the message volume node 4, and redistribute the data stored in the original node 2.
Further, the message flow cluster further comprises a data flow calculation module;
the data flow calculation module is in communication connection with the message service node;
the data flow calculation module is used for generating a calculation function according to a function code uploaded by a user, acquiring a first message in the message service node, carrying out data flow processing on the first message according to the calculation function, generating a target message, and sending the target message to a consumer through the message service node.
It should be noted that, the message flow cluster includes a data flow calculation module, which can receive the function code uploaded by the user, generate a calculation task according to the logic set in the code, run the task in the thread pool to calculate the message of the designated Topic, and generate the message to the new Topic.
Wherein the message is stored on a local disk of a host where the message service is located, the message belongs to a Topic (Topic) for classifying the message.
The user self-defines a plurality of function codes, wherein the function codes comprise the calculation logic of the message (which Topic reads the message from which Topic the generated message belongs to, and the frequency and the times of the calculation tasks) and the message is uploaded to a message flow platform or is deployed along with the message flow platform, and after the function codes are received, a data flow calculator generates the calculation tasks according to the calculation logic and starts a thread pool to execute the calculation tasks.
Further, in the embodiment of the present application, in the form of calculation supporting streaming, as shown in fig. 11, fig. 11 shows a schematic diagram of data flow calculation in the distributed message processing method provided in the embodiment of the present application, that is, the last output message may be used as input to participate in the next calculation, 2 functions are defined by the user, 2 calculation tasks are generated, the task 1 reads the messages of the Topic1 and the Topic2, a new Topic3 message is generated after processing according to the logic calculation of the user, and the message and the original message of the Topic2 participate in the calculation of the task 2 again, so as to generate a new Topic4 message again.
Further, the data stream calculation module stores a plurality of theme files and calculation tasks;
when the computing tasks comprise a first computing task and a second computing task, the data stream computing module comprises a first theme file, a second theme file, a third theme file and a fourth theme file, wherein the first theme file comprises commodity order information sent by a production end, the second theme file comprises payment order information sent by the production end, the third theme file comprises customer point information, and the fourth theme file comprises coupon information.
The customer point information is generated by the data flow calculation module through a first calculation task according to the commodity order information and the payment order information;
the coupon message is generated by the data flow computing module performing a second computing task based on the customer loyalty message.
It should be noted that, the theme file is Topic, and the calculation task is generated by analyzing the calculation rule according to the code uploaded by the user.
Therefore, the data stream computing module may store a plurality of theme files and computing tasks, and when the computing tasks include the first computing task and the second computing task, the data stream computing module may include Topic1, topic2, topic3, and Topic4.
Therefore, the data flow calculation module can be used for receiving commodity order information sent by the production end and payment order information corresponding to the commodity order information; judging whether the payment amount is larger than a target payment threshold value according to the payment order information; if the payment amount is larger than the target point, determining a target point according to the payment amount; determining a client number according to the payment serial number in the payment order message; generating a client point message according to the client number, the target point and the payment serial number, and storing the client point message in a third theme file in a data stream calculation module; and generating a coupon message according to the client point message and the payment order message, and storing the coupon message and a fourth theme file in the data stream calculation module.
Specifically, the user uploads 2 real-time computing functions to a data stream computing module in the distributed message processing system, the data stream computing module generates 2 real-time computing tasks according to the function content, the first computing task can be represented as task 1, task 1 is used for computing the integral of the client, the second computing task can be represented as task 2, and task 2 is used for computing the coupon of the client.
The production end generates commodity order information (Topic 1) and corresponding payment order information (Topic 2) and sends the commodity order information and the payment order information to the information flow platform, the task 1 reads the order information and the payment information in real time and carries out real-time calculation, and the calculation rule corresponding to the first calculation task comprises: for each payment message, if the payment is successful and the payment amount is greater than 10, the user's credit is increased (e.g., credit pts equals the amount divided by 10), then customer Id (cusId) is found from the payment serial number (payId) traversal order message, and the customer credit message is generated along with the credit (pts) and the payment serial number (payId) and saved to Topic3. Task 2 reads the customer loyalty message (Topic 3) and the payment order message (Topic 2) in real time, calculates and generates a coupon message, and saves the coupon message to Topic4. The calculation function (function 2) is as follows: for each credit message, if the single credit exceeds 100, a payment channel (channel) is found by traversing the payment message according to a payment serial number (payId), if the channel is a 'stored value number', coupons are added by dividing the credit by 100 (1 coupon is added per 100 credits), and the amount of each coupon is fixed to 10.
Further, the message flow cluster also includes a registry;
The registry is configured to maintain a message service list and a message volume list, where the message service list includes all message service nodes and states corresponding to the message service nodes, and the message volume list includes all message volume nodes, states corresponding to the message volume nodes, and disk usage.
The state includes at least one of: a non-operating state, and an operating state.
It should be noted that, in the embodiment of the present application, 2 lists are maintained in the registry, and the Broker list stores the currently surviving Broker nodes, and the Broker and the registry do real-time heartbeat detection and update the list. The message volume list records the current surviving message volume nodes and disk usage of the nodes, and the message volume nodes update the heartbeat (surviving) and usage in real time.
The node is in a non-working state, namely a node which is alive at present, and the node is in a working state, namely a node which is occupied at present or the disk utilization rate exceeds a preset threshold value.
The embodiment of the application is only responsible for storing the messages by introducing the independent message volume nodes, can independently perform transverse expansion or contraction, and further separates the calculation and storage of the messages, namely the processing and storage of the messages are not bound in the same 1 node but are performed in 2 nodes (message service nodes and message volume nodes), the message service nodes are used for processing the messages, the message volume nodes are used for storing the messages and mutually independent, can independently perform transverse expansion or contraction without mutual influence, and can realize that the request of a user (the first message produced by the production end) is sequentially forwarded to one message service node for processing through the response of the load balancing node to the allocation request of the production end.
In addition, the application solves the single-point pressure problem of the original master Broker node by arranging the distributed message processing system, namely designing a framework with separated computation and storage, wherein the computation and the storage are mutually independent and can be independently expanded or contracted transversely without mutual influence. The streaming computation can perform the computation processing again on the original message, and a set of platform is deployed, so that the real-time transmission and streaming computation of the message can be completed simultaneously.
Referring to fig. 6, fig. 6 shows a schematic diagram of a distributed message processing system according to an embodiment of the present application, where the distributed message processing system is applied to a load balancing node, and the apparatus includes:
a receiving module 601, configured to receive an allocation request corresponding to a first message sent by a production end;
a query module 602, configured to query a registry for a list of message services and a list of message volumes in a non-working state in response to the allocation request;
and the determining module 603 is configured to determine, according to the message service list, a message service node corresponding to the first message, so that the message service node receives the first message, and send the first message to all message volume nodes in the message volume list, so that all message volume nodes store the first message.
The embodiment of the application is only responsible for storing the messages by introducing the independent message volume nodes, can independently perform transverse expansion or contraction, and further separates the calculation and storage of the messages, namely the processing and storage of the messages are not bound in the same 1 node but are performed in 2 nodes (message service nodes and message volume nodes), the message service nodes are used for processing the messages, the message volume nodes are used for storing the messages and mutually independent, can independently perform transverse expansion or contraction without mutual influence, and can realize that the request of a user (the first message produced by the production end) is sequentially forwarded to one message service node for processing through the response of the load balancing node to the allocation request of the production end.
Referring to fig. 7, fig. 7 shows a schematic diagram of another distributed message processing system provided by an embodiment of the present application, applied to a message service node, where the apparatus includes:
a receiving module 701, configured to receive a first message, where the first message is sent by the production end to the load balancing node, and the load balancing node queries a message service list and a message volume list in a non-working state in a registry according to an allocation request sent by the production end, and determines a message service node corresponding to the first message according to the message service list;
A sending module 702, configured to send the first message to all message volume nodes in the message volume list, so that all the message volume nodes store the first message.
The embodiment of the application is only responsible for storing the messages by introducing the independent message volume nodes, can independently perform transverse expansion or contraction, and further separates the calculation and storage of the messages, namely the processing and storage of the messages are not bound in the same 1 node but are performed in 2 nodes (message service nodes and message volume nodes), the message service nodes are used for processing the messages, the message volume nodes are used for storing the messages and mutually independent, can independently perform transverse expansion or contraction without mutual influence, and can realize that the request of a user (the first message produced by the production end) is sequentially forwarded to one message service node for processing through the response of the load balancing node to the allocation request of the production end.
The embodiment of the present application also provides a communication device, as shown in fig. 8, including a processor 801, a communication interface 802, a memory 803, and a communication bus 804, where the processor 801, the communication interface 802, and the memory 803 complete communication with each other through the communication bus 804,
A memory 803 for storing a computer program;
the processor 801, when executing the program stored in the memory 803, may implement the following steps:
receiving an allocation request corresponding to a first message sent by a production end;
querying a message service list and a message volume list in a non-working state in a registry in response to the allocation request;
and determining a message service node corresponding to the first message according to the message service list so that the message service node receives the first message and sends the first message to all message volume nodes in the message volume list so that all the message volume nodes store the first message.
Or, receiving a first message, wherein the first message is sent to the load balancing node by the production end, and the load balancing node queries a message service list and a message volume list in a non-working state in a registry according to an allocation request sent by the production end, and determines a message service node corresponding to the first message according to the message service list;
and sending the first message to all message volume nodes in the message volume list so that all the message volume nodes store the first message.
The communication bus mentioned by the above terminal may be a peripheral component interconnect standard (Peripheral Component Interconnect, abbreviated as PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, abbreviated as EISA) bus, etc. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.
The communication interface is used for communication between the terminal and other devices.
The memory may include random access memory (Random Access Memory, RAM) or non-volatile memory (non-volatile memory), such as at least one disk memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.
The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), etc.; but also digital signal processors (Digital Signal Processing, DSP for short), application specific integrated circuits (Application Specific Integrated Circuit, ASIC for short), field-programmable gate arrays (Field-Programmable Gate Array, FPGA for short) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
In yet another embodiment of the present application, there is also provided a computer readable storage medium having instructions stored therein that, when run on a computer, cause the computer to perform congestion control as described in any of the above embodiments.
In yet another embodiment of the present application, there is also provided a computer program product containing instructions that, when run on a computer, cause the computer to perform congestion control as described in any of the above embodiments.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present application, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or third database to another website, computer, server, or third database by a wired (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device including one or more servers, third databases, etc. that can be integrated with the available medium. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), etc.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.
The foregoing description is only of the preferred embodiments of the present application and is not intended to limit the scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.
Claims (20)
1. A distributed message processing method, applied to a load balancing node, the distributed message processing method comprising:
receiving an allocation request corresponding to a first message sent by a production end;
querying a message service list and a message volume list in a non-working state in a registry in response to the allocation request;
and determining a message service node corresponding to the first message according to the message service list so that the message service node receives the first message and sends the first message to all message volume nodes in the message volume list so that all the message volume nodes store the first message.
2. The method according to claim 1, wherein determining a message service node corresponding to the first message according to the message service list comprises:
and determining a message service node corresponding to the first message in the message service list based on a polling access mode.
3. The distributed message processing method of claim 1, further comprising:
receiving a message reading request corresponding to a second message sent by a consumer;
responding to the message reading request, and inquiring a message service list and a message volume list in a non-working state in a registry;
and determining a message service node and a message volume node corresponding to the second message according to the message service list and the message volume list, wherein the message volume node comprises the second message corresponding to the message reading request, and the message service node is used for sending the second message to the consumption terminal.
4. A distributed message processing method, applied to a message service node, the distributed message processing method comprising:
receiving a first message, wherein the first message is sent to a load balancing node by a production end, the load balancing node queries a message service list and a message volume list which are in a non-working state in a registry according to an allocation request sent by the production end, and determines a message service node corresponding to the first message according to the message service list;
And sending the first message to all message volume nodes in the message volume list so that all the message volume nodes store the first message.
5. The method of distributed message processing according to claim 4, wherein after the step of sending the first message to all of the message volume nodes in the message volume list to cause all of the message volume nodes to store the first message, the method comprises:
obtaining a target message, wherein the target message is generated by a data stream calculation module according to a function code uploaded by a user to generate a calculation function and carrying out data stream processing on the first message according to the calculation function;
and sending the target message to a consumer.
6. The method of claim 5, wherein the computation function includes computation rules corresponding to the first message, and the target message is the data flow computation module
And generating a calculation task according to the calculation rule, and starting a thread pool to execute the calculation task, wherein the calculation rule comprises a message service node corresponding to the first message, a message service node corresponding to the target message, and frequency and times corresponding to the calculation task.
7. A distributed message processing system, the system comprising: a message flow cluster comprising a load balancing node, at least one message service node, and at least one message volume node;
the load balancing node is used for receiving an allocation request corresponding to a first message sent by a production end; querying a message service list and a message volume list in a non-working state in a registry in response to the allocation request; determining a message service node corresponding to the first message according to the message service list, so that the message service node receives the first message and sends the first message to all message volume nodes in the message volume list, so that all the message volume nodes store the first message;
the message service node is used for receiving a first message; and sending the first message to all message volume nodes in the message volume list so that all the message volume nodes store the first message.
8. The distributed message processing system of claim 7, wherein the system further comprises: a production end;
the production end is used for sending a first message to the load balancing nodes in the message flow cluster and an allocation request corresponding to the first message.
9. The distributed message processing system of claim 7, wherein the system further comprises: a consumer end;
the consumer is configured to send a message read request for a second message to the message flow cluster.
10. The distributed message processing system of claim 7, wherein the message volume node is configured to store the first message and a second message.
11. The distributed message processing system of claim 10, wherein the message volume node includes a disk usage alert for monitoring usage of a local disk on the node in real time and reporting the usage to a registry.
12. The distributed message processing system of claim 11, wherein the message volume node is further configured to send alert information to the registry upon detecting that the local disk usage is greater than a preset threshold, such that the registry sends message volume node expansion information to a user based on the alert information.
13. The distributed message processing system of claim 7, wherein the message flow cluster further comprises a data flow calculation module;
The data flow calculation module is in communication connection with the message service node;
the data flow calculation module is used for generating a calculation function according to a function code uploaded by a user, acquiring a first message in the message service node, carrying out data flow processing on the first message according to the calculation function, generating a target message, and sending the target message to a consumer through the message service node.
14. The distributed message processing system of claim 13, wherein the data stream computing module has a plurality of theme files and computing tasks stored therein;
when the computing tasks comprise a first computing task and a second computing task, the data stream computing module comprises a first theme file, a second theme file, a third theme file and a fourth theme file, wherein the first theme file comprises commodity order information sent by a production end, the second theme file comprises payment order information sent by the production end, the third theme file comprises customer point information, and the fourth theme file comprises coupon information.
15. The distributed message processing system of claim 14, wherein the customer loyalty message is generated by the data flow calculation module performing a first calculation task from the merchandise order message and the payment order message;
The coupon message is generated by the data flow computing module performing a second computing task based on the customer loyalty message.
16. The distributed message processing system of claim 7, wherein the message flow cluster further comprises a registry;
the registry is configured to maintain a message service list and a message volume list, where the message service list includes all message service nodes and states corresponding to the message service nodes, and the message volume list includes all message volume nodes, states corresponding to the message volume nodes, and disk usage.
17. A distributed message processing apparatus for use with a load balancing node, the apparatus comprising:
the receiving module is used for receiving an allocation request corresponding to the first message sent by the production end;
the inquiring module is used for responding to the allocation request and inquiring a message service list and a message volume list which are in a non-working state in a registry;
and the determining module is used for determining the message service node corresponding to the first message according to the message service list so that the message service node receives the first message and sends the first message to all message volume nodes in the message volume list so that all the message volume nodes store the first message.
18. A distributed message processing apparatus for use with a message service node, the apparatus comprising:
the receiving module is used for receiving a first message, wherein the first message is sent to a load balancing node by a production end, the load balancing node inquires a message service list and a message volume list which are in a non-working state in a registry according to an allocation request sent by the production end, and the message service node corresponding to the first message is determined according to the message service list;
and the sending module is used for sending the first message to all message volume nodes in the message volume list so as to enable all the message volume nodes to store the first message.
19. A communication device, comprising: a transceiver, a memory, a processor, and a program stored on the memory and executable on the processor;
the processor is configured to read a program in the memory to implement the distributed message processing method according to any one of claims 1 to 3 or to implement the distributed message processing method according to any one of claims 4 to 6.
20. A readable storage medium storing a program, wherein the program when executed by a processor implements the distributed message processing method according to any one of claims 1-3 or the distributed message processing method according to any one of claims 4-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310961369.8A CN116662022B (en) | 2023-08-02 | 2023-08-02 | Distributed message processing method, system, device, communication equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310961369.8A CN116662022B (en) | 2023-08-02 | 2023-08-02 | Distributed message processing method, system, device, communication equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116662022A true CN116662022A (en) | 2023-08-29 |
| CN116662022B CN116662022B (en) | 2024-02-20 |
Family
ID=87710416
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310961369.8A Active CN116662022B (en) | 2023-08-02 | 2023-08-02 | Distributed message processing method, system, device, communication equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116662022B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116909779A (en) * | 2023-09-11 | 2023-10-20 | 中移(苏州)软件技术有限公司 | Message processing method and device and related equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103281384A (en) * | 2013-05-31 | 2013-09-04 | 汉柏科技有限公司 | Management system and method for message service cluster without center node |
| CN108628688A (en) * | 2018-03-30 | 2018-10-09 | 阿里巴巴集团控股有限公司 | A kind of message treatment method, device and equipment |
| CN114827145A (en) * | 2022-04-24 | 2022-07-29 | 阿里巴巴（中国）有限公司 | Server cluster system, and metadata access method and device |
| CN115022318A (en) * | 2022-05-30 | 2022-09-06 | 在线途游（北京）科技有限公司 | Message publishing method and system based on micro-service architecture |
| CN115185705A (en) * | 2021-04-06 | 2022-10-14 | 腾讯科技（深圳）有限公司 | Message notification method, device, medium and equipment |
-
2023
- 2023-08-02 CN CN202310961369.8A patent/CN116662022B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103281384A (en) * | 2013-05-31 | 2013-09-04 | 汉柏科技有限公司 | Management system and method for message service cluster without center node |
| CN108628688A (en) * | 2018-03-30 | 2018-10-09 | 阿里巴巴集团控股有限公司 | A kind of message treatment method, device and equipment |
| CN115185705A (en) * | 2021-04-06 | 2022-10-14 | 腾讯科技（深圳）有限公司 | Message notification method, device, medium and equipment |
| CN114827145A (en) * | 2022-04-24 | 2022-07-29 | 阿里巴巴（中国）有限公司 | Server cluster system, and metadata access method and device |
| CN115022318A (en) * | 2022-05-30 | 2022-09-06 | 在线途游（北京）科技有限公司 | Message publishing method and system based on micro-service architecture |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116909779A (en) * | 2023-09-11 | 2023-10-20 | 中移(苏州)软件技术有限公司 | Message processing method and device and related equipment |
| CN116909779B (en) * | 2023-09-11 | 2024-01-26 | 中移(苏州)软件技术有限公司 | Message processing method and device and related equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116662022B (en) | 2024-02-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2017167044A1 (en) | Distributed cluster training method and device | |
| KR101871383B1 (en) | Method and system for using a recursive event listener on a node in hierarchical data structure | |
| CN116662022B (en) | Distributed message processing method, system, device, communication equipment and storage medium | |
| CN111966289B (en) | Partition optimization method and system based on Kafka cluster | |
| CN110928851B (en) | Method, device and equipment for processing log information and storage medium | |
| KR101908377B1 (en) | Meter data management system and computer readable recording medium to member deterioration | |
| CN111049928B (en) | Data synchronization method, system, electronic device and computer readable storage medium | |
| CN110187995B (en) | Method for fusing opposite end node and fusing device | |
| CN111625592A (en) | Load balancing method and device for distributed database | |
| CN113067875A (en) | Access method, device and equipment based on dynamic flow control of micro-service gateway | |
| CN108932241B (en) | Log data statistical method, device and node | |
| CN113254245A (en) | Fault detection method and system for storage cluster | |
| CN111538772A (en) | Data exchange processing method and device, electronic equipment and storage medium | |
| CN111884932B (en) | Link determining method, device, equipment and computer readable storage medium | |
| CN113037420B (en) | Reading time stamp obtaining method and device, electronic equipment and storage medium | |
| CN113094397B (en) | Data storage and query response method and block execution method | |
| CN112131198B (en) | Log analysis method and device and electronic equipment | |
| CN109254880A (en) | A kind of method and device handling database delay machine | |
| CN114443415A (en) | Acquisition automatic balancing method, task distributor and system for Prometheus | |
| CN115114316A (en) | Processing method, device, cluster and storage medium for high-concurrency data | |
| CN109151016B (en) | Flow forwarding method and device, service system, computing device and storage medium | |
| CN109510855B (en) | Event distribution system, method and device | |
| CN112765277A (en) | Data synchronization method, device and system | |
| CN112215527A (en) | Logistics management method and device | |
| CN112543213B (en) | Data processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |