CN113687956A - Message routing distribution method and device, computer equipment and storage medium - Google Patents
Message routing distribution method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN113687956A CN113687956A CN202110839374.2A CN202110839374A CN113687956A CN 113687956 A CN113687956 A CN 113687956A CN 202110839374 A CN202110839374 A CN 202110839374A CN 113687956 A CN113687956 A CN 113687956A
- Authority
- CN
- China
- Prior art keywords
- message
- processed
- sending
- queue
- target system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
Abstract
The invention relates to the technical field of message middleware, and discloses a message routing distribution method, a device, computer equipment and a storage medium, wherein the message routing distribution method comprises the steps of obtaining a message to be processed through a message receiving interface; determining a message middleware matched with the event type according to the event type of the message to be processed; transmitting the message to be processed to the message middleware; determining a message queue and a sending processing mode matched with the event type of the message to be processed according to the configuration management information; the message to be processed is transmitted to the corresponding message queue through the message transmitting interface according to the transmitting and processing mode; and sending the message to be processed to the target system through the message queue. The method can shield the difference among different message middleware by integrating various message middleware, thereby realizing the purpose of providing a uniform message receiving and sending function to the outside. Meanwhile, a message routing and distribution processing mechanism among different services is provided for the service scene.
Description
Technical Field
The present invention relates to the field of message middleware technologies, and in particular, to a message routing distribution method and apparatus, a computer device, and a storage medium.
Background
At present, in distributed application, most message middleware realizes a simple message routing function of a certain message middleware through a complex configuration file. For example, the producer forwards the produced message to the corresponding message queue according to the rule, and then the consumer does simple pull consumption. For various complicated and complicated business and complicated service calling relations of banks, no better reference technology exists at present. Therefore, a unified platform capable of performing routing distribution processing on events among different services to solve the problem of consistency among services is lacking at present.
Disclosure of Invention
Therefore, it is necessary to provide a message routing distribution method, device, computer device, and storage medium for solving the problem of lack of a unified platform that can perform routing distribution processing on events between different services to solve the problem of consistency between services.
A message routing distribution method comprises the steps of obtaining a message to be processed through a message receiving interface; determining a message middleware matched with the event type according to the event type of the message to be processed; the message to be processed is transmitted to the message middleware; determining a message queue and a sending processing mode matched with the event type of the information to be processed according to the configuration management information; the message to be processed is transmitted to the message queue through a message transmitting interface according to the transmitting and processing mode; and sending the message to be processed to a target system through the message queue.
The message routing distribution method provides a simple and easy-to-use uniform message interface for the service system, thereby providing transparent message sending and receiving services. Multiple message middlewares are encapsulated so that messages of different event types can be processed using message queues of different message middlewares. And the event platform determines a message middleware matched with the event type according to the event type of the message to be processed, and transmits the message to be processed to the message middleware. The configuration management information includes configuration information of which message queue the to-be-processed messages of various event types can be transmitted to by which sending and processing mode. The event platform can determine a message queue and a sending processing mode matched with the event type of the message to be processed according to the configuration management information. And the event platform transparently transmits the message to be processed to the corresponding message queue through the message transmitting interface according to the corresponding transmitting and processing mode of the message to be processed. And sending the messages to be processed to the corresponding target systems by using the message queues. The message routing distribution method provided by the invention can shield the difference and complexity of the use of different message middleware by integrating and encapsulating different message middleware, and can provide simple and easy-to-use unified message receiving and sending functions to the outside. Meanwhile, by providing an event routing and distribution processing mechanism among different services, different message middleware is integrated, and messages to be processed with different scenes and different event types can be processed, so that the increasing business and transaction requirements are met.
In one embodiment, the message queue has a preset sending priority, and before the message to be processed is transmitted to the corresponding message queue through a message sending interface according to the sending processing mode, the message queue which is sent preferentially is determined according to the sending priority; the transmitting the to-be-processed message to the corresponding message queue through the message transmitting interface according to the transmitting and processing mode comprises transmitting the to-be-processed message corresponding to the message queue which is preferentially transmitted through the message transmitting interface according to the transmitting and processing mode.
In one embodiment, the message middleware comprises Kafka, RockketMQ, ActiveMQ, RabbitMQ.
In one embodiment, after the to-be-processed message is transmitted to the corresponding message queue through the message transmission interface according to the transmission processing mode, the method further includes classifying and caching the to-be-processed message according to a target system corresponding to the to-be-processed message; when the number of the messages to be processed which are transmitted to the same target system in the classified cache is accumulated to a preset number, combining the messages to be processed which are transmitted to the same target system in the classified cache into a first batch of message sets; and/or merging the messages to be processed which are transmitted to the same target system in the classified cache into a second batch of message sets at preset time intervals; the sending the message to be processed to the target system through the message queue comprises: and sending the first batch message set and/or the second batch message set to a corresponding target system through the message queue.
In one embodiment, after the to-be-processed message is sent to the target system through the message queue, the method further includes recording a sending operation of the to-be-processed message in an event list; and migrating the data in the event list at regular time.
In one embodiment, after the message to be processed is sent to the target system through the message queue, the method further includes receiving a consumption result of the message to be processed, which is fed back by the target system; and when the consumption result fed back by the target system is successful, deleting the to-be-processed message in the message queue, and changing the state of the to-be-processed message in the event detail table into consumed.
In one embodiment, after the to-be-processed message is sent to the target system through the message queue, the method further includes sending the to-be-processed message to the target system through the message queue again when the consumption result is consumption failure.
A message routing and distributing device comprises a message interface module, a message queue module and a message sending module, wherein the message interface module is used for acquiring a message to be processed through a message receiving interface and also used for transmitting the message to be processed to the corresponding message queue through a message sending interface; the message middleware configuration module is used for determining the message middleware matched with the event type according to the event type of the message to be processed; the message to be processed is also used for transmitting the message to be processed to a message queue corresponding to the message middleware; the message queue and the sending processing mode matched with the event type of the message to be processed are determined according to the configuration management information; the message queue module is also used for transmitting the message to be processed to the corresponding message queue through a message sending interface according to the sending processing mode; and the message middleware module is used for sending the message to be processed to a target system through the message queue.
A computer device comprising a memory storing a computer program and a processor implementing the steps of the message routing distribution method of any one of the above when the processor executes the computer program.
A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the message routing distribution method of any one of the above.
Drawings
In order to more clearly illustrate the embodiments of the present specification or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the specification, and other drawings can be obtained by those skilled in the art without inventive labor.
Fig. 1 is a schematic flow chart of a method for message routing and distribution according to an embodiment of the present invention;
fig. 2 is a flowchart illustrating a method for merging and forwarding messages according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating a data migration method according to another embodiment of the present invention;
FIG. 4 is a flowchart illustrating a method of a consumption result verification mechanism according to an embodiment of the present invention;
FIG. 5 is a block diagram of a message routing and distributing apparatus according to an embodiment of the present invention;
fig. 6 is a block diagram of a message routing and distributing apparatus or system according to an embodiment of the present invention.
Detailed Description
To facilitate an understanding of the invention, the invention will now be described more fully with reference to the accompanying drawings. Preferred embodiments of the present invention are shown in the drawings. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
Currently, a single message middleware is usually used to perform storage routing of the service event message, however, the single message middleware may have problems such as incomplete monitoring, disordered messages, lost data and the like.
With the increase of message types and the rapid increase of the number of messages, usage scenarios tend to be diversified, and a single message middleware is gradually difficult to meet message processing requirements. For a complex system, a message processing system capable of coping with different scenarios and processing various types of messages is needed, and a scenario-oriented high-quality message can be provided for a distributed service system, without being limited by performance bottleneck caused by using a message middleware.
Fig. 1 is a schematic flow chart of a method for distributing event routes according to an embodiment of the present invention, where in an embodiment, the method for distributing event routes may include the following steps S100 to S600.
Step S100: and acquiring the message to be processed through a message receiving interface.
The event platform abstracts the events of the business transaction and classifies the events according to different characteristics of different events. Different message middleware may handle different types of events. However, different message middleware has different message receiving interfaces. In the prior art, a simple message routing function of a certain message middleware is mostly realized through a complex configuration file. Some embodiments of the present disclosure may shield differences and complexities in use of different message queues by integrating and encapsulating message receiving interfaces of each message middleware, provide a simple and easy-to-use messaging function to the outside by using a unified message receiving interface and a message relaxing interface, and enable an event platform to enjoy a high-quality messaging service without configuring a complex configuration file. And providing a uniform message receiving interface for the event platform and each application system to acquire the messages to be processed.
Step S200: and determining the message middleware matched with the event type according to the event type of the message to be processed.
Because different message middleware can process different types of events, after the message to be processed is acquired through the uniform message interface, when the event needs to be distributed, the event platform can determine to which message middleware the event to be processed needs to be transmitted according to the event type of the message to be processed.
Step S300: and transmitting the message to be processed to the message middleware.
After determining which message middleware the pending event needs to be transmitted to according to the event type of the pending message, the pending message may be transparently transmitted to the message middleware for buffering. The transparent transmission is transparent to the outside in the transmission process, and is only responsible for transmitting the message to be transmitted to the message queue no matter what message is transmitted, and meanwhile, the transmission quality is ensured, and the transmitted message is not processed.
Step S400: and determining a message queue and a sending processing mode matched with the event type of the message to be processed according to the configuration management information.
Because the message middleware comprises a plurality of message queues and different target systems subscribe different message queues, the target system corresponding to the message to be processed can determine to which message queue the message queue needs to be transmitted for buffering and the sending and processing mode adopted when the message to be processed is transmitted to the message queue. The event platform can determine a message queue corresponding to the message to be processed and a sending and processing mode of the response according to the configuration management information.
In one embodiment, the configuration management information may further include configuration information corresponding to each message middleware. Different message middleware has different configuration information, and most of the prior art realizes the simple message routing function of the message middleware through complex configuration files. The method and the device for the message middleware integration and encapsulation manage the configuration information corresponding to the message middleware while integrating and encapsulating the message middleware. Various information of the message middleware, such as a message policy, can be acquired according to the configuration information.
In some embodiments, if the message queue corresponding to the message middleware needs to send the to-be-processed message in a specific data format, the to-be-processed event may also be subjected to corresponding data processing according to the corresponding configuration management information.
In some embodiments of the present disclosure, the transmission processing manner may include point-to-point transmission, broadcast transmission, routing transmission in groups, and the like. The point-to-point sending is to send the message to a designated queue to wait for consumption of a consumer; the broadcast sending is to send the message to each queue to wait for the consumer to consume; and sending the configuration table of the target system group for searching the message according to the group route, distributing the message and routing the message to a specified target system. The sending processing method of each to-be-processed message stored in the configuration management information may be analyzed in combination with a specific service scenario, and then it is determined which sending processing method the message queue should distribute the message according to.
Meanwhile, each target system is made to monitor the messages belonging to the system. For example, when a broadcast method is used for routing messages, different target systems subscribe to different message queues, and the messages are not sent to a designated location, but different messages to be processed are correspondingly transmitted to different message queues, so that all the target systems are required to monitor the subscribed message queues and determine whether messages belonging to the system exist in the message queues. The target system may dynamically execute the target method.
Step S500: and transmitting the message to be processed to the corresponding message queue through the message transmitting interface according to the transmitting and processing mode.
And transmitting the message to be processed to the corresponding message queue through the message transmitting interface according to the determined sending and processing mode of the message to be processed. Some embodiments of the present disclosure may shield differences and complexities in use of different message queues by performing integration and encapsulation on message sending interfaces of each message middleware, provide a simple and easy-to-use message sending function to the outside by using a unified message sending interface, and enable an event platform to enjoy high-quality messaging service without configuring a complex configuration file. And transmitting the message to be processed to the corresponding message queue by using a uniform message transmitting interface.
By calling the native API (Application Programming Interface) corresponding to the message queue in the unified message Interface. By integrating and packaging the interfaces of each message queue, the difference and complexity of different message queues in use can be shielded, a uniform message sending function is provided for the outside, and a scene-oriented high-quality message sending service is provided for a distributed service system. And the message is distributed to the corresponding message queue through the message sending interface corresponding to the message queue.
Step S600: and sending the message to be processed to the target system through the message queue.
The message routing distribution method provided by the disclosure is compatible with different message middleware products by integrating and packaging various message middleware products, and can shield the difference and complexity in the use of different message queues, thereby realizing the purpose of providing a uniform message receiving and sending function to the outside and providing flexible product support and expansion.
Meanwhile, the event platform also provides a message routing and distribution processing mechanism among different services for the business scene. Various service application scenes are fully considered, service transactions of different service types are abstracted into messages and classified, the functions of receiving, processing and distributing events of different types are realized, messages can be distributed asynchronously in a low coupling mode, and flow peak clipping and peak load shifting flow control are realized.
In one embodiment, the message queue has a preset sending priority, and before the processed message is transmitted to the corresponding message queue through the message sending interface according to the sending processing mode, the message queue which is sent preferentially is determined according to the sending priority. In some embodiments of the present disclosure, the pending messages corresponding to the message queue with higher priority are preferentially sent. For example, when the message queue includes a first message queue and a second message queue, and the priority of the first message queue is higher than that of the second message queue, the pending messages required to be sent to the first message queue may be sent preferentially.
And transmitting the messages to be processed in the message queue to the unified message interface through the message transmitting interface according to the transmitting and processing mode. When the message to be processed is transmitted transparently, a connection pool can be established, and multithreading is started for consumption and transmission transparently. In some embodiments of the present disclosure, a first thread set and a second thread set may be included in the connection pool, the number of threads in the first thread set being greater than the number of threads in the second thread set. The first thread set preferentially consumes the message queue with higher sending priority, namely preferentially sends the messages which need to be transmitted to the first message queue. The first thread set may also be used to send messages in the second message queue after the sending of messages in the first message queue is completed. The second thread set is used only to consume messages in the second message queue.
In some embodiments of the disclosure, the message queue is divided into a normal queue and a VIP queue. The VIP queue has a higher priority than the normal queue. The first thread set is used for sending messages needing to be sent to the VIP queue, and the second thread set is used for sending messages needing to be sent to the consumption common queue. And the first thread set sends the messages corresponding to the VIP queue preferentially, and after the sending of the messages corresponding to the VIP queue is finished, the first thread set automatically sends the messages corresponding to the common queue. Therefore, the waste of resources can be prevented while the preferential consumption of the VIP queue is ensured.
In one embodiment, the encapsulation integration is performed in the message middleware of the four main flows of Kafka, RockketMQ, ActiveMQ and RabbitMQ in the disclosure. Since any machine may face an unknown risk of downtime in a distributed system, the four message middleware Kafka, rockmq, ActiveMQ, RabbitMQ all achieve their high availability through a cluster deployment approach. The message middleware used in the method is relatively mature software, has relatively strong fault-tolerant capability and high reliability.
In one embodiment, before the message to be processed is acquired through the unified message interface, the method may further include updating configuration information of the message middleware. For example, the configuration information may include configuration information of the transmission processing method corresponding to each message queue, and the configuration information of the transmission processing method corresponding to each message queue may be modified after being analyzed in combination with a specific service scenario. The real-time message routing distribution method performs unified management on the configuration information of various message middleware, so that the application end does not need to update the configuration file after the configuration information is updated every time.
And updating the configuration management information in real time according to the corresponding modification requirement of the data in the configuration management information. In some embodiments, the configuration management information may include event query information, event type configuration information, target system configuration information, messaging information. When the method is applied, the information can be set or modified in real time according to the actual application condition. According to the configuration management information, the information such as the target system corresponding to each message, which message middleware each message needs to pass through for routing and the like can be obtained.
The configuration management information may also include event detail history, message transmission history, and the like for storing the operation status. The management of the configuration management information further includes setting the configuration information of each message queue, such as setting the sending mode of the message queue, whether to persist, and other attributes. By updating and setting the configuration management information, necessary management and control can be performed on the whole platform.
In one embodiment, the different event types may include transaction type events, scratch confirmation type events, feedback type events, and instant type events. The event platform abstracts the behaviors of business transaction and the like into messages of different event types, classifies the messages according to the characteristics of the messages, can achieve low coupling, and can realize flow peak clipping and peak shifting flow control by asynchronously distributing the messages. In the message processing procedure, different processing is performed for messages of different event types.
The message routing distribution method provided by the disclosure fully considers different service scenes, and can realize the functions of receiving, processing and distributing messages of different service types under different scenes. A message processing method of a scene high-quality message is provided for a distributed service system, and is not limited by using a message middleware to generate performance bottleneck.
Fig. 2 is a schematic flow chart of a method for merging and forwarding messages according to an embodiment of the present invention, where in an embodiment, after a message to be processed is transmitted to a corresponding message queue through a message sending interface according to a sending processing mode, the method further includes the following steps S510 to S540.
Step S510: and classifying and caching the messages to be processed according to the target system corresponding to the messages to be processed.
And classifying and caching the messages according to the target systems to which the messages need to be sent. And when the data in the classification cache is accumulated to a certain degree, merging the messages sent to the same destination.
Step S520: when the number of the messages to be processed which are transmitted to the same target system in the classified cache is accumulated to a preset number, combining the messages to be processed which are transmitted to the same target system in the classified cache into a first batch of message sets; and/or.
In some embodiments of the present disclosure, messages destined for the same destination may be merged into a first set of bulk messages when all of the target systems accumulate to a preset number of messages for the same target system.
Step S530: and merging the messages to be processed which are transmitted to the same target system in the classified cache into a second batch of message set at preset intervals.
In some embodiments of the present disclosure, all messages in the cache that are addressed to the same destination may also be merged into a second set of bulk messages at preset intervals.
Sending the pending message to the target system through the message queue includes step S540: and sending the first batch message set and/or the second batch message set to the corresponding target system through the message queue.
And after the message combination is completed, sending the first batch of message sets and/or the second batch of message sets to the corresponding target systems. The present disclosure reduces the number of messages by merging messages that need to be sent to the same destination, thereby reducing the processing pressure of the message middleware and the server.
Fig. 3 is a flowchart illustrating a method for data migration according to another embodiment of the present invention, in which in one embodiment, after sending a to-be-processed message to a target system through a message queue, the method further includes the following steps S610 to S620.
Step S610: the sending operation of the message to be processed is recorded in the event list.
And recording the working processes of receiving, processing, sending and the like of the message in an event list. The event list may be used for checking. The event detail table can be used for backing up, tracing and reconciling events, and can be matched with the check and sending of global transaction events in a distributed scene. For example, when the target system does not receive the corresponding message due to the abnormal message transmission, the reason for the abnormal transmission can be determined by checking the message transmission operation record in the event list, so as to perform the abnormal processing in a targeted manner. In some embodiments of the present disclosure, the event detail table may be checked at regular time, and whether there is an abnormal condition such as missing transmission or mistransmission of the message is determined according to the event detail table, and if there is an abnormal condition, the missing transmission or mistransmission message may be retransmitted. By recording the working data, the reliability and safety of the operation can be ensured.
Step S620: and migrating the data in the event list at regular time.
The data recorded in the event list is accumulated all the time, so that the data is excessive, and the system operation is influenced. In order to avoid excessive data due to daily use of the event list, the data can be migrated after being accumulated to a certain degree. In some embodiments of the present disclosure, the amount of data in the event list is guaranteed to be within a controllable range by periodically migrating messages in the event list to the log history. In some other embodiments, after the occupied space of the data in the event list is accumulated to a certain data amount, the message in the event list can be migrated to the record list history, so as to ensure the stable operation of the system platform.
Fig. 4 is a flowchart illustrating a method of a consumption result validation mechanism according to an embodiment of the present invention, where in an embodiment, after sending a to-be-processed message to a target system through a message queue, the method further includes the following steps S630 to S640.
Step S630: and receiving the consumption result of the message to be processed fed back by the target system.
The target system is used as a message queue for caching the messages to be processed after the consumers consume the messages to be processed stored in the message queue subscribed by the consumers, and a consumption result is fed back. The message queue can determine whether the target system is consumed successfully according to the consumption result, so as to judge whether the cached data needs to be cleared.
Step S640: and when the consumption result fed back by the target system is successful, deleting the to-be-processed message in the message queue, and changing the state of the to-be-processed message in the event list into consumed.
And if the consumption result fed back by the target system is successful, deleting the data of the to-be-processed message cached in the message queue, and changing the state of the to-be-processed message in the event list into consumed.
In one embodiment, after the to-be-processed message is sent to the target system subscribed to the message queue through the message queue, the method further includes sending the to-be-processed message to the target system through the unified message interface again when the consumption result fed back by the target system is consumption failure.
And after consuming the to-be-processed messages cached in the message queue, the target system serving as the consumer returns a consumption result to the message queue and the event platform. And if the consumption result is consumption failure, the message queue sends the message to be processed to the unified message interface again, retransmits the stored message to be processed to the target system through the unified message interface, and receives the consumption result returned by the target system again. And when the repeated sending times of the same message to be processed exceed the preset times, the message queue still does not receive the consumption result of successful consumption fed back by the target system, and the message queue does not continue to send the message. And error reporting information can be sent out to remind the operation and maintenance personnel that the message to be processed cannot be sent normally.
The consumption validation mechanism described above may be used to ensure that message data buffered in the message queue is not lost until consumed. And only after the consumption queue receives the confirmation information of successful consumption, the message data cached in the consumption queue is cleared, otherwise, the message data is always stored in the message queue, so that the messages forwarded by the event platform are all consumed by the corresponding target system.
It should be understood that although the various steps in the flowcharts of fig. 1-4 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 1-4 may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed in turn or alternately with other steps or at least some of the other steps or stages.
Based on the above description of the message routing distribution method embodiment, the present disclosure also provides a message routing distribution apparatus. The apparatus may include systems (including distributed systems), software (applications), modules, components, etc. that employ the methods described in embodiments of the specification in conjunction with hardware where necessary to implement the apparatus. Based on the same innovative concept, the embodiments of the present disclosure provide an apparatus in one or more embodiments as described in the following embodiments. Since the implementation scheme of the apparatus for solving the problem is similar to that of the method, the specific implementation of the apparatus in the embodiment of the present specification may refer to the implementation of the foregoing method, and repeated details are not repeated. As used hereinafter, the term "unit" or "module" may be a combination of software and/or hardware that implements a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware, or a combination of software and hardware is also possible and contemplated.
Based on the description of the foregoing method embodiment, the present disclosure also provides a message routing and distributing apparatus. Fig. 5 is a block diagram of a message routing and distributing apparatus according to an embodiment of the present invention, in which the message routing and distributing apparatus may include a message interface module 100, a message middleware configuration module 200, and a message middleware 300.
The message interface module 100 is configured to obtain a to-be-processed message through a message receiving interface, and further configured to transparently transmit the to-be-processed message to a corresponding message queue through a message sending interface. The message middleware configuration module 200 is configured to determine, according to an event type of a message to be processed, a message middleware matching the event type; the message middleware is also used for transmitting the message to be processed to the message middleware; the message queue and the sending processing mode matched with the event type of the message to be processed are determined according to the configuration management information; and the message processing module is also used for transmitting the messages to be processed to the corresponding message queues through the message sending interface according to the sending processing mode. The message middleware module 300 is used for sending the message to be processed to the target system through the message queue.
Different types of messages are cached and routed through different message middleware, the message interface module 100 can provide a simple and easy-to-use message receiving interface and a message sending interface corresponding to the event platform and each target system according to the message middleware corresponding to the message to be processed, and the message to be processed is acquired through the unified message interface. By integrating and encapsulating each message queue, the unified message interface of the message interface module 100 can shield the difference and complexity of different message middleware in use, provide a unified messaging function for the outside, and provide a scene-oriented high-quality message receiving service for a distributed service system.
Message queues corresponding to different message middleware have different configuration management information, and most of the prior art realizes a simple message routing function of the message middleware through a complex configuration file. According to the method, after the message middleware is integrated and packaged, the configuration information of the message middleware is integrated, a unified message receiving and sending function is provided for the outside, and unified storage and management are performed on the integrated configuration information.
The message routing distribution device provided by the disclosure can shield the difference and complexity of different message queues in use by integrating and packaging various message queues, thereby realizing the purpose of providing a uniform message transceiving function to the outside. Meanwhile, the message routing and distributing device can also provide a message routing and distributing processing mechanism among different services for the service scene. Various service application scenes are fully considered, service transactions of different service types are abstracted into messages and classified, the functions of receiving, processing and distributing messages of different types of events are realized, low coupling and asynchronous message distribution can be realized, and meanwhile, flow peak clipping and peak staggering flow control are also realized.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
It is to be understood that the various embodiments of the methods, apparatus, etc. described above are described in a progressive manner, and like/similar elements may be referred to one another, with each embodiment focusing on differences from the other embodiments. Reference may be made to the description of other method embodiments for relevant points.
Fig. 6 is a block diagram illustrating a message routing distribution apparatus or system S00 according to an example embodiment. Referring to FIG. 6, a message routing distribution apparatus or system S00 includes a processing component S20 that further includes one or more processors and memory resources, represented by memory S22, for storing instructions, such as applications, executable by the processing component S20. The application program stored in the memory S22 may include one or more modules each corresponding to a set of instructions. Further, the processing component S20 is configured to execute instructions to perform the above-described method.
The message route distribution apparatus or system S00 may further include: the power supply component S24 is configured to perform power management of the message routing distribution apparatus or system S00, the wired or wireless network interface S26 is configured to connect the message routing distribution apparatus or system S00 to a network, and the input output (I/O) interface S28. The message routing distribution apparatus or system S00 may be operable based on an operating system stored in the memory S22, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, or the like.
In an exemplary embodiment, a computer-readable storage medium is also provided that includes instructions, such as the memory S22 including instructions, that are executable by a processor of the message routing distribution apparatus or system S00 to perform the above-described method. The storage medium may be a computer-readable storage medium, which may be, for example, a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
In an exemplary embodiment, there is also provided a computer program product including instructions executable by a processor of the message routing distribution apparatus or system S00 to perform the above method.
In the description herein, references to the description of "some embodiments," "other embodiments," "desired embodiments," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, a schematic description of the above terminology may not necessarily refer to the same embodiment or example.
The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A message routing distribution method, comprising:
acquiring a message to be processed through a message receiving interface;
determining a message middleware matched with the event type according to the event type of the message to be processed;
the message to be processed is transmitted to the message middleware;
determining a message queue and a sending processing mode matched with the event type of the message to be processed according to the configuration management information;
the message to be processed is transmitted to the corresponding message queue through a message transmitting interface according to the transmitting and processing mode;
and sending the message to be processed to a target system through the message queue.
2. The message routing and distribution method according to claim 1, wherein the message queue has a preset sending priority, and before the message to be processed is transmitted to the corresponding message queue through a message sending interface according to the sending processing mode, the method further comprises:
determining a message queue to be sent preferentially according to the sending priority;
the passing through the message to be processed to the corresponding message queue through the message sending interface according to the sending processing mode includes:
and transmitting the to-be-processed message corresponding to the message queue which is sent preferentially through a message sending interface according to the sending processing mode.
3. The message routing distribution method of claim 1, wherein the message middleware comprises Kafka, rockmq, ActiveMQ, RabbitMQ.
4. The message routing and distribution method according to claim 1, wherein after the to-be-processed message is transmitted to the corresponding message queue through a message transmission interface according to the transmission processing manner, the method further comprises:
classifying and caching the message to be processed according to a target system corresponding to the message to be processed;
when the number of the messages to be processed which are transmitted to the same target system in the classified cache is accumulated to a preset number, combining the messages to be processed which are transmitted to the same target system in the classified cache into a first batch of message sets; and/or
Merging the messages to be processed which are transmitted to the same target system in the classified cache into a second batch of message sets at preset time intervals;
the sending the message to be processed to the target system through the message queue comprises:
and sending the first batch message set and/or the second batch message set to a corresponding target system through the message queue.
5. The message routing and distribution method according to claim 1, wherein after sending the pending message to a target system through the message queue, the method further comprises:
recording the sending operation of the message to be processed in an event list;
and migrating the data in the event list at regular time.
6. The message routing and distribution method according to claim 5, wherein after sending the pending message to the target system through the message queue, the method further comprises:
receiving a consumption result of the message to be processed fed back by the target system;
and when the consumption result fed back by the target system is successful, deleting the to-be-processed message in the message queue, and changing the state of the to-be-processed message in the event detail table into consumed.
7. The message routing and distribution method according to claim 6, wherein after sending the pending message to the target system through the message queue, the method further comprises:
and when the consumption result is consumption failure, the message to be processed is sent to the target system again through the message queue.
8. A message routing distribution apparatus, comprising:
the message interface module is used for acquiring a message to be processed through a message receiving interface and transmitting the message to be processed to the corresponding message queue through a message sending interface;
the message middleware configuration module is used for determining the message middleware matched with the event type according to the event type of the message to be processed; the message middleware is also used for transmitting the message to be processed to the message middleware; the message queue and the sending processing mode matched with the event type of the message to be processed are determined according to the configuration management information; the message queue module is also used for transmitting the message to be processed to the corresponding message queue through a message sending interface according to the sending processing mode;
and the message middleware module is used for sending the message to be processed to a target system through the message queue.
9. A computer device comprising a memory and a processor, the memory storing a computer program, wherein the processor when executing the computer program implements the steps of the message routing distribution method according to any one of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the message routing distribution method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110839374.2A CN113687956A (en) | 2021-07-23 | 2021-07-23 | Message routing distribution method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110839374.2A CN113687956A (en) | 2021-07-23 | 2021-07-23 | Message routing distribution method and device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113687956A true CN113687956A (en) | 2021-11-23 |
Family
ID=78577829
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110839374.2A Pending CN113687956A (en) | 2021-07-23 | 2021-07-23 | Message routing distribution method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113687956A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115086419A (en) * | 2022-07-24 | 2022-09-20 | 浙江大学 | Message mechanism-based cross-three-party system message transmission center system |
| CN115103017A (en) * | 2022-06-13 | 2022-09-23 | 珠海格力电器股份有限公司 | Message middleware processing system and method |
| CN115550280A (en) * | 2022-11-24 | 2022-12-30 | 云账户技术(天津)有限公司 | Multi-level message queue implementation method, system, electronic device and readable storage medium |
-
2021
- 2021-07-23 CN CN202110839374.2A patent/CN113687956A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115103017A (en) * | 2022-06-13 | 2022-09-23 | 珠海格力电器股份有限公司 | Message middleware processing system and method |
| CN115103017B (en) * | 2022-06-13 | 2023-12-08 | 珠海格力电器股份有限公司 | Message middleware processing system and method |
| CN115086419A (en) * | 2022-07-24 | 2022-09-20 | 浙江大学 | Message mechanism-based cross-three-party system message transmission center system |
| CN115550280A (en) * | 2022-11-24 | 2022-12-30 | 云账户技术(天津)有限公司 | Multi-level message queue implementation method, system, electronic device and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107729139B (en) | Method and device for concurrently acquiring resources | |
| CN113687956A (en) | Message routing distribution method and device, computer equipment and storage medium | |
| CN105472042B (en) | The message-oriented middleware system and its data transferring method of WEB terminal control | |
| KR100324976B1 (en) | Data processing apparatus, method and computer program product for carrying out workload management with respect to a group of servers in an asynchronous client/server computing system | |
| CN112882813B (en) | Task scheduling method, device and system and electronic equipment | |
| US20040015368A1 (en) | High availability for asynchronous requests | |
| US20170187672A1 (en) | Message delivery in messaging networks | |
| CN105338086A (en) | Distributed message forwarding method | |
| CN110515748B (en) | Message processing method and related device | |
| US5960178A (en) | Queue system and method for point-to-point message passing having a separate table for storing message state and identifier of processor assigned to process the message | |
| CN110138876B (en) | Task deployment method, device, equipment and platform | |
| WO2021088641A1 (en) | Data transmission method, data processing method, data reception method and device, and storage medium | |
| CN113422842B (en) | Distributed power utilization information data acquisition system considering network load | |
| CN105260842B (en) | Communication method and system between heterogeneous ERP systems | |
| US9641604B1 (en) | Ranking candidate servers in order to select one server for a scheduled data transfer | |
| CN110351366A (en) | A kind of service scheduling method of Internet application, system and computer readable storage medium | |
| CN111414241A (en) | Batch data processing method, device and system, computer equipment and computer readable storage medium | |
| CN113703954A (en) | Message backup method and device, electronic equipment and computer storage medium | |
| CN113703997A (en) | Bidirectional asynchronous communication middleware system integrating multiple message agents and implementation method | |
| CN114710571B (en) | Data packet processing system | |
| CN110138753B (en) | Distributed message service system, method, apparatus, and computer-readable storage medium | |
| CN103685363A (en) | Efficient and reliable method and system for multitask processing | |
| CN113971098A (en) | RabbitMQ consumption management method and system | |
| CN112039985B (en) | Heterogeneous cloud management method and system | |
| CN113630310B (en) | Distributed high-availability gateway system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |