CN112579274A - Multi-channel access message forwarding method and device - Google Patents
Multi-channel access message forwarding method and device Download PDFInfo
- Publication number
- CN112579274A CN112579274A CN202011484464.6A CN202011484464A CN112579274A CN 112579274 A CN112579274 A CN 112579274A CN 202011484464 A CN202011484464 A CN 202011484464A CN 112579274 A CN112579274 A CN 112579274A
- Authority
- CN
- China
- Prior art keywords
- message
- sending
- event message
- queue
- forwarding
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 78
- 230000007246 mechanism Effects 0.000 claims abstract description 62
- 230000002093 peripheral effect Effects 0.000 claims abstract description 40
- 230000008569 process Effects 0.000 claims abstract description 35
- 230000003139 buffering effect Effects 0.000 claims abstract description 27
- 230000005540 biological transmission Effects 0.000 claims abstract description 13
- 238000012545 processing Methods 0.000 claims description 26
- 238000004590 computer program Methods 0.000 claims description 16
- 238000003860 storage Methods 0.000 claims description 9
- 239000003999 initiator Substances 0.000 claims description 7
- 238000010586 diagram Methods 0.000 description 17
- 238000012790 confirmation Methods 0.000 description 11
- 230000008878 coupling Effects 0.000 description 9
- 238000010168 coupling process Methods 0.000 description 9
- 238000005859 coupling reaction Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 8
- 238000011161 development Methods 0.000 description 6
- 230000001360 synchronised effect Effects 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 230000003247 decreasing effect Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013467 fragmentation Methods 0.000 description 1
- 238000006062 fragmentation reaction Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002688 persistence Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention provides a method and a device for forwarding a multi-channel access message, wherein the method comprises the following steps: configuring a forwarding parameter; establishing a message queue through a platform scheduling task; wherein, the message queue includes: a waiting queue, an error queue and a buffering queue; acquiring an event message generated when a local transaction is executed by using a scheduling task; matching the event message with the forwarding parameter, and respectively putting the event message into a waiting queue and a buffering queue according to the service limit of the event message after the matching is successful; reading the event message in the buffer queue through a message forwarding service, and configuring a sending sequence for the event message by using a strong sequence guarantee mechanism; sending the event messages to a peripheral system according to a sending buffer mechanism and dynamically adjusting a sending quota in the sending process; when the transmission to the peripheral system fails, the event message is put into an error queue.
Description
Technical Field
The invention relates to the technical field of computer data processing, in particular to a method and a device for forwarding a multi-channel access message.
Background
This section is intended to provide a background or context to the embodiments of the invention that are recited in the claims. The description herein is not admitted to be prior art by inclusion in this section.
With the development of society and economy, an IT software system needs to be subjected to modular splitting inevitably in order to meet the increasing business requirements and guarantee maintainability, the modules are more and more mutually interacted and the scene is more and more complex as the modules are more and more finely split, each module is subjected to corresponding logic processing, and the requirements on the prepositive and postpositive conditions are more rigorous than before.
On the whole, the call of the multiple services involves multiple data sources, which can be regarded as a special distributed transaction, but the requirement on consistency is higher than that of the distributed transaction, the business logic processing of each module is very complex, and the subsequent business process can be determined only after the result of the previous step is definitely obtained. The current popular distributed architecture is basically designed by taking final consistency as an ultimate target, and does not meet the requirement that each link needs consistency.
One service relates to a plurality of environments, each link needs to process complex service logic, the implementation modes of the system are different, time consumption, cache and exception handling mechanisms are different, and in order to ensure the continuity of the service, the service transfer system generally needs to be compatible with various complex and unpredictable scenes, and sequential forwarding, a cache mechanism and an error retransmission mechanism need to be implemented.
In IT practice in banking industry, the importance degree of accounting data consistency often exceeds other non-functional characteristics, and under the SOA architecture, a "store and forward" mechanism is generally adopted at present. The mechanism starts from a transaction initiator, messages are sent in a chain mode according to a business process, a plurality of services of different modules are sequentially pushed, and after the last message is pushed successfully, the whole business process is completed. If the status of any one message returned result is unknown or failed after a certain time, in order to ensure the overall data consistency, it is necessary to initiate correction to each pushed message, and whether the last execution is successful or not, the pushed message needs to be cancelled, and the main flow and the execution sequence are shown in fig. 2.
Under the message synchronous calling mechanism, the push message needs to synchronously wait for receiving the service and return a message processing result, although the realization is simpler and the execution sequence is also ensured, the defects are also obvious:
1. when the call chain is long, the response time cannot be guaranteed by the synchronous waiting mechanism.
2. The synchronous waiting occupies more system resources, so that the application system cannot support high-concurrency scenes.
In terms of data consistency, the hedging mechanism, while providing a reverse revocation function in case of a party failure, does not provide a query or automatic retry function for the case where the result of the service party processing the message is unknown.
Therefore, how to provide a new solution, which can solve the above technical problems, is a technical problem to be solved in the art.
Disclosure of Invention
The embodiment of the invention provides a multi-channel access message forwarding method, which realizes a mechanism capable of dynamically adjusting, controlling message sequence and caching and retransmitting through parameterized configuration, and ensures normal service processing of a downstream system, and comprises the following steps:
configuring a forwarding parameter;
establishing a message queue through a platform scheduling task; wherein, the message queue includes: a waiting queue, an error queue and a buffering queue;
acquiring an event message generated when a local transaction is executed by using a scheduling task;
matching the event message with the forwarding parameter, and respectively putting the event message into a waiting queue and a buffering queue according to the service limit of the event message after the matching is successful;
reading the event message in the buffer queue through a message forwarding service, and configuring a sending sequence for the event message by using a strong sequence guarantee mechanism;
sending the event messages to a peripheral system according to a sending buffer mechanism and dynamically adjusting a sending quota in the sending process;
when the transmission to the peripheral system fails, the event message is put into an error queue.
The embodiment of the present invention further provides a device for forwarding a multi-channel access message, including:
the parameter configuration module is used for configuring forwarding parameters;
the message queue establishing module is used for establishing a message queue through a platform scheduling task; wherein, the message queue includes: a waiting queue, an error queue and a buffering queue;
the event message acquisition module is used for acquiring an event message generated when the local transaction is executed by using the scheduling task;
the forwarding parameter matching module is used for matching the event message with the forwarding parameters, and after the matching is successful, the event message is respectively put into a waiting queue and a buffering queue according to the service limit of the event message;
a sending sequence configuration module, which is used for reading the event message in the buffer queue through the message forwarding service and configuring the sending sequence for the event message by using a strong sequence guarantee mechanism;
the event message forwarding module is used for sending the event messages to the peripheral system according to the sending buffer mechanism and dynamically adjusting the sending quota in the sending process;
and the error processing module is used for putting the event message into an error queue when the event message is failed to be sent to the peripheral system.
The embodiment of the invention also provides computer equipment which comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein the processor realizes the multi-channel access message forwarding method when executing the computer program.
An embodiment of the present invention further provides a computer-readable storage medium, where a computer program for executing the above-mentioned multichannel access message forwarding method is stored in the computer-readable storage medium.
The embodiment of the invention provides a method and a device for forwarding a multi-channel access message, which comprises the following steps: firstly, configuring forwarding parameters; then, establishing a message queue through a platform scheduling task; wherein, the message queue includes: a waiting queue, an error queue and a buffering queue; continuously utilizing the scheduling task to acquire an event message generated when the local transaction is executed; matching the event message with the forwarding parameter, and respectively putting the event message into a waiting queue and a buffering queue according to the service limit of the event message after the matching is successful; next, reading the event message in the buffer queue through a message forwarding service, and configuring a sending sequence for the event message by using a strong sequence guarantee mechanism; then, according to a sending buffer mechanism, sending the event messages to a peripheral system according to a sending sequence, and dynamically adjusting a sending quota in the sending process; and finally, when the event message is failed to be sent to the peripheral system, the event message is put into an error queue. The embodiment of the invention realizes a mechanism capable of dynamic adjustment, message sequence control and cache retransmission through parametric configuration, and ensures normal service processing of a downstream system; the main technical effects comprise:
1. through parameter configuration, a service scene is automatically identified, the coupling between service codes in the system and message forwarding is reduced, key elements of the message service are obtained, the service expansion is simple and convenient, the possibility of parallel development is increased, the online speed of a new service is improved, the forwarding of multi-system and multi-message is realized, the transaction consistency is ensured, the service scene can be restored in the forwarding process, and the requirements of a downstream system are met.
2. A strong order guarantee mechanism is provided, which can ensure that the application service execution message is forwarded according to a preset order, the disorder of the downstream message due to overtime and the like is avoided, the service complexity of the downstream system is reduced, and the peripheral system can still receive the message according to a specific order under the condition that the message arrives in the disorder; and meanwhile, no matter how the arrival sequence of the confirmation messages is, the confirmation messages are guaranteed to be sent to the downstream in a predefined sequence according to the service requirements of the downstream system.
3. When the message is sent to the downstream peripheral system, a flow control mechanism is adopted, the message sending process is dynamically adjusted according to the sending quota, the sequence is ensured, meanwhile, the dynamically adjustable buffer is provided for the peripheral system, the downstream cannot suddenly receive a large number of messages, and the system pressure is reduced.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts. In the drawings:
fig. 1 is a schematic diagram of a method for forwarding a multi-channel access message according to an embodiment of the present invention.
Fig. 2 is a flow and execution sequence diagram of a "store and forward" mechanism in the prior art.
Fig. 3 is a schematic diagram illustrating a process of generating an event message when a local transaction is executed according to a method for forwarding a multichannel access message in an embodiment of the present invention.
Fig. 4 is a schematic diagram of a computer device for executing a multi-channel access message forwarding method implemented by the present invention.
Fig. 5 is a schematic diagram of a multi-channel access message forwarding apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention are further described in detail below with reference to the accompanying drawings. The exemplary embodiments and descriptions of the present invention are provided to explain the present invention, but not to limit the present invention.
Fig. 1 is a schematic diagram of a method for forwarding a multi-channel access message according to an embodiment of the present invention, and as shown in fig. 1, an embodiment of the present invention provides a method for forwarding a multi-channel access message, which implements a mechanism capable of dynamically adjusting, controlling a message sequence, and retransmitting a cache through parameterized configuration, so as to ensure that a downstream system performs normal service processing, and includes:
step 101: configuring a forwarding parameter;
step 102: establishing a message queue through a platform scheduling task; wherein, the message queue includes: a waiting queue, an error queue and a buffering queue;
step 103: acquiring an event message generated when a local transaction is executed by using a scheduling task;
step 104: matching the event message with the forwarding parameter, and respectively putting the event message into a waiting queue and a buffering queue according to the service limit of the event message after the matching is successful;
step 105: reading the event message in the buffer queue through a message forwarding service, and configuring a sending sequence for the event message by using a strong sequence guarantee mechanism;
step 106: sending the event messages to a peripheral system according to a sending buffer mechanism and dynamically adjusting a sending quota in the sending process;
step 107: when the transmission to the peripheral system fails, the event message is put into an error queue.
The embodiment of the invention provides a method for forwarding a multi-channel access message, which comprises the following steps: firstly, configuring forwarding parameters; then, establishing a message queue through a platform scheduling task; wherein, the message queue includes: a waiting queue, an error queue and a buffering queue; continuously utilizing the scheduling task to acquire an event message generated when the local transaction is executed; matching the event message with the forwarding parameter, and respectively putting the event message into a waiting queue and a buffering queue according to the service limit of the event message after the matching is successful; next, reading the event message in the buffer queue through a message forwarding service, and configuring a sending sequence for the event message by using a strong sequence guarantee mechanism; then, according to a sending buffer mechanism, sending the event messages to a peripheral system according to a sending sequence, and dynamically adjusting a sending quota in the sending process; and finally, when the event message is failed to be sent to the peripheral system, the event message is put into an error queue. The embodiment of the invention realizes a mechanism capable of dynamic adjustment, message sequence control and cache retransmission through parametric configuration, and ensures normal service processing of a downstream system; the main technical effects are as follows:
1. through parameter configuration, a service scene is automatically identified, the coupling between service codes in the system and message forwarding is reduced, key elements of the message service are obtained, the service expansion is simple and convenient, the possibility of parallel development is increased, the online speed of a new service is improved, the forwarding of multi-system and multi-message is realized, the transaction consistency is ensured, the service scene can be restored in the forwarding process, and the requirements of a downstream system are met.
2. A strong order guarantee mechanism is provided, which can ensure that the application service execution message is forwarded according to a preset order, the disorder of the downstream message due to overtime and the like is avoided, the service complexity of the downstream system is reduced, and the peripheral system can still receive the message according to a specific order under the condition that the message arrives in the disorder; and meanwhile, no matter how the arrival sequence of the confirmation messages is, the confirmation messages are guaranteed to be sent to the downstream in a predefined sequence according to the service requirements of the downstream system.
3. When the message is sent to the downstream peripheral system, a flow control mechanism is adopted, the message sending process is dynamically adjusted according to the sending quota, the sequence is ensured, meanwhile, the dynamically adjustable buffer is provided for the peripheral system, the downstream cannot suddenly receive a large number of messages, and the system pressure is reduced.
With the increase of service complexity and user scale, service requirements are more and more complex, related systems are more and more, respective service processing logics of each system are different, the requirement on the consistency of objects is higher than that of distributed systems, the conflict cost is very high, single system rollback cannot be achieved, and strong consistency needs to be adopted.
The existing business can not complete all business processing in a single link, the interactivity among all systems is very strong, sometimes, one transaction needs to be interacted for 3 to 4 times, the requirement of time sequence is better, on the premise of ensuring the normal end of the business, strong consistency is needed, and the interaction among the systems is completed in the business processing logic. The access channels are more and more, the design service scenes are more and more complex, the coupling between codes is very strong, synchronous confirmation, fault-tolerant reissue and ordered transaction are completed, a little content is modified sometimes, and the adjustment range is very large. Especially, when a new access channel or a new service requirement is completed, the related service logic needs to be completely combed and adjusted, so that the service is normally completed.
The invention aims to establish a comprehensive scheduling platform, realize parametric configuration, realize dynamic adjustment, increase message sequence control and buffer retransmission mechanism and ensure normal service processing of a downstream system.
The main functions realized by the invention comprise:
1. on the premise of reducing code coupling, the function of parametric configuration is realized, and the global transaction consistency of data fragmentation/multiple data sources is realized;
2. providing a message sequence guarantee mechanism, finishing the sequential processing of the out-of-order messages by a platform, and reading the messages in a downstream according to a specific sequence;
3. dynamically adjustable buffering is provided for downstream consumption while ensuring order, ensuring that downstream does not suddenly receive a large number of messages.
When the method for forwarding the multi-channel access message provided by the embodiment of the present invention is implemented specifically, the method may include: configuring a forwarding parameter; establishing a message queue through a platform scheduling task; wherein, the message queue includes: a waiting queue, an error queue and a buffering queue; acquiring an event message generated when a local transaction is executed by using a scheduling task; matching the event message with the forwarding parameter, and respectively putting the event message into a waiting queue and a buffering queue according to the service limit of the event message after the matching is successful; reading the event message in the buffer queue through a message forwarding service, and configuring a sending sequence for the event message by using a strong sequence guarantee mechanism; sending the event messages to a peripheral system according to a sending buffer mechanism and dynamically adjusting a sending quota in the sending process; when the transmission to the peripheral system fails, the event message is put into an error queue.
In the ordinary service logic processing process, besides the service of the system, a large amount of message forwarding logic processing is occupied, the codes are coupled more and more tightly, when an access channel is newly added or the forwarding requirement of a new service type is newly added, the original service code needs to be modified, the modification cost is high, and the online time is long.
Meanwhile, because the unreliability of the network and the service processing capability of the downstream system are difficult to be truly synchronized, and the downstream has certain service dependence on the sequence of the messages, the sequence of the messages finally arriving at the downstream after the traditional store-and-forward and distributed multi-segment message aggregation can not be ensured, so that the sequence of the messages sent to the downstream can not be ensured finally, and the service failure is caused.
The invention establishes a set of parameterized configuration logic, configures forwarding parameters, ensures the ordered arrival of messages by platform scheduling, controls the sending speed by adopting a sending quota mechanism, dynamically adjusts the quota and achieves the effect of caching.
The embodiment of the invention realizes the forwarding of multi-system multi-message by parameter configuration on the premise of reducing the code coupling in the system, and ensures the transaction consistency; providing a message sequence guarantee mechanism to ensure that a peripheral system can still receive messages according to a specific sequence under the condition that the messages arrive out of sequence; the buffer which can be dynamically adjusted is provided for the peripheral system while the sequence is ensured, thereby ensuring that the downstream can not suddenly receive a large amount of messages and reducing the system pressure.
In an embodiment of the present invention, when the method for forwarding a multi-channel access message is implemented specifically, the configuring of the forwarding parameter includes:
and forming parameters by taking the service initiator, the service type, the service receiver and the original service type as key elements, and configuring the parameters into forwarding parameters.
In the embodiment, a parameterized configuration logic is established, a service initiator, a service type, a service receiver and an original service type are used as key elements to form parameters, the parameters are recorded into forwarding configuration information, message forwarding is decoupled from a service processing logic, and the message forwarding is completed by a platform scheduling task. The embodiment of the invention automatically identifies the service scene through parameter configuration, reduces the coupling of the service code in the system and the message forwarding, acquires the key elements of the message service, has simple and convenient service expansion, increases the possibility of parallel development, improves the online speed of new service, realizes the forwarding of multiple messages of multiple systems, ensures the transaction consistency, can restore the service scene in the forwarding process and meets the requirements of downstream systems.
In an embodiment of the method for forwarding a multi-channel access message according to an embodiment of the present invention, the establishing a message queue by a scheduling task of a platform includes: the platform scheduling task establishes 3 queues, a waiting queue, an error queue and a buffer queue, and each queue respectively establishes a channel queue according to an accessed channel; the scheduling task of the platform is also used to retrieve event messages.
Fig. 3 is a schematic diagram of a process of generating an event message when a local transaction is executed in a method for forwarding a multichannel access message according to an embodiment of the present invention, and as shown in fig. 3, when the method for forwarding a multichannel access message according to an embodiment of the present invention is specifically implemented, in an embodiment, the generating an event message when a local transaction is executed includes:
step 301: when the execution of the local transaction is successful, the generated event message is acquired by the scheduling task and is put into a message queue; wherein the event message is persisted and supports repeated consumption;
step 302: when execution of the local transaction fails, no event message is generated.
In an embodiment, to ensure reliable event messages, at least two conditions need to be satisfied:
1) persistence of the message itself: event messages submitted to the message queue must be persisted and support repeated consumption, a feature that may be provided by the message queue, such as Apache Kafka;
2) atomicity between local transactions and transaction events: if the local transaction is successfully executed, the correspondingly generated event message must be acquired by the scheduling task and submitted to the message queue, otherwise, the local transaction which is failed to be executed cannot push the event message at all, and the pushing is stopped by directly adopting a mode of not generating the event message when the local transaction is failed to be executed.
In a specific embodiment of the method for forwarding a multi-channel access message according to the embodiments of the present invention, in the method for forwarding a multi-channel access message, the matching of the event message with the forwarding parameter and the successful matching are respectively placed in a waiting queue and a buffering queue according to the service restriction of the event message, including:
matching the event message with the forwarding parameters, scheduling the forwarding process when the matching is successful, calculating the HASH value of the event message when the service limit of the event message accords with the forwarding logic but needs to wait for the preposed message, instantiating the HASH value, establishing an event message index, putting the event message into a waiting queue, and waiting for the preposed message to arrive;
and when the service limit of the event message accords with the forwarding logic and the pre-message exists, putting the event message into a buffer queue, searching whether a subsequent message waiting for sending exists in the waiting queue, and taking out the subsequent message waiting from the waiting queue and putting the subsequent message waiting into the buffer queue when the subsequent message waiting for sending exists.
In the embodiment, after the scheduling task acquires the event message, the event message is matched with the forwarding parameter, when the matching fails, no peripheral system subscribes the message, and the event message is set as successfully forwarded;
when the matching is successful, scheduling a forwarding flow, calculating the HASH value of the event message when the service limit of the event message accords with the forwarding logic but needs to wait for the preposed message, instantiating the HASH value, establishing an event message index, putting the event message into a waiting queue, and waiting for the preposed message to arrive;
and when the service limit of the event message accords with the forwarding logic and the pre-message exists, putting the event message into a buffer queue, searching whether a subsequent message waiting for sending exists in the waiting queue, and taking out the subsequent message waiting from the waiting queue and putting the subsequent message waiting into the buffer queue when the subsequent message waiting for sending exists.
Due to the fact that transaction events need to be processed and confirmed through multiple stages to be finally forwarded to corresponding destination queues, and due to the introduction of a timeout confirmation mechanism and a concurrency mechanism of a message queue, part of events may not reach the destination queues in the original order. In some specific sequence-sensitive scenes, the out-of-order events will affect the correctness or success rate of subsequent processing logic, and in order to solve the above problems, the embodiment of the invention particularly designs a strong sequence guarantee mechanism of event messages; in an embodiment of the method for forwarding a multi-channel access message according to an embodiment of the present invention, in a specific implementation, the configuring a sending sequence for an event message by using a strong sequence guarantee mechanism includes:
and setting a serial number Key and a front transaction Key for each event message by using a strong sequence guarantee mechanism, and configuring a sending sequence.
The mechanism sets a serial number Key and a preposed affair Key for each event message, and can send the current message only after the preposed message is successfully sent, and the strong order ensures that the order of sending the preparation stage message by the affair initiator is the standard. The aforementioned keys represent a kind of sequential encoding of event messages. The strong-order guarantee mechanism provided by the embodiment of the invention can ensure that the application service execution message is forwarded according to a preset order, the disorder of the downstream message due to overtime and the like is avoided, the service complexity of the downstream system is reduced, and the peripheral system can still receive the message according to a specific order under the condition that the message arrives in the disorder; and meanwhile, no matter how the arrival sequence of the confirmation messages is, the confirmation messages are guaranteed to be sent to the downstream in a predefined sequence according to the service requirements of the downstream system.
The buffer queue stores and maintains the messages which are available for being sent to the downstream but are not sent temporarily under each sequential chain, if the messages are searched from the beginning of the sequential chain to the downstream in sequence, and once the messages which have received the confirmation are sent directly, a large number of messages are sent in a short time when the sequential chain is long, and great pressure can be caused to downstream application. In order to solve the above problem, a concept of a sending quota and a time period T is introduced, and the sending quota is dynamically adjusted in an operation process, and when the method for forwarding a multichannel access message provided by an embodiment of the present invention is specifically implemented, in an embodiment, the dynamically adjusting the sending quota in the sending process includes:
the sending task runs regularly, and the initial sending quota is set to be miStep quota is set to Δ m, maximum quota is set to mmax；
Triggering the sending task, recording the sending number c and the task time T, stopping sending if the task time T exceeds the time period T or the sending number exceeds the set sending quota, checking the task execution condition,
if T is less than T, changing the sending quota m to c + delta m;
if c is smaller than m, changing the sending quota m to be c-delta m;
if c <10, changing the sending quota m to 1;
if m is less than delta m/2, changing the sending quota m to be 2 m;
if the current sending quota t is tmaxThen the transmit quota is not increased.
In the embodiment, a flow control mechanism is adopted when the message is sent to a downstream peripheral system, the message sending process is dynamically adjusted according to the sending quota, the sequence is ensured, meanwhile, dynamically adjustable buffering is provided for the peripheral system, the downstream cannot suddenly receive a large number of messages, and the system pressure is reduced; and within the maximum quota range, if the consumed time exceeds the current quota, increasing the quota for the next batch of transmission, and if the consumed time does not exceed the current quota, decreasing the quota for the next batch of transmission.
When the event messages are sent to the peripheral system according to the sending buffer mechanism and the sending sequence, deleting the messages when the sending is successful, continuing to send the next message, and adjusting the sending quota; adjusting the sending quota when the time is out or the response time is over long; and if the message fails, the message is put into an error queue, and the next message is continuously sent.
When the peripheral system fails to transmit due to error report of the forwarding peripheral system, the message is put into an error queue, the transmission frequency is added by 1 to reach the maximum frequency, the forwarding fails, the message is deleted, and the failure state is recorded.
The method for forwarding the multi-channel access message can realize that:
1. the method reduces the coupling of service codes through parametric configuration, is simple and convenient for service expansion, increases the possibility of parallel development, and improves the online speed of new services.
2. The message conversion platform provides a forced sequence guarantee mechanism, which can ensure that the application service execution messages are forwarded according to a preset sequence, the downstream message disorder caused by overtime and the like is avoided, and the service complexity of a downstream system is reduced.
3. The message sending is dynamically adjusted according to the sending quota, so that the execution efficiency of the program is improved, and meanwhile, the downstream pressure cannot be brought.
Fig. 4 is a schematic diagram of a computer device for executing a multi-channel access message forwarding method implemented by the present invention, and as shown in fig. 4, an embodiment of the present invention further provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the multi-channel access message forwarding method when executing the computer program.
An embodiment of the present invention further provides a computer-readable storage medium, where a computer program for implementing the above-mentioned multichannel access message forwarding method is stored in the computer-readable storage medium.
The embodiment of the invention also provides a device for forwarding the multi-channel access message, which is described in the following embodiment. Because the principle of the device for solving the problems is similar to that of a multi-channel access message forwarding method, the implementation of the device can refer to the implementation of the multi-channel access message forwarding method, and repeated parts are not described again.
Fig. 5 is a schematic diagram of a multichannel access message forwarding apparatus according to an embodiment of the present invention, and as shown in fig. 5, an embodiment of the present invention further provides a multichannel access message forwarding apparatus, which may include:
a parameter configuration module 501, configured to configure forwarding parameters;
a message queue establishing module 502, configured to establish a message queue through a platform scheduling task; wherein, the message queue includes: a waiting queue, an error queue and a buffering queue;
an event message acquiring module 503, configured to acquire, by using the scheduling task, an event message generated when the local transaction is executed;
a forwarding parameter matching module 504, configured to match the event message with a forwarding parameter, and after the matching is successful, place the event message into a waiting queue and a buffering queue according to service restrictions of the event message;
a sending sequence configuration module 505, configured to read the event message in the buffer queue through the message forwarding service, and configure a sending sequence for the event message by using a strong sequence guarantee mechanism;
an event message forwarding module 506, configured to send event messages to a peripheral system according to a sending buffer mechanism, where a sending quota is dynamically adjusted in a sending process;
the error handling module 507 is configured to place the event message into an error queue when the sending to the peripheral system fails.
In an embodiment of the present invention, when the device for forwarding a multichannel access message is implemented specifically, the parameter configuration module is configured to:
and forming parameters by taking the service initiator, the service type, the service receiver and the original service type as key elements, and configuring the parameters into forwarding parameters.
In an embodiment of the present invention, when the device for forwarding a multi-channel access message provided in the embodiment of the present invention is implemented specifically, the acquiring an event message by the event message acquiring module when a local transaction is executed includes:
when the execution of the local transaction is successful, the generated event message is acquired by the scheduling task and is put into a message queue; wherein the event message is persisted and supports repeated consumption;
when execution of the local transaction fails, no event message is generated.
In an embodiment of the present invention, when the apparatus for forwarding a multi-channel access message provided in the embodiment of the present invention is implemented specifically, the forwarding parameter matching module is specifically configured to:
matching the event message with the forwarding parameters, scheduling the forwarding process when the matching is successful, calculating the HASH value of the event message when the service limit of the event message accords with the forwarding logic but needs to wait for the preposed message, instantiating the HASH value, establishing an event message index, putting the event message into a waiting queue, and waiting for the preposed message to arrive;
and when the service limit of the event message accords with the forwarding logic and the pre-message exists, putting the event message into a buffer queue, searching whether a subsequent message waiting for sending exists in the waiting queue, and taking out the subsequent message waiting from the waiting queue and putting the subsequent message waiting into the buffer queue when the subsequent message waiting for sending exists.
In an embodiment of the present invention, when the device for forwarding a multi-channel access message is implemented specifically, the sending sequence configuration module is specifically configured to:
and setting a serial number Key and a front transaction Key for each event message by using a strong sequence guarantee mechanism, and configuring a sending sequence.
In an embodiment of the present invention, when the device for forwarding a multichannel access message is implemented specifically, the event message forwarding module is specifically configured to:
the sending task runs regularly, and the initial sending quota is set to be miStep quota is set to Δ m, maximum quota is set to mmax；
Triggering the sending task, recording the sending number c and the task time T, stopping sending if the task time T exceeds the time period T or the sending number exceeds the set sending quota, checking the task execution condition,
if T is less than T, changing the sending quota m to c + delta m;
if c is smaller than m, changing the sending quota m to be c-delta m;
if c <10, changing the sending quota m to 1;
if m is less than delta m/2, changing the sending quota m to be 2 m;
if the current sending quota t is tmaxThen the transmit quota is not increased.
The embodiment of the invention automatically identifies the service scene through parameter configuration, reduces the coupling of the service code and the message forwarding, key elements of the message service, and can restore the service scene in the platform forwarding process, thereby meeting the requirements of a downstream system. Message order assurance mechanisms are provided to ensure that messages are sent downstream in a predefined order, regardless of the order of arrival of acknowledgement messages, in accordance with downstream system traffic needs. When the message is sent to the downstream, a flow control mechanism is adopted, the process is limited by sending quota, in the maximum quota range, if the consumed time exceeds the current quota, the quota is increased for the next batch of sending, and if the consumed time does not exceed the current quota, the quota is decreased for the next batch of sending.
To sum up, a method and an apparatus for forwarding a multi-channel access message provided by the embodiments of the present invention include: firstly, configuring forwarding parameters; then, establishing a message queue through a platform scheduling task; wherein, the message queue includes: a waiting queue, an error queue and a buffering queue; continuously utilizing the scheduling task to acquire an event message generated when the local transaction is executed; matching the event message with the forwarding parameter, and respectively putting the event message into a waiting queue and a buffering queue according to the service limit of the event message after the matching is successful; next, reading the event message in the buffer queue through a message forwarding service, and configuring a sending sequence for the event message by using a strong sequence guarantee mechanism; then, according to a sending buffer mechanism, sending the event messages to a peripheral system according to a sending sequence, and dynamically adjusting a sending quota in the sending process; and finally, when the event message is failed to be sent to the peripheral system, the event message is put into an error queue. The embodiment of the invention realizes a mechanism capable of dynamic adjustment, message sequence control and cache retransmission through parametric configuration, and ensures normal service processing of a downstream system; the main technical effects are as follows:
1. through parameter configuration, a service scene is automatically identified, the coupling between service codes in the system and message forwarding is reduced, key elements of the message service are obtained, the service expansion is simple and convenient, the possibility of parallel development is increased, the online speed of a new service is improved, the forwarding of multi-system and multi-message is realized, the transaction consistency is ensured, the service scene can be restored in the forwarding process, and the requirements of a downstream system are met.
2. A strong order guarantee mechanism is provided, which can ensure that the application service execution message is forwarded according to a preset order, the disorder of the downstream message due to overtime and the like is avoided, the service complexity of the downstream system is reduced, and the peripheral system can still receive the message according to a specific order under the condition that the message arrives in the disorder; and meanwhile, no matter how the arrival sequence of the confirmation messages is, the confirmation messages are guaranteed to be sent to the downstream in a predefined sequence according to the service requirements of the downstream system.
3. When the message is sent to a downstream peripheral system, a flow control mechanism is adopted, the message sending process is dynamically adjusted according to the sending quota, the sequence is ensured, meanwhile, dynamically adjustable buffering is provided for the peripheral system, the downstream cannot suddenly receive a large number of messages, and the system pressure is reduced; and within the maximum quota range, if the consumed time exceeds the current quota, increasing the quota for the next batch of transmission, and if the consumed time does not exceed the current quota, decreasing the quota for the next batch of transmission.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are only exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (14)
1. A multi-channel access message forwarding method is characterized by comprising the following steps:
configuring a forwarding parameter;
establishing a message queue through a platform scheduling task; wherein, the message queue includes: a waiting queue, an error queue and a buffering queue;
acquiring an event message generated when a local transaction is executed by using a scheduling task;
matching the event message with the forwarding parameter, and respectively putting the event message into a waiting queue and a buffering queue according to the service limit of the event message after the matching is successful;
reading the event message in the buffer queue through a message forwarding service, and configuring a sending sequence for the event message by using a strong sequence guarantee mechanism;
sending the event messages to a peripheral system according to a sending buffer mechanism and dynamically adjusting a sending quota in the sending process;
when the transmission to the peripheral system fails, the event message is put into an error queue.
2. The method of claim 1, wherein configuring forwarding parameters comprises:
and forming parameters by taking the service initiator, the service type, the service receiver and the original service type as key elements, and configuring the parameters into forwarding parameters.
3. The method of claim 1, wherein generating an event message when executing the local transaction comprises:
when the execution of the local transaction is successful, the generated event message is acquired by the scheduling task and is put into a message queue; wherein the event message is persisted and supports repeated consumption;
when execution of the local transaction fails, no event message is generated.
4. The method of claim 1, wherein matching the event message with the forwarding parameter, and after the matching is successful, placing the event message into a waiting queue and a buffering queue according to the service limit of the event message, respectively, comprises:
matching the event message with the forwarding parameters, scheduling the forwarding process when the matching is successful, calculating the HASH value of the event message when the service limit of the event message accords with the forwarding logic but needs to wait for the preposed message, instantiating the HASH value, establishing an event message index, putting the event message into a waiting queue, and waiting for the preposed message to arrive;
and when the service limit of the event message accords with the forwarding logic and the pre-message exists, putting the event message into a buffer queue, searching whether a subsequent message waiting for sending exists in the waiting queue, and taking out the subsequent message waiting from the waiting queue and putting the subsequent message waiting into the buffer queue when the subsequent message waiting for sending exists.
5. The method of claim 1, wherein configuring a sending order for event messages using a strong order assurance mechanism comprises:
and setting a serial number Key and a front transaction Key for each event message by using a strong sequence guarantee mechanism, and configuring a sending sequence.
6. The method of claim 1, wherein dynamically adjusting the transmission quota during the transmission comprises:
the sending task runs regularly, and the initial sending quota is set to be miStep quota is set to Δ m, maximum quota is set to mmax；
Triggering the sending task, recording the sending number c and the task time T, stopping sending if the task time T exceeds the time period T or the sending number exceeds the set sending quota, checking the task execution condition,
if T is less than T, changing the sending quota m to c + delta m;
if c is smaller than m, changing the sending quota m to be c-delta m;
if c <10, changing the sending quota m to 1;
if m is less than delta m/2, changing the sending quota m to be 2 m;
if the current sending quota t is tmaxThen the transmit quota is not increased.
7. A multi-channel access message forwarding apparatus, comprising:
the parameter configuration module is used for configuring forwarding parameters;
the message queue establishing module is used for establishing a message queue through a platform scheduling task; wherein, the message queue includes: a waiting queue, an error queue and a buffering queue;
the event message acquisition module is used for acquiring an event message generated when the local transaction is executed by using the scheduling task;
the forwarding parameter matching module is used for matching the event message with the forwarding parameters, and after the matching is successful, the event message is respectively put into a waiting queue and a buffering queue according to the service limit of the event message;
a sending sequence configuration module, which is used for reading the event message in the buffer queue through the message forwarding service and configuring the sending sequence for the event message by using a strong sequence guarantee mechanism;
the event message forwarding module is used for sending the event messages to the peripheral system according to the sending buffer mechanism and dynamically adjusting the sending quota in the sending process;
and the error processing module is used for putting the event message into an error queue when the event message is failed to be sent to the peripheral system.
8. The apparatus of claim 7, wherein the parameter configuration module is to:
and forming parameters by taking the service initiator, the service type, the service receiver and the original service type as key elements, and configuring the parameters into forwarding parameters.
9. The apparatus of claim 7, wherein the event message retrieving module retrieves the event message generated when the local transaction is executed, comprising:
when the execution of the local transaction is successful, the generated event message is acquired by the scheduling task and is put into a message queue; wherein the event message is persisted and supports repeated consumption;
when execution of the local transaction fails, no event message is generated.
10. The apparatus of claim 7, wherein the forwarding parameter matching module is specifically configured to:
matching the event message with the forwarding parameters, scheduling the forwarding process when the matching is successful, calculating the HASH value of the event message when the service limit of the event message accords with the forwarding logic but needs to wait for the preposed message, instantiating the HASH value, establishing an event message index, putting the event message into a waiting queue, and waiting for the preposed message to arrive;
and when the service limit of the event message accords with the forwarding logic and the pre-message exists, putting the event message into a buffer queue, searching whether a subsequent message waiting for sending exists in the waiting queue, and taking out the subsequent message waiting from the waiting queue and putting the subsequent message waiting into the buffer queue when the subsequent message waiting for sending exists.
11. The apparatus of claim 7, wherein the transmission order configuration module is specifically configured to:
and setting a serial number Key and a front transaction Key for each event message by using a strong sequence guarantee mechanism, and configuring a sending sequence.
12. The apparatus of claim 7, wherein the event message forwarding module is specifically configured to:
the sending task runs regularly, and the initial sending quota is set to be miStep quota is set to Δ m, maximum quota is set to mmax；
Triggering the sending task, recording the sending number c and the task time T, stopping sending if the task time T exceeds the time period T or the sending number exceeds the set sending quota, checking the task execution condition,
if T is less than T, changing the sending quota m to c + delta m;
if c is smaller than m, changing the sending quota m to be c-delta m;
if c <10, changing the sending quota m to 1;
if m is less than delta m/2, changing the sending quota m to be 2 m;
if the current sending quota t is tmaxThen the transmit quota is not increased.
13. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of any of claims 1 to 6 when executing the computer program.
14. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program for executing a method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011484464.6A CN112579274A (en) | 2020-12-16 | 2020-12-16 | Multi-channel access message forwarding method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011484464.6A CN112579274A (en) | 2020-12-16 | 2020-12-16 | Multi-channel access message forwarding method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112579274A true CN112579274A (en) | 2021-03-30 |
Family
ID=75135720
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011484464.6A Pending CN112579274A (en) | 2020-12-16 | 2020-12-16 | Multi-channel access message forwarding method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112579274A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117290122A (en) * | 2023-02-28 | 2023-12-26 | 北京荣大科技股份有限公司 | Kafka-based multi-environment ordered production and consumption method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5729540A (en) * | 1995-10-19 | 1998-03-17 | Qualcomm Incorporated | System and method for scheduling messages on a common channel |
| WO2003010624A2 (en) * | 2001-07-23 | 2003-02-06 | Cognis Corporation | On-site analysis system with central processor and method of analysing |
| US6904014B1 (en) * | 2000-04-27 | 2005-06-07 | Cisco Technology, Inc. | Method and apparatus for performing high-speed traffic shaping |
| CN110071873A (en) * | 2019-04-09 | 2019-07-30 | 华为技术有限公司 | A kind of method, apparatus and relevant device sending data |
| CN111866191A (en) * | 2020-09-24 | 2020-10-30 | 深圳市易博天下科技有限公司 | Message event distribution method, distribution platform, system and server |
-
2020
- 2020-12-16 CN CN202011484464.6A patent/CN112579274A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5729540A (en) * | 1995-10-19 | 1998-03-17 | Qualcomm Incorporated | System and method for scheduling messages on a common channel |
| US6904014B1 (en) * | 2000-04-27 | 2005-06-07 | Cisco Technology, Inc. | Method and apparatus for performing high-speed traffic shaping |
| WO2003010624A2 (en) * | 2001-07-23 | 2003-02-06 | Cognis Corporation | On-site analysis system with central processor and method of analysing |
| CN110071873A (en) * | 2019-04-09 | 2019-07-30 | 华为技术有限公司 | A kind of method, apparatus and relevant device sending data |
| CN111866191A (en) * | 2020-09-24 | 2020-10-30 | 深圳市易博天下科技有限公司 | Message event distribution method, distribution platform, system and server |
Non-Patent Citations (1)
| Title |
|---|
| 金凤华;: "分布式消息中间件应用场景及建设设想", 中国金融电脑, no. 09, 7 September 2020 (2020-09-07) * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117290122A (en) * | 2023-02-28 | 2023-12-26 | 北京荣大科技股份有限公司 | Kafka-based multi-environment ordered production and consumption method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111539726B (en) | Block chain consensus system and method | |
| CN108023908B (en) | Data updating method, device and system | |
| CN108769099B (en) | Message de-duplication realization method of message middleware | |
| CN107341062B (en) | Data pushing method, device, equipment and storage medium | |
| US8984530B2 (en) | Queued message dispatch | |
| US7546613B2 (en) | Methods and systems for efficient queue propagation using a single protocol-based remote procedure call to stream a batch of messages | |
| CN110134534B (en) | System and method for optimizing message processing for big data distributed system based on NIO | |
| CN111949614B (en) | Bank system file conversion method and device | |
| CN111045839A (en) | Sequence calling method and device based on two-phase transaction message in distributed environment | |
| CN112019597A (en) | Distributed data receiving system and data receiving method | |
| CN112579274A (en) | Multi-channel access message forwarding method and device | |
| CN114629904B (en) | Distributed event processing method, system, equipment and medium | |
| CN113259408B (en) | Data transmission method and system | |
| CN114063936A (en) | Method, system, equipment and storage medium for optimizing timing task | |
| CN114500416A (en) | Delivery method and delivery system for at most one message delivery | |
| WO2023231723A1 (en) | Streaming media data processing method and system | |
| CN112702146B (en) | Data processing method and device | |
| CN113691466B (en) | Data transmission method, intelligent network card, computing device and storage medium | |
| CN110445874B (en) | Session processing method, device, equipment and storage medium | |
| CN113992609B (en) | Method and system for processing multilink service data disorder | |
| CN108337285B (en) | Communication system and communication method | |
| CN116016555A (en) | Message synchronization method, device, equipment and computer storage medium | |
| US7895362B2 (en) | Multiple message source electronic data interchange (EDI) enveloper with batching support | |
| US11210089B2 (en) | Vector send operation for message-based communication | |
| US5754856A (en) | MVS/ESA message transport system using the XCF coupling facility |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |