CN113961372A - MQ multi-instance message sending method and device - Google Patents
MQ multi-instance message sending method and device Download PDFInfo
- Publication number
- CN113961372A CN113961372A CN202111247686.0A CN202111247686A CN113961372A CN 113961372 A CN113961372 A CN 113961372A CN 202111247686 A CN202111247686 A CN 202111247686A CN 113961372 A CN113961372 A CN 113961372A
- Authority
- CN
- China
- Prior art keywords
- message
- queue manager
- queue
- available
- abnormal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/544—Buffers; Shared memory; Pipes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/548—Queue
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Telephonic Communication Services (AREA)
Abstract
The invention provides an MQ multi-instance message sending method and a device, belonging to big data, and the method comprises the following steps: configuring a plurality of MQ queue managers to join an MQ queue manager group; when an application system sends a message, an MQ queue manager is selected through an MQ queue manager group, and an available MQ queue manager list is determined and cached; the method comprises the steps that messages sent by an application system are received through an MQ queue, an MQ queue manager is selected from a cache by utilizing an MQ queue manager group, and the messages are sent to a corresponding channel through the selected MQ queue manager; monitoring the message sending result, and when the message sending is abnormal, replacing a new MQ queue manager according to an abnormality processing mechanism to resend the message. The invention manages a plurality of MQ instances uniformly from the application angle, provides a high-availability solution of MQ multi-instance, and realizes the configuration and message transmission of the multi-MQ queue manager by application.
Description
Technical Field
The invention relates to the technical field of computer data processing, in particular to an MQ multi-instance message sending method and device.
Background
This section is intended to provide a background or context to the embodiments of the invention that are recited in the claims. The description herein is not admitted to be prior art by inclusion in this section.
MQ (IBM Message queue) is an existing commercial Message intermediate product, and is suitable for being used in a distributed computing environment or a heterogeneous system. Message queuing is a technique for exchanging information between distributed applications. The message queue may reside on memory or disk, the queue storing messages until they are read by the application. With the message queue, applications can execute independently-they need not know each other's location, or wait for the receiving program to receive the message before continuing execution.
In a conventional application architecture, MQ adopts single-point deployment, as shown in fig. 2, a single-instance architecture diagram of MQ in the prior art, when an application system sends a message, the application system needs to send the message through a queue, and the queue sends the message to a specified channel according to a configured single queue manager, so that the risk of unavailability after a failure exists.
Therefore, how to provide a new solution, which can solve the above technical problems, is a technical problem to be solved in the art.
Disclosure of Invention
The embodiment of the invention provides an MQ multi-instance message sending method, which carries out unified management on a plurality of MQ instances from the application angle, provides a high-availability solution of the MQ multi-instance, and realizes the configuration and message sending of a multi-MQ queue manager by application, and comprises the following steps:
configuring a plurality of MQ queue managers to join an MQ queue manager group;
when an application system sends a message, an MQ queue manager is selected through an MQ queue manager group, and an available MQ queue manager list is determined and cached;
the method comprises the steps that messages sent by an application system are received through an MQ queue, an MQ queue manager is selected from a cache by utilizing an MQ queue manager group, and the messages are sent to a corresponding channel through the selected MQ queue manager;
monitoring the message sending result, and when the message sending is abnormal, replacing a new MQ queue manager according to an abnormality processing mechanism to resend the message.
An embodiment of the present invention further provides an MQ multi-instance message sending apparatus, including:
the MQ queue manager configuration module is used for configuring a plurality of MQ queue managers to join the MQ queue manager group;
the available MQ queue manager list determining module is used for selecting an MQ queue manager through an MQ queue manager group when the application system sends a message, determining an available MQ queue manager list and caching the available MQ queue manager list;
the MQ multi-instance scheduling module is used for receiving the message sent by the application system through the MQ queue, selecting an MQ queue manager from the cache by utilizing the MQ queue manager group, and sending the message to a corresponding channel through the selected MQ queue manager;
and the abnormal message processing and retransmitting module is used for monitoring the message sending result and updating a new MQ queue manager to retransmit the message according to the abnormal processing mechanism when the message sending is abnormal.
The embodiment of the present invention further provides a computer device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the MQ multi-instance message sending method when executing the computer program.
An embodiment of the present invention further provides a computer-readable storage medium, where a computer program for executing the MQ multi-instance message sending method is stored in the computer-readable storage medium.
The embodiment of the invention provides an MQ multi-instance message sending method and a device, comprising the following steps: firstly, configuring a plurality of MQ queue managers to join an MQ queue manager group; then when the application system sends a message, an MQ queue manager is selected through an MQ queue manager group, and an available MQ queue manager list is determined and cached; next, receiving a message sent by the application system through the MQ queue, selecting an MQ queue manager from the cache by using the MQ queue manager group, and sending the message to a corresponding channel through the selected MQ queue manager; and finally, monitoring a message sending result, and replacing a new MQ queue manager to resend the message according to an exception handling mechanism when the message sending is abnormal. The invention manages a plurality of MQ instances uniformly from the application angle, provides a high-availability solution of MQ multi-instance, and realizes the configuration and message transmission of the multi-MQ queue manager by application. Scheduling management is carried out on a plurality of MQ queue managers by adding an MQ queue manager group, and an exception handling mechanism is introduced under the abnormal condition; the high availability mechanism aiming at a plurality of MQ instances is realized, and the mechanism is designed aiming at the abnormity in the message sending process; an exception handling mechanism; the method meets the hard requirement of high availability of the MQ in the production environment, has relatively low development invasiveness, can be realized by configuring the MQ queue management group only, and has low application modification cost.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts. In the drawings:
fig. 1 is a schematic diagram of an MQ multi-instance message sending method according to an embodiment of the present invention.
Fig. 2 is a prior art MQ single instance architecture diagram.
Fig. 3 is an MQ multi-instance architecture diagram of an MQ multi-instance messaging method according to an embodiment of the present invention.
Fig. 4 is a flowchart of a scheduling mechanism of a queue manager group of the MQ multi-instance message sending method according to the embodiment of the present invention.
Fig. 5 is a schematic diagram of an exception message handling retransmission process of the MQ multi-instance message sending method according to the embodiment of the present invention.
Fig. 6 is a flow chart of an exception round robin mechanism of an MQ multi-instance message sending method according to an embodiment of the present invention.
Fig. 7 is a schematic diagram of a computer apparatus running an MQ multi-instance messaging method implemented by the present invention.
Fig. 8 is a diagram of an MQ multi-instance messaging device according to an embodiment of the invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention are further described in detail below with reference to the accompanying drawings. The exemplary embodiments and descriptions of the present invention are provided to explain the present invention, but not to limit the present invention.
The invention relates to big data technology. Fig. 1 is a schematic diagram of an MQ multi-instance message sending method according to an embodiment of the present invention, and as shown in fig. 1, an embodiment of the present invention provides an MQ multi-instance message sending method, where the present invention performs unified management on multiple MQ instances from an application perspective, and provides a high availability solution for MQ multi-instance, and an application implements configuration and message sending of a multi-MQ queue manager, where the method includes:
step 101: configuring a plurality of MQ queue managers to join an MQ queue manager group;
step 102: when an application system sends a message, an MQ queue manager is selected through an MQ queue manager group, and an available MQ queue manager list is determined and cached;
step 103: the method comprises the steps that messages sent by an application system are received through an MQ queue, an MQ queue manager is selected from a cache by utilizing an MQ queue manager group, and the messages are sent to a corresponding channel through the selected MQ queue manager;
step 104: monitoring the message sending result, and when the message sending is abnormal, replacing a new MQ queue manager according to an abnormality processing mechanism to resend the message.
The MQ multi-instance message sending method provided by the embodiment of the invention comprises the following steps: firstly, configuring a plurality of MQ queue managers to join an MQ queue manager group; then when the application system sends a message, an MQ queue manager is selected through an MQ queue manager group, and an available MQ queue manager list is determined and cached; next, receiving a message sent by the application system through the MQ queue, selecting an MQ queue manager from the cache by using the MQ queue manager group, and sending the message to a corresponding channel through the selected MQ queue manager; and finally, monitoring a message sending result, and replacing a new MQ queue manager to resend the message according to an exception handling mechanism when the message sending is abnormal. The invention manages a plurality of MQ instances uniformly from the application angle, provides a high-availability solution of MQ multi-instance, and realizes the configuration and message transmission of the multi-MQ queue manager by application. Scheduling management is carried out on a plurality of MQ queue managers by adding an MQ queue manager group, and an exception handling mechanism is introduced under the abnormal condition; the high availability mechanism aiming at a plurality of MQ instances is realized, and the mechanism is designed aiming at the abnormity in the message sending process; an exception handling mechanism; the method meets the hard requirement of high availability of the MQ in the production environment, has relatively low development invasiveness, can be realized by configuring the MQ queue management group only, and has low application modification cost.
In the embodiment of the invention, the related professional vocabularies are as follows:
message: in MQ, data transmitted by an application program through MQ is defined as a message, and the content of the message can be defined and the message can be understood in a broad sense, such as: various types of data files of a user, processing requests from an application to other applications, and the like may be used as messages. The message is composed of two parts: message descriptors, describing the characteristics of a message, such as: the priority, life cycle, message Id, etc. of the message; the message body, i.e. the user data part. In MQ, messages are divided into two types, non-persistent messages and persistent messages, the non-persistent messages being stored in memory and designed to improve performance and are not recoverable when the system is powered down or the MQ queue manager is restarted. The permanent message is stored on the hard disk, and has high reliability, and can ensure no loss and no duplication of the message under the condition of network and system failure and the like.
And (3) queue: a queue is a container that holds messages.
Queue manager is a top-level concept in MQ systems that provides queue-based message services.
Fig. 3 is an MQ multi-instance architecture diagram of an MQ multi-instance message sending method according to an embodiment of the present invention, as shown in fig. 3, when an application system connects MQ multi-instances, a queue needs to be configured, a plurality of corresponding queue managers need to be configured in the queue, and the queue manager configures information such as IP addresses, ports, and channels of the MQ instances. When the application system sends the message, the message is sent to the appointed channel by using the MQ queue managers to manage the configured MQ queue managers. When the MQ multi-instance message sending method provided by the embodiment of the present invention is implemented specifically, in an embodiment, the method may include:
configuring a plurality of MQ queue managers to join an MQ queue manager group;
when an application system sends a message, an MQ queue manager is selected through an MQ queue manager group, and an available MQ queue manager list is determined and cached;
the method comprises the steps that messages sent by an application system are received through an MQ queue, an MQ queue manager is selected from a cache by utilizing an MQ queue manager group, and the messages are sent to a corresponding channel through the selected MQ queue manager;
monitoring the message sending result, and when the message sending is abnormal, replacing a new MQ queue manager according to an abnormality processing mechanism to resend the message.
The invention realizes the configuration and the message sending of the multi-queue manager through application. After the scheme of the embodiment of the invention is adopted, the application system still adopts the configured queue to transmit the message, but a plurality of MQ queue managers are configured inside, the queue can judge whether the currently connected queue manager is effective according to the transmission state of the message, if the currently connected queue manager is in an invalid state, the queue selects another queue manager to retransmit the message. The method meets the hard requirement of high availability of the MQ in the production environment, has relatively low development invasiveness, can be realized by configuring the MQ queue management group only, and has low application modification cost.
When the MQ multi-instance message sending method provided by the embodiment of the present invention is specifically implemented, in an embodiment, configuring a plurality of MQ queue managers to join an MQ queue manager group includes: in order to ensure the high availability of the MQ multi-instance, the original MQ queue is reformed, an MQ queue manager group is added, each queue manager is managed by the queue manager group, and a plurality of MQ queue managers are added into the MQ queue manager group by configuring, and are configured at first to serve as the basis for subsequently selecting the MQ queue managers.
In one embodiment, when an application system sends a message, the MQ queue manager group selects an MQ queue manager to determine and cache an available MQ queue manager list, where the method includes:
when the application system sends a message, triggering a selection action, selecting an MQ queue manager through an MQ queue manager group according to the sent message, determining an available MQ queue manager list and caching; in the cache, a list of available MQ queue managers is stored; wherein the MQ queue managers in the available MQ queue manager list can be increased or decreased, and the control of the number is realized by the MQ queue manager group.
Fig. 4 is a flowchart of a scheduling mechanism of a queue manager group of an MQ multi-instance message sending method according to an embodiment of the present invention, as shown in fig. 4, when an MQ multi-instance message sending method provided by the embodiment of the present invention is implemented specifically, in an embodiment, an MQ queue manager is selected from a buffer by using the MQ queue manager group, and a message is sent to a corresponding channel through the selected MQ queue manager, where the method includes:
searching an available MQ queue manager list from a cache by utilizing the MQ queue manager group;
when the available MQ queue manager list exists in the cache, one MQ queue manager is selected from the available MQ queue manager list, and the message is sent to the corresponding channel through the selected MQ queue manager.
In the embodiment, messages sent by an application system are received through an MQ queue, an available MQ queue manager list is searched from a cache by using an MQ queue manager group, and the available MQ queue manager list exists in the cache
Then, an MQ queue manager is randomly selected from the available MQ queue manager list, and the message is selected
To the corresponding channel. The invention schedules and manages the MQ multi-queue manager by adding the queue manager group.
Further, as shown in fig. 4, when the MQ multi-instance message sending method provided by the embodiment of the present invention is implemented specifically, in an embodiment, the method further includes:
when the available MQ queue manager list does not exist in the cache, selecting one MQ queue manager from a plurality of MQ queue managers configured in the MQ queue manager group, and sending the message to a corresponding channel through the selected MQ queue manager;
and when the message is successfully sent, adding the selected MQ queue manager into the available MQ queue manager list and caching.
In the embodiment, when an available MQ queue manager list is searched from a cache by using an MQ queue manager group and the available MQ queue manager list does not exist in the cache, an MQ queue manager needs to be randomly selected from a plurality of MQ queue managers configured in the MQ queue manager group at the moment, and a message is sent to a corresponding channel through the selected MQ queue manager; monitoring the message sending result, and adding the selected MQ queue manager into the available MQ queue manager list and caching when the message is successfully sent, so as to realize the dynamic change of the available MQ queue manager list and ensure that the MQ queue manager in the available MQ queue manager list is in an available state at any time.
From a development perspective, a technician needs to configure and join multiple MQ queue managers into an MQ queue manager group through which the MQ queues select the MQ queue managers when sending messages.
After the scheme is adopted, the application system still adopts the configured queue to transmit the message, but a plurality of MQ queue managers are configured in the application system, the queue can judge whether the currently connected queue manager is effective or not according to the transmission state of the message, if the currently connected queue manager is in an invalid state, the queue selects another queue manager to retransmit the message, and the high availability under the MQ multi-instance is solved.
Fig. 5 is a schematic diagram of an exception message processing and retransmitting process of an MQ multi-instance message transmission method according to an embodiment of the present invention, and as shown in fig. 5, when an MQ multi-instance message transmission method provided by an embodiment of the present invention is implemented specifically, in an embodiment, a message transmission result is monitored, and when a message transmission is abnormal, an MQ queue manager updated according to an exception handling mechanism retransmits a message, where the process includes:
step 501: monitoring a message sending result;
step 502: when the message is successfully sent, marking the selected MQ queue manager to be available;
step 503: when the message is sent abnormally, judging the type of the abnormal message;
step 504: if the type of the abnormal message is system type abnormality, an abnormal rotation mechanism is triggered, and a new MQ queue manager is replaced to resend the message;
step 505: and if the type of the abnormal message is non-system type abnormal, resending the message according to a message resending mechanism.
In an embodiment, when an exception occurs when a queue manager is acquired and a message is sent, it needs to perform respective processing according to the type of the exception, and when processing and retransmitting an exception message, the main process includes:
firstly, monitoring a message sending result; the message sending result is classified as success or abnormity; then, when the message is successfully sent, the selected MQ queue manager is marked to be available; then, when the message is sent abnormally, judging the type of the abnormal message; if the type of the abnormal message is system type abnormality, an abnormal rotation mechanism is triggered, and a new MQ queue manager is replaced to resend the message; and finally, if the type of the abnormal message is non-system type abnormal, resending the message according to a message resending mechanism.
The embodiment of the invention focuses on realizing a high availability mechanism aiming at a plurality of MQ instances through application and designing exception handling aiming at exceptions in a message sending process, and the design and development of an exception rotation mechanism and a message retransmission mechanism are the most difficult for the invention. To implement the high availability characteristic of MQ multiple instances through an application, the multiple instances need to be scheduled and managed through an MQ queue manager group, and the high availability is implemented through exception handling and message retransmission, which is the most core flow of the embodiment of the present invention. The high availability characteristic of the MQ multiple instances realized by the application in the embodiment of the invention, wherein the scheduling management of the MQ multiple instances, the mechanism and the cooperation of the abnormal message processing and the message retransmission module are the key points of the embodiment of the invention.
Fig. 6 is a flowchart of an exception round robin mechanism of an MQ multi-instance message sending method according to an embodiment of the present invention, and as shown in fig. 6, when an MQ multi-instance message sending method provided by an embodiment of the present invention is specifically implemented, in an embodiment, if a type of an exception message is a system class exception, an exception round robin mechanism is triggered, and a new MQ queue manager is replaced to resend the message, where the exception round robin mechanism includes:
if the type of the abnormal message is system type abnormity including connection rejection and connection overtime, triggering an abnormal rotation mechanism, and judging whether an available MQ queue manager exists in the cache;
when an available MQ queue manager exists in the cache, selecting an MQ queue manager as a new MQ queue manager, and deleting the abnormal MQ queue manager;
resending the message to the corresponding channel by using the new MQ queue manager;
when no available MQ queue manager exists in the buffer, selecting one MQ queue manager from a plurality of configured MQ queue managers in the MQ queue manager group as a new MQ queue manager,
and utilizing the new MQ queue manager to try to resend the message to the corresponding channel until the message is successfully sent, and adding the new MQ queue manager to the available MQ queue manager list and caching.
In the embodiment, if the exception is a system exception such as connection rejection, connection timeout and the like, an exception rotation mechanism is triggered, and whether an available MQ queue manager exists in the cache is judged; when an available MQ queue manager exists in the cache, an MQ queue manager is randomly selected as a new MQ queue manager, and the abnormal MQ queue manager is deleted by utilizing an MQ queue manager group, so that the MQ queue manager in the cache is ensured to be in an available state, the unavailable MQ queue manager is eliminated, the message sending efficiency is improved, and the waste of system resources is reduced; resending the message to the corresponding channel by using the new MQ queue manager; when the available MQ queue manager does not exist in the buffer memory, one MQ queue manager is randomly selected from a plurality of MQ queue managers configured in the MQ queue manager group to serve as a new MQ queue manager, the new MQ queue manager is utilized to try to resend the message to the corresponding channel until the message is successfully sent, the new MQ queue manager is added into the available MQ queue manager list by the MQ queue manager group and buffered, and the dynamic increase and decrease of the available MQ queue manager list are realized.
If the abnormality is system abnormality such as connection rejection, connection overtime and the like, triggering a rotation mechanism, namely judging whether an available queue manager exists in the cache, and if the available queue manager exists, randomly selecting a next queue manager and deleting the manager with the problem; if no available queue manager exists in the buffer, one queue manager is randomly selected from the configuration and is tried to be sent again until the queue manager is buffered after the transmission is successful.
And if the exception is a non-system exception, entering a message retransmission mechanism.
In an embodiment, when the MQ multi-instance message sending method provided by the embodiment of the present invention is specifically implemented, if the type of the abnormal message is a non-system-class abnormality, the message is resent according to a message resending mechanism, including:
if the type of the abnormal message is non-system type abnormal, the message is introduced into a message retransmission queue;
and retransmitting the message to the corresponding channel according to the retransmission strategy configured by the message retransmission mechanism.
In the embodiment, for non-system type exception occurring when sending a message, the message enters a message retransmission queue, and an application will process according to a configured retransmission policy, where general policies include maximum number of retransmissions and infinite retransmissions.
When the MQ multi-instance message sending method provided by the embodiment of the present invention is implemented specifically, in an embodiment, when the retransmission policy is the maximum number of retransmissions,
setting a message retransmission time threshold, gradually increasing the interval of each retransmission according to a set multiplying factor, and if the message transmission time reaches the message retransmission time threshold and each time is a non-system type exception, throwing an exception outwards and processing the next message.
In the embodiment, the maximum retransmission is that the transmission is tried for the specified number of times, the retransmission interval of each time is gradually increased, if the maximum retransmission is reached and each time is a non-system type exception, the exception is thrown outwards, and then the next message is processed.
When the MQ multi-instance message sending method provided by the embodiment of the present invention is implemented, in an embodiment, when the retransmission policy is infinite retransmission,
and repeatedly sending the message until the message is successfully sent to the corresponding channel or system type abnormity occurs.
The message is repeatedly sent without limit until success or system class exception occurs.
After the scheme is adopted, the application system still adopts the configured queue to transmit the message, but a plurality of MQ queue managers are configured in the application system, the queue can judge whether the currently connected queue manager is effective or not according to the transmission state of the message, if the currently connected queue manager is in an invalid state, the queue selects another queue manager to retransmit the message, and the high availability under the MQ multiple instances is solved; scheduling management is carried out on a plurality of MQ queue managers by adding an MQ queue manager group, and a queue manager rotation and message retransmission mechanism is introduced under the abnormal condition; the high availability mechanism aiming at a plurality of MQ instances is realized, and an exception handling and message retransmission mechanism is designed aiming at the exception in the message sending process; the method meets the hard requirement of high availability of the MQ in the production environment, has relatively low development invasiveness, can be realized by configuring the MQ queue management group only, and has low application modification cost.
The following briefly describes, in conjunction with a specific scenario, an MQ multi-instance message sending method provided by the embodiment of the present invention:
the invention manages a plurality of MQ instances uniformly from the application point of view, and provides a high-availability solution of MQ multi-instance. The invention realizes the configuration and the message sending of the multi-queue manager through application.
After the scheme is adopted, the application system still adopts the configured queue to transmit the message, but a plurality of MQ queue managers are configured inside, the queue can judge whether the currently connected queue manager is valid according to the transmission state of the message, if the currently connected queue manager is in an invalid state, the queue selects another queue manager to retransmit the message.
The invention schedules and manages the MQ multi-queue manager by adding the queue manager group, introduces a queue manager rotation and message retransmission mechanism under the abnormal condition, and mainly comprises the following mechanisms:
scheduling mechanism for MQ multi-instances:
in order to ensure the high availability of MQ multiple instances, an original MQ queue is reformed, an MQ queue manager group is added, each queue manager is managed by the queue manager group, when the application sends a message, if an available MQ queue manager list is stored in a cache, an MQ queue manager is randomly selected from the cache, if the available MQ queue manager list does not exist in the cache, a configured queue manager is randomly selected, the queue manager in an available state is cached according to the result of sending the message, and the exception handling mechanism is entered when the message is failed to be sent.
2. Round robin mechanism in abnormal situations:
when an exception occurs when a queue manager is acquired and a message is sent, the exception needs to be respectively processed according to the types of the exceptions:
if the abnormality is system abnormality such as connection rejection, connection overtime and the like, triggering a rotation mechanism, namely judging whether an available queue manager exists in the cache, and if the available queue manager exists, randomly selecting a next queue manager and deleting the manager with the problem; if no available queue manager exists in the buffer, one queue manager is randomly selected from the configuration and is tried to be sent again until the queue manager is buffered after the transmission is successful.
And if the exception is a non-system exception, entering a message retransmission mechanism.
3. Message retransmission mechanism:
the method comprises the steps that a message is sent into a message retransmission queue aiming at non-system type abnormity appearing during message sending, application is processed according to a configured retransmission strategy, the general strategy comprises maximum retransmission and infinite retransmission, the maximum retransmission is the appointed number of times of trying to send, the retransmission interval of each time is gradually increased, if the maximum retransmission is reached, the non-system type abnormity is processed every time, the next message is processed after the abnormity is thrown outwards; the message is repeatedly sent without limit until success or system class exception occurs.
From a development perspective, the technician need only configure and join multiple MQ queue managers in the MQ queue manager group through which the MQ queue selects the MQ queue manager when sending messages.
The essence of the embodiment of the invention is that a high availability mechanism for a plurality of MQ instances is realized through application, and an exception handling and message retransmission mechanism is designed for exceptions in the message sending process, and the design and development of the mechanisms are the most difficult for the invention.
The embodiment of the invention also provides a modular instance of the MQ multi-instance message sending method, which comprises an MQ multi-instance scheduling module, a message exception processing module and a message retransmitting module.
The MQ multi-instance scheduling module finishes the selection and the caching of a plurality of MQ queue managers, and enters a message exception handling module for exception handling after the message is sent by the selected queue manager, the exception handling module respectively handles different exceptions, and the message retransmitting module enters the message retransmitting module for message retransmission according to a retransmission strategy if the message is not system-class exception; and the system class exception enters a rotation mechanism, namely a next MQ queue manager is selected through a buffer memory or a group of MQ queue managers in the configuration and the message is sent.
The embodiment of the invention realizes the high availability characteristic of the MQ multi-instance through application, and is specifically completed through the cooperation of the MQ multi-instance scheduling module, the message exception processing module and the message retransmission module.
The method comprises the following steps that 1, an MQ multi-instance scheduling module receives messages sent by an application, an MQ queue manager is selected to send the messages, and an available MQ queue manager list is cached;
2. after receiving the message and the abnormal information, the message exception processing module selects the next available MQ queue manager for sending the system class exception according to the type of the abnormal information and caches the message after the message exception processing module succeeds; a module for retransmitting the abnormal entry message of the non-system class;
3. the message retransmission module receives the message to be retransmitted and retransmits the message according to the retransmission strategy;
to realize the high availability characteristic of MQ multi-instance through application, the multi-instance needs to be scheduled and managed through an MQ queue manager group, and the high availability is realized through exception handling and message retransmission, which is the most core flow of the invention.
The high availability characteristic of the MQ multi-instance realized by the application in the embodiment of the invention, wherein the scheduling management of the MQ multi-instance, the mechanism and the cooperation of the abnormal message processing and the message retransmission module are the key points of the invention.
The embodiment of the invention realizes a novel MQ multi-instance framework through a series of mechanisms and module series connection. The advantages are as follows: the hard requirement of high availability of MQ in the production environment is met; the development intrusiveness is relatively low, and the development only needs to configure the MQ queue management group; the method is realized by a bottom layer frame, and the application reconstruction cost is low.
Fig. 7 is a schematic diagram of a computer apparatus for operating an MQ multi-instance message sending method implemented by the present invention, and as shown in fig. 7, an embodiment of the present invention further provides a computer device including a memory, a processor, and a computer program stored in the memory and operable on the processor, where the processor implements the MQ multi-instance message sending method when executing the computer program.
An embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program for executing the MQ multi-instance message sending method.
An embodiment of the present invention further provides an MQ multi-instance message sending apparatus, as described in the following embodiments. Because the principle of the device for solving the problem is similar to that of an MQ multi-instance message sending method, the implementation of the device can refer to the implementation of the MQ multi-instance message sending method, and repeated parts are not described again.
Fig. 8 is a schematic diagram of an MQ multi-instance message sending device according to an embodiment of the present invention, and as shown in fig. 8, an MQ multi-instance message sending device according to an embodiment of the present invention may be implemented by:
an MQ queue manager configuring module 801, configured to configure a plurality of MQ queue managers to join an MQ queue manager group;
an available MQ queue manager list determining module 802, configured to select an MQ queue manager through an MQ queue manager group when an application system sends a message, determine an available MQ queue manager list, and cache the available MQ queue manager list;
the MQ multi-instance scheduling module 803 is configured to receive a message sent from the application system through an MQ queue, select an MQ queue manager from the cache by using an MQ queue manager group, and send the message to a corresponding channel through the selected MQ queue manager;
and the abnormal message processing and retransmitting module 804 is configured to monitor a message sending result, and when the message sending is abnormal, replace the new MQ queue manager to retransmit the message according to the abnormal processing mechanism.
In an embodiment, when the MQ multi-instance message sending device provided by the embodiment of the present invention is implemented specifically, the MQ multi-instance scheduling module is specifically configured to:
searching an available MQ queue manager list from a cache by utilizing the MQ queue manager group;
when the available MQ queue manager list exists in the cache, one MQ queue manager is selected from the available MQ queue manager list, and the message is sent to the corresponding channel through the selected MQ queue manager.
In an embodiment, when the MQ multi-instance message sending device provided by the embodiment of the present invention is implemented specifically, the MQ multi-instance scheduling module is further configured to:
when the available MQ queue manager list does not exist in the cache, selecting one MQ queue manager from a plurality of MQ queue managers configured in the MQ queue manager group, and sending the message to a corresponding channel through the selected MQ queue manager;
and when the message is successfully sent, adding the selected MQ queue manager into the available MQ queue manager list and caching.
In an embodiment of an MQ multi-instance message sending device according to an embodiment of the present invention, when the MQ multi-instance message sending device is implemented, the exception message processing resending module is specifically configured to:
monitoring a message sending result;
when the message is successfully sent, marking the selected MQ queue manager to be available;
when the message is sent abnormally, judging the type of the abnormal message;
if the type of the abnormal message is system type abnormality, an abnormal rotation mechanism is triggered, and a new MQ queue manager is replaced to resend the message;
and if the type of the abnormal message is non-system type abnormal, resending the message according to a message resending mechanism.
In an embodiment of the MQ multi-instance message sending device according to the embodiment of the present invention, when the MQ multi-instance message sending device is implemented specifically, the exception message handling retransmission module is further configured to:
if the type of the abnormal message is system type abnormity including connection rejection and connection overtime, triggering an abnormal rotation mechanism, and judging whether an available MQ queue manager exists in the cache;
when an available MQ queue manager exists in the cache, selecting an MQ queue manager as a new MQ queue manager, and deleting the abnormal MQ queue manager;
resending the message to the corresponding channel by using the new MQ queue manager;
when no available MQ queue manager exists in the buffer, selecting one MQ queue manager from a plurality of configured MQ queue managers in the MQ queue manager group as a new MQ queue manager,
and utilizing the new MQ queue manager to try to resend the message to the corresponding channel until the message is successfully sent, and adding the new MQ queue manager to the available MQ queue manager list and caching.
In an embodiment of the MQ multi-instance message sending device according to the embodiment of the present invention, when the MQ multi-instance message sending device is implemented specifically, the exception message handling retransmission module is further configured to:
if the type of the abnormal message is non-system type abnormal, the message is introduced into a message retransmission queue;
and retransmitting the message to the corresponding channel according to the retransmission strategy configured by the message retransmission mechanism.
In an embodiment of the MQ multi-instance message sending device according to the embodiment of the present invention, when the MQ multi-instance message sending device is implemented specifically, the exception message handling retransmission module is further configured to: when the retransmission strategy is the maximum number of retransmissions,
setting a message retransmission time threshold, gradually increasing the interval of each retransmission according to a set multiplying factor, and if the message transmission time reaches the message retransmission time threshold and each time is a non-system type exception, throwing an exception outwards and processing the next message.
In an embodiment of the MQ multi-instance message sending device according to the embodiment of the present invention, when the MQ multi-instance message sending device is implemented specifically, the exception message handling retransmission module is further configured to: when the retransmission strategy is an infinite retransmission,
and repeatedly sending the message until the message is successfully sent to the corresponding channel or system type abnormity occurs.
To sum up, an MQ multi-instance message sending method and apparatus provided by the embodiments of the present invention include: firstly, configuring a plurality of MQ queue managers to join an MQ queue manager group; then when the application system sends a message, an MQ queue manager is selected through an MQ queue manager group, and an available MQ queue manager list is determined and cached; next, receiving a message sent by the application system through the MQ queue, selecting an MQ queue manager from the cache by using the MQ queue manager group, and sending the message to a corresponding channel through the selected MQ queue manager; and finally, monitoring a message sending result, and replacing a new MQ queue manager to resend the message according to an exception handling mechanism when the message sending is abnormal. The invention manages a plurality of MQ instances uniformly from the application angle, provides a high-availability solution of MQ multi-instance, and realizes the configuration and message transmission of the multi-MQ queue manager by application. Scheduling management is carried out on a plurality of MQ queue managers by adding an MQ queue manager group, and an exception handling mechanism is introduced under the abnormal condition; the high availability mechanism aiming at a plurality of MQ instances is realized, and the mechanism is designed aiming at the abnormity in the message sending process; an exception handling mechanism; the method meets the hard requirement of high availability of the MQ in the production environment, has relatively low development invasiveness, can be realized by configuring the MQ queue management group only, and has low application modification cost.
After the scheme of the embodiment of the invention is adopted, the application system still adopts the configured queue to send the message, but a plurality of MQ queue managers are configured in the application system, the queue can judge whether the currently connected queue manager is effective according to the sending state of the message, if the currently connected queue manager is in an invalid state, the queue selects another queue manager to resend the message, and the problem of high availability under multiple instances of MQ is solved; scheduling management is carried out on a plurality of MQ queue managers by adding an MQ queue manager group, and a queue manager rotation and message retransmission mechanism is introduced under the abnormal condition; the high availability mechanism aiming at a plurality of MQ instances is realized, and an exception handling and message retransmission mechanism is designed aiming at the exception in the message sending process; the method meets the hard requirement of high availability of the MQ in the production environment, has relatively low development invasiveness, can be realized by configuring the MQ queue management group only, and has low application modification cost.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are only exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (18)
1. An MQ multi-instance messaging method, comprising:
configuring a plurality of MQ queue managers to join an MQ queue manager group;
when an application system sends a message, an MQ queue manager is selected through an MQ queue manager group, and an available MQ queue manager list is determined and cached;
the method comprises the steps that messages sent by an application system are received through an MQ queue, an MQ queue manager is selected from a cache by utilizing an MQ queue manager group, and the messages are sent to a corresponding channel through the selected MQ queue manager;
monitoring the message sending result, and when the message sending is abnormal, replacing a new MQ queue manager according to an abnormality processing mechanism to resend the message.
2. The method as claimed in claim 1, wherein selecting the MQ queue manager from the cache using the MQ queue manager group, and sending the message to the corresponding channel through the selected MQ queue manager comprises:
searching an available MQ queue manager list from a cache by utilizing the MQ queue manager group;
when the available MQ queue manager list exists in the cache, one MQ queue manager is selected from the available MQ queue manager list, and the message is sent to the corresponding channel through the selected MQ queue manager.
3. The method of claim 2, further comprising:
when the available MQ queue manager list does not exist in the cache, selecting one MQ queue manager from a plurality of MQ queue managers configured in the MQ queue manager group, and sending the message to a corresponding channel through the selected MQ queue manager;
and when the message is successfully sent, adding the selected MQ queue manager into the available MQ queue manager list and caching.
4. The method of claim 2, wherein monitoring the messaging results, and in the event of an exception to the messaging, renewing the MQ queue manager to resend the message in accordance with the exception handling mechanism comprises:
monitoring a message sending result;
when the message is successfully sent, marking the selected MQ queue manager to be available;
when the message is sent abnormally, judging the type of the abnormal message;
if the type of the abnormal message is system type abnormality, an abnormal rotation mechanism is triggered, and a new MQ queue manager is replaced to resend the message;
and if the type of the abnormal message is non-system type abnormal, resending the message according to a message resending mechanism.
5. The method of claim 4, wherein triggering the exception rotation mechanism to replace the new MQ queue manager to resend the message if the type of exception message is a system-class exception comprises:
if the type of the abnormal message is system type abnormity including connection rejection and connection overtime, triggering an abnormal rotation mechanism, and judging whether an available MQ queue manager exists in the cache;
when an available MQ queue manager exists in the cache, selecting an MQ queue manager as a new MQ queue manager, and deleting the abnormal MQ queue manager;
resending the message to the corresponding channel by using the new MQ queue manager;
when no available MQ queue manager exists in the buffer, selecting one MQ queue manager from a plurality of configured MQ queue managers in the MQ queue manager group as a new MQ queue manager,
and utilizing the new MQ queue manager to try to resend the message to the corresponding channel until the message is successfully sent, and adding the new MQ queue manager to the available MQ queue manager list and caching.
6. The method of claim 4, wherein if the type of the exception message is a non-system-class exception, resending the message according to a message resending mechanism, comprises:
if the type of the abnormal message is non-system type abnormal, the message is introduced into a message retransmission queue;
and retransmitting the message to the corresponding channel according to the retransmission strategy configured by the message retransmission mechanism.
7. The method of claim 6, wherein when the retransmission policy is a maximum number of retransmissions,
setting a message retransmission time threshold, gradually increasing the interval of each retransmission according to a set multiplying factor, and if the message transmission time reaches the message retransmission time threshold and each time is a non-system type exception, throwing an exception outwards and processing the next message.
8. The method of claim 6, wherein when the retransmission policy is an infinite retransmission,
and repeatedly sending the message until the message is successfully sent to the corresponding channel or system type abnormity occurs.
9. An MQ multi-instance messaging device, comprising:
the MQ queue manager configuration module is used for configuring a plurality of MQ queue managers to join the MQ queue manager group;
the available MQ queue manager list determining module is used for selecting an MQ queue manager through an MQ queue manager group when the application system sends a message, determining an available MQ queue manager list and caching the available MQ queue manager list;
the MQ multi-instance scheduling module is used for receiving the message sent by the application system through the MQ queue, selecting an MQ queue manager from the cache by utilizing the MQ queue manager group, and sending the message to a corresponding channel through the selected MQ queue manager;
and the abnormal message processing and retransmitting module is used for monitoring the message sending result and updating a new MQ queue manager to retransmit the message according to the abnormal processing mechanism when the message sending is abnormal.
10. The apparatus as in claim 9, wherein the MQ multi-instance scheduling module is to:
searching an available MQ queue manager list from a cache by utilizing the MQ queue manager group;
when the available MQ queue manager list exists in the cache, one MQ queue manager is selected from the available MQ queue manager list, and the message is sent to the corresponding channel through the selected MQ queue manager.
11. The apparatus of claim 10, wherein the MQ multi-instance scheduling module is further to:
when the available MQ queue manager list does not exist in the cache, selecting one MQ queue manager from a plurality of MQ queue managers configured in the MQ queue manager group, and sending the message to a corresponding channel through the selected MQ queue manager;
and when the message is successfully sent, adding the selected MQ queue manager into the available MQ queue manager list and caching.
12. The apparatus of claim 10, wherein the exception message handling retransmission module is specifically configured to:
monitoring a message sending result;
when the message is successfully sent, marking the selected MQ queue manager to be available;
when the message is sent abnormally, judging the type of the abnormal message;
if the type of the abnormal message is system type abnormality, an abnormal rotation mechanism is triggered, and a new MQ queue manager is replaced to resend the message;
and if the type of the abnormal message is non-system type abnormal, resending the message according to a message resending mechanism.
13. The apparatus of claim 12, wherein the exception message handling retransmission module is further configured to:
if the type of the abnormal message is system type abnormity including connection rejection and connection overtime, triggering an abnormal rotation mechanism, and judging whether an available MQ queue manager exists in the cache;
when an available MQ queue manager exists in the cache, selecting an MQ queue manager as a new MQ queue manager, and deleting the abnormal MQ queue manager;
resending the message to the corresponding channel by using the new MQ queue manager;
when no available MQ queue manager exists in the buffer, selecting one MQ queue manager from a plurality of configured MQ queue managers in the MQ queue manager group as a new MQ queue manager,
and utilizing the new MQ queue manager to try to resend the message to the corresponding channel until the message is successfully sent, and adding the new MQ queue manager to the available MQ queue manager list and caching.
14. The apparatus of claim 12, wherein the exception message handling retransmission module is further configured to:
if the type of the abnormal message is non-system type abnormal, the message is introduced into a message retransmission queue;
and retransmitting the message to the corresponding channel according to the retransmission strategy configured by the message retransmission mechanism.
15. The apparatus of claim 14, wherein the exception message handling retransmission module is further configured to: when the retransmission strategy is the maximum number of retransmissions,
setting a message retransmission time threshold, gradually increasing the interval of each retransmission according to a set multiplying factor, and if the message transmission time reaches the message retransmission time threshold and each time is a non-system type exception, throwing an exception outwards and processing the next message.
16. The apparatus of claim 14, wherein the exception message handling retransmission module is further configured to: when the retransmission strategy is an infinite retransmission,
and repeatedly sending the message until the message is successfully sent to the corresponding channel or system type abnormity occurs.
17. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of any of claims 1 to 8 when executing the computer program.
18. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program for executing a method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111247686.0A CN113961372A (en) | 2021-10-26 | 2021-10-26 | MQ multi-instance message sending method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111247686.0A CN113961372A (en) | 2021-10-26 | 2021-10-26 | MQ multi-instance message sending method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113961372A true CN113961372A (en) | 2022-01-21 |
Family
ID=79467011
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111247686.0A Pending CN113961372A (en) | 2021-10-26 | 2021-10-26 | MQ multi-instance message sending method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113961372A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114710311A (en) * | 2022-02-11 | 2022-07-05 | 浙江高信技术股份有限公司 | Multi-project message management method and system |
-
2021
- 2021-10-26 CN CN202111247686.0A patent/CN113961372A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114710311A (en) * | 2022-02-11 | 2022-07-05 | 浙江高信技术股份有限公司 | Multi-project message management method and system |
| CN114710311B (en) * | 2022-02-11 | 2024-01-02 | 浙江高信技术股份有限公司 | Multi-project message management method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7945819B2 (en) | Message state maintenance at a message log | |
| US7676580B2 (en) | Message delivery with configurable assurances and features between two endpoints | |
| US10601754B2 (en) | Message delivery system using message metadata | |
| RU2345408C2 (en) | Improvement of availability and scalability in system of message transfer by method, transparent for application | |
| JP3932994B2 (en) | Server handover system and method | |
| CA2547880C (en) | Improved distributed kernel operating system | |
| EP1729465A2 (en) | Distributed kernel operating system | |
| JP2004520640A (en) | Method and apparatus used in an asynchronous message exchange system | |
| CN113961372A (en) | MQ multi-instance message sending method and device | |
| US20110060957A1 (en) | Reliable Messaging Using Publish Subscribe Mechanism | |
| CN108880994B (en) | Method and device for retransmitting mails | |
| CN113992433B (en) | Network equipment concurrency fuzzy test method and device based on variation strategy | |
| CN117097597A (en) | Resource state information pushing method, system, electronic equipment and storage medium | |
| Kümmel et al. | An environment-aware transport mechanism for the mobile workforce |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |