CN114090297A - Service message processing method and related device - Google Patents
Service message processing method and related device Download PDFInfo
- Publication number
- CN114090297A CN114090297A CN202111407374.1A CN202111407374A CN114090297A CN 114090297 A CN114090297 A CN 114090297A CN 202111407374 A CN202111407374 A CN 202111407374A CN 114090297 A CN114090297 A CN 114090297A
- Authority
- CN
- China
- Prior art keywords
- message
- exception
- preset
- thread pool
- processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
Abstract
The application discloses a method for processing service messages and a related device, and relates to the technical field of data processing. Therefore, the storage of the message and the buffering of the system during the sudden service increase are realized. The method and the device have the advantages that the corresponding preset exception is set according to each node in the processing flow, and the processing flow of the message is monitored to determine whether the preset exception is sent or not in the process of inputting the message into the thread pool. The message queue of the message with the irretrievable exception is removed by classifying the exception types of each node in the message processing process, so that the processing result of the message can be accurately obtained, and the processing speed of the message is further improved.
Description
Technical Field
The present invention relates to the field of data processing, and in particular, to a method and a related apparatus for processing a service message.
Background
With the development of social informatization, banking business gradually realizes digitization. Aiming at low fault tolerance services such as bills and the like, a bank system mostly adopts a synchronous online transaction mode to acquire a real-time and accurate message circulation state. However, in practical scenarios, there are situations where the ticketing services are traded centrally, i.e. market participants tend to trade centrally for a period of time before the time of the day of the rest, resulting in a large number of trades occurring during that period. If the downstream system does not provide a reasonable flow control and transaction degradation mechanism, sudden pressure is applied to the downstream system, and thus messages in the memory are lost.
In the related technology, the database persistence is performed on the message in advance, and the database is scanned through a timing and quantitative strategy, so that the impact of the instantaneous peak value of the transaction amount on the system is relieved. Although the method can effectively avoid the loss of the message, the method not only causes a large number of database connection pools to be occupied, but also causes the phenomenon of waiting for message processing during the occurrence of a transaction peak value, and limits the capability of the system for dynamically adjusting concurrent processing.
Disclosure of Invention
The embodiment of the application provides a method and a related device for processing a service message, which implement buffer protection on a service system by adding a received message to a message queue for caching. And the processing result of the message is determined by classifying the abnormal type of each node in the message processing process, so that the processing speed of the message is improved.
In a first aspect, an embodiment of the present application provides a method for processing a service message, where the method includes:
after adding the received message to a message queue, processing the message and monitoring the processing flow of the message; the processing flow is a process of inputting the message in the message queue into a thread pool;
if preset abnormity occurs in the message in the processing process and the class of the preset abnormity is detected to be the unreleasable abnormity, removing the message from the message queue;
if the exception of the message in the processing process is not monitored, the message is recorded into a thread pool and then removed from the message queue.
The embodiment of the application adds the received message to the message queue for caching, and processes the message in the message queue based on the processing capacity of the downstream system. Therefore, the storage of the message and the buffering of the system during the sudden service increase are realized. The method and the device have the advantages that the corresponding preset exception is set according to each node in the processing flow, and the processing flow of the message is monitored to determine whether the preset exception is sent or not in the process of inputting the message into the thread pool. The message queue of the message with the irretrievable exception is removed by classifying the exception types of each node in the message processing process, so that the processing result of the message can be accurately obtained, and the processing speed of the message is further improved.
In some possible embodiments, after monitoring that a preset exception occurs in the processing process of the message, the method further includes:
if the class of the preset exception is detected to be retriable exception, a retry strategy corresponding to the preset exception is adopted to reprocess the message, and exception monitoring is carried out on the reprocessing process.
In the processing flow of monitoring the message, the embodiment of the application sets a corresponding retry strategy for the retry exception. The message is reprocessed by adopting a corresponding retry strategy so that the message is recorded into the thread pool, and the processing speed of the message is improved.
In some possible embodiments, the preset anomalies include at least the following:
the security check of the message fails;
failing to convert the format of the message;
failing to store the message;
and the message is failed to be recorded into the thread pool.
In the embodiment of the application, corresponding preset exceptions are set in each processing node aiming at the processing flow of the message. The method comprises the steps of firstly carrying out safety verification on a newly received message, detecting an exception triggered by failure in verification, carrying out format conversion on the message, detecting an exception triggered by failure in conversion, storing the message, and finally recording an exception triggered by failure when the message is recorded into a thread pool from a message queue.
In some possible embodiments, the detecting the category of the preset anomaly comprises:
if the preset exception is failure in performing security check on the message, determining that the category of the preset exception is an unreleasable exception;
if the preset exception is failure in format conversion of the message, determining that the category of the preset exception is an unreleasable exception;
if the preset exception is that the message is failed to be stored and the field naming rule of the message is detected to be not in accordance with the storage rule, determining that the preset exception is a retry-impossible exception;
if the preset exception is failure in storing the message and the database for storing the message is detected to be in an off-line state, determining that the preset exception is a retry-able exception;
if the preset exception is that the message is failed to be recorded into the thread pool, determining that the type of the preset exception is a retriable exception.
The method and the device for node resetting comprise the steps that corresponding categories are set for each node preset exception, and the categories can comprise retriable exceptions and non-retriable exceptions. The retry-able exception can be avoided in a processing mode so that the message is successfully recorded into the thread pool. For the reason that the unrepeatable exception is that the self format, specification and the like cannot be processed by the system, the message needs to be removed from the message queue to improve the processing rate.
In some possible embodiments, the reprocessing the message by using the retry policy corresponding to the preset exception includes:
if the preset exception is that the message is failed to be stored and the database for storing the message is detected to be in an off-line state, storing the message into the database after controlling the database to be on-line;
if the preset exception is that the message is failed to be recorded into the thread pool, the message is recorded into the thread pool again after the thread pool is detected to have idle resources.
In the embodiment of the application, if the off-line of the database is detected when the message is stored in the database, a preset exception of which the category is retry exception occurs. The exception can be avoided by restarting the database and logging the message again. In the embodiment of the application, when the message is recorded into the thread pool from the message queue, if the recording fails, it indicates that no idle resource exists in the thread pool. And re-recording the message into the thread pool after waiting for the thread pool to have idle resources so as to avoid the exception.
In some possible embodiments, before the reprocessing the message by using the retry policy corresponding to the preset exception, the method further includes:
and after determining that the category of the preset exception is a retriable exception, stopping adding a new message to the message queue.
According to the embodiment of the application, after the preset exception is determined to be the retriable exception, the exception can be avoided by reprocessing the message. Before the message is reprocessed, new message is added into the message queue, and the pressure of a large number of requests on a service system is relieved by expanding the time interval for receiving the message.
In some possible embodiments, before performing the entering the message into a thread pool, the second removal module is further configured to:
determining that the message is not stored in a database;
the second removal module is further configured to:
if the message information is detected to be stored in the database, removing the message information from the message queue; wherein, the message in the database is added before the message is recorded into the thread pool, so as to determine that the message triggering the recovery mechanism is not recorded into the thread pool again; the recovery mechanism is used for putting the message into the thread pool after the message to be put into the thread pool is lost.
In the embodiment of the application, the message information in the message queue needs to be recorded into the database for storage before being recorded into the thread pool. When the message messages to be put into the thread pool are lost due to the problems of system downtime, breakdown and the like when the message messages are input into the thread pool, the lost message messages in the input process can be input into the thread pool again in the message queue through a recovery mechanism. Because the recovery mechanism and the message queue are mutually opposite operation flows, the message is not known to be recorded into the thread pool from the perspective of the message queue. Therefore, by querying the database, if the message is stored in the database, it is represented that the message has been processed by the recovery mechanism, and the message does not need to be processed again. Therefore, the problem of repeatedly recording the message into the thread is solved.
In a second aspect, an embodiment of the present application provides an apparatus for processing a service message, where the apparatus includes:
the flow monitoring module is configured to add the received message to a message queue, process the message and monitor the processing flow of the message; the processing flow is a process of inputting the message in the message queue into a thread pool;
the first removing module is configured to remove the message from the message queue if preset abnormity occurs in the processing process of the message and the class of the preset abnormity is detected to be the unreleasable abnormity;
and the second removing module is configured to remove the message from the message queue after the message is recorded into a thread pool if the message is not monitored to be abnormal in the processing process.
In some possible embodiments, after the monitoring that a preset exception occurs in the message during the processing process is executed, the process monitoring module is further configured to:
if the type of the preset exception is detected to be retriable exception, a retrying strategy corresponding to the preset exception is adopted to reprocess the message, and exception monitoring is carried out on the reprocessing process.
In some possible embodiments, the preset anomalies include at least the following:
the security check of the message fails;
failing to convert the format of the message;
failing to store the message;
and the message is failed to be recorded into the thread pool.
In some possible embodiments, said detecting of said category of preset anomalies is performed, said first removal module being configured to:
if the preset exception is failure in performing security check on the message, determining that the category of the preset exception is an unreleasable exception;
if the preset exception is failure in format conversion of the message, determining that the category of the preset exception is an unreleasable exception;
if the preset exception is failure in storing the message and the field naming rule of the message is detected to be not accordant with the storage rule, determining that the preset exception is a retry-impossible exception;
if the preset exception is failure in storing the message and the database for storing the message is detected to be in an off-line state, determining that the preset exception is a retriable exception;
if the preset exception is that the message is failed to be recorded into the thread pool, determining that the type of the preset exception is a retriable exception.
In some possible embodiments, the performing the reprocessing of the message by using the retry policy corresponding to the preset exception, wherein the first removing module is configured to:
if the preset exception is that the message is failed to be stored and the database for storing the message is detected to be in an off-line state, the message is reprocessed after the database is controlled to be on-line, and the message is stored in the database;
if the preset exception is that the message is failed to be recorded into the thread pool, the message is reprocessed after the thread pool is detected to have idle resources, and the message is recorded into the thread pool again.
In some possible embodiments, before performing the reprocessing of the message by using the retry policy corresponding to the preset exception, the first removing module is further configured to:
and after determining that the category of the preset exception is a retriable exception, stopping adding a new message to the message queue.
In some possible embodiments, before performing the entering of the message into a thread pool, the method further includes:
determining that the message is not stored in a database;
the method further comprises the following steps:
if the message information is detected to be stored in the database, removing the message information from the message queue; wherein, the message in the database is added before the message is recorded into the thread pool, so as to determine that the message triggering the recovery mechanism is not recorded into the thread pool again; the recovery mechanism is used for putting the message into the thread pool after the message to be put into the thread pool is lost.
In a third aspect, an embodiment of the present application further provides an electronic device, including:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement any of the methods as provided in the first aspect of the application.
In a fourth aspect, embodiments of the present application further provide a computer-readable storage medium, where instructions, when executed by a processor of an electronic device, enable the electronic device to perform any one of the methods as provided in the first aspect of the present application.
Additional features and advantages of the application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the application. The objectives and other advantages of the application may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments of the present application will be briefly described below, and it is obvious that the drawings described below are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a diagram illustrating a database scanning task according to an embodiment of the present application;
fig. 2a is a flowchart illustrating an overall method for processing a service message according to an embodiment of the present application;
FIG. 2b is a schematic diagram of a timed and quantitative scanning strategy according to an embodiment of the present application;
fig. 2c is a schematic diagram illustrating a message processing flow according to an embodiment of the present application;
fig. 2d is a schematic diagram illustrating categories of preset exceptions of each processing node according to the embodiment of the present application;
fig. 3 is a block diagram of a processing apparatus 300 for service messages according to an embodiment of the present application;
fig. 4 is a schematic diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be described in detail and clearly with reference to the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, "a face will mean or means, for example, a/B may mean a or B; "and/or" in the text is only an association relationship describing an associated object, and means that three relationships may exist, for example, a and/or B may mean: three cases of a alone, a and B both, and B alone exist, and in addition, "a plurality" means two or more than two in the description of the embodiments of the present application.
In the description of the embodiments of the present application, the term "plurality" means two or more unless otherwise specified, and other terms and the like should be understood similarly, and the preferred embodiments described herein are only for the purpose of illustrating and explaining the present application, and are not intended to limit the present application, and features in the embodiments and examples of the present application may be combined with each other without conflict.
To further illustrate the technical solutions provided by the embodiments of the present application, the following detailed description is made with reference to the accompanying drawings and the detailed description. Although the embodiments of the present application provide method steps as shown in the following embodiments or figures, more or fewer steps may be included in the method based on conventional or non-inventive efforts. In steps where no necessary causal relationship exists logically, the order of execution of these steps is not limited to the order of execution provided by the embodiments of the present application. The method can be executed in the order of the embodiments or the method shown in the drawings or in parallel in the actual process or the control device.
Aiming at low fault tolerance services such as bills and the like, a bank system mostly adopts a synchronous online transaction mode to acquire a real-time and accurate message circulation state. Specifically, the complete message is put into the large field of the transaction message, and the flow state of the message is accurately mastered in real time in a synchronous waiting mode. If the transaction fails, the upstream system may retry until an acknowledgement is obtained from the downstream system. However, due to the higher complexity of the service, there is a phenomenon that the transaction chain is longer. When the transaction chain of the downstream system for message processing and consumption confirmation is long, the upstream and downstream systems can keep long-time network connection waiting, and more network resources are occupied. In addition, there is a case of centralized transaction in the application scenario of the ticket service. The concentrated trading represents a tendency for market participants to concentrate on trading half an hour or an hour before the break, which results in a large volume of trade leaps over time. If the downstream system processing the transaction message has no reasonable flow control and transaction degradation mechanism, sudden pressure is caused to the downstream system, so that the operation capability and stability of the system are influenced. When the downstream system fails to respond in time and the upstream system goes down, the messages in the memory are lost.
In the related technology, the database persistence is performed on the message in advance, and the database is scanned through a timing and quantitative strategy, so that the impact of the instantaneous peak value of the transaction amount on the system is relieved. Specifically, as shown in fig. 1, the unified message processing cluster preferentially stores the message into the database for persistence, and sets the task state of the message to be the pending state. Furthermore, another service system cluster starts a database timing scanning task thread, starts different scanning threads for different message types, and scans the task to be processed at a certain time interval to execute subsequent service processing. And updating the state of the message according to the service execution result of the message, for example, updating the state of the message with a successful service execution from the state to be processed to the state of the message with a successful service execution. And for the message with failed execution or abnormal execution, the state of the message needs to be updated to be failed or abnormal. And finally, determining whether the scanning task can be called again according to the corresponding state value.
The processing mode can relieve the impact of the instantaneous peak value of the transaction amount on the system through a timing and quantitative strategy, and plays a role in protecting the system. However, with the development of services, the transaction amount and the message types are increasing, and new data scanning threads need to be started continuously, so that a large number of database connection pools are occupied continuously. Moreover, although the timing and quantitative strategy protects the transaction from impact, the message processing waiting phenomenon is caused during the transaction peak value occurrence period, and the capability of the system for dynamically adjusting the concurrent processing is limited.
In order to solve the above problem, in the embodiments of the present application, a message queue is used as a buffering component, and efficient processing of a message is achieved by using the characteristic of high throughput. The invention conception of the application is as follows: the received message is added into the message queue for caching, and the message in the message queue is processed based on the processing capacity of a downstream system. Therefore, the storage of the message and the buffering of the system during the sudden service increase are realized. The method and the device have the advantages that the corresponding preset exception is set according to each node in the processing flow, and the processing flow of the message is monitored to determine whether the preset exception is sent or not in the process of inputting the message into the thread pool. The message queue of the message with the irretrievable exception is removed by classifying the exception types of each node in the message processing process, so that the processing result of the message can be accurately obtained, and the processing speed of the message is further improved.
A method for processing a service message according to an embodiment of the present application is described in detail below with reference to the accompanying drawings. As shown in fig. 2a, the method comprises the following steps:
step 201: after adding the received message to a message queue, processing the message and monitoring the processing flow of the message; the processing flow is a process of inputting the message in the message queue into a thread pool;
as mentioned above, in the prior art, after the message is stored in a storage for a long time, the message to be processed is recorded into the thread pool by scanning the database periodically and quantitatively. The specific method may be, for example, adding a state value to the message in advance, setting the state value to 0 when the message is not recorded in the thread pool, and setting the state value to 1 when the message is recorded in the thread pool. The so-called timing and quantitative scanning strategy is to process the messages in the database by setting the scanning interval time and the maximum processing number, specifically as shown in fig. 2b, for example, 1000 messages are stored in the database, and 800 messages are to-be-processed messages. The 700 messages to be processed, which are preferentially scanned in the database, can be recorded into the thread pool by, for example, scanning the database every 10 seconds with the scanning strategy of which the maximum processing amount is 700. This approach is less real-time due to the setting of the interval time and the maximum number of processes.
In the embodiment of the application, a newly received message needs to be encapsulated into an executable thread, and the executable thread is recorded into a thread pool and is processed by a downstream system. In implementation, the Kafka message queue can be used as a middleware system for message transmission, and the message messages in the message queue are processed based on the processing capability of a downstream system. Kafka, among others, is an open-source, streaming-enabled, high-throughput, distributed message middleware. The Kafka message queue can be used as a buffer component to relieve the problem of system stability reduction caused by sudden service increase, and the characteristic of high throughput can be utilized to realize efficient processing of message messages.
Step 202: if preset abnormity occurs in the message in the processing process and the class of the preset abnormity is detected to be the unreleasable abnormity, removing the message from the message queue;
step 203: if the exception of the message in the processing process is not monitored, the message is recorded into a thread pool and then removed from the message queue.
The Kafka message queue is used as the middleware for message transmission, and how to determine that the message is successfully consumed, namely, the message queue completes processing, needs to be considered. Based on this, in the processing flow of monitoring the message, the embodiment of the application sets the corresponding preset exception for each processing node, and divides each preset exception into two major categories, i.e., retriable exception and non-retriable exception. The retriable exception is an exception that can be avoided by processing, and the non-retriable exception is an exception that cannot be processed by the system.
In the embodiment of the application, the message queue of the message with the irretrievable exception is removed by classifying and judging the exception type of each node in the message processing process, so that the processing result of the message can be accurately obtained, and the message triggering the preset exception is reprocessed by adopting a retry strategy which is preset for the preset exception aiming at the preset exception with the retry exception belonging to the category. So as to avoid the preset exception when the message is reprocessed, thereby improving the processing speed of the message.
In the ticket service, the message sent by the upstream system is a text in an XML (EXtensible Markup Language) format. After receiving the message according to the Kafka listening thread, the message is recorded into the thread pool for processing by the downstream system after the processing flow shown in fig. 2c is needed. The processing flow may include four processing nodes of a Message security check, such as a signature check, an MD5 check (Message-Digest Algorithm 5, one-way hash Algorithm), a Message JAXB (Java Architecture for XML Binding, specification of Java ee) format conversion, a Message persistence warehousing, and a Message entry thread pool.
Based on the four processing nodes, in the embodiment of the present application, each processing node is provided with a preset processing exception, and each preset processing exception is classified, specifically, as shown in fig. 2d, the preset exception of the security check node is a security-related exception, which may be represented as a failure in performing security check on the message. The abnormity is caused by errors generated when the system utilizes the SM2 (elliptic curve public key cryptography algorithm) asymmetric verification and MD5 verification on the message. This is because the message is a tampered illegal message, which does not conform to the agreed security standard, and the subsequent service operation cannot be executed for the message, so that the exception is an unreretriable exception.
The preset exception of the JAXB format conversion node is the conversion related exception, which can be expressed as a failure of format conversion on the message. The abnormal condition is an error which occurs when the system converts the XML text into the JAVA object tree after completing the safety check, the error indicates that the message does not accord with the appointed standard, and the system considers that the message does not accord with the appointed standard and the follow-up service operation can not be carried out, so the abnormal condition is the abnormal condition which can not be retried.
The preset exception of the persistent library entry node is a database related exception, and specifically, two cases can be classified, one is that the field rule of the message is not matched with the definition of the database, for example, the database cannot be put in a library due to a naming error. The class of exceptions pertains to non-retriable exceptions. The other is that the database is currently in an offline state, and the type of exception is a retriable exception. And through a corresponding retry strategy, the database is controlled to restart and restore the online and then the message is put in the database again, so that the abnormality can be avoided.
The preset exception of the logging thread pool node is the thread pool related exception, after the message content is stored in the database, the system encapsulates the message into an executable thread and puts the executable thread into the thread pool to wait for execution, the processing concurrency of the service system is dynamically adjusted through the thread pool, and when the exception occurs during the storage of the thread, the thread pool does not have an available thread currently, a waiting queue is full, the consumption speed of the message queue is larger than the processing capacity of the system, and the consumption speed of the message queue needs to be reduced, so that the thread pool has idle resources for the message to use. The exception belongs to retriable exception, and the message can be recorded into the thread pool after idle resources exist in the thread pool through a corresponding retry strategy, so that the exception can be avoided.
In some possible embodiments, for a retriable exception, adding a new message to the message queue should be stopped before the message is reprocessed using the corresponding retry policy. The time for stopping adding new message messages into the message queue is the time interval for receiving the expanded message messages, so that the system has time to process the message messages to be processed, the pressure of a large number of transaction requests (namely receiving a large number of message messages) on a service system is relieved, and the stability of the system is improved.
It should be noted that the "packet persistent entry node" mentioned in the above processing flow of the present application is to perform the idempotent determination. The idempotent judgment is used for avoiding the waste of resources caused by the repeated execution of the service. Specifically, when a message in a message queue is recorded into a thread pool, if a problem such as a system crash or breakdown occurs and the message waiting to be put into the thread pool in a memory is lost, a recovery mechanism after the system is restarted can retrieve the lost message from the message queue and record the message into the thread pool. The recovery mechanism and the processing of the kafka message queue are two mutually independent processes, the kafka message queue does not know that the message is recorded into the thread pool, and in order to avoid recording the message into the thread pool again through the kafka message queue, whether the message is stored in the thread pool needs to be inquired before the message is recorded into the thread pool. Specifically, unique identification information may be set for each message, and whether the message is put in storage may be determined by querying the identification information in the database. If the message is detected to be put in storage, the message is indicated to be recorded into the thread pool through a recovery mechanism, and the message does not need to be recorded again through the message queue. At this point, the message should be removed from the message queue.
Based on the same inventive concept, an embodiment of the present application further provides a method 300 for processing a service message, specifically as shown in fig. 3, including:
a process monitoring module 301, configured to add the received message to a message queue, process the message, and monitor a processing process of the message; the processing flow is a process of inputting the message in the message queue into a thread pool;
a first removing module 302, configured to remove the message from the message queue if it is detected that a preset exception occurs in a processing process of the message and it is detected that a category to which the preset exception belongs is an unrepeatable exception;
the second removing module 303 is configured to, if it is not monitored that the message is abnormal in the processing process, enter the message into a thread pool, and then remove the message from the message queue.
In some possible embodiments, after the monitoring that the preset exception occurs in the processing process of the message is performed, the process monitoring module 301 is further configured to:
if the class of the preset exception is detected to be retriable exception, a retry strategy corresponding to the preset exception is adopted to reprocess the message, and exception monitoring is carried out on the reprocessing process.
In some possible embodiments, the preset anomalies include at least the following:
the security check of the message fails;
failing to convert the format of the message;
failing to store the message;
and the message is failed to be recorded into the thread pool.
In some possible embodiments, performing said detecting of said category of preset anomalies, said first removing module 302 is configured to:
if the preset exception is failure in performing security check on the message, determining that the category of the preset exception is an unreleasable exception;
if the preset exception is failure in format conversion of the message, determining that the category of the preset exception is an unreleasable exception;
if the preset exception is that the message is failed to be stored and the field naming rule of the message is detected to be not in accordance with the storage rule, determining that the preset exception is a retry-impossible exception;
if the preset exception is failure in storing the message and the database for storing the message is detected to be in an off-line state, determining that the preset exception is a retry-able exception;
if the preset exception is that the message is failed to be recorded into the thread pool, determining that the type of the preset exception is a retriable exception.
In some possible embodiments, performing the reprocessing of the message by using the retry policy corresponding to the preset exception, where the first removing module 302 is configured to:
if the preset exception is that the message is failed to be stored and the database for storing the message is detected to be in an off-line state, the message is reprocessed after the database is controlled to be on-line, and the message is stored in the database;
if the preset exception is that the message is failed to be recorded into the thread pool, the message is reprocessed after the thread pool is detected to have idle resources, and the message is recorded into the thread pool again.
In some possible embodiments, before performing the reprocessing of the message by using the retry policy corresponding to the preset exception, the first removing module 302 is further configured to:
and after determining that the category of the preset exception is a retriable exception, stopping adding a new message to the message queue.
In some possible embodiments, before performing the entering of the message into a thread pool, the method further includes:
determining that the message is not stored in a database;
the method further comprises the following steps:
if the message information is detected to be stored in the database, removing the message information from the message queue; wherein, the message in the database is added before the message is recorded into the thread pool, so as to determine that the message triggering the recovery mechanism is not recorded into the thread pool again; the recovery mechanism is used for putting the message into the thread pool after the message to be put into the thread pool is lost.
The electronic device 130 according to this embodiment of the present application is described below with reference to fig. 4. The electronic device 130 shown in fig. 4 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.
As shown in fig. 4, the electronic device 130 is represented in the form of a general electronic device. The components of the electronic device 130 may include, but are not limited to: the at least one processor 131, the at least one memory 132, and a bus 133 that connects the various system components (including the memory 132 and the processor 131).
The memory 132 may include readable media in the form of volatile memory, such as Random Access Memory (RAM)1321 and/or cache memory 1322, and may further include Read Only Memory (ROM) 1323.
The electronic device 130 may also communicate with one or more external devices 134 (e.g., keyboard, pointing device, etc.), with one or more devices that enable a user to interact with the electronic device 130, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 130 to communicate with one or more other electronic devices. Such communication may occur via input/output (I/O) interfaces 135. Also, the electronic device 130 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet) via the network adapter 136. As shown, network adapter 136 communicates with other modules for electronic device 130 over bus 133. It should be understood that although not shown in the figures, other hardware and/or software modules may be used in conjunction with electronic device 130, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
In an exemplary embodiment, a computer-readable storage medium comprising instructions, such as the memory 132 comprising instructions, executable by the processor 131 of the apparatus 400 to perform the above-described method is also provided. Alternatively, the computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
In an exemplary embodiment, there is also provided a computer program product comprising computer programs/instructions which, when executed by the processor 131, implement any of the methods of processing business messages as provided herein.
In exemplary embodiments, various aspects of a service message processing method provided in this application can also be implemented in the form of a program product, which includes program code for causing a computer device to perform the steps in a service message processing method according to various exemplary embodiments of this application described above in this specification when the program product runs on the computer device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The program product for processing of business messages of embodiments of the present application may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on an electronic device. However, the program product of the present application is not limited thereto, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations for aspects of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "for example" programming language or similar programming languages. The program code may execute entirely on the consumer electronic device, partly on the consumer electronic device, as a stand-alone software package, partly on the consumer electronic device and partly on a remote electronic device, or entirely on the remote electronic device or server. In the case of remote electronic devices, the remote electronic devices may be connected to the consumer electronic device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to external electronic devices (e.g., through the internet using an internet service provider).
It should be noted that although several units or sub-units of the apparatus are mentioned in the above detailed description, such division is merely exemplary and not mandatory. Indeed, the features and functions of two or more units described above may be embodied in one unit, according to embodiments of the application. Conversely, the features and functions of one unit described above may be further divided into embodiments by a plurality of units.
Further, while the operations of the methods of the present application are depicted in the drawings in a particular order, this does not require or imply that these operations must be performed in this particular order, or that all of the illustrated operations must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable image scaling apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable image scaling apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable image scaling apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable image scaling device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer implemented process such that the instructions which execute on the computer or other programmable device provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While the preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all alterations and modifications as fall within the scope of the application.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.
Claims (10)
1. A method for processing a service message, the method comprising:
after adding the received message to a message queue, processing the message and monitoring the processing flow of the message; the processing flow is a process of inputting the message in the message queue into a thread pool;
if preset abnormity occurs in the message in the processing process and the class of the preset abnormity is detected to be the unreleasable abnormity, removing the message from the message queue;
if the exception of the message in the processing process is not monitored, the message is recorded into a thread pool and then removed from the message queue.
2. The method according to claim 1, wherein after monitoring that a predetermined exception occurs in the processing of the message, the method further comprises:
if the class of the preset exception is detected to be retriable exception, a retry strategy corresponding to the preset exception is adopted to reprocess the message, and exception monitoring is carried out on the reprocessing process.
3. The method according to any one of claims 1 or 2, wherein the predetermined anomalies comprise at least:
the security check of the message fails;
failing to convert the format of the message;
failing to store the message;
and the message is failed to be recorded into the thread pool.
4. The method of claim 3, wherein said detecting the category of the predetermined anomaly comprises:
if the preset exception is failure in performing security check on the message, determining that the category of the preset exception is an unreleasable exception;
if the preset exception is failure in format conversion of the message, determining that the category of the preset exception is a retry-impossible exception;
if the preset exception is that the message is failed to be stored and the field naming rule of the message is detected to be not in accordance with the storage rule, determining that the preset exception is a retry-impossible exception;
if the preset exception is failure in storing the message and the database for storing the message is detected to be in an off-line state, determining that the preset exception is a retry-able exception;
if the preset exception is that the message is failed to be recorded into the thread pool, determining that the type of the preset exception is a retriable exception.
5. The method according to claim 4, wherein the reprocessing the message using the retry strategy corresponding to the preset exception comprises:
if the preset exception is that the message is failed to be stored and the database for storing the message is detected to be in an off-line state, the message is reprocessed after the database is controlled to be on-line, and the message is stored in the database;
if the preset exception is that the message is failed to be recorded into the thread pool, the message is reprocessed after the thread pool is detected to have idle resources, and the message is recorded into the thread pool again.
6. The method of claim 4, wherein before the reprocessing the message using the retry strategy corresponding to the preset exception, the method further comprises:
and after determining that the category of the preset exception is a retriable exception, stopping adding a new message to the message queue.
7. The method of claim 1, wherein prior to entering the message into a thread pool, the method further comprises:
determining that the message is not stored in a database;
the method further comprises the following steps:
if the message information is detected to be stored in the database, removing the message information from the message queue; wherein, the message in the database is added before the message is recorded into the thread pool, so as to determine that the message triggering the recovery mechanism is not recorded into the thread pool again; the recovery mechanism is used for putting the message into the thread pool after the message to be put into the thread pool is lost.
8. An apparatus for processing service messages, the apparatus comprising:
the flow monitoring module is configured to add the received message to a message queue, process the message and monitor the processing flow of the message; the processing flow is a process of inputting the message in the message queue into a thread pool;
the first removing module is configured to remove the message from the message queue if preset abnormity occurs in the processing process of the message and the class of the preset abnormity is detected to be the unreleasable abnormity;
and the second removing module is configured to remove the message from the message queue after the message is recorded into a thread pool if the message is not monitored to be abnormal in the processing process.
9. An electronic device, comprising:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the method of processing a service message according to any one of claims 1 to 7.
10. A computer-readable storage medium, wherein instructions in the computer-readable storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the method of processing a service message of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111407374.1A CN114090297A (en) | 2021-11-24 | 2021-11-24 | Service message processing method and related device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111407374.1A CN114090297A (en) | 2021-11-24 | 2021-11-24 | Service message processing method and related device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114090297A true CN114090297A (en) | 2022-02-25 |
Family
ID=80304171
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111407374.1A Pending CN114090297A (en) | 2021-11-24 | 2021-11-24 | Service message processing method and related device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114090297A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115981828A (en) * | 2023-02-09 | 2023-04-18 | 中国证券登记结算有限责任公司 | Service message processing method and device |
| CN116934271A (en) * | 2023-08-08 | 2023-10-24 | 上海中汇亿达金融信息技术有限公司 | Service processing method, device, electronic equipment and medium |
-
2021
- 2021-11-24 CN CN202111407374.1A patent/CN114090297A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115981828A (en) * | 2023-02-09 | 2023-04-18 | 中国证券登记结算有限责任公司 | Service message processing method and device |
| CN115981828B (en) * | 2023-02-09 | 2023-09-22 | 中国证券登记结算有限责任公司 | Service message processing method and device |
| CN116934271A (en) * | 2023-08-08 | 2023-10-24 | 上海中汇亿达金融信息技术有限公司 | Service processing method, device, electronic equipment and medium |
| CN116934271B (en) * | 2023-08-08 | 2024-02-02 | 上海中汇亿达金融信息技术有限公司 | Service processing method, device, electronic equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR101835458B1 (en) | Method, system and computer-readable storage medium for restarting data processing systems | |
| JP6556110B2 (en) | Resume process | |
| US7624309B2 (en) | Automated client recovery and service ticketing | |
| CN114090297A (en) | Service message processing method and related device | |
| CN110838065A (en) | Transaction data processing method and device | |
| US9158606B2 (en) | Failure repetition avoidance in data processing | |
| CN111880906A (en) | Virtual machine high-availability management method, system and storage medium | |
| US20120166893A1 (en) | Recording and Preventing Crash in an Appliance | |
| CN112231403A (en) | Consistency checking method, device, equipment and storage medium for data synchronization | |
| CN112333262A (en) | Data updating prompting method and device, computer equipment and readable storage medium | |
| US9286112B2 (en) | Determining and dispatching processing agent for transaction processing | |
| US20230109607A1 (en) | Transaction exchange platform with a messenger microservice to update transactions | |
| US20230106852A1 (en) | Transaction exchange platform with a watchdog microservice to handle stalled transactions | |
| US20230108180A1 (en) | Transaction exchange platform with a pause microservice to pause processing of workflows | |
| US20230107687A1 (en) | Transaction exchange platform defining conditions for the processing of transaction objects | |
| US8661296B2 (en) | Dynamic data store for failed jobs in a systems complex | |
| WO2024064133A1 (en) | Transaction exchange platform using blockchain and data integrity microservices to validate transaction object integrity | |
| CN114626839A (en) | Payment channel monitoring method and system, electronic equipment and storage medium | |
| CN117082546A (en) | Method and system for processing disaster recovery of core network element | |
| CN114629694A (en) | Detection method and related device for distributed denial of service (DDoS) | |
| CN114706872A (en) | Asynchronous queuing processing method and system for payment transaction | |
| CN117539843A (en) | Interactive database alarm and fault processing system based on enterprise WeChat | |
| CN117762740A (en) | Method, system, equipment and medium for data security monitoring | |
| JP2007094727A (en) | System, method and program for connection to bank of japan |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |