CN111190747A

CN111190747A - Message loss detection method and device for message queue

Info

Publication number: CN111190747A
Application number: CN201911330718.6A
Authority: CN
Inventors: 马鹏程; 胡伟
Original assignee: Beijing Kingsoft Cloud Network Technology Co Ltd
Current assignee: Beijing Kingsoft Cloud Network Technology Co Ltd
Priority date: 2019-12-20
Filing date: 2019-12-20
Publication date: 2020-05-22
Also published as: WO2021121370A1

Abstract

The application discloses a message loss detection method and device for a message queue. Wherein, the method comprises the following steps: receiving a service message sent by a producer and a message identifier of the service message, wherein the service message is used for representing a target task to be executed by a consumer; storing the service message to a message queue and storing the message identifier to a database, wherein the message queue and the database are in independent communication with the consumer; sending the service message to the consumer through the message queue so that the consumer executes the target task and stores the message identifier of the service message; and when the database receives a query request of a consumer, the message identification is sent to the consumer through the database, so that the consumer determines whether the lost service message exists according to the message identification stored in the database and the message identification stored in the consumer. The method and the device solve the technical problem that the lost message cannot be accurately detected in the related technology.

Description

Message loss detection method and device for message queue

Technical Field

The present application relates to the field of internet, and in particular, to a method and an apparatus for detecting message loss for a message queue.

Background

Remote Procedure Calls (RPC) to request services from remote computer programs over a network without knowledge of the underlying network technology protocols, the RPC protocol assuming the existence of some transport protocol, such as TCP or UDP, to carry information data between the communicating programs. In the OSI network communication model, RPC spans the transport and application layers, and RPC makes it easier to develop applications including network distributed multiprogrammers.

In the distributed system architecture, RPC is a general technique for implementing reliable and ordered delivery of messages, and is one of the key factors ensuring stable operation of the system in the distributed system environment, taking openstack-neutron component (a component for network virtualization) as an example, and the system architecture is shown in fig. 1. In the system architecture shown in fig. 1, if a user sends an HTTP request to a neutron-server (producer), where the request is used to request to create a routing entry between two network segments, so as to implement IP three-layer accessibility between the two network segments, if the producer does not accurately send the request to an L3 proxy (or L3-agent, that is, a consumer's proxy that receives a service message sent in a FANOUT manner), the L3 proxy cannot configure a routing entry for a corresponding gateway, and the function of this HTTP request cannot be implemented.

During the process of the producer sending a request to the L3 agent, the producer sends a message to the consumer xxx-agent via a message queue MQ (collectively referred to as MessageQueue), where "xxx" is the identity of a particular consumer, such as L3 and OVS, and several exceptions may occur that result in the producer not sending the request accurately to the consumer:

(1) when a producer generates an exception when sending a message to a message queue, such as TCP connection interruption, so that the message is not sent to the message queue, the exception producer should be capturable;

(2) the message Queue is suddenly down, the messages stored in the Queue of the message Queue are not persistent, and the messages are lost after restarting;

(3) the connection between the consumer and the message queue is broken, but the producer also sends a message to the consumer, resulting in the accumulation of messages in the corresponding queue in the message queue and the failure to send to the producer.

In the above several abnormal situations that may cause message loss, the following solutions are commonly adopted in the industry:

scheme one, message persistence: the method mainly comprises the steps of setting persistence for Exchange, setting persistence for Queue, and sending Message persistence, wherein a sending mode DeliveryMode is set to be 2 in the Message sending setting, under the default condition, if a Message Queue cluster is down and restarted, all Exchange and Queue and service messages which are not consumed are lost, and through the steps of Exchange persistence, Queue persistence and service Message persistence, after the Message Queue is down and restarted, the state before the down can be quickly recovered.

Scheme two, ack acknowledgement mechanism: in the current message sending confirmation, the ConfirmCallback only confirms whether the message correctly arrives in Exchange; return Callback triggers a callback when a message does not arrive correctly in the queue, and does not execute if it arrives correctly in the queue. When the message is received and confirmed, the message consumer automatically sends the ack confirmation message under the default condition, the scheme adopts automatic confirmation, and the message can be immediately confirmed after being sent to the consumer proxy, so that the possibility of losing the message exists, such as the message is not successfully stored by the consumer proxy and is not successfully sent to the consumer.

In the disadvantages of the first and second schemes, firstly, the first scheme relates to a persistence process, wherein persistence is an operation of frequently reading and writing a hard disk, when the scale of a distributed cluster is large, the pressure of a message queue is large, and the persistence of the message queue can generate a bottleneck on the scale of the cluster; there are several common problems with both solution one and solution two: firstly, if an error occurs in the process of sending a message to Queue by Exchange, a producer and a consumer cannot effectively identify the error, and then the message is lost, namely the problem cannot be solved; secondly, if the TCP connection between the consumer and the message queue is disconnected, the consumer cannot receive the Fanout type message, and the message queue cannot perform persistence and acknowledgement retransmission on the message.

It can be seen that the above two solutions still cannot guarantee that the message is accurately transmitted to the receiving end (consumer).

In view of the above problems, no effective solution has been proposed.

Disclosure of Invention

The embodiment of the application provides a message loss detection method and device for a message queue, so as to at least solve the technical problem that the lost message cannot be accurately detected in the related technology.

According to an aspect of the embodiments of the present application, there is provided a method for detecting a message loss of a message queue, which is applied to a message processing system, where the message processing system includes the message queue and a database, the method includes: receiving a service message sent by a producer and a message identifier of the service message, wherein the service message is used for representing a target task to be executed by a consumer; storing the service message to a message queue and storing the message identifier to a database, wherein the message queue and the database are in independent communication with the consumer; sending the service message to the consumer through the message queue so that the consumer executes the target task and stores the message identifier of the service message; and when the database receives a query request of a consumer, the message identification is sent to the consumer through the database, so that the consumer determines whether the lost service message exists according to the message identification stored in the database and the message identification stored in the consumer.

According to another aspect of the embodiments of the present application, there is provided a message loss detection method for a message queue, applied to a consumer of a message, the method including: receiving a service message sent by a message queue from a message processing system and locally storing a message identifier of the service message, wherein the service message is used for representing a target task to be executed by a consumer; sending an identification acquisition request to a database of the message processing system to acquire a message identification stored in the database; and determining whether the lost service message exists according to the locally stored message identification and the message identification stored in the database.

According to another aspect of the embodiments of the present application, there is provided a message loss detection method for a message queue, applied to a producer of a message, the method including: generating a service message to be sent, wherein the service message comprises a message identifier; sending the service message to a message queue of the message processing system so that the service message is stored in the message queue for a consumer to read and consume; and sending the message identification of the service message to a database of the message processing system, so that the message identification is stored in the database for a consumer to inquire when determining whether the lost service message exists.

According to another aspect of the embodiments of the present application, there is also provided a message loss detection apparatus for a message queue, which is applied to a message processing system, where the message processing system includes the message queue and a database, and the apparatus includes: the first receiving unit is used for receiving a service message sent by a producer and a message identifier of the service message, wherein the service message is used for representing a target task to be executed by a consumer; the storage unit is used for storing the service message to a message queue and storing the message identifier to a database, wherein the message queue and the database are independently communicated with the consumer; the first sending unit is used for sending the business message to the consumer through the message queue so that the consumer can execute the target task and store the message identifier of the business message; the first detection unit is used for sending the message identification to the consumer through the database when the database receives the query request of the consumer, so that the consumer can determine whether the lost service message exists according to the message identification stored in the database and the message identification stored in the consumer.

According to another aspect of the embodiments of the present application, there is also provided a message loss detection apparatus for a message queue, applied to a consumer of a message, the apparatus including: the second receiving unit is used for receiving the service message sent by the message queue of the message processing system and storing the message identifier of the service message locally, wherein the service message is used for representing a target task to be executed by a consumer; the system comprises an acquisition unit, a storage unit and a processing unit, wherein the acquisition unit is used for sending an identification acquisition request to a database of the message processing system so as to acquire a message identification stored in the database; and the second detection unit is used for determining whether the lost service message exists according to the locally stored message identifier and the message identifier stored in the database.

According to another aspect of the embodiments of the present application, there is also provided a message loss detection apparatus for a message queue, applied to a producer of a message, the apparatus including: the generating unit is used for generating a service message to be sent, wherein the service message comprises a message identifier; the second sending unit is used for sending the service message to a message queue of the message processing system so that the service message is stored in the message queue for a consumer to read and consume; and the third sending unit is used for sending the message identifier of the service message to a database of the message processing system so that the message identifier is stored in the database for a consumer to inquire when determining whether the lost service message exists.

According to another aspect of the embodiments of the present application, there is also provided a storage medium including a stored program which, when executed, performs the above-described method.

According to another aspect of the embodiments of the present application, there is also provided an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the above method through the computer program.

In the embodiment of the application, the data synchronization can be realized for the producer and the consumer through the provided database, and the consumer can determine whether the lost message exists by comparing the message identifier stored in the database with the local message identifier, so that the technical problem that the lost message cannot be accurately detected in the related technology can be solved, and the technical effect of monitoring the lost message is further achieved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

fig. 1 is a schematic diagram of a hardware environment of a system for processing a service message in the related art;

FIG. 2 is a schematic diagram of a hardware environment of a system for processing business messages according to an embodiment of the present application;

FIG. 3 is a flow chart of an alternative message loss detection method for a message queue according to an embodiment of the present application;

FIG. 4 is a flow chart of an alternative message loss detection method for a message queue according to an embodiment of the present application;

FIG. 5 is a flow chart of an alternative message loss detection method for a message queue according to an embodiment of the present application;

FIG. 6 is a flow chart of an alternative message loss detection method for a message queue according to an embodiment of the present application;

FIG. 7 is a diagram of an alternative message loss detection apparatus for a message queue according to an embodiment of the present application;

FIG. 8 is a diagram of an alternative message loss detection apparatus for a message queue according to an embodiment of the present application;

FIG. 9 is a diagram of an alternative message loss detection apparatus for a message queue according to an embodiment of the present application; and the number of the first and second groups,

fig. 10 is a block diagram of a terminal according to an embodiment of the present application.

Detailed Description

In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

First, partial nouns or terms appearing in the description of the embodiments of the present application are applicable to the following explanations:

the Transmission Control Protocol (TCP) is a transport Protocol specifically designed to provide a reliable end-to-end byte stream over an unreliable internet network.

A user Datagram protocol UDP (user Datagram protocol), which is a connectionless transport layer protocol in an OSI (Open system interconnection ) reference model, and provides a transaction-oriented simple unreliable information transfer service, and a protocol number of UDP in an IP packet is 17.

The Internet Protocol Address IP (english: Internet Protocol Address, translated to Internet Protocol Address) is a digital label assigned to an Internet Protocol device used by a user to access the Internet.

Http (hypertext Transfer protocol): is a simple request-response protocol (hypertext transfer protocol) that typically runs on top of TCP, which specifies what messages a client may send to a server and what responses to get. The headers of the request and response messages are given in ASCII code form; and the message content has a MIME-like format.

According to an aspect of embodiments of the present application, there is provided a method embodiment of a method for detecting a message loss of a message queue.

Alternatively, the above method may be applied in a hardware environment as shown in fig. 2. As shown in fig. 2, the distributed system includes a device where a consumer is located (which may be referred to as a consumer for short), a device where a producer is located (which may be referred to as a producer for short), a device where a queue is located (such as a memory and a database), and optionally, the distributed system may further include an agent device where an L3 agent is located and an agent device where an OVS (OpenvSwitch, a kind of virtual switching software) agent is located, and the device of the consumer and the agent device may be referred to as a consumer collectively.

The message loss detection method for the message queue according to the embodiment of the present application may be performed by a device (e.g., a producer, a consumer, and a device in which the message queue is located) in a distributed system.

Fig. 3 is a flowchart of an alternative message loss detection method for a message queue, which is applied to a message processing system including the message queue and a database (storing message identifiers in the database), according to an embodiment of the present application, and as shown in fig. 3, the method may include the following steps:

step S302, the message processing system receives the service message sent by the producer and the message identifier of the service message, and the service message is used for representing the target task to be executed by the consumer.

The service message may be a message packet generated according to a format for communication between a producer and a consumer, and the message may represent that a task to be executed is a target task by carrying a task identifier of the task itself or the target task.

Step S304, storing the service message to the message queue and storing the message identifier to the database, where the message queue and the database are independently communicated with the consumer, for example, the message queue and the database may be respectively stored in memories located at different positions, so as to avoid that the memory in which the message queue is located also fails when the memory in which the message queue is located fails.

Step S306, the service message is sent to the consumer through the message queue, so that the consumer executes the target task and stores the message identification of the service message.

Step S308, when the database receives the query request of the consumer, the message identifier is sent to the consumer through the database, so that the consumer can determine whether the lost service message exists according to the message identifier stored in the database and the message identifier stored in the consumer.

In the above technical solution of the present application, a producer sends a service Message to a Message queue, and also stores a Message identifier (i.e., Message _ id) of the service Message in a database (e.g., a Redis database), and each service Message consumed by a consumer also stores the Message identifier in a local cache, so as to periodically query the Message identifier in the database, and check whether the queried Message identifier is consumed by the consumer, and for a Message identifier that is not consumed, it indicates that the Message queue loses a Message corresponding to the Message identifier in a Message transmission process.

In the technical scheme, the data synchronization can be realized for the producer and the consumer through the provided database, and the consumer can determine whether the lost message exists by comparing the message identifier stored in the database with the local message identifier, so that the technical problem that the lost message cannot be accurately detected in the related technology can be solved, and the technical effect of monitoring the lost message is further achieved. The technical solution of the present application is further detailed below with reference to the steps shown in fig. 3:

in the technical solution provided in step S302, the producer receives a request for the consumer to execute a target task, and further generates a service message, and sends the service message to the message processing system, and the message processing system receives the service message sent by the producer and the message identifier of the service message.

The producer may provide an interface (e.g., a rest interface) to the user, through which the user may invoke by way of an HTTP request, such as by sending an HTTP-based task request.

In the technical solution provided in step S304, the service message is stored in the message queue and the message identifier is stored in the database.

Optionally, when storing the service message to the message queue and storing the message identifier to the database, the sending object may be selected according to the type of the message, and when the type of the service message is a first type (e.g., Fanout type), the service message is stored to all message queues, and the message identifier is associated with the first type and then stored to the database, and the message of the first type is set to be sent to all message queues for all consumers to consume; and if the type of the service message is a second type (such as a Direct type message in openstack), storing the service message to a message queue associated with a consumer specified by the service message, associating a message identifier with the consumer specified by the service message, and then storing the message identifier in a database, wherein the message of the second type is set to be sent to the message queue associated with the specified consumer for consumption by the specified consumer.

The association may be configured by configuring a specified character (e.g., character "0" indicates a first type, and character "1" indicates a second type) for the message identifier, or may be stored according to a format communicated in advance to indicate the type, such as a specified position in a table.

Optionally, after receiving a message identifier of a service message sent by a producer, the message identifier may be stored in a preset storage space, where the preset storage space is used to perform redundant backup on the message identifier stored in the database; and under the condition that the message identification stored in the database is lost, copying the message identification backed up by the preset storage space to the database for storage.

In the computer structure, there is an important part, namely memory, which is a component for storing program and data. The types of the memories are divided into a main memory and an auxiliary memory according to the purposes of the memories, wherein the main memory is also called an internal memory (called an internal memory for short), the internal memory is only used for temporarily storing programs and data, and the programs and the data are lost once the power supply is turned off or the power is cut off; the auxiliary memory is also called an external memory (external memory for short), can store information for a long time, and does not depend on electricity to store the information.

The message queue of the application can be stored by adopting an internal memory, the database can also be established on an external memory, such as a magnetic disk, in order to improve the data processing speed, the database can also be established on the internal memory which keeps data synchronization with the external memory, the data processing speed is ensured by the internal memory, and meanwhile, the data loss caused by the abnormality of the internal memory is avoided by the external memory (namely the preset storage space).

In the technical scheme, the message queue can be realized in a rabbitmq (which is open source message agent software for realizing an advanced message queue protocol AMQP and is also called as message-oriented middleware) on an internal memory; a database is established in an internal memory, the database (such as Redis) can be remotely accessed, and different processes can be connected to the same database; the database adopts the operation based on the memory, and the speed is high; the database not only supports simple key-value type data, but also provides storage of data structures such as lists list, sets, ordered sets zsets, hash and the like.

The rabbitmq where the message queue is located and the database can not be directly interacted, normal communication among processes can be realized through the rabbitmq, the database is used as a storage tool, when a producer sends a message to a consumer, the message identifier is stored in the database, and then the consumer can check whether the message sent by the producer is completely consumed according to the message identifier stored in the database by the producer.

In the technical solution provided in step S306, the service message is sent to the consumer through the message queue, so that the consumer executes the target task and stores the message identifier of the service message.

Similarly, when a message queue distributes service messages therein to consumers, the consumption queue also has two types, one type is a message queue corresponding to the first type, and the message queue corresponding to the first type can directly store the service messages of the first type without storing information of all consumers (storing information of any consumer herein is equivalent to specifying that the service message needs to be associated with all consumers); the other type is a message queue corresponding to the second type, and the message queue corresponding to the second type can store only the service messages of the same consumer although the message queue corresponding to the second type also directly stores the service messages of the second type, in other words, each message queue under the type is bound with one consumer.

The above describes a message handling scheme in different scenarios (i.e. different types), where a message of a first type corresponds to a broadcast and is received by all consumers, so that there is no need to consider to whom the message is intended, and for consumers each consumer consumes all messages of the first type. For the second type of message, the producer sends the message to the consumer designated by the request (such as carrying the device identification of the consumer in the task request), for example, the producer sends the message to the consumer on a certain host, and other consumers do not consume the message, the host of the message can be indicated when the message identification of the message is saved in the message queue on the database, so that other consumers irrelevant to the message can be prevented from processing the message by mistake.

In the technical solution provided in step S308, when the database receives the query request of the consumer, the message identifier is sent to the consumer through the database, so that the consumer determines whether there is a lost service message according to the message identifier stored in the database and the message identifier stored in the consumer.

Optionally, in order to relieve the storage pressure of the database, after the message identifier is sent to the consumer through the database, the message identifier which is queried by the consumer and corresponds to the service message that has been consumed by the consumer in the database is deleted, so as to reduce resource waste caused by repeated comparison.

Fig. 4 is a flowchart of an alternative message loss detection method for a message queue according to an embodiment of the present application, which may be performed by a consumer as shown in fig. 4, and which may include the following steps:

step S402, the consumer receives the service message sent by the message queue from the message processing system and stores the message identification of the service message locally, and the service message is used for representing the target task to be executed by the consumer.

Step S404, the consumer sends an identifier obtaining request to a database of the message processing system to obtain the message identifier stored in the database.

Optionally, since the number of service messages increases with time, the identifier obtaining request may be periodically sent to the database according to a preset query period (e.g., ten minutes) so as to query whether there is a lost service message in the period.

Step S406, the consumer determines whether there is a lost service message according to the locally stored message identifier and the message identifier stored in the database.

Optionally, determining whether there is a lost service message according to the locally stored message identifier and the message identifier stored in the database includes: comparing the locally stored message identification with the message identification stored in the database to obtain a comparison result; and determining whether the lost service message exists according to the comparison result.

Optionally, determining whether there is a lost service message according to the comparison result includes: determining that the service message corresponding to the first type message identifier (the specific number can be an integer greater than or equal to 0) is a lost service message under the condition that the comparison result indicates that the first type message identifier exists, wherein the first type message identifier represents a message identifier which is stored in a database and cannot be searched in a local storage; and determining that no lost service message exists under the condition that the comparison result shows that the first-class message identifier does not exist.

Optionally, some service messages are lost due to network delay, and some service messages are actually lost service messages, and in order to avoid detection errors, when the comparison result shows that the first type of message identifier exists, a timer is started; when the timing duration of the timer reaches a first duration (such as 1 minute), determining whether a service message corresponding to the first type message identifier is received; determining that the service message corresponding to the first type message identifier is a lost service message includes: if the service message corresponding to the first type message identification is determined not to be received, determining that the service message corresponding to the first type message identification is lost; otherwise, it is determined that the first type message identifies that the corresponding service message is not lost (i.e., the above-mentioned transient loss due to network delay).

Optionally, when the timing duration of the timer reaches the first duration, determining whether to receive the service message corresponding to the first type message identifier includes: when the timing duration of the timer reaches a first time duration, judging whether a first type message identifier still exists; and when the first-class message identification does not exist, determining that the service message corresponding to the first-class message identification is received. When judging whether the first type message identification exists, if so, determining the number of the first type message identification at the moment, judging whether the number of the first type message identification is reduced relative to the previous time, if so, determining that the service message corresponding to the reduced first type message identification is received through comparison with the previous time.

Optionally, after determining that the service message corresponding to the first type message identifier is lost, an alarm message may be generated to prompt the user that the service message corresponding to the message identifier of the first type message identifier is lost, and a corresponding policy is adopted, for example, a producer is notified to resend the service message.

Fig. 5 is a flowchart of an alternative message loss detection method for a message queue according to an embodiment of the present application, which may be performed by a producer as shown in fig. 5, and which may include the following steps:

step S502, the producer generates a service message to be sent, and the service message comprises a message identifier.

Step S504, the producer sends the service message to the message queue of the message processing system, so that the service message is stored in the message queue for the consumer to read and consume.

Step S506, the producer sends the message identifier of the service message to the database of the message processing system, so that the message identifier is stored in the database for the consumer to query when determining whether there is a lost service message.

Optionally, sending the service message to a message queue of the message processing system includes: under the condition that the type of the business message is a first type, the business message is sent to all message queues so that the business message is stored in all message queues, and the message of the first type is set to be sent to all message queues for all consumers to consume; and in the case that the type of the business message is a second type, sending the business message to a message queue associated with the specified consumer of the business message, wherein the message of the second type is set to be sent to the message queue associated with the specified consumer for consumption by the specified consumer.

In the embodiment of the present application, taking the use of the database Redis as an example (the database used here may be replaced by another database, such as Mysql), the Redis is an open-source log-type, Key-Value database written in ANSI C language, supporting network, being based on memory and being persistent, and provides API in multiple languages. According to the scheme, by means of Redis, a consumer can directly compare data with a producer to detect whether the consumer consumes all messages sent to the consumer or not, and the reliability of a message queue is not relied on. After a producer sends a message to a message queue, a message identifier of the message is stored in Redis, each message consumed by the consumer also stores the message identifier in a local variable, the message identifier in Redis is inquired periodically, whether the message identifier searched from the Redis is consumed or not is checked, and if the message identifier in the Redis which is not consumed is inquired, the message queue is indicated that the message corresponding to the message identifier is lost in the process of transmitting the message.

As an alternative example, the technical solution of the present application is further described below with reference to the specific embodiments.

The producer and the consumer respectively build a connection to the same database Redis and then process according to the message type.

If a producer sends a message marked by a Fanout type to all consumers, the message identification of the message is stored in a queue of Redis before the message is sent, for example, one or more message identifications are inserted into the head of a list by using an Lpush command, then a message (namely a service message) is sent to the message queue, and when the consumer consumes the message, the message identification of the taken message is stored in a cache of the consumer, and then the message is processed. In the consumer process, a timing task can be set, a message identifier of which the Exchange mode belongs to a Fanout type is inquired from the Redis at a fixed time, and then the message identifier taken from the Redis is compared with the message identifier in the cache of the consumer, so that the message identifier which is not consumed in the Redis is found, namely the service message which is lost or accumulated is found. For Fanout type messages, if a certain consumer is disconnected from a message queue, the message queue and a producer are both noninductive, and lost messages can be detected by comparing message identifications stored in Redis by the producer with the consumer.

If a producer uses a Direct mode (in the mode, a unique Queue can receive messages determined by a routingKey and Exchange of a routing host in class), the messages are sent to partial consumers, namely, the consumers of the designated host are used as routing keys, the sent host and message identifications are used as key words and stored in a Redis, and when the consumers consume the messages, if the Exchange is Direct, the message identifications and the own host are stored in a cache, and then the service messages are processed. In the consumer process, a timing task is set, the message identifications of the Exchange and host which the Redis belongs to can be inquired at regular time, and then the message identifications which are not consumed in the Redis, namely the lost or message accumulated service messages, are found out by comparing the message identifications with the message identifications in the cache. In Direct mode, if the TCP connection between the consumer and the message queue is disconnected, Exchange will also send the message to the designated message queue through the routing key matching, but the disconnection of the consumer and the message queue will cause message accumulation.

Because the message identifier needs to be processed periodically, in order to avoid resource waste, the processed message identifier should be deleted from the cache and Redis of the consumer process, so as to avoid repeated checksum formation and data accumulation. The solution here is that before the producer sends the Message, the producer stores the Message identifier in the Redis list, as implemented by Lpush (key, Message _ id), and prunes a list, as implemented by a command trim (e.g., ltrim, which can enable the list to only retain elements in a specified interval, and elements not in the specified interval are all deleted), and sets the LENGTH to MAX _ LENGTH, ltrim (key,0, self.

To control the amount of cache in the consumer process, the consumer may periodically perform a message detection function, during the detection process, care should be taken to delete the verified message from the cache, and a message identifier that exists in the Redis but does not exist in its own cache is a message identifier that should be consumed but is not consumed, store the message identifier, set a timestamp, and then wait for an alarm to be issued when the time threshold is exceeded.

The specific implementation process of the above scheme can be seen in fig. 6:

in step S602, the producer receives 10 HTTP requests (i.e., task requests), and the request _ ids (i.e., request identifiers) are 0 to 9, respectively.

Each HTTP request can send a message queue to OVS-AGENT of the HOST-A, the mut mutexchange type of the corresponding service message is Direct, the name of the mut mutexchange-AGENT is OVS-AGENT, and the ROUTE-KEY is HOST-A.

Step S604, before the producer sends the message to the message queue rabbitmq (or MQ), the producer stores the message identifier in the database Redis in the manner of Lpush (OVS-AGENT-HOST-a,0) … Lpush (OVS-AGENT-HOST-a,9), and once prunes the queue when storing the Redis each time, so as to avoid that the LENGTH of the queue (if the LENGTH needs to be controlled in LENGTH) is too large due to long-term storage of the message identifier that has been checked by the consumer process, and the method can be implemented by ltrim (OVS-AGENT-HOST-a,0, LENGTH).

The Redis driver may provide an interface to operate Redis, and similarly, the message queue driver provides an interface to operate the message queue.

In step S606, the OVS-AGENT process on HOST-a stores the Message identifier (also referred to as request _ id) into the variable when consuming the Message, and if the Message identifier is received from 0 to 6 and 9, a list local _ Message _ ids [0,1,2,3,4,5,6,9] mutexists in the cache of the consumer process to indicate the received Message identifier.

Step S608, the OVS-agent (consumer) process of HOST-a mutexecutes the periodic task, and obtains the message identifier stored in the database.

Key is obtained from Redis as OVS-AGENT-HOST-a (combination of mutexchange name and routing key) list element, and Redis _ Message _ ids is [0,1,2,3,4,5,6,7,8,9 ].

The Message identifier miss _ Message _ ids that the consumer does not consume can be obtained through calculation, and the calculation formula is miss _ Message _ ids, which is Redis _ Message _ ids-local _ Message _ ids. The calculation result is miss _ Message _ ids ═ 7, 8.

At this time, the Message identifiers in the Redis may be synchronized, because the Message identifiers are saved in the Redis and then pruned, which limits the length of the list in the Redis, may cause that the old Message identifiers have been deleted in the Redis, and these Message identifiers deleted by the Redis should also be deleted in the consumer process, so as to avoid that the larger the local _ Message _ ids list in the consumer process is, the larger the list is. The treatment protocol was as follows: the local _ Message _ ids-Redis _ Message _ ids; local _ Message _ ids. difference _ update (amplified _ ids).

In the above steps, the consumer process (ovs-agent) finds out which message identifiers are not consumed by itself, but the message identifier may also be caused by a time delay rather than a failure problem, so a time threshold may be set for performing a timeout detection, and the message identifier cannot be simply considered to be lost, which is specifically referred to as the following:

taking out the elements in the miss _ Message _ ids one by one, taking the miss _ Message _ id as key, obtaining the timestamp at the moment as value, and storing the value into a matching table map, wherein the miss _ Message _ map [ '7' ] ═ now _ time, and the miss _ Message _ map [ '8' ] ═ now _ time;

traversing the miss _ Message _ map, acquiring a current timestamp, subtracting the value of each option in the miss _ Message _ map from the current timestamp, if the value exceeds the threshold value, considering that the Message identifier serving as the key value is lost, and notifying the operation and maintenance personnel through an alarm interface.

In the scheme, Redis is taken as an example to realize data transmission between a producer and a consumer, and databases such as mysql and the like can be adopted to replace the Redis or synchronize message identifications in other interprocess communication modes. By adopting the technical scheme, the producer and the consumer realize Remote Procedure Call (RPC) by using the rabbitmq, and realize inter-process communication by using components of other third parties, the RPC is realized by the rabbitmq, and the rabbitmq is actually also an inter-process communication, and two tools are mainly used for realizing the inter-process communication to detect whether the communication data between the producer and the consumer is abnormal or not. The scheme can add detection outside the rabbitmq, and avoids the condition that a worker or a process log is not sensed due to message loss caused by the problem of the rabbitmq.

It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present application is not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application.

Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present application.

According to another aspect of the embodiments of the present application, there is also provided a message loss detection apparatus for a message queue, for implementing the message loss detection method for a message queue. Fig. 7 is a schematic diagram of an alternative message loss detection apparatus for a message queue according to an embodiment of the present application, and as shown in fig. 7, the apparatus may include:

a first receiving unit 701, configured to receive a service message sent by a producer and a message identifier of the service message, where the service message is used to indicate a target task to be executed by a consumer;

a storage unit 703 for storing the service message to a message queue and storing the message identifier to a database, wherein the message queue and the database are in independent communication with the consumer;

a first sending unit 705, configured to send the service message to the consumer through the message queue, so that the consumer executes the target task and stores a message identifier of the service message;

a first detecting unit 707, configured to send, when the database receives a query request of a consumer, a message identifier to the consumer through the database, so that the consumer determines whether there is a lost service message according to the message identifier stored in the database and the message identifier stored in the consumer.

It should be noted that the first receiving unit 701 in this embodiment may be configured to execute step S302 in this embodiment, the storage unit 703 in this embodiment may be configured to execute step S304 in this embodiment, the first sending unit 705 in this embodiment may be configured to execute step S306 in this embodiment, and the first detecting unit 707 in this embodiment may be configured to execute step S308 in this embodiment.

Through the module, the producer and the consumer can realize data synchronization through the provided database, and the consumer can determine whether the lost message exists by comparing the message identifier stored in the database with the local message identifier, so that the technical problem that the lost message cannot be accurately detected in the related technology can be solved, and the technical effect of monitoring the lost message is further achieved.

Optionally, the storage unit may further be configured to: after receiving a message identifier of a service message sent by a producer, storing the message identifier into a preset storage space, wherein the preset storage space is used for performing redundant backup on the message identifier stored in a database; so that the message identifier backed up by the preset storage space is copied to the database for storage under the condition that the message identifier stored in the database is lost.

Optionally, the storage unit may further be configured to: under the condition that the type of the service message is a first type, the service message is stored in all message queues, the message identification is associated with the first type and then stored in a database, wherein the message of the first type is set to be sent to all message queues for all consumers to consume; and under the condition that the type of the business message is a second type, storing the business message to a message queue associated with the consumer specified by the business message, and storing a message identifier to a database after associating the message identifier with the consumer specified by the business message, wherein the message of the second type is set to be sent to the message queue associated with the specified consumer for consumption by the specified consumer.

Optionally, the apparatus may further include: and the first deleting unit is used for deleting the message identification which is inquired by the consumer and the corresponding service message is consumed by the consumer in the database after the message identification is sent to the consumer through the database.

According to another aspect of the embodiments of the present application, there is also provided a message loss detection apparatus for a message queue, for implementing the message loss detection method for a message queue. Fig. 8 is a schematic diagram of an alternative message loss detection apparatus for a message queue according to an embodiment of the present application, and as shown in fig. 8, the apparatus may include:

a second receiving unit 801, configured to receive a service message sent from a message queue of a message processing system and store a message identifier of the service message locally, where the service message is used to indicate a target task to be performed by a consumer.

An obtaining unit 803, configured to send an identifier obtaining request to a database of the message processing system to obtain the message identifier stored in the database.

A second detecting unit 805, configured to determine whether there is a lost service message according to the locally stored message identifier and the message identifier stored in the database.

It should be noted that the second receiving unit 801 in this embodiment may be configured to execute step S402 in this embodiment, the obtaining unit 803 in this embodiment may be configured to execute step S404 in this embodiment, and the second detecting unit 805 in this embodiment may be configured to execute step S406 in this embodiment.

Optionally, the second detection unit may be further configured to: comparing the locally stored message identification with the message identification stored in the database to obtain a comparison result; and determining whether the lost service message exists according to the comparison result.

Optionally, the second detection unit may be further configured to: determining the service message corresponding to the first type message identifier as a lost service message under the condition that the comparison result shows that the first type message identifier exists; determining that no lost service message exists under the condition that the comparison result shows that the first type message identification does not exist; the first type message identification represents the message identification which is stored in the database and cannot be searched in the local storage.

Optionally, the apparatus may further include: and the second deleting unit is used for deleting the second type of message identification from the local storage, wherein the second type of message identification is the message identification which is determined to exist in the database and the local storage through the comparison result.

Optionally, the second detection unit may be further configured to: starting a timer when the comparison result shows that the first type of message identification exists; when the timing duration of the timer reaches a first time duration, determining whether a service message corresponding to the first type message identifier is received; and when determining that the service message corresponding to the first type message identifier is a lost service message, if determining that the service message corresponding to the first type message identifier is not received, determining that the service message corresponding to the first type message identifier is lost.

Optionally, the apparatus may further include: and the warning unit is used for generating warning information after determining that the service message corresponding to the first type message identifier is lost so as to prompt the user that the service message corresponding to the message identifier of the first type message identifier is lost.

Optionally, the second detection unit may be further configured to: when the timing duration of the timer reaches a first time duration, judging whether a first type message identifier still exists; and when the first-class message identification does not exist, determining that the service message corresponding to the first-class message identification is received.

Optionally, the obtaining unit may be further configured to: and periodically sending an identification acquisition request to the database according to a preset query period.

According to another aspect of the embodiments of the present application, there is also provided a message loss detection apparatus for a message queue, for implementing the message loss detection method for a message queue. Fig. 9 is a schematic diagram of an alternative message loss detection apparatus for a message queue according to an embodiment of the present application, and as shown in fig. 9, the apparatus may include:

a generating unit 901, configured to generate a service message to be sent, where the service message includes a message identifier;

a second sending unit 903, configured to send a service message to a message queue of the message processing system, so that the service message is stored in the message queue for a consumer to read and consume;

a third sending unit 905, configured to send a message identifier of the service message to a database of the message processing system, so that the message identifier is stored in the database, so that a consumer may query when determining whether there is a lost service message.

It should be noted that the generating unit 901 in this embodiment may be configured to execute step S502 in this embodiment, the second sending unit 903 in this embodiment may be configured to execute step S504 in this embodiment, and the third sending unit 905 in this embodiment may be configured to execute step S506 in this embodiment.

Optionally, the third sending unit may be further configured to: under the condition that the type of the business message is a first type, the business message is sent to all message queues so that the business message is stored in all message queues, wherein the message of the first type is set to be sent to all message queues for all consumers to consume; and in the case that the type of the business message is a second type, sending the business message to a message queue associated with the specified consumer of the business message, wherein the message of the second type is set to be sent to the message queue associated with the specified consumer for consumption by the specified consumer.

It should be noted here that the modules described above are the same as the examples and application scenarios implemented by the corresponding steps, but are not limited to the disclosure of the above embodiments. It should be noted that the modules described above as a part of the apparatus may be run in a hardware environment as shown in fig. 2, may be implemented by software, and may also be implemented by hardware, where the hardware environment includes a network environment.

According to another aspect of the embodiments of the present application, there is also provided a server or a terminal for implementing the message loss detection method for a message queue.

Fig. 10 is a block diagram of a terminal according to an embodiment of the present application, and as shown in fig. 10, the terminal may include: one or more processors 1001 (only one of which is shown in fig. 10), memory 1003, and a transmission apparatus 1005, the terminal may further include an input-output device 1007, as shown in fig. 10.

The memory 1003 may be used to store software programs and modules, such as program instructions/modules corresponding to the method and apparatus for detecting a message loss of a message queue in this embodiment, and the processor 1001 executes various functional applications and data processing by running the software programs and modules stored in the memory 1003, that is, implements the above-described method for detecting a message loss of a message queue. The memory 1003 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 1003 may further include memory located remotely from the processor 1001, which may be connected to a terminal over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The transmitting device 1005 is used for receiving or transmitting data via a network, and can also be used for data transmission between a processor and a memory. Examples of the network may include a wired network and a wireless network. In one example, the transmitting device 1005 includes a Network adapter (NIC) that can be connected to a router via a Network cable and other Network devices to communicate with the internet or a local area Network. In one example, the transmitting device 1005 is a Radio Frequency (RF) module, which is used for communicating with the internet in a wireless manner.

Among them, the memory 1003 is used to store an application program, in particular.

The processor 1001 may call an application stored in the memory 1003 via the transmitting device 1005 to perform the following steps:

receiving a service message sent by a producer and a message identifier of the service message, wherein the service message is used for representing a target task to be executed by a consumer;

storing the business message to a message queue and storing the message identification to a database, wherein the message queue and the database are in independent communication with the consumer;

sending the service message to the consumer through the message queue so that the consumer executes the target task and stores the message identifier of the service message;

and when the database receives a query request of a consumer, the message identification is sent to the consumer through the database, so that the consumer determines whether the lost service message exists according to the message identification stored in the database and the message identification stored in the consumer.

The processor 1001 is further configured to perform the following steps:

receiving a service message sent by a message queue from a message processing system and locally storing a message identifier of the service message, wherein the service message is used for representing a target task to be executed by a consumer;

sending an identification acquisition request to a database of the message processing system to acquire a message identification stored in the database;

and determining whether the lost service message exists according to the locally stored message identification and the message identification stored in the database.

The processor 1001 is further configured to perform the following steps:

generating a service message to be sent, wherein the service message comprises a message identifier;

sending the service message to a message queue of the message processing system so that the service message is stored in the message queue for a consumer to read and consume;

and sending the message identification of the service message to a database of the message processing system, so that the message identification is stored in the database for a consumer to inquire when determining whether the lost service message exists.

Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment is not described herein again.

It will be understood by those skilled in the art that the structure shown in fig. 10 is merely an illustration, and the terminal may be a terminal device such as a smart phone (e.g., an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, and a Mobile Internet Device (MID), a PAD, etc. Fig. 10 is a diagram illustrating a structure of the electronic device. For example, the terminal may also include more or fewer components (e.g., network interfaces, display devices, etc.) than shown in FIG. 10, or have a different configuration than shown in FIG. 10.

Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by a program instructing hardware associated with the terminal device, where the program may be stored in a computer-readable storage medium, and the storage medium may include: flash disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.

Embodiments of the present application also provide a storage medium. Alternatively, in this embodiment, the storage medium may be used to execute a program code of a message loss detection method for a message queue.

Optionally, in this embodiment, the storage medium may be located on at least one of a plurality of network devices in a network shown in the above embodiment.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps:

Optionally, the storage medium is further arranged to store program code for performing the steps of:

Optionally, in this embodiment, the storage medium may include, but is not limited to: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.

The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.

The integrated unit in the above embodiments, if implemented in the form of a software functional unit and sold or used as a separate product, may be stored in the above computer-readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or a part of or all or part of the technical solution contributing to the prior art may be embodied in the form of a software product stored in a storage medium, and including instructions for causing one or more computer devices (which may be personal computers, servers, network devices, or the like) to execute all or part of the steps of the method described in the embodiments of the present application.

In the above embodiments of the present application, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

In the several embodiments provided in the present application, it should be understood that the disclosed client may be implemented in other manners. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one type of division of logical functions, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The foregoing is only a preferred embodiment of the present application and it should be noted that those skilled in the art can make several improvements and modifications without departing from the principle of the present application, and these improvements and modifications should also be considered as the protection scope of the present application.

Claims

1. A message loss detection method for a message queue, the method is applied to a message processing system, the message processing system comprises the message queue and a database, and the method is characterized by comprising the following steps:

storing the business message to the message queue and the message identification to the database, wherein the message queue and database are in independent communication with the consumer;

sending the business message to the consumer through the message queue so that the consumer executes the target task and stores the message identification of the business message;

and when the database receives the query request of the consumer, the message identification is sent to the consumer through the database, so that the consumer determines whether the lost service message exists according to the message identification stored in the database and the message identification stored in the consumer.

2. The method of claim 1, wherein after receiving the message identifier of the service message sent by the producer, the method further comprises:

storing the message identification to a preset storage space, wherein the preset storage space is used for carrying out redundancy backup on the message identification stored in the database;

and under the condition that the message identification stored in the database is lost, copying the message identification backed up by the preset storage space to the database for storage.

3. The method of claim 1 or 2, wherein storing the service message to the message queue and the message identification to the database comprises:

under the condition that the type of the service message is a first type, storing the service message to all the message queues, associating the message identification with the first type, and storing the message identification to the database, wherein the message of the first type is set to be sent to all the message queues for all the consumers to consume;

and under the condition that the type of the business message is a second type, storing the business message to the message queue associated with the consumer specified by the business message, and storing the message identifier to the database after being associated with the consumer specified by the business message, wherein the message of the second type is set to be sent to the message queue associated with the specified consumer for consumption.

4. The method of claim 1 or 2, wherein after sending the message identification to the consumer via the database, the method further comprises:

and deleting the message identification which is inquired by the consumer and the corresponding service message is consumed by the consumer in the database.

5. A message loss detection method for a message queue, the method being applied to a consumer of messages, comprising:

sending an identification acquisition request to a database of the message processing system to acquire the message identification stored in the database;

and determining whether the lost service message exists according to the locally stored message identifier and the message identifier stored in the database.

6. The method of claim 5, wherein determining whether there is a lost service message based on a locally stored message identification and a message identification stored in the database comprises:

comparing the locally stored message identification with the message identification stored in the database to obtain a comparison result;

and determining whether the lost service message exists according to the comparison result.

7. The method of claim 6, wherein determining whether there is a lost service message according to the comparison comprises:

determining that the service message corresponding to the first type message identifier is a lost service message under the condition that the comparison result is that the first type message identifier exists;

determining that no lost service message exists under the condition that the comparison result indicates that the first type message identification does not exist;

the first type message identification represents the message identification which is stored in the database and can not be searched in the local storage.

8. The method of claim 6, further comprising:

and deleting a second type of message identification from the local storage, wherein the second type of message identification is the message identification which is determined by the comparison result and exists in the database and the local storage.

9. The method of claim 7, further comprising:

starting a timer when the comparison result shows that the first type of message identification exists;

when the timing duration of the timer reaches a first duration, determining whether a service message corresponding to the first type message identifier is received;

the determining that the service message corresponding to the first type message identifier is a lost service message includes:

and if determining that the service message corresponding to the first type message identifier is not received, determining that the service message corresponding to the first type message identifier is lost.

10. The method of claim 9, further comprising:

and generating alarm information to prompt a user that the service message corresponding to the first type message identifier is lost after determining that the service message corresponding to the first type message identifier is lost.

11. The method according to claim 9, wherein the determining whether the service message corresponding to the first type message identifier is received when the timing duration of the timer reaches a first duration comprises:

when the timing duration of the timer reaches a first time duration, judging whether a first type message identifier still exists;

and when the first-class message identification does not exist, determining that the service message corresponding to the first-class message identification is received.

12. The method of claim 5, wherein sending an identification acquisition request to a database of the message processing system comprises:

and periodically sending the identification acquisition request to the database according to a preset query period.

13. A message loss detection method for a message queue, the method being applied to a producer of a message, the method comprising:

sending the service message to a message queue of a message processing system so that the service message is stored in the message queue for a consumer to read and consume;

and sending the message identification of the service message to a database of the message processing system so that the message identification is stored in the database for the consumer to inquire when determining whether the lost service message exists.

14. The method of claim 13, wherein sending the traffic message to a message queue of a message processing system comprises:

under the condition that the type of the business message is a first type, the business message is sent to all the message queues so that the business message is stored in all the message queues, wherein the message of the first type is set to be sent to all the message queues for all the consumers to consume;

and if the type of the business message is a second type, sending the business message to the message queue associated with the consumer designated by the business message, wherein the message of the second type is set to be sent to the message queue associated with the designated consumer for consumption by the designated consumer.

15. A message loss detection device for a message queue, applied to a message processing system, wherein the message processing system comprises the message queue and a database, and is characterized by comprising:

the system comprises a first receiving unit, a second receiving unit and a processing unit, wherein the first receiving unit is used for receiving a service message sent by a producer and a message identifier of the service message, and the service message is used for representing a target task to be executed by a consumer;

the storage unit is used for storing the service message to the message queue and storing the message identifier to the database, wherein the message queue and the database are in independent communication with the consumer;

the first sending unit is used for sending the business message to the consumer through the message queue so that the consumer can execute the target task and store the message identifier of the business message;

and the first detection unit is used for sending the message identifier to the consumer through the database when the database receives the query request of the consumer, so that the consumer can determine whether the lost service message exists according to the message identifier stored in the database and the message identifier stored in the consumer.

16. A message loss detection apparatus for a message queue, applied to a consumer of a message, comprising:

the second receiving unit is used for receiving a service message sent by a message queue of a message processing system and locally storing a message identifier of the service message, wherein the service message is used for representing a target task to be executed by a consumer;

an obtaining unit, configured to send an identifier obtaining request to a database of the message processing system to obtain a message identifier stored in the database;

and the second detection unit is used for determining whether the lost service message exists according to the locally stored message identifier and the message identifier stored in the database.

17. A message loss detection apparatus for a message queue, applied to a producer of a message, comprising:

a generating unit, configured to generate a service message to be sent, where the service message includes a message identifier;

the second sending unit is used for sending the business message to a message queue of a message processing system so that the business message is stored in the message queue for a consumer to read and consume;

and the third sending unit is used for sending the message identifier of the service message to a database of the message processing system so that the message identifier is stored in the database for the consumer to inquire when determining whether the lost service message exists.

18. A storage medium, characterized in that the storage medium comprises a stored program, wherein the program when executed performs the method of any of the preceding claims 1 to 14.

19. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the method of any of the preceding claims 1 to 14 by means of the computer program.