WO2021180025A1 - Message processing method and apparatus, electronic device and medium - Google Patents

Message processing method and apparatus, electronic device and medium Download PDF

Info

Publication number
WO2021180025A1
WO2021180025A1 PCT/CN2021/079512 CN2021079512W WO2021180025A1 WO 2021180025 A1 WO2021180025 A1 WO 2021180025A1 CN 2021079512 W CN2021079512 W CN 2021079512W WO 2021180025 A1 WO2021180025 A1 WO 2021180025A1
Authority
WO
WIPO (PCT)
Prior art keywords
message
backup
queue
consumed
processing
Prior art date
Application number
PCT/CN2021/079512
Other languages
French (fr)
Chinese (zh)
Inventor
郑舒力
Original Assignee
北京金山云网络技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京金山云网络技术有限公司 filed Critical 北京金山云网络技术有限公司
Publication of WO2021180025A1 publication Critical patent/WO2021180025A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/54Interprogram communication
    • G06F9/546Message passing systems or structures, e.g. queues
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14Error detection or correction of the data by redundancy in operation
    • G06F11/1402Saving, restoring, recovering or retrying
    • G06F11/1446Point-in-time backing up or restoration of persistent data
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/06Buying, selling or leasing transactions
    • G06Q30/0601Electronic shopping [e-shopping]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00Indexing scheme relating to G06F9/00
    • G06F2209/54Indexing scheme relating to G06F9/54
    • G06F2209/547Messaging middleware
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00Indexing scheme relating to G06F9/00
    • G06F2209/54Indexing scheme relating to G06F9/54
    • G06F2209/548Queue

Definitions

  • This application relates to the field of communication technology, and in particular to a message processing method, device, electronic equipment, and medium.
  • the producer refers to the source of the message, which is used to write the produced message into the message system; the consumer is used to obtain and process the message from the message system.
  • the order message generated by the user through the terminal is recorded in the message system, and each order message in the message system needs to be processed by the server (consumer).
  • the message system can be RocketMQ or Kafka.
  • Kafka Kafka
  • RocketMQ and Kafka are distributed messaging systems, they are composed of multiple components.
  • Kafka includes components such as controller, coordinator, log management, and copy manager. Because RocketMQ and Kafka include many components, the deployment and operation of RocketMQ and Kafka are complicated, and the development workload for terminals used by producers and consumers is relatively large, which leads to more complicated methods for processing messages.
  • the purpose of the embodiments of the present application is to provide a message processing method, device, electronic device, and medium, so as to implement a simplified method for processing messages.
  • the specific technical solutions are as follows:
  • an embodiment of the present application provides a message processing method.
  • the method includes: monitoring whether a message to be consumed is included in the message main queue of the remote dictionary service Redis, and the message to be consumed is a message written by the message producer to the message main queue ; When it is monitored that the message to be consumed is included in the main message queue, the message to be consumed is obtained, and the message to be consumed is transferred to the message backup queue; the message to be consumed is processed.
  • an embodiment of the present application provides a message processing device, which includes: a monitoring module configured to monitor whether messages to be consumed are included in the main message queue of the remote dictionary service Redis, and the messages to be consumed are written by the message producer Messages in the main message queue; acquisition module, used to obtain the messages to be consumed when the listening module monitors that the message main queue includes messages to be consumed, and transfer the messages to be consumed to the message backup queue; processing module, used to process the acquisition The message to be consumed obtained by the module.
  • a monitoring module configured to monitor whether messages to be consumed are included in the main message queue of the remote dictionary service Redis, and the messages to be consumed are written by the message producer Messages in the main message queue
  • acquisition module used to obtain the messages to be consumed when the listening module monitors that the message main queue includes messages to be consumed, and transfer the messages to be consumed to the message backup queue
  • processing module used to process the acquisition The message to be consumed obtained by the module.
  • an embodiment of the present application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus.
  • the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory is used for storing Computer program; processor, used to implement the steps of any of the above message processing methods when executing the program stored on the memory.
  • the embodiments of the present application also provide a computer-readable storage medium.
  • the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above message processing methods are implemented.
  • the embodiments of the present application also provide a computer program product containing instructions, which when run on a computer, cause the computer to execute any of the above message processing methods.
  • the embodiments of the present application provide a computer program, which when the computer program runs on a computer, causes the computer to execute the steps of the message processing method provided in the embodiments of the present application.
  • the message producer can write the message to be consumed into the message main queue in Redis, the consumer can obtain the message to be consumed from Redis and transfer the message to be consumed to the message backup queue of Redis , And then process the messages to be consumed. Since the message to be consumed is transferred to the message backup queue at the same time as it is obtained, even if the consumer does not process the message to be consumed successfully, the message to be consumed can still be obtained from the message backup queue and processed again, so the message producer generates The omissions will not be dealt with when each message is to be consumed. And because Redis is an in-memory database, compared to RocketMQ and Kafka and other messaging systems that include multiple components, Redis has a simple structure, convenient deployment, operation and maintenance, and simplifies the method of processing messages.
  • FIG. 1 is a schematic structural diagram of a message processing system provided by an embodiment of this application.
  • FIG. 2 is a flowchart of a message processing method provided by an embodiment of the application
  • FIG. 3 is a flowchart of another message processing method provided by an embodiment of the application.
  • FIG. 4 is a schematic structural diagram of a message processing apparatus provided by an embodiment of the application.
  • FIG. 5 is a schematic structural diagram of an electronic device provided by an embodiment of the application.
  • the device that generates the message is called the message producer, and the device that processes the message is called the message consumer.
  • the message producer can be a terminal or server, and the message consumer can also be a terminal or server.
  • the terminal can be a mobile phone, a tablet computer, a computer, and other electronic devices.
  • the embodiment of the present application does not specifically limit the specific equipment corresponding to the message producer and the message consumer.
  • the embodiment of the present application provides a message processing system.
  • the system includes: a message producer 101, a message consumer 102, and a database 103.
  • the message producer 101 and the message consumer 102 in FIG. 1 both take a server as an example.
  • the message producer 101 is set to produce messages, and writes the produced messages into the database 103.
  • the message consumer 102 is set to read and process messages from the database 103.
  • the database 103 is set up to store messages.
  • the database can be a remote dictionary service (Redis).
  • the Redis database may be deployed in a Redis server.
  • the Redis server in the embodiment of the present application includes a main Redis server and a backup Redis server corresponding to the main Redis server.
  • the main Redis server includes the Redis message main queue
  • the backup Redis server includes the Redis message backup queue
  • the Redis message backup queue is used to back up messages to be consumed in the Redis message main queue.
  • Each master Redis server corresponds to at least one slave Redis server, which is equivalent to one Redis message master queue corresponding to at least one Redis message backup queue.
  • the message main queue of Redis is referred to as the message main queue for short
  • the message backup queue of Redis is referred to as the message backup queue for short.
  • the number of devices in the message processing system is only an example provided in the embodiment of the present application, and the embodiment of the present application does not specifically limit the number of devices in the message processing system.
  • an embodiment of the present application provides a message processing method, which can be applied to the message consumer of the message processing system shown in FIG. 1, and the method includes the following steps.
  • Step 201 Monitor whether the message to be consumed is included in the message main queue of Redis.
  • the message to be consumed is the message written by the message producer into the main message queue.
  • the message consumer can monitor whether the message to be consumed is included in the message main queue of Redis through the blocking command (BRPOPLPUSH).
  • the BRPOPLPUSH command is a blocking command.
  • the BRPOPLPUSH command is in a waiting state until the main queue includes messages to be consumed, and the messages to be consumed are obtained from the main queue.
  • the message producer can write a batch of produced messages into the message main queue of Redis each time through a push command (LPUSH).
  • LUSH push command
  • the message producer will receive a return message indicating that the batch of messages was successfully written. If the message producer does not receive the return message within the preset period of time when the message is written to the main message queue, the batch of messages can be rewritten into the main message queue.
  • Redis in the embodiment of the present application can enable Append Only File (AOF) persistence, so that the data in the message main queue and message backup queue of Redis can be written to disk In, the integrity of the message is guaranteed.
  • OAF Append Only File
  • Step 202 When it is monitored that the message to be consumed is included in the main message queue, the message to be consumed is obtained, and the message to be consumed is transferred to the message backup queue.
  • the message consumer can start multiple threads, while obtaining the message to be consumed from the main message queue through the BRPOPLPUSH command, delete the message to be consumed from the main message queue, and insert the message to be consumed into the message backup In the queue.
  • Atomicity refers to the indivisibility of a transaction, all operations of a transaction are executed without interruption, or an operation is not executed.
  • the above-mentioned atomicity means that the operation process of deleting messages to be consumed from the main message queue and inserting them into the message backup queue cannot be interrupted, and there will be no interruption of only messages to be consumed from the main message queue. Delete from the column, or just insert the message to be consumed into the message backup queue.
  • the message to be consumed is deleted from the main message queue and the message to be consumed is not inserted into the backup message queue, the message to be consumed is not stored in Redis at this time.
  • the message consumer fails to process the message to be consumed, the message to be consumed cannot be obtained again, so that the message to be consumed is not successfully processed.
  • the operation of transferring the message to be consumed to the message backup queue is atomic, which reduces the occurrence of the above situation and reduces the omission of the message to be consumed in the message processing process.
  • Step 203 Process the message to be consumed.
  • the technical solution provided by the embodiments of the present application brings at least the following beneficial effects: since the message producer can write the message to be consumed into the message main queue in Redis, the consumer can obtain the message to be consumed from Redis and transfer the message to be consumed Transfer to the message backup queue of Redis, and then process the messages to be consumed. Since the message to be consumed is transferred to the message backup queue at the same time as it is obtained, even if the consumer does not process the message to be consumed successfully, the message to be consumed can still be obtained from the message backup queue and processed again, so the message producer generates The omissions will not be dealt with when each message is to be consumed. And because Redis is an in-memory database, compared to RocketMQ and Kafka and other messaging systems that include multiple components, Redis has a simple structure, convenient deployment, operation and maintenance, and simplifies the method of processing messages.
  • message main queues and message backup queues there may be one or more message main queues and message backup queues in Redis, and message consumers can monitor one of the message main queues by opening multiple threads, or monitor multiple message main queues separately, The messages to be consumed in each message main queue are transferred to the corresponding message backup queue.
  • the number of message main queues may be the same as the number of message backup queues, and there is a one-to-one correspondence.
  • the message consumer monitors multiple message main queues at the same time, and when a message to be consumed is monitored from one message main queue, the message to be consumed is transferred to the message backup queue corresponding to the message main queue.
  • the message main queue and message backup queue may store messages in the form of key-value, where key is the identifier of the message queue, and value is the identifier of the message.
  • key is the identifier of the message queue
  • value is the identifier of the message.
  • One key can correspond to multiple values.
  • the main message queue and the message backup queue may be monitored by multiple message consumers.
  • the message consumer can delete the message to be consumed in the message backup queue.
  • the message to be consumed in the message backup queue can be deleted through a delete command (LREM).
  • LREM command is a Redis command used to remove elements in the list.
  • the LREM command is used to delete backup messages in the message backup queue.
  • the message consumer keeps the message to be consumed in the message backup queue.
  • the technical solution of the embodiment of the present application can also bring the following beneficial effects: the message consumer will delete the message to be consumed in the message backup queue only when the message to be consumed is successfully processed. If the processing of the message to be consumed fails, the message to be consumed can also be acquired and processed again until the processing of the message to be consumed is successful, so the embodiment of the present application reduces the omission of the message to be consumed in the message processing process.
  • the message consumer since the message consumer obtains the message to be consumed from the main message queue at the same time, deletes the message to be consumed from the message main queue, avoiding the situation that other message consumers repeatedly obtain and process the message to be consumed from the main message queue, and reduce The situation where the same message to be consumed is processed repeatedly.
  • message consumers can also process backup messages in the message backup queue, including the following steps:
  • Step 301 Determine whether a backup message is included in the message backup queue every preset time duration.
  • the preset duration of monitoring blocking can be set in the BRPOPLPUSH command of the monitoring message main queue, and the message consumer can monitor whether the backup message is included in the message backup queue by using the BRPOPLPUSH command to monitor whether the backup message is included in the message backup queue at each interval preset duration.
  • the preset duration may be a random value within a preset range, and the preset range may be 5-10 minutes.
  • Step 302 If a backup message is included in the message backup queue, obtain the backup message.
  • the message consumer can use the LINDEX command to obtain the backup message that enters the message backup queue first among the backup messages included in the message backup queue according to the order in which each backup message enters the message backup queue.
  • the LINDEX command is a Redis command used to obtain elements in the list.
  • the LINDEX command is used to obtain the backup message that first enters the backup queue.
  • Step 303 Process the backup message.
  • the message consumer can process the obtained backup message.
  • Step 304 If the backup message is processed successfully, delete the backup message from the message backup queue.
  • the message consumer can delete the backup message in the message backup queue through the LREM command.
  • the message consumer can delete the backup message from the message backup queue. If the backup message is not processed successfully, the message consumer does not delete the backup message from the backup queue.
  • the technical solution provided by the embodiments of the present application can also bring the following beneficial effects: because the message consumer can obtain and process the backup message from the message backup queue every interval preset time, and the message backup can be deleted when the backup message is processed successfully.
  • the backup messages in the queue reduce the omission of messages during message processing.
  • the message backup queue in the embodiment of the present application may further include the timestamp of each backup message.
  • the time stamp is the time when the backup message enters the message backup queue.
  • Step 1 Obtain the timestamp of the backup message.
  • Step 2 If the time difference between the current time and the time stamp of the backup message is greater than the preset time difference, execute the step of processing the backup message.
  • the time difference between the current time and the timestamp of the backup message is greater than the preset time difference, it means that the backup message has been in the message backup queue for a long time and has not been deleted, and the processing of the backup message is more likely to fail. , Needs to be processed again.
  • the acquired backup message is not processed.
  • the time difference between the current time and the timestamp of the backup message is not greater than the preset time difference, it means that the backup message has entered the message backup queue for a short time and has not been deleted, and the backup message may be being processed by the message consumer , So you don’t need to process the backup message.
  • the preset time difference is ten minutes.
  • the technical solution provided by the embodiments of this application can also bring about the following beneficial effects: since it takes a certain time for the message consumer to process the message, and the message consumer will delete the backup message in the backup message queue after the processing is successful, the implementation of this application For example, it can process backup messages that take a long time to enter the message backup queue, reducing the number of repeated processing of messages.
  • an embodiment of the present application provides a message processing device, which includes: a monitoring module 401, an acquisition module 402, and a processing module 403;
  • the monitoring module 401 is configured to monitor whether messages to be consumed are included in the main message queue of the remote dictionary service Redis, and the messages to be consumed are messages written by the message producer to the main message queue;
  • the obtaining module 402 is configured to obtain the message to be consumed when the monitoring module 701 detects that the message to be consumed is included in the main message queue, and transfer the message to be consumed to the message backup queue;
  • the processing module 403 is configured to process the message to be consumed acquired by the acquiring module 402.
  • the device may further include: a deletion module
  • the delete module is set to delete the message to be consumed in the message backup queue if the message to be consumed is processed successfully after processing the message to be consumed.
  • the device may further include: a determination module and a deletion module;
  • the determining module is set to the preset duration of each interval to determine whether the backup message is included in the message backup queue;
  • the obtaining module 402 is further configured to obtain the backup message if the backup message is included in the message backup queue;
  • the processing module 403 is also set to process backup messages
  • the delete module is set to delete the backup message from the message backup queue if the backup message is processed successfully.
  • the obtaining module 402 may be configured as:
  • the backup message that enters the message backup queue first among the backup messages included in the message backup queue is obtained.
  • the message backup queue further includes a timestamp of each backup message, and the timestamp is the time when the backup message enters the message backup queue;
  • the device may further include: a calling module;
  • the obtaining module is also set to obtain the timestamp of the backup message before processing the backup message;
  • the calling module is set to call the processing module to execute the step of processing the backup message when the time difference between the current time and the time stamp of the backup message is greater than the preset time difference.
  • the embodiment of the present application also provides an electronic device. As shown in FIG. Complete the communication between each other,
  • the memory 503 is set to store computer programs
  • the processor 501 is configured to implement the method steps in the foregoing method embodiment when it is configured to execute the program stored in the memory 503.
  • the communication bus mentioned in the above electronic device may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc.
  • PCI Peripheral Component Interconnect
  • EISA Extended Industry Standard Architecture
  • the communication bus can be divided into address bus, data bus, control bus and so on. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
  • the communication interface is used for communication between the above-mentioned electronic device and other devices.
  • the memory may include random access memory (Random Access Memory, RAM), and may also include non-volatile memory (Non-Volatile Memory, NVM), such as at least one disk storage.
  • NVM non-Volatile Memory
  • the memory may also be at least one storage device located far away from the foregoing processor.
  • the above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (Network Processor, NP), etc.; it can also be a digital signal processor (Digital Signal Processing, DSP), a dedicated integrated Circuit (Application Specific Integrated Circuit, ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
  • CPU central processing unit
  • NP Network Processor
  • DSP Digital Signal Processing
  • ASIC Application Specific Integrated Circuit
  • FPGA Field-Programmable Gate Array
  • a computer-readable storage medium stores a computer program, and the computer program implements any of the above message processing methods when executed by a processor. A step of.
  • a computer program product containing instructions is also provided, which when running on a computer, causes the computer to execute any message processing method in the above-mentioned embodiments.
  • a computer program which when running on a computer, causes the computer to execute any message processing method in the above-mentioned embodiments.
  • the computer may be implemented in whole or in part by software, hardware, firmware, or any combination thereof.
  • software it can be implemented in the form of a computer program product in whole or in part.
  • the computer program product includes one or more computer instructions.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
  • the computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium.
  • the computer instructions may be transmitted from a website, computer, server, or data center.
  • the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center integrated with one or more available media.
  • the usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, and a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)).
  • the message can use the message main queue and message backup queue in Redis as message middleware, which can simplify the difficulty of deploying message middleware and simplify the process of message processing. method.

Abstract

A message processing method and apparatus, an electronic device and a medium, which relate to the technical field of communications and can simplify a method for message processing. The method comprises: monitoring whether a message main queue of a remote dictionary server (Redis) comprises a message to be consumed (201), wherein the message to be consumed is a message written into the message main queue by a message producer; then, when it is detected that the message main queue comprises the message to be consumed, acquiring the message to be consumed, and transferring the message to be consumed to a message backup queue (202); and processing the message to be consumed (203).

Description

一种消息处理方法、装置、电子设备及介质Message processing method, device, electronic equipment and medium
本申请要求于2020年03月13日提交中国专利局、申请号为202010174823.1发明名称为“一种消息处理方法、装置、电子设备及介质”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on March 13, 2020 with the application number 202010174823.1 and the invention title is "a message processing method, device, electronic equipment and medium", the entire content of which is incorporated by reference In this application.
技术领域Technical field
本申请涉及通讯技术领域,特别是涉及一种消息处理方法、装置、电子设备及介质。This application relates to the field of communication technology, and in particular to a message processing method, device, electronic equipment, and medium.
背景技术Background technique
随着互联网与信息技术的发展,网络中存在大量的消息,在一些特殊的场景下,需要保证生产者产生的消息被消费者处理时不会发生遗漏。其中,生产者指的是消息的产生源头,用于将生产的消息写入消息系统;消费者用于从消息系统中获取并处理消息。例如,在网购场景下,为了保证用户购物成功,用户通过终端(生产者)生成的订单消息被记录在消息系统中,消息系统中的各订单消息均需要被服务端(消费者)处理。With the development of the Internet and information technology, there are a large number of messages in the network. In some special scenarios, it is necessary to ensure that the messages generated by the producers will not be omitted when processed by the consumers. Among them, the producer refers to the source of the message, which is used to write the produced message into the message system; the consumer is used to obtain and process the message from the message system. For example, in the online shopping scenario, in order to ensure the user's successful shopping, the order message generated by the user through the terminal (producer) is recorded in the message system, and each order message in the message system needs to be processed by the server (consumer).
相关技术中,消息系统可以为RocketMQ或者Kafka,以Kafka为例,处理消息系统中各消息的方法为:生产者将生产的消息写入Kafka消息队列后,消费者在第一次从Kafka消息队列中获取消息时,从Kafka消息队列中获取最先进入消息队列的消息的序号以及预设数量的消息。消费者在处理获得的消息后,更新获得的序号(更新后的序号=获得的序号+预设数量),然后从Kafka消息队列中继续获取更新后的序号对应的消息。In related technologies, the message system can be RocketMQ or Kafka. Taking Kafka as an example, the method of processing each message in the message system is: after the producer writes the produced message to the Kafka message queue, the consumer first reads it from the Kafka message queue When getting messages from the Kafka message queue, the sequence number of the message that first enters the message queue and the preset number of messages are obtained from the Kafka message queue. After processing the obtained message, the consumer updates the obtained sequence number (updated sequence number = obtained sequence number + preset number), and then continues to obtain the message corresponding to the updated sequence number from the Kafka message queue.
而由于RocketMQ以及Kafka均属于分布式消息系统,均由多个组件组成,例如,Kafka包括控制器、协调器、日志管理、副本管理器等组件。因RocketMQ以及Kafka包括的组件较多,使得RocketMQ以及Kafka的部署运维复杂,而且对于生产者和消费者使用的终端的开发工作量较大,导致实现处理消息的方法较为复杂。Because RocketMQ and Kafka are distributed messaging systems, they are composed of multiple components. For example, Kafka includes components such as controller, coordinator, log management, and copy manager. Because RocketMQ and Kafka include many components, the deployment and operation of RocketMQ and Kafka are complicated, and the development workload for terminals used by producers and consumers is relatively large, which leads to more complicated methods for processing messages.
发明内容Summary of the invention
本申请实施例的目的在于提供一种消息处理方法、装置、电子设备及介质,以实现简化处理消息的方法。具体技术方案如下:The purpose of the embodiments of the present application is to provide a message processing method, device, electronic device, and medium, so as to implement a simplified method for processing messages. The specific technical solutions are as follows:
第一方面,本申请实施例提供了一种消息处理方法,该方法包括:监听远程字典服务Redis的消息主队列中是否包括待消费消息,待消费消息为消息生产者写入消息主队列的消息;当监听到消息主队列中包括待消费消息时,获取待消费消息,并将待消费消息转移到消息备份队列中;处理待消费消息。In the first aspect, an embodiment of the present application provides a message processing method. The method includes: monitoring whether a message to be consumed is included in the message main queue of the remote dictionary service Redis, and the message to be consumed is a message written by the message producer to the message main queue ; When it is monitored that the message to be consumed is included in the main message queue, the message to be consumed is obtained, and the message to be consumed is transferred to the message backup queue; the message to be consumed is processed.
第二方面,本申请实施例提供了一种消息处理装置,该装置包括:监听模块,设置为监听远程字典服务Redis的消息主队列中是否包括待消费消息,待消费消息为消息生产者写入消息主队列的消息;获取模块,用于当监听模块监听到消息主队列中包括待消费消息时,获取待消费消息,并将待消费消息转移到消息备份队列中;处理模块,用于处理获取模块获取的待消费消息。In a second aspect, an embodiment of the present application provides a message processing device, which includes: a monitoring module configured to monitor whether messages to be consumed are included in the main message queue of the remote dictionary service Redis, and the messages to be consumed are written by the message producer Messages in the main message queue; acquisition module, used to obtain the messages to be consumed when the listening module monitors that the message main queue includes messages to be consumed, and transfer the messages to be consumed to the message backup queue; processing module, used to process the acquisition The message to be consumed obtained by the module.
第三方面,本申请实施例提供了一种电子设备,包括处理器、通信接口、存储器和通信总线,其中,处理器,通信接口,存储器通过通信总线完成相互间的通信;存储器,用于存放计算机程序;处理器,用于执行存储器上所存放的程序时,实现上述任一消息处理方法的步骤。In the third aspect, an embodiment of the present application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus. The processor, the communication interface, and the memory communicate with each other through the communication bus; the memory is used for storing Computer program; processor, used to implement the steps of any of the above message processing methods when executing the program stored on the memory.
第四方面,本申请实施例还提供了一种计算机可读存储介质,计算机可读存储介质内存储有计算机程序,计算机程序被处理器执行时实现上述任一消息处理方法的步骤。In a fourth aspect, the embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above message processing methods are implemented.
第五方面,本申请实施例还提供了一种包含指令的计算机程序产品,当其在计算机上运行时,使得计算机执行上述任一消息处理方法。In a fifth aspect, the embodiments of the present application also provide a computer program product containing instructions, which when run on a computer, cause the computer to execute any of the above message processing methods.
第六方面,本申请实施例提供了一种计算机程序,计算机程序在计算机上运行时,使得计算机执行本申请实施例所提供的消息处理方法的步骤。In a sixth aspect, the embodiments of the present application provide a computer program, which when the computer program runs on a computer, causes the computer to execute the steps of the message processing method provided in the embodiments of the present application.
本申请实施例有益效果:由于消息生产者可以将待消费消息写入在Redis 中的消息主队列,消费者可以从Redis中获取待消费消息,并将待消费消息转移到Redis的消息备份队列中,再处理待消费消息。由于待消费消息被获取的同时被转移到消息备份队列,使得即使消费者处理待消费消息时没有处理成功,仍然可以从消息备份队列中获取该待消费消息并再次处理,因此处理消息生产者生成的各待消费消息时不会处理遗漏。而且由于Redis是一种内存数据库,相比于RocketMQ以及Kafka等包括多个组件的消息系统,Redis的结构简单,部署运维方便,简化了处理消息的方法。The beneficial effects of the embodiments of the application: since the message producer can write the message to be consumed into the message main queue in Redis, the consumer can obtain the message to be consumed from Redis and transfer the message to be consumed to the message backup queue of Redis , And then process the messages to be consumed. Since the message to be consumed is transferred to the message backup queue at the same time as it is obtained, even if the consumer does not process the message to be consumed successfully, the message to be consumed can still be obtained from the message backup queue and processed again, so the message producer generates The omissions will not be dealt with when each message is to be consumed. And because Redis is an in-memory database, compared to RocketMQ and Kafka and other messaging systems that include multiple components, Redis has a simple structure, convenient deployment, operation and maintenance, and simplifies the method of processing messages.
当然,实施本申请的任一产品或方法并不一定需要同时达到以上所述的所有优点。Of course, implementing any product or method of the present application does not necessarily need to achieve all the advantages described above at the same time.
附图说明Description of the drawings
为了更清楚地说明本申请实施例或相关技术中的技术方案,下面将对实施例或相关技术描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。In order to more clearly describe the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings that need to be used in the description of the embodiments or related technologies. Obviously, the drawings in the following description are merely present For some of the embodiments of the application, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work.
图1为本申请实施例提供的一种消息处理系统的结构示意图;FIG. 1 is a schematic structural diagram of a message processing system provided by an embodiment of this application;
图2为本申请实施例提供的一种消息处理方法流程图;FIG. 2 is a flowchart of a message processing method provided by an embodiment of the application;
图3为本申请实施例提供的另一种消息处理方法流程图;FIG. 3 is a flowchart of another message processing method provided by an embodiment of the application;
图4为本申请实施例提供的一种消息处理装置的结构示意图;FIG. 4 is a schematic structural diagram of a message processing apparatus provided by an embodiment of the application;
图5为本申请实施例提供的一种电子设备的结构示意图。FIG. 5 is a schematic structural diagram of an electronic device provided by an embodiment of the application.
具体实施方式Detailed ways
下面将结合本申请实施例中的附图,对本申请实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例仅仅是本申请一部分实施例,而不是全部的实施例。基于本申请中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例,都属于本申请保护的范围。The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.
由于相关技术中利用RocketMQ以及Kafka实现消息处理的方法较为复杂, 对于处理的消息量较少的情况下,相关技术中的消息处理方法并不适用。为了提供一种更轻量简单的处理消息的方法,本申请实施例提供了一种消息处理方法,以下进行详细描述。Since the method of using RocketMQ and Kafka to implement message processing in the related technology is relatively complicated, the message processing method in the related technology is not applicable when the amount of processed messages is small. In order to provide a lighter and simpler message processing method, an embodiment of the present application provides a message processing method, which will be described in detail below.
在本申请实施例中,将产生消息的设备称为消息生产者,将处理消息的设备称为消息消费者。消息生产者可以为终端或者服务器,消息消费者也可以为终端或者服务器。其中终端可以为手机、平板电脑、计算机等电子设备。本申请实施例对消息生产者和消息消费者所对应的具体设备不作具体限定。In the embodiments of the present application, the device that generates the message is called the message producer, and the device that processes the message is called the message consumer. The message producer can be a terminal or server, and the message consumer can also be a terminal or server. The terminal can be a mobile phone, a tablet computer, a computer, and other electronic devices. The embodiment of the present application does not specifically limit the specific equipment corresponding to the message producer and the message consumer.
本申请实施例提供了一种消息处理系统,如图1所示,该系统包括:消息生产者101、消息消费者102和数据库103。其中,图1中的消息生产者101和消息消费者102均以一台服务器为例。The embodiment of the present application provides a message processing system. As shown in FIG. 1, the system includes: a message producer 101, a message consumer 102, and a database 103. Among them, the message producer 101 and the message consumer 102 in FIG. 1 both take a server as an example.
消息生产者101设置为生产消息,并将生产的消息写入数据库103。The message producer 101 is set to produce messages, and writes the produced messages into the database 103.
消息消费者102设置为从数据库103中读取并处理消息。The message consumer 102 is set to read and process messages from the database 103.
数据库103设置为存储消息。其中,数据库可以为远程字典服务(Redis)。The database 103 is set up to store messages. Among them, the database can be a remote dictionary service (Redis).
其中,Redis数据库可以部署于Redis服务器中,本申请实施例中的Redis服务器包括主Redis服务器和主Redis服务器对应的备份Redis服务器。主Redis服务器中包括Redis的消息主队列,备份Redis服务器中包括Redis的消息备份队列,Redis的消息备份队列用于对Redis的消息主队列中的待消费消息进行备份。The Redis database may be deployed in a Redis server. The Redis server in the embodiment of the present application includes a main Redis server and a backup Redis server corresponding to the main Redis server. The main Redis server includes the Redis message main queue, the backup Redis server includes the Redis message backup queue, and the Redis message backup queue is used to back up messages to be consumed in the Redis message main queue.
本申请实施例中Redis服务器的数量可根据实际需求设置,每台主Redis服务器对应至少一台从Redis服务器,相当于一个Redis的消息主队列对应于至少一个Redis的消息备份队列。The number of Redis servers in the embodiment of the present application can be set according to actual needs. Each master Redis server corresponds to at least one slave Redis server, which is equivalent to one Redis message master queue corresponding to at least one Redis message backup queue.
在本申请实施例中,将Redis的消息主队列简称为消息主队列,将Redis的消息备份队列简称为消息备份队列。In the embodiment of the present application, the message main queue of Redis is referred to as the message main queue for short, and the message backup queue of Redis is referred to as the message backup queue for short.
图1所示的消息处理系统中,各设备的数量仅为本申请实施例提供的一种示例,本申请实施例对消息处理系统中各设备的数量不作具体限定。In the message processing system shown in FIG. 1, the number of devices in the message processing system is only an example provided in the embodiment of the present application, and the embodiment of the present application does not specifically limit the number of devices in the message processing system.
参见图2,本申请实施例提供了一种消息处理方法,可以应用于图1所示的消息处理系统的消息消费者,该方法包括如下步骤。Referring to FIG. 2, an embodiment of the present application provides a message processing method, which can be applied to the message consumer of the message processing system shown in FIG. 1, and the method includes the following steps.
步骤201,监听Redis的消息主队列中是否包括待消费消息。Step 201: Monitor whether the message to be consumed is included in the message main queue of Redis.
其中,待消费消息为消息生产者写入消息主队列的消息。Among them, the message to be consumed is the message written by the message producer into the main message queue.
一种实施方式中,消息消费者可以通过阻塞命令(BRPOPLPUSH),监听Redis的消息主队列中是否包括待消费消息。In an implementation manner, the message consumer can monitor whether the message to be consumed is included in the message main queue of Redis through the blocking command (BRPOPLPUSH).
其中,BRPOPLPUSH命令是一个阻塞命令,当BRPOPLPUSH命令监听的消息主队列不包括待消费消息时,该BRPOPLPUSH命令处于等待状态,直至该主队列包括待消费消息时,从主队列中获取待消费消息。The BRPOPLPUSH command is a blocking command. When the main message queue monitored by the BRPOPLPUSH command does not include messages to be consumed, the BRPOPLPUSH command is in a waiting state until the main queue includes messages to be consumed, and the messages to be consumed are obtained from the main queue.
在本申请实施例中,消息生产者可以通过推送命令(LPUSH),每次将一批生产的消息写入Redis的消息主队列中。在这批消息成功写入消息主队列时,消息生产者将接收到用于表示该批消息写入成功的返回消息。若消息生产者在将消息写入消息主队列的预设时间段内未接收到返回消息,则可以重新将该批消息写入消息主队列。In the embodiment of the present application, the message producer can write a batch of produced messages into the message main queue of Redis each time through a push command (LPUSH). When this batch of messages is successfully written to the main message queue, the message producer will receive a return message indicating that the batch of messages was successfully written. If the message producer does not receive the return message within the preset period of time when the message is written to the main message queue, the batch of messages can be rewritten into the main message queue.
在本申请实施例的一实施方式中,本申请实施例中的Redis可以开启仅追加文件(Append Only File,AOF)持久化,使得Redis的消息主队列和消息备份队列中的数据可以写入磁盘中,保证了消息的完整性。In an implementation of the embodiment of the present application, Redis in the embodiment of the present application can enable Append Only File (AOF) persistence, so that the data in the message main queue and message backup queue of Redis can be written to disk In, the integrity of the message is guaranteed.
步骤202,当监听到消息主队列中包括待消费消息时,获取待消费消息,并将待消费消息转移到消息备份队列中。Step 202: When it is monitored that the message to be consumed is included in the main message queue, the message to be consumed is obtained, and the message to be consumed is transferred to the message backup queue.
一种实施方式中,消息消费者可以开启多个线程,在通过BRPOPLPUSH命令从消息主队列中获取待消费消息的同时,将待消费消息从消息主队列中删除,并将待消费消息插入消息备份队列中。In one embodiment, the message consumer can start multiple threads, while obtaining the message to be consumed from the main message queue through the BRPOPLPUSH command, delete the message to be consumed from the main message queue, and insert the message to be consumed into the message backup In the queue.
其中,将待消费消息从消息主队列中删除,并插入消息备份队列这一操作具有原子性。原子性(Atomicity)是指事务的不可分割性,一个事务的所有操作不间断地全部被执行,或者一个操作也没有被执行。应用在本申请实施例中,上述原子性是指将待消费消息从的消息主队列中删除,并插入的消息备份队列中的操作过程不可发生中断,不会出现只将待消费消息从消息主队列中删除,或者只将待消费消息插入消息备份队列的情况。Among them, the operation of deleting the message to be consumed from the main message queue and inserting it into the message backup queue is atomic. Atomicity (Atomicity) refers to the indivisibility of a transaction, all operations of a transaction are executed without interruption, or an operation is not executed. Applied in the embodiments of this application, the above-mentioned atomicity means that the operation process of deleting messages to be consumed from the main message queue and inserting them into the message backup queue cannot be interrupted, and there will be no interruption of only messages to be consumed from the main message queue. Delete from the column, or just insert the message to be consumed into the message backup queue.
若将待消费消息从消息主队列中删除,且未将待消费消息插入备份消息 队列,此时Redis中未存储该待消费消息。在消息消费者对待消费消息处理失败时,该待消费消息无法被再次获取,使得该待消费消息没有被成功处理。If the message to be consumed is deleted from the main message queue and the message to be consumed is not inserted into the backup message queue, the message to be consumed is not stored in Redis at this time. When the message consumer fails to process the message to be consumed, the message to be consumed cannot be obtained again, so that the message to be consumed is not successfully processed.
而由于在本申请实施例中,将待消费消息转移到消息备份队列的操作具有原子性,减少了上述情况的发生,减少了消息处理过程中对待消费消息的遗漏。However, in the embodiment of the present application, the operation of transferring the message to be consumed to the message backup queue is atomic, which reduces the occurrence of the above situation and reduces the omission of the message to be consumed in the message processing process.
步骤203,处理待消费消息。Step 203: Process the message to be consumed.
本申请实施例提供的技术方案至少带来以下有益效果:由于消息生产者可以将待消费消息写入在Redis中的消息主队列,消费者可以从Redis中获取待消费消息,并将待消费消息转移到Redis的消息备份队列中,再处理待消费消息。由于待消费消息被获取的同时被转移到消息备份队列,使得即使消费者处理待消费消息时没有处理成功,仍然可以从消息备份队列中获取该待消费消息并再次处理,因此处理消息生产者生成的各待消费消息时不会处理遗漏。而且由于Redis是一种内存数据库,相比于RocketMQ以及Kafka等包括多个组件的消息系统,Redis的结构简单,部署运维方便,简化了处理消息的方法。The technical solution provided by the embodiments of the present application brings at least the following beneficial effects: since the message producer can write the message to be consumed into the message main queue in Redis, the consumer can obtain the message to be consumed from Redis and transfer the message to be consumed Transfer to the message backup queue of Redis, and then process the messages to be consumed. Since the message to be consumed is transferred to the message backup queue at the same time as it is obtained, even if the consumer does not process the message to be consumed successfully, the message to be consumed can still be obtained from the message backup queue and processed again, so the message producer generates The omissions will not be dealt with when each message is to be consumed. And because Redis is an in-memory database, compared to RocketMQ and Kafka and other messaging systems that include multiple components, Redis has a simple structure, convenient deployment, operation and maintenance, and simplifies the method of processing messages.
在本申请实施例中,Redis中的消息主队列和消息备份队列可以为一个或多个,消息消费者可以通过开启多个线程监听其中一个消息主队列,或者分别监听多个消息主队列,将每个消息主队列中的待消费消息转移到对应的消息备份队列中。在一种实施方式中,消息主队列的数量可以与消息备份队列数量相同,并一一对应。消息消费者同时监听多个消息主队列,当从一个消息主队列中监听到待消费消息时,将该待消费消息转移到该消息主队列对应的消息备份队列中。In the embodiment of this application, there may be one or more message main queues and message backup queues in Redis, and message consumers can monitor one of the message main queues by opening multiple threads, or monitor multiple message main queues separately, The messages to be consumed in each message main queue are transferred to the corresponding message backup queue. In an implementation manner, the number of message main queues may be the same as the number of message backup queues, and there is a one-to-one correspondence. The message consumer monitors multiple message main queues at the same time, and when a message to be consumed is monitored from one message main queue, the message to be consumed is transferred to the message backup queue corresponding to the message main queue.
消息主队列和消息备份队列存储消息的形式可以为:键(key)-值(value)的形式,其中key为消息队列的标识,value为消息的标识。一个key可以对应多个value。The message main queue and message backup queue may store messages in the form of key-value, where key is the identifier of the message queue, and value is the identifier of the message. One key can correspond to multiple values.
在一种实施方式中,消息主队列以及消息备份队列可以被多个消息消费者监听。In one embodiment, the main message queue and the message backup queue may be monitored by multiple message consumers.
在本申请另一实施方式中,在上述步骤203之后,若待消费消息处理成功,则消息消费者可以删除消息备份队列中的待消费消息。In another embodiment of the present application, after the above step 203, if the message to be consumed is processed successfully, the message consumer can delete the message to be consumed in the message backup queue.
一种实施方式中,可以通过删除命令(LREM)删除消息备份队列中的待消费消息。LREM命令是一种Redis命令,用于移除列表中的元素,应用在本申请实施例中,该LREM命令用于删除消息备份队列中的备份消息。In an implementation manner, the message to be consumed in the message backup queue can be deleted through a delete command (LREM). The LREM command is a Redis command used to remove elements in the list. In the embodiment of the present application, the LREM command is used to delete backup messages in the message backup queue.
若待消费消息处理失败,则消息消费者保留消息备份队列中的待消费消息。If the processing of the message to be consumed fails, the message consumer keeps the message to be consumed in the message backup queue.
本申请实施例的技术方案还可以带来以下有益效果:由于在待消费消息处理成功时,消息消费者才会删除消息备份队列中的待消费消息。若待消费消息处理失败,该待消费消息还可以被再次获取并处理,直至待消费消息处理成功,所以本申请实施例减少了消息处理过程中对待消费消息的遗漏。The technical solution of the embodiment of the present application can also bring the following beneficial effects: the message consumer will delete the message to be consumed in the message backup queue only when the message to be consumed is successfully processed. If the processing of the message to be consumed fails, the message to be consumed can also be acquired and processed again until the processing of the message to be consumed is successful, so the embodiment of the present application reduces the omission of the message to be consumed in the message processing process.
另外,由于消息消费者从消息主队列中获取待消费的同时,从消息主队列中删除了待消费消息,避免其他消息消费者从消息主队列中重复获取并处理待消费消息的情况,减少了同一个待消费消息被重复处理的情况。In addition, since the message consumer obtains the message to be consumed from the main message queue at the same time, deletes the message to be consumed from the message main queue, avoiding the situation that other message consumers repeatedly obtain and process the message to be consumed from the main message queue, and reduce The situation where the same message to be consumed is processed repeatedly.
在本申请另一实施例中,消息消费者在从消息主队列中获取待消费消息,并将待消费消息作为备份消息转移到消息备份队列后,若对待消费消息处理失败,则待消费消息的备份消息保留在消息备份队列中,所以消息备份队列中可能包括处理失败的待消费消息的备份消息。参见图3,为了避免对待消费消息处理时的遗漏,消息消费者还可以处理消息备份队列中的备份消息,包括如下步骤:In another embodiment of the present application, after the message consumer obtains the message to be consumed from the main message queue and transfers the message to be consumed to the message backup queue as a backup message, if the processing of the message to be consumed fails, the message to be consumed The backup messages are kept in the message backup queue, so the message backup queue may include backup messages of messages to be consumed that have failed to be processed. Referring to Figure 3, in order to avoid omissions in processing messages to be consumed, message consumers can also process backup messages in the message backup queue, including the following steps:
步骤301,每间隔预设时长,确定消息备份队列中是否包括备份消息。Step 301: Determine whether a backup message is included in the message backup queue every preset time duration.
一种实施方式中,监听消息主队列的BRPOPLPUSH命令中可以设置监听阻塞的预设时长,消息消费者每间隔预设时长,通过BRPOPLPUSH命令监听消息备份队列中是否包括备份消息。In an implementation manner, the preset duration of monitoring blocking can be set in the BRPOPLPUSH command of the monitoring message main queue, and the message consumer can monitor whether the backup message is included in the message backup queue by using the BRPOPLPUSH command to monitor whether the backup message is included in the message backup queue at each interval preset duration.
例如,预设时长可以是预设范围内的一个随机值,预设范围可以是5-10分钟。For example, the preset duration may be a random value within a preset range, and the preset range may be 5-10 minutes.
步骤302,若消息备份队列中包括备份消息,则获取备份消息。Step 302: If a backup message is included in the message backup queue, obtain the backup message.
一种实施方式中,消息消费者可以通过LINDEX命令,按照各备份消息进入消息备份队列的先后顺序,获取消息备份队列所包括的备份消息中,最先 进入消息备份队列的备份消息。其中,LINDEX命令为一种Redis命令,用于获取列表中的元素,应用在本申请实施例中,该LINDEX命令用于获取最先进入备份队列中的备份消息。In an implementation manner, the message consumer can use the LINDEX command to obtain the backup message that enters the message backup queue first among the backup messages included in the message backup queue according to the order in which each backup message enters the message backup queue. Wherein, the LINDEX command is a Redis command used to obtain elements in the list. In the embodiment of the present application, the LINDEX command is used to obtain the backup message that first enters the backup queue.
步骤303,处理备份消息。Step 303: Process the backup message.
其中,消息消费者可以对获取到的备份消息进行处理。Among them, the message consumer can process the obtained backup message.
步骤304,若备份消息处理成功,则从消息备份队列中删除备份消息。Step 304: If the backup message is processed successfully, delete the backup message from the message backup queue.
一种实施方式中,消息消费者可以通过LREM命令删除消息备份队列中的备份消息。In an implementation manner, the message consumer can delete the backup message in the message backup queue through the LREM command.
在本步骤中,若备份消息被消息消费者处理成功,则消息消费者可以从消息备份队列中删除该备份消息。若该备份消息未被处理成功,则消息消费者则不从备份队列中删除该备份消息。In this step, if the backup message is successfully processed by the message consumer, the message consumer can delete the backup message from the message backup queue. If the backup message is not processed successfully, the message consumer does not delete the backup message from the backup queue.
本申请实施例提供的技术方案还可以带来以下有益效果:由于消息消费者可以每间隔预设时长,从消息备份队列中获取并处理备份消息,在备份消息处理成功时,还可以删除消息备份队列中的备份消息,减少了消息处理过程中对消息的遗漏。The technical solution provided by the embodiments of the present application can also bring the following beneficial effects: because the message consumer can obtain and process the backup message from the message backup queue every interval preset time, and the message backup can be deleted when the backup message is processed successfully. The backup messages in the queue reduce the omission of messages during message processing.
在一种实施方式中,本申请实施例中的消息备份队列中还可以包括各备份消息的时间戳。其中,时间戳为备份消息进入消息备份队列的时刻。在上述步骤303,处理备份消息之前,消息消费者还可以判断是否需要处理获取的备份消息,包括如下步骤:In an implementation manner, the message backup queue in the embodiment of the present application may further include the timestamp of each backup message. Among them, the time stamp is the time when the backup message enters the message backup queue. In the above step 303, before processing the backup message, the message consumer can also determine whether the acquired backup message needs to be processed, including the following steps:
步骤一、获取备份消息的时间戳。Step 1: Obtain the timestamp of the backup message.
步骤二、若当前时刻与备份消息的时间戳之间的时间差大于预设时间差,则执行处理所述备份消息的步骤。Step 2: If the time difference between the current time and the time stamp of the backup message is greater than the preset time difference, execute the step of processing the backup message.
可以理解的,如果当前时刻与备份消息的时间戳之间的时间差大于预设时间差,说明该备份消息进入消息备份队列的时间较长且未被删除,该备份消息被处理失败的可能性较大,需要被再次处理。It is understandable that if the time difference between the current time and the timestamp of the backup message is greater than the preset time difference, it means that the backup message has been in the message backup queue for a long time and has not been deleted, and the processing of the backup message is more likely to fail. , Needs to be processed again.
在另一种实施方式中,若当前时刻与备份消息的时间戳之间的时间差不 大于预设时间差,则不处理获取的备份消息。In another embodiment, if the time difference between the current time and the time stamp of the backup message is not greater than the preset time difference, the acquired backup message is not processed.
可以理解的,如果当前时刻与备份消息的时间戳之间的时间差不大于预设时间差,说明该备份消息进入消息备份队列的时间较短且未被删除,该备份消息可能正在被消息消费者处理,所以可以不处理该备份消息。Understandably, if the time difference between the current time and the timestamp of the backup message is not greater than the preset time difference, it means that the backup message has entered the message backup queue for a short time and has not been deleted, and the backup message may be being processed by the message consumer , So you don’t need to process the backup message.
例如,预设时间差为十分钟。For example, the preset time difference is ten minutes.
本申请实施例提供的技术方案还可以带来以下有益效果:由于消息消费者处理消息需要一定的时间,而且消息消费者在处理成功后,才会删除备份消息队列中的备份消息,本申请实施例可以处理进入消息备份队列时间较长的备份消息,减少了消息的重复处理次数。The technical solution provided by the embodiments of this application can also bring about the following beneficial effects: since it takes a certain time for the message consumer to process the message, and the message consumer will delete the backup message in the backup message queue after the processing is successful, the implementation of this application For example, it can process backup messages that take a long time to enter the message backup queue, reducing the number of repeated processing of messages.
对应于上述方法实施例,如图4所示,本申请实施例提供了一种消息处理装置,该装置包括:监听模块401、获取模块402和处理模块403;Corresponding to the foregoing method embodiment, as shown in FIG. 4, an embodiment of the present application provides a message processing device, which includes: a monitoring module 401, an acquisition module 402, and a processing module 403;
监听模块401,设置为监听远程字典服务Redis的消息主队列中是否包括待消费消息,待消费消息为消息生产者写入消息主队列的消息;The monitoring module 401 is configured to monitor whether messages to be consumed are included in the main message queue of the remote dictionary service Redis, and the messages to be consumed are messages written by the message producer to the main message queue;
获取模块402,设置为当监听模块701监听到消息主队列中包括待消费消息时,获取待消费消息,并将待消费消息转移到消息备份队列中;The obtaining module 402 is configured to obtain the message to be consumed when the monitoring module 701 detects that the message to be consumed is included in the main message queue, and transfer the message to be consumed to the message backup queue;
处理模块403,设置为处理获取模块402获取的待消费消息。The processing module 403 is configured to process the message to be consumed acquired by the acquiring module 402.
在本申请另一实施例中,该装置还可以包括:删除模块;In another embodiment of the present application, the device may further include: a deletion module;
删除模块,设置为在处理待消费消息之后,若待消费消息处理成功,则删除消息备份队列中的待消费消息。The delete module is set to delete the message to be consumed in the message backup queue if the message to be consumed is processed successfully after processing the message to be consumed.
在本申请另一实施例中,该装置还可以包括:确定模块和删除模块;In another embodiment of the present application, the device may further include: a determination module and a deletion module;
确定模块,设置为每间隔预设时长,确定消息备份队列中是否包括备份消息;The determining module is set to the preset duration of each interval to determine whether the backup message is included in the message backup queue;
获取模块402,还设置为若消息备份队列中包括备份消息,则获取备份消息;The obtaining module 402 is further configured to obtain the backup message if the backup message is included in the message backup queue;
处理模块403,还设置为处理备份消息;The processing module 403 is also set to process backup messages;
删除模块,设置为若备份消息处理成功,则从消息备份队列中删除备份消息。The delete module is set to delete the backup message from the message backup queue if the backup message is processed successfully.
在本申请另一实施例中,获取模块402,可以设置为:In another embodiment of the present application, the obtaining module 402 may be configured as:
按照各备份消息进入消息备份队列的先后顺序,获取消息备份队列所包括的备份消息中,最先进入消息备份队列的备份消息。According to the sequence in which each backup message enters the message backup queue, the backup message that enters the message backup queue first among the backup messages included in the message backup queue is obtained.
在本申请另一实施例中,消息备份队列中还包括各备份消息的时间戳,时间戳为备份消息进入消息备份队列的时刻;该装置还可以包括:调用模块;In another embodiment of the present application, the message backup queue further includes a timestamp of each backup message, and the timestamp is the time when the backup message enters the message backup queue; the device may further include: a calling module;
获取模块,还设置为在处理备份消息之前,获取备份消息的时间戳;The obtaining module is also set to obtain the timestamp of the backup message before processing the backup message;
调用模块,设置为在当前时刻与备份消息的时间戳之间的时间差大于预设时间差时,调用处理模块执行处理备份消息的步骤。The calling module is set to call the processing module to execute the step of processing the backup message when the time difference between the current time and the time stamp of the backup message is greater than the preset time difference.
本申请实施例还提供了一种电子设备,如图5所示,包括处理器501、通信接口502、存储器503和通信总线504,其中,处理器501,通信接口502,存储器503通过通信总线504完成相互间的通信,The embodiment of the present application also provides an electronic device. As shown in FIG. Complete the communication between each other,
存储器503,设置为存放计算机程序;The memory 503 is set to store computer programs;
处理器501,设置为执行存储器503上所存放的程序时,实现上述方法实施例中的方法步骤。The processor 501 is configured to implement the method steps in the foregoing method embodiment when it is configured to execute the program stored in the memory 503.
上述电子设备提到的通信总线可以是外设部件互连标准(Peripheral Component Interconnect,PCI)总线或扩展工业标准结构(Extended Industry Standard Architecture,EISA)总线等。该通信总线可以分为地址总线、数据总线、控制总线等。为便于表示,图中仅用一条粗线表示,但并不表示仅有一根总线或一种类型的总线。The communication bus mentioned in the above electronic device may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into address bus, data bus, control bus and so on. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
通信接口用于上述电子设备与其他设备之间的通信。The communication interface is used for communication between the above-mentioned electronic device and other devices.
存储器可以包括随机存取存储器(Random Access Memory,RAM),也可以包括非易失性存储器(Non-Volatile Memory,NVM),例如至少一个磁盘存储器。可选的,存储器还可以是至少一个位于远离前述处理器的存储装置。The memory may include random access memory (Random Access Memory, RAM), and may also include non-volatile memory (Non-Volatile Memory, NVM), such as at least one disk storage. Optionally, the memory may also be at least one storage device located far away from the foregoing processor.
上述的处理器可以是通用处理器,包括中央处理器(Central Processing  Unit,CPU)、网络处理器(Network Processor,NP)等;还可以是数字信号处理器(Digital Signal Processing,DSP)、专用集成电路(Application Specific Integrated Circuit,ASIC)、现场可编程门阵列(Field-Programmable Gate Array,FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (Network Processor, NP), etc.; it can also be a digital signal processor (Digital Signal Processing, DSP), a dedicated integrated Circuit (Application Specific Integrated Circuit, ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
在本申请提供的又一实施例中,还提供了一种计算机可读存储介质,该计算机可读存储介质内存储有计算机程序,所述计算机程序被处理器执行时实现上述任一消息处理方法的步骤。In yet another embodiment provided in this application, a computer-readable storage medium is also provided. The computer-readable storage medium stores a computer program, and the computer program implements any of the above message processing methods when executed by a processor. A step of.
在本申请提供的又一实施例中,还提供了一种包含指令的计算机程序产品,当其在计算机上运行时,使得计算机执行上述实施例中任一消息处理方法。In another embodiment provided by the present application, a computer program product containing instructions is also provided, which when running on a computer, causes the computer to execute any message processing method in the above-mentioned embodiments.
在本申请提供的又一实施例中,还提供了一种计算机程序,当其在计算机上运行时,使得计算机执行上述实施例中任一消息处理方法。In another embodiment provided in this application, a computer program is also provided, which when running on a computer, causes the computer to execute any message processing method in the above-mentioned embodiments.
在上述实施例中,可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时,可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时,全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中,或者从一个计算机可读存储介质向另一个计算机可读存储介质传输,例如,所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质,(例如,软盘、硬盘、磁带)、光介质(例如,DVD)、或者半导体介质(例如固态硬盘Solid State Disk(SSD))等。In the above-mentioned embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented by software, it can be implemented in the form of a computer program product in whole or in part. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website, computer, server, or data center. Transmission to another website, computer, server or data center via wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center integrated with one or more available media. The usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, and a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)).
需要说明的是,在本文中,诸如第一和第二等之类的关系术语仅仅用来 将一个实体或者操作与另一个实体或操作区分开来,而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且,术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含,从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素,而且还包括没有明确列出的其他要素,或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下,由语句“包括一个……”限定的要素,并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply one of these entities or operations. There is any such actual relationship or order between. Moreover, the terms "include", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes those that are not explicitly listed Other elements of, or also include elements inherent to this process, method, article or equipment. If there are no more restrictions, the element defined by the sentence "including a..." does not exclude the existence of other identical elements in the process, method, article, or equipment that includes the element.
本说明书中的各个实施例均采用相关的方式描述,各个实施例之间相同相似的部分互相参见即可,每个实施例重点说明的都是与其他实施例的不同之处。尤其,对于装置实施例而言,由于其基本相似于方法实施例,所以描述的比较简单,相关之处参见方法实施例的部分说明即可。The various embodiments in this specification are described in a related manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, as for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for related parts, please refer to the part of the description of the method embodiment.
以上所述仅为本申请的较佳实施例而已,并非用于限定本申请的保护范围。凡在本申请的精神和原则之内所作的任何修改、等同替换、改进等,均包含在本申请的保护范围内。The foregoing descriptions are only preferred embodiments of the present application, and are not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application are all included in the protection scope of this application.
工业实用性Industrial applicability
基于本申请实施例提供的消息处理方法、装置、电子设备及介质,消息可以将Redis中的消息主队列和消息备份队列作为消息中间件,可以简化对消息中间件的部署难度,简化处理消息的方法。Based on the message processing method, device, electronic device, and medium provided by the embodiments of the present application, the message can use the message main queue and message backup queue in Redis as message middleware, which can simplify the difficulty of deploying message middleware and simplify the process of message processing. method.

Claims (14)

  1. 一种消息处理方法,所述方法包括:A message processing method, the method includes:
    监听远程字典服务Redis的消息主队列中是否包括待消费消息,所述待消费消息为消息生产者写入所述消息主队列的消息;Monitoring whether the message main queue of the remote dictionary service Redis includes a message to be consumed, and the message to be consumed is a message written by a message producer to the message main queue;
    当监听到所述消息主队列中包括待消费消息时,获取所述待消费消息,并将所述待消费消息转移到消息备份队列中;When it is monitored that the message to be consumed is included in the main message queue, acquiring the message to be consumed, and transferring the message to be consumed to the message backup queue;
    处理所述待消费消息。Process the message to be consumed.
  2. 根据权利要求1所述的方法,其中,在所述处理所述待消费消息之后,所述方法还包括:The method according to claim 1, wherein, after the processing the message to be consumed, the method further comprises:
    若所述待消费消息处理成功,则删除所述消息备份队列中的所述待消费消息。If the processing of the message to be consumed is successful, the message to be consumed in the message backup queue is deleted.
  3. 根据权利要求1所述的方法,其中,所述方法还包括:The method according to claim 1, wherein the method further comprises:
    每间隔预设时长,确定所述消息备份队列中是否包括备份消息;Each interval preset time length, determine whether the message backup queue includes a backup message;
    若所述消息备份队列中包括备份消息,则获取所述备份消息;If the message backup queue includes a backup message, acquiring the backup message;
    处理所述备份消息;Processing the backup message;
    若所述备份消息处理成功,则从所述消息备份队列中删除所述备份消息。If the processing of the backup message is successful, the backup message is deleted from the message backup queue.
  4. 根据权利要求3所述的方法,其中,所述获取所述备份消息,包括:The method according to claim 3, wherein said obtaining said backup message comprises:
    按照各备份消息进入所述消息备份队列的先后顺序,获取所述消息备份队列所包括的备份消息中,最先进入所述消息备份队列的备份消息。According to the sequence in which each backup message enters the message backup queue, among the backup messages included in the message backup queue, the backup message that first enters the message backup queue is obtained.
  5. 根据权利要求3或4所述的方法,其中,所述消息备份队列中还包括各备份消息的时间戳,所述时间戳为备份消息进入所述消息备份队列的时刻;在所述处理所述备份消息之前,所述方法还包括:The method according to claim 3 or 4, wherein the message backup queue further includes a time stamp of each backup message, and the time stamp is the time when the backup message enters the message backup queue; Before backing up the message, the method further includes:
    获取所述备份消息的时间戳;Obtaining the timestamp of the backup message;
    若当前时刻与所述备份消息的时间戳之间的时间差大于预设时间差,则 执行所述处理所述备份消息的步骤。If the time difference between the current time and the time stamp of the backup message is greater than the preset time difference, the step of processing the backup message is executed.
  6. 一种消息处理装置,所述装置包括:A message processing device, the device includes:
    监听模块,设置为监听远程字典服务Redis的消息主队列中是否包括待消费消息,所述待消费消息为消息生产者写入所述消息主队列的消息;A monitoring module, configured to monitor whether messages to be consumed are included in the message main queue of the remote dictionary service Redis, and the messages to be consumed are messages written into the message main queue by a message producer;
    获取模块,设置为当所述监听模块监听到所述消息主队列中包括待消费消息时,获取所述待消费消息,并将所述待消费消息转移到消息备份队列中;An acquiring module, configured to acquire the message to be consumed when the monitoring module detects that the message to be consumed is included in the main message queue, and transfer the message to be consumed to the message backup queue;
    处理模块,设置为处理所述获取模块获取的所述待消费消息。The processing module is configured to process the message to be consumed acquired by the acquisition module.
  7. 根据权利要求6所述的装置,其中,所述装置还包括:删除模块;The device according to claim 6, wherein the device further comprises: a deletion module;
    所述删除模块,设置为在所述处理所述待消费消息之后,若所述待消费消息处理成功,则删除所述消息备份队列中的所述待消费消息。The deleting module is configured to delete the message to be consumed in the message backup queue if the message to be consumed is successfully processed after the processing of the message to be consumed.
  8. 根据权利要求6所述的装置,其中,所述装置还包括:确定模块和删除模块;The device according to claim 6, wherein the device further comprises: a determination module and a deletion module;
    所述确定模块,设置为每间隔预设时长,确定所述消息备份队列中是否包括备份消息;The determining module is set to a preset duration of each interval, and determines whether a backup message is included in the message backup queue;
    所述获取模块,还设置为若所述消息备份队列中包括备份消息,则获取所述备份消息;The acquiring module is further configured to acquire the backup message if the backup message is included in the message backup queue;
    所述处理模块,还设置为处理所述备份消息;The processing module is further configured to process the backup message;
    所述删除模块,设置为若所述备份消息处理成功,则从所述消息备份队列中删除所述备份消息。The deletion module is configured to delete the backup message from the message backup queue if the processing of the backup message is successful.
  9. 根据权利要求8所述的装置,其中,所述获取模块,设置为:The device according to claim 8, wherein the acquiring module is configured to:
    按照各备份消息进入所述消息备份队列的先后顺序,获取所述消息备份队列所包括的备份消息中,最先进入所述消息备份队列的备份消息。According to the sequence in which each backup message enters the message backup queue, among the backup messages included in the message backup queue, the backup message that first enters the message backup queue is obtained.
  10. 根据权利要求8或9所述的装置,其中,所述消息备份队列中还包括各备份消息的时间戳,所述时间戳为备份消息进入所述消息备份队列的时刻;所述装置还包括:调用模块;The device according to claim 8 or 9, wherein the message backup queue further includes a timestamp of each backup message, and the timestamp is the time when the backup message enters the message backup queue; the device further comprises: Call module
    所述获取模块,还设置为在所述处理所述备份消息之前,获取所述备份消息的时间戳;The acquiring module is further configured to acquire the time stamp of the backup message before the processing of the backup message;
    所述调用模块,设置为在当前时刻与所述备份消息的时间戳之间的时间差大于预设时间差时,调用所述处理模块执行所述处理所述备份消息的步骤。The calling module is configured to call the processing module to execute the step of processing the backup message when the time difference between the current moment and the time stamp of the backup message is greater than a preset time difference.
  11. 一种电子设备,包括处理器、通信接口、存储器和通信总线,其中,处理器,通信接口,存储器通过通信总线完成相互间的通信;An electronic device, including a processor, a communication interface, a memory, and a communication bus. The processor, the communication interface, and the memory communicate with each other through the communication bus;
    存储器,设置为存放计算机程序;Memory, set to store computer programs;
    处理器,设置为执行存储器上所存放的程序时,实现权利要求1-5任一所述的方法步骤。The processor is configured to implement the method steps described in any one of claims 1-5 when it is configured to execute the program stored in the memory.
  12. 一种计算机可读存储介质,所述计算机可读存储介质内存储有计算机程序,所述计算机程序被处理器执行时实现权利要求1-5任一所述的方法步骤。A computer-readable storage medium in which a computer program is stored, and when the computer program is executed by a processor, the method steps according to any one of claims 1 to 5 are realized.
  13. 一种包含指令的计算机程序产品,所述包含指令的计算机程序产品在计算机上运行时,使得计算机执行权利要求1-5任一所述的方法步骤。A computer program product containing instructions, which when running on a computer, causes the computer to execute the method steps of any one of claims 1-5.
  14. 一种计算机程序,所述计算机程序在计算机上运行时,使得计算机执行权利要求1-5任一所述的方法步骤。A computer program that, when run on a computer, causes the computer to execute the method steps described in any one of claims 1-5.
PCT/CN2021/079512 2020-03-13 2021-03-08 Message processing method and apparatus, electronic device and medium WO2021180025A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010174823.1 2020-03-13
CN202010174823.1A CN111381987A (en) 2020-03-13 2020-03-13 Message processing method and device, electronic equipment and medium

Publications (1)

Publication Number Publication Date
WO2021180025A1 true WO2021180025A1 (en) 2021-09-16

Family

ID=71217266

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/079512 WO2021180025A1 (en) 2020-03-13 2021-03-08 Message processing method and apparatus, electronic device and medium

Country Status (2)

Country Link
CN (1) CN111381987A (en)
WO (1) WO2021180025A1 (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113961379A (en) * 2021-12-22 2022-01-21 零犀(北京)科技有限公司 Message processing method and device based on man-machine conversation
CN114124846A (en) * 2021-11-15 2022-03-01 聚好看科技股份有限公司 Service queue consumption method and server
CN114490091A (en) * 2022-04-06 2022-05-13 江苏金恒信息科技股份有限公司 Method and device for monitoring rule engine performance in industrial data acquisition management system
CN114489480A (en) * 2021-12-23 2022-05-13 深圳市世强元件网络有限公司 Method and system for high-concurrency data storage
CN115048418A (en) * 2022-08-12 2022-09-13 深圳市必凡娱乐科技有限公司 Data processing method and system
CN115134262A (en) * 2022-08-31 2022-09-30 飞狐信息技术(天津)有限公司 RocktMQ monitoring method and device, storage medium and electronic equipment

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111381987A (en) * 2020-03-13 2020-07-07 北京金山云网络技术有限公司 Message processing method and device, electronic equipment and medium
CN112527527A (en) * 2020-12-16 2021-03-19 深圳市分期乐网络科技有限公司 Consumption speed control method and device of message queue, electronic equipment and medium
CN113014618B (en) * 2021-01-12 2022-04-29 腾讯科技(深圳)有限公司 Message processing method and system and electronic equipment
CN112965839B (en) * 2021-03-29 2024-01-05 远景智能国际私人投资有限公司 Message transmission method, device, equipment and storage medium
CN113742107A (en) * 2021-09-03 2021-12-03 广州新丝路信息科技有限公司 Processing method for avoiding message loss in message queue and related equipment
CN115001998B (en) * 2022-04-26 2024-02-23 北京贝壳时代网络科技有限公司 Disaster recovery method and device for message service
CN116821245A (en) * 2023-07-05 2023-09-29 贝壳找房(北京)科技有限公司 Data aggregation synchronization method and storage medium in distributed scene

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106528306A (en) * 2016-11-08 2017-03-22 天津海量信息技术股份有限公司 Message queue storage method
CN107197017A (en) * 2017-05-23 2017-09-22 努比亚技术有限公司 A kind of consuming method, terminal and computer-readable recording medium based on consumption queue
US20180375783A1 (en) * 2017-06-27 2018-12-27 Atlassian Pty Ltd Retry handling in messaging queues
CN111381987A (en) * 2020-03-13 2020-07-07 北京金山云网络技术有限公司 Message processing method and device, electronic equipment and medium

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107092533A (en) * 2017-03-29 2017-08-25 弘成科技发展有限公司 Synchronization message queue based on ActiveMQ+Redis
CN108965355B (en) * 2017-05-18 2021-05-25 北京京东尚科信息技术有限公司 Method, apparatus and computer readable storage medium for data transmission
CN109561151B (en) * 2018-12-12 2021-09-17 北京达佳互联信息技术有限公司 Data storage method, device, server and storage medium

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106528306A (en) * 2016-11-08 2017-03-22 天津海量信息技术股份有限公司 Message queue storage method
CN107197017A (en) * 2017-05-23 2017-09-22 努比亚技术有限公司 A kind of consuming method, terminal and computer-readable recording medium based on consumption queue
US20180375783A1 (en) * 2017-06-27 2018-12-27 Atlassian Pty Ltd Retry handling in messaging queues
CN111381987A (en) * 2020-03-13 2020-07-07 北京金山云网络技术有限公司 Message processing method and device, electronic equipment and medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
ZENHOBBY: "Java Jedis operation Redis example (two)-list producer/consumer model to achieve message queue", BLOG CSDN, CSDN, CN, 1 January 2017 (2017-01-01), CN, pages 1 - 20, XP055846770, Retrieved from the Internet <URL:https://blog.csdn.net/KingCat666/article/details/77887613> [retrieved on 20211001] *

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114124846A (en) * 2021-11-15 2022-03-01 聚好看科技股份有限公司 Service queue consumption method and server
CN114124846B (en) * 2021-11-15 2023-08-11 聚好看科技股份有限公司 Service queue consumption method and server
CN113961379A (en) * 2021-12-22 2022-01-21 零犀(北京)科技有限公司 Message processing method and device based on man-machine conversation
CN114489480A (en) * 2021-12-23 2022-05-13 深圳市世强元件网络有限公司 Method and system for high-concurrency data storage
CN114490091A (en) * 2022-04-06 2022-05-13 江苏金恒信息科技股份有限公司 Method and device for monitoring rule engine performance in industrial data acquisition management system
CN115048418A (en) * 2022-08-12 2022-09-13 深圳市必凡娱乐科技有限公司 Data processing method and system
CN115134262A (en) * 2022-08-31 2022-09-30 飞狐信息技术(天津)有限公司 RocktMQ monitoring method and device, storage medium and electronic equipment
CN115134262B (en) * 2022-08-31 2023-01-06 飞狐信息技术(天津)有限公司 RocktMQ monitoring method and device, storage medium and electronic equipment

Also Published As

Publication number Publication date
CN111381987A (en) 2020-07-07

Similar Documents

Publication Publication Date Title
WO2021180025A1 (en) Message processing method and apparatus, electronic device and medium
US10831776B2 (en) On-demand file synchronization
CN107832100B (en) APK plug-in loading method and terminal thereof
US20180253458A1 (en) Synergizing real-time and polling connectors for data ingestion
CN112650576B (en) Resource scheduling method, device, equipment, storage medium and computer program product
CN110865888A (en) Resource loading method and device, server and storage medium
CN110008041B (en) Message processing method and device
CN112527879B (en) Kafka-based real-time data extraction method and related equipment
US20230030856A1 (en) Distributed table storage processing method, device and system
CN107341062B (en) Data pushing method, device, equipment and storage medium
CN112115200B (en) Data synchronization method, device, electronic equipment and readable storage medium
CN111382206B (en) Data storage method and device
CN111385255B (en) Asynchronous call implementation method and device, server and server cluster
CN110881224B (en) Network long connection method, device, equipment and storage medium
WO2022134638A1 (en) Logic clock synchronization method and apparatus, and central time service cluster
CN112653736B (en) Parallel source returning method and device and electronic equipment
CN108108126B (en) Data processing method, device and equipment
CN112865927B (en) Message delivery verification method, device, computer equipment and storage medium
CN111078418B (en) Operation synchronization method, device, electronic equipment and computer readable storage medium
CN116701020A (en) Message delay processing method, device, equipment, medium and program product
CN109445966B (en) Event processing method, device, medium and computing equipment
CN116627333A (en) Log caching method and device, electronic equipment and computer readable storage medium
CN111309693A (en) Data synchronization method, device and system, electronic equipment and storage medium
CN111159298A (en) Service request processing method and device, electronic equipment and storage medium
CN112445860B (en) Method and device for processing distributed transaction

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21767307

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21767307

Country of ref document: EP

Kind code of ref document: A1