CN109739659B - Message consumption method and system of distributed message queue - Google Patents
Message consumption method and system of distributed message queue Download PDFInfo
- Publication number
- CN109739659B CN109739659B CN201811527275.5A CN201811527275A CN109739659B CN 109739659 B CN109739659 B CN 109739659B CN 201811527275 A CN201811527275 A CN 201811527275A CN 109739659 B CN109739659 B CN 109739659B
- Authority
- CN
- China
- Prior art keywords
- consumption
- message
- period
- current
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Debugging And Monitoring (AREA)
Abstract
The invention provides a method and a system for message consumption of a distributed message queue, wherein the method comprises the following steps: the consumption control thread or the consumption conventional thread sends message consumption state data of a subarea to the consumption progress coordination device; when the consumption progress coordinating device judges that all the messages of each subarea in the current period are consumed and finished according to the message consumption state data, the consumption control thread switches the current period of the synchronous inspection data to the next period; each consumption control thread and the consumption conventional thread acquire the synchronous check data from the consumption progress coordinating device; the method and the system for consuming the messages of the distributed message queue provided by the invention are used for consuming the messages in the period of the synchronous check data by the consumption control threads and the consumption conventional threads, based on specific service requirements, some scenes need to be counted strictly according to a time period, particularly when the data is recalculated, the statistics cannot be carried out in a cross-period mode, otherwise, the statistical error is caused, and the method and the system for consuming the messages of the distributed message queue are also very suitable for the scenes.
Description
Technical Field
The present invention relates to the field of information technologies, and in particular, to a method and a system for message consumption of a distributed message queue.
Background
In real-time calculation, due to the design principle of a distributed message queue, messages in a single partition are sequential, the partitions are independent, the number of the messages in each partition is inconsistent with the consumption speed of each consumption thread, and message time sequences from log reading to log calculation are inconsistent. For example, the message time of partition 1 is consumed to 6:00, while the message time of partition 2 is more, the accumulation is more serious, and may be consumed to 5:50, and the time difference between partitions is 10 minutes or more. In a computing scenario where the message time order is insensitive, the impact of log time differences can be ignored. However, some service requirements are strict on time sequence, and it is difficult to accept that messages are out of order, which results in difference of statistical results. For example, a user may be charged at 23:55, may be processed after a 00:05 message due to pile-up delay, and may be counted for different periods.
watermark is now the most common solution to the problem of dealing with distributed message queue consumption timing, which is implemented in many real-time processing frameworks. watermark, when used, can be understood as a delayed time, i.e. how long a task is delayed to trigger. If we want to compute once per minute with watermark set to (current time-10) seconds, the last time period of task computation would be triggered at 10 seconds per minute. Therefore, normally, a watermark is not set too large, otherwise the statistical delay is large. The watermark scheme is used for solving the problem of log disorder by setting a delay time which can be tolerated, and the solution can cause statistical errors in scenes requiring strict time sequence. Especially if the real-time statistics program exits abnormally and needs to recover data from a certain period of time, the inconsistency of consumption speed can cause that the problem cannot be solved even if the watermark method is used, because the size of the tolerated out-of-order time window cannot be enlarged. There is also a scenario that if the real-time processing is delayed due to data accumulation, after the watermark is set, the accumulated log is ignored, and the log is not correctly calculated.
The idea of reordering is to order the consumed data once before calculating, and this solution can only solve the ordering of the data in the current execution batch, and cannot solve the problem of inconsistent consumption speed fundamentally.
Disclosure of Invention
The invention aims to provide a method and a system for message consumption of distributed message queues, which aim to solve the problem of coordinating consumption speed when each distributed message queue is consumed.
The invention provides a message consumption method of a distributed message queue, which comprises the following steps:
the consumption control thread or the consumption conventional thread sends message consumption state data of a subarea to the consumption progress coordination device;
when the consumption progress coordinating device judges that all the messages of each subarea in the current period are consumed and finished according to the message consumption state data, the consumption control thread switches the current period of the synchronous inspection data to the next period;
each consumption control thread and the consumption conventional thread acquire the synchronous check data from the consumption progress coordinating device;
each of the consumption control thread and the consumption regular thread consumes a message in a cycle of the synchronization check data.
Further, the method of the present invention further comprises:
setting a time checkpoint in the synchronous inspection data by the consumption control thread;
and when the time reaches the time check point, the consumption conventional thread and the consumption control thread judge whether all the messages of each partition in the current period are consumed completely according to the message consumption state data.
Further, the method of the present invention further comprises:
when the time reaches the time checkpoint, the consumption control thread sends the message consumption state data to the consumption progress coordinating device and acquires the synchronization check data from the consumption progress coordinating device.
Further, in the method of the present invention, the data structure of the message consumption state data includes:
consumption group, topic, total partition number, current partition, current consumption location, and current log timestamp.
Further, in the method of the present invention, the data structure of the synchronization check data includes:
consumption groups, total partition number, cycle and time checkpoints.
The invention provides a message consumption system of a distributed message queue, which comprises: a business processing device and a consumption progress coordinating device;
the service processing device comprises: a consumption control thread and at least one consumption regular thread; wherein each consumption control thread or consumption conventional thread corresponds to one partition; wherein each topic comprises at least two of the partitions;
wherein the service processing device is configured to:
causing each of said consumption control threads or said consumption regular threads to send message consumption status data of one of said partitions to said consumption progress coordinating means;
each consumption control thread and the consumption conventional thread acquire the synchronous check data from the consumption progress coordinating device;
each consumption control thread and the consumption conventional thread consume the messages in the period of the synchronous check data;
the consumption progress coordinating apparatus includes: a consumption control thread;
wherein the consumption progress coordinating means is for:
and when the consumption control thread judges that all the messages of each partition in the current period are consumed completely according to the message consumption state data, the consumption control thread switches the current period of the synchronous inspection data into the next period.
Further, in the system of the present invention, the consumption progress coordinating device is configured to:
setting a time checkpoint in the synchronous inspection data by the consumption control thread;
and when the data consumed by the consumption control thread reaches the time check point, the consumption control thread judges whether all the messages of each partition in the current period are consumed completely according to the message consumption state data.
Further, in the system of the present invention, the service processing apparatus is further configured to:
when the time reaches the time checkpoint, the consumption control thread sends the message consumption state data to the consumption progress coordinating device and acquires the synchronization check data from the consumption progress coordinating device.
Further, in the system of the present invention, the data structure of the message consumption state data includes:
consumption group, topic, total partition number, current partition, current consumption location, and current log timestamp.
Further, in the system of the present invention, the data structure of the synchronization check data includes:
consumption groups, total partition number, cycle and time checkpoints.
The present invention also provides a storage device storing computer program instructions for performing the method according to the present invention.
The method and the system for message consumption of the distributed message queue can coordinate the consumption speed of each partition by a mechanism of checking in the process and among the processes, achieve the aim of mutual coordination, and control the message consumption speed of different partitions during consumption. In addition, based on specific service requirements, some scenes need to be counted strictly according to a time period, particularly when data is recalculated, the statistics cannot be carried out across the periods, otherwise, statistical errors are caused, and the method and the system for message consumption of the distributed message queue are also very suitable for the scenes.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments made with reference to the following drawings:
fig. 1 is a flowchart illustrating a method for message consumption of a distributed message queue according to an embodiment of the present invention;
FIG. 2 is a schematic diagram illustrating a method for message consumption of a distributed message queue according to an embodiment of the present invention;
fig. 3 is a flowchart illustrating a message consumption method of a distributed message queue according to a second embodiment of the present invention;
fig. 4 is a schematic structural diagram of a system for message consumption of a distributed message queue according to a third embodiment of the present invention.
The same or similar reference numbers in the drawings identify the same or similar elements.
Detailed Description
The present invention is described in further detail below with reference to the attached drawing figures.
In a typical configuration of the invention, the terminals, the devices serving the network each comprise one or more processors (CPU), input/output interfaces, network interfaces and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, which include both non-transitory and non-transitory, removable and non-removable media, may implement the information storage by any method or technology. The information may be computer readable instructions, data structures, program means, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.
Fig. 1 is a flowchart illustrating a method for message consumption of a distributed message queue according to an embodiment of the present invention, where as shown in fig. 1, the method for message consumption of a distributed message queue according to an embodiment of the present invention includes:
step S101, the consumption control thread or the consumption conventional thread sends message consumption state data of a partition to the consumption progress coordination device.
Step S102, when the consumption progress coordinating device judges that all the messages of each subarea in the current period are consumed and completed according to the message consumption state data, the consumption control thread switches the current period of the synchronous inspection data to the next period.
Step S103, each consumption control thread and each consumption regular thread acquire the synchronization check data from the consumption progress coordinating device.
And step S104, each consumption control thread and each consumption conventional thread consume the messages in the period of the synchronous check data.
Wherein, the consumption conventional thread and the consumption control thread have the following relationship: the consuming threads are consuming conventional threads, and a consuming thread with a control coordination function is elected or appointed as a consuming control thread. The code consuming the regular thread and the consumption control thread are identical, only have different roles and execute different functions. The consumption control thread and the consumption conventional thread consume data, only the consumption control thread has a consumption function and a control coordination function, and the consumption conventional thread only has a consumption function.
Wherein the data structure of the message consumption state data comprises:
consumption group, topic, total partition number, current partition, current consumption location, and current log timestamp.
For example, the data structure of the message consumption state data is:
$group|$topic|$paritition_cnt|$partition|$offset|$timestamp
wherein the data structure of the synchronization check data comprises:
consumption groups, total partition number, cycle and time checkpoints.
For example, the data structure of the sync check data is:
$group|$partition_cnt|period|$timestamp
the topic (topic) is used for distinguishing different service classifications in the distributed message queue, and a plurality of partitions are arranged under each topic (topic).
Fig. 2 is a schematic diagram of a message consumption method of a distributed message queue according to an embodiment of the present invention, and as shown in fig. 2, the overall system is divided into three logic portions, namely, a distributed message queue, service processing, and consumption progress coordination, where each consumption thread corresponds to one partition (where the consumption thread refers to a consumption control thread and a consumption normal thread), for example, the consumption control thread corresponds to partition 1, and the consumption normal thread corresponds to other partitions. The partitions may be deployed in a distributed manner, forming a distributed message queue for each partition. And the consumption control thread arranges the message consumption state data of the corresponding subarea according to a standard data structure and sends the message consumption state data to the shared memory. The shared memory realizes a consumption progress coordination mechanism through a consumption progress coordination device, and a consumption control thread of the consumption progress coordination device checks whether all messages of a partition 1 to a partition n in the current period are consumed completely according to the current consumption position and the current log timestamp of each received message consumption state data and the period of synchronous check data, namely whether the current consumption position and the current log timestamp of the message consumption state data are in the period of the synchronous check data. When the message is completely consumed, the consumption control thread switches the current period of the synchronous inspection data to the next period, namely, the next period is started, otherwise, the inspection is continued. And the consumption conventional thread acquires the synchronous check data from the shared memory and judges whether the message to be consumed is in the period of the synchronous check data according to the current consumption position and the current log timestamp of the message consumption state data and the period of the synchronous check data. When the message to be consumed is within the period, the message is consumed. When the message to be consumed is not within the period, the consumption of the message is suspended.
Fig. 3 is a schematic flowchart of a method for message consumption of a distributed message queue according to a second embodiment of the present invention, and as shown in fig. 3, the method for message consumption of a distributed message queue according to the second embodiment of the present invention includes:
step S301, a time checkpoint is set on the synchronization check data by the consumption control thread.
Step S302, when the consumption data of the message conventional threads reaches the time check point, each consumption conventional thread sends the message consumption state data of one partition to the consumption progress coordinating device.
Step S303, when the consumption data of the message control thread reaches the time check point, the consumption control thread judges whether all the messages of each subarea in the current period are consumed or not according to the message consumption state data.
Step S304, when the consumption control thread of the consumption progress coordinating apparatus determines that all the messages of each partition in the current period are consumed according to the message consumption state data, the consumption control thread switches the current period of the synchronous inspection data to the next period.
Step S305, when the current cycle of the consumption check data is switched to the next cycle, the consumption control threads and the consumption regular threads respectively obtain the synchronous check data from the consumption progress coordinating device.
Step S306, each of the consumption control thread and the consumption regular thread consumes the message in the cycle of the synchronization check data.
Wherein the data structure of the message consumption state data comprises:
consumption group, topic, total partition number, current partition, current consumption location, and current log timestamp.
Wherein the data structure of the synchronization check data comprises:
consumption groups, total partition number, cycle and time checkpoints.
Wherein, the time check point can be set by the consumption control thread in the synchronous check data, so as to satisfy more flexible check mechanism. In order to avoid occupying excessive computing resources, the consumption control thread judges whether all the messages of each partition in the current period are completely consumed only when the time check point is reached, and the consumption conventional thread sends message consumption state data and acquires synchronous check data. During the switching period, it is necessary to ensure that the data of the previous period enters into the calculation. When the consumption log reaches the checkpoint, it is possible that the consumption control thread and the consumption regular thread may wait and check whether the message can be continuously consumed. In order to improve the efficiency, the waiting time of each check is not more than a preset waiting time threshold value, for example, 1 second, so as to ensure the real-time performance.
Specifically, the consumption regular thread sends the message consumption state data of the partition corresponding to the consumption regular thread to the consumption progress coordination device of the shared memory. And the consumption control thread of the consumption progress coordination device checks whether all the messages of each subarea in the current period are consumed completely according to the current consumption position and the current log timestamp of each received message consumption state data and the period of the synchronous check data, namely whether the current consumption position and the current log timestamp of the message consumption state data are in the period of the synchronous check data. When the message is completely consumed, the consumption control thread switches the current period of the synchronous inspection data to the next period, namely, the next period is started, otherwise, the inspection is continued. And the consumption conventional thread acquires the synchronous check data from the shared memory and judges whether the message to be consumed is in the period of the synchronous check data according to the current consumption position and the current log timestamp of the message consumption state data and the period of the synchronous check data. When the message to be consumed is within the period, the message is consumed. When the message to be consumed is not within the period, the consumption of the message is suspended.
Wherein, when the abnormal condition is detected, the consumption progress coordinating device needs to send out an alarm in time. In handling coordination, attention is paid to the mechanism for capturing exceptions and increasing retries to ensure system stability.
Fig. 4 is a schematic structural diagram of a system for message consumption of a distributed message queue according to a third embodiment of the present invention, and as shown in fig. 4, the system for message consumption of a distributed message queue according to the present invention includes: a business processing device 41 and a consumption progress coordinating device 42.
The service processing device 41 includes: at least one consumption routine thread and one consumption control thread; wherein each consumption regular thread or consumption control thread corresponds to one partition; wherein each body comprises at least two of said partitions.
Wherein, the service processing device 41 is configured to:
causing each of said consuming regular threads or said consuming control threads to send message consumption status data of one of said partitions to said consuming progress coordinating means;
each consumption conventional thread and the consumption control thread acquire the synchronous check data from the consumption progress coordinating device;
each of the consumption regular thread and the consumption control thread consumes a message in a cycle of the synchronization check data.
Wherein the data structure of the message consumption state data comprises: consumption group, topic, total partition number, current partition, current consumption location, and current log timestamp.
Wherein the data structure of the synchronization check data comprises: consumption groups, total partition number, cycle and time checkpoints.
The service processing device 41 may further be configured to:
when the time reaches the time checkpoint, the consumption routine thread sends the message consumption status data to the consumption progress coordination device and acquires the synchronization check data from the consumption progress coordination device.
The consumption progress coordinating means 42 includes: and consuming the control thread.
Wherein, the consumption progress coordinating device 42 is used for:
and when the consumption control thread judges that all the messages of each partition in the current period are consumed completely according to the message consumption state data, the consumption control thread switches the current period of the synchronous inspection data into the next period.
The consumption progress coordinating means 42 may be further configured to:
setting a time checkpoint in the synchronous inspection data by the consumption control thread;
and when the time reaches the time check point, the consumption control thread judges whether all the messages of each partition in the current period are consumed or not according to the message consumption state data.
For the system in the third embodiment of the present invention and the implementation apparatus of the method in the second embodiment of the present invention, reference may be made to the second embodiment of the present invention, and details are not described herein again.
The invention also provides a storage device storing computer program instructions for execution according to the method of the first or second embodiment of the invention.
It should be noted that the present invention may be implemented in software and/or in a combination of software and hardware, for example, as an Application Specific Integrated Circuit (ASIC), a general purpose computer or any other similar hardware device. In some embodiments, the software program of the present invention may be executed by a processor to implement the above steps or functions. Also, the software programs (including associated data structures) of the present invention can be stored in a computer readable recording medium, such as RAM memory, magnetic or optical drive or diskette and the like. Further, some of the steps or functions of the present invention may be implemented in hardware, for example, as circuitry that cooperates with the processor to perform various steps or functions.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned. Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the apparatus claims may also be implemented by one unit or means in software or hardware.
Claims (3)
1. A method of message consumption for a distributed message queue, the system for message consumption for a distributed message queue comprising: a business processing device and a consumption progress coordinating device;
the service processing device comprises: a consumption control thread and at least one consumption regular thread; wherein each consumption control thread or consumption conventional thread corresponds to one partition; each topic comprising at least two of said partitions;
the consumption progress coordinating apparatus includes: a consumption control thread;
the method comprises the following steps:
setting a time check point at synchronous check data through a consumption control thread of the consumption progress coordinating device; the data structure of the synchronization check data includes: consumption groups, total partition number, cycle and time checkpoints;
the consumption control thread or the consumption conventional thread of the business processing device sends the message consumption state data of one subarea to the consumption progress coordination device; the data structure of the message consumption state data includes: consumption groups, topics, total partition number, current partition, current consumption location, and current log timestamp;
when the time reaches the time check point, the consumption control thread of the consumption progress coordinating device judges whether the current consumption position and the current log timestamp of the message consumption state data of all the messages of each partition in the current period are in the period of synchronous check data according to the current consumption position and the current log timestamp of each message consumption state data and the period of the synchronous check data, and the consumption regular thread of the business processing device sends the message consumption state data to the consumption progress coordinating device and acquires the synchronous check data from the consumption progress coordinating device;
when the consumption progress coordination device judges that the current consumption positions and the current log timestamps of the consumption state data of all the messages of each partition in the current period are not in the period of synchronous inspection data according to the message consumption state data, a consumption control thread of the consumption progress coordination device switches the current period of the synchronous inspection data into the next period; otherwise, continuing checking;
the consumption control thread and the consumption conventional thread of each business processing device acquire the synchronous check data from the consumption progress coordinating device;
the consumption control thread and the consumption conventional thread of each business processing device consume the messages in the period of the synchronous check data;
when the message to be consumed is not in the period, suspending the consumption of the message;
when the abnormality is detected, the consumption progress coordinating device gives an alarm.
2. A system for message consumption by a distributed message queue, comprising: a business processing device and a consumption progress coordinating device;
the service processing device comprises: a consumption control thread and at least one consumption regular thread; wherein each consumption control thread or consumption conventional thread corresponds to one partition; wherein each topic comprises at least two of the partitions;
wherein the service processing device is configured to:
causing each of said consumption control threads or said consumption regular threads to send message consumption status data of one of said partitions to said consumption progress coordinating means; the data structure of the message consumption state data includes: consumption groups, topics, total partition number, current partition, current consumption location, and current log timestamp;
each consumption control thread and the consumption conventional thread acquire synchronous check data from the consumption progress coordinating device; the data structure of the synchronization check data includes: consumption groups, total partition number, cycle and time checkpoints;
each consumption control thread and the consumption conventional thread consume the messages in the period of the synchronous check data;
when the time reaches the time check point, the consumption regular thread sends the message consumption state data to the consumption progress coordination device and acquires the synchronous check data from the consumption progress coordination device;
the consumption progress coordinating apparatus includes: a consumption control thread;
wherein the consumption progress coordinating means is for:
setting a time checkpoint in the synchronous inspection data by the consumption control thread;
when the consumption control thread judges that the current consumption positions and the current log timestamps of the message consumption state data of all the messages of each subarea in the current period are not in the period of the synchronous check data according to the current consumption position and the current log timestamp of each message consumption state data and the period of the synchronous check data, the consumption control thread switches the current period of the synchronous check data into the next period; otherwise, continuing checking;
when the time reaches the time check point, the consumption control thread judges whether the current consumption position and the current log timestamp of the message consumption state data of all the messages of each partition in the current period are in the period of the synchronous check data or not according to the current consumption position and the current log timestamp of each message consumption state data and the period of the synchronous check data;
when the message to be consumed is not in the period, suspending the consumption of the message;
and when the abnormity is detected, an alarm is given.
3. A storage device storing computer program instructions for performing the method of claim 1.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811527275.5A CN109739659B (en) | 2018-12-13 | 2018-12-13 | Message consumption method and system of distributed message queue |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811527275.5A CN109739659B (en) | 2018-12-13 | 2018-12-13 | Message consumption method and system of distributed message queue |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109739659A CN109739659A (en) | 2019-05-10 |
| CN109739659B true CN109739659B (en) | 2020-06-19 |
Family
ID=66358901
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811527275.5A Active CN109739659B (en) | 2018-12-13 | 2018-12-13 | Message consumption method and system of distributed message queue |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109739659B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111600774B (en) * | 2020-05-13 | 2021-11-12 | 北京奇艺世纪科技有限公司 | Consumption delay determination method, system, device, equipment and readable storage medium |
| CN111966943A (en) * | 2020-08-13 | 2020-11-20 | 上海哔哩哔哩科技有限公司 | Streaming data distribution method and system |
| CN112615929B (en) * | 2020-12-24 | 2023-01-31 | 上海掌门科技有限公司 | Method and equipment for pushing messages |
| CN112905668B (en) * | 2021-03-05 | 2023-06-06 | 北京中经惠众科技有限公司 | Database derivative method, device and medium based on distributed data stream processing engine |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120066694A1 (en) * | 2010-09-10 | 2012-03-15 | International Business Machines Corporation | Event overflow handling by coalescing and updating previously-queued event notification |
| CN103259737B (en) * | 2013-04-18 | 2016-01-13 | 西安交通大学 | A kind of method for rapidly positioning of flow of parallel storage high-speed network |
| CN107015872B (en) * | 2016-12-09 | 2018-03-16 | 上海壹账通金融科技有限公司 | The processing method and processing device of monitoring data |
| CN106598760B (en) * | 2016-12-19 | 2020-07-10 | 北京奇虎科技有限公司 | Consumption method and device of message queue |
| CN108595483B (en) * | 2018-03-13 | 2020-11-24 | 腾讯科技（深圳）有限公司 | Data processing method and related device |
| CN108874562B (en) * | 2018-06-21 | 2022-08-02 | 北京顺丰同城科技有限公司 | Distributed high-concurrency message queue pushing system |
-
2018
- 2018-12-13 CN CN201811527275.5A patent/CN109739659B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN109739659A (en) | 2019-05-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109739659B (en) | Message consumption method and system of distributed message queue | |
| US10509585B2 (en) | Data synchronization method, apparatus, and system | |
| US11157061B2 (en) | Processor management via thread status | |
| US10212226B2 (en) | Node cluster synchronization | |
| US9418129B2 (en) | Adaptive high-performance database redo log synchronization | |
| JP6818014B2 (en) | Operation retry method and equipment for jobs | |
| CN110018914B (en) | Shared memory based message acquisition method and device | |
| US9459949B2 (en) | Methods and apparatus to provide failure detection | |
| CN109308170B (en) | Data processing method and device | |
| US20200264777A1 (en) | Method and apparatus for upgrading a distributed storage system | |
| CN107769943B (en) | Method and equipment for switching main and standby clusters | |
| CN107341062B (en) | Data pushing method, device, equipment and storage medium | |
| US20170344439A1 (en) | System and method for input data fault recovery in a massively parallel real time computing system | |
| EP2972852B1 (en) | System management interrupt handling for multi-core processors | |
| GB2582783A (en) | Checkpointing | |
| CN104346307A (en) | System and Method for Direct Memory Access Transfers | |
| US9141439B2 (en) | System and method for reporting a synchronization event in a runtime system of a computer system | |
| CN106815094B (en) | Method and equipment for realizing transaction submission in master-slave synchronization mode | |
| US9606879B2 (en) | Multi-partition networking device and method therefor | |
| CN111831408A (en) | Asynchronous task processing method and device, electronic equipment and medium | |
| CN112463514A (en) | Monitoring method and device for distributed cache cluster | |
| US20180309702A1 (en) | Method and device for processing data after restart of node | |
| CN115658356A (en) | Watchdog feeding method and system in Linux system | |
| CN111143127B (en) | Method, device, storage medium and equipment for supervising network equipment | |
| CN107122489B (en) | Data comparison method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PP01 | Preservation of patent right | ||
| PP01 | Preservation of patent right |
Effective date of registration: 20210305 Granted publication date: 20200619 |