CN109739659A - A kind of method and system of the information consumption of Distributed Message Queue - Google Patents
A kind of method and system of the information consumption of Distributed Message Queue Download PDFInfo
- Publication number
- CN109739659A CN109739659A CN201811527275.5A CN201811527275A CN109739659A CN 109739659 A CN109739659 A CN 109739659A CN 201811527275 A CN201811527275 A CN 201811527275A CN 109739659 A CN109739659 A CN 109739659A
- Authority
- CN
- China
- Prior art keywords
- consumption
- thread
- control
- information
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Debugging And Monitoring (AREA)
Abstract
The present invention provides a kind of method and systems of the information consumption of Distributed Message Queue, this method comprises: control of consumption thread or consumption common thread send the information consumption status data extremely consumption progress conditioning unit of a subregion；When the consumption progress conditioning unit judges each subregion when the entire message of current period has consumed completion according to the information consumption status data, the current period of sync check data is switched to next period by the control of consumption thread；Each control of consumption thread and the consumption common thread obtain the sync check data from the consumption progress conditioning unit；Each control of consumption thread and the consumption common thread consume to the message in the period in the sync check data method and system of the information consumption of Distributed Message Queue provided by the invention, based on specific business demand, some scenes needs are counted in strict accordance with the time cycle, when especially recalculating data, it cannot across period statistics, otherwise it will cause statistical error, the method and system of the information consumption of Distributed Message Queue of the invention is also very applicable for this scene.
Description
Technical field
The present invention relates to a kind of method of information technology field more particularly to information consumption of Distributed Message Queue and it is
System.
Background technique
In real-time calculate, due to the design principle of Distributed Message Queue, message is sequence, subregion in single subregion
Between be it is independent, the inconsistent and each consuming thread consumption rate of the message number of each subregion is inconsistent, will result in from
It is inconsistent for reading message time sequence when log is calculated to log.For example, the message time of subregion 1, which has arrived consumption, arrives 6:00,
And subregion 2 is since message is relatively more, packing ratio is more serious, may only to 5:50 points, between subregion, poor 10 minutes even for consumption
The longer time.In the insensitive calculating scene of message time sequence, the time difference bring that can ignore log influences.
But some business demands are stringenter to time sequence requirement, it is very difficult to which received message random ordering leads to statistical result difference.Than
Such as, user supplements order with money at 23:55 points, probably due to accumulation delay, just processed after 00:05 points of message, can be united
It counts in the different periods.
Watermark is the solution of the most common processing Distributed Message Queue consumption sequence problem now, this
Scheme has realization in many frames of processing in real time.Watermark is when in use, it is possible to understand that the time postponed for one,
That is how long task delay triggers.If we will calculate once per minute, watermark is set as (current time -10)
Second, then it can be in each minute task computation for triggering the last period for 10 seconds.So will not be arranged under normal circumstances too big
Otherwise it is very big to will cause statistical delay by watermark.The mode of watermark scheme is can to tolerate to prolong by setting one
The slow time solves the problems, such as log random ordering, and this settling mode will also result in statistics and miss in the scene for needing strict sequential order
Difference.If needing to restore the data from some period before especially after real-time statistics program exception exits, consumption rate
Inconsistent will cause can not solve the problems, such as even if using watermark mode, because the out-of-order time window size of tolerance is not
What can be put is very big.If being exactly that processing in real time is being provided with because postponing caused by data stacking there are also a kind of scene
After watermark, the log that will cause accumulation can be ignored, and log is caused not calculate correctly.
The thought to reorder is that the data consumed once are sorted before calculating, and this settling mode can only solve
The order of the data in batch is certainly currently executed, and cannot fundamentally solve the problems, such as that consumption rate is inconsistent.
Summary of the invention
It is an object of the present invention to provide a kind of method and systems of the information consumption of Distributed Message Queue, to solve
When being consumed to each Distributed Message Queue, coordinate consumption rate the problem of.
The method of the information consumption of Distributed Message Queue provided by the invention, comprising:
Information consumption status data to the consumption progress that control of consumption thread or consumption common thread send a subregion is assisted
Adjust device；
When the consumption progress conditioning unit judges each subregion current according to the information consumption status data
When the entire message in period has consumed completion, the current period of sync check data is switched to down by the control of consumption thread
One period；
Each control of consumption thread and the consumption common thread are from described in consumption progress conditioning unit acquisition
Sync check data；
Each control of consumption thread and the consumption common thread are in the period in the sync check data
Message consumed.
Further, method of the present invention, further includes:
Time check point is set in the sync check data by the control of consumption thread；
When the time reaching the time check point, the consumption common thread and control of consumption thread are according to the message
Consumption status data judge whether each subregion consumes completion in the entire message of current period.
Further, method of the present invention, further includes:
When the time reaching the time check point, the control of consumption thread sends the information consumption status data extremely
The consumption progress conditioning unit and the sync check data are obtained from the consumption progress conditioning unit.
Further, the data structure of method of the present invention, the information consumption status data includes:
Consumption group, theme, total number of partitions, current partition, current consumption position and current log timestamp.
Further, the data structure of method of the present invention, the sync check data includes:
Consumption group, total number of partitions, period and time check point.
The system of the information consumption of Distributed Message Queue provided by the invention, comprising: business processing device and consume into
Spend conditioning unit；
The business processing device includes: a control of consumption thread and at least one consumption common thread；Wherein, each
The control of consumption thread or the corresponding subregion of the consumption common thread；Wherein, each theme includes described at least two
Subregion；
Wherein, the business processing device is used for:
Each control of consumption thread or the consumption common thread is set to send the information consumption shape of a subregion
State data are to the consumption progress conditioning unit；
Each control of consumption thread and the consumption common thread are from described in consumption progress conditioning unit acquisition
Sync check data；
Each control of consumption thread and the consumption common thread are in the period in the sync check data
Message consumed；
The consumption progress conditioning unit includes: control of consumption thread；
Wherein, the consumption progress conditioning unit is used for:
When the control of consumption thread judges each subregion in current period according to the information consumption status data
Entire message when having consumed completion, the current period of sync check data is switched to next week by the control of consumption thread
Phase.
Further, system of the present invention, the consumption progress conditioning unit are used for:
Time check point is set in the sync check data by the control of consumption thread；
When the data of control of consumption thread consumption reach the time check point, the control of consumption thread is according to
Information consumption status data judges whether each subregion consumes completion in the entire message of current period.
Further, system of the present invention, the business processing device are also used to:
When the time reaching the time check point, the control of consumption thread sends the information consumption status data extremely
The consumption progress conditioning unit and the sync check data are obtained from the consumption progress conditioning unit.
Further, the data structure of system of the present invention, the information consumption status data includes:
Consumption group, theme, total number of partitions, current partition, current consumption position and current log timestamp.
Further, the data structure of system of the present invention, the sync check data includes:
Consumption group, total number of partitions, period and time check point.
The present invention also provides a kind of storage equipment, the storage equipment stores computer program instructions, the computer journey
Sequence instructs method according to the present invention to be executed.
The method and system of the information consumption of Distributed Message Queue provided by the invention, can be in process between process
The mechanism of inspection, come coordinate each subregion consumption speed, achieve the purpose that coordinate mutually, can in consumption control it is different
The speed of information consumption under subregion.In addition, being based on specific business demand, some scenes needs are united in strict accordance with the time cycle
Meter, when especially recalculating data, cannot across period statistics, otherwise will cause statistical error, distributed message of the invention
The method and system of the information consumption of queue is also very applicable for this scene.
Detailed description of the invention
By reading a detailed description of non-restrictive embodiments in the light of the attached drawings below, of the invention other
Feature, objects and advantages will become more apparent upon:
Fig. 1 is the flow diagram of the method for the information consumption for the Distributed Message Queue that the embodiment of the present invention one provides；
Fig. 2 is the schematic illustration of the method for the information consumption for the Distributed Message Queue that the embodiment of the present invention one provides；
Fig. 3 is the flow diagram of the method for the information consumption of Distributed Message Queue provided by Embodiment 2 of the present invention；
Fig. 4 is the structural schematic diagram of the system of the information consumption for the Distributed Message Queue that the embodiment of the present invention three provides.
The same or similar appended drawing reference represents the same or similar component in attached drawing.
Specific embodiment
Present invention is further described in detail with reference to the accompanying drawing.
In the present invention one typical configuration, terminal, the equipment of service network include one or more processors
(CPU), input/output interface, network interface and memory.
Memory may include the non-volatile memory in computer-readable medium, random access memory (RAM) and/or
The forms such as Nonvolatile memory, such as read-only memory (ROM) or flash memory (flashRAM).Memory is showing for computer-readable medium
Example.
Computer-readable medium includes permanent and non-permanent, removable and non-removable media, can be by any side
Method or technology realize that information stores.Information can be the device or other numbers of computer readable instructions, data structure, program
According to.The example of the storage medium of computer includes, but are not limited to phase change memory (PRAM), static random access memory
(SRAM), dynamic random access memory (DRAM), other kinds of random access memory (RAM), read-only memory
(ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory techniques, CD-ROM (CD-
ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storages
Equipment or any other non-transmission medium, can be used for storage can be accessed by a computing device information.
Fig. 1 is the flow diagram of the method for the information consumption for the Distributed Message Queue that the embodiment of the present invention one provides,
As shown in Figure 1, the method for the information consumption for the Distributed Message Queue that the embodiment of the present invention one provides, comprising:
Step S101, control of consumption thread or consumption common thread send the information consumption status data of a subregion to disappearing
Take progress conditioning unit.
Step S102, when the consumption progress conditioning unit judges each described point according to the information consumption status data
When the entire message of current period has consumed completion, the control of consumption thread is by the current period of sync check data in area
It is switched to next period.
Step S103, each control of consumption thread and the consumption common thread are from the consumption progress conditioning unit
Obtain the sync check data.
Step S104, each control of consumption thread and the consumption common thread are in the sync check data
Period in message consumed.
Wherein, consuming common thread and control of consumption thread has following relationship: consuming thread is all consumption common thread,
It therefrom elects or specified one has the consuming thread of control coordination function as control of consumption thread.Consume common thread and
The code of control of consumption thread is all consistent, and only role is different, executes different functions.Control of consumption thread and consumption
Common thread all can consumption data, only control of consumption thread have consumption function and control coordination function, consume common thread
Only consumption function.
Wherein, the data structure of the information consumption status data includes:
Consumption group, theme, total number of partitions, current partition, current consumption position and current log timestamp.
For example, the data structure of information consumption status data are as follows:
$group|$topic|$paritition_cnt|$partition|$offset|$timestamp
Wherein, the data structure of the sync check data includes:
Consumption group, total number of partitions, period and time check point.
For example, the data structure of sync check data are as follows:
$group|$partition_cnt|period|$timestamp
Wherein, theme (topic) is in the distributed message queue for distinguishing different business classification, in each theme
(topic) there are several subregions under, due to being distributed deployment, can suitably amplify the quantity of subregion, it is bigger to meet
Data throughout demand.
Fig. 2 is the schematic illustration of the method for the information consumption for the Distributed Message Queue that the embodiment of the present invention one provides,
As shown in Fig. 2, total system, which is divided into Distributed Message Queue, business processing, consumption progress, coordinates three logical gates, each disappear
Take the corresponding subregion of thread (consuming thread herein refers to control of consumption thread and consumption common thread), for example, consumption control
Thread processed corresponds to subregion 1, and consumption common thread corresponds to other subregions.Each subregion can use distributed deployment, to be formed
The Distributed Message Queue of each subregion.Control of consumption thread is by the information consumption status data of corresponding subregion according to mark
Quasi- data structure is arranged and is sent to shared drive.Shared drive realizes consumption progress association by consumption progress conditioning unit
Tune mechanism consumes control of consumption thread currently the disappearing according to each information consumption status data received of progress conditioning unit
Take cyclic check (check) subregion 1 to subregion n of position and current log timestamp and sync check data in current week
All the message of phase whether complete by consumption, i.e., whether the current consumption position of information consumption status data and current log timestamp
In period in sync check data.When message all complete by consumption, control of consumption thread is worked as sync check data
The preceding period is switched to next period, that is, starts next period, otherwise continue checking.Common thread is consumed to obtain together from shared drive
Step checks data, and according to the current consumption position of information consumption status data and current log timestamp and sync check
The period of data judges whether message to be consumed was in the period of sync check data.When message to be consumed is in the period
When interior, which is consumed.When message to be consumed was not in the period, suspend the consumption to the message.
Fig. 3 is the flow diagram of the method for the information consumption of Distributed Message Queue provided by Embodiment 2 of the present invention,
As shown in figure 3, the method for the information consumption of Distributed Message Queue provided by Embodiment 2 of the present invention, comprising:
Time check point is arranged in the sync check data by the control of consumption thread in step S301.
Step S302, when message common thread consumption data reaches the time check point, each consumption common thread
Send the information consumption status data extremely consumption progress conditioning unit an of subregion.
Step S303, when message control thread consumption data reaches the time check point, the control of consumption thread
Judge whether each subregion consumes completion in the entire message of current period according to the information consumption status data.
Step S304, when the control of consumption thread of the consumption progress conditioning unit is according to the information consumption status data
Each subregion is judged when the entire message of current period has consumed completion, and the control of consumption thread is by sync check
The current period of data is switched to next period.
Step S305, when the current period of consumption inspection data is switched to next period, each control of consumption line
Journey and consumption common thread obtain the sync check data from the consumption progress conditioning unit.
Step S306, each control of consumption thread and consumption common thread are to the week for being in the sync check data
Message in phase is consumed.
Wherein, the data structure of the information consumption status data includes:
Consumption group, theme, total number of partitions, current partition, current consumption position and current log timestamp.
Wherein, the data structure of the sync check data includes:
Consumption group, total number of partitions, period and time check point.
Wherein it is possible to time check point is arranged in sync check data by control of consumption thread, it is more flexible to meet
Checking mechanism.In order to avoid occupying excessive computing resource, only at arrival time checkpoint, control of consumption thread judges each
Subregion current period message whether all consumption complete, and consume common thread send information consumption status data and
Obtain sync check data.In switching cycle, it need to guarantee that the data in last period all enter and calculate.And it is reached in consumption log
When checkpoint, control of consumption thread and consumption common thread, which are possible to wait and check whether, can continue to disappear to message
Take.In order to improve efficiency, it need to be arranged and check that the time waited is no more than default waiting time threshold value, such as 1 second every time, to guarantee
Real-time.
Specifically, the information consumption status data of corresponding subregion is sent to shared drive by consumption common thread
Consumption progress conditioning unit.The control of consumption thread of consumption progress conditioning unit is according to each information consumption status number received
According to current consumption position and current log timestamp and sync check data each subregion of cyclic check in current period
All message whether complete by consumption, i.e., whether the current consumption position of information consumption status data and current log timestamp are in
In the period of sync check data.When message all complete by consumption, control of consumption thread is by the current week of sync check data
Phase is switched to next period, that is, starts next period, otherwise continue checking.It consumes common thread and obtains synchronous inspection from shared drive
Data are looked into, and according to the current consumption position of information consumption status data and current log timestamp and sync check data
Period judge whether message to be consumed was in the period of sync check data.When message to be consumed was in the period
When, which is consumed.When message to be consumed was not in the period, suspend the consumption to the message.
Wherein, when having checked abnormal, consumption progress conditioning unit need to issue alarm in time.In processing universal time coordinated, need
The mechanism retried to abnormal capture and increase is paid attention to, to guarantee system stability.
Fig. 4 is the structural schematic diagram of the system of the information consumption for the Distributed Message Queue that the embodiment of the present invention three provides,
As shown in figure 4, the system of the information consumption of Distributed Message Queue provided by the invention, comprising: business processing device 41 and disappear
Take progress conditioning unit 42.
The business processing device 41 includes: at least one consumption common thread and a control of consumption thread；Wherein, often
A consumption common thread or the corresponding subregion of the control of consumption thread；Wherein, each main body includes at least two institutes
State subregion.
Wherein, the business processing device 41 is used for:
Each consumption common thread or the control of consumption thread is set to send the information consumption shape of a subregion
State data are to the consumption progress conditioning unit；
Each consumption common thread and the control of consumption thread are from described in consumption progress conditioning unit acquisition
Sync check data；
Each consumption common thread and the control of consumption thread are in the period in the sync check data
Message consumed.
Wherein, the data structure of the information consumption status data includes: consumption group, theme, total number of partitions, current point
Area, current consumption position and current log timestamp.
Wherein, the data structure of the sync check data includes: consumption group, total number of partitions, period and time check point.
The business processing device 41 can be also used for:
When the time reaching the time check point, the consumption common thread sends the information consumption status data extremely
The consumption progress conditioning unit and the sync check data are obtained from the consumption progress conditioning unit.
The consumption progress conditioning unit 42 includes: control of consumption thread.
Wherein, the consumption progress conditioning unit 42 is used for:
When the control of consumption thread judges each subregion in current period according to the information consumption status data
Entire message when having consumed completion, the current period of sync check data is switched to next week by the control of consumption thread
Phase.
The consumption progress conditioning unit 42 can be also used for:
Time check point is set in the sync check data by the control of consumption thread；
When the time reaching the time check point, the control of consumption thread is sentenced according to the information consumption status data
Whether each subregion that breaks in the entire message of current period consumes completion.
The system of the embodiment of the present invention three, the realization device of the method for the embodiment of the present invention two, specifically refers to the present invention
Embodiment two, details are not described herein again.
The present invention also provides a kind of storage equipment, the storage equipment stores computer program instructions, the computer journey
Sequence instruct according to embodiments of the present invention one or the method for embodiment two executed.
It should be noted that the present invention can be carried out in the assembly of software and/or software and hardware, for example, can adopt
With specific integrated circuit (ASIC), general purpose computer or any other realized similar to hardware device.In some embodiments
In, software program of the invention can be executed by processor to realize above step or function.Similarly, software of the invention
Program (including relevant data structure) can be stored in computer readable recording medium, for example, RAM memory, magnetic or
CD-ROM driver or floppy disc and similar devices.In addition, some of the steps or functions of the present invention may be implemented in hardware, for example,
As the circuit cooperated with processor thereby executing each step or function.
It is obvious to a person skilled in the art that invention is not limited to the details of the above exemplary embodiments, Er Qie
In the case where without departing substantially from spirit or essential attributes of the invention, the present invention can be realized in other specific forms.Therefore, no matter
From the point of view of which point, the present embodiments are to be considered as illustrative and not restrictive, and the scope of the present invention is by appended power
Benefit requires rather than above description limits, it is intended that all by what is fallen within the meaning and scope of the equivalent elements of the claims
Variation is included in the present invention.Any reference signs in the claims should not be construed as limiting the involved claims.This
Outside, it is clear that one word of " comprising " does not exclude other units or steps, and odd number is not excluded for plural number.That states in device claim is multiple
Unit or device can also be implemented through software or hardware by a unit or device.
Claims (11)
1. a kind of method of the information consumption of Distributed Message Queue characterized by comprising
Information consumption status data to the consumption progress that control of consumption thread or consumption common thread send a subregion coordinates dress
It sets；
When the consumption progress conditioning unit judges each subregion in current period according to the information consumption status data
Entire message when having consumed completion, the current period of sync check data is switched to next week by the control of consumption thread
Phase；
Each control of consumption thread obtains with the consumption common thread from the consumption progress conditioning unit described synchronous
Check data；
Each control of consumption thread and the consumption common thread are to disappearing in the period in the sync check data
Breath is consumed.
2. the method according to claim 1, wherein further include:
Time check point is set in the sync check data by the control of consumption thread；
When the time reaching the time check point, the control of consumption thread judges each according to the information consumption status data
Whether a subregion consumes completion in the entire message of current period.
3. according to the method described in claim 2, it is characterized by further comprising:
When the time reaching the time check point, the consumption common thread sends the information consumption status data to described
Consumption progress conditioning unit and the sync check data are obtained from the consumption progress conditioning unit.
4. according to the method in any one of claims 1 to 3, which is characterized in that the number of the information consumption status data
Include: according to structure
Consumption group, theme, total number of partitions, current partition, current consumption position and current log timestamp.
5. according to the method in any one of claims 1 to 3, which is characterized in that the data knot of the sync check data
Structure includes:
Consumption group, total number of partitions, period and time check point.
6. a kind of system of the information consumption of Distributed Message Queue characterized by comprising business processing device and consume into
Spend conditioning unit；
The business processing device includes: a control of consumption thread and at least one consumption common thread；Wherein, each described
Control of consumption thread or the corresponding subregion of the consumption common thread；Wherein, each theme includes at least two subregions；
Wherein, the business processing device is used for:
Each control of consumption thread or the consumption common thread is set to send the information consumption status number of a subregion
According to the extremely consumption progress conditioning unit；
Each control of consumption thread obtains with the consumption common thread from the consumption progress conditioning unit described synchronous
Check data；
Each control of consumption thread and the consumption common thread are to disappearing in the period in the sync check data
Breath is consumed；
The consumption progress conditioning unit includes: control of consumption thread；
Wherein, the consumption progress conditioning unit is used for:
When the control of consumption thread judges each subregion in the complete of current period according to the information consumption status data
When portion's message has consumed completion, the current period of sync check data is switched to next period by the control of consumption thread.
7. system according to claim 6, which is characterized in that the consumption progress conditioning unit is used for:
Time check point is set in the sync check data by the control of consumption thread；
When the time reaching the time check point, the control of consumption thread judges each according to the information consumption status data
Whether a subregion consumes completion in the entire message of current period.
8. system according to claim 7, which is characterized in that the business processing device is also used to:
When the time reaching the time check point, the consumption common thread sends the information consumption status data to described
Consumption progress conditioning unit and the sync check data are obtained from the consumption progress conditioning unit.
9. the system according to any one of claim 6 to 8, which is characterized in that the number of the information consumption status data
Include: according to structure
Consumption group, theme, total number of partitions, current partition, current consumption position and current log timestamp.
10. the system according to any one of claim 6 to 8, which is characterized in that the data knot of the sync check data
Structure includes:
Consumption group, total number of partitions, period and time check point.
11. a kind of storage equipment, which is characterized in that the storage equipment stores computer program instructions, the computer program
Instruction the method according to any one of claims 1 to 5 is executed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811527275.5A CN109739659B (en) | 2018-12-13 | 2018-12-13 | Message consumption method and system of distributed message queue |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811527275.5A CN109739659B (en) | 2018-12-13 | 2018-12-13 | Message consumption method and system of distributed message queue |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109739659A true CN109739659A (en) | 2019-05-10 |
| CN109739659B CN109739659B (en) | 2020-06-19 |
Family
ID=66358901
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811527275.5A Active CN109739659B (en) | 2018-12-13 | 2018-12-13 | Message consumption method and system of distributed message queue |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109739659B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111600774A (en) * | 2020-05-13 | 2020-08-28 | 北京奇艺世纪科技有限公司 | Consumption delay determination method, system, device, equipment and readable storage medium |
| CN111966943A (en) * | 2020-08-13 | 2020-11-20 | 上海哔哩哔哩科技有限公司 | Streaming data distribution method and system |
| CN112615929A (en) * | 2020-12-24 | 2021-04-06 | 上海掌门科技有限公司 | Method and equipment for pushing messages |
| CN112905668A (en) * | 2021-03-05 | 2021-06-04 | 北京中经惠众科技有限公司 | Database derivative method, apparatus, and medium based on distributed data stream processing engine |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120198477A1 (en) * | 2010-09-10 | 2012-08-02 | International Business Machines Corporation | Event overflow handling by coalescing and updating previously-queued event notification |
| CN103259737A (en) * | 2013-04-18 | 2013-08-21 | 西安交通大学 | Method for quickly positioning parallel storage high speed network flow |
| CN106598760A (en) * | 2016-12-19 | 2017-04-26 | 北京奇虎科技有限公司 | Message queue consumption method and apparatus |
| CN107015872A (en) * | 2016-12-09 | 2017-08-04 | 上海壹账通金融科技有限公司 | The processing method and processing device of monitoring data |
| CN108595483A (en) * | 2018-03-13 | 2018-09-28 | 腾讯科技（深圳）有限公司 | Data processing method and relevant apparatus |
| CN108874562A (en) * | 2018-06-21 | 2018-11-23 | 北京顺丰同城科技有限公司 | Distributed high concurrent message queue supplying system |
-
2018
- 2018-12-13 CN CN201811527275.5A patent/CN109739659B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120198477A1 (en) * | 2010-09-10 | 2012-08-02 | International Business Machines Corporation | Event overflow handling by coalescing and updating previously-queued event notification |
| CN103259737A (en) * | 2013-04-18 | 2013-08-21 | 西安交通大学 | Method for quickly positioning parallel storage high speed network flow |
| CN107015872A (en) * | 2016-12-09 | 2017-08-04 | 上海壹账通金融科技有限公司 | The processing method and processing device of monitoring data |
| CN106598760A (en) * | 2016-12-19 | 2017-04-26 | 北京奇虎科技有限公司 | Message queue consumption method and apparatus |
| CN108595483A (en) * | 2018-03-13 | 2018-09-28 | 腾讯科技（深圳）有限公司 | Data processing method and relevant apparatus |
| CN108874562A (en) * | 2018-06-21 | 2018-11-23 | 北京顺丰同城科技有限公司 | Distributed high concurrent message queue supplying system |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111600774A (en) * | 2020-05-13 | 2020-08-28 | 北京奇艺世纪科技有限公司 | Consumption delay determination method, system, device, equipment and readable storage medium |
| CN111966943A (en) * | 2020-08-13 | 2020-11-20 | 上海哔哩哔哩科技有限公司 | Streaming data distribution method and system |
| CN112615929A (en) * | 2020-12-24 | 2021-04-06 | 上海掌门科技有限公司 | Method and equipment for pushing messages |
| CN112615929B (en) * | 2020-12-24 | 2023-01-31 | 上海掌门科技有限公司 | Method and equipment for pushing messages |
| CN112905668A (en) * | 2021-03-05 | 2021-06-04 | 北京中经惠众科技有限公司 | Database derivative method, apparatus, and medium based on distributed data stream processing engine |
| CN112905668B (en) * | 2021-03-05 | 2023-06-06 | 北京中经惠众科技有限公司 | Database derivative method, device and medium based on distributed data stream processing engine |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109739659B (en) | 2020-06-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109739659A (en) | A kind of method and system of the information consumption of Distributed Message Queue | |
| CN100449651C (en) | Memory system having fast and slow data reading mechanisms | |
| CN111417935B (en) | Automatic data chip address sequencer for address/command chip synchronization for distributed buffer memory systems | |
| US20170006097A1 (en) | Node cluster synchronization | |
| CN106817295A (en) | A kind of message processing apparatus and method | |
| US9459949B2 (en) | Methods and apparatus to provide failure detection | |
| CN102521265B (en) | Dynamic consistency control method in massive data management | |
| CN102750257B (en) | On-chip multi-core shared storage controller based on access information scheduling | |
| JPS6121562A (en) | Faul allowance synchronizer for multiple processor system | |
| CN110231995B (en) | Task scheduling method, device and storage medium based on Actor model | |
| US20090319712A1 (en) | Reducing core wake-up latency in a computer system | |
| US20150378782A1 (en) | Scheduling of tasks on idle processors without context switching | |
| US20170293516A1 (en) | Temporal Relationship Extension of State Machine Observer | |
| WO2018024076A1 (en) | Flow velocity control method and device | |
| CN111078422A (en) | Message processing method, message processing device, readable storage medium and electronic equipment | |
| US9606879B2 (en) | Multi-partition networking device and method therefor | |
| CN113778650A (en) | Task scheduling method and device, electronic equipment and storage medium | |
| US10318466B2 (en) | Method and apparatus for handling outstanding interconnect transactions | |
| JP2000259429A (en) | Device and method for managing timer | |
| KR101499890B1 (en) | Low Latency Framework System | |
| EP1766510B1 (en) | Microprocessor output ports and control of instructions provided therefrom | |
| CN106774397A (en) | A kind of four redundance flight control systems computer synchronous method | |
| CN110795239A (en) | Application memory leakage detection method and device | |
| CN103793207B (en) | A kind of intelligent dispatching method of single-threaded multipriority system | |
| KR100433079B1 (en) | Input data processing circuit |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PP01 | Preservation of patent right | ||
| PP01 | Preservation of patent right |
Effective date of registration: 20210305 Granted publication date: 20200619 |