CN108733839A - A kind of statistical method and device of mass data - Google Patents
A kind of statistical method and device of mass data Download PDFInfo
- Publication number
- CN108733839A CN108733839A CN201810528191.7A CN201810528191A CN108733839A CN 108733839 A CN108733839 A CN 108733839A CN 201810528191 A CN201810528191 A CN 201810528191A CN 108733839 A CN108733839 A CN 108733839A
- Authority
- CN
- China
- Prior art keywords
- statistics
- task
- data
- statistics task
- statistic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Abstract
The present invention relates to big data platform, technical field of data processing, more particularly to the statistical method and device of a kind of mass data.This method is：Based on preset statistics task state table, the initial data got is counted in the data statistic of corresponding time granularity, wherein, exist between different time granularity comprising or by inclusion relation, the data statistic of different time granularity is respectively used to store the initial data got in the corresponding period, including at least the task names and task status of the corresponding statistics task of all time granularities in the current statistic period in statistics task state table, it is respectively arranged with corresponding timing statistics for each statistics task, task status is for characterizing whether statistics task is completed in corresponding timing statistics；The anomaly statistics task in statistic processes is detected based on preset detection cycle, and carries out statistical disposition for anomaly statistics task.
Description
Technical field
The present invention relates to big data platform, technical field of data processing, more particularly to a kind of statistical method of mass data
And device.
Background technology
In big data platform technical field, a big data platform can access multiple devices, and an equipment is sent to
The daily record amount of big data platform is very huge, and daily record usually has corresponding retention periods, then, big data platform database
The middle daily record data that can store magnanimity, then, it is slow to directly result in interface queries velocity anomaly, is even reading big data
It is overtime when the Rest API of the database of platform.In this way, big data platform needs to be counted for the mass data received
And merger, satisfactory data are counted according to certain demands, statistical result is returned and is stored in corresponding database table
In, to reduce resource occupation and data redundancy, improve page interrogation performance.
Currently, the data statistics mode that big data platform uses includes any one in following manner：
Mode one, by establishing a data statistic, based on preset statistical rules by the daily record data received into
Row statistics, and be stored in the data statistic.However, employing mode one carries out data statistics, the number being stored in data statistic
Can be increasing according to amount, when data volume reaches a certain threshold value, it is inefficient to may result in interface queries, even cannot respond to
Page interrogation is as a result, and be unable to ensure the accuracy of statistical data.
Mode two, by establishing multiple data statistics, e.g., data statistic is established as unit of day, by daily day
Will data are stored in corresponding data statistic, although reducing the data volume stored in individual data statistic, number
The quantity of table is more and more according to statistics, and when interface statistical query may need to cross over multiple data statistics, so as to cause the page
Search efficiency is not high, even cannot respond to page interrogation as a result, and being unable to ensure the accuracy of statistical data.
Mode three, by be arranged the corresponding data statistic of different time granularity, initial data is integrated into the small time
In the corresponding data statistic of granularity, then that the corresponding data statistic of small time granularity is integrated into big time granularity is opposite
In the data statistic answered.However, during carrying out statistics merger to initial data, any data statistics task all may
Lead to the appearance of statistical delay due to data jamming, and then influence whether next data statistics task, with timing statistics
The statistical delay of the growth of span, accumulation can be increasing, in this way, the real-time of statistical data will be directly influenced, and not
The processing that statistical result verification and anomaly statistics task are carried out to statistics task is unable to ensure statistics when measurement period is longer
The accuracy of data.
Above data statistics mode often only focuses on data statistics simultaneously, has not focused on management and the system of statistics task
Frame is counted, when actual deployment, it is also possible to cause statistical framework or statistics task to manage excessively complicated, so that after
Phase maintenance cost is high.
Invention content
The purpose of the embodiment of the present invention is to provide a kind of statistical method of high magnitude of data and device, to solve in the prior art
Existing statistical data accuracy, real-time be not high, the inefficient problem of page interrogation.
The specific technical solution provided in the embodiment of the present invention is as follows：
In a first aspect, the present invention provides a kind of statistical method of mass data, which includes：Based on preset system
Task status table is counted, the initial data got is counted in the data statistic of corresponding time granularity, wherein different time
Exist between granularity and is respectively used to store the corresponding time comprising or by inclusion relation, the data statistic of above-mentioned different time granularity
The initial data that gets in section includes at least all time granularities point in the current statistic period in above-mentioned statistics task state table
The task names and task status of not corresponding statistics task are respectively arranged with corresponding timing statistics for each statistics task,
Above-mentioned task status is for characterizing whether statistics task is completed in corresponding timing statistics；It is detected based on preset detection cycle
The anomaly statistics task of statistics is not completed in statistic processes in corresponding timing statistics, and is carried out for above-mentioned anomaly statistics task
Statistical disposition.
Using the statistical method of mass data provided by the invention, by the way that statistics task state table is arranged, for each
Corresponding timing statistics are arranged in business, carry out statistical disposition to the initial data got, all statistics tasks can be according to system
Meter task status table judges whether executed, and corresponding timing statistics are respectively set for each statistics task, can be more accurate
Really, easily management statistics task, and so that whether on time each statistics task complete not interfering with next system by statistics
The execution of meter task is avoided the accumulation for being led to statistical delay due to data jamming, and then ensures the real-time of statistical data
Property, further, provided in the statistical method of mass data provided by the invention in statistic processes in corresponding timing statistics
The scheme that interior unfinished statistics task is counted again, it is ensured that the accuracy of statistical data.
Optionally, which further comprises：By resident task manager, timing task management device and statistics task
Queue forms statistical framework, and the statistical framework is for statistics task state table and the statistics task shape described in maintenance and management
Each statistics task in state table；The statistics task administrative mechanism of the statistical framework includes Resident Process management and timing process pipe
Reason, and executed respectively by Resident Process manager and timing process manager；Running frequency is more than or equal to the system of first frequency
Meter task is set as resident statistics task, by the Resident Process manager administration and starts statistics task queue；And it will operation
The statistics task that frequency is less than or equal to second frequency is set as timing statistics task, by the timing process manager start by set date
Timing statistics task, and timing statistics task is added in statistics task queue, wherein the first frequency is more than described the
Two frequencies.
Above-mentioned optional embodiment characterization, also discloses in of the invention including resident task manager, timed task pipe
Manage device and statistics task queue and form statistical framework, and by the different statistics task of running frequency be divided into timing statistics task and
Resident statistics task, and the process task in statistical framework is divided into timing statistics task and resident statistics process, and two
The different process of kind is each responsible for the execution of different statistics tasks, and respectively since taking-up statistics times in statistics task state table
Business is handled, and is avoided occupying a large amount of CPU and memory source as possible, is improved system statistics efficiency.
Optionally, the timed task by corresponding to big time granularity statistics task and other timed tasks form；Institute
State resident task by corresponding to small time granularity statistics task and the very high other tasks of running frequency form.
Above-mentioned optional embodiment characterization, a kind of preferable specific implementation mode is in the present invention, can be by grain of big time
Statistics task and other tasks of timing being needed to count corresponding to degree (i.e. statistic frequency is relatively low) are set as timed task, and incite somebody to action
Statistics task corresponding to small time granularity (i.e. statistic frequency is higher) is set as resident task.For example, it is assumed that by different time
Granularity is set as minute, hour, day, the moon, corresponding, you can by minute corresponding statistics task be set as it is resident count into
Journey sets hour, day, the moon corresponding statistics task to timing statistics task.
Optionally, it is based on preset statistics task state table, the initial data got is counted on into corresponding time granularity
Data statistic in, including：Object statistics task is obtained from statistics task state table, and above-mentioned object statistics task is added
Enter in statistics task queue；Based on the above-mentioned object statistics task corresponding statistics time started, completes above-mentioned object statistics and appoint
Code function needed for business, the initial data got in the above-mentioned corresponding time granularity of object statistics task is counted
Merger, and count in the above-mentioned corresponding data statistic of object statistics task；If it is determined that in above-mentioned object statistics task phase
Above-mentioned object statistics task is completed in the corresponding object statistics time, then is changed to the task status of above-mentioned object statistics task
It is completed, otherwise, it is abnormal that the task status of above-mentioned object statistics task is changed to statistics.
Above-mentioned optional embodiment characterization, it is a kind of in the present invention that specifically the initial data got is counted
Mode is independent from each other between each statistics task, current statistic task whether can be timely completed can not influence it is next
The normal statistics of a statistics task ensure that the real-time of statistical data.
Optionally, above-mentioned statistical method further comprises：Judge whether the history initial data not counted；?
Judgement result is corresponding historical statistics task to be arranged, and above-mentioned statistics task state table is added in historical statistics task when being,
And it is based on above-mentioned statistics task state table, above-mentioned history initial data is counted in the data statistic of corresponding time granularity.
Above-mentioned optional embodiment characterizes, and further includes to not uniting in the statistical method of mass data provided by the invention
The statistical disposition of the history initial data of meter, passes through the statistics to history initial data so that statistical data is more complete, when useful
When family needs to inquire the data, it is no longer necessary to be obtained from initial data, to improve efficiency data query, improve user
Experience Degree.
Optionally, corresponding historical statistics task is set, including：Earliest time stamp letter based on above-mentioned historical statistical data
Breath and latest time stab information, and the corresponding historical statistics task of above-mentioned different time granularity is arranged, wherein above-mentioned history system
Meter task is resident statistics task.
Above-mentioned optional embodiment characterizes, can be according to this in the present invention when being counted to history initial data
The timestamp information of each data in history initial data, determines the time span of the history initial data, and then when according to this
Between span, in conjunction with the different time granularity that has been arranged, corresponding historical statistics task is set.
Optionally, above-mentioned statistical method further comprises：After the completion of determining above-mentioned historical statistics task, detect above-mentioned
The exception history statistics task of statistics is not completed in history initial data statistic processes in corresponding timing statistics, and for above-mentioned
Exception history statistics task carries out statistical disposition.
Above-mentioned optional embodiment characterizes, can also be periodically right in the statistical method of mass data provided by the invention
The exception that statistics task is completed not in corresponding timing statistics that (including historical statistics and real-time statistics) occur in statistic processes
Statistics task carries out statistical disposition again, it is ensured that the accuracy and integrality of statistical data.
Optionally, above-mentioned statistical method further comprises：Based on data query instruction is received, from the number that statistics is completed
The target data statistical form of each time granularity corresponding with query time is determined in table according to statistics；It unites from above-mentioned target data
The statistical data for meeting querying condition is determined in meter table, and returns to query result.
Second aspect, the present invention provide a kind of statistic device of mass data, which includes：Statistic unit is used
In based on preset statistics task state table, the initial data got is counted on to the data statistic of corresponding time granularity
In, wherein exist between different time granularity comprising or by inclusion relation, the data statistic difference of above-mentioned different time granularity
For storing the initial data got in the corresponding period, the current statistic period is included at least in above-mentioned statistics task state table
The task names and task status of the corresponding statistics task of interior all time granularities, are respectively arranged with for each statistics task
Corresponding timing statistics, above-mentioned task status is for characterizing whether statistics task is completed in corresponding timing statistics；Abnormality processing
Unit, for detecting that not completing the abnormal of statistics in corresponding timing statistics in statistic processes unites based on preset detection cycle
Meter task, and carry out statistical disposition for above-mentioned anomaly statistics task.
Optionally, statistical framework is formed by resident task manager, timing task management device and statistics task queue, it is described
Statistical framework is for each statistics task in statistics task state table and the statistics task state table described in maintenance and management；Institute
The statistics task administrative mechanism for stating statistical framework includes Resident Process management and timing management of process, and respectively by Resident Process pipe
It manages device and timing process manager executes；The statistics task that running frequency is more than or equal to first frequency is set as resident statistics times
Business by the Resident Process manager administration and starts statistics task queue；And running frequency is less than or equal to second frequency
Statistics task is set as timing statistics task, by the timing process manager start by set date timing statistics task, and will determine
When statistics task be added statistics task queue in, wherein the first frequency be more than the second frequency.
Optionally, the timed task by corresponding to big time granularity statistics task and other timed tasks form；Institute
State resident task by corresponding to small time granularity statistics task and the very high other tasks of running frequency form.
Optionally, based on preset statistics task state table, the initial data got is counted on into corresponding time grain
When in the data statistic of degree, above-mentioned statistic unit is used for：Object statistics task is obtained from statistics task state table, and will be upper
Object statistics task is stated to be added in statistics task queue；It is complete based on the above-mentioned object statistics task corresponding statistics time started
At the code function of above-mentioned object statistics required by task, by what is got in the above-mentioned corresponding time granularity of object statistics task
Initial data carries out statistics merger, and counts in the above-mentioned corresponding data statistic of object statistics task；If it is determined that upper
It states and completes above-mentioned object statistics task in the object statistics task corresponding object statistics time, then by above-mentioned object statistics task
Task status be changed to be completed, otherwise, it is abnormal that the task status of above-mentioned object statistics task is changed to statistics.
Optionally, above-mentioned statistic device further comprises historical data processing unit：For judging whether not carry out
The history initial data of statistics；When it is to be to judge result, corresponding historical statistics task is set, and historical statistics task is added
Enter above-mentioned statistics task state table, and be based on above-mentioned statistics task state table, when above-mentioned history initial data is counted on corresponding
Between granularity data statistic in.
Optionally, when corresponding historical statistics task is arranged, above-mentioned historical data processing unit is used for：It is gone through based on above-mentioned
The earliest time stamp information and latest time of history statistical data stab information, and the corresponding history of above-mentioned different time granularity is arranged
Statistics task, wherein above-mentioned historical statistics task is resident statistics task.
Optionally, above-mentioned exception processing unit is further used for：After the completion of determining above-mentioned historical statistics task, detect
The exception history statistics task of statistics is not completed in above-mentioned history initial data statistic processes in corresponding timing statistics, and is directed to
Above-mentioned exception history statistics task carries out statistical disposition.
Optionally, above-mentioned statistic device further comprises data query unit：For being based on receiving data query instruction,
The target data statistical form of each time granularity corresponding with query time is determined from the data statistic for be completed statistics；
The statistical data for meeting querying condition is determined from above-mentioned target data statistical form, and returns to query result.
The third aspect, the present invention provide a kind of computing device, which includes：Memory refers to for storing program
It enables；Processor is executed according to the program of acquisition in above-mentioned first aspect for calling the program instruction stored in above-mentioned memory
Any one method.
Fourth aspect, the present invention provide a kind of computer storage media, which has calculating
Machine executable instruction, above computer executable instruction is for making above computer execute any one of above-mentioned first aspect side
Method.
The present invention has the beneficial effect that：
In conclusion in the embodiment of the present invention, during being counted to mass data, appointed based on preset statistics
Business state table, the initial data got is counted in the data statistic of corresponding time granularity, wherein different time granularity
Between exist and be respectively used to store in the corresponding period comprising or by inclusion relation, the data statistic of above-mentioned different time granularity
It is right respectively to include at least all time granularities in the current statistic period in above-mentioned statistics task state table for the initial data got
The task names and task status for the statistics task answered are respectively arranged with corresponding timing statistics for each statistics task, above-mentioned
Task status is for characterizing whether statistics task is completed in corresponding timing statistics；And it is detected based on preset detection cycle
The anomaly statistics task of statistics is not completed in statistic processes in corresponding timing statistics, and is carried out for above-mentioned anomaly statistics task
Statistical disposition.
Using the above method, by the way that statistics task state table is arranged, corresponding timing statistics are set for each task, it is right
The initial data got carries out statistical disposition, and all statistics tasks can judge whether to have held according to statistics task state table
Row, can more accurately, easily management statistics task, and make each statistics task on time whether complete will not shadow for statistics
The execution to next statistics task is rung, the accumulation of statistical delay is avoided due to data jamming and cause, and then ensures system
The real-time counted, further, provided in the statistical method of mass data provided by the invention in statistic processes
The scheme that the statistics task not completed in corresponding timing statistics is counted again, it is ensured that the accuracy of statistical data, in addition,
Statistical framework deployment provided by the invention is simple, and process manager, the realization of task queue are flexible.
Description of the drawings
Fig. 1 is a kind of detail flowchart of the statistical method of mass data in the embodiment of the present invention；
Fig. 2 is a kind of Governance framework schematic diagram of real-time statistics task and historical statistics task in the embodiment of the present invention；
Fig. 3 is a kind of statistical framework schematic diagram of mass data in the embodiment of the present invention；
Fig. 4 is a kind of structural schematic diagram of the statistic device of mass data in the embodiment of the present invention.
Specific implementation mode
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation describes, it is clear that described embodiments are only a part of the embodiments of the present invention, is not whole embodiment.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
First, term in the embodiment of the present invention " and ", a kind of only incidence relation of description affiliated partner, expression can be with
There are three kinds of relationships, for example, A and B, can indicate：Individualism A exists simultaneously A and B, these three situations of individualism B.Separately
Outside, character "/" herein, it is a kind of relationship of "or" to typically represent forward-backward correlation object.
When the present invention refers to ordinal numbers such as " first ", " second ", " third " or " the 4th ", unless based on context its
The meaning of certain order of representation, it is appreciated that being only to distinguish to be used.
The solution of the present invention will be described in detail by specific embodiment below, certainly, the present invention is not limited to
Lower embodiment.
As shown in fig.1, in the embodiment of the present invention, a kind of detailed process of the statistical method of mass data is as follows：
Step 100：The data statistic of different time granularity is set, wherein there is packet between above-mentioned different time granularity
Contain or by inclusion relation, the data statistic of above-mentioned different time granularity is respectively used to store the original got in the corresponding period
Beginning data.
In practical application, statistical module is initialized, according to the query demand of user, different time granularity is set, and is directed to
Corresponding data statistic is respectively set in each time granularity.In this way, can will be obtained in any time granularity corresponding period
To initial data count in the corresponding data statistic of any of the above-described time granularity.
In the embodiment of the present invention, a kind of preferable real-time mode is, the different time granularity of setting includes minute rank, small
When rank, day rank, moon rank.Illustratively, refering to shown in table 1, between the corresponding data statistic of different time granularity
Using father's sub-table structure.Corresponding data statistic is set for the time granularity of minute rank, for example, every 10 minutes are divided into
The time granularity of one minute rank, e.g., when 17 days 14 March in 2017：00 point：When -2017 years 00 second 17 days 14 March：10 points：
00 second correspondence, one data statistic；Corresponding data statistic is set for the time granularity of hour rank, for example, every 1 is small
When be divided into the time granularity of a hour rank, e.g., when 17 days 14 March in 2017：00 point：- 2017 years 00 second 17 days 15 March
When：00 point：00 second correspondence, one data statistic, it is clear that seen from the above description, the time granularity of a hour rank is opposite
The data statistic answered includes the corresponding data statistic of time granularity of 6 minute ranks；For the time of day rank
Corresponding data statistic is arranged in granularity, for example, every 1 day time granularity for being divided into a day rank, e.g., March 17 in 2017
When day 00：00 point：When -2017 years 00 second 18 days 00 March：00 point：00 second correspondence, one data statistic, it is clear that retouched by above-mentioned
It states it is found that including the time granularity phase of 24 hour ranks in the corresponding data statistic of the time granularity of a day rank
Corresponding data statistic；Corresponding data statistic is set for the time granularity of moon rank, for example, every 1 month is divided into
The time granularity of one month rank, e.g., when 1 day 00 March in 2017：00 point：When -2017 years 00 second 1 day 00 April：00 point：00 second
A corresponding data statistic, it is clear that seen from the above description, the corresponding data statistic of time granularity of a month rank
In include N number of day rank the corresponding data statistic of time granularity, N is of that month total number of days.
Table 1
Further, the history initial data not counted is judged whether；When it is to be to judge result, phase is set
The historical statistics task answered, and above-mentioned statistics task state table is added in historical statistics task, and it is based on above-mentioned statistics task shape
State table is counted on above-mentioned history initial data in the data statistic of corresponding time granularity by the way of non-obstruction.Specifically
, when corresponding historical statistics task is arranged, specifically include：Based on above-mentioned historical statistical data earliest time stamp information and
Latest time stabs information, and the corresponding historical statistics task of above-mentioned different time granularity is arranged, wherein above-mentioned historical statistics is appointed
Business is resident statistics task.
In the embodiment of the present invention, in statistical module initialization procedure, need judge whether not counted to go through
History initial data, and if it exists, then statistics task state can be written into historical statistics task according to the different time granularity of setting
In table, and statistical disposition is carried out to history initial data according to historical statistics task by historical statistics module, wherein statistics task
The task names and task status that all historical statistics tasks are included at least in state table, set respectively for each historical statistics task
Corresponding timing statistics are equipped with, above-mentioned task status is for characterizing whether historical statistics task is completed in corresponding timing statistics.
Certainly, in the embodiment of the present invention, the specific set-up mode of different time granularity can also be according to different user not
It is different with actual demand and/or the different actual demands of different application scene, in the embodiment of the present invention, do not do specific limit herein
It is fixed.
For example, for the time granularity of minute rank, could be provided as being used as a time granularity in every 10 minutes, it can also
It is set as every 5 minutes or every 20 minutes time granularities as a minute rank.
Step 110：Based on preset statistics task state table, the initial data got is counted on into corresponding time granularity
Data statistic in, wherein all time granularities point in the current statistic period are included at least in above-mentioned statistics task state table
The task names and task status of not corresponding statistics task are respectively arranged with corresponding timing statistics for each statistics task,
Above-mentioned task status is for characterizing whether statistics task is completed in corresponding timing statistics.
Specifically, in the embodiment of the present invention, when executing step 110, object statistics are obtained from statistics task state table
Task, and above-mentioned object statistics task is added in statistics task queue；Based on the above-mentioned corresponding statistics of object statistics task
Time started completes the code function of above-mentioned object statistics required by task, by the corresponding time grain of above-mentioned object statistics task
The initial data got in degree carries out statistics merger, and counts on the above-mentioned corresponding data statistic of object statistics task
In；If it is determined that complete above-mentioned object statistics task within the above-mentioned object statistics task corresponding object statistics time, then it will be upper
The task status for stating object statistics task is changed to be completed, and otherwise, the task status of above-mentioned object statistics task is changed to
Statistics is abnormal.
Further, in the embodiment of the present invention, by resident task manager, timing task management device and statistics task queue
Statistical framework is formed, above-mentioned statistical framework is used for the above-mentioned statistics task state table of maintenance and management and above-mentioned statistics task state table
In each statistics task；The statistics task administrative mechanism of above-mentioned statistical framework includes Resident Process management and timing management of process,
And it is executed respectively by Resident Process manager and timing process manager；The statistics that running frequency is more than or equal to first frequency is appointed
Business is set as resident statistics task, by above-mentioned Resident Process manager administration and starts statistics task queue；And by running frequency
Statistics task less than or equal to second frequency is set as timing statistics task, by the timing of above-mentioned timing process manager start by set date
Statistics task, and timing statistics task is added in statistics task queue, wherein above-mentioned first frequency is more than above-mentioned second frequency
Rate.
As shown in fig.2, in the embodiment of the present invention, the Governance framework of real-time statistics task and historical statistics task is illustrated.
The execution of more statistics tasks can be responsible for scheduling by message task queue and complete (some multithreading task queues e.g., can be used), after
The management that platform resides statistics process is responsible for (e.g., process manager can be realized by shell scripts) by Resident Process manager, is responsible for
Management and startup statistics task queue, to complete to reside the statistics task of completion needed for statistics process, timed task process
Management can be responsible for (the timed task frame carried in programming language e.g., can be used or linux is included by timing process manager
Crontab realize), be responsible for start by set date timing statistics task, and timing statistics task is added in statistics task queue, from
And the statistics task completed needed for completion timing statistics task.
Specifically, in the embodiment of the present invention, in the statistics of same time granularity, since the multiple statistics of setting may be needed
Task is in from statistical data in multiple raw data tables to multiple data statistics of different business scene, then, once may
There are multiple statistics tasks, in the embodiment of the present invention, in order to reduce required computing resource, the reality of minimum time granularity can be directed to
When statistics task and historical statistics task by the way of the Resident Process of backstage, and for other times granularity real-time statistics appoint
Business uses timed task.
Illustratively, refering to shown in table 2, in the embodiment of the present invention, if the different time granularity of setting include minute, it is small
When, day, the moon, then, you can minute corresponding real-time statistics task is set to resident statistics process, by hour, day, the phases of the moon
Corresponding real-time statistics task is set as timing statistics task.Certainly, in the embodiment of the present invention, exist in judgement and do not counted
History initial data, when needing to count history initial data, then, you can go through all time granularities are corresponding
History statistics task is set as resident statistics process.In practical application, current statistic task may be caused to exist due to data jamming
It cannot complete to count in corresponding timing statistics, then, it is anomaly statistics by current statistic task flagging when determining statistics time-out
Task directly skips current statistic task, carries out the statistics of next statistics task, in order to avoid the appearance of statistical delay, and for not
The anomaly statistics task that statistics is completed in corresponding timing statistics is united again by the exception processing module of start by set date is unified
Meter processing.
Table 2
In the embodiment of the present invention, the different time granularity based on setting is preset with corresponding statistics task state table, can use
In task status, task names, job start time, the task of all statistics tasks in storage designated time period (such as 1 day)
The information such as end time, field structure is refering to shown in table 3.
Table 3
For example, being that every 10 minutes statistics are primary with the time granularity of minute rank, the time granularity of hour rank is every 1 small
Shi Tongji is primary, and the time granularity of day rank is for every 1 day statistics is primary, can be advance before the statistics at one day starts
The statistics task state table on the same day is configured, is included at least in the statistics task state table：6 × 24=144 minute rank
The corresponding statistics task of time granularity, the corresponding statistics task of time granularity of 1 × 24 hour rank, 1 day rank
The corresponding statistics task of time granularity.
In practical application, statistical module initialization after, by historical statistics task (if there are history initial data) and after
It is continuous that the real-time statistics task executed is needed to be written in corresponding statistics task state table, and statistics task state table is inserted into subsequently
Statistics task information in, e.g., can be at daily 23:00 point：00 second system that second day is inserted into statistics task information
Count task status table.Certainly, after statistical module is restarted, it is also desirable to judge whether the statistics task state table on the same day has been inserted
Enter.
Further, in the embodiment of the present invention, exception processing module handles anomaly statistics task in timing same
When, the statistics task information deletion of the real-time statistics task and historical statistics task of statistics can also will be completed.
In the embodiment of the present invention, if it is determined that in the presence of the history initial data not counted, then need historical statistics module to going through
History initial data is counted.In the embodiment of the present invention, a kind of preferable specific implementation mode is that historical statistics module is according to system
The historical statistics task in task list is counted, the statistics task of big time granularity corresponds to the statistics of the small time granularity in period at it
Start to execute after the completion of task statistics, if any historical statistics task does not complete corresponding statistics (i.e. in corresponding timing statistics
Exception history statistics task), then the task status of any historical statistics task is not changed, or be revised as not completing, directly
Execute next historical statistics task；If any historical statistics task is completed to count accordingly in corresponding timing statistics, will
It is completed after the task status modification of any historical statistics task.
In the embodiment of the present invention, if it is determined that there is no the history initial data not counted, then it is not necessarily to that corresponding history is arranged
Statistics task, only need to real-time reception to initial data count, i.e., in statistics task state table only include real-time statistics
The real-time statistics task that module executes.In the embodiment of the present invention, a kind of preferable specific implementation mode was, by one day all reality
When statistics task write-in statistics task state table in, and by the one day statistics task information (resident statistics process and/or timing
Statistics task) it updates into Resident Process manager and/or timing task management device, start to execute real-time statistics task, will work as
The initial data got in the preceding time granularity corresponding period counts in corresponding data statistic, if some is in real time
Statistics task does not complete in corresponding timing statistics, then it is anomaly statistics task to mark a certain task, and is directly executed next
Real-time statistics task, if some real-time statistics task is completed in corresponding timing statistics, by a certain real-time statistics task
Task status is revised as being completed, after real-time statistics task is completed on the day of, using exception processing module to the exception on the same day
Statistics task carries out statistical disposition again.Certainly, the startup period of exception processing module can be real according to the difference of different user
The different actual demands of border demand and/or different application scene and it is different, in the embodiment of the present invention, be not specifically limited herein.
Illustratively, refering to shown in table 4, in the embodiment of the present invention, statistics of the real-time statistics module to each time granularity
The scheduling mode of task process.
Table 3
Certainly, the scheduling mode of the statistics task process of each time granularity can be according to the different actual demands of user
And/or the different actual demands of different application scene and it is different, in the embodiment of the present invention, be not specifically limited herein.
Step 120：It detects not completing statistics in statistic processes in corresponding timing statistics based on preset detection cycle
Anomaly statistics task, and for above-mentioned anomaly statistics task carry out statistical disposition.
In practical application, since the present invention appoints the real-time statistics in statistics task state table using non-blocking fashion
Business and/or historical statistics task are counted, and each statistics task is provided with corresponding timing statistics, if any statistics is appointed
Business does not count completion in corresponding timing statistics, then will be labeled as anomaly statistics task, then, in the embodiment of the present invention, i.e.,
The anomaly statistics mission bit stream occurred in statistic processes can be detected according to pre-set detection cycle, and to above-mentioned exception
Statistics task carries out statistical disposition again.
Specifically, for real-time statistics task, exception can be passed through after one day statistics task is fully completed
Reason module carries out statistical disposition again to the anomaly statistics task detected；It, can also be and for historical statistics task
After historical statistics task is fully completed, united again to the exception history statistics task detected using exception processing module
Meter processing.While ensuring statistical data real-time, the accuracy of statistical data ensure that.
Further, in the data query instruction for receiving user's triggering, based on data query instruction is received, from
The target data statistical form of each time granularity corresponding with query time is determined in the data statistic of completion statistics；From upper
The statistical data for determining to meet querying condition in target data statistical form is stated, and returns to query result.
Such as：There are one data statistic (e.g., s_attack) based on attack source IP, polling cycle are 2017 3
On the moon 18 16:30:00 to 2017 on March 25,16:30:00, due to 18 days March in 2017 for needing to inquire and in March, 2017
25 days data non-all day, be also the data of non-whole hour, and the data on March 24th, 18 days 1 March in 2017 are
The data of all day, then, it needs to inquire corresponding data from following several periods corresponding data statistic：2017
Minute table (the s_attack_min_2017.03.18_16 on March 18:30:00~2017.3.19_00:00:00), 2017 3
Day table (the s_attack_day_2017.03.19_00 on March 24th, 19 days 1 moon:00:00~s_attack_day_
2017.03.24_00:00:And the minute table (s_attack_min_2017.3.25_00 on March 25th, 2,017 00):00:00~
2017.3.25_16:30:00)。
Above-described embodiment is described in further detail using specific application scenarios below, as shown in fig.3, of the invention
In embodiment, a kind of statistical framework signal of mass data, for initial configuration for initializing statistics merger information, setting is different
The data statistic of time granularity (e.g., minute, hour, day, the moon), respectively by the system of historical statistics module and real-time statistics module
Meter task is written in statistics task state table；Judgement exist do not count history initial data when, historical statistics module from
Historical statistics task is obtained in statistics task state table, by historical statistics task and process scheduling frame, using non-obstruction side
History initial data statistics is integrated into corresponding data statistic by formula, meanwhile, real-time statistics module is from statistics task state
Real-time statistics task corresponding with current time is obtained in table, by real-time statistics task and process scheduling frame, use is non-
The initial data got in real time statistics is integrated into corresponding data statistic by obstruction mode；Based on preset abnormality processing
Rule, after corresponding measurement period task terminates, the detection statistics period, the interior statistics task in timing statistics did not completed,
Task status carries out Statistics Division again by exception processing module labeled as abnormal statistics task to the anomaly statistics task
Reason, at the same time it can also remove in measurement period, statistics task is completed in timing statistics, and task status is labeled as having counted
At statistics task mission bit stream；When receiving Web page inquiry instruction, the query time indicated by inquiry instruction
Span selects the statistical data of suitable particle size from data statistic, returns to Web page query result.
Based on above-described embodiment, as shown in fig.4, in the embodiment of the present invention, a kind of statistic device of mass data, at least
Including statistic unit 40 and exception processing unit 41, wherein
Statistic unit 40 counts on the initial data got accordingly for being based on preset statistics task state table
In the data statistic of time granularity, wherein exist between different time granularity comprising or by inclusion relation, above-mentioned different time
The data statistic of granularity is respectively used to store the initial data got in the corresponding period, in above-mentioned statistics task state table
Including at least the task names and task status of the corresponding statistics task of all time granularities in the current statistic period, for
Each statistics task is respectively arranged with corresponding timing statistics, and above-mentioned task status is for characterizing whether statistics task is accordingly counting
It is completed in time；
Exception processing unit 41 detects in statistic processes for being based on preset detection cycle in corresponding timing statistics
The anomaly statistics task of statistics is not completed, and statistical disposition is carried out for above-mentioned anomaly statistics task.
Optionally, further comprise：The corresponding statistics task of minimum time granularity in the different time granularity is set
It is set to resident statistics task, by Resident Process manager administration and starts statistics task queue；And by the different time granularity
In the corresponding statistics task of other times granularity in addition to the minimum time granularity be set as timing statistics task, by fixed
When process manager start by set date timing statistics task, and will timing statistics task be added statistics task queue in.
Optionally, above-mentioned different time granularity includes minute, hour, day, the moon；Above-mentioned minute corresponding statistics task is
Resident statistics process, above-mentioned hour, day, the moon corresponding statistics task are timing statistics task.
Optionally, based on preset statistics task state table, the initial data got is counted on into corresponding time grain
When in the data statistic of degree, above-mentioned statistic unit 40 is used for：Object statistics task is obtained from statistics task state table, and will
Above-mentioned object statistics task is added in statistics task queue；Based on the above-mentioned object statistics task corresponding statistics time started,
The code function for completing above-mentioned object statistics required by task will be got in the above-mentioned corresponding time granularity of object statistics task
Initial data carry out statistics merger, and count in the above-mentioned corresponding data statistic of object statistics task；If it is determined that
Above-mentioned object statistics task is completed in the above-mentioned object statistics task corresponding object statistics time, then is appointed above-mentioned object statistics
The task status of business is changed to be completed, and otherwise, it is abnormal that the task status of above-mentioned object statistics task is changed to statistics.
Optionally, above-mentioned statistic device further comprises historical data processing unit：For judging whether not carry out
The history initial data of statistics；When it is to be to judge result, corresponding historical statistics task is set, and historical statistics task is added
Enter above-mentioned statistics task state table, and be based on above-mentioned statistics task state table, when above-mentioned history initial data is counted on corresponding
Between granularity data statistic in.
Optionally, when corresponding historical statistics task is arranged, above-mentioned historical data processing unit is used for：It is gone through based on above-mentioned
The earliest time stamp information and latest time of history statistical data stab information, and the corresponding history of above-mentioned different time granularity is arranged
Statistics task, wherein above-mentioned historical statistics task is resident statistics task.
Optionally, above-mentioned exception processing unit 41 is further used for：After the completion of determining above-mentioned historical statistics task, detection
Go out in above-mentioned history initial data statistic processes not completing the exception history statistics task of statistics, and needle in corresponding timing statistics
Statistical disposition is carried out to above-mentioned exception history statistics task.
Optionally, above-mentioned statistic device further comprises data query unit：For being based on receiving data query instruction,
The target data statistical form of each time granularity corresponding with query time is determined from the data statistic for be completed statistics；
The statistical data for meeting querying condition is determined from above-mentioned target data statistical form, and returns to query result.
In conclusion in the embodiment of the present invention, during being counted to mass data, appointed based on preset statistics
Business state table, the initial data got is counted in the data statistic of corresponding time granularity, wherein different time granularity
Between exist and be respectively used to store in the corresponding period comprising or by inclusion relation, the data statistic of above-mentioned different time granularity
It is right respectively to include at least all time granularities in the current statistic period in above-mentioned statistics task state table for the initial data got
The task names and task status for the statistics task answered are respectively arranged with corresponding timing statistics for each statistics task, above-mentioned
Task status is for characterizing whether statistics task is completed in corresponding timing statistics；And it is detected based on preset detection cycle
The anomaly statistics task of statistics is not completed in statistic processes in corresponding timing statistics, and is carried out for above-mentioned anomaly statistics task
Statistical disposition.
Using the above method, by the way that statistics task state table is arranged, corresponding timing statistics are set for each task, it is right
The initial data got carries out statistical disposition, and all statistics tasks can judge whether to have held according to statistics task state table
Row, can more accurately, easily management statistics task, and make each statistics task on time whether complete will not shadow for statistics
The execution to next statistics task is rung, the accumulation of statistical delay is avoided due to data jamming and cause, and then ensures system
The real-time counted, further, provided in the statistical method of mass data provided by the invention in statistic processes
The scheme that the statistics task not completed in corresponding timing statistics is counted again, it is ensured that the accuracy of statistical data.In addition,
Statistical framework deployment provided by the invention is simple, and process manager, the realization of task queue are flexible.
It should be understood by those skilled in the art that, the embodiment of the present invention can be provided as method, system or computer program
Product.Therefore, complete hardware embodiment, complete software embodiment or reality combining software and hardware aspects can be used in the present invention
Apply the form of example.Moreover, the present invention can be used in one or more wherein include computer usable program code computer
The computer program production implemented in usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.)
The form of product.
The present invention be with reference to according to the method for the embodiment of the present invention, the flow of equipment (system) and computer program product
Figure and/or block diagram describe.It should be understood that can be realized by computer program instructions every first-class in flowchart and/or the block diagram
The combination of flow and/or box in journey and/or box and flowchart and/or the block diagram.These computer programs can be provided
Instruct the processor of all-purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce
A raw machine so that the instruction executed by computer or the processor of other programmable data processing devices is generated for real
The device for the function of being specified in present one flow of flow chart or one box of multiple flows and/or block diagram or multiple boxes.
These computer program instructions, which may also be stored in, can guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works so that instruction generation stored in the computer readable memory includes referring to
Enable the manufacture of device, the command device realize in one flow of flow chart or multiple flows and/or one box of block diagram or
The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device so that count
Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, in computer or
The instruction executed on other programmable devices is provided for realizing in one flow of flow chart or multiple flows and/or block diagram one
The step of function of being specified in a box or multiple boxes.
Although preferred embodiments of the present invention have been described, it is created once a person skilled in the art knows basic
Property concept, then additional changes and modifications may be made to these embodiments.So it includes excellent that the following claims are intended to be interpreted as
It selects embodiment and falls into all change and modification of the scope of the invention.
Obviously, those skilled in the art can carry out the embodiment of the present invention various modification and variations without departing from this hair
The spirit and scope of bright embodiment.In this way, if these modifications and variations of the embodiment of the present invention belong to the claims in the present invention
And its within the scope of equivalent technologies, then the present invention is also intended to include these modifications and variations.
Claims (11)
1. a kind of statistical method of mass data, which is characterized in that including：
Based on preset statistics task state table, the initial data got is counted on to the data statistic of corresponding time granularity
In, wherein exist between different time granularity comprising or by inclusion relation, the data statistic difference of the different time granularity
For storing the initial data got in the corresponding period, the current statistic period is included at least in the statistics task state table
The task names and task status of the corresponding statistics task of interior all time granularities, are respectively arranged with for each statistics task
Corresponding timing statistics, the task status is for characterizing whether statistics task is completed in corresponding timing statistics；
Detect that the anomaly statistics for not completing statistics in statistic processes in corresponding timing statistics are appointed based on preset detection cycle
Business, and carry out statistical disposition for the anomaly statistics task.
2. the method as described in claim 1, which is characterized in that further comprise：
Statistical framework is formed by resident task manager, timing task management device and statistics task queue, the statistical framework is used
Each statistics task in the statistics task state table described in maintenance and management and the statistics task state table；
The statistics task administrative mechanism of the statistical framework includes Resident Process management and timing management of process, and respectively by residing
Process manager and timing process manager execute；
The statistics task that running frequency is more than or equal to first frequency is set as resident statistics task, is managed by the Resident Process
Device management and startup statistics task queue；And the statistics task that running frequency is less than or equal to second frequency is set as timing and counts
Statistics task is added by the timing process manager start by set date timing statistics task, and by timing statistics task in task
In queue, wherein the first frequency is more than the second frequency.
3. method as claimed in claim 2, which is characterized in that the timed task is appointed by the statistics corresponding to big time granularity
Business and other timed tasks composition；
The resident task by corresponding to small time granularity statistics task and the very high other tasks of running frequency form.
4. method as described in any one of claims 1-3, which is characterized in that be based on preset statistics task state table, will obtain
The initial data got counts in the data statistic of corresponding time granularity, including：
Object statistics task is obtained from statistics task state table, and statistics task queue is added in the object statistics task
In；
Based on the object statistics task corresponding statistics time started, the code letter of the object statistics required by task is completed
Number, carries out statistics merger, and count on by the initial data got in the corresponding time granularity of object statistics task
In the corresponding data statistic of object statistics task；
If it is determined that completing the object statistics task within the object statistics task corresponding object statistics time, then by institute
The task status for stating object statistics task is changed to be completed, and otherwise, the task status of the object statistics task is changed to
Statistics is abnormal.
5. the method as described in claim 1, which is characterized in that further comprise：
Judge whether the history initial data not counted；
When it is to be to judge result, corresponding historical statistics task is set, and the statistics task is added in historical statistics task
State table, and it is based on the statistics task state table, the data that the history initial data is counted on to corresponding time granularity are united
It counts in table.
6. method as claimed in claim 5, which is characterized in that corresponding historical statistics task is set, including：
Earliest time stamp information and latest time based on the historical statistical data stab information, and the different time granularity is arranged
Corresponding historical statistics task, wherein the historical statistics task is resident statistics task.
7. method as claimed in claim 6, which is characterized in that further comprise：
After the completion of determining the historical statistics task, detect in the history initial data statistic processes in corresponding statistics
The exception history statistics task of interior unfinished statistics, and carry out statistical disposition for the exception history statistics task.
8. the method as described in claim 1, which is characterized in that further comprise：
Based on data query instruction is received, determined from the data statistic for be completed statistics corresponding with query time
The target data statistical form of each time granularity；
The statistical data for meeting querying condition is determined from the target data statistical form, and returns to query result.
9. a kind of statistic device of mass data, which is characterized in that including：
The initial data got is counted on corresponding time grain by storage unit for being based on preset statistics task state table
In the data statistic of degree, wherein exist between different time granularity comprising or by inclusion relation, the different time granularity
Data statistic is respectively used to store the initial data got in the corresponding period, is at least wrapped in the statistics task state table
The task names and task status for including the corresponding statistics task of all time granularities in the current statistic period, for each statistics
Task is respectively arranged with corresponding timing statistics, and the task status is for characterizing statistics task whether in corresponding timing statistics
It completes；
Exception processing unit detects in statistic processes not completing in corresponding timing statistics for being based on preset detection cycle
The anomaly statistics task of statistics, and carry out statistical disposition for the anomaly statistics task.
10. a kind of computing device, which is characterized in that including：
Memory, for storing program instruction；
Processor, for calling the program instruction stored in the memory, according to acquisition program execute as claim 1 to
8 any one of them methods.
11. a kind of computer storage media, which is characterized in that the computer-readable recording medium storage has computer executable
Instruction, the computer executable instructions are for making the computer execute such as claim 1 to 8 any one of them method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810528191.7A CN108733839A (en) | 2018-05-25 | 2018-05-25 | A kind of statistical method and device of mass data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810528191.7A CN108733839A (en) | 2018-05-25 | 2018-05-25 | A kind of statistical method and device of mass data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108733839A true CN108733839A (en) | 2018-11-02 |
Family
ID=63935667
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810528191.7A Pending CN108733839A (en) | 2018-05-25 | 2018-05-25 | A kind of statistical method and device of mass data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108733839A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110515989A (en) * | 2019-09-02 | 2019-11-29 | 四川长虹电器股份有限公司 | A kind of data real-time statistical method based on financial data management platform |
| CN111625541A (en) * | 2020-05-11 | 2020-09-04 | 紫光云技术有限公司 | Method for measuring IP (Internet protocol) of elastic public network according to flow |
| CN113268487A (en) * | 2021-06-16 | 2021-08-17 | 中移（杭州）信息技术有限公司 | Data statistical method, device and computer readable storage medium |
| CN113468246A (en) * | 2021-07-20 | 2021-10-01 | 上海齐屹信息科技有限公司 | Intelligent data counting and subscribing system and method based on OLTP |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1747398A (en) * | 2004-09-08 | 2006-03-15 | 大唐移动通信设备有限公司 | Mass performance data statistical method in network element management system |
| CN1897615A (en) * | 2005-07-15 | 2007-01-17 | 中兴通讯股份有限公司 | Method for realizing customer definition statistical function |
| US20170228661A1 (en) * | 2014-04-17 | 2017-08-10 | Sas Institute Inc. | Systems and methods for machine learning using classifying, clustering, and grouping time series data |
| CN107066519A (en) * | 2017-02-13 | 2017-08-18 | 阿里巴巴集团控股有限公司 | A kind of task detection method and device |
| CN107402806A (en) * | 2017-04-20 | 2017-11-28 | 阿里巴巴集团控股有限公司 | The task processing method and device of distributed document framework |
| CN107562797A (en) * | 2017-08-02 | 2018-01-09 | 贵州工程应用技术学院 | A kind of universal intelligent design method based on data target statistics |
-
2018
- 2018-05-25 CN CN201810528191.7A patent/CN108733839A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1747398A (en) * | 2004-09-08 | 2006-03-15 | 大唐移动通信设备有限公司 | Mass performance data statistical method in network element management system |
| CN1897615A (en) * | 2005-07-15 | 2007-01-17 | 中兴通讯股份有限公司 | Method for realizing customer definition statistical function |
| US20170228661A1 (en) * | 2014-04-17 | 2017-08-10 | Sas Institute Inc. | Systems and methods for machine learning using classifying, clustering, and grouping time series data |
| CN107066519A (en) * | 2017-02-13 | 2017-08-18 | 阿里巴巴集团控股有限公司 | A kind of task detection method and device |
| CN107402806A (en) * | 2017-04-20 | 2017-11-28 | 阿里巴巴集团控股有限公司 | The task processing method and device of distributed document framework |
| CN107562797A (en) * | 2017-08-02 | 2018-01-09 | 贵州工程应用技术学院 | A kind of universal intelligent design method based on data target statistics |
Non-Patent Citations (4)
| Title |
|---|
| WENCHUAN YANG等: "A visual DSS prototype in statistic data warehouse", 《IEEE》 * |
| 于春凡: "《IBM-PC INTEL 8088/80X86 宏汇编语言程序设计》", 30 November 1998 * |
| 曾兆棋: "《中国计算机应用文集 DJS100系列机软件专辑》", 31 December 1983 * |
| 薛会男: "电力营销数据可视化统计分析方法及系统", 《科技咨询》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110515989A (en) * | 2019-09-02 | 2019-11-29 | 四川长虹电器股份有限公司 | A kind of data real-time statistical method based on financial data management platform |
| CN110515989B (en) * | 2019-09-02 | 2022-03-01 | 四川长虹电器股份有限公司 | Data real-time statistical method based on financial data management platform |
| CN111625541A (en) * | 2020-05-11 | 2020-09-04 | 紫光云技术有限公司 | Method for measuring IP (Internet protocol) of elastic public network according to flow |
| CN113268487A (en) * | 2021-06-16 | 2021-08-17 | 中移（杭州）信息技术有限公司 | Data statistical method, device and computer readable storage medium |
| CN113268487B (en) * | 2021-06-16 | 2023-04-11 | 中移（杭州）信息技术有限公司 | Data statistical method, device and computer readable storage medium |
| CN113468246A (en) * | 2021-07-20 | 2021-10-01 | 上海齐屹信息科技有限公司 | Intelligent data counting and subscribing system and method based on OLTP |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108733839A (en) | A kind of statistical method and device of mass data | |
| TWI742045B (en) | Task resource scheduling method and device | |
| CN112162865B (en) | Scheduling method and device of server and server | |
| CN108776934B (en) | Distributed data calculation method and device, computer equipment and readable storage medium | |
| US9658910B2 (en) | Systems and methods for spatially displaced correlation for detecting value ranges of transient correlation in machine data of enterprise systems | |
| US7453910B1 (en) | Synchronization of independent clocks | |
| CN104809201B (en) | A kind of method and apparatus of database synchronization | |
| CN104601696B (en) | Service processing method, service calling system, device and system | |
| US9235440B2 (en) | Managing job execution | |
| US20170180220A1 (en) | Techniques to Generate Workload Performance Fingerprints for Cloud Infrastructure Elements | |
| US11005700B2 (en) | System and method for quickly determining reasonable alarm threshold of network | |
| WO2015139164A1 (en) | Task scheduling method, apparatus and device | |
| Soualhia et al. | A dynamic and failure-aware task scheduling framework for hadoop | |
| US10009237B1 (en) | Cross silo time stiching | |
| CN109146381A (en) | Logistics data monitoring method, device, electronic equipment and computer storage medium | |
| CN109271447A (en) | Method of data synchronization, device, computer equipment and storage medium | |
| US9716638B1 (en) | Push pull data collection | |
| CN106484503B (en) | A kind of calculation method and network processing unit verifying sum | |
| CN110347407A (en) | A kind of method, apparatus, computer equipment and medium obtaining EMS memory occupation amount | |
| CN110324211A (en) | A kind of data capture method and device | |
| CN116414914A (en) | Data synchronization method and device, processor and electronic equipment | |
| CN110689429B (en) | Method, device and system for storing certificate transaction data | |
| von der Brüggen | Realistic Scheduling Models and Analyses for Advanced Real-Time Embedded Systems | |
| Ivers et al. | Probabilistic network loads with dependencies and the effect on queue sojourn times | |
| CN109165238A (en) | It is a kind of for generating the data processing method and device of cyclical indicator data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181102 |