CN104199942A - Hadoop platform time series data incremental computation method and system - Google Patents
Hadoop platform time series data incremental computation method and system Download PDFInfo
- Publication number
- CN104199942A CN104199942A CN201410456262.9A CN201410456262A CN104199942A CN 104199942 A CN104199942 A CN 104199942A CN 201410456262 A CN201410456262 A CN 201410456262A CN 104199942 A CN104199942 A CN 104199942A
- Authority
- CN
- China
- Prior art keywords
- time series
- series data
- computing
- sub
- historical
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 19
- 238000004364 calculation method Methods 0.000 claims description 45
- 238000004064 recycling Methods 0.000 claims description 8
- 230000011218 segmentation Effects 0.000 claims description 8
- 230000003252 repetitive effect Effects 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 10
- 238000004422 calculation algorithm Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 3
- 238000013467 fragmentation Methods 0.000 description 2
- 238000006062 fragmentation reaction Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000000714 time series forecasting Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a Hadoop platform time series data incremental computation method and system. The method includes the steps that when a time series data incremental computation task is started, the historical computational state of time series data is obtained from a cache server; incremental computation is carried out by means of a segmented time series data incremental computation method containing a SubCp sub-operation and a ReduceCp sub-operation according to the historical computational state, wherein the SubCp sub-operation is used for self-definition of segmented time series data and storage of intermediate results, the ReduceCp sub-operation is carried out in an operation merging stage and used for merging operation on computed results of the segmented time series data according to the self-defined operation, and the computational state of the SubCp sub-operation and the computational state of the ReduceCp sub-operation are maintained through the cache server. By the adoption of the method and system, plenty of unnecessary repetitive computation can be saved through incremental computation, and therefore data processing efficiency is improved.
Description
Technical field
The present invention relates to field of computer technology, relate in particular to a kind of Hadoop platform time series data incremental calculation method.
Background technology
Along with the develop rapidly of current Internet technology, the widespread use of information acquiring technology etc. produces and has accumulated the various data that exist with time series form of magnanimity in many science-cum-industry fields such as telecommunications, meteorology, geology, electric power, finance.Traditional time Series Processing method is generally to select the relevant mathematics computational tools such as Matlab to carry out, but in the time that the problem scale of processing becomes large, problem often allows people insufferable computing time.
Current, along with large data processing is taken seriously gradually, some companies, research institution have also started the research of this respect, and related work mainly concentrates on Hadoop and increases income on Distributed Computing Platform.Hadoop is as a Distributed Architecture, can distributed operation mass data, there are a lot of advantages processing in mass data, such as thering is the features such as high fault tolerance, high scalability, high reliability.
At present, to time series data, processing does not provide good support to Hadoop platform, and fewer to the incremental computations correlative study of time series data, needs double counting while causing time series data newly-increased, thereby reduces the efficiency of data processing.
Summary of the invention
The object of this invention is to provide a kind of Hadoop platform time series data incremental calculation method and system, can save a large amount of unnecessary double countings by incremental computations, thereby improved the efficiency of data processing.
The object of the invention is to be achieved through the following technical solutions:
A kind of Hadoop platform time series data incremental calculation method, the method comprises:
In the time starting time series data incremental computations task, from caching server, obtain the historical computing mode of this time series data;
The subsection timing sequence data increment computing method that comprise SubCp and the sub-computing of ReduceCP according to described historical computing mode utilization are carried out incremental computations;
Wherein, the sub-computing of SubCp is for carrying out self-defining sub-computing and intermediate result is preserved subsection timing sequence data respectively; The sub-computing of ReduceCP is computing merging phase, the result of calculation merger operation according to self-defining operation to subsection timing sequence data, and the computing mode of described SubCp and the sub-computing of ReduceCP is safeguarded by caching server.
A kind of Hadoop platform time series data incremental computations system, this system comprises:
Time series data incremental processing module TSI in the time starting time series data incremental computations task, obtains the historical computing mode of this time series data from caching server; The subsection timing sequence data increment computing method that comprise SubCp and the sub-computing of ReduceCP according to described historical computing mode utilization are carried out incremental computations; Wherein, the sub-computing of SubCp is for carrying out self-defining sub-computing and intermediate result is preserved subsection timing sequence data respectively; The sub-computing of ReduceCP is computing merging phase, the result of calculation merger operation according to self-defining operation to subsection timing sequence data, and the computing mode of described SubCp and the sub-computing of ReduceCP is safeguarded by caching server;
Caching server, for preserving the historical computing mode of time series data.
As seen from the above technical solution provided by the invention, by the historical computing mode of caching server buffer memory time series data, in the time starting incremental computations, according to the historical computing mode getting, directly carry out the calculating of incremental data, multiplexing historical result of calculation fast again, has avoided unnecessary double counting, thereby has improved the efficiency of data processing.
Brief description of the drawings
In order to be illustrated more clearly in the technical scheme of the embodiment of the present invention, below the accompanying drawing of required use during embodiment is described is briefly described, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skill in the art, do not paying under the prerequisite of creative work, can also obtain other accompanying drawings according to these accompanying drawings.
The process flow diagram of a kind of Hadoop platform time series data incremental calculation method that Fig. 1 provides for the embodiment of the present invention one;
The schematic diagram of a kind of time series data fragmentation scheme that Fig. 2 provides for the embodiment of the present invention one;
The schematic diagram of a kind of subsection timing sequence data increment computing method that Fig. 3 provides for the embodiment of the present invention one;
The schematic diagram of the moving window incremental calculation method of a kind of stationary window width with state that Fig. 4 provides for the embodiment of the present invention one;
The schematic diagram of a kind of incremental calculation method with the fixing monotone increasing window of the starting point of state that Fig. 5 provides for the embodiment of the present invention one;
The schematic diagram of a kind of Hadoop platform time series data incremental computations system that Fig. 6 provides for the embodiment of the present invention two;
The schematic diagram that the existing Hadoop platform that Fig. 7 provides for the embodiment of the present invention two and incremental computations system are mutually integrated.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is clearly and completely described, obviously, described embodiment is only the present invention's part embodiment, instead of whole embodiment.Based on embodiments of the invention, those of ordinary skill in the art, not making the every other embodiment obtaining under creative work prerequisite, belong to protection scope of the present invention.
Embodiment mono-
The process flow diagram of a kind of Hadoop platform time series data incremental calculation method that Fig. 1 provides for the embodiment of the present invention one.As shown in Figure 1, the method mainly comprises the steps:
Step 11, in the time starting time series data incremental computations task, from caching server, obtain the historical computing mode of this time series data.
Wherein, described time series data, is divided into multiple segmentations continuous time series data taking section sometime as unit, and the time series data computing in each unit interval section is a sub-computing; And time series data after segmentation need to meet the one semigroup character.
Described time series data incremental computations task indicates newly-increased subsection timing sequence data.
Step 12, the subsection timing sequence data increment computing method that comprise SubCp and the sub-computing of ReduceCP according to described historical computing mode utilization are carried out incremental computations.
Wherein, the sub-computing of SubCp is for carrying out self-defining sub-computing and intermediate result is preserved subsection timing sequence data respectively; The sub-computing of ReduceCP is computing merging phase, the result of calculation merger operation according to self-defining operation to subsection timing sequence data, and the computing mode of described SubCp and the sub-computing of ReduceCP is safeguarded by caching server.
Further, described subsection timing sequence data increment computing method comprise:
Moving window incremental calculation method with the stationary window width of state: the historical computing mode of the time series data that described state representation caching server is safeguarded, described window width is fixing represents that the time period number comprising fixes; If the width of window is fixed as n, and the 1st time series data to n time period has completed and has calculated and deposit in described caching server, in the time having n+1 newly-increased time series data to arrive, according to the historical computing mode of this time series data in caching server, utilize the sub-computing of SubCp only to carry out the calculating of n+1 newly-increased time series data, the result merger in n+1 newly-increased time series data and historical computing mode is carried out in the sub-computing of recycling ReduceCP, and deducts the time series data of the 1st time period;
Incremental calculation method with the fixing monotone increasing window of the starting point of state: the historical computing mode of the time series data that described state representation caching server is safeguarded, its window start time point is fixed, and the size of window increases progressively in time; If the starting point of window is the time series data of the 1st time period, and the 1st time series data to n time period has completed and has calculated and deposit in described caching server, in the time having n+1 newly-increased time series data to arrive, according to the historical computing mode of this time series data in caching server, utilize the sub-computing of SubCp only to carry out the calculating of n+1 newly-increased time series data, the result merger in n+1 newly-increased time series data and historical computing mode is carried out in the sub-computing of recycling ReduceCP.
For the ease of understanding, below in conjunction with accompanying drawing, 2-5 is described further the present invention.
As shown in Figure 2, for just following the schematic diagram that the time series data fragmentation scheme providing is provided.As shown in Figure 2, for time series data, can continuous time series data be divided into multiple sections as unit taking section sometime, the time series data computing in each like this unit interval section is a sub-computing.Wherein, the sub-computing after division need to meet the one semigroup character, can carry out Merging to corresponding sub-computing.
As shown in Figure 3, for subsection timing sequence data increment calculation flow chart, this process has been utilized the subsection timing sequence data mechanism of Fig. 2, these computing method comprise two sub-computings of sub-computing: SubCp and the sub-computing of ReduceCP, wherein, the sub-computing of SubCp is for carrying out self-defining sub-computing and intermediate result is preserved subsection timing sequence data respectively; Exemplary, in the subsection timing sequence data of statistics taking sky as unit, certain page access flow of website in each time period.ReduceCP is computing merging phase, the result of calculation merger operation according to self-defining operation to segment data; Exemplary, in the subsection timing sequence data of merger taking sky as unit, the n days total flowing of access of certain page of this website recently.And the state of the sub-computing of above-mentioned SubCp and the sub-computing of ReduceCP is safeguarded by caching server (Cache Server).
The embodiment of the present invention can be saved a large amount of unnecessary double countings by incremental computations, thereby has improved the efficiency of data processing; In the embodiment of the present invention, in conjunction with the correlation properties of subsection timing sequence data increment computing method and subsection timing sequence data, two kinds of moving window incremental calculation methods with state are proposed: fixed width window, the time period number that window comprises is fixed; Monotone increasing window, window start time point is fixed, and passes in time window size and increases progressively.Specific as follows:
Shown in Fig. 4, be the moving window incremental computations of the stationary window width with state, described state refers to the correlation computations state that Cache Server safeguards.In conjunction with correlation properties and the incremental calculation method of subsection timing sequence data in Fig. 2,3, as shown in Figure 4, here the width of supposing window is fixed as n, in the time having n+1 newly-increased time series data to arrive, data (the 1st to n segments order sequenced data) on the left of learning according to the historical computing mode in Cache Server had been calculated, now only need to calculate incremental data (n+1 newly-increased time series data) and and the merger of part historical results just can obtain results needed, after merger, also need to deduct the 1st segments order sequenced data because the width of window is fixed as n; The newly-increased number sequence data of final combination and historical result of calculation can obtain and carry out global data and calculate the same result, and this method can be avoided a large amount of unnecessary double countings, thereby have improved the efficiency of data processing.
As shown in Figure 5, be the incremental computations of the fixing monotone increasing window of the starting point with state, described state refers to the correlation computations state that Cache Server safeguards.In conjunction with Fig. 2, the correlation properties of subsection timing sequence data and incremental calculation method in 3, as shown in Figure 5, the starting point of supposing window is 1, in the time having n+1 newly-increased number sequence data to arrive, data (the 1st to n segments order sequenced data) on the left of learning according to the historical computing mode in Cache Server had been calculated, now only need to calculate incremental data (n+1 newly-increased time series data) and and the merger of part historical results just can obtain results needed, the newly-increased data of final combination and historical result of calculation can obtain and carry out global data and calculate the same result, this method can be avoided a large amount of unnecessary double countings, thereby improve the efficiency of data processing.
On the other hand, the caching server in the embodiment of the present invention also can arrange timing mechanism to the data of inserting, and it is identifying and removing useless legacy data to guarantee the memory database expansion of can breaking after section sometime.
Meanwhile, time series data computational algorithm can also be combined with incremental calculation method provided by the invention; Wherein, time series data computational algorithm comprises the algorithm that following conventional time series is calculated: time series forecasting algorithm, comprises simple sequential average method, moving average method, weighted moving average method etc.; Time Series Similarity metric algorithm, comprises ED, DTW, FastDTW etc.
The technical scheme that the embodiment of the present invention provides compared with prior art, has following beneficial effect:
1) based on Hadoop platform, do not change Hadoop bottom architecture structure, facilitate programming personnel's coding;
2) on Hadoop platform, support the processing of time series data;
3) incremental computations of the time series data of support Hadoop platform, reduces unnecessary double counting, improves incremental data counting yield.
Embodiment bis-
The schematic diagram of a kind of Hadoop platform time series data incremental computations system that Fig. 6 provides for the embodiment of the present invention two.As shown in Figure 6, this system mainly comprises:
Time series data incremental processing module TSI11 in the time starting time series data incremental computations task, obtains the historical computing mode of this time series data from caching server; The subsection timing sequence data increment computing method that comprise SubCp and the sub-computing of ReduceCP according to described historical computing mode utilization are carried out incremental computations; Wherein, the sub-computing of SubCp is for carrying out self-defining sub-computing and intermediate result is preserved subsection timing sequence data respectively; The sub-computing of ReduceCP is computing merging phase, the result of calculation merger operation according to self-defining operation to subsection timing sequence data, and the computing mode of described SubCp and the sub-computing of ReduceCP is safeguarded by caching server;
Caching server 12, for preserving the historical computing mode of time series data.
Further, described subsection timing sequence data increment computing method comprise:
Moving window incremental calculation method with the stationary window width of state: the historical computing mode of the time series data that described state representation caching server is safeguarded, described window width is fixing represents that the time period number comprising fixes; If the width of window is fixed as n, and the 1st time series data to n time period has completed and has calculated and deposit in described caching server, in the time having n+1 newly-increased time series data to arrive, according to the historical computing mode of this time series data in caching server, utilize the sub-computing of SubCp only to carry out the calculating of n+1 newly-increased time series data, the result merger in n+1 newly-increased time series data and historical computing mode is carried out in the sub-computing of recycling ReduceCP, and deducts the time series data of the 1st time period;
Incremental calculation method with the fixing monotone increasing window of the starting point of state: the historical computing mode of the time series data that described state representation caching server is safeguarded, its window start time point is fixed, and the size of window increases progressively in time; If the starting point of window is the time series data of the 1st time period, and the 1st time series data to n time period has completed and has calculated and deposit in described caching server, in the time having n+1 newly-increased time series data to arrive, according to the historical computing mode of this time series data in caching server, utilize the sub-computing of SubCp only to carry out the calculating of n+1 newly-increased time series data, the result merger in n+1 newly-increased time series data and historical computing mode is carried out in the sub-computing of recycling ReduceCP.
Further, described time series data, is divided into multiple segmentations continuous time series data taking section sometime as unit, and the time series data computing in each unit interval section is a sub-computing; Wherein, the time series data after segmentation meets the one semigroup character.
Because native system can be realized based on Hadoop platform, for ease of understanding, above-mentioned module can be combined with existing Hadoop platform.As shown in Figure 7, based on Hadoop platform extension caching server Cache Server and time series data incremental processing module TSI; Caching server is data cached library module, its buffer memory necessary computing mode result, the buffer memory service that comparing Hadoop self provides has abundanter data structure to represent function; TSI module is mainly used in time series data incremental computations.
It should be noted that, in the specific implementation of the function that each functional module comprising in said system realizes each embodiment above, have a detailed description, therefore here repeat no more.
Those skilled in the art can be well understood to, for convenience and simplicity of description, only be illustrated with the division of above-mentioned each functional module, in practical application, can above-mentioned functions be distributed and completed by different functional modules as required, be divided into different functional modules by the inner structure of system, to complete all or part of function described above.
Through the above description of the embodiments, those skilled in the art can be well understood to above-described embodiment and can realize by software, and the mode that also can add necessary general hardware platform by software realizes.Based on such understanding, the technical scheme of above-described embodiment can embody with the form of software product, it (can be CD-ROM that this software product can be stored in a non-volatile memory medium, USB flash disk, portable hard drive etc.) in, comprise that some instructions are in order to make a computer equipment (can be personal computer, server, or the network equipment etc.) carry out the method described in each embodiment of the present invention.
The above; only for preferably embodiment of the present invention, but protection scope of the present invention is not limited to this, is anyly familiar with in technical scope that those skilled in the art disclose in the present invention; the variation that can expect easily or replacement, within all should being encompassed in protection scope of the present invention.Therefore, protection scope of the present invention should be as the criterion with the protection domain of claims.
Claims (6)
1. a Hadoop platform time series data incremental calculation method, is characterized in that, the method comprises:
In the time starting time series data incremental computations task, from caching server, obtain the historical computing mode of this time series data;
The subsection timing sequence data increment computing method that comprise SubCp and the sub-computing of ReduceCP according to described historical computing mode utilization are carried out incremental computations;
Wherein, the sub-computing of SubCp is for carrying out self-defining sub-computing and intermediate result is preserved subsection timing sequence data respectively; The sub-computing of ReduceCP is computing merging phase, the result of calculation merger operation according to self-defining operation to subsection timing sequence data, and the computing mode of described SubCp and the sub-computing of ReduceCP is safeguarded by caching server.
2. method according to claim 1, is characterized in that, described subsection timing sequence data increment computing method comprise:
Moving window incremental calculation method with the stationary window width of state: the historical computing mode of the time series data that described state representation caching server is safeguarded, described window width is fixing represents that the time period number comprising fixes; If the width of window is fixed as n, and the 1st time series data to n time period has completed and has calculated and deposit in described caching server, in the time having n+1 newly-increased time series data to arrive, according to the historical computing mode of this time series data in caching server, utilize the sub-computing of SubCp only to carry out the calculating of n+1 newly-increased time series data, the result merger in n+1 newly-increased time series data and historical computing mode is carried out in the sub-computing of recycling ReduceCP, and deducts the time series data of the 1st time period;
Incremental calculation method with the fixing monotone increasing window of the starting point of state: the historical computing mode of the time series data that described state representation caching server is safeguarded, its window start time point is fixed, and the size of window increases progressively in time; If the starting point of window is the time series data of the 1st time period, and the 1st time series data to n time period has completed and has calculated and deposit in described caching server, in the time having n+1 newly-increased time series data to arrive, according to the historical computing mode of this time series data in caching server, utilize the sub-computing of SubCp only to carry out the calculating of n+1 newly-increased time series data, the result merger in n+1 newly-increased time series data and historical computing mode is carried out in the sub-computing of recycling ReduceCP.
3. method according to claim 1 and 2, is characterized in that, described time series data is divided into multiple segmentations continuous time series data taking section sometime as unit, and the time series data computing in each unit interval section is a sub-computing; Wherein, the time series data after segmentation meets the one semigroup character.
4. a Hadoop platform time series data incremental computations system, is characterized in that, this system comprises:
Time series data incremental processing module TSI in the time starting time series data incremental computations task, obtains the historical computing mode of this time series data from caching server; The subsection timing sequence data increment computing method that comprise SubCp and the sub-computing of ReduceCP according to described historical computing mode utilization are carried out incremental computations; Wherein, the sub-computing of SubCp is for carrying out self-defining sub-computing and intermediate result is preserved subsection timing sequence data respectively; The sub-computing of ReduceCP is computing merging phase, the result of calculation merger operation according to self-defining operation to subsection timing sequence data, and the computing mode of described SubCp and the sub-computing of ReduceCP is safeguarded by caching server;
Caching server, for preserving the historical computing mode of time series data.
5. system according to claim 4, is characterized in that, described subsection timing sequence data increment computing method comprise:
Moving window incremental calculation method with the stationary window width of state: the historical computing mode of the time series data that described state representation caching server is safeguarded, described window width is fixing represents that the time period number comprising fixes; If the width of window is fixed as n, and the 1st time series data to n time period has completed and has calculated and deposit in described caching server, in the time having n+1 newly-increased time series data to arrive, according to the historical computing mode of this time series data in caching server, utilize the sub-computing of SubCp only to carry out the calculating of n+1 newly-increased time series data, the result merger in n+1 newly-increased time series data and historical computing mode is carried out in the sub-computing of recycling ReduceCP, and deducts the time series data of the 1st time period;
Incremental calculation method with the fixing monotone increasing window of the starting point of state: the historical computing mode of the time series data that described state representation caching server is safeguarded, its window start time point is fixed, and the size of window increases progressively in time; If the starting point of window is the time series data of the 1st time period, and the 1st time series data to n time period has completed and has calculated and deposit in described caching server, in the time having n+1 newly-increased time series data to arrive, according to the historical computing mode of this time series data in caching server, utilize the sub-computing of SubCp only to carry out the calculating of n+1 newly-increased time series data, the result merger in n+1 newly-increased time series data and historical computing mode is carried out in the sub-computing of recycling ReduceCP.
6. according to the system described in claim 4 or 5, it is characterized in that, described time series data, is divided into multiple segmentations continuous time series data taking section sometime as unit, and the time series data computing in each unit interval section is a sub-computing; Wherein, the time series data after segmentation meets the one semigroup character.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410456262.9A CN104199942B (en) | 2014-09-09 | 2014-09-09 | A kind of Hadoop platform time series data incremental calculation method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410456262.9A CN104199942B (en) | 2014-09-09 | 2014-09-09 | A kind of Hadoop platform time series data incremental calculation method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104199942A true CN104199942A (en) | 2014-12-10 |
| CN104199942B CN104199942B (en) | 2017-11-07 |
Family
ID=52085235
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410456262.9A Expired - Fee Related CN104199942B (en) | 2014-09-09 | 2014-09-09 | A kind of Hadoop platform time series data incremental calculation method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104199942B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105843891A (en) * | 2016-03-22 | 2016-08-10 | 浙江大学 | Incremental online characteristic extraction and analysis method and system |
| WO2017113865A1 (en) * | 2015-12-31 | 2017-07-06 | 华为技术有限公司 | Method and device for big data increment calculation |
| CN108846636A (en) * | 2018-06-01 | 2018-11-20 | 北京字节跳动网络技术有限公司 | Data dispatching method, device, computer readable storage medium |
| CN109948007A (en) * | 2019-03-21 | 2019-06-28 | 浙江邦盛科技有限公司 | A kind of clock synchronization ordinal number maximum processing method for being increased continuously number and number of increments according to statistics |
| CN110008544A (en) * | 2019-03-21 | 2019-07-12 | 浙江邦盛科技有限公司 | A kind of processing method of clock synchronization ordinal number number of increments and reduced degree according to statistics |
| CN110019367A (en) * | 2017-12-28 | 2019-07-16 | 北京京东尚科信息技术有限公司 | A kind of method and apparatus of statistical data feature |
| CN112488412A (en) * | 2020-12-11 | 2021-03-12 | 北京字跳网络技术有限公司 | Duration information determination method and device, electronic equipment and computer storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103049556A (en) * | 2012-12-28 | 2013-04-17 | 中国科学院深圳先进技术研究院 | Fast statistical query method for mass medical data |
| CN103676645A (en) * | 2013-12-11 | 2014-03-26 | 广东电网公司电力科学研究院 | Mining method for association rules in time series data flows |
| US20140214372A1 (en) * | 2013-01-25 | 2014-07-31 | International Business Machines Corporation | Interpolation techniques used for time alignment of multiple simulation models |
-
2014
- 2014-09-09 CN CN201410456262.9A patent/CN104199942B/en not_active Expired - Fee Related
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103049556A (en) * | 2012-12-28 | 2013-04-17 | 中国科学院深圳先进技术研究院 | Fast statistical query method for mass medical data |
| US20140214372A1 (en) * | 2013-01-25 | 2014-07-31 | International Business Machines Corporation | Interpolation techniques used for time alignment of multiple simulation models |
| CN103676645A (en) * | 2013-12-11 | 2014-03-26 | 广东电网公司电力科学研究院 | Mining method for association rules in time series data flows |
Non-Patent Citations (2)
| Title |
|---|
| 刘学军等: "基于滑动窗口的在线数据流增量聚集查询", 《计算机工程》 * |
| 王文胜: "基于集群计算的网络信息采集系统的设计与实现", 《中国优秀硕士学位论文全文数据库》 * |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017113865A1 (en) * | 2015-12-31 | 2017-07-06 | 华为技术有限公司 | Method and device for big data increment calculation |
| CN106933882A (en) * | 2015-12-31 | 2017-07-07 | 华为技术有限公司 | A kind of big data incremental calculation method and device |
| CN106933882B (en) * | 2015-12-31 | 2020-09-29 | 华为技术有限公司 | Big data increment calculation method and device |
| CN105843891A (en) * | 2016-03-22 | 2016-08-10 | 浙江大学 | Incremental online characteristic extraction and analysis method and system |
| CN110019367A (en) * | 2017-12-28 | 2019-07-16 | 北京京东尚科信息技术有限公司 | A kind of method and apparatus of statistical data feature |
| CN110019367B (en) * | 2017-12-28 | 2022-04-12 | 北京京东尚科信息技术有限公司 | Method and device for counting data characteristics |
| CN108846636A (en) * | 2018-06-01 | 2018-11-20 | 北京字节跳动网络技术有限公司 | Data dispatching method, device, computer readable storage medium |
| CN109948007A (en) * | 2019-03-21 | 2019-06-28 | 浙江邦盛科技有限公司 | A kind of clock synchronization ordinal number maximum processing method for being increased continuously number and number of increments according to statistics |
| CN110008544A (en) * | 2019-03-21 | 2019-07-12 | 浙江邦盛科技有限公司 | A kind of processing method of clock synchronization ordinal number number of increments and reduced degree according to statistics |
| CN109948007B (en) * | 2019-03-21 | 2020-07-14 | 浙江邦盛科技有限公司 | Processing method for inquiring maximum continuous increasing times and decreasing times of time sequence data statistics |
| CN112488412A (en) * | 2020-12-11 | 2021-03-12 | 北京字跳网络技术有限公司 | Duration information determination method and device, electronic equipment and computer storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104199942B (en) | 2017-11-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104199942A (en) | Hadoop platform time series data incremental computation method and system | |
| US10423635B2 (en) | Processing time series | |
| US10281284B2 (en) | Hybrid road network and grid based spatial-temporal indexing under missing road links | |
| JP6852392B2 (en) | Stochastic price and sudden change forecast | |
| CN111339240A (en) | Object recommendation method and device, computing equipment and storage medium | |
| CN110633594A (en) | Target detection method and device | |
| CN113052265B (en) | Moving object track simplifying method based on feature selection | |
| CN103473036A (en) | Input method skin push method and system | |
| CN114817651B (en) | Data storage method, data query method, device and equipment | |
| US20170061208A1 (en) | Determining Severity of a Geomagnetic Disturbance on a Power Grid Using Similarity Measures | |
| Liu et al. | Segmented analysis for reducing data movement | |
| CN118246413A (en) | Text generation method, device and model based on attention mechanism | |
| CN109657071B (en) | Vocabulary prediction method, device, equipment and computer readable storage medium | |
| CN110309974B (en) | Logistics transportation destination prediction method and device, electronic equipment and storage medium | |
| US20230224493A1 (en) | Segment-wise prediction machine learning frameworks | |
| CN115630733A (en) | Order delivery time estimation method, system, medium and electronic device | |
| Chen et al. | Discrete wavelet transform application for bike sharing system check-in/out demand prediction | |
| CN105302894A (en) | Parallel association rule based tracking method and tracking apparatus for hotspots of public opinions | |
| CN113468866B (en) | Method and device for analyzing non-standard JSON string | |
| CN114691918A (en) | Radar image retrieval method and device based on artificial intelligence and electronic equipment | |
| Kim et al. | Are Self-Attentions Effective for Time Series Forecasting? | |
| Kang et al. | Integration of cloud and big data analytics for future smart cities | |
| Zhang et al. | Wind power error compensation prediction model based on CEEMD-SE-ELM-TCN | |
| CN115908427B (en) | Pavement disease maintenance cost prediction method and system based on semantic segmentation and SVM | |
| CN109815312A (en) | A kind of method, apparatus of document query calculates equipment and computer storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20171107 |