CN110515681A - A kind of method of the given delay repeatability of real-time judge flow data itself - Google Patents

A kind of method of the given delay repeatability of real-time judge flow data itself Download PDF

Info

Publication number
CN110515681A
CN110515681A CN201810526517.2A CN201810526517A CN110515681A CN 110515681 A CN110515681 A CN 110515681A CN 201810526517 A CN201810526517 A CN 201810526517A CN 110515681 A CN110515681 A CN 110515681A
Authority
CN
China
Prior art keywords
calculation window
adjustment
component
data
delay
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201810526517.2A
Other languages
Chinese (zh)
Other versions
CN110515681B (en
Inventor
吕纪竹
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Priority to CN201810526517.2A priority Critical patent/CN110515681B/en
Publication of CN110515681A publication Critical patent/CN110515681A/en
Application granted granted Critical
Publication of CN110515681B publication Critical patent/CN110515681B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/451—Execution arrangements for user interfaces

Abstract

The auto-correlation of given delay can be used for judging the repeatability of time series or the given delay of flow data itself.Judge the method for time series or the given delay repeatability of flow data itself, system and calculating device program product in real time the invention discloses a kind of auto-correlation of specified delay by decrement calculating time series or the calculation window of flow data.Embodiment of the present invention includes autocorrelative two components above of the specified delay of calculation window after autocorrelative two components above decrement calculating based on the specified delay of calculation window before adjusting adjusts, and then generates the auto-correlation of the specified delay of calculation window after adjusting based on two components above that decrement calculates as needed.All data elements and execution after decrement calculating auto-correlation avoids access from adjusting in calculation window compute repeatedly to improve computational efficiency, it saves computing resource and reduces computing system energy consumption, so that given repeated efficient low-consume and the given delay repeatability of some real-time judge flow datas itself of postponing of real-time judge flow data itself is from being very unlikely to become possibility.

Description

A kind of method of the given delay repeatability of real-time judge flow data itself
Technical field
Big data or flow data analysis.
Background technique
Internet, mobile communication, navigation, game on line, cognition technology and large-scale calculations infrastructure all generate daily The data of magnanimity.Big data precisely due to its huge size, quickly variation and growth rate and have exceeded conventional database systems Processing capacity and traditional analysis analysis ability data.
Flow data is exactly the data for constantly being transmitted and being continuously received by a supplier.Flow data can be collection Real time data from sensor simultaneously continuously transfers data on calculating equipment or electronic equipment.Usually this includes that reception is identical The data element of format continuously divided by time interval.Flow data is also possible to the data continuously read from storage equipment, That is storage equipment stores large data sets on multiple computing device.
Auto-correlation, also referred to as delay correlation or sequence correlation are a specific time serieses and delay l time One measurement of the degree of correlation of the time series of point itself.It can be separated by l time by time series Association's correlation of the observed value of point is obtained divided by its standard variance.The autocorrelation value of some delay is 1 or close to 1 it is believed that fluxion According to or fluidisation big data occur after the delay itself repeat rule, therefore the auto-correlation based on given delay judge flow data oneself The repeatability of the given delay of body is it is clear that and difficult and challenge is how to calculate auto-correlation on flow data in real time.
Auto-correlation may need to be recalculated after out-of-date data are removed or adjust calculation window scale.Example It such as, to be perhaps to be removed with received data element and downsizing calculation window calculating auto-correlation.Every reception One data element, the data element can be removed from calculation window.Data element from scale be n calculation window removal after, N-1 data element in calculation window adjusted will be accessed to recalculate auto-correlation.In this way, each data variation The sub-fraction data in calculation window may only be changed.It is usually weighed using all data elements in calculation window adjusted The new auto-correlation that calculates is related to repeated data access and calculates, therefore time-consuming and waste of resource.
Depending on needing, the scale of calculation window may be very big, such as the data element in calculation window may be distributed In thousands of calculating equipment of cloud platform.It is recalculated certainly on the flow data after some data variations with conventional method Correlation can not accomplish to handle and occupy and waste a large amount of computing resources in real time, but also some judge flow data itself in real time The repeatability of given delay is realized in which may be unable to satisfy demand.
Summary of the invention
The present invention is extended to method, and system and calculating device program product calculate the auto-correlation of given delay in a manner of being reduced So as to judge the repeatability of the given delay of big data itself in real time after adjusting calculation window scale.After an adjustment It includes the auto-correlation that the delay based on calculation window before adjusting is l that calculation window decrement, which calculates the specified auto-correlation for postponing l (l > 0), Two or more (p (p > 1)) component decrement to calculate autocorrelative more than two components that the delay of calculation window after adjustment is l right Generate the auto-correlation that the delay of calculation window after adjusting is l based on more than two components that decrement calculates as needed afterwards.Decrement Calculate the component that auto-correlation only needs to access and calculate using decrement, calculation window two after the data element of removal, and adjustment Each l data element on side and all data elements and execution after avoiding access from adjusting in calculation window compute repeatedly to drop Low data access delay improves computational efficiency, saves computing resource and reduces computing system energy consumption, makes some judgement streams in real time The repeatability of the given delay of data itself is from being very unlikely to become possibility.
The auto-correlation given for one is reduced algorithm, it is assumed that the component that all decrements calculate in same wheel decrement calculates (including calculation window and/or average value) sum is p (p > 1).The number of components that directly decrement calculates is v (1≤v≤p), then The number of components that decrement calculates indirectly is w=p-v (w >=0).Wherein calculation window and/or average value must be reduced calculating Particular components.And/or average value can be calculated by direct or indirect decrement.
Computing system initialization delay l (l > 0), calculation window scale n (n > l) and it is stored in one or more storage media On large data sets scale be n adjustment before calculation window delay be l autocorrelative p (p > 1) a component.
Computing system includes a buffer area to store flow data element.It this buffer area can be in memory or other calculating Machine readable media, in hard disk or other media, it might even be possible to it is the multiple distributed documents distributed in multiple calculating equipment, They it is end-to-end interconnection and formed one " cyclic buffer ".
The calculating window of calculation window before one of the data flow that computing system initialization is stored on a buffer area adjusts Mouthful scale counter n (n > l), one and/or an average value or one and and an average value, and specified delay l (l > 0) Autocorrelative other one or more components.The initialization of the two or more component includes based in calculation window before adjusting Data element calculates more than two components by the definition of component or has calculated from reception or access on device-readable media is calculated More than two components.The initialization of calculation window scale counter includes the number data elements setting one for using tricks to calculate in window A value or access receive a specific calculation window scale.
Computing system receives one will be from the related letter for adjusting data element or the data element that preceding calculation window removes Breath (for example, index or address).Computing system removes data element to be removed from data buffer zone.Computing system passes through Data element to be removed is removed from calculation window before the adjustment of non-empty and calculation window scale counter is subtracted 1 to adjust Calculation window before adjusting.
Computing system be directly reduced calculate adjustment after calculation window specified delay it is autocorrelative remove and and average value it Outer one or more components.It includes: calculation window both sides after access adjustment that directly decrement, which calculates the one or more component, Each l data element and the data element of removal；V component of the specified delay of calculation window before access adjusts；From v component Each of any contribution of the data element being removed mathematically is removed in part.
Computing system is reduced the autocorrelative w=p-v of the specified delay of calculation window after calculating adjusts indirectly as needed A component.Indirectly decrement calculate w component of specified delay include singly be reduced indirectly it is each in w component of calculating A component.It includes: access and using the specified delay in addition to the component that decrement, which calculates a component of specified delay, indirectly One or more components calculate the component.The one or more component may be by initialization, and directly decrement calculates Or it is reduced calculating indirectly.
Computing system is reduced autocorrelative group of the specified delay of calculation window after the adjustment of calculating based on one or more The auto-correlation of the specified delay of calculation window after one adjustment of part generation.
Computing system can constantly receive a data element to be removed, and the data element is removed from buffer area Element, calculation window before adjustment adjusts, directly v (1≤v≤p) a component of the specified delay of decrement calculating, is reduced indirectly as needed It calculates the component of w=p-v specified delays and calculates the auto-correlation of specified delay.Computing system can according to need multiplicating This process.
This summary is to introduce the concept of some selections in a simplified manner, they will be described in further detail below. This summary is to be not configured to the key features or necessary feature of identification claimed subject matter, nor in order to be used to help confirm Range included by claimed subject matter.
Other features and advantages of the present invention will embody in the following description, understand the partly obvious body from description It is existing, or acquire from the practice of the present invention.What the features and advantages of the present invention can be particularly pointed out from appended claims It realizes and obtains in method equipment and combinations thereof.These and other features of the invention will be in following description and additional right Become more full apparent in claim or practice of the invention.
Detailed description of the invention
To describe that the mode of above-mentioned and other advantage and feature of the invention, the present invention being briefly described above can be obtained A more specifically description will be shown by referring to specific embodiment shown in additional chart.These Chart only describes typical embodiments of the invention, therefore they are not understood that or are construed to the scope of the present invention Limitation:
Fig. 1 illustrates one to support decrement to calculate the high level summary of autocorrelative example computing system.
Fig. 1-1 shows the auto-correlation for supporting decrement to calculate flow data and all components are calculated in a manner of being directly reduced One example computing system framework.
Fig. 1-2 show support decrement calculate flow data auto-correlation and members calculated in a manner of being directly reduced and The example computing system framework that members are calculated in a manner of being reduced indirectly.
Fig. 2 shows that decrement calculates the flow chart of an autocorrelative exemplary techniques for flow data.
Fig. 3-1 shows the data removed from the left side of calculation window 300A.
Fig. 3-2 shows that decrement calculates the data that auto-correlation accesses when removing data from the left side of calculation window 300A.
Fig. 3-3 shows the data removed from the right of calculation window 300B.
Fig. 3-4 shows that decrement calculates the data that auto-correlation accesses when removing data from the right of calculation window 300B.
Fig. 4-1 shows autocorrelative definition and calculates autocorrelative traditional equation.
Fig. 4-2 shows first available equation of auto-correlation decrement computational algorithm (decrement algorithm 1).
Fig. 4-3 shows second available equation of auto-correlation decrement computational algorithm (decrement algorithm 2).
Fig. 4-4 shows third auto-correlation decrement computational algorithm (decrement algorithm 3) available equation.
Fig. 5-1 shows first calculation window for a calculated examples.
Fig. 5-2 shows second calculation window for a calculated examples.
Fig. 5-3 shows the third calculation window for a calculated examples.
Fig. 6-1 shows the calculation amount of tradition and decrement auto-correlation algorithm when calculation window length be 4 delays is 1.
Fig. 6-2 shows the calculating of tradition and decrement auto-correlation algorithm when calculation window length be 1000000 delays is 1 Amount.
Specific embodiment
Calculating auto-correlation is the effective ways for judging time series or fluidizing the given delay repeatability of big data itself.This hair It is bright to be extended to the auto-correlation by the specified delay l (1≤l<n) for the calculation window that decrement computational length is n (n>1) so as to reality When judge the method for the given delay repeatability of flow data itself, system and calculate device program product.One computing system packet Containing one or more processor-based calculating equipment and one or more storage media.Each calculating equipment includes one or more A processor.The computing system includes an input block.In store flow data element in the input block.This buffering It area can be in memory or other computer-readable medias, in hard disk or other media, it might even be possible to be distribution in multiple calculating Multiple distributed documents in equipment, they it is end-to-end interconnection and formed one " cyclic buffer ".From the data flow, relate to And the calculation window before an adjustment is formed to multiple data elements of autocorrelation calculation.Computing system keeps a calculation window Scale counter n (n > l) indicates the data element number in a calculation window of buffer area.Delay l is indicated for auto-correlation The delay used when calculating.Embodiment of the present invention includes autocorrelative two based on the specified delay of calculation window before adjusting Autocorrelative more than two components of the specified delay of calculation window after a above (p (p > 1)) component decrement calculating adjustment, then The auto-correlation of the specified delay of calculation window after adjusting is generated based on more than two components that decrement calculates as needed.Decrement meter All data elements and execution after auto-correlation avoids access from adjusting in calculation window are calculated to compute repeatedly to improve computational efficiency, It saves computing resource and reduces computing system energy consumption, so that itself repeatability of the given delay of some real-time judge flow datas is from can not It can become possible.
Auto-correlation, also referred to as delay correlation or sequence correlation are a specific time serieses and delay l time One measurement of the degree of correlation of the time series of point itself.It can be separated by l time by time series Association's correlation of the observed value of point is obtained divided by its standard variance.If calculating all different delays values an of time series Auto-correlation just obtain the auto-correlation function of the time series.The time series not changed over time for one, auto-correlation Value can exponentially be reduced to 0.The range of autocorrelative value is between -1 and+1.Value+1 shows the past and future of time series Value has a complete positive linear relationships, and value -1 show the past of time series and following value have one it is complete negative Relationship.Calculate given delay it is autocorrelative on the basis of judge the repeatability of the given delay of flow data itself obviously.
Herein, a calculation window contains data involved in autocorrelation calculation.Data element in calculation window It is known as ordinal relation, that is, the sequence for changing the data element in calculation window can influence autocorrelation calculation result.
Herein, an autocorrelative component occurs from auto-correlation defined formula or any turn of its defined formula An amount or expression formula in changing.Auto-correlation is the maximum component of own.It is the example of some autocorrelative components below.
(l is delay)
Auto-correlation component or their combination can be calculated based on one or more, so polyalgorithm supports decrement auto-correlation meter It calculates.
One component can be calculated by directly decrement or decrement calculates indirectly.Their difference is when a component is direct The component is the value that is calculated by the component in previous round to calculate when decrement calculates, and when the component is by indirect decrement calculating When the component be with other Assembly calculations except the component.
The component given for one, it is perhaps by directly decrement calculating but the quilt in another algorithm in an algorithm Decrement calculates indirectly.
Calculation window and/or average value be the particular components that must be reduced calculating.For any one algorithm, at least can It is decremented calculating there are two component, one of component is and/or average value, the two components above can be direct or indirect Decrement calculates, but efficient mode is that at least one component is calculated by directly decrement.The algorithm given for one, it is assumed that make The sum of different components is (p > 1) p, if the number of components that directly decrement calculates is v (1≤v≤p), is subtracted indirectly The number for measuring the component calculated is w=p-v (0≤w < p).Component that may be all calculates (v in this case by directly decrement =p > 1 and w=0).But no matter whether autocorrelative result is required and accesses in a specific round, is directly reduced The component of calculating must all be calculated.
For a given algorithm, if a component is calculated by directly decrement, the component must be calculated (i.e. whenever When an existing data element is removed from calculation window).It, should but if a component is calculated by indirect decrement Component can by using other one or more components except the component come as needed, i.e., only when auto-correlation need by When calculating and accessing, calculated.In this way, only a small amount of component needs when auto-correlation is when some calculates round and is not accessed It calculates with being decremented.Perhaps, the direct decrement that one component that decrement calculates indirectly can be used for a component calculates, at this In the case of kind, the calculating for the component which calculates can not be omitted.
Implementation of the invention includes that two or more (p (p > 1)) component that calculation window calculates before being based upon adjustment subtracts Amount ground calculates autocorrelative more than two (p (p > 1)) components of calculation window adjusted.
Computing system had been initialised from non-empty, calculation window scale counter and two or more components Calculation window starts the two or more autocorrelative components of decrement calculating before adjusting.If do not initialized, calculation window scale Counter and two or more components can be initialized according to the data element in calculation window before adjusting.Initialize calculation window Scale counter includes setting a value or access with the data element number in calculation window or receiving one specifically to calculate Window scale.
Computing system initialize a given scale n (n > 1) adjustment before calculation window given delay l (l >=1) from Relevant two or more (p (p > 1)) component.The initialization of the two or more component includes being defined based on before the adjustment according to it The data element in window is calculated usually to calculate or be computed from access or reception on one or more calculating device-readable media Component.
Computing system receives one will be from the data element or the data element of calculation window removal before the adjustment of non-empty Relevant information (for example, index or address).
Computing system removes received data element from data buffer zone.
Computing system by before the adjustment of non-empty calculation window remove data element to be removed and calculating window Mouth scale counter subtracts 1 to adjust and adjust preceding calculation window.
Computing system is that calculation window is directly reduced autocorrelative one or more v (1≤v that computing relay is l after adjusting ≤ p) component.The autocorrelative v component that directly decrement computing relay is l includes being reduced to calculate indirectly respectively one by one to prolong It is late each of the v component of l.It is directly reduced in delay l and calculates calculation window after v component includes: to access the adjustment Each l data element on both sides, the data element being removed, and to adjust the v group that the delay of preceding calculation window calculating is l Part；Any contribution for the data element being removed mathematically is removed from each of the v component that delay is l part.
Computing system is reduced autocorrelative w=p-v that computing relay is l as needed for calculation window after adjustment indirectly Component.The autocorrelative w component that decrement computing relay is l indirectly includes that decrement computing relay is indirectly respectively one by one Each of w component of l.The component that decrement computing relay is l indirectly includes: that the delay accessed except the component is The one or more components of l and the Assembly calculation component based on access.The one or more components that these delays are l can be It was initialised, and was directly reduced calculated or was reduced indirectly calculated.
Computing system is as needed, is based upon autocorrelative one that the delay that calculation window decrement calculates after adjusting is l Or multiple components, the auto-correlation for being l for calculation window computing relay after adjustment.
Computing system can continue to receive data element to be removed, the data element from data buffer zone It removes, adjusts calculation window before the adjustment, directly decrement calculates one or more v (1≤v≤p) specified component postponed, saves straight The component that decrement calculates is connect, is reduced the component for calculating w=p-v specified delays indirectly as needed, it is as needed to be based on one Or the given auto-correlation postponed of Assembly calculation that multiple decrements calculate, and this process is repeated as needed.
Embodiment of the present invention may include or using comprising calculate device hardware, such as one or more processors and The storage equipment of following more detail, dedicated or general calculating equipment.The range of embodiment of the present invention also includes object It is reason and it is other for carry or store calculate equipment can operating instruction and/or data structure calculating device-readable media.This A little device-readable media that calculate can be the general or specialized addressable any media of calculating equipment.Storage calculates equipment and can run The calculating device-readable media of instruction are storages media (equipment).Carry calculate equipment can operating instruction calculating device-readable matchmaker Body is transmission media.Therefore, in mode for example and not limitation, embodiment of the present invention may include at least two inhomogeneities The calculating device-readable media of type: storage media (equipment) and transmission media.
Storing media (equipment) includes random access memory (RAM), read-only memory (ROM), electrically erasable Read-only memory (EEPROM), compact disc-ROM (CD-ROM), solid state hard disk (SSD), flash memory (Flash Memory), phase Transition storage (PCM), other type memories, other optical disc storages, magnetic disk storage or other magnetic storage devices, or It is any other can be used to store required for calculate equipment can the program code that is constituted of operating instruction or data structure form simultaneously And it can be by the media of general or specialized calculating equipment access.
One " network " is defined as that calculating equipment and/or module and/or other electronic equipments is enable to transmit electron number According to one or more data links.When information by network or other communication connection (it is wired, it is wirelessly or wired and wireless Combination) it transmission or is supplied to when calculating equipment, it calculates equipment and connection is considered as transmission media.Transmission media may include for carrying It is required with calculate equipment can the program code that is constituted of operating instruction or data structure form, and it can be by general or special With the network and/or data link for calculating equipment access.Above combination also should include calculating device-readable media Within the scope of.
In addition, in different the calculatings apparatus assemblies of application, calculating equipment can operating instruction or data structure form program Code can be automatically transferred to storage media (equipment) (or in turn) from transmission media.For example, from network or data link Received calculating equipment can operating instruction or data structure can be kept in into random in Network Interface Module (for example, NIC) Access memory in, be then finally transferred to calculate equipment random access memory and/or to calculate equipment it is one smaller Variable storage media (equipment).It also (or even main) is answered therefore, it is to be understood that storage media (equipment) can be included in With in the calculating apparatus assembly of transmission media.
Calculate equipment can operating instruction include, for example, instruction and data, when being run by processor, so that general-purpose computations Equipment or dedicated computing equipment go to execute a specific function or one group of function.Calculate equipment can operating instruction can be, for example, Binary system, intermediate format instructions such as assembly code, or even source code.Although the object of description be with structure feature and/or Method movement concrete syntax description, it should be understood that the object defined in the appended claims may be not necessarily limited to The feature or movement of upper description.But the feature of description or movement are only disclosed to realize the example form of claim.
Embodiment of the present invention can by it is a plurality of types of calculate device configurations network computing environments in realize, this A little equipment that calculate include PC, desktop computer, laptop, message handler, handheld device, and multiprocessing system is based on Microprocessor or programmable consumer electronics, network computer, minicomputer, master computer, supercomputer, mobile electricity Words, palm PC, tablet computer, pager, router, interchanger and similar products.Embodiment of the present invention can also be applied (it can be linked by cable data, wireless data link in by the network interconnection, be also possible to cable data link and no line number According to the combination of link) execution task Local or Remote calculate equipment constitute distributed system environment.In distributed system In environment, program module can be stored locally or remotely in storage equipment.
Embodiment of the present invention can also be realized in cloud computing environment.In this description and following claims In, " cloud computing is defined as the mould for making it possible to be accessed the shared pool of configurable computing resource by network on demand Type.For example, cloud computing can utilize the shared pool for going offer universal with the convenient configurable computing resource of access on demand by market. The shared pool of configurable computing resource can be prepared quickly by virtualization and mutual with low management cost or low service provider It moves to provide, then does corresponding adjustment.
Cloud computing model may include various features for example, on demand from servicing, and broadband network access, resource is collected, quickly Folding and unfolding, metering service etc..Cloud computing model can also be embodied with various service modes, for example, software is as service (" SaaS "), platform is as service (" PaaS ") and facility as service (" IaaS ").Cloud computing model can also be by not With deployment model such as private clound, community cloud, public cloud, mixed cloud etc. disposes.
Be described herein and claims in, one " cyclic buffer " be a data structure its be one single Have regular length seems end to end buffer area.One cyclic buffer is either one commonly distributed in memory Block space is also possible to one " virtual cyclic buffer ", a file not necessarily in memory, but on hard disk, even It is the distributed document being physically distributed in more storage equipment, as long as they are interconnected to form one in logic end-to-endly A " cyclic buffer ".
Several examples can be provided in following chapters and sections.
Fig. 1 illustrates the high level overviews that an autocorrelative example computing system 100 is calculated for flow data decrement.With reference to figure 1, computing system 100 include by heterogeneous networks, such as local area network 1021, wireless network 1022 and internet 1023 etc., connection Multiple equipment.Multiple equipment includes, for example, data analysis engine 1007, storage system 1011, real-time stream 1006, and It can be with arrangement data analysis task and/or the calculating equipment for more distributions for inquiring data analysis result, such as PC 1016, handheld device 1017 and desktop computer 1018 etc..
Data analysis engine 1007 may include one or more processors, such as CPU 1009 and CPU1010, one or Multiple Installed System Memories, such as Installed System Memory 1008 and Assembly calculation module 131 and autocorrelation calculation module 192.Module 131 Details can illustrate (for example, Fig. 1-1 and Fig. 1-2) in more detail in other charts.Storage system 1011 may include one or Multiple storage media, such as storage media 1012 and storage media 1014, can be used for storing large data sets.For example, 1012 And/or 1014 may include data set 123.Data set in storage system 1011 can be accessed by data analysis engine 1007.
In general, data flow 1006 may include the flow data from different data sources, for example, share price, audio data, video Data, geographical spatial data, internet data, mobile communication data, network game data, bank transaction data, sensing data, And/or closed-caption data etc..Here citing depicts several, and real time data 1000 may include from inductor 1001, stock 1002, communicate the data of 1003 and bank 1004 etc. real-time collecting.Data analysis engine 1007 can receive from data flow 1006 data element.Data from different data sources can be stored in storage system 1011 and for big data analysis institute Access, such as data set 123 can come from different data sources and accessed by big data analysis.
It please understand Fig. 1 and be and introduce some concepts in very simplified form, for example, distribution apparatus 1016 and 1017 may be through It crosses firewall and is just linked to data analysis engine 1007, data analysis engine 1007 is from data flow 1006 and/or storage system 1011 Access or received data may be by data filter screenings, etc..
Fig. 1-1, which illustrates to be reduced for flow data, calculates auto-correlation, owns (v=p > 1) component and is calculated by directly decrement, Example computing system framework 100A.About computing system framework 100A, here by the main component first only introduced in the framework Function and correlation, and about these components how to cooperate the common process for completing decrement autocorrelation calculation further in connection with Flow chart described in Fig. 2 is introduced together.Fig. 1-1 illustrates 1006 and 1007 that Fig. 1 is shown.With reference to Fig. 1-1, computing system frame Structure 100A includes Assembly calculation module 131 and autocorrelation calculation module 192.Assembly calculation module 131 can be through high speed number It is couple according to bus with one or more storage media tights or pass through a network, such as local area network, wide area network or even internet With one or more storage media loose couplings by system management memory.Correspondingly, Assembly calculation module 131 and it is any its The calculating equipment and their component that it is connected, can send and receive message relevant data (for example, internet protocol on network (" IP ") datagram and other upper-layer protocols using IP datagram are discussed, for example, User Datagram Protocol (" UDP "), real-time streams Agreement (" RTSP "), real-time transport protocol (" RTP "), Microsoft Media Server (" MMS "), transmission control protocol (" TCP ") surpass Text Transfer Protocol (" HTTP "), simple message transfer protocol (SMTP) (" SMTP "), etc.).The output of Assembly calculation module 131 can quilt As the input of autocorrelation calculation module 192, auto-correlation 193 is can be generated in autocorrelation calculation module 192.
In general, the digitally encoded signal (i.e. the packets or data packet of data) that data flow 190 can be sequence is respectively used to pass Information in defeated or reception transmission process.Data flow 190 may include from different types of data, for example, share price, audio Data, video data, geographical spatial data, internet data, mobile communication data, network game data, bank transaction data, sensing Device data, closed-caption data and real-time text etc..Data flow 190 can be real-time time series or flow data.
With the reception of flow data element, data element can be removed from data buffer zone 121.For example, data element 101 Or its relevant information (such as index or address) can be received, data element 101 can be removed from position 121A.Then, number It can be received according to element 102 or its relevant information, then be removed from position 121B.
If shown, data buffer zone 121 has multiple before receiving data element 101 or its relevant information Data element is in position 121A, 121B, 121C, 121D, 121E and other positions.Calculation window scale can reduce and calculate window Mouthful from the removal of data buffer zone 121 it can become new calculation window with data element.
Calculation window scale counter 118 can be by hardware or software realization.When implemented in hardware, counter 118 can To be the skinny device for updating calculation window scale.When implemented in software, counter 118 can be there are memory, hard disk or Other calculate the variable of device-readable storage media, and inside perhaps value is calculation window scale.One data element of every reception, meter Calculation system can adjust meter by removing the data element from calculation window and the value of calculation window scale counter 118 being subtracted 1 Calculate window.Calculation window scale counter 118 can be accessed or be received by Assembly calculation module 131, can also be reset module The 129 specific values of setting one.In description herein, calculation window scale counter and calculation window scale be it is equivalent, It is interchangeable.
With reference to computing system framework 100A, before receiving data element 101 or its relevant information, calculation window 122 is wrapped Containing multiple data elements in position 121A, 121B ..., so calculation window scale should use the data element in calculation window 122 Plain number initialization.When data element 101 or its relevant information are received and data element 101 is removed from position 121A, Data element 101 can be removed from calculation window 122, and calculation window 122 will become calculation window 122A after adjustment before adjusting, Calculation window scale counter 118 can subtract 1.Calculation window counter 118 can be saved in storage equipment 119.Calculation window Scale counter 118 and data element 101 can all be accessed by Assembly calculation module 131.
Later, data element 102 or its relevant information can be received, and started new round decrement and calculated, last round of tune Calculation window 122A becomes calculation window before the adjustment that new round decrement calculates after whole.Data element 102 can be from position 121B quilt Removal.Calculation window 122A will become calculation window 122B after new adjustment.Calculation window scale counter 118 can subtract 1.It calculates Window scale counter 118 and data element 102 can all be accessed by Assembly calculation module 131.
With reference to computing system framework 100A, usual Assembly calculation module 131 is included as that direct decrement calculates calculation window V Assembly calculation module of a component of autocorrelative v (v=p > 1).V be one given delay decrement calculate it is autocorrelative to Determine the number that the component of calculating is directly reduced in algorithm, it is different and different with the decrement algorithm used.Such as institute in Fig. 1-1 Show, Assembly calculation module 131 includes a component Cd1Computing module 161 and a component CdvComputing module 162, between them There are also v-2 other Assembly calculation modules, they can be component Cd2Computing module, component Cd3Computing module ... ..., and Component Cdv-1Computing module.Each Assembly calculation module calculates the specific component of a given delay.Each Assembly calculation mould It is to calculate after adjusting that calculation window, which initializes the initialization module an of component and one, before block is adjusted comprising one for first Window is directly reduced the algorithm for calculating the component.For example, component Cd1Computing module 161 includes initialization module 132 to initialize The component Cd of given delay1The component Cd for calculating given delay is reduced with decrement algorithm 1331, component CdvComputing module 162 wraps The component Cd of given delay is initialized containing initialization module 138vThe component for calculating given delay is reduced with decrement algorithm 139 Cdv。
Initialization module 132 can be in initialization component Cd1When use when being reset using or autocorrelation calculation.Together Sample, initialization module 138 can be in initialization component CdvWhen use when being reset using or autocorrelation calculation.Component Cd1 141 be component Cd1Initial value.Initialization module 132 can be used to initialization component Cd1Or reset autocorrelation calculation.Initially Change module 132 can based on adjust before calculation window data element according to component Cd1Definition initialization component Cd1141 or Initialization component Cd1To reset the particular value of the transmission of module 123 (for example, contribution 151,152 and 153).Initialization module 138 can To be used to initialization component CdvOr reset autocorrelation calculation.Initialization module 138 can be based on the number of calculation window before adjusting According to element according to component CdvDefinition initialization component Cdv145 or initialization component CdvTo reset the specific of the transmission of module 123 Value is (for example, contribution 181,182 and 183).
Decrement algorithm includes directly being reduced calculating for the data element after an adjustment in calculation window and giving to postpone to be l's Autocorrelative v component.The component Cd that the access of decrement algorithm 133 or the delay for receiving last round of calculating are l1, calculated after adjustment Each l data element in window both sides and the data element removed in calculation window before adjusting are as input.It is reduced algorithm 133 The component Cd that delay using last round of calculating is l1, each l data element in calculation window both sides and before adjustment after adjustment The data element removed in calculation window, which is directly reduced, calculates the New Parent Cd that a delay is l1.Contribution removal module 133A can It is to adjust the component Cd that the delay of preceding calculation window calculating is l from last round of calculating1In mathematically removal be removed data element Any contribution of the element to the component, thus the New Parent Cd that the delay for being adjusted rear calculation window is l1.Be reduced algorithm 139 with The mode similar with decrement algorithm 133 works.The component that the access of decrement algorithm 139 or the delay for receiving last round of calculating are l Cdv, each l data element in calculation window both sides and the data element removed in calculation window before adjusting are as defeated after adjustment Enter.It is reduced the component Cd that algorithm 139 is l using the delay of last round of calculatingv, each l data element in calculation window both sides after adjustment Element and the data element removed in calculation window before adjusting, which are directly reduced, calculates the New Parent Cd that a delay is lv.Contribution Removal module 139A can be the component Cd that the delay that calculation window calculates before adjusting is l from last round of calculatingvIn mathematically go Except any contribution of the data element to the component is removed, thus the New Parent that the delay for being adjusted rear calculation window is l Cdv。
With reference to Fig. 1-1, computing system framework 100A further includes autocorrelation calculation module 192.Autocorrelation calculation module 192 can It is reduced the Assembly calculation that the delay of calculating is l based on one or more as needed and postpones the auto-correlation for being l.
Computing system can continue to data element, data element be removed from data buffer zone 121, before adjustment adjustment Calculation window, directly decrement calculates v (1≤v≤p) a component on delay l, calculates auto-correlation on delay l as needed, and This process is repeated as needed.
Fig. 1-2 illustrates as a flow data decrement calculating auto-correlation and part (v (1≤v < p)) component is directly reduced It calculates, the example that part (w=p-v) component is reduced calculating indirectly calculates device structure 100B.In some implementations, it calculates Difference between system architecture 100B and 100A is that framework 100B includes Assembly calculation module 135.In addition to this have with 100A together The part of sample label number all works in the same manner.In order not to thing explained inside 100A description before repetition, only There is different parts that can discuss herein.Digital v inside the 100B and digital v inside 100A may be different, because some The component that calculating is directly reduced in 100A can be calculated in 100B by indirect decrement.In 100A, v=p > 1, but in 100B In, 1≤v < p.With reference to Fig. 1-2, computing system framework 100B includes Assembly calculation module 135.The output of Assembly calculation module 131 It can be used as the input of Assembly calculation module 135, the output of computing module 131 and 135 can be used as autocorrelation calculation module 192 Input, auto-correlation 193 can be generated in autocorrelation calculation module 192.Assembly calculation module 135 generally includes w=p-v component Computing module, which to be reduced indirectly, calculates w component.For example, Assembly calculation module 135 includes Assembly calculation module 163 for indirect It is reduced computation module Ci1, Assembly calculation module 164 for being reduced computation module Ci indirectlywAnd other w-2 between them Assembly calculation module.It includes singly being reduced each for calculating w component indirectly that decrement, which calculates w component, indirectly.Between Connecing decrement and calculating a component includes accessing and using the one or more components in addition to the component itself.That is one or more Component, which can be, to be initialised, and directly decrement is calculated or is reduced indirectly calculated.
With reference to computing system framework 100B, once w=p-v component after given delay is calculated by indirect decrement, from phase Close the Assembly calculation auto-correlation 193 that computing module 192 can be calculated in given delay with one or more decrements.
Fig. 2 illustrates the stream that an autocorrelative exemplary techniques 200 are calculated for large data sets or the data flow decrement of fluidisation Cheng Tu.Method 200 can be described together respectively in connection with the component and data of computing system framework 100A and 100B.
Method 200 is including initializing the given size of a data buffer zone as the finger of calculation window before the adjustment of n (n > 1) Fixed delay is a component (201) of autocorrelative p (p>1) of l (0<l<n).For example, initialization module 132 can be in given delay With 151 (contributions of data element 101) of contribution, 152 (contributions of data element 102) and 153 (contributions of other data elements) Initialization component Cd1141.Similarly, initialization module 138 can be in the given delay 181 (tributes of data element 101 of contribution Offer), 182 (contributions of data element 102) and 183 (contributions of other data elements) initialization component Cdv 145。
Method 200 include when the i.e. not all component of v < p all by directly decrement calculate when, as needed singly between It connects decrement and calculates each of w=p-v component component based on the one or more components wanted except computation module.This w Component only can just be calculated (209) when auto-correlation is accessed.For example, being counted with reference to its members of Fig. 1-2 by directly decrement It calculates and members are calculated by indirect decrement, computing module 163 can be based on component Ci1Except one or more components come it is indirect It is reduced computation module Ci1, computing module 164 can be based on component CiwExcept one or more components to be reduced computation module indirectly Ciw.The one or more component can be initialization, and directly decrement calculates, or is reduced indirectly calculated.
Method 200 includes the auto-correlation for calculating a delay as needed and being l.When auto-correlation is accessed, the auto-correlation It can be calculated based on the component (210) of one or more initialization or decrement calculating, otherwise only have that v component to be decremented meter It calculates.
Method 200 includes receiving the data element (202) to remove from the preceding calculation window of adjustment.For example, data element Element 101 can be received.
Method 200 includes that received data element (203) are removed from data buffer zone.For example, receiving data element 101 Afterwards, data element can be removed from data buffer zone 121.
Method 200 includes calculation window before adjustment adjusts, comprising: removes received data element in calculation window before adjusting Element simultaneously adjusts calculation window scale counter (204).For example, data element 101 is removed from calculation window 122 before adjustment, so Calculation window 122 is transformed into calculation window 122A after adjustment before adjusting afterwards.The value of calculation window scale counter 118 can subtract 1.
Method 200 includes for be directly reduced the autocorrelative v (1≤v≤p) that computing relay is l a for calculation window after adjustment Component (205), comprising: each l data element on calculation window both sides and the data element (206) of removal after access adjustment；It visits The delay for asking calculation window before adjusting is the autocorrelative v component (207) of l；Mathematically from each of v component component Remove any contribution (208) for the data element being removed.Datail description is as follows.
After the autocorrelative v component for calculating specified delay l is directly reduced for calculation window after adjustment including access adjustment Each l data element on calculation window both sides and the data element (206) of removal.For example, if specified delay l=1, decrement are calculated Method 133 calculates Cd1When may have access to data element 101, the data element (data element 102) and calculation window of 121B in position The data element of 122 rightmosts.Similarly, if specified delay l=1, decrement algorithm 139 calculate CdvWhen may have access to data element Element 101, the data element of calculation window 122A rightmost after the data element (data element 102) of position 121B and adjustment.
Directly being reduced the autocorrelative v component that computing relay is l for calculation window after adjustment includes the preceding meter of access adjustment The delay for calculating window is a component (207) of autocorrelative v (1≤v≤p) of l.For example, if specified delay l=1, is reduced algorithm 133 may have access to the component Cd that delay is 11141, if specified delay l=2, decrement algorithm 133 may have access to the component that delay is 2 Cd1141…….Similarly, if specified delay l=1, decrement algorithm 139 may have access to the component Cd that delay is 1v145, if Specified delay l=2, decrement algorithm 139 may have access to the component Cd that delay is 2v 145……。
Directly being reduced for calculation window after adjustment and calculating the autocorrelative v component of specified delay l includes from v component Each component mathematically remove removal data element any contribution (208).For example, if specified delay l=2, directly Meet the component Cd that decrement computing relay is 21143 may include the component Cd that contribution removal module 133A is 2 from delay1Number in 141 Learn ground removal contribution 151.Similarly, directly it is reduced the component Cd that computing relay is 2v147 may include contribution removal module 139A The component Cd for being 2 from delayvMathematically removal contribution 181 in 145.Contribution 151 and 181 is from data element 101.
As shown in Fig. 1-1 and 1-2, component Cd1143 include contribution 152 (contributions from data element 102) and other Contribute 153 (contributions from data element 103-106).Similarly, component Cdv147 include that contribution 182 (comes from data element 102 contribution) and 183 (contributions from data element 103-106) of other contributions.
When auto-correlation is accessed and v < p (that is, not all component is all calculated by directly decrement), method 200 includes Decrement calculates the component (209) that w=p-v delay is l indirectly as needed.This w component is only when auto-correlation is accessed It can just calculate.For example, its members is directly reduced calculating with reference to Fig. 1-2, members are reduced calculating, computing module indirectly 163 can be based on component Ci1Except one or more components to be reduced computation module Ci indirectly1, computing module 164 can be with base In component CiwExcept one or more components to be reduced computation module Ci indirectlyw.The one or more component can be initially Change, directly decrement calculates, or is reduced indirectly calculated.
Method 200 includes calculating auto-correlation on an as-needed basis.When auto-correlation is accessed, auto-correlation can be by calculating base In the component that one or more decrements calculate；Otherwise only have v component that can be calculated by directly decrement.When auto-correlation is accessed, Method 200 includes that can be reduced the w component (209) that computing relay is l indirectly as needed.For example, in framework 100A, from phase The auto-correlation 193 of given delay can be calculated by closing module 192.In framework 100B, computing module 163 can be based on component Ci1Except One or more components be reduced indirectly calculate Ci1And computing module 164 can be based on component CiwExcept one or more components Decrement calculates Ci indirectlyw... ..., autocorrelation calculation module 192 can calculate the auto-correlation 193 (210) of given delay.Once given The auto-correlation of delay is calculated, and method 200 includes receiving next flow data element.
As the access 202-208 of more data elements can be repeated, 209-210, which can according to need, to be repeated.Example Such as, in component Cd1143 arrive component CdvAfter component in 147 ranges is calculated, data element 102 can be received (202).Once a new data element is received, method 200 includes that received data element is removed from data buffer zone (203).Method 200 includes removing received data element from calculation window and adjusting calculation window scale counter (204).Example Such as, it can be removed from buffer area after data element 102 is received.Calculation window 122A can remove data element 102 and handle The value of calculation window scale counter subtracts 1.
Method 200 includes directly being reduced calculating based on v component for adjusting preceding calculation window for calculation window after adjustment to prolong It is late the autocorrelative v component (205) of l, this includes l data element for accessing calculation window both sides and the data element of removal Plain (206), the v component (207) of calculation window before access adjusts, and mathematically removed from each of v component component Any contribution (208) of the data element of removal.For example, in specified delay such as l=1, being reduced algorithm with reference to 100A and 100B 133 can be used for directly being reduced the component Cd that computing relay is 1 for calculation window 122B1144 are based upon calculation window 122A calculating Delay be 1 component Cd1143(205).Decrement algorithm 133 may have access to the data element 102 of data element 103 and removal (206).Being reduced algorithm 133 may have access to the component Cd that delay is 11143(207).Directly it is reduced the component Cd that computing relay is 11 144 include the component Cd that contribution removal module 133A is 1 from delay1Mathematically 152 namely data element are contributed in removal in 143 102 contribution (208).Similarly, can be used for directly subtracting for calculation window 122B in specified delay such as l=1, decrement algorithm 139 Measure the component Cd that computing relay is 1vThe component Cd that 148 delays for being based upon calculation window 122A calculating are 1v147.Decrement is calculated Method 139 may have access to the data element 102 of data element 103 and removal.Being reduced algorithm 139 may have access to the component Cd that delay is 1v 147.Directly it is reduced the component Cd that computing relay is 1v148 include the component Cd that contribution removal module 139A is 1 from delayv 147 In mathematically removal contribution 182 namely data element 102 contribution.
As shown, delay is the component Cd of l1144 include the 153 (tributes from data element 103-106 of other contributions Offer), postpone the component Cd for lv148 include 183 (contributions from data element 103-106) of other contributions.
Method 200 includes being reduced the w component and auto-correlation that calculate given delay indirectly as needed.
Method 200 includes, and when i.e. only auto-correlation is accessed as needed, decrement calculates w group of given delay indirectly Part and auto-correlation.If auto-correlation is not accessed, including continuing as, next calculation window reception is next to be removed method 200 Data element (202).If auto-correlation is accessed, method 200 includes the w component that decrement calculates given delay indirectly (209), it is reduced the auto-correlation (210) of the given delay of Assembly calculation of the given delay of calculating based on one or more.
When next flow data element is received, component Cd1144 can be used to directly be reduced the next component Cd of calculating1, Component Cdv148 can be used to directly be reduced the next component Cd of calculatingv。
Method 200 includes resetting 211.Resetting 211 can be used to reset decrement autocorrelation calculation.When resetting 211 is 205 Or when calling after 210, calculation window scale counter and autocorrelative v (1≤v≤p) a component in given delay can be with It is initialised.For example, component Cd1141 can by being initialised with the data element in calculation window according to definition or initially Turn to the given value calculated.The latter can combine decrement autocorrelation calculation and iteration autocorrelation calculation or increment certainly Occur when relevant calculation.Component Cdv145 can initialize in a similar manner.
Fig. 3-1 illustrates the data element removed when decrement calculates auto-correlation on flow data from the left side of calculation window 300A Element.Calculation window 300A is the calculation window of a non-empty (assuming that comprising n data element xm+1,xm+2,…,xm+n).With the time Passage, the data element in calculation window 300A, for example, data element xm+1, then xm+2, then xm+3..., successively divided It is not removed from the left side of calculation window 300A.
Fig. 3-2 illustrates the data element accessed when decrement calculates auto-correlation on flow data from calculation window 300A.It adjusts The autocorrelative v component that the delay of calculation window is l after whole can pass through calculation window after being removed data element and adjusting Each l data element on both sides and v component for adjusting preceding calculation window are directly reduced calculating.If specified delay is 1, 1 data element of leftmost 1 data element of calculation window and rightmost can be interviewed after being removed data element and adjustment It asks.If specified delay is 2, it is removed after data element and adjustment leftmost 2 data elements of calculation window and most right 2 data elements on side can be accessed.If specified delay is l, calculation window is most left after being removed data element and adjustment The l data element on side and l data element of rightmost are accessed.So the delay given for one, data access amount It is reduced with calculation amount and is constant.Calculation window scale n is bigger, then the reduction of data access amount and calculation amount is more aobvious It writes.
Fig. 3-3 illustrates the data element removed on the right of calculation window 300B when decrement calculates auto-correlation on flow data Element.Calculation window 300B is the calculation window of a non-empty (assuming that comprising n data element xm+1,xm+2,…,xm+n).With the time Passage, the data element in calculation window 300B, for example, data element xm+n, then xm+n-1, then xm+n-2..., it is successive It is removed respectively from the right of calculation window 300B.
Fig. 3-4 illustrates the data accessed when decrement calculates auto-correlation on flow data from calculation window 300B.After adjustment The autocorrelative v component that the delay of calculation window is l can pass through calculation window both sides after being removed data element and adjusting L data element and adjustment before v component of calculation window be directly reduced calculating.If specified delay is 1, gone Except 1 data element of leftmost 1 data element of calculation window and rightmost can be accessed after data element and adjustment.Such as The specified delay of fruit is 2, is removed 2 of leftmost 2 data elements of calculation window and rightmost after data element and adjustment A data element can be accessed.If specified delay is l, it is removed the leftmost l of calculation window after data element and adjustment L data element of a data element and rightmost is accessed.So the delay given for one, data access amount and calculating Amount is reduced and is constant.Calculation window scale n is bigger, then the reduction of data access amount and calculation amount is more significant.
Fig. 4-1 illustrates autocorrelative definition.Assuming that having a time series or data flow: receiving data at time point 1 x1..., data x is received in time point m+1m+1, time point m+2 reception data xm+2..., time point m+n receives data xm+n..., And a calculation window X includes n data element of the time series or data flow: X={ xi| i=m+1, m+2 ..., m+ n}.And whenever auto-correlation needs are calculated a data element again after removing in calculation window.Assuming that computing system connects The input received indicates the data element x to remove from calculation window Xr(m+1≤r≤m+n) or its relevant information (such as rope Draw or address) and every receive such a data element or the relevant information just removal data element x from calculation window Xr, so After recalculate auto-correlation.Whenever a data element is removed, calculation window is considered as a new calculation window, new One wheel decrement, which calculates, to be started.Calculation window before removing data element, which is referred to as, adjusts preceding calculation window.After removing data element Calculation window be referred to as adjust after calculation window.In the decrement of a new round calculates, calculation window becomes after adjustment originally Calculation window before the adjustment that a new round calculates.
Autocorrelative calculated result is not only related with the value of each data element in calculation window, also with each data element The sequence of element is related.So position removal data element different from calculation window needs different processing.There are three types of situations. First, a data element x is removed from the Far Left of calculation windowr.Second, a data are removed from the rightmost of calculation window Element xr.Third removes a data element x from any position other than calculation window both sidesr.First two situation is often changing Occur when calculation window scale, when the third situation Chang You exceptional value occurs.The equation that three kinds of situations are used is different.It is limited to a piece Width only discusses first two situation here.In order to distinguish, defining the calculation window adjusted in the first situation is XI, second Kind situation is XII。
Equation 401 and 402 is that respectively kth wheel calculates the summation S for adjusting all data elements in preceding calculation window XkWith Average valueTraditional equation.Equation 403 is that the auto-correlation for adjusting that the given delay of preceding calculation window X is l is calculated for kth wheel ρ(k,l)Traditional equation.Equation 404 and 405 is that the respectively wheel of kth+1 calculates calculation window X after adjustmentIIn all data elements Summation SI k+1And average valueTraditional equation.As previously mentioned, when one number of calculation window Far Left removal before adjusting According to element xm+1(i.e. xr(r=m+1)) when, calculation window is defined as X after adjustmentI.Equation 406 is after taking turns calculating adjustment for kth+1 Calculation window XIGiven delay be l auto-correlation ρI (k+1,l)Traditional equation.As previously mentioned, when calculation window is gone from rightmost Except a data element xrWhen, calculation window is defined as X after adjustmentII.Equation 407 is to calculate window after taking turns calculating adjustment for kth+1 Mouth XIIGiven delay be l auto-correlation ρII (k+1,l)Traditional equation.
To show that three different decrement auto-correlation algorithms are provided as example how using component decrement calculating auto-correlation Son.It is begun to whenever the wheel that calculation window has a data variation stylish calculates.One and/or average value are calculated from phase The basic module of pass.Decrement, which calculates one and/or the equation of average value, all to be used by all examples decrement autocorrelation calculation algorithm Decrement component equation.
Fig. 4-2 shows first available equation of example decrement autocorrelation calculation algorithm (decrement algorithm 1).Equation 401 Initialization component S can be used, respectively, to 402kAnd/orAs data element xm+1The calculation window Far Left removal before adjusting When, decrement algorithm 1 includes component Sk+1OrSSk+1,SXk+1, and covXI (k+1,l)Decrement calculate, once component SXk+1With cocXI (k+1,l)It is calculated, auto-correlation ρI (k+1,l)It can be calculated based on them.Once component SkAnd/orIt can use, equation 412 Calculation window X after decrement calculating adjusts can be used, respectively, to 413IComponent Sk+1WithEquation 408 is computation module SSk Traditional equation.Once component SSkIt can use, equation 414 can be used for directly being reduced calculation window X after calculating adjustmentIComponent SSk+1.Equation 409 is computation module SXkTraditional equation.Once component Sk+1OrAnd SSk+1It can use, equation 415 can be used for Decrement calculates calculation window X after adjustment indirectlyIComponent SXk+1.Equation 410 is computation module covX(k,l)Traditional equation.One Denier component covX(k,l), SSk+1, SkOrAnd Sk+1OrIt can use, equation 416 can be used for after being directly reduced calculating adjustment calculating Window XIComponent covXI (k+1,l).412,413,415 and 416 separately include multiple equations but only one of them are needed to take respectively Certainly in whether and/or average value or both it is all available.Once component covXI (k+1,l)And SXk+1It is calculated, between equation 417 can be used for It connects decrement and calculates calculation window X after adjustmentIGiven delay be l component ρI (k+1,l).As data element xm+nIt is calculated before adjustment When window rightmost removes, decrement algorithm 1 includes component Sk+1OrSSk+1,SXk+1, and covXII (k+1,l)Decrement calculate, Once component SXk+1And covXII (k+1,l)It is calculated, auto-correlation ρII (k+1,l)It can be calculated based on them.Once component SkWith/ OrIt can use, equation 418 and 419 can be used, respectively, to decrement and calculate calculation window X after adjustmentIIComponent Sk+1WithEquation 408 be computation module SSkTraditional equation.Once component SSkIt can use, equation 420 can be used for after being directly reduced calculating adjustment calculating Window XIIComponent SSk+1.Equation 409 is computation module SXkTraditional equation.Once component Sk+1OrAnd SSk+1It can use, Equation 421 can be used for being reduced calculation window X after calculating adjustment indirectlyIIComponent SXk+1.Equation 410 is computation module covX(k,l) Traditional equation.Once component covX(k,l), SSk+1, SkOrAnd Sk+1OrIt can use, equation 422 can be used for directly being reduced Calculate calculation window X after adjustingIIComponent covXII (k+1,l).418,419,421 and 422 separately include multiple equations but difference only Need that one of them is depended on whether and/or average value or both is all available.Once component covXII (k+1,l)And SXk+1It is calculated, Equation 423 can be used for being reduced calculation window X after calculating adjustment indirectlyIIGiven delay be l component ρII (k+1,l)。
Fig. 4-3 shows second available equation of example decrement autocorrelation calculation algorithm (decrement algorithm 2).Work as data element Plain xm+1When calculation window Far Left removes before adjusting, decrement algorithm 2 includes component Sk+1OrSXk+1And covXI (k+1,l) Decrement calculate, once component SXk+1And covXI (k+1,l)It is calculated, auto-correlation ρI (k+1,l)It can be calculated based on them.Once Component SkAnd/orIt can use, equation 427 and 428 can be used, respectively, to directly be reduced calculation window X after calculating adjustsIComponent Sk+1WithEquation 424 is computation module SXkTraditional equation.Once component SXk,xkOr SkAnd Sk+1OrIt can use, equation 429 can be used for directly being reduced calculation window X after calculating adjustmentIComponent SXk+1.Equation 425 is computation module covX(k,l)Biography System equation.Equation 430 can be used for directly being reduced calculation window X after calculating adjustmentIComponent covXI (k+1,l)Once component covX(k,l), SkOrAnd Sk+1OrIt can use.427,428,429 and 430 separately include multiple equations but only need it respectively In one depend on whether and/or average value or both is all available.Once component covXI (k+1,l)And SXk+1It is calculated, equation 431 It can be used for being reduced calculation window X after calculating adjusts indirectlyIGiven delay be l component ρI (k+1,l).As data element xm+nFrom tune When whole preceding calculation window rightmost removes, decrement algorithm 2 includes component Sk+1OrSXk+1, and covXII (k+1,l)Decrement meter It calculates, once component SXk+1And covXII (k+1,l)It is calculated, auto-correlation ρII (k+1,l)It can be calculated based on them.Once component Sk And/orIt can use, equation 432 and 433 can be used, respectively, to directly be reduced calculation window X after calculating adjustsIIComponent Sk+1WithEquation 424 is computation module SXkTraditional equation.Once component SXk,xkOr SkAnd Sk+1OrIt can use, equation 434 It can be used for directly being reduced calculation window X after calculating adjustsIIComponent SXk+1.Equation 425 is computation module covX(k,l)Tradition Equation.Equation 435 can be used for directly being reduced calculation window X after calculating adjustmentIIComponent covXII (k+1,l)Once component covX(k,l), SkOrAnd Sk+1OrIt can use.432,433,434 and 435 separately include multiple equations but only need it respectively In one depend on whether and/or average value or both is all available.Once component covXII (k+1,l)And SXk+1It is calculated, equation 436 It can be used for being reduced calculation window X after calculating adjusts indirectlyIIGiven delay be l component ρII (k+1,l)。
Fig. 4-4 shows third example decrement autocorrelation calculation algorithm (decrement algorithm 3) available equation.Work as data element Plain xm+1When calculation window Far Left removes before adjusting, decrement algorithm 3 includes component Sk+1OrSXk+1And covXI (k+1,l) Decrement calculate, once component SXk+1And covXI (k+1,l)It is calculated, auto-correlation ρI (k+1,l)It can be calculated based on them.Equation 440 and 441 can be used, respectively, to directly be reduced calculation window X after calculating adjustsIComponent Sk+1WithEquation 437 is to calculate Component SXkTraditional equation.Equation 442 can be used for directly being reduced calculation window X after calculating adjustmentIComponent SXk+1Once component SXk, SkOrAnd Sk+1OrIt can use.Equation 438 is computation module covX(k,l)Traditional equation.Equation 443 can be used for Directly decrement calculates calculation window X after adjustmentIComponent covXI (k+1,l)Once component covX(k,l), SkOrAnd Sk+1OrIt can use.440,441,442 and 443 separately include multiple equations but only need one of them to depend on whether and/or put down respectively Mean value or both is all available.Equation 444 can be used for being reduced calculation window X after calculating adjustment indirectlyIGiven delay be l component ρI (k+1,l)Once component covXI (k+1,l)And SXk+1It is calculated.As data element xm+nThe calculation window rightmost removal before adjusting When, decrement algorithm 3 includes component Sk+1OrSXk+1And covXII (k+1,l)Decrement calculate, once component SXk+1With covXII (k+1,l)It is calculated, auto-correlation ρII (k+1,l)It can be calculated based on them.Once component SkAnd/orIt can use, equation 445 and 446 can be used, respectively, to directly be reduced calculation window X after calculating adjustsIIComponent Sk+1WithEquation 437 is to calculate Component SXkTraditional equation.Equation 447 can be used for directly being reduced calculation window X after calculating adjustmentIIComponent SXk+1Once component SXk, SkOrAnd Sk+1OrIt can use.Equation 438 is computation module covX(k,l)Traditional equation.Equation 448 can be used for Directly decrement calculates calculation window X after adjustmentIIComponent covXII (k+1,l)Once component covX(k,l), SkOrAnd Sk+1OrIt can use.445,446,447 and 448 separately include multiple equations but only need one of them to depend on whether and/or put down respectively Mean value or both is all available.Equation 449 can be used for being reduced calculation window X after calculating adjustment indirectlyIIGiven delay be l component ρII (k+1,l)Once component covXII (k+1,l)And SXk+1It is calculated.
For show decrement auto-correlation algorithm and they compared with traditional algorithm, three examples are given below.Use 3 The data of calculation window.For traditional algorithm, the calculating process of all 3 calculation windows is identical.For being reduced algorithm, the One calculation window carries out the initialization of two or more components, and second and third calculation window carry out decrement calculating.
Fig. 5-1, Fig. 5-2, Fig. 5-3 respectively illustrate first calculation window for a calculated examples, second meter Calculate window and third calculation window.Calculation window 502 includes 6 data elements of head of data flow 501: 8,3,6,1,9,2. Calculation window scale 503 (n) is 6.Calculation window 504 includes 5 data elements of data flow 501: 3,6,1,9,2.Calculate window Mouth scale 505 (n) is 5.Calculation window 506 includes 4 data elements of data flow 501: 6,1,9,2.Calculation window scale 507 It (n) is 4.The calculated examples assume that data element is removed from the Far Left of calculation window.Data flow 501 can be fluidisation Big data or flow data.
Calculation window 502,504, and the auto-correlation that 506 delays are 1 are calculated separately with traditional algorithm first.For calculation window The auto-correlation that 502 computing relays are 1:
It is that auto-correlation that calculation window computing relay that scale is 6 is 1 shares 2 divisions in the case where there is no any optimization, 11 Secondary multiplication, 14 sub-additions and 16 subtractions.
It is 1 from phase that identical equation and process, which can be used to 504 computing relay of calculation window of respectively Fig. 5-2 display, The auto-correlation closed and be 1 for Fig. 5-3 506 computing relay of calculation window shown.The delay of calculation window 504 for being 5 for scale is 1 Auto-correlation It is every in this calculating One includes 2 divisions, 9 multiplication, 11 sub-additions and 13 subtractions without optimization.The calculating window for being 4 for scale The auto-correlation that 506 delay of mouth is 1 Each of this calculating includes 2 divisions, 7 multiplication, 8 sub-additions and 10 subtractions without optimization.It passes It is that n usually requires completion 2 when giving the auto-correlation that delay is l that system algorithm calculates calculation window scale without optimization Secondary division, 2n-l multiplication, 3n- (l+3) sub-addition and 3n-2l subtraction.
The auto-correlation that the delay of calculation window 502,504 and 506 is 1 is calculated separately with decrement algorithm 1 below.
For scale be 6 502 computing relay of calculation window be 1 auto-correlation:
1. using equation 402,410,411, and 412 initialize the 1st component taken turns respectivelySS1,SX1, and covX(1,1):
2.
3. calculating the auto-correlation ρ of the 1st wheel with equation 413(1,1):
For 502 computing relay of calculation window be 1 auto-correlation when share 2 divisions, 13 multiplication, 14 additions and 16 subtract Method.
The auto-correlation for being 1 for 504 computing relay of calculation window:
1. using equation 415,416,417, and 418 are reduced the component for calculating the 2nd wheel respectivelySS2,SX2, and covX(2,1):
SS2=SS1+xm+1+4 2-xm+1 2=110+92-82=110+81-64=127
2. calculating the auto-correlation ρ of the 2nd wheel with equation 419(2,1):
Calculation window 504 shares 2 divisions when being reduced the auto-correlation that computing relay is 1,10 multiplication, and 8 additions and 7 subtract Method.
The auto-correlation for being 1 for 506 computing relay of calculation window:
1. using equation 415,416,417, and 418 are reduced the component for calculating the 3rd wheel respectivelySS3,SX3, and covX(3,1):
SS3=SS2+xm+1+4 2-xm+1 2=127+22-32=127+4-9=122
2. calculating the auto-correlation ρ of the 3rd wheel with equation 419(3,1):
For 505 computing relay of calculation window be 1 auto-correlation when share 2 divisions, 10 multiplication, 8 additions and 7 subtractions.
The auto-correlation that the delay of calculation window 502,504 and 506 is 1 is calculated separately with decrement algorithm 2 below.
The auto-correlation for being 1 for 502 computing relay of calculation window:
1. using equation 402,426, and the component of the 1st wheel of 427 initializationSX1, and covX(1,1):
2. calculating the auto-correlation ρ of the 1st wheel with equation 428(1,1):
For 502 computing relay of calculation window be 1 auto-correlation when share 2 divisions, 7 multiplication, 8 additions and 10 subtractions.
The auto-correlation for being 1 for 504 computing relay of calculation window:
1. using equation 430,431, and 432 are reduced the component for calculating the 2nd wheel respectivelySX2, and covX(2,1):
2. calculating the auto-correlation ρ of the 2nd wheel with equation 433(2,1):
Calculation window 504 shares 2 divisions when being reduced the auto-correlation that computing relay is 1,7 multiplication, and 10 additions and 7 subtract Method.
The auto-correlation for being 1 for 506 computing relay of calculation window:
1. using equation 430,431, and 432 are reduced the component for calculating the 3rd wheel respectivelySX3, and covX(3,1):
2. calculating the auto-correlation ρ of the 3rd wheel with equation 433(3,1):
Calculation window 506 shares 2 divisions when being reduced the auto-correlation that computing relay is 1,7 multiplication, and 10 additions and 7 subtract Method.
The auto-correlation that the delay of calculation window 502,504 and 506 is 1 is calculated separately with decrement algorithm 3 below.
The auto-correlation for being 1 for 502 computing relay of calculation window:
1. using equation 402,439, and the component of the 1st wheel of 440 initializationSX1, and covX(1,1):
2. calculating the auto-correlation ρ of the 1st wheel with equation 441(1,1):
For 502 computing relay of calculation window be 1 auto-correlation when share 2 divisions, 7 multiplication, 8 additions and 10 subtractions.
The auto-correlation for being 1 for 504 computing relay of calculation window:
1. using equation 443,444, and 445 are reduced the component for calculating the 2nd wheel respectivelySX2, and covX(2,1):
2. calculating the auto-correlation ρ of the 2nd wheel with equation 446(2,1):
Calculation window 504 shares 2 divisions, 7 multiplication, 9 additions and 8 subtractions when being reduced the auto-correlation that computing relay is 1.
The auto-correlation for being 1 for 506 computing relay of calculation window:
1. using equation 443,444, and 445 are reduced the component for calculating the 3rd wheel respectivelySX3, and covX(3,1):
2. calculating the auto-correlation ρ of the 3rd wheel with equation 446(3,1):
Calculation window 506 shares 2 divisions, 7 multiplication, 9 additions and 8 subtractions when being reduced the auto-correlation that computing relay is 1.
In three above example, average value be used to be reduced autocorrelation calculation.It is counted with may be alternatively used for auto-correlation decrement It calculates, only operand is different.In addition, data element is the Far Left removal of the calculation window before adjusting in above three example 's.When the rightmost that data element is the calculation window before adjusting removes, the similar only application of its calculating process is a different set of Equation.
Fig. 6-1 illustrate n=4 delay be 1 (i.e. l=1) when, traditional auto-correlation algorithm and decrement auto-correlation algorithm calculating The comparison of amount.As shown, the divide operations of any one decrement algorithm and traditional algorithm, multiplication operation, add operation and subtract Method operation is all similar.
Fig. 6-2 illustrates n=1, and when 000,000 delay is 1 (i.e. l=1), traditional auto-correlation algorithm and decrement auto-correlation are calculated The comparison of the calculation amount of method.As shown, many multiplication operations all fewer than traditional algorithm of any one decrement algorithm, add operation It is operated with subtraction.Decrement algorithm can be completed on single machine the data handled on thousands of computers are needed only.Greatly It is big to improve computational efficiency, computing resource is reduced, reduces and calculates equipment energy consumption, so that some real-time judge time serieses or fluidisation are big The given delay repeatability of data is from being very unlikely to become possibility.
The present invention can in the case where not departing from its thought or substantive characteristics by it is other it is specific in a manner of realize.This Shen The implementation that please be described is only as exemplary rather than restrictive in all aspects.Therefore, model of the invention It encloses by appended claims rather than the description of front indicates.With the meaning and scope of claim in claims All changes of equal value are included in the scope of the claims.

Claims (10)

1. it is a kind of, by calculating what the computing system that equipment is constituted was realized based on one or more, flow data is judged in real time The method of itself given delay repeatability, it is characterised in that:
By being calculation window before one of a data flow adjusts, initializing one based on a computing system for calculating equipment Calculation window scale counter n (n>1) and a given delay are autocorrelative more than two components of l (0<l<n), should Calculation window counter specifies the data element number that calculation window before the adjustment includes, the data of calculation window before the adjustment Element is stored in a buffer area of the computing system；
By based on calculate equipment the computing system, receive will from before the adjustment calculation window remove a data element or with Its relevant information；
It, will be from the number removed in calculation window before the adjustment from removal in the buffer area by the computing system based on calculating equipment According to element；
By adjusting calculation window before the adjustment, passing through based on the computing system for calculating equipment:
From the data element to be removed in calculation window before the adjustment；And
Correspondingly adjust the calculation window scale counter；
By based on the computing system for calculating equipment, the delay at least based on calculation window before the adjustment is autocorrelative the two of l A components above is reduced autocorrelative more than two components that computing relay is l for calculation window after the adjustment, and is being reduced It avoids accessing and usually dropping using all data elements in calculation window after the adjustment during calculating the two or more component Low data access delay improves computational efficiency, saves computing resource and reduces the computing system energy consumption；And
By being based on one or more the group of calculation window decrement calculating after the adjustment based on the computing system for calculating equipment Part generates the auto-correlation that delay is l for calculation window after the adjustment.
2. this method described in accordance with the claim 1 realized by the computing system, it is characterised in that: the reception one will be gone The data element or relative information removed include receive it is multiple will from before the adjustment calculation window remove data elements or Relative information, this method also further comprise for each of multiple data elements to be removed data element point Data element to be removed, calculation window before adjustment adjusts Jin Hang not be removed from the buffer area, decrement computing relay is l's Autocorrelative two or more component, and the auto-correlation that delay is l is generated for calculation window after adjustment.
3. this method realized according to claim 2 by the computing system, it is characterised in that: described is to calculate after adjusting Window, which generates, postpones the auto-correlation for being l when the auto-correlation is accessed.
4. the method described in accordance with the claim 3 realized by the computing system, it is characterised in that: described is to calculate window after adjusting It further comprises by being that calculation window is indirect after adjusting based on the computing system for calculating equipment that mouth, which generates the auto-correlation that delay is l, The autocorrelative one or more components that computing relay is l are reduced, it includes being based on that decrement, which calculates the one or more component, indirectly One or more components except the component to be calculated calculate separately the one or more component one by one.
5. a computing system, it is characterised in that:
One or more calculates equipment；
Each calculating equipment includes one or more processors；
One or more storage media；And
One or more computing modules, when the one or more computing module calculates at least one of equipment by one or more When calculating equipment execution, time series is judged in real time or fluidizes the repeatability of the given delay of big data itself, this or more A computing module is configured as:
It a. is one for being stored in a buffer area in the computing system in one or more storage equipment of a data flow Calculation window before adjusting, initializing a calculation window scale counter n (n>1) and a given delay is l (0<l<n) Autocorrelative more than two components, which specifies the data element that calculation window before the adjustment includes Number；
B. the data element or relative information that one will remove from calculation window before the adjustment are received；
It c. will be from the data element removed in calculation window before the adjustment from removal removal in the buffer area；
D. calculation window before the adjustment is adjusted, comprising:
From the data element to be removed in calculation window before the adjustment；And
Correspondingly adjust the calculation window scale counter；
E. autocorrelative more than two components that the delay at least based on calculation window before the adjustment is l, to be calculated after the adjustment Window is reduced autocorrelative more than two components that computing relay is l, and calculates the process of the two or more component in decrement In avoid access and usually reduce data access delay using all data elements in calculation window after adjustment, improve and calculate effect Rate saves computing resource and reduces the computing system energy consumption；And
F. it is based on one or more the component that calculation window decrement calculates after the adjustment, prolongs for calculation window generation after the adjustment It is late the auto-correlation of l.
6. the computing system according to claim 5, it is characterised in that: the one or more computing module, when their quilts When the one or more calculates the calculating equipment execution of at least one of equipment, b, c, d, e and f is performed a plurality of times.
7. the computing system according to claim 6, it is characterised in that: execute f calculation window after the adjustment Delay be l auto-correlation it is accessed when.
8. the computing system according to claim 7, it is characterised in that: described to generate delay for calculation window after the adjustment It is that calculation window is reduced computing relay indirectly after the adjustment is l from phase that auto-correlation for l, which further comprises by the computing system, The one or more components of pass, it includes based on one except the component to be calculated that decrement, which calculates the one or more component, indirectly Or multiple components calculate separately the one or more component one by one.
9. a computing system program product runs on one and includes one or more computing systems for calculating equipment, the calculating System includes one or more processors and one or more storage media, the computing system program product include a plurality of calculating Machine executable instructions are run when these calculate machine executable instructions by least one calculating equipment in the computing system When, so that the computing system executes the side for judging time series in real time or fluidizing the given delay repeatability of big data itself Method, it is characterised in that:
It is a buffer area being stored at least one storage media of the computing system an of data flow by the computing system One adjust before calculation window, initializing the given delay of a calculation window scale counter n (n > 1) and one is l (0 < l < n) autocorrelative more than two components, which specifies the data that calculation window before the adjustment includes Element number；
The data element or relative information that one will remove from calculation window before the adjustment are received by the computing system；
It will be from the data element removed in calculation window before the adjustment from removal in the buffer area by the computing system；
Calculation window before the adjustment is adjusted by the computing system, is passed through:
From the data element to be removed in calculation window before the adjustment；And
Correspondingly adjust the calculation window scale counter；
Autocorrelative more than two components that delay by the computing system at least based on calculation window before the adjustment is l, for this Autocorrelative more than two components that calculation window decrement computing relay is l after adjustment, and the two or more is calculated in decrement It avoids accessing during component and usually reduces data access delay using all data elements in calculation window after the adjustment, Computational efficiency is improved, computing resource is saved and reduces the computing system energy consumption；And
It is based on one or more the component that calculation window decrement calculates after the adjustment, to be counted after the adjustment by the computing system It calculates window and generates the auto-correlation that delay is l.
10. computing system program product as claimed in claim 9, it is characterised in that: one data element to be removed of the reception Plain or relative information includes receiving multiple data elements or relative to remove from calculation window before the adjustment Information, this method also further comprise that each of multiple data elements to be removed data element is carried out respectively from this Data element to be removed, calculation window before adjustment adjusts are removed in buffer area, decrement computing relay is autocorrelative the two of l A components above, and the auto-correlation that delay is l is generated for calculation window after adjustment.
CN201810526517.2A 2018-05-21 2018-05-21 Method for judging given delay repeatability of stream data in real time Active CN110515681B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810526517.2A CN110515681B (en) 2018-05-21 2018-05-21 Method for judging given delay repeatability of stream data in real time

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810526517.2A CN110515681B (en) 2018-05-21 2018-05-21 Method for judging given delay repeatability of stream data in real time

Publications (2)

Publication Number Publication Date
CN110515681A true CN110515681A (en) 2019-11-29
CN110515681B CN110515681B (en) 2024-04-09

Family

ID=68622308

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810526517.2A Active CN110515681B (en) 2018-05-21 2018-05-21 Method for judging given delay repeatability of stream data in real time

Country Status (1)

Country Link
CN (1) CN110515681B (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5329473A (en) * 1991-09-17 1994-07-12 Peter Cangiane Fast window/presum operation
CN1585290A (en) * 2003-08-18 2005-02-23 因芬尼昂技术股份公司 Sliding window
WO2014138894A1 (en) * 2013-03-15 2014-09-18 Imagine Communications Corp. Systems and methods for controlling branch latency within computing applications
CN104731691A (en) * 2013-12-18 2015-06-24 国际商业机器公司 Dynamically adjusting the number of replicas of a file according to the probability that the file will be accessed within a distributed file system
CN107092649A (en) * 2017-03-13 2017-08-25 浙江工业大学 A kind of topological replacement method of unaware towards real-time stream calculation
CN107423046A (en) * 2007-07-26 2017-12-01 起元技术有限责任公司 For handling method, system and the computer-readable medium of the calculating based on figure

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5329473A (en) * 1991-09-17 1994-07-12 Peter Cangiane Fast window/presum operation
CN1585290A (en) * 2003-08-18 2005-02-23 因芬尼昂技术股份公司 Sliding window
CN107423046A (en) * 2007-07-26 2017-12-01 起元技术有限责任公司 For handling method, system and the computer-readable medium of the calculating based on figure
WO2014138894A1 (en) * 2013-03-15 2014-09-18 Imagine Communications Corp. Systems and methods for controlling branch latency within computing applications
CN104731691A (en) * 2013-12-18 2015-06-24 国际商业机器公司 Dynamically adjusting the number of replicas of a file according to the probability that the file will be accessed within a distributed file system
CN107092649A (en) * 2017-03-13 2017-08-25 浙江工业大学 A kind of topological replacement method of unaware towards real-time stream calculation

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
刘华成;涂承胜;: "一种动态调度的延迟敏感流网络挖掘算法", 计算机工程与应用, no. 15 *

Also Published As

Publication number Publication date
CN110515681B (en) 2024-04-09

Similar Documents

Publication Publication Date Title
Trifunovic et al. Paradigm shift in big data supercomputing: dataflow vs. controlflow
US9959248B1 (en) Iterative simple linear regression coefficient calculation for big data using components
KR20200029387A (en) Data aggregation method for cache optimization and efficient processing
US9760539B1 (en) Incremental simple linear regression coefficient calculation for big data or streamed data using components
US20180270158A1 (en) Decremental autocorrelation calculation for big data using components
CN110515681A (en) A kind of method of the given delay repeatability of real-time judge flow data itself
CN109002925A (en) Traffic prediction method and apparatus
CN110515680A (en) A kind of method of the given delay repeatability of real-time judge big data itself
Bruneel et al. Analysis of a discrete-time single-server queue with an occasional extra server
CN110457340A (en) A kind of real-time method found big data itself and repeat rule
US10275488B1 (en) Incremental covariance calculation for big data or streamed data using components
CN110362365A (en) A kind of method of real-time prediction flow data variation tendency
CN109995834A (en) Massive dataflow processing method, calculates equipment and storage medium at device
CN112035521A (en) Method for judging self-set delay repeatability of streaming data in real time
Vasilyev et al. High-order non-uniform grid scheme for numerical analysis of queueing system with a small parameter
CN110363321A (en) A kind of method of real-time prediction big data variation tendency
US10310910B1 (en) Iterative autocorrelation calculation for big data using components
Lu et al. Analytical performance analysis of network-processor-based application designs
CN110909305B (en) Method for judging data flow change isotropy and degree thereof in real time
US10467326B1 (en) Decremental simple linear regression coefficient calculation for big data or streamed data using components
US10313250B1 (en) Incremental autocorrelation calculation for streamed data using components
CN112035520A (en) Method for judging self-set delay repeatability of streaming data in real time
WO2019136178A1 (en) Collaborative algorithm development, deployment, and tuning platform
CN111488380A (en) Method for judging asymmetry of stream data distribution in real time
CN112035791A (en) Method for judging self-given delay repeatability of big data in real time

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant