WO2018036641A1 - Device and method arranged for executing information processing on a data stream - Google Patents

Device and method arranged for executing information processing on a data stream Download PDF

Info

Publication number
WO2018036641A1
WO2018036641A1 PCT/EP2016/070249 EP2016070249W WO2018036641A1 WO 2018036641 A1 WO2018036641 A1 WO 2018036641A1 EP 2016070249 W EP2016070249 W EP 2016070249W WO 2018036641 A1 WO2018036641 A1 WO 2018036641A1
Authority
WO
WIPO (PCT)
Prior art keywords
level
stream
window
hierarchy
sub
Prior art date
Application number
PCT/EP2016/070249
Other languages
French (fr)
Inventor
Radu TUDORAN
Goetz BRASCHE
Xing ZHU
Original Assignee
Huawei Technologies Co., Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co., Ltd. filed Critical Huawei Technologies Co., Ltd.
Priority to CN202010872238.9A priority Critical patent/CN112148753B/en
Priority to CN201680083495.6A priority patent/CN109196494B/en
Priority to PCT/EP2016/070249 priority patent/WO2018036641A1/en
Publication of WO2018036641A1 publication Critical patent/WO2018036641A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2455Query execution
    • G06F16/24568Data stream processing; Continuous queries
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10Complex mathematical operations
    • G06F17/18Complex mathematical operations for evaluating statistical data, e.g. average values, frequency distributions, probability functions, regression analysis

Definitions

  • the present invention is directed to a device and method, both arranged for executing information processing on a data stream and both arranged for generation of partial results during the information processing. Further, the present invention relates to a correspondingly arranged computer program product and to a correspondingly arranged computer-readable medium.
  • Stream processing methods become more and more important in view of growing amount of systems and applications where data generated by one or more devices is communicated to a further device responsible for analysis of said data.
  • Stream processing is implemented in a plurality of use cases.
  • stream processing comprises analysis of one or more data streams from one or more sources providing the data streams.
  • the processing is executed in real time. Therefore, the speed of the processing and accuracy of the results are main issues.
  • one or more sensors may continuously report measurement data to a device processing and analyzing the measurement data.
  • These measurement data may, for example, be important for a control process, e.g. control of a system or production process. If the system, to which the measurement data relates, has to be kept in a particular state, it is important to analyze also partial results of the analysis of the measurement data for constantly monitoring the state of the system.
  • the object of the present invention is to provide a method and a device that enable an improved data stream processing, in which also partial results are determined.
  • the present invention provides a method and device that are arranged to generate partial results by executing information processing on a data stream in an efficient and accurate way.
  • a stream window hierarchy is implemented according to the present invention such that the required partial results are generated in an efficient and reliable way in each level of the stream window hierarchy.
  • Stream windows as such are known. They subdivide or discretize a data stream into non overlapping subsequent windows, i.e. parts of the data stream such that the stream processing is done in portions or in windows respectively, wherein each time the processing is executed with regard to the respective part of the data stream. Usually only one window is used. By use of the window, the part of the data stream is processed that is covered by the window, defined by a particular window size.
  • the window is slide along the stream to a next part of the stream which was not processed until then and which fits into the window, and the processing is started again with regard to the next part of the data stream.
  • the windows have a particular pre-determined size such that data streams of particular pre-determined length are processed.
  • the partial results represent intermediate results of a data processing process. Such results may be required and requested, for example, in time sensitive applications or systems.
  • the partial results may be output or provided at pre-defined time intervals while the computation or determining of the final result is executed.
  • the stream window hierarchy is divided into several levels, arranged in a certain order. With other words, the stream window hierarchy comprises a plurality of levels.
  • the device is configured such that a partial result, which has been generated in a level of the stream window hierarchy that is below a highest level of the stream window hierarchy, is provided for further processing in an overlying level of the stream window hierarchy, which is arranged over the level, in which the partial level has been generated. In this way, the efficiency of the calculation and generation of the partial results is increased. Additionally, the amount resources required for the execution of the data processing is kept small.
  • the device is configured such that each partial result is generated according to a pre-determined procedure.
  • a modular and flexible partial result generation is enabled because each level of the stream window hierarchy only the procedure to be used for generating corresponding partial results has to be told. This enables also a flexible exchange and modification of the procedures to be used for the generation of the partial results. In this way, the device is adaptable with regard to a plurality of possible use cases, applications and/or systems.
  • the device is configured such that each partial result is outputted at a respective time point.
  • the stream window hierarchy comprises a main window and one or more sub-windows of the main window.
  • each sub-window of the one or more sub-windows is associated with a respective time period and is arranged for processing information received during the respective time period. Also in this way, it is ensured that the further processing of the partial results becomes possible at desired time points. Also here, the reliability on the availability of the partial results at desired time points is increased.
  • a sub-window which is associated with a smallest time period, is arranged in the lowest level of the stream window hierarchy, and each sub- window, which is located in a level between the lowest level and the highest level of the stream window hierarchy, is associated with a time period that is greater than a time period of the sub-window located below the level.
  • the main window is arranged in the highest level of the stream window hierarchy. Also by this feature an efficient re-use of the partial results in the further levels of the hierarchy is enabled.
  • the stream window hierarchy levels below the highest level are arranged according to the lengths of the time periods, with which the sub- windows are associated. In this way, it is ensured that the partial results of the lower levels may be always reused at the higher levels. Thus, the efficiency of the determination of the partial and, subsequently, also of the final result is ensured.
  • a partial result, which has been generated in the highest level of the stream window hierarchy, is a final result.
  • the device comprises one or more processing entities configured to generate the partial results by executing the information processing on the data stream by implementing the stream window hierarchy and by generating the partial results at each level of the stream window hierarchy.
  • the device comprises at least one confidence value calculating entity configured to calculate a confidence value for a partial result generated in at least one level of the stream window hierarchy. In this way, the reliability of the partial results is provided additionally. This improves the further handling of the partial results by further devices or entities. For example, in control systems the knowledge on the reliability of the results is highly important. The calculation of the confidence value improves proper operation of further devices or entities that use the generated partial results.
  • the confidence value may be a value that indicates or measures the confidence of that the partial results are equal to or approximately the final result.
  • the confidence value may express a prediction about how far off the partial results are from the expected final value, or may indicate the amount of data processed from the whole part of the input stream data covered by the main window.
  • the confidence value is a percentage.
  • the present invention provides according to a second aspect also a method arranged to generate partial results by executing information processing on a data stream, wherein information, comprised in the data stream, is processed by implementing a stream window hierarchy, comprising a plurality of levels, and by generating partial results at each level of the stream window hierarchy.
  • the steps of the method correspond to the steps or activities executed by the device introduced above and explained in more detail below.
  • the effects described herein are achieved.
  • the present invention refers according to a third aspect to a computer program product comprising computer readable program code that is configured to cause a computing device to execute steps of the method introduced above and explained in more detail below.
  • the present invention relates according to a fourth aspect to a computer- readable recording medium configured to store therein said computer program product.
  • Fig. 1 shows an exemplary arrangement of a device, arranged to generate partial results by executing information processing on a data stream, according to an embodiment of the present invention.
  • Fig. 2 shows an exemplary data processing, executed through all levels of a stream window hierarchy, according to an embodiment of the present invention.
  • Fig. 3 shows a further exemplary arrangement of the device, arranged to generate partial results by executing information processing on a data stream, according to an embodiment of the present invention.
  • Fig. 4 shows a further exemplary arrangement of the device, arranged to generate partial results by executing information processing on a data stream, according to an embodiment of the present invention.
  • Fig. 5 shows a flow-diagram of steps for generating partial results by executing information processing on a data stream, according to an embodiment of the present invention.
  • Fig. 1 shows an exemplary arrangement of a device 100, arranged to generate partial results by executing information processing on a data stream, according to an embodiment of the present invention.
  • the device 100 implements a stream window hierarchy 103 with n levels of windows, wherein n is an integer and is greater than or equal to two.
  • n windows 103 1, 103 2, ..., 103_n are provided in the stream window hierarchy 103, wherein each window is provided in a particular level i, 1 ⁇ i ⁇ n of the stream window hierarchy 103.
  • the device 100 uses the structure of the stream window hierarchy and, particularly, the windows 103 1, 103 2, ..., 103_n for generating the partial results.
  • the window 103_n of the highest hierarchy level n represents a main window, i.e. a window that is slid over an input data stream 102 for processing the input data stream 201.
  • the device 100 comprises an input buffer 101, in which a received input data stream 102 is stored for processing via the stream window hierarchy 103.
  • the main window 103_n is slid over the input data stream 102 such that a part of the data stream covered by the main window 103 is processed via the stream window hierarchy.
  • the main window 103_n is slid again over the input data stream 102, and a next part of the input data stream 102 covered by the main window 103 is processed then.
  • the main window 103 has a pre-determined length.
  • the input data stream 102 is divided into parts that have said pre-determined length.
  • the parts of the input data stream 102, divided by the main window 103, do not overlap.
  • this part of the input data stream 102 is processed within the stream window hierarchy 103.
  • the windows 103 1, 103 2, ..., 103_n of the stream window hierarchy 103 are arranged such that the windows of the levels below the highest level n represent sub-windows of the main window 103_n. Thus, they are smaller, that is, have a shorter length, than the main window 103_n, and are slid over the part of the input data stream that is covered by the main window 103_n.
  • each of the sub-windows 103 1, 103 2 is associated with a respective time period and is arranged for generation of partial results 104 1, 104 2, 104_n-l of the data that occurred during the respective time period.
  • 103 2 is configured to be slid over the part of the input data stream 102 determined by the main window 103_n. Each time, a partial result 104 1, 104 2, 104_n-l has been determined, the sub-window 103 1, 103 2 is slid again over the part of the input data stream 102, determined by the main window 103_n, for determining or generating the next partial result
  • the sub-windows 103 1, 103 2 are arranged according to the lengths of the time periods, with which the sub-windows 103 1, 103 2 are associated.
  • the sub-window 103 1 is associated with the smallest time period and is located in the lowest level of the stream window hierarchy 103.
  • the sub-window 103 2 is associated with a time period that is greater than the time period of the sub-window 103 1 but that is smaller than the time period of the sub-window of the next level (i.e. level 3), and so on.
  • the processing is started at the lowest level 1 of the stream window hierarchy 103.
  • the sliding of the main window 103_n is executed by the device 100. Further, the device 100 uses the sub-windows 103 1, 103 2 (including the other possible sub-windows not shown in Fig. 1) for
  • the processing is started at the lowest level 1 of the stream window hierarchy 103.
  • the part of the input stream 102, covered by the main window 103_n is used as an input for executing the processing at the lower level of the stream window hierarchy 103.
  • the device 100 slides the sub-window 103 1 over the part of the input stream 102, covered by the main window 103_n and determines partial results 104 1 with regard to data covered by the sub-window 103 1.
  • the device 100 determines a first partial result 104 1 of the data provided during the time period associated with the sub-window 103 1. After the device 100 has generated the first partial result 104 1 on data covered by the sub-window 103 1 of the lowest level 1 of the stream window hierarchy 103, the partial result 104 1 is output (see the arrow leading away from the stream window hierarchy) and can be used (e.g., analyzed) by further devices or entities. Additionally, the partial result 104 1 is provided to the next level 2, where it is reused within the processing, executed by the device 100 with regard to the sub-window 103 2 of the next level.
  • the sub-window 103 1 is then slid again over the part of the input data stream 102, determined by the main window 103_n, and the processing is executed again by the device 100. This sliding of the sub-window 103 1 and the corresponding processing of the data covered by the sub-window 103 1 are executed until the whole part of the input data stream 102, covered by the main window 103_n, has been processed via the sub-window 103 1 of the lowest level 1.
  • the processing in the next level 2 is done by sliding the sub-window 103 2 over the part of the input data stream 102, determined by the main window 103_n.
  • the processing in the next level 1 is started as soon as all partial results have been received from the previous level, i.e. level 1, that refer to data that is covered by the sub-window 103 2.
  • the sub-window 103 2 is associated with a time period that is greater than the time period of the sub-window 103 1 of the lower level 1.
  • the partial results 104 1 generated during the previous processing at the level below the level of the sub-window 103 2 are used for data processing at the level of the sub-window 103 2.
  • the data covered by the sub-window 103 2 comprise the data covered by the sub-window 103 1 of the lower level 1.
  • the partial result 104 2 generated by the data processing, is output (see the arrow leading away from the stream window hierarchy) to a further device or entity (e.g., for further analysis). Additionally, the partial result 104 2 is provided to the next level 3, where it is reused within the processing, executed by the device 100 with regard to the sub-window of the next level 3.
  • This sliding of the sub-window 103 2 and the corresponding processing of the data covered by the sub-window 103 2 are executed until the whole part of the input data stream 102, covered by the main window 103_n, has been processed via the sub-window 103 2 of the level 2.
  • the processing in each level of the next levels of the stream window hierarchy 103 is done in similar ways by sliding the corresponding sub-windows over the part of the input data stream 102, covered by the main window 103_n, and by processing the data, covered by the corresponding sub-window.
  • partial results, determined in the level, below the level, in which the data processing is executed, are reused.
  • the processing in each of the next levels is started as soon as all partial results have been received from the previous level that refer to data that is covered by the corresponding sub-window.
  • the device 100 executes the data processing at the highest level n with regard to the main window 103_n by using all partial results provided after the data processing in the level n-1 below the highest level n.
  • the data processing in each of the levels 103 1, 103 2, ..., 103_n is done, according to an embodiment, by executing the same processing function. For example, if sensor data is analyzed, the same analysis function is executed in each of the levels 103 1, 103 2, ..., 103 n.
  • Fig. 2 data processing is executed with regard to four time moments T, T-1, T-2, and T-3 because, according to the embodiment of Fig. 2, the main window covers data of the input data stream 102 that has been received at four subsequent time moments.
  • the processing is executed with regard to the main window at three time moments because the sub-window is set such that it covers data received at two time moments.
  • the processing starts with the newest received data, i.e. with data received at the newest time moments covered by the main window.
  • the sub-window in the level 1 is set such that it starts at the end of the main window comprising the newest data.
  • the newest data has been provided at the time moment T.
  • the sub-window of the level 1 covers two time moments, i.e. the time period of the sub-window of the level 1 refers to or comprises two time moments.
  • the data received at the time moments T and T-1 is processed by the device 100 according to the sub-window of the level 1.
  • the partial result, obtained after the processing with regard to the time period T to T-1 is provided to the further level, which in the embodiment of Fig. 2 is the highest level comprising the main window.
  • the sub-window of the level 1 is slid one time moment further.
  • the time period T-1 to T-2 is considered via the sub-window of level 1, and the data received at the time moments T-1 and T-2 is processed by the device 100 according to the sub-window of the level 1.
  • the partial result, obtained after the processing with regard to the time period T-1 to T-2, is provided to the further level, which in the embodiment of Fig. 2 is the highest level comprising the main window.
  • the sub-window of the level 1 is slid again one time moment further.
  • the time period T-2 to T-3 is considered via the sub-window of level 1.
  • the data received at the time moments T-2 and T-3 is processed by the device 100 according to the sub-window of the level 1.
  • the partial result, obtained after the processing with regard to the time period T-2 to T-3, is provided to the further level, which in the embodiment of Fig. 2 is the highest level comprising the main window.
  • the data processing via the sub-window of level 1 terminates because the sub-window of level 1 has been shifted up to the end of the part of the input data stream 102 that is covered by the main window.
  • all partial results required for the processing at the highest level via the main window are present.
  • data processing with regard to the main window is executed by the device 100.
  • the partial results provided at the previous level, here level 1 are used, and a final result is provided with regard to the data received in the time period T to T-3 at the highest level.
  • the time period T to T-3 refers to data that is covered by the main window in the data stream.
  • Fig. 3 shows a further exemplary arrangement of the device 100.
  • the device 100 comprises further a confidence value calculating entity 300.
  • the confidence value calculating entity is configured to receive a partial result 104_i generated at a level i of the stream window hierarchy 103, wherein 1 ⁇ i ⁇ n, and to determine a confidence value for the partial result 104_i.
  • the confidence value is a value that: indicates or measures the confidence of that the partial result 104_i is equal to or is approximately the final result; expresses a prediction about how far off the partial result 104_i is from the expected final value; or indicates the amount of data processed from the whole part of input stream data 102 covered by the main window 103_n.
  • the confidence value is a percentage. Which of the above-listed three indications the confidence value expresses, can be chosen or set by the user of the device 100. Alternatively, a custom predictor for determining the confidence value by the confidence value calculating entity 300 can be specified via a correspondingly arranged interface (not shown).
  • the output 301_i of the confidence value calculating entity 300 is a tuple of the calculated confidence value and the partial result 104_i. According to an embodiment of the present invention, one confidence value calculating entity 300 per level i (1 ⁇ i ⁇ n) and, thus, per sub-window 103 1, 103 2 is provided.
  • one confidence value calculating entity 300 for more than one level i is provided.
  • a plurality of confidence value calculating entities 300 may be provided in device 100.
  • the confidence value calculating entity 300 is configured offline, i.e. before the calculations via the stream window hierarchy 103 are executed. In this way, an application- specific confidence value calculating entity 300 can be arranged in the device 100 before the calculations via the stream window hierarchy 103 are executed.
  • the confidence value calculating entity 300 is configurable to be aware of the level i, from which the partial result 104_i, is received, and thus is aware on the amount of data processed for generating the partial result 104_i. This enables the confidence value calculating entity 300 to estimate the significance of the partial result 104_i as the confidence value of the partial result 104_i, said significance being determined in view of the amount of data used for calculating the partial result 104_i and in view of the amount of data covered by the main window 103_n.
  • the confidence value calculating entity 300 for calculating the confidence value for a partial result 104_i.
  • the confidence value calculating entity 300 may be interconnected in the stream window hierarchy 103 and may be hard- wired for providing notification on the confidence value for the partial result 104_i with a specific time frequency.
  • the output 301_i of the confidence calculating entity 300 is provided to a further device or entity for further analysis, while the partial result 104_i is provided also to the next level i+1 of the stream window hierarchy 300 for further processing.
  • Fig. 4 shows a further exemplary arrangement of the device 100. The embodiment of Fig. 4 can be combined with any of the embodiments described here.
  • the device 100 comprises additionally a hierarchy generating entity 400 that is configured to generate the stream window hierarchy 103.
  • the hierarchy generating entity 400 is operated, according to an embodiment, offline, i.e. before the partial result generation by use of the stream window hierarchy 103 has been started.
  • the hierarchy generating entity 300 uses at least one of the following input parameters for generating the stream window hierarchy 300:
  • said settings comprise, for example, the function(s), according to which the confidence values will be calculated by the at least one confidence value calculating entity 300.
  • the hierarchy generating entity 400 is configured to create automatically stream window hierarchy 103 comprising windows 103 1, 103 2, 103_n based on the input parameters. According to an embodiment, the hierarchy generating entity 400 generates and adjusts the windows 103 1, 103 2, 103_n according to a known stream processing engine such as Flink, Spark or any other appropriate stream processing engine.
  • a known stream processing engine such as Flink, Spark or any other appropriate stream processing engine.
  • the number n of hierarchy levels of the stream depends on a number of partial results that are awaited during a processing of a main window 103_n.
  • the number of partial results that are awaited during a processing of a main window 103_n is set, for example, by a user with regard to the use case or application with regard to which the stream window hierarchy 103 will be used and implemented.
  • the hierarchy generating entity 400 configures also the sub-windows 103 1, 103 2 etc.
  • the hierarchy generating entity 400 executes at least one of the following: sets the size of the sub-windows 103 1, 103 2 etc. and, thus, the respective time periods of each of the sub-windows 103 1, 103 2 etc.;
  • the hierarchy generating entity 400 comprises a user interface, via which user settings with regard to the stream window hierarchy 103 may be
  • the hierarchy generating entity 400 comprises also an application programming interface (API) for setting and/or determining parameters, routine definitions, protocols and/or tools, wherein said parameters, routine definitions, protocols and/or tools are then used for implementing the stream window hierarchy 103 and the functions of the windows 103 1, 103 2, ..., 103_n of the stream window hierarchy 103.
  • API application programming interface
  • Fig. 5 shows a flow-diagram of steps for generating partial results by executing information processing on a data stream, according to an embodiment of the present invention.
  • the steps of Fig. 5 represent, in particular, activities of the device 100 and are executed by the device 100.
  • the stream window hierarchy 103 is implemented, wherein the use of the stream window hierarchy 103 is started with regard to a received input data stream 102.
  • partial results 104 1, 104 2, 104_n-l are generated at each level i (1 ⁇ i ⁇ n) of the stream window hierarchy, as described above.
  • the present invention is directed to a device and to a method, both arranged to generate partial results by executing information processing on a data stream, wherein the device is configured to process information comprised in the data stream by implementing a stream window hierarchy, comprising a plurality of levels, and by generating partial results at each level of the stream window hierarchy.
  • a stream window hierarchy comprising a plurality of levels
  • the present invention an effective and resource-saving approach for generating partial results is achieved.
  • the partial results are, particularly, generated for further use by further devices or entities.
  • the efficient, fast and reliable generation of the partial results ensures also a fast and proper further operation of the further devices or entities.
  • the invention has been described in conjunction with various embodiments herein.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Mathematical Analysis (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Algebra (AREA)
  • Operations Research (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Software Systems (AREA)
  • Evolutionary Biology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computational Linguistics (AREA)
  • User Interface Of Digital Computer (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

The present invention is directed to a device and to a method, both arranged to generate partial results by executing information processing on a data stream, wherein the device is configured to process information comprised in the data stream by implementing a stream window hierarchy, comprising a plurality of levels, and by generating partial results at each level of the stream window hierarchy.

Description

DEVICE AND METHOD ARRANGED FOR EXECUTING INFORMATION
PROCESSING ON A DATA STREAM
TECHNICAL FIELD
The present invention is directed to a device and method, both arranged for executing information processing on a data stream and both arranged for generation of partial results during the information processing. Further, the present invention relates to a correspondingly arranged computer program product and to a correspondingly arranged computer-readable medium.
BACKGROUND
Stream processing methods become more and more important in view of growing amount of systems and applications where data generated by one or more devices is communicated to a further device responsible for analysis of said data. Stream processing is implemented in a plurality of use cases. Particularly, stream processing comprises analysis of one or more data streams from one or more sources providing the data streams. The processing is executed in real time. Therefore, the speed of the processing and accuracy of the results are main issues.
Often, not only a final result of a stream processing process but also intermediate results or partial results respectively are of interest. Use cases are present where such intermediate or partial result give an overview or an assessment of a given situation or of a state of a system, for example.
For example, one or more sensors may continuously report measurement data to a device processing and analyzing the measurement data. These measurement data may, for example, be important for a control process, e.g. control of a system or production process. If the system, to which the measurement data relates, has to be kept in a particular state, it is important to analyze also partial results of the analysis of the measurement data for constantly monitoring the state of the system.
Here, several scenarios are present where continuous presentation of partial results is useful and sometimes even important (e.g., to maintain a correct operation of a system). Therefore, devices and methods are required that are able to execute stream processing in a fast and effective way and that provide also fast and reliable partial results.
SUMMARY
The object of the present invention is to provide a method and a device that enable an improved data stream processing, in which also partial results are determined.
The present invention provides a method and device that are arranged to generate partial results by executing information processing on a data stream in an efficient and accurate way. In particular, a stream window hierarchy is implemented according to the present invention such that the required partial results are generated in an efficient and reliable way in each level of the stream window hierarchy. Stream windows as such are known. They subdivide or discretize a data stream into non overlapping subsequent windows, i.e. parts of the data stream such that the stream processing is done in portions or in windows respectively, wherein each time the processing is executed with regard to the respective part of the data stream. Usually only one window is used. By use of the window, the part of the data stream is processed that is covered by the window, defined by a particular window size. After the processing of the part of the stream covered by the window, the window is slide along the stream to a next part of the stream which was not processed until then and which fits into the window, and the processing is started again with regard to the next part of the data stream. The windows have a particular pre-determined size such that data streams of particular pre-determined length are processed.
The object of the present invention is achieved by the solution provided in the enclosed independent claims. Advantageous implementations of the present invention are further defined in the respective dependent claims, in the present description, and/or in the appended figures.
According to a first aspect, a device is provided, which is arranged to generate partial results by executing information processing on a data stream, wherein the device is configured to process information comprised in the data stream by implementing a stream window hierarchy, comprising a plurality of levels i, i = 1, ... n, where n > 2, and by generating partial results at each level of the stream window hierarchy.
The partial results represent intermediate results of a data processing process. Such results may be required and requested, for example, in time sensitive applications or systems. The partial results may be output or provided at pre-defined time intervals while the computation or determining of the final result is executed. The stream window hierarchy is divided into several levels, arranged in a certain order. With other words, the stream window hierarchy comprises a plurality of levels.
By implementing the stream window hierarchy for generating partial results an effective and fast instrument for providing partial results and for calculating or determining the final result is provided. The efficiency of calculation of partial and final results is a crucial issue when data streams are processed. The processing of large data is resource and time consuming. On the other side, a fast provision of the partial and final results is expected because said partial and final results are used for execution of further processes, in which further decisions (e.g. on controlling, on further steps to be executed in a system or application) are met based on said results. By the stream window hierarchy, the efficiency problem is solved. Additionally, the efficiency problem is solved without increasing the amount of resources required for processing the information of a data stream. Rather, the amount of the required resources is kept small.
In a first possible implementation form of the device according to the first aspect, the device is configured such that a partial result, which has been generated in a level of the stream window hierarchy that is below a highest level of the stream window hierarchy, is provided for further processing in an overlying level of the stream window hierarchy, which is arranged over the level, in which the partial level has been generated. In this way, the efficiency of the calculation and generation of the partial results is increased. Additionally, the amount resources required for the execution of the data processing is kept small.
In a second possible implementation form of the device according to the first aspect as such or according to the first implementation form of the first aspect, the device is configured such that in each level i = 2, ..., n of the stream window hierarchy an underlying level partial result, which has been generated in an underlying level j = 1, ..., n-1 of the stream window hierarchy, is used for generating a respective partial result in a respective level i of the stream window hierarchy, wherein the underlying level j is a level that is arranged under the respective level i. A level i with i = 2, ..., n denotes a level which is not the lowest level of the stream window hierarchy.
In this way, an efficient and resource-saving computation and generation of partial results is enabled.
In a third possible implementation form of the device according to the first aspect as such or according to any of the preceding implementation forms of the first aspect, the device is configured such that each partial result is generated according to a pre-determined procedure. Thus, a modular and flexible partial result generation is enabled because each level of the stream window hierarchy only the procedure to be used for generating corresponding partial results has to be told. This enables also a flexible exchange and modification of the procedures to be used for the generation of the partial results. In this way, the device is adaptable with regard to a plurality of possible use cases, applications and/or systems.
In a fourth possible implementation form of the device according to the first aspect as such or according to any of the preceding implementation forms of the first aspect, the device is configured such that each partial result is outputted at a respective time point. Thus, it is ensured that the further processing of the partial results becomes possible at desired time points. Further, the reliability on the availability of the partial results at desired time points is increased. In a fifth possible implementation form of the device according to the first aspect as such or according to any of the preceding implementation forms of the first aspect, the stream window hierarchy comprises a main window and one or more sub-windows of the main window. Thus a simple architecture of the stream window hierarchy is provided that supports the desired efficient determination of the partial results.
In a sixth possible implementation form of the device according to the fifth implementation form of the first aspect, each sub-window of the one or more sub-windows is associated with a respective time period and is arranged for processing information received during the respective time period. Also in this way, it is ensured that the further processing of the partial results becomes possible at desired time points. Also here, the reliability on the availability of the partial results at desired time points is increased.
In a seventh possible implementation form of the device according to the sixth implementation form of the first aspect, a sub-window, which is associated with a smallest time period, is arranged in the lowest level of the stream window hierarchy, and each sub- window, which is located in a level between the lowest level and the highest level of the stream window hierarchy, is associated with a time period that is greater than a time period of the sub-window located below the level. In this way, an efficient re-use of the partial results in the further levels of the hierarchy is enabled.
In an eight possible implementation form of the device according to one of the fifth to seventh implementation forms of the first aspect, the main window is arranged in the highest level of the stream window hierarchy. Also by this feature an efficient re-use of the partial results in the further levels of the hierarchy is enabled.
In a ninth possible implementation form of the device according to one of the sixth to eight implementation forms of the first aspect, the stream window hierarchy levels below the highest level are arranged according to the lengths of the time periods, with which the sub- windows are associated. In this way, it is ensured that the partial results of the lower levels may be always reused at the higher levels. Thus, the efficiency of the determination of the partial and, subsequently, also of the final result is ensured.
In a tenth possible implementation form of the device according to the first aspect as such or according to any of the preceding implementation forms of the first aspect, a partial result, which has been generated in the highest level of the stream window hierarchy, is a final result.
In an eleventh possible implementation form of the device according to the first aspect as such or according to any of the preceding implementation forms of the first aspect, the device comprises one or more processing entities configured to generate the partial results by executing the information processing on the data stream by implementing the stream window hierarchy and by generating the partial results at each level of the stream window hierarchy. In a twelfth possible implementation form of the device according to the first aspect as such or according to any of the preceding implementation forms of the first aspect, the device comprises at least one confidence value calculating entity configured to calculate a confidence value for a partial result generated in at least one level of the stream window hierarchy. In this way, the reliability of the partial results is provided additionally. This improves the further handling of the partial results by further devices or entities. For example, in control systems the knowledge on the reliability of the results is highly important. The calculation of the confidence value improves proper operation of further devices or entities that use the generated partial results.
The confidence value may be a value that indicates or measures the confidence of that the partial results are equal to or approximately the final result. The confidence value may express a prediction about how far off the partial results are from the expected final value, or may indicate the amount of data processed from the whole part of the input stream data covered by the main window. Preferably, the confidence value is a percentage.
The present invention provides according to a second aspect also a method arranged to generate partial results by executing information processing on a data stream, wherein information, comprised in the data stream, is processed by implementing a stream window hierarchy, comprising a plurality of levels, and by generating partial results at each level of the stream window hierarchy. Generally, the steps of the method correspond to the steps or activities executed by the device introduced above and explained in more detail below. Also by the method, the effects described herein are achieved. Further, the present invention refers according to a third aspect to a computer program product comprising computer readable program code that is configured to cause a computing device to execute steps of the method introduced above and explained in more detail below.
Furthermore, the present invention relates according to a fourth aspect to a computer- readable recording medium configured to store therein said computer program product. BRIEF DESCRIPTION OF THE DRAWINGS
The above-described aspects and implementation forms of the present invention will be explained in the following description of specific embodiments in relation to the enclosed drawings, in which:
Fig. 1 shows an exemplary arrangement of a device, arranged to generate partial results by executing information processing on a data stream, according to an embodiment of the present invention.
Fig. 2 shows an exemplary data processing, executed through all levels of a stream window hierarchy, according to an embodiment of the present invention. Fig. 3 shows a further exemplary arrangement of the device, arranged to generate partial results by executing information processing on a data stream, according to an embodiment of the present invention.
Fig. 4 shows a further exemplary arrangement of the device, arranged to generate partial results by executing information processing on a data stream, according to an embodiment of the present invention.
Fig. 5 shows a flow-diagram of steps for generating partial results by executing information processing on a data stream, according to an embodiment of the present invention.
DETAILED DESCRIPION OF THE EMBODIMENTS
Generally, it has to be noted that all arrangements, devices, modules, components, models, elements, units, entities, and means and so forth described in the present application could be implemented by software or hardware elements or any kind of combination thereof. All steps which are performed by the various entities described in the present application as well as the functionality described to be performed by the various entities are intended to mean that the respective entity is adapted to or configured to perform the respective steps and functionalities. Even if in the following description of the specific embodiments, a specific functionality or step to be performed by a general entity is not reflected in the description of a specific detailed element of the entity which performs the specific step or functionality, it should be clear for a skilled person that these methods and functionalities can be implemented in respective hardware or software elements, or any kind of combination thereof. Further, the method of the present invention and its various steps are embodied in the functionalities of the various described apparatus elements. Moreover, any of the embodiments and features of any of the embodiments, described herein, may be combined with each other, unless a combination is explicitly excluded.
Fig. 1 shows an exemplary arrangement of a device 100, arranged to generate partial results by executing information processing on a data stream, according to an embodiment of the present invention. The device 100 implements a stream window hierarchy 103 with n levels of windows, wherein n is an integer and is greater than or equal to two. According to the present embodiment, n windows 103 1, 103 2, ..., 103_n are provided in the stream window hierarchy 103, wherein each window is provided in a particular level i, 1 < i < n of the stream window hierarchy 103. The device 100 uses the structure of the stream window hierarchy and, particularly, the windows 103 1, 103 2, ..., 103_n for generating the partial results.
The window 103_n of the highest hierarchy level n represents a main window, i.e. a window that is slid over an input data stream 102 for processing the input data stream 201. According to the present embodiment, the device 100 comprises an input buffer 101, in which a received input data stream 102 is stored for processing via the stream window hierarchy 103. Thus, the main window 103_n is slid over the input data stream 102 such that a part of the data stream covered by the main window 103 is processed via the stream window hierarchy. After the processing of the part of the data stream, the main window 103_n is slid again over the input data stream 102, and a next part of the input data stream 102 covered by the main window 103 is processed then. The main window 103 has a pre-determined length. Thus, the input data stream 102 is divided into parts that have said pre-determined length. The parts of the input data stream 102, divided by the main window 103, do not overlap. After it has been determined, by sliding the main window 103_n, which part of the input data stream 102 will be processed next, this part of the input data stream 102 is processed within the stream window hierarchy 103. The windows 103 1, 103 2, ..., 103_n of the stream window hierarchy 103 are arranged such that the windows of the levels below the highest level n represent sub-windows of the main window 103_n. Thus, they are smaller, that is, have a shorter length, than the main window 103_n, and are slid over the part of the input data stream that is covered by the main window 103_n.
According to the present embodiment, each of the sub-windows 103 1, 103 2 is associated with a respective time period and is arranged for generation of partial results 104 1, 104 2, 104_n-l of the data that occurred during the respective time period. Each sub-window 103 1,
103 2 is configured to be slid over the part of the input data stream 102 determined by the main window 103_n. Each time, a partial result 104 1, 104 2, 104_n-l has been determined, the sub-window 103 1, 103 2 is slid again over the part of the input data stream 102, determined by the main window 103_n, for determining or generating the next partial result
104 1, 104 2, 104_n-l . Further, according to the present embodiment, the sub-windows 103 1, 103 2 are arranged according to the lengths of the time periods, with which the sub-windows 103 1, 103 2 are associated. The sub-window 103 1 is associated with the smallest time period and is located in the lowest level of the stream window hierarchy 103. The sub-window 103 2 is associated with a time period that is greater than the time period of the sub-window 103 1 but that is smaller than the time period of the sub-window of the next level (i.e. level 3), and so on.
After the determination, by sliding the main window 103_n, of the part of the input data stream 102 that will be processed via the stream window hierarchy 103, the processing is started at the lowest level 1 of the stream window hierarchy 103. The sliding of the main window 103_n is executed by the device 100. Further, the device 100 uses the sub-windows 103 1, 103 2 (including the other possible sub-windows not shown in Fig. 1) for
determining the partial results and, subsequently, the final result. The processing is started at the lowest level 1 of the stream window hierarchy 103. For this purpose, the part of the input stream 102, covered by the main window 103_n, is used as an input for executing the processing at the lower level of the stream window hierarchy 103. The device 100 slides the sub-window 103 1 over the part of the input stream 102, covered by the main window 103_n and determines partial results 104 1 with regard to data covered by the sub-window 103 1.
Particularly, the device 100 determines a first partial result 104 1 of the data provided during the time period associated with the sub-window 103 1. After the device 100 has generated the first partial result 104 1 on data covered by the sub-window 103 1 of the lowest level 1 of the stream window hierarchy 103, the partial result 104 1 is output (see the arrow leading away from the stream window hierarchy) and can be used (e.g., analyzed) by further devices or entities. Additionally, the partial result 104 1 is provided to the next level 2, where it is reused within the processing, executed by the device 100 with regard to the sub-window 103 2 of the next level. The sub-window 103 1 is then slid again over the part of the input data stream 102, determined by the main window 103_n, and the processing is executed again by the device 100. This sliding of the sub-window 103 1 and the corresponding processing of the data covered by the sub-window 103 1 are executed until the whole part of the input data stream 102, covered by the main window 103_n, has been processed via the sub-window 103 1 of the lowest level 1.
Similarly, the processing in the next level 2 is done by sliding the sub-window 103 2 over the part of the input data stream 102, determined by the main window 103_n. The processing in the next level 1 is started as soon as all partial results have been received from the previous level, i.e. level 1, that refer to data that is covered by the sub-window 103 2.
The sub-window 103 2 is associated with a time period that is greater than the time period of the sub-window 103 1 of the lower level 1. Thus, when the device 100 executes data processing with regard to the sub-window 103 2, the partial results 104 1 generated during the previous processing at the level below the level of the sub-window 103 2 are used for data processing at the level of the sub-window 103 2. This is possible because the data covered by the sub-window 103 2 comprise the data covered by the sub-window 103 1 of the lower level 1. Similarly to the data processing in level 1 , each time the sub-window 103 2 of the level 2 has been slid over the part of the sub-stream, covered by the main window 103_n, and the corresponding data processing has been done by the device 100, the partial result 104 2, generated by the data processing, is output (see the arrow leading away from the stream window hierarchy) to a further device or entity (e.g., for further analysis). Additionally, the partial result 104 2 is provided to the next level 3, where it is reused within the processing, executed by the device 100 with regard to the sub-window of the next level 3.
This sliding of the sub-window 103 2 and the corresponding processing of the data covered by the sub-window 103 2 are executed until the whole part of the input data stream 102, covered by the main window 103_n, has been processed via the sub-window 103 2 of the level 2.
The processing in each level of the next levels of the stream window hierarchy 103 is done in similar ways by sliding the corresponding sub-windows over the part of the input data stream 102, covered by the main window 103_n, and by processing the data, covered by the corresponding sub-window. For processing of the data, partial results, determined in the level, below the level, in which the data processing is executed, are reused. The processing in each of the next levels is started as soon as all partial results have been received from the previous level that refer to data that is covered by the corresponding sub-window.
At the highest level n, all partial results 104_n-l of the level n-1 below the highest level n are provided for data processing in the highest level n. Thus, the device 100 executes the data processing at the highest level n with regard to the main window 103_n by using all partial results provided after the data processing in the level n-1 below the highest level n.
The data processing in each of the levels 103 1, 103 2, ..., 103_n is done, according to an embodiment, by executing the same processing function. For example, if sensor data is analyzed, the same analysis function is executed in each of the levels 103 1, 103 2, ..., 103 n.
This data processing, executed by the device 100 through all levels of a stream window hierarchy, is exemplary shown in Fig. 2 of the present application. For sake of conciseness and for sake of clarity, only two levels are shown in Fig. 2, wherein below (similarly to Fig. 1) the upper level n of the stream window hierarchy is shown and above, the lower level 1 of the stream window hierarchy is shown. According to the embodiment of Fig. 2, n = 2.
In Fig. 2, data processing is executed with regard to four time moments T, T-1, T-2, and T-3 because, according to the embodiment of Fig. 2, the main window covers data of the input data stream 102 that has been received at four subsequent time moments. The processing is executed with regard to the main window at three time moments because the sub-window is set such that it covers data received at two time moments.
According to the embodiment of Fig. 2, the processing starts with the newest received data, i.e. with data received at the newest time moments covered by the main window. At the moment T-1 the sub-window in the level 1 is set such that it starts at the end of the main window comprising the newest data. According to the present embodiment, the newest data has been provided at the time moment T. The sub-window of the level 1 covers two time moments, i.e. the time period of the sub-window of the level 1 refers to or comprises two time moments. Thus, the data received at the time moments T and T-1 is processed by the device 100 according to the sub-window of the level 1. The partial result, obtained after the processing with regard to the time period T to T-1, is provided to the further level, which in the embodiment of Fig. 2 is the highest level comprising the main window.
At the moment T-2, the sub-window of the level 1 is slid one time moment further. Thus, at the moment T-2, the time period T-1 to T-2 is considered via the sub-window of level 1, and the data received at the time moments T-1 and T-2 is processed by the device 100 according to the sub-window of the level 1. The partial result, obtained after the processing with regard to the time period T-1 to T-2, is provided to the further level, which in the embodiment of Fig. 2 is the highest level comprising the main window.
At the moment T-3, the sub-window of the level 1 is slid again one time moment further. Thus, at the moment T-3, the time period T-2 to T-3 is considered via the sub-window of level 1. Thus, the data received at the time moments T-2 and T-3 is processed by the device 100 according to the sub-window of the level 1. The partial result, obtained after the processing with regard to the time period T-2 to T-3, is provided to the further level, which in the embodiment of Fig. 2 is the highest level comprising the main window.
At the moment T-3, the data processing via the sub-window of level 1 terminates because the sub-window of level 1 has been shifted up to the end of the part of the input data stream 102 that is covered by the main window. Further, after the provision of the partial result, obtained after the processing with regard to the time period T-2 to T-3, all partial results required for the processing at the highest level via the main window are present. Thus, data processing with regard to the main window is executed by the device 100. In said data processing, the partial results provided at the previous level, here level 1, are used, and a final result is provided with regard to the data received in the time period T to T-3 at the highest level. The time period T to T-3 refers to data that is covered by the main window in the data stream. Thus, if the main window is slid further over the data stream, the next data covered by the main window will be data received at time period T+l to T+4, comprising data received at four time moments again.
Fig. 3 shows a further exemplary arrangement of the device 100. According to the embodiment of Fig. 3, which can be combined with any of the embodiments described herein, the device 100 comprises further a confidence value calculating entity 300. The confidence value calculating entity is configured to receive a partial result 104_i generated at a level i of the stream window hierarchy 103, wherein 1 < i < n, and to determine a confidence value for the partial result 104_i. The confidence value is a value that: indicates or measures the confidence of that the partial result 104_i is equal to or is approximately the final result; expresses a prediction about how far off the partial result 104_i is from the expected final value; or indicates the amount of data processed from the whole part of input stream data 102 covered by the main window 103_n. According to an embodiment, the confidence value is a percentage. Which of the above-listed three indications the confidence value expresses, can be chosen or set by the user of the device 100. Alternatively, a custom predictor for determining the confidence value by the confidence value calculating entity 300 can be specified via a correspondingly arranged interface (not shown). The output 301_i of the confidence value calculating entity 300 is a tuple of the calculated confidence value and the partial result 104_i. According to an embodiment of the present invention, one confidence value calculating entity 300 per level i (1 < i < n) and, thus, per sub-window 103 1, 103 2 is provided.
According to a further embodiment of the present invention, one confidence value calculating entity 300 for more than one level i is provided. Thus, also a plurality of confidence value calculating entities 300 may be provided in device 100.
According to an embodiment, the confidence value calculating entity 300 is configured offline, i.e. before the calculations via the stream window hierarchy 103 are executed. In this way, an application- specific confidence value calculating entity 300 can be arranged in the device 100 before the calculations via the stream window hierarchy 103 are executed.
Furthermore, the confidence value calculating entity 300 is configurable to be aware of the level i, from which the partial result 104_i, is received, and thus is aware on the amount of data processed for generating the partial result 104_i. This enables the confidence value calculating entity 300 to estimate the significance of the partial result 104_i as the confidence value of the partial result 104_i, said significance being determined in view of the amount of data used for calculating the partial result 104_i and in view of the amount of data covered by the main window 103_n.
Generally, any appropriate statistic calculation or artificial intelligence approach can be implemented by the confidence value calculating entity 300 for calculating the confidence value for a partial result 104_i. The confidence value calculating entity 300 may be interconnected in the stream window hierarchy 103 and may be hard- wired for providing notification on the confidence value for the partial result 104_i with a specific time frequency.
The output 301_i of the confidence calculating entity 300 is provided to a further device or entity for further analysis, while the partial result 104_i is provided also to the next level i+1 of the stream window hierarchy 300 for further processing. Fig. 4 shows a further exemplary arrangement of the device 100. The embodiment of Fig. 4 can be combined with any of the embodiments described here. According to the present embodiment, the device 100 comprises additionally a hierarchy generating entity 400 that is configured to generate the stream window hierarchy 103. The hierarchy generating entity 400 is operated, according to an embodiment, offline, i.e. before the partial result generation by use of the stream window hierarchy 103 has been started.
According to the present embodiment, the hierarchy generating entity 300 uses at least one of the following input parameters for generating the stream window hierarchy 300:
• one or more moment notifications, each moment notification indicating a time
moment at which a partial result should be outputted;
• a parameter indicating whether an output stream is generated for each specified time moment at which a partial result is output or only one stream needs to be created for partial results output at more than one specified time moments;
• a processing function, that is executed at the sub-windows for calculating and
generating the partial results;
• settings for implementing at least one confidence value calculating entity 300,
wherein said settings comprise, for example, the function(s), according to which the confidence values will be calculated by the at least one confidence value calculating entity 300.
The hierarchy generating entity 400 is configured to create automatically stream window hierarchy 103 comprising windows 103 1, 103 2, 103_n based on the input parameters. According to an embodiment, the hierarchy generating entity 400 generates and adjusts the windows 103 1, 103 2, 103_n according to a known stream processing engine such as Flink, Spark or any other appropriate stream processing engine.
The number n of hierarchy levels of the stream depends on a number of partial results that are awaited during a processing of a main window 103_n. The number of partial results that are awaited during a processing of a main window 103_n is set, for example, by a user with regard to the use case or application with regard to which the stream window hierarchy 103 will be used and implemented. Starting from the number of the awaited partial results during a processing of the main window 103_n, the hierarchy generating entity 400 configures also the sub-windows 103 1, 103 2 etc. For example, the hierarchy generating entity 400 executes at least one of the following: sets the size of the sub-windows 103 1, 103 2 etc. and, thus, the respective time periods of each of the sub-windows 103 1, 103 2 etc.;
determines the re-usage options for partial results received at a level i of the hierarchy 103 from a level i-1 below the level i; and/or determines the connections of the windows 103 1, 103 2, ..., 103_n of the hierarchy 103, wherein the determined connections comprise, for example, also connections from the windows 103 1, 103 2, ..., 103_n to further devices or entities.
According to an embodiment, the hierarchy generating entity 400 comprises a user interface, via which user settings with regard to the stream window hierarchy 103 may be
communicated. According to a further embodiment, the hierarchy generating entity 400 comprises also an application programming interface (API) for setting and/or determining parameters, routine definitions, protocols and/or tools, wherein said parameters, routine definitions, protocols and/or tools are then used for implementing the stream window hierarchy 103 and the functions of the windows 103 1, 103 2, ..., 103_n of the stream window hierarchy 103.
Fig. 5 shows a flow-diagram of steps for generating partial results by executing information processing on a data stream, according to an embodiment of the present invention. The steps of Fig. 5 represent, in particular, activities of the device 100 and are executed by the device 100. In step 501, the stream window hierarchy 103 is implemented, wherein the use of the stream window hierarchy 103 is started with regard to a received input data stream 102. In step 502 partial results 104 1, 104 2, 104_n-l are generated at each level i (1 < i< n) of the stream window hierarchy, as described above.
Thus, as shown above, the present invention is directed to a device and to a method, both arranged to generate partial results by executing information processing on a data stream, wherein the device is configured to process information comprised in the data stream by implementing a stream window hierarchy, comprising a plurality of levels, and by generating partial results at each level of the stream window hierarchy. By use of the present invention, an effective and resource-saving approach for generating partial results is achieved. The partial results are, particularly, generated for further use by further devices or entities. The efficient, fast and reliable generation of the partial results ensures also a fast and proper further operation of the further devices or entities. The invention has been described in conjunction with various embodiments herein. However, other variations to the enclosed embodiments can be understood and effected by those skilled in the art and practicing the claimed invention, from a study of the drawings, the disclosure and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

A device arranged to generate partial results by executing information processing on a data stream, wherein the device is configured to process information comprised in the data stream by implementing a stream window hierarchy, comprising a plurality of levels i, i = 1, ... n, where n > 2, and by generating partial results at each level of the stream window hierarchy.
The device according to claim 1, wherein the device is configured such that a partial result, which has been generated in a level of the stream window hierarchy that is below a highest level of the stream window hierarchy, is provided for further processing in an overlying level of the stream window hierarchy, which is arranged over the level, in which the partial level has been generated.
The device according to claim 1 or 2, wherein the device is configured such that in each level i = 2, n of the stream window hierarchy, an underlying level partial result, which has been generated in an underlying level j = 1, n-1 of the stream window hierarchy, is used for generating a respective partial result in a respective level i of the stream window hierarchy, wherein the underlying level j is a level that is arranged under the respective level i.
The device according to any one of the preceding claims, wherein the device is configured such that each partial result is generated according to a pre-determined procedure.
The device according to any one of the preceding claims, wherein the device is configured such that each partial result is outputted at a respective time point.
The device according to any one of the preceding claims, wherein the stream window hierarchy comprises a main window and one or more sub-windows of the main window.
The device according to claim 6, wherein each sub-window of the one or more sub- windows is associated with a respective time period and is arranged for processing information received during the respective time period.
8. The device according to claim 7, wherein a sub-window, which is associated with a smallest time period, is arranged in the lowest level of the stream window hierarchy, and each sub-window, which is located in a level between the lowest level and the highest level of the stream window hierarchy, is associated with a time period that is greater than a time period of the sub-window located below the level.
9. The device according to any one of claims 6 to 8, wherein the main window is arranged in the highest level of the stream window hierarchy.
10. The device according to any one of claims 7 to 9, wherein the stream window hierarchy levels below the highest level are arranged according to the lengths of the time periods, with which the sub-windows are associated.
11. The device according to any one of the preceding claims, wherein a partial result, which has been generated in the highest level of the stream window hierarchy, is a final result.
12. The device according to any one of the preceding claims, wherein the device comprises one or more processing entities configured to generate the partial results by executing the information processing on the data stream by implementing the stream window hierarchy and by generating the partial results at each level of the stream window hierarchy.
13. The device according to any one of the preceding claims, wherein the device comprises at least one confidence value calculating entity configured to calculate a confidence value for a partial result generated in at least one level of the stream window hierarchy.
14. A method arranged to generate partial results by executing information processing on a data stream, wherein information, comprised in the data stream, is processed by implementing a stream window hierarchy, comprising a plurality of levels, and by generating partial results at each level of the stream window hierarchy.
15. A computer program product comprising computer readable program code that is configured to cause a computing device to execute steps of method of claim 14.
6. A computer-readable recording medium configured to store therein a computer program product, which comprises a computer readable program code that is configured to cause a computing device to execute steps of method of claim 14.
PCT/EP2016/070249 2016-08-26 2016-08-26 Device and method arranged for executing information processing on a data stream WO2018036641A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CN202010872238.9A CN112148753B (en) 2016-08-26 2016-08-26 Apparatus and method for performing information processing on data stream
CN201680083495.6A CN109196494B (en) 2016-08-26 2016-08-26 Apparatus and method for performing information processing on data stream
PCT/EP2016/070249 WO2018036641A1 (en) 2016-08-26 2016-08-26 Device and method arranged for executing information processing on a data stream

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/EP2016/070249 WO2018036641A1 (en) 2016-08-26 2016-08-26 Device and method arranged for executing information processing on a data stream

Publications (1)

Publication Number Publication Date
WO2018036641A1 true WO2018036641A1 (en) 2018-03-01

Family

ID=56853606

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2016/070249 WO2018036641A1 (en) 2016-08-26 2016-08-26 Device and method arranged for executing information processing on a data stream

Country Status (2)

Country Link
CN (2) CN109196494B (en)
WO (1) WO2018036641A1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023077451A1 (en) * 2021-11-05 2023-05-11 中国科学院计算技术研究所 Stream data processing method and system based on column-oriented database

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070294247A1 (en) * 2006-06-20 2007-12-20 International Business Machines Corporation Identifying optimal multi-scale patterns in time-series streams

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CA2565756A1 (en) * 2006-10-26 2008-04-26 Daniel Langlois Interface system
US20080120283A1 (en) * 2006-11-17 2008-05-22 Oracle International Corporation Processing XML data stream(s) using continuous queries in a data stream management system
US8181003B2 (en) * 2008-05-29 2012-05-15 Axis Semiconductor, Inc. Instruction set design, control and communication in programmable microprocessor cores and the like
CN103024815A (en) * 2011-09-21 2013-04-03 宏碁股份有限公司 Method for transmitting data in wireless network system
US8880493B2 (en) * 2011-09-28 2014-11-04 Hewlett-Packard Development Company, L.P. Multi-streams analytics
CN102315974B (en) * 2011-10-17 2014-08-27 北京邮电大学 Stratification characteristic analysis-based method and apparatus thereof for on-line identification for TCP, UDP flows
CN103246430A (en) * 2013-04-24 2013-08-14 深圳市同洲电子股份有限公司 Terminal and method for managing multiple windows
CN103488757A (en) * 2013-09-25 2014-01-01 浪潮电子信息产业股份有限公司 Clustering feature equivalent histogram maintaining method based on cloud computing

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070294247A1 (en) * 2006-06-20 2007-12-20 International Business Machines Corporation Identifying optimal multi-scale patterns in time-series streams

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
XIAO-BO FAN ET AL: "MRST-An Efficient Monitoring Technology of Summarization on Stream Data", JOURNAL OF COMPUTER SCIENCE AND TECHNOLOGY, vol. 22, no. 2, 17 April 2007 (2007-04-17), pages 190 - 196, XP019510132, ISSN: 1860-4749, DOI: 10.1007/S11390-007-9025-7 *

Also Published As

Publication number Publication date
CN112148753A (en) 2020-12-29
CN109196494B (en) 2020-09-11
CN112148753B (en) 2024-01-16
CN109196494A (en) 2019-01-11

Similar Documents

Publication Publication Date Title
US11216741B2 (en) Analysis apparatus, analysis method, and non-transitory computer readable medium
US20160231738A1 (en) Information processing apparatus and analysis method
US11165799B2 (en) Anomaly detection and processing for seasonal data
JP2016076218A5 (en)
JPWO2016079972A1 (en) Factor analysis device, factor analysis method and program, and factor analysis system
US10139840B2 (en) System, device, and method for fluid dispensing control
US11755605B2 (en) Controlling technical equipment through quality indicators using parameterized batch-run monitoring
US9922143B2 (en) Method and control for carrying out a calculation of a data-based function model
RU2020111006A (en) VERIFICATION DEVICE, INFORMATION PROCESSING METHOD AND PROGRAM
CA2965283C (en) Systems and methods of subject state change notification
US10635078B2 (en) Simulation system, simulation method, and simulation program
US8793106B2 (en) Continuous prediction of expected chip performance throughout the production lifecycle
WO2018036641A1 (en) Device and method arranged for executing information processing on a data stream
WO2016026653A1 (en) Detection of consistent steady-state behavior in process plants
US9235489B2 (en) Testing scheduling system and method
CN111224830A (en) Data monitoring method and device, Internet of things network element and computer readable storage medium
EP3665895A1 (en) Data processing device, data analyzing device, data processing system and method for processing data
CN114187552A (en) Method and system for monitoring power environment of machine room
US10380290B1 (en) Systems and methods for parallel transient analysis and simulation
WO2016143337A1 (en) Information processing device, information processing method, and recording medium
US10962966B2 (en) Equipment process monitoring system with automatic configuration of control limits and alert zones
Yulianto et al. The MSOF-DTW Method for Checking Timeseries Similarities
JP7258253B1 (en) Normal model generation program, normal model generation device, and normal model generation method
CN111523452B (en) Method and device for detecting human body position in image
US20240235949A1 (en) Queue-based orchestration simulation

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16760028

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16760028

Country of ref document: EP

Kind code of ref document: A1