PL423219A1 - Method for extraction and transformation of stream-oriented measuring data, using the parallel computing - Google Patents
Method for extraction and transformation of stream-oriented measuring data, using the parallel computingInfo
- Publication number
- PL423219A1 PL423219A1 PL423219A PL42321917A PL423219A1 PL 423219 A1 PL423219 A1 PL 423219A1 PL 423219 A PL423219 A PL 423219A PL 42321917 A PL42321917 A PL 42321917A PL 423219 A1 PL423219 A1 PL 423219A1
- Authority
- PL
- Poland
- Prior art keywords
- block
- module
- input
- data
- mask
- Prior art date
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Sposób ekstrakcji i transformacji strumieniowych danych pomiarowych wykorzystujący obliczenia równoległe polega na tym, że w module formowania bloków (2) formuje się wejściowe dane pomiarowe (1) w bloki danych (3), następnie poddaje się je operacji wyznaczania statystyk wstępnych w module statystyk wstępnych (4) i otrzymuje się: tablicę liczebności (5), tablicę pozycji (6) oraz tablicę indeksów (7), z których korzysta moduł grupowania (8) wykorzystujący blok danych (3), w wyniku jego działania powstaje pogrupowany blok danych (9), który jest przekazywany na wejście modułu transpozycji (10), który produkuje transponowany blok pogrupowanych danych (11) stanowiący wejście dla modułu różnicy (12), w wyniku którego powstaje blok różnic (13) wykorzystywany przez moduł generowania maski ID (14) oraz moduł generowania maski czasu (15), na których wyjściach powstają odpowiednio: blok maski ID (16) oraz blok maski czasu (17), które są podawane na wejście modułu iloczynu (18), który produkuje blok maski(19) wykorzystywany przez moduł nakładania maski (20), wykorzystujący dodatkowo blok różnic (13) do wytworzenia bloku podzielonych danych (21), który stanowi wejście dla modułu sumującego (22), w wyniku działania którego powstaje blok przedziałów (23), wykorzystywany przez moduł rekoncyliacji (24), produkujący blok przedziałów rekoncyliacji (25) stanowiący wejście dla modułu filtracji (26), na którego wyjściu powstaje blok przefiltrowanych danych (27) wykorzystywany przez moduł wykrywania trendów (28), produkujący blok trendów (29), który podawany jest na wejście modułu zapisu (30), po czym zapisuje dane do bazy danych (31).The method of extraction and transformation of streaming measurement data using parallel calculations is that in the block forming module (2) the input measurement data (1) is formed into data blocks (3), then it is subjected to the operation of determining preliminary statistics in the module of preliminary statistics ( 4) and the following is obtained: number table (5), position table (6) and index table (7) used by the grouping module (8) using a data block (3), as a result of its operation a grouped data block is created (9) , which is passed to the input of the transposition module (10), which produces the transposed block of grouped data (11) constituting the input for the difference module (12), resulting in a block of differences (13) used by the ID mask generation module (14) and the module generating a time mask (15), on which outputs respectively the ID mask block (16) and the time mask block (17), which are fed to the input the product module (18) that produces the mask block (19) used by the mask overlay module (20), which additionally uses the difference block (13) to create the split data block (21), which is the input for the summation module (22) as a result the action which creates a block of compartments (23), used by the reconciliation module (24), producing a block of reconciliation compartments (25) constituting the input for the filtration module (26), at the output of which a block of filtered data (27) is created, used by the trend detection module (28) ), producing a trend block (29), which is fed to the input of the write module (30), and then writes the data to the database (31).
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PL423219A PL233157B1 (en) | 2017-10-20 | 2017-10-20 | Method for extraction and transformation of stream-oriented measuring data, using the parallel computing |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PL423219A PL233157B1 (en) | 2017-10-20 | 2017-10-20 | Method for extraction and transformation of stream-oriented measuring data, using the parallel computing |
Publications (2)
Publication Number | Publication Date |
---|---|
PL423219A1 true PL423219A1 (en) | 2019-04-23 |
PL233157B1 PL233157B1 (en) | 2019-09-30 |
Family
ID=66167918
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PL423219A PL233157B1 (en) | 2017-10-20 | 2017-10-20 | Method for extraction and transformation of stream-oriented measuring data, using the parallel computing |
Country Status (1)
Country | Link |
---|---|
PL (1) | PL233157B1 (en) |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101533407A (en) * | 2009-04-10 | 2009-09-16 | 中国科学院软件研究所 | Method for detecting exceptional data in ETL flow |
CN101587477A (en) * | 2008-05-23 | 2009-11-25 | 阿里巴巴集团控股有限公司 | Method and system for automatically maintaining ETL modules |
CN105930384A (en) * | 2016-04-14 | 2016-09-07 | 南京理工大学 | Sensing cloud data storage system based on Hadoop system and implementation method thereof |
US20170220654A1 (en) * | 2016-02-03 | 2017-08-03 | Wipro Limited | Method for automatically generating extract transform load (etl) codes using a code generation device |
CN107169023A (en) * | 2017-04-07 | 2017-09-15 | 广东精点数据科技股份有限公司 | Data lineage analysis system and method based on sql semantic automatic analysis |
-
2017
- 2017-10-20 PL PL423219A patent/PL233157B1/en unknown
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101587477A (en) * | 2008-05-23 | 2009-11-25 | 阿里巴巴集团控股有限公司 | Method and system for automatically maintaining ETL modules |
CN101533407A (en) * | 2009-04-10 | 2009-09-16 | 中国科学院软件研究所 | Method for detecting exceptional data in ETL flow |
US20170220654A1 (en) * | 2016-02-03 | 2017-08-03 | Wipro Limited | Method for automatically generating extract transform load (etl) codes using a code generation device |
CN105930384A (en) * | 2016-04-14 | 2016-09-07 | 南京理工大学 | Sensing cloud data storage system based on Hadoop system and implementation method thereof |
CN107169023A (en) * | 2017-04-07 | 2017-09-15 | 广东精点数据科技股份有限公司 | Data lineage analysis system and method based on sql semantic automatic analysis |
Also Published As
Publication number | Publication date |
---|---|
PL233157B1 (en) | 2019-09-30 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
PH12019500288A1 (en) | Video encoding and decoding | |
Li et al. | Sequence parallelism: Long sequence training from system perspective | |
WO2018111116A3 (en) | Method for handling multidimensional data | |
RU2011104986A (en) | METHOD AND DEVICE FOR GROUPING TRACKS AND SUBSETS OF TRACES | |
JP2014041486A5 (en) | ||
MX2024006502A (en) | Three-dimensional data encoding method, three-dimensional data decoding method, three-dimensional data encoding device, and three-dimensional data decoding device. | |
MY193732A (en) | Apparatus and method for efficient synthesis of sinusoids and sweeps by employing spectral patterns | |
GB201310419D0 (en) | Seismic trace attribute | |
Weiss et al. | Invariant manifolds for nonsmooth systems with sliding mode | |
MX2021006569A (en) | Three-dimensional data encoding method, three-dimensional data decoding method, three-dimensional data encoding device, and three-dimensional data decoding device. | |
PL423219A1 (en) | Method for extraction and transformation of stream-oriented measuring data, using the parallel computing | |
MX2016012001A (en) | Apparatus, method and corresponding computer program for generating an error concealment signal using individual replacement lpc representations for individual codebook information. | |
MX2015014862A (en) | Method for characterising a product by means of topological spectral analysis. | |
IN2013MU02590A (en) | ||
SA519401514B1 (en) | Method and apparatus for compact representation of bioinformatics data | |
MX348811B (en) | Apparatus and method for spatial audio object coding employing hidden objects for signal mixture manipulation. | |
Deng et al. | Estimating ultimate bound and finding topological horseshoe for a new chaotic system | |
BR112015028914A2 (en) | METHOD AND APPARATUS FOR REBUILDING A TIME/FREQUENCY BLOCK OF N AUDIO OBJECTS, METHOD AND CODER FOR GENERATING AT LEAST ONE WEIGHTING PARAMETER, AND COMPUTER READABLE MEDIA | |
PL421668A1 (en) | Method for storage and aggregation of multidimensional stream-oriented measuring data, with consideration of a locality aspect | |
Kluge | Comparison and end-to-end performance analysis of parallel filesystems | |
Amadane et al. | The Quantum Parafermi Gases and Statistical Physics | |
Mennucci | An intuitive presentation of Faà di Bruno’s formula | |
Beresnyak | Reply to Comment on" Spectra of strong magnetohydrodynamic turbulence from high-resolution simulations" | |
Burov | Imitation Modelling of Computing Algorithms for Identification of Technical Objects. | |
Ordokova | Lyrical Reports in Halimat Bairamukova’s Prose |