WO2012095839A4 - Systems and methods for performing online analytical processing - Google Patents
Systems and methods for performing online analytical processing Download PDFInfo
- Publication number
- WO2012095839A4 WO2012095839A4 PCT/IL2012/000012 IL2012000012W WO2012095839A4 WO 2012095839 A4 WO2012095839 A4 WO 2012095839A4 IL 2012000012 W IL2012000012 W IL 2012000012W WO 2012095839 A4 WO2012095839 A4 WO 2012095839A4
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- data
- aggregated
- database
- transactional data
- aggregation
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3452—Performance evaluation by statistical analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3409—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment
- G06F11/3419—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment by assessing time
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3466—Performance evaluation by tracing or monitoring
- G06F11/3495—Performance evaluation by tracing or monitoring for systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/283—Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
- G06F2201/80—Database-specific techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
- G06F2201/87—Monitoring of transactions
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- Databases & Information Systems (AREA)
- Quality & Reliability (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Probability & Statistics with Applications (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Complex Calculations (AREA)
Abstract
Near real time online analytical processing system, the system includes an analysis and aggregation module, a database and a query engine, the analysis and aggregation module, the analysis and aggregation module receiving transactional data, the analysis and aggregation module producing raw data, measured metrics statistics data, aggregated transactional data, and aggregated measured metrics statistics data, the database being coupled with the analysis and aggregation module, the database storing the raw data, the measured metrics statistics data, the aggregated transactional data and the aggregated measured metrics statistics data, the query engine being coupled with the database, the query engine receiving a user query, and answering the user query by retrieving data from the database.
Claims
1. A near Reai-Time On-Line Analytical Processing system (RTOLAP) associated with a monitored computing environment, the RTOLAP system comprising:
a database;
a storage for storing executable instructions;
a central processing unit {CPU} coupled with said storage and with said database, said CPU executing said executable instructions for:
implementing an analysis and aggregation module coupled with said database, said analysis and aggregation module analyzing transactional data respective of transaction instances executed on said monitored computing environment, wherein transactional data respective of a selected transaction instance contains a plurality of descriptive fields which values characterize said selected transaction instance, and contains measured metrics which values characterize the execution of said selected transaction instance on said monitored computing environment, said analysis and aggregation module aggregating transactional data for producing aggregated transactional data, said analysis and aggregation module storing aggregated transactional data in said database in a partitioned and redundant manner, wherein said analysis and aggregation module stores aggregated transactional data respective of transaction instances, which start in different periods of time, in separate partitions in said database, and stores aggregated transactional data which is aggregated according to different combinations of said plurality of descriptive fields in separate sub-partitions in said partitions in said database; and implementing a query engine coupled with said database, said query engine receiving a user query and answering said user query by retrieving data from said database, wherein said query engine answers said user query while said analysis and aggregation module stores aggregated transactional data in said database.
2. The RTOLAP system of claim 1 , wherein said analysis and aggregation module is further coupled with at least one monitoring agent monitoring at least a portion of said multi-tier computing environment for producing transactional data.
3. The RTOLAP system of claim 2, wherein said analysis and aggregation module includes an analysis engine, said analysis engine receiving transactional data from said at least one monitoring agent, wherein said analysis engine producing raw aggregated data by aggregating transactional data over the finest time period and over all of said plurality of descriptive fields, producing measured metrics statistics data by calculating statistics of measured metrics of raw aggregated data, producing aggregated transactional data by aggregating transactional data over coarser time periods and over different combinations of said plurality of descriptive fields, and producing aggregated measured metrics statistics data by calculating statistics of measured metrics of aggregated transactional data.
4. The RTOLAP system of claim 1 , wherein said query engine is further coupled with at least one client for receiving said user query data from said at least one client.
5. The RTOLAP system of claim 1 , wherein said user query specifying at least a queried time period detailing the queried start time and the queried end time, and specifying a set of queried ones of said plurality of descriptive fields,
wherein an answer to said user query being aggregated metrics statistics data respective of a!! transaction instances executed on said multi-tier computing environment and corresponding to said queried time period and to said queried ones of said plurality of descriptive fields.
6. The RTOLAP system of claim 5, wherein said query engine selecting an answer aggregation type from a plurality of aggregations types of said RTOLAP system, and selecting at least one answer time periods, each of said aggregation types being a combination of said plurality of descriptive fields, said answer aggregation type including at least said queried ones of said plurality of descriptive fields and further including the least amount of un-queried ones of said plurality of descriptive fields, said answer time periods being the minimum amount of time periods required for covering said queried time period.
7. The RTOLAP system of claim 1 , wherein said database stores raw aggregated data in logical table named period details table, stores aggregated transactional data in a logical table named period details aggregated table, stores measured metrics statistics data in a logical table named metrics statistics table and stores aggregated measured metrics statistics data in a logical table named metrics statistics aggregated table.
8. The RTOLAP system of claim 7, wherein said database stores the aggregated transactional data in said period details aggregated table redundantly in several time levels, and further redundantly according to different combinations of said plurality of descriptive fields.
9. The RTOLAP system of claim 7, wherein said analysis and aggregation module includes a merge engine and wherein said database includes a late arrivals data store, said merge engine retrieving late arrivals data from said late arrivals data store and merging the retrieved late arrivals data into said period details table, said metric statistics fable, said period details aggregated table and into said metric statistics aggregated table, said merge engine operates while said analysis and aggregation module stores aggregated transactional data in said database,
wherein late arrivals data is produced by said analysis and aggregation module from transactional data received by said RTOLAP system after co-starting transactional data respective of co-starting transaction instances has already been stored in said database, and wherein said late arrivals data is available for said query engine when answering said user query.
10. The RTOLAP system of claim 1 , wherein said analysis and aggregation module includes an aggregation engine, for retrieving aggregated transactional data from said database and aggregating the retrieved aggregated transactional data info aggregations of higher time levels or into aggregations of different aggregation types, wherein each said aggregation types being a combination of said plurality of descriptive fields, said aggregation engine producing the aggregated measured metrics statistics data for said retrieved aggregated data and storing said aggregated measured metrics statistics data in said database.
1 1. The RTOLAP system of claim 1 , wherein said analysis and aggregation module includes a purge engine for purging data stored in said database according to the age, the aggregation time level and the aggregation type of the data to be purged, and according to pre-configured retention times of data.
12. A method for operating a near Real-Time On-Line Analytical Processing (RTOLAP) system for a transaction monitoring system associated with a monitored computing environment, the method comprising the following procedures:
analyzing transactional data, respective of transaction instances executed on said monitored computing environment, wherein transactional data respective of a selected transaction instance contains a plurality of descriptive fields which values characterize said selected transaction instance, and contains measured metrics which values characterize the execution of said selected transaction instance on said monitored computing environment,
aggregating transactional data for producing aggregated transactional data, and storing aggregated transactional data in a database in a partitioned and redundant manner, wherein aggregated transactional data respective of transaction instances, which start in different periods of time, are stored in separate partitions in said database, and aggregated transactional data which is aggregated according to different combinations of said plurality of descriptive fields are stored in separate sub-partitions in said partitions in said database; and
receiving a user query, and answering said user query by retrieving data from said database, wherein said user query being answered while transactional data being stored in said database.
13. The method of claim 12, further including a procedure of purging data stored in said database according to the age, the aggregation time level and the aggregation type of the data to be purged, and according to pre-configured retention times of data.
14. The method of claim 12, further including a procedure of incorporating late arrivals data into said database
wherein late arrivals data is produced from transactional data received after co-starting transactional data respective of co-starting transaction instances has already been stored in said database, wherein said procedure of incorporating late arrivals data is performed simultaneously with said procedure of analyzing transactional data and said procedure of aggregating transactional data.
15. The method of claim 12, wherein said user query specifying at least a queried time period detailing the queried start time and the queried end time, and specifying a set of queried ones of said plurality of descriptive fields, and wherein said user query being answered by selecting an answer aggregation type from a plurality of aggregations types, and selecting at least one answer time periods, each of said aggregation types being a combination of said plurality of descriptive fields, said answer aggregation type including at least said queried ones of said plurality of descriptive fields and further including the least amount of un-queried ones of said plurality of descriptive fields, said answer time periods being the minimum amount of time periods required for covering said queried time period, and wherein said user query being answered further by retrieving aggregated data respective of the answer aggregation dimensions type and respective of the at least one answer time periods from said database, the answer to said user query being aggregated metrics statistics data respective of all transaction instances executed on said multi-tier computing environment and corresponding to said queried time period and to said queried ones of said plurality of descriptive fields.
16. The method of claim 15, further including the procedure of searching for late arrivals data which is relevant to said user query and incorporating late arrivals data into said answer.
17. The method of claim 12, wherein said procedure of aggregating transactional data includes producing raw aggregated data by aggregating transactional data over the finest time period and over all of said plurality of descriptive fields, producing measured metrics statistics data by calculating statistics of measured metrics of raw aggregated data, producing aggregated transactional data by aggregating transactional data over coarser time periods and over different combinations of said plurality of descriptive fields, and producing aggregated measured metrics statistics data by calculating statistics of measured metrics of aggregated transactional data.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US201161431175P | 2011-01-10 | 2011-01-10 | |
| US61/431,175 | 2011-01-10 |
Publications (3)
| Publication Number | Publication Date |
|---|---|
| WO2012095839A2 WO2012095839A2 (en) | 2012-07-19 |
| WO2012095839A3 WO2012095839A3 (en) | 2012-12-06 |
| WO2012095839A4 true WO2012095839A4 (en) | 2013-01-24 |
Family
ID=45755444
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/IL2012/000012 WO2012095839A2 (en) | 2011-01-10 | 2012-01-09 | Systems and methods for performing online analytical processing |
Country Status (1)
| Country | Link |
|---|---|
| WO (1) | WO2012095839A2 (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10169442B1 (en) * | 2012-06-29 | 2019-01-01 | Open Text Corporation | Methods and systems for multi-dimensional aggregation using composition |
| US9430453B1 (en) | 2012-12-19 | 2016-08-30 | Emc Corporation | Multi-page document recognition in document capture |
| EP3114620A1 (en) * | 2014-03-07 | 2017-01-11 | Systema Systementwicklung Dip.-Inf. Manfred Austen Gmbh | Real-time information systems and methodology based on continuous homomorphic processing in linear information spaces |
| US10419452B2 (en) | 2015-07-28 | 2019-09-17 | Sap Se | Contextual monitoring and tracking of SSH sessions |
| US10015178B2 (en) | 2015-07-28 | 2018-07-03 | Sap Se | Real-time contextual monitoring intrusion detection and prevention |
| CN109800129A (en) * | 2019-01-17 | 2019-05-24 | 青岛特锐德电气股份有限公司 | A kind of real-time stream calculation monitoring system and method for processing monitoring big data |
| WO2023219572A1 (en) * | 2022-05-12 | 2023-11-16 | Gp Network Asia Pte. Ltd. | Method and system for adaptively processing a request for data |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5958010A (en) * | 1997-03-20 | 1999-09-28 | Firstsense Software, Inc. | Systems and methods for monitoring distributed applications including an interface running in an operating system kernel |
| US6493699B2 (en) | 1998-03-27 | 2002-12-10 | International Business Machines Corporation | Defining and characterizing an analysis space for precomputed views |
| US6385604B1 (en) | 1999-08-04 | 2002-05-07 | Hyperroll, Israel Limited | Relational database management system having integrated non-relational multi-dimensional data store of aggregated data elements |
| US7805509B2 (en) * | 2004-06-04 | 2010-09-28 | Optier Ltd. | System and method for performance management in a multi-tier computing environment |
-
2012
- 2012-01-09 WO PCT/IL2012/000012 patent/WO2012095839A2/en active Application Filing
Also Published As
| Publication number | Publication date |
|---|---|
| WO2012095839A3 (en) | 2012-12-06 |
| WO2012095839A2 (en) | 2012-07-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2012095839A4 (en) | Systems and methods for performing online analytical processing | |
| US20210286786A1 (en) | Database performance tuning method, apparatus, and system, device, and storage medium | |
| Kanellis et al. | Too many knobs to tune? towards faster database tuning by pre-selecting important knobs | |
| US20220277042A1 (en) | Related content identification for different types of machine-generated data | |
| US10002149B2 (en) | Relevance ranking for data and transformations | |
| US8577871B2 (en) | Method and mechanism for out-of-the-box real-time SQL monitoring | |
| Katsipoulakis et al. | A holistic view of stream partitioning costs | |
| CN107643983B (en) | Test data processing method and system | |
| US10860465B2 (en) | Automatically rerunning test executions | |
| US20170132291A1 (en) | Event analysis apparatus, an event analysis system, an event analysis method, and an event analysis program | |
| EP3259681A1 (en) | Method and device for deciding where to execute subqueries of an analytics continuous query | |
| Hao et al. | Ts-benchmark: A benchmark for time series databases | |
| US20200136889A1 (en) | Near real time analytics | |
| CN112506743A (en) | Log monitoring method and device and server | |
| CN112286961B (en) | SQL optimization query method and device | |
| Sîrbu et al. | Towards data-driven autonomics in data centers | |
| US10970295B2 (en) | Collecting statistics in unconventional database environments | |
| CN110807145A (en) | Query engine acquisition method, device and computer-readable storage medium | |
| US10248544B2 (en) | System and method for automatic root cause detection | |
| Zhou et al. | Llm as dba | |
| US20100262593A1 (en) | Automated filtered index recommendations | |
| US20160078070A1 (en) | Database table column annotation | |
| US20150006567A1 (en) | Estimating most frequent values for a data set | |
| US20130097119A1 (en) | Method for Analyzing Performance Data for a Database | |
| US20210019288A1 (en) | Adapting time series database schema |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 12705458 Country of ref document: EP Kind code of ref document: A2 |
|
| NENP | Non-entry into the national phase in: |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 12705458 Country of ref document: EP Kind code of ref document: A2 |
|
| WWE | Wipo information: entry into national phase |
Ref document number: 231202 Country of ref document: IL |