CN105224635A - A kind of parallel OLAP construction device based on mixture model and construction method - Google Patents
A kind of parallel OLAP construction device based on mixture model and construction method Download PDFInfo
- Publication number
- CN105224635A CN105224635A CN201510618062.3A CN201510618062A CN105224635A CN 105224635 A CN105224635 A CN 105224635A CN 201510618062 A CN201510618062 A CN 201510618062A CN 105224635 A CN105224635 A CN 105224635A
- Authority
- CN
- China
- Prior art keywords
- model
- information
- snowflake
- module
- output unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/283—Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
Abstract
The invention provides a kind of parallel OLAP construction device based on mixture model and construction method, this construction device comprises: model makes module, core operation module, log pattern and parallel OLAP query module in advance.Beneficial effect of the present invention is compared with the prior art: a kind of parallel OLAP construction device based on mixture model provided by the invention and construction method thereof can to different data sources, the corresponding mixture model of automatic structure, the mixture model constructed combines the advantage of snowflake model and Star Model, solve the problem that traditional snowflake model is difficult to realize parallel query, under storage space consumption increases few prerequisite, significantly reduce the cost of parallel OLAP multidimensional analysis inquiry, ensure that the efficient execution of OLAP.In addition, use apparatus and method provided by the invention, only need perform according to the script write in advance, learning cost and running time cost can be saved for user of service.
Description
Technical field
The present invention relates to field of computer technology, be specifically related to a kind of parallel OLAP construction device based on mixture model and construction method.
Background technology
OLAP (OnlineAnalyticalProcessing) i.e. on-line analytical processing, OLAP system is the core application of data warehouse.Decision maker can be drilled through data by OLAP system, cut into slices and the multidimensional analysis such as stripping and slicing and rotation operates, and obtains directly perceived, understandable form Query Result to support decision-making.Current OLAP system can be divided into three types according to the data memory format of its storer, i.e. relational OLAP (RelationalOLAP, be called for short ROLAP), multidimensional OLAP (MultidimensionalOLAP, be called for short MOLAP) and mixed type OLAP (HybridOLAP is called for short HOLAP).Traditional ROLAP mainly organizes data according to Star Model or snowflake model.Star Model is a kind of non-normalized structure, and each dimension of cube is directly connected with fact table, so data have certain redundancy and can not embody the hierarchical structure of dimension well.Snowflake model obtains after standardizing to data on Star Model basis, it is to the further stratification of dimension table, the dimension table of original band hierarchy attributes is expanded to little fact table, form the level regions of some local, these list catenation be decomposed are to primary dimension table, show the hierarchical structure of dimension, solve the problem of Star Model data redundancy simultaneously.Although snowflake model solves Star Model in dimension hierarchy modeling and the defect of data redundancy, lose the feature that Star Model can carry out efficient parallel inquiry.
The problem how obtaining the ability of efficient parallel inquiry and multidimensional analysis operation while reducing storage cost is not still effectively solved.
In view of above-mentioned defect, creator of the present invention, through research and test for a long time, finally obtains the present invention.
Summary of the invention
For achieving the above object, a kind of parallel OLAP construction device based on mixture model of the present invention and construction method.
Technical scheme of the present invention is: provide a kind of parallel OLAP construction device based on mixture model on the one hand, this construction device comprises: model makes module, core operation module, log pattern and parallel OLAP query module in advance;
Described model makes module in advance, for receiving and store the data source extracted from data warehouse or relevant database, snowflake model is set up according to data source, and all information had by snowflake model output in described core operation module, and all operations information set up in the operating process of snowflake model is outputted in described log pattern; Described core operation module, all information that the snowflake model making module output in advance for receiving described model has, the snowflake model after upgrading is obtained after upgrading snowflake model, and according to all information that the snowflake model after upgrading has, set up join index between table, obtain final mixture model, and all operations information upgrading join index between all operations information of snowflake model and foundation table is outputted in described log pattern, all information simultaneously had by mixture model output in described parallel OLAP query module; Described log pattern, the all operations information of module, core operation module and parallel OLAP query module output is made in advance for receiving described model, and filtration treatment is carried out to operation information, obtain the operation information after filtering, the operation information after filtering all is outputted in described parallel OLAP query module; Described parallel OLAP query module, operation information after the filtration of all information that the mixture model exported for receiving described core operation module has and the output of described log pattern, and carry out multidimensional analysis query manipulation to all information that mixture model has, and all operations information of all information had mixture model being carried out multidimensional analysis inquiry outputs in described log pattern.
Further, described model makes module in advance and comprises: the first receiving element, storage unit, snowflake model set up unit, the first output unit and the second output unit; Described first receiving element, for receiving the data source extracted from data warehouse or relevant database; Described storage unit, for storing described data source; Described snowflake model sets up unit, for setting up corresponding snowflake model according to described data source; Described first output unit, all information for described snowflake model is had output in described core operation module; Described second output unit, all operations information set up in the operating process of described snowflake model for described snowflake model being set up unit outputs in described log pattern.
Further, described core operation module comprises: the second receiving element, updating block, the 3rd output unit, the 4th output unit and mixture model set up unit; Described second receiving element, all information that the snowflake model exported for receiving described first output unit has; Described updating block, for the key information of the highest non-for Dimensional level in snowflake model dimension table is all added in central facts table, obtains the snowflake model after upgrading; Described 3rd output unit, all information for the snowflake model after renewal is had output to described mixture model and set up in unit; Described 4th output unit, outputs in described log pattern for all operations information described updating block being upgraded snowflake model; Described mixture model sets up unit, for setting up mixture model.
Further, described mixture model is set up unit and is comprised: the 3rd receiving element, set up indexing units, the 5th output unit and the 6th output unit; Described 3rd receiving element, for receiving all information that have of snowflake model after renewal that described 3rd output unit exports; Describedly set up indexing units, for according to the key information added in central facts table, by Dimensional level from high in the end, set up successively central facts table to the dimension table of each Dimensional level table between join index, obtain final mixture model; Described 5th output unit, for outputting in log pattern by the described all operations information setting up join index between indexing units foundation table; Described 6th output unit, all information for being had by described mixture model output in described parallel OLAP query module.
Further, described log pattern comprises: the 4th receiving element, filter element and the 7th output unit; Described 4th receiving element, for receiving all operations information making the output of module, core operation module and parallel OLAP query module from described model in advance; Described filter element, for carrying out filtration treatment by what have nothing to do in operation information with the operation information of mistake, obtains the operation information after filtering; Described 7th output unit, for all outputting to the operation information after filtration in described parallel OLAP query module.
Further, described parallel OLAP query module comprises: the 5th receiving element, parallel processing element and the 8th output unit; Described 5th receiving element, for receiving the operation information after filtration that in all information that has of mixture model and described log pattern that export from the 6th output unit in described core operation module, the 7th output unit exports; Described parallel processing element, for carrying out multidimensional analysis query manipulation by parallel method to all information that mixture model has; Described 8th output unit, for outputting in described log pattern by described parallel processing element to all operations information that all information that mixture model has carry out multidimensional analysis inquiry; Described multidimensional analysis query manipulation comprises data mining, section and the multidimensional analysis such as stripping and slicing and rotation query manipulation.
Another aspect provides a kind of parallel OLAP construction method based on mixture model, and the method comprises the following steps:
Step S1: one model makes module in advance and receives and store the data source extracted from data warehouse or relevant database, snowflake model is set up according to data source, and all information had by snowflake model output in a core operation module, and all operations information set up in the operating process of snowflake model is outputted in a log pattern; Step S2: one core operation module receives described model and makes all information of having of snowflake model that module exports in advance, the snowflake model after upgrading is obtained after upgrading snowflake model, and according to all information that the snowflake model after upgrading has, set up join index between table, obtain final mixture model, and all operations information upgrading join index between all operations information of snowflake model and foundation table is outputted in described log pattern, all information simultaneously had by mixture model output in a parallel OLAP query module; Step S3: one log pattern receives all operations information that described model makes the output of module, core operation module and parallel OLAP query module in advance, and filtration treatment is carried out to operation information, obtain the operation information after filtering, the operation information after filtering all is outputted in described parallel OLAP query module; Operation information after the filtration of all information that the mixture model that step S4: one parallel OLAP query module receives the output of described core operation module has and the output of described log pattern, and carry out multidimensional analysis query manipulation to all information that mixture model has, and all operations information of all information had mixture model being carried out multidimensional analysis inquiry outputs in described log pattern.
Further, described step S1 specifically comprises the following steps:
Step S11: one first receiving element receives the data source extracted from data warehouse or relevant database; Data source described in step S12: one cell stores; Step S13: one snowflake model is set up unit and is set up corresponding snowflake model according to described data source; Step S14: one first output unit outputs to all information that described snowflake model has in described core operation module, and described snowflake model is set up the unit all operations information set up in the operating process of described snowflake model and outputted in described log pattern by one second output unit;
Described step S2 specifically comprises the following steps:
All information that the snowflake model that step S21: one second receiving element receives described first output unit output has; Step S22: one updating block all adds the key information of the highest non-for Dimensional level in snowflake model dimension table in central facts table, obtains the snowflake model after upgrading; Step S23: one the 3rd output unit outputs to a mixture model all information that the snowflake model after renewal has and sets up in unit, and all operations information that described updating block upgrades snowflake model outputs in described log pattern by one the 4th output unit; Step S24: one mixture model is set up unit and is set up mixture model.
Further, described step S24 specifically comprises the following steps:
All information that snowflake model after step S241: one the 3rd receiving element receives the renewal of described 3rd output unit output has; Step S242: one sets up indexing units according to the key information added in central facts table, by Dimensional level from high in the end, set up successively central facts table to the dimension table of each Dimensional level table between join index, obtain final mixture model; The described all operations information setting up join index between indexing units foundation table outputs in log pattern by step S243: one the 5th output unit, and all information that described mixture model has by one the 6th output unit output in described parallel OLAP query module.
Further, described step S3 specifically comprises the following steps:
Step S31: one the 4th receiving element receives all operations information making the output of module, core operation module and parallel OLAP query module from described model in advance; Step S32: one filter element carries out filtration treatment by what have nothing to do in operation information with the operation information of mistake, obtains the operation information after filtering; Operation information after filtration all outputs in described parallel OLAP query module by step S33: one the 7th output unit;
Described step S4 specifically comprises the following steps:
Step S41: one the 5th receiving element receives the operation information after the filtration that in all information that has of mixture model and described log pattern exported from the 6th output unit in described core operation module, the 7th output unit exports; Step S42: a line processing unit carries out multidimensional analysis query manipulation by parallel method to all information that mixture model has; Described multidimensional analysis query manipulation comprises data mining, section and the multidimensional analysis such as stripping and slicing and rotation query manipulation; Described parallel processing element outputs in described log pattern all operations information that all information that mixture model has carry out multidimensional analysis inquiry by step S43: one the 8th output unit.
Beneficial effect of the present invention is compared with the prior art: a kind of parallel OLAP construction device based on mixture model provided by the invention and construction method thereof can to different data sources, the corresponding mixture model of automatic structure, the mixture model constructed combines the advantage of snowflake model and Star Model, solve the problem that traditional snowflake model is difficult to realize parallel query, under storage space consumption increases few prerequisite, significantly reduce the cost of parallel OLAP multidimensional analysis inquiry, ensure that the efficient execution of OLAP.In addition, use apparatus and method provided by the invention, only need perform according to the script write in advance, learning cost and running time cost can be saved for user of service.
Accompanying drawing explanation
Fig. 1 is the structure function block diagram of a kind of parallel OLAP construction device based on mixture model of the present invention;
Fig. 2 of the present inventionly a kind ofly makes the structure function block diagram of module based on model in the parallel OLAP construction device of mixture model in advance;
Fig. 3 is a kind of structure function block diagram based on core operational module in the parallel OLAP construction device of mixture model of the present invention;
Fig. 4 is the structure function block diagram that in core operation module, mixture model sets up unit;
Fig. 5 is a kind of structure function block diagram based on log pattern in the parallel OLAP construction device of mixture model of the present invention;
Fig. 6 is a kind of structure function block diagram based on OLAP query module parallel in the parallel OLAP construction device of mixture model of the present invention;
Fig. 7 is the process flow diagram of a kind of parallel OLAP construction method based on mixture model of the present invention;
Fig. 8 is a kind of process flow diagram based on step S1 in the parallel OLAP construction method of mixture model of the present invention;
Fig. 9 is a kind of process flow diagram based on step S2 in the parallel OLAP construction method of mixture model of the present invention;
Figure 10 is a kind of process flow diagram based on step S24 in the parallel OLAP construction method of mixture model of the present invention;
Figure 11 is a kind of process flow diagram based on step S3 in the parallel OLAP construction method of mixture model of the present invention;
Figure 12 is a kind of process flow diagram based on step S4 in the parallel OLAP construction method of mixture model of the present invention;
Figure 13 is the data source extracted from data warehouse or relevant database;
Figure 14 is the snowflake model constructed according to data source;
Figure 15 is the snowflake model after upgrading;
Figure 16 is the final mixture model obtained.
Embodiment
Below in conjunction with accompanying drawing, to above-mentioned being described in more detail with other technical characteristic and advantage of the present invention.
As shown in Figure 1, be the structure function block diagram of a kind of parallel OLAP construction device based on mixture model of the present invention, this device comprises: model makes module 10, core operation module 20, log pattern 30 and parallel OLAP query module 40 in advance.
Described model makes module 10 in advance for receiving and store the data source extracted from data warehouse or relevant database, snowflake model is set up according to data source, and all information had by snowflake model output in described core operation module 20, and all operations information set up in the operating process of snowflake model is outputted in described log pattern 30.
Described core operation module 20 makes for receiving described model all information of having of snowflake model that module 10 exports in advance, the snowflake model after upgrading is obtained after upgrading snowflake model, and according to all information that the snowflake model after upgrading has, set up join index between table, obtain final mixture model, and all operations information upgrading join index between all operations information of snowflake model and foundation table is outputted in described log pattern 30, all information simultaneously had by mixture model output in described parallel OLAP query module 40.
Described log pattern 30 makes all operations information of module 10, core operation module 20 and parallel OLAP query module 40 output in advance for receiving described model, and filtration treatment is carried out to operation information, obtain the operation information after filtering, the operation information after filtering all is outputted in described parallel OLAP query module 40.
Operation information after the filtration that described parallel OLAP query module 40 exports for all information of having of mixture model of receiving described core operation module 20 and exporting and described log pattern 30, and carry out multidimensional analysis query manipulation to all information that mixture model has, and all operations information of all information had mixture model being carried out multidimensional analysis inquiry outputs in described log pattern 30.
As shown in Figure 2, a kind ofly the structure function block diagram of module is made in advance based on model in the parallel OLAP construction device of mixture model for of the present invention.This model makes module 10 in advance and comprises: the first receiving element 101, storage unit 102, snowflake model set up unit 103, first output unit 104 and the second output unit 105.Described first receiving element 101 is for receiving the data source extracted from data warehouse or relevant database, described storage unit 102 is for storing described data source, described snowflake model sets up unit 103 for setting up corresponding snowflake model according to described data source, described first output unit 104 outputs in described core operation module 30 for all information that described snowflake model is had, described second output unit 105 outputs in described log pattern 30 for described snowflake model being set up the unit 103 all operations information set up in the operating process of described snowflake model.
As shown in Figure 3, be a kind of structure function block diagram based on core operational module in the parallel OLAP construction device of mixture model of the present invention.As shown in Figure 4, for mixture model in core operation module sets up the structure function block diagram of unit.This core operation module 20 comprises: the second receiving element 201, updating block 202, the 3rd output unit 203, the 4th output unit 204 and mixture model set up unit 205.All information that described second receiving element 201 has for the snowflake model receiving described first output unit 104 output.Described updating block 202, for the key information of the highest non-for Dimensional level in snowflake model dimension table is all added in central facts table, obtains the snowflake model after upgrading.Described 3rd output unit 203 outputs to described mixture model for all information that the snowflake model after renewal is had and sets up in unit 205.Described 4th output unit 204 outputs in described log pattern 30 for all operations information described updating block 202 being upgraded snowflake model.Described mixture model sets up unit 205 for setting up mixture model.
Described mixture model is set up unit 205 and is comprised: the 3rd receiving element 2051, set up indexing units 2052, the 5th output unit 2053 and the 6th output unit 2054.Described 3rd receiving element 2051 is for all information that have of snowflake model after the renewal that receives described 3rd output unit 203 and export.Described indexing units 2052 of setting up for according to the key information added in central facts table, by Dimensional level from high in the end, set up successively central facts table to the dimension table of each Dimensional level table between join index, obtain final mixture model.Described 5th output unit 2053 is for outputting to the described all operations information setting up join index between indexing units 2052 foundation table in log pattern 30.Described 6th output unit 2054 outputs in described parallel OLAP query module 40 for all information had by described mixture model.
As shown in Figure 5, be a kind of structure function block diagram based on log pattern in the parallel OLAP construction device of mixture model of the present invention.Described log pattern 30 comprises: the 4th receiving element 301, filter element 302 and the 7th output unit 303.Described 4th receiving element 301 makes all operations information of module 10, core operation module 20 and parallel OLAP query module 40 output in advance from described model for receiving.Described filter element 302, for carrying out filtration treatment by what have nothing to do in operation information with the operation information of mistake, obtains the operation information after filtering.Described 7th output unit 303 is for all outputting to the operation information after filtration in described parallel OLAP query module 40.
As shown in Figure 6, be a kind of structure function block diagram based on OLAP query module parallel in the parallel OLAP construction device of mixture model of the present invention.Described parallel OLAP query module 40 comprises: the 5th receiving element 401, parallel processing element 402 and the 8th output unit 403.Described 5th receiving element 401 is for receiving the operation information after filtration that in all information that has of mixture model and described log pattern 30 that export from the 6th output unit 2054 in described core operation module 20, the 7th output unit 303 exports.Described parallel processing element 402 is for carrying out multidimensional analysis query manipulation by parallel method to all information that mixture model has; Described multidimensional analysis query manipulation comprises data mining, section and the multidimensional analysis such as stripping and slicing and rotation query manipulation.The all operations information that described 8th output unit 403 carries out multidimensional analysis inquiry for all information had by described parallel processing element 402 pairs of mixture models outputs in described log pattern 30.
As shown in Figure 7, be the process flow diagram of a kind of parallel OLAP construction method based on mixture model of the present invention, the method comprises the following steps:
Step S1: one model makes module in advance and receives and store the data source extracted from data warehouse or relevant database, snowflake model is set up according to data source, and all information had by snowflake model output in a core operation module, and all operations information set up in the operating process of snowflake model is outputted in a log pattern.
Step S2: one core operation module receives described model and makes all information of having of snowflake model that module exports in advance, the snowflake model after upgrading is obtained after upgrading snowflake model, and according to all information that the snowflake model after upgrading has, set up join index between table, obtain final mixture model, and all operations information upgrading join index between all operations information of snowflake model and foundation table is outputted in described log pattern, all information simultaneously had by mixture model output in a parallel OLAP query module.
Step S3: one log pattern receives all operations information that described model makes the output of module, core operation module and parallel OLAP query module in advance, and filtration treatment is carried out to operation information, obtain the operation information after filtering, the operation information after filtering all is outputted in described parallel OLAP query module.
Operation information after the filtration of all information that the mixture model that step S4: one parallel OLAP query module receives the output of described core operation module has and the output of described log pattern, and carry out multidimensional analysis query manipulation to all information that mixture model has, and all operations information of all information had mixture model being carried out multidimensional analysis inquiry outputs in described log pattern.
As shown in Figure 8, be a kind of process flow diagram based on step S1 in the parallel OLAP construction method of mixture model of the present invention.Described step S1 specifically comprises the following steps:
Step S11: one first receiving element receives the data source extracted from data warehouse or relevant database.As shown in figure 13, the data source for extracting from data warehouse or relevant database.
Data source described in step S12: one cell stores.
Step S13: one snowflake model is set up unit and is set up corresponding snowflake model according to described data source.As shown in figure 14, the snowflake model for constructing according to data source.
Step S14: one first output unit outputs to all information that described snowflake model has in described core operation module, and described snowflake model is set up all operations information that unit sets up in the operating process of described snowflake model and outputted in described log pattern by one second output unit.
As shown in Figure 9, be a kind of process flow diagram based on step S2 in the parallel OLAP construction method of mixture model of the present invention.Described step S2 specifically comprises the following steps:
All information that the snowflake model that step S21: one second receiving element receives described first output unit output has.
Step S22: one updating block all adds the key information of the highest non-for Dimensional level in snowflake model dimension table in central facts table, obtains the snowflake model after upgrading.
As shown in figure 15, be the snowflake model after renewal.
Step S23: one the 3rd output unit outputs to a mixture model all information that the snowflake model after renewal has and sets up in unit, and all operations information that described updating block upgrades snowflake model outputs in described log pattern by one the 4th output unit.
Step S24: one mixture model is set up unit and is set up mixture model.
As shown in Figure 10, be a kind of process flow diagram based on step S24 in the parallel OLAP construction method of mixture model of the present invention, described step S24 specifically comprises the following steps:
All information that snowflake model after step S241: one the 3rd receiving element receives the renewal of described 3rd output unit output has.
Step S242: one sets up indexing units according to the key information added in central facts table, by Dimensional level from high in the end, set up successively central facts table to the dimension table of each Dimensional level table between join index, obtain final mixture model.As shown in figure 16, the final mixture model for obtaining.
The described all operations information setting up join index between indexing units foundation table outputs in log pattern by step S243: one the 5th output unit, and all information that described mixture model has by one the 6th output unit output in described parallel OLAP query module.
As shown in figure 11, be a kind of process flow diagram based on step S3 in the parallel OLAP construction method of mixture model of the present invention.Described step S3 specifically comprises the following steps:
Step S31: one the 4th receiving element receives all operations information making the output of module, core operation module and parallel OLAP query module from described model in advance.
Step S32: one filter element carries out filtration treatment by what have nothing to do in operation information with the operation information of mistake, obtains the operation information after filtering.
Operation information after filtration all outputs in described parallel OLAP query module by step S33: one the 7th output unit.
As shown in figure 12, be a kind of process flow diagram based on step S4 in the parallel OLAP construction method of mixture model of the present invention.Described step S4 specifically comprises the following steps:
Step S41: one the 5th receiving element receives the operation information after the filtration that in all information that has of mixture model and described log pattern exported from the 6th output unit in described core operation module, the 7th output unit exports.
Step S42: a line processing unit carries out multidimensional analysis query manipulation by parallel method to all information that mixture model has; Described multidimensional analysis query manipulation comprises data mining, section and the multidimensional analysis such as stripping and slicing and rotation query manipulation.
Described parallel processing element outputs in described log pattern all operations information that all information that mixture model has carry out multidimensional analysis inquiry by step S43: one the 8th output unit.
A kind of parallel OLAP construction device based on mixture model provided by the invention and construction method thereof can to different data sources, the corresponding mixture model of automatic structure, the mixture model constructed combines the advantage of snowflake model and Star Model, solve the problem that traditional snowflake model is difficult to realize parallel query, under storage space consumption increases few prerequisite, significantly reduce the cost of parallel OLAP multidimensional analysis inquiry, ensure that the efficient execution of OLAP.In addition, use apparatus and method provided by the invention, only need perform according to the script write in advance, learning cost and running time cost can be saved for user of service.
The foregoing is only preferred embodiment of the present invention, is only illustrative for invention, and nonrestrictive.Those skilled in the art is understood, and can carry out many changes in the spirit and scope that invention claim limits to it, amendment, even equivalence, but all will fall within the scope of protection of the present invention.
Claims (10)
1. based on a parallel OLAP construction device for mixture model, it is characterized in that, this construction device comprises: model makes module, core operation module, log pattern and parallel OLAP query module in advance;
Described model makes module in advance, for receiving and store the data source extracted from data warehouse or relevant database, snowflake model is set up according to data source, and all information had by snowflake model output in described core operation module, and all operations information set up in the operating process of snowflake model is outputted in described log pattern;
Described core operation module, all information that the snowflake model making module output in advance for receiving described model has, the snowflake model after upgrading is obtained after upgrading snowflake model, and according to all information that the snowflake model after upgrading has, set up join index between table, obtain final mixture model, and all operations information upgrading join index between all operations information of snowflake model and foundation table is outputted in described log pattern, all information simultaneously had by mixture model output in described parallel OLAP query module;
Described log pattern, the all operations information of module, core operation module and parallel OLAP query module output is made in advance for receiving described model, and filtration treatment is carried out to operation information, obtain the operation information after filtering, the operation information after filtering all is outputted in described parallel OLAP query module;
Described parallel OLAP query module, operation information after the filtration of all information that the mixture model exported for receiving described core operation module has and the output of described log pattern, and carry out multidimensional analysis query manipulation to all information that mixture model has, and all operations information of all information had mixture model being carried out multidimensional analysis inquiry outputs in described log pattern.
2. a kind of parallel OLAP construction device based on mixture model according to claim 1, it is characterized in that, described model makes module in advance and comprises: the first receiving element, storage unit, snowflake model set up unit, the first output unit and the second output unit; Described first receiving element, for receiving the data source extracted from data warehouse or relevant database; Described storage unit, for storing described data source; Described snowflake model sets up unit, for setting up corresponding snowflake model according to described data source; Described first output unit, all information for described snowflake model is had output in described core operation module; Described second output unit, all operations information set up in the operating process of described snowflake model for described snowflake model being set up unit outputs in described log pattern.
3. a kind of parallel OLAP construction device based on mixture model according to claim 2, it is characterized in that, described core operation module comprises: the second receiving element, updating block, the 3rd output unit, the 4th output unit and mixture model set up unit; Described second receiving element, all information that the snowflake model exported for receiving described first output unit has; Described updating block, for the key information of the highest non-for Dimensional level in snowflake model dimension table is all added in central facts table, obtains the snowflake model after upgrading; Described 3rd output unit, all information for the snowflake model after renewal is had output to described mixture model and set up in unit; Described 4th output unit, outputs in described log pattern for all operations information described updating block being upgraded snowflake model; Described mixture model sets up unit, for setting up mixture model.
4. a kind of parallel OLAP construction device based on mixture model according to claim 3, it is characterized in that, described mixture model is set up unit and is comprised: the 3rd receiving element, set up indexing units, the 5th output unit and the 6th output unit; Described 3rd receiving element, for receiving all information that have of snowflake model after renewal that described 3rd output unit exports; Describedly set up indexing units, for according to the key information added in central facts table, by Dimensional level from high in the end, set up successively central facts table to the dimension table of each Dimensional level table between join index, obtain final mixture model; Described 5th output unit, for outputting in log pattern by the described all operations information setting up join index between indexing units foundation table; Described 6th output unit, all information for being had by described mixture model output in described parallel OLAP query module.
5. a kind of parallel OLAP construction device based on mixture model according to claim 4, it is characterized in that, described log pattern comprises: the 4th receiving element, filter element and the 7th output unit; Described 4th receiving element, for receiving all operations information making the output of module, core operation module and parallel OLAP query module from described model in advance; Described filter element, for carrying out filtration treatment by what have nothing to do in operation information with the operation information of mistake, obtains the operation information after filtering; Described 7th output unit, for all outputting to the operation information after filtration in described parallel OLAP query module.
6. a kind of parallel OLAP construction device based on mixture model according to claim 5, it is characterized in that, described parallel OLAP query module comprises: the 5th receiving element, parallel processing element and the 8th output unit; Described 5th receiving element, for receiving the operation information after filtration that in all information that has of mixture model and described log pattern that export from the 6th output unit in described core operation module, the 7th output unit exports; Described parallel processing element, for carrying out multidimensional analysis query manipulation by parallel method to all information that mixture model has; Described 8th output unit, for outputting in described log pattern by described parallel processing element to all operations information that all information that mixture model has carry out multidimensional analysis inquiry; Described multidimensional analysis query manipulation comprises data mining, section and the multidimensional analysis such as stripping and slicing and rotation query manipulation.
7., based on a parallel OLAP construction method for mixture model, it is characterized in that, the method comprises the following steps:
Step S1: one model makes module in advance and receives and store the data source extracted from data warehouse or relevant database, snowflake model is set up according to data source, and all information had by snowflake model output in a core operation module, and all operations information set up in the operating process of snowflake model is outputted in a log pattern;
Step S2: one core operation module receives described model and makes all information of having of snowflake model that module exports in advance, the snowflake model after upgrading is obtained after upgrading snowflake model, and according to all information that the snowflake model after upgrading has, set up join index between table, obtain final mixture model, and all operations information upgrading join index between all operations information of snowflake model and foundation table is outputted in described log pattern, all information simultaneously had by mixture model output in a parallel OLAP query module;
Step S3: one log pattern receives all operations information that described model makes the output of module, core operation module and parallel OLAP query module in advance, and filtration treatment is carried out to operation information, obtain the operation information after filtering, the operation information after filtering all is outputted in described parallel OLAP query module;
Operation information after the filtration of all information that the mixture model that step S4: one parallel OLAP query module receives the output of described core operation module has and the output of described log pattern, and carry out multidimensional analysis query manipulation to all information that mixture model has, and all operations information of all information had mixture model being carried out multidimensional analysis inquiry outputs in described log pattern.
8. a kind of parallel OLAP construction method based on mixture model according to claim 7, it is characterized in that, described step S1 specifically comprises the following steps:
Step S11: one first receiving element receives the data source extracted from data warehouse or relevant database;
Data source described in step S12: one cell stores;
Step S13: one snowflake model is set up unit and is set up corresponding snowflake model according to described data source;
Step S14: one first output unit outputs to all information that described snowflake model has in described core operation module, and described snowflake model is set up the unit all operations information set up in the operating process of described snowflake model and outputted in described log pattern by one second output unit;
Described step S2 specifically comprises the following steps:
All information that the snowflake model that step S21: one second receiving element receives described first output unit output has;
Step S22: one updating block all adds the key information of the highest non-for Dimensional level in snowflake model dimension table in central facts table, obtains the snowflake model after upgrading;
Step S23: one the 3rd output unit outputs to a mixture model all information that the snowflake model after renewal has and sets up in unit, and all operations information that described updating block upgrades snowflake model outputs in described log pattern by one the 4th output unit;
Step S24: one mixture model is set up unit and is set up mixture model.
9. a kind of parallel OLAP construction method based on mixture model according to claim 8, it is characterized in that, described step S24 specifically comprises the following steps:
All information that snowflake model after step S241: one the 3rd receiving element receives the renewal of described 3rd output unit output has;
Step S242: one sets up indexing units according to the key information added in central facts table, by Dimensional level from high in the end, set up successively central facts table to the dimension table of each Dimensional level table between join index, obtain final mixture model;
The described all operations information setting up join index between indexing units foundation table outputs in log pattern by step S243: one the 5th output unit, and all information that described mixture model has by one the 6th output unit output in described parallel OLAP query module.
10. a kind of parallel OLAP construction method based on mixture model according to claim 9, it is characterized in that, described step S3 specifically comprises the following steps:
Step S31: one the 4th receiving element receives all operations information making the output of module, core operation module and parallel OLAP query module from described model in advance;
Step S32: one filter element carries out filtration treatment by what have nothing to do in operation information with the operation information of mistake, obtains the operation information after filtering;
Operation information after filtration all outputs in described parallel OLAP query module by step S33: one the 7th output unit;
Described step S4 specifically comprises the following steps:
Step S41: one the 5th receiving element receives the operation information after the filtration that in all information that has of mixture model and described log pattern exported from the 6th output unit in described core operation module, the 7th output unit exports;
Step S42: a line processing unit carries out multidimensional analysis query manipulation by parallel method to all information that mixture model has; Described multidimensional analysis query manipulation comprises data mining, section and the multidimensional analysis such as stripping and slicing and rotation query manipulation;
Described parallel processing element outputs in described log pattern all operations information that all information that mixture model has carry out multidimensional analysis inquiry by step S43: one the 8th output unit.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510618062.3A CN105224635A (en) | 2015-09-24 | 2015-09-24 | A kind of parallel OLAP construction device based on mixture model and construction method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510618062.3A CN105224635A (en) | 2015-09-24 | 2015-09-24 | A kind of parallel OLAP construction device based on mixture model and construction method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105224635A true CN105224635A (en) | 2016-01-06 |
Family
ID=54993603
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510618062.3A Pending CN105224635A (en) | 2015-09-24 | 2015-09-24 | A kind of parallel OLAP construction device based on mixture model and construction method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105224635A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107451142A (en) * | 2016-05-31 | 2017-12-08 | 北京京东尚科信息技术有限公司 | The method and apparatus and its management system of data are write and inquired about in database |
| CN116303834A (en) * | 2023-05-19 | 2023-06-23 | 北京弘维大数据技术有限公司 | Data warehouse historical data storage and processing method, system and device |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103927337A (en) * | 2014-03-26 | 2014-07-16 | 北京国双科技有限公司 | Method and device for processing data of association relationships in online analytical processing |
| CN104391948A (en) * | 2014-12-01 | 2015-03-04 | 广东电网有限责任公司清远供电局 | Data standardization construction method and system of data warehouse |
-
2015
- 2015-09-24 CN CN201510618062.3A patent/CN105224635A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103927337A (en) * | 2014-03-26 | 2014-07-16 | 北京国双科技有限公司 | Method and device for processing data of association relationships in online analytical processing |
| CN104391948A (en) * | 2014-12-01 | 2015-03-04 | 广东电网有限责任公司清远供电局 | Data standardization construction method and system of data warehouse |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107451142A (en) * | 2016-05-31 | 2017-12-08 | 北京京东尚科信息技术有限公司 | The method and apparatus and its management system of data are write and inquired about in database |
| CN107451142B (en) * | 2016-05-31 | 2022-05-27 | 北京京东尚科信息技术有限公司 | Method and apparatus for writing and querying data in database, management system and computer-readable storage medium thereof |
| CN116303834A (en) * | 2023-05-19 | 2023-06-23 | 北京弘维大数据技术有限公司 | Data warehouse historical data storage and processing method, system and device |
| CN116303834B (en) * | 2023-05-19 | 2024-03-08 | 北京弘维大数据技术有限公司 | Data warehouse historical data storage and processing method, system and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11366829B2 (en) | System and method for analysis and management of data distribution in a distributed database environment | |
| CN110019396B (en) | Data analysis system and method based on distributed multidimensional analysis | |
| CN106682213A (en) | Internet-of-things task customizing method and system based on Hadoop platform | |
| CN102023983B (en) | Managing method of statistical space-time database | |
| CN104915793A (en) | Public information intelligent analysis platform based on big data analysis and mining | |
| CN112199433A (en) | Data management system for city-level data middling station | |
| CN104205039A (en) | Interest-driven business intelligence systems and methods of data analysis using interest-driven data pipelines | |
| CN105320679A (en) | Data table index set generation method and device | |
| CN109508355A (en) | A kind of data pick-up method, system and terminal device | |
| CN114218218A (en) | Data processing method, device and equipment based on data warehouse and storage medium | |
| CN112148578A (en) | IT fault defect prediction method based on machine learning | |
| CN105069153A (en) | Patent analysis system | |
| CN115858513A (en) | Data governance method, data governance device, computer equipment and storage medium | |
| CN102945262A (en) | Comparing method and device for RNC (Radio Network Controller) configuration data | |
| CN105224635A (en) | A kind of parallel OLAP construction device based on mixture model and construction method | |
| CN105183824A (en) | Data integration method and apparatus | |
| CN106599241A (en) | Big data visual management method for GIS software | |
| CN106815320B (en) | Investigation big data visual modeling method and system based on expanded three-dimensional histogram | |
| CN110825744B (en) | Cluster environment-based air quality monitoring big data partition storage method | |
| Gu | Integration and optimization of ancient literature information resources based on big data technology | |
| CN111475602A (en) | Multi-version knowledge graph storage method and device, storage medium and electronic equipment | |
| CN111079809A (en) | Intelligent system method for electric connector | |
| CN109376148B (en) | Data processing method and device for slow change dimension table and electronic equipment | |
| Ren et al. | Efficient processing of shortest path queries in evolving graph sequences | |
| CN115982177A (en) | Data collection method, device, equipment and medium based on tree dimensionality |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160106 |
|
| RJ01 | Rejection of invention patent application after publication |