CN108875008A - A kind of Large Volume Data analysis method and device - Google Patents
A kind of Large Volume Data analysis method and device Download PDFInfo
- Publication number
- CN108875008A CN108875008A CN201810623366.2A CN201810623366A CN108875008A CN 108875008 A CN108875008 A CN 108875008A CN 201810623366 A CN201810623366 A CN 201810623366A CN 108875008 A CN108875008 A CN 108875008A
- Authority
- CN
- China
- Prior art keywords
- cube
- database
- data
- analysis
- large volume
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention provides a kind of Large Volume Data analysis method and devices, select the guide of data source, the supplier of Analysis Service, the position of Analysis server and multi-dimensional database and cube；It calls AddNew method to create Analysis Service database in the MDStores set of Server object, adds the association attributes including connection string, specify a new data source for Analysis Service database；Creation cube is closed in Database.MDStores collection, creates true table, dimension table, defines the data column of measurement, and according to business need, sets the granularity of dimension；Setting OlapMode attribute is closed in the MDStores collection of Server object, selects the storage mode of cube；The processing mode of cube is selected by specifying an optional parameters to Process method.
Description
Technical field
The present invention relates to data analysis technique fields, more particularly, to a kind of Large Volume Data analysis method and dress
It sets.
Background technique
Realize that multidimensional data analysis is the major function that data warehouse needs to have, it receives the more of user
Dimension data inquiry request generates cube, on-line analytical processing (On_LineAnalyticalProcessing, OLAP)
It is an important component of data warehouse technology.OLAP technology is to be taken out the data of data warehouse by olap server
Multidimensional data structure is taken and be converted to, to reflect true " dimension " of user enterprise to understand, then passes through multidimensional point
Multiple angles of the analysis tool to information, multiple sides carry out quick, consistent and interaction access, to make analysis personnel, warp
Reason and administrative staff can carry out deep analysis and observation to data.Wherein, data pick-up and crossover tool by data according to
Certain requirement, imported into multidimensional data warehouse, and the user of data warehouse is by front-end access and analysis tool, to more
The data set that dimension data analysis generates carries out display output according to certain mode.And multidimensional data analysis, then according to front end work
The data inquiry request of tool input obtains data, and generates the data set of certain format.
Generally using open source modeling tool MechanicalAPDL and ANSYS Workbench, but excessively complicated behaviour
The experience that user is affected as process, reduces service efficiency.Due to the limitation of data Layer, application layer is realizing looking into for data
When asking function, the small-scale data of centering can accomplish instant response, but to the mass data stored in database and greatly
The data stored in data platform, it is difficult to guarantee the real-time of inquiry.Secondly, OLAP query engine generally uses
Mondrian, it realizes inquiry with MDX language, only supports to read data from relevant database, but to big data platform
Lack mating interface and the aggregate function supported is limited.In addition, OLAP query engine do not have to the depth analysis of data and
Data mining ability.
Summary of the invention
The present invention provides a kind of a kind of Large Volume Data for overcoming the above problem or at least being partially solved the above problem
Analysis method and device.
According to an aspect of the present invention, a kind of Large Volume Data analysis method is proposed, including：
Select the guide of data source, the supplier of Analysis Service, the position of Analysis server and multi-dimensional database and
Cube；
AddNew method is called to create Analysis Service database, addition packet in the MDStores set of Server object
The association attributes including connection string are included, specify a new data source for Analysis Service database；
Creation cube is closed in Database.MDStores collection, true table, dimension table is created, defines measurement
Data column, and according to business need, set the granularity of dimension；
Setting OlapMode attribute is closed in the MDStores collection of Server object, selects the storage side of cube
Formula；
The processing mode of cube is selected by specifying an optional parameters to Process method.
Preferably, the storage mode of the cube includes multidimensional OLAP, relational OLAP and mixing OLAP.
Preferably, the granularity of setting dimension specifically includes：
Creation measurement, position of the specified metric in source database, type and size are concentrated in multidimensional data, and is pointed out
The polymerization methods of measurement.
A kind of Large Volume Data analytical equipment, including：
Multiple processors, multiple memories, communication interface and bus；Wherein,
The processor, memory, communication interface complete mutual communication by the bus；
The communication interface is for the information transmission between the test equipment and the communication equipment of display device；
The memory is stored with the program instruction that can be executed by the processor, and the processor calls described program
Instruction is able to carry out such as above-mentioned Large Volume Data analysis method.
A kind of Large Volume Data analytical equipment, including：
Multiple processors；And
The multiple memories being connect with the processor communication, wherein：
The memory is stored with the program instruction that can be executed by the processor, and the processor calls described program
Instruction is able to carry out such as above-mentioned Large Volume Data analysis method.
A kind of non-transient computer readable storage medium, the non-transient computer readable storage medium store computer
Instruction, the computer instruction make the computer execute such as above-mentioned Large Volume Data analysis method.
A kind of Large Volume Data analysis method proposed by the present invention and device, in DSO, ADO MD and MDX (multidimensional data
Expression formula) etc. on the basis of technologies, constructing a multidimensional data analysis device implementations described herein can be embedded in
In other application programs, to realize the integrated of entire data warehouse applications, unprofessional user's answering using management tool is reduced
Polygamy.
Detailed description of the invention
Fig. 1 is Large Volume Data analysis method schematic diagram of the invention；
Fig. 2 is Large Volume Data analytical equipment schematic diagram of the invention.
Specific embodiment
With reference to the accompanying drawings and examples, specific embodiments of the present invention will be described in further detail.Implement below
Example is not intended to limit the scope of the invention for illustrating the present invention.
As shown in Figure 1, a kind of Large Volume Data analysis method is shown in figure, including：
Select the guide of data source, the supplier of Analysis Service, the position of Analysis server and multi-dimensional database and
Cube；
AddNew method is called to create Analysis Service database, addition packet in the MDStores set of Server object
The association attributes including connection string are included, specify a new data source for Analysis Service database；
Creation cube is closed in Database.MDStores collection, true table, dimension table is created, defines measurement
Data column, and according to business need, set the granularity of dimension；
Setting OlapMode attribute is closed in the MDStores collection of Server object, selects the storage side of cube
Formula；The storage mode of cube includes multidimensional OLAP (OlapmodeMolapIndex), relational OLAP
(OlapmodeRolap) and OLAP (OlapmodeHybridIndex) is mixed.
The processing mode of cube is selected by specifying an optional parameters to Process method.
Microsoft SQL Server 2000 and the above version both provide relevant data warehouse development technique, make
Obtaining user can according to need the application program of exploitation oneself, Multidimensional Expressions MDX (Multi-dimensionalExpress)
Support the operation to cube.ADO MD is the extension of ADO, it allows the application program based on COM to pass through for OLAP
O LE DB interface realize operation to multi-dimensional data source.Including in olap database to cube hierarchical structure
Read-only access；Query processing and data retrieval function .DSO (Decisi on SupportObject, decision support objects) are provided
To the managerial ability of olap database.
In the present embodiment, on the basis of Analysis server and applying PivotTable services, to the number in cube
According to multidimensional analysis is carried out, the guide of data source is selected by user oneself；It can be by the offer of user oneself selection analysis service
Person, the position of Analysis server and multi-dimensional database and cube.Simultaneity factor also arranges local cube
Table shows that user can directly select data source of the cube as multidimensional data analysis.In multidimensional analysis module
In, the service provided by ADO MD object model and PivotTable Service accesses cube.List display
All dimensions and metric of cube out.The dimension and metric for select data to show by user oneself, according to user
The dimension and metric of selection, construction MDX query statement carry out multi-dimensional query analysis, utilize the C of ADO MD object model
Ellset object stores multidimensional analysis as a result, finally showing multidimensional data, checks data for user.
AddNew method is called to create Analysis Service database grammer shape in the MDStores set of Server object
Formula is as follows：
DsoServer.MDStores.AddNew database name
The AddNew method of DataSources set is called, the association attributes including connection string is added, is
Analytical database specifies a new data source.
Creation cube is closed in Database.MDStores collection：
Create a multidimensional data set name:Set dsoCube=dsoDb.MDStores.AddNew；
The data source of cube is set:dsoCube.DataSources.AddNew dsoDb.D ataSources
(dataSource link) .Name；
Specified fact table:DsoCube.SourceTable=fact table name is concentrated in multidimensional data increases dimension:
DsoCube.Dimensions.AddNew dimension name；
Creation measurement is concentrated in multidimensional data, firstly the need of specified metric；Position, type in source database and big
It is small；Secondly, it should be pointed out that how measurement is polymerize.
The present embodiment discloses a kind of Large Volume Data analytical equipment, including：
Multiple processors；And
The multiple memories being connect with the processor communication, wherein：
The memory is stored with the program instruction that can be executed by the processor, and the processor calls described program
Instruction is able to carry out such as above-mentioned Large Volume Data analysis method, for example including：
Select the guide of data source, the supplier of Analysis Service, the position of Analysis server and multi-dimensional database and
Cube；
AddNew method is called to create Analysis Service database, addition packet in the MDStores set of Server object
The association attributes including connection string are included, specify a new data source for Analysis Service database；
Creation cube is closed in Database.MDStores collection, true table, dimension table is created, defines measurement
Data column, and according to business need, set the granularity of dimension；
Setting OlapMode attribute is closed in the MDStores collection of Server object, selects the storage side of cube
Formula；
The processing mode of cube is selected by specifying an optional parameters to Process method.
Fig. 2 is the structural block diagram for showing the Large Volume Data analytical equipment of the embodiment of the present application.
Reference Fig. 2, the Large Volume Data analytical equipment, including：Processor (processor) 810, memory
(memory) 830, communication interface (Communications Interface) 820 and bus 840；
Wherein,
The processor 810, memory 830, communication interface 820 complete mutual communication by the bus 840；
The communication interface 820 is for the information transmission between the test equipment and the communication equipment of display device；
The processor 810 is used to call the program instruction in the memory 830, is implemented with executing above-mentioned each method
Large Volume Data analysis method provided by example, for example including：
Select the guide of data source, the supplier of Analysis Service, the position of Analysis server and multi-dimensional database and
Cube；
AddNew method is called to create Analysis Service database, addition packet in the MDStores set of Server object
The association attributes including connection string are included, specify a new data source for Analysis Service database；
Creation cube is closed in Database.MDStores collection, true table, dimension table is created, defines measurement
Data column, and according to business need, set the granularity of dimension；
Setting OlapMode attribute is closed in the MDStores collection of Server object, selects the storage side of cube
Formula；
The processing mode of cube is selected by specifying an optional parameters to Process method.
The present embodiment discloses a kind of computer program product, and the computer program product includes being stored in non-transient meter
Computer program on calculation machine readable storage medium storing program for executing, the computer program include program instruction, when described program instruction is counted
When calculation machine executes, computer is able to carry out Large Volume Data analysis method provided by above-mentioned each method embodiment, such as wraps
It includes：
Select the guide of data source, the supplier of Analysis Service, the position of Analysis server and multi-dimensional database and
Cube；
AddNew method is called to create Analysis Service database, addition packet in the MDStores set of Server object
The association attributes including connection string are included, specify a new data source for Analysis Service database；
Creation cube is closed in Database.MDStores collection, true table, dimension table is created, defines measurement
Data column, and according to business need, set the granularity of dimension；
Setting OlapMode attribute is closed in the MDStores collection of Server object, selects the storage side of cube
Formula；
The processing mode of cube is selected by specifying an optional parameters to Process method.
The present embodiment provides a kind of non-transient computer readable storage medium, the non-transient computer readable storage medium
Matter stores computer instruction, and the computer instruction makes the computer execute great Rong provided by above-mentioned each method embodiment
Data analysing method is measured, for example including：
Select the guide of data source, the supplier of Analysis Service, the position of Analysis server and multi-dimensional database and
Cube；
AddNew method is called to create Analysis Service database, addition packet in the MDStores set of Server object
The association attributes including connection string are included, specify a new data source for Analysis Service database；
Creation cube is closed in Database.MDStores collection, true table, dimension table is created, defines measurement
Data column, and according to business need, set the granularity of dimension；
Setting OlapMode attribute is closed in the MDStores collection of Server object, selects the storage side of cube
Formula；
The processing mode of cube is selected by specifying an optional parameters to Process method.
In conclusion the present invention proposes a kind of Large Volume Data analysis method and device, it is (more in DSO, ADO MD and MDX
Dimension data expression formula) etc. on the basis of technologies, construct a multidimensional data analysis device implementations described herein energy quilt
It is embedded into other application programs, to realize the integrated of entire data warehouse applications, reduces unprofessional user and use management work
The complexity of tool.
Those of ordinary skill in the art will appreciate that：Realize that all or part of the steps of above method embodiment can lead to
The relevant hardware of program instruction is crossed to complete, program above-mentioned can be stored in a computer readable storage medium, the journey
Sequence when being executed, executes step including the steps of the foregoing method embodiments；And storage medium above-mentioned includes：ROM, RAM, magnetic disk or
The various media that can store program code such as person's CD.
The embodiments such as the test equipment of display device described above are only schematical, wherein described be used as is divided
Unit from part description may or may not be physically separated, component shown as a unit can be or
Person may not be physical unit, it can and it is in one place, or may be distributed over multiple network units.It can be with
Some or all of the modules therein is selected to achieve the purpose of the solution of this embodiment according to the actual needs.This field is common
Technical staff is without paying creative labor, it can understands and implements.
Through the above description of the embodiments, those skilled in the art can be understood that each embodiment
It can realize by means of software and necessary general hardware platform, naturally it is also possible to pass through hardware.Based on such reason
Solution, substantially the part that contributes to existing technology can embody above-mentioned technical proposal in the form of software products in other words
Out, which may be stored in a computer readable storage medium, such as ROM/RAM, magnetic disk, CD,
It uses including some instructions so that a computer equipment (can be personal computer, server or the network equipment etc.) is held
Method described in certain parts of each embodiment of row or embodiment.
Finally it should be noted that：The above various embodiments is only to illustrate the technical solution of the embodiment of the present invention, rather than right
It is limited；Although the embodiment of the present invention is described in detail referring to foregoing embodiments, the ordinary skill of this field
Personnel should understand that：It is still possible to modify the technical solutions described in the foregoing embodiments, or to part
Or all technical features are equivalently replaced；And these are modified or replaceed, and do not make the essence of corresponding technical solution de-
Range from each embodiment technical solution of the embodiment of the present invention.
Finally, method of the invention is only preferable embodiment, it is not intended to limit the scope of the present invention.It is all
Within the spirit and principles in the present invention, any modification, equivalent replacement, improvement and so on should be included in guarantor of the invention
Within the scope of shield.
Claims (6)
1. a kind of Large Volume Data analysis method, which is characterized in that including：
Select the guide of data source, the supplier of Analysis Service, the position of Analysis server and multi-dimensional database and multidimensional number
According to collection；
AddNew method is called to create Analysis Service database in the MDStores set of Server object, addition includes connection
Association attributes including character string specify a new data source for Analysis Service database；
Creation cube is closed in Database.MDStores collection, true table, dimension table is created, defines the data of measurement
Column, and according to business need, set the granularity of dimension；
Setting OlapMode attribute is closed in the MDStores collection of Server object, selects the storage mode of cube；
The processing mode of cube is selected by specifying an optional parameters to Process method.
2. Large Volume Data analysis method according to claim 1, which is characterized in that the storage side of the cube
Formula includes multidimensional OLAP, relational OLAP and mixing OLAP.
3. Large Volume Data analysis method according to claim 1, which is characterized in that the granularity for setting dimension is specifically wrapped
It includes：
Creation measurement, position of the specified metric in source database, type and size are concentrated in multidimensional data, and points out measurement
Polymerization methods.
4. a kind of Large Volume Data analytical equipment, which is characterized in that including：
Multiple processors, multiple memories, communication interface and bus；Wherein,
The processor, memory, communication interface complete mutual communication by the bus；
The communication interface is for the information transmission between the test equipment and the communication equipment of display device；
The memory is stored with the program instruction that can be executed by the processor, and the processor calls described program to instruct energy
Enough methods executed as described in claims 1 to 3 is any.
5. a kind of Large Volume Data analytical equipment, which is characterized in that including：
Multiple processors；And
The multiple memories being connect with the processor communication, wherein：
The memory is stored with the program instruction that can be executed by the processor, and the processor calls described program to instruct energy
Enough methods executed as described in claims 1 to 3 is any.
6. a kind of non-transient computer readable storage medium, which is characterized in that the non-transient computer readable storage medium is deposited
Computer instruction is stored up, the computer instruction makes the computer execute the method as described in claims 1 to 3 is any.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810623366.2A CN108875008A (en) | 2018-06-15 | 2018-06-15 | A kind of Large Volume Data analysis method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810623366.2A CN108875008A (en) | 2018-06-15 | 2018-06-15 | A kind of Large Volume Data analysis method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108875008A true CN108875008A (en) | 2018-11-23 |
Family
ID=64339431
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810623366.2A Pending CN108875008A (en) | 2018-06-15 | 2018-06-15 | A kind of Large Volume Data analysis method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108875008A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112970012A (en) * | 2019-01-15 | 2021-06-15 | 谷歌有限责任公司 | System and method for specifying OLAP multidimensional datasets at query time |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103106188A (en) * | 2013-02-21 | 2013-05-15 | 用友软件股份有限公司 | Imaging analysis system and imaging analysis method of data model |
| CN106484875A (en) * | 2016-10-13 | 2017-03-08 | 广州视源电子科技股份有限公司 | Data processing method based on MOLAP and device |
-
2018
- 2018-06-15 CN CN201810623366.2A patent/CN108875008A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103106188A (en) * | 2013-02-21 | 2013-05-15 | 用友软件股份有限公司 | Imaging analysis system and imaging analysis method of data model |
| CN106484875A (en) * | 2016-10-13 | 2017-03-08 | 广州视源电子科技股份有限公司 | Data processing method based on MOLAP and device |
Non-Patent Citations (1)
| Title |
|---|
| 段江娇等: "数据仓库的多维数据分析器设计与实现", 《厦门大学学报（自然科学版）》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112970012A (en) * | 2019-01-15 | 2021-06-15 | 谷歌有限责任公司 | System and method for specifying OLAP multidimensional datasets at query time |
| US11941031B2 (en) | 2019-01-15 | 2024-03-26 | Google Llc | Systems and methods for specifying OLAP cube at query time |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8655861B2 (en) | Query metadata engine | |
| US9223847B2 (en) | Using dimension substitutions in OLAP cubes | |
| US20130166498A1 (en) | Model Based OLAP Cube Framework | |
| US9182963B2 (en) | Computerized migration tool and method | |
| US20160224594A1 (en) | Schema Definition Tool | |
| US20140181154A1 (en) | Generating information models in an in-memory database system | |
| US10922282B2 (en) | On-demand collaboration user interfaces | |
| CN103020158A (en) | Report form creation method, device and system | |
| US20150293947A1 (en) | Validating relationships between entities in a data model | |
| US20140244690A1 (en) | Generate in-memory views from universe schema | |
| US9807169B2 (en) | Distributed tagging of data in a hybrid cloud environment | |
| US20140229223A1 (en) | Integrated erp based planning | |
| US20010037228A1 (en) | System and method for using metadata to flexibly analyze data | |
| US9330140B1 (en) | Transient virtual single tenant queries in a multi-tenant shared database system | |
| US20110145005A1 (en) | Method and system for automatic business content discovery | |
| US10747786B2 (en) | Spontaneous networking | |
| US10552423B2 (en) | Semantic tagging of nodes | |
| EP3486798A1 (en) | Reporting and data governance management | |
| CN112667598B (en) | Data model rapid construction system based on business demand change | |
| CN114238379A (en) | Report query method and system based on SQL generator | |
| CN110377668A (en) | Data analysing method and system | |
| CN108875008A (en) | A kind of Large Volume Data analysis method and device | |
| US20190147082A1 (en) | Reporting and data governance management | |
| US20140136257A1 (en) | In-memory analysis scenario builder | |
| EP2851812A1 (en) | Exposing relationships between universe objects |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181123 |