CN106446279B - Concurrent data analysis method based on Analysis Service pond and dynamic virtual data set - Google Patents
Concurrent data analysis method based on Analysis Service pond and dynamic virtual data set Download PDFInfo
- Publication number
- CN106446279B CN106446279B CN201610928392.7A CN201610928392A CN106446279B CN 106446279 B CN106446279 B CN 106446279B CN 201610928392 A CN201610928392 A CN 201610928392A CN 106446279 B CN106446279 B CN 106446279B
- Authority
- CN
- China
- Prior art keywords
- analysis
- analysis service
- service
- data set
- pond
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
- G06F16/24534—Query rewriting; Transformation
- G06F16/24549—Run-time optimisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/256—Integrating or interfacing systems involving database management systems in federated or virtual databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0637—Strategic management or analysis, e.g. setting a goal or target of an organisation; Planning actions based on goals; Analysis or evaluation of effectiveness of goals
- G06Q10/06375—Prediction of business process outcome or impact based on a proposed change
Abstract
The invention discloses the concurrent data analysis methods based on Analysis Service pond and dynamic virtual data set: step 1, creation analysis service pool；After step 2, client receive the analysis request of user, it is connected to Analysis Service pond application Analysis Service and obtains an idle Analysis Service connection: wherein, when there are multiple analysis requests, each analysis request successively obtains Analysis Service connection from Analysis Service pond；Step 3, client, which connect to service to on-line analysis by Analysis Service, sends analysis request；Step 4, on-line analysis service are analyzed and processed.Overcome the problems, such as that dispersion data set can not be handled present in existing on-line analysis technology and concurrently analyze difficulty, support the data source for carrying out self-dispersing data set and supports simultaneously to analyze this data source.
Description
Technical field
The concurrent data analysis method based on Analysis Service pond and dynamic virtual data set that the present invention relates to a kind of.
Background technique
With the high speed development of information level of the enterprise, enterprises are produced, are monitored, resource management and office are automatic
The application system of change etc. continues to bring out, and forms the IT application in enterprises complication system being made of multiple subsystems.With when
Between passage, have accumulated a large amount of multidimensional model data and business datum in the storage system of complication system.Due to business datum
Excessively huge, the query processing efficiency of individual tables of data storage substantially reduces, therefore numerous systems use temporally period, model
The modes such as object type, object generation sequence carry out data and divide table, and material is thus formed a complicated enterprise-level discrete datas
Collection.
On-line analysis technology is a kind of for relational database or the real time data query analysis hand of data warehouse data collection
Section quickly, neatly carries out complex query and the analysis of mass data according to the requirement of analysis personnel or administrative decision personnel
Processing, its main feature is that complicated query requirement can be responded in a short time.Business datum point is being carried out using on-line analysis technology
When analysis, different user can be from different angles come the business of examining closely, not according to the combination producing of different analytic angle and angle
Same report.
Carrying out on-line analysis currently based on complication system, there are the following problems:
The first, for data distribution in multiple points of tables, on-line analysis is difficult to the querying condition for supporting flexibly to define.
The second, due to the complexity of complication system data and existing analytical technology the characteristics of, on-line analysis concurrency is insufficient,
It is difficult to support multiple users while carrying out complex query.
Summary of the invention
In view of the above-mentioned problems, the present invention provides a kind of concurrent data based on Analysis Service pond and dynamic virtual data set point
Analysis method overcomes the problems, such as that dispersion data set can not be handled present in existing on-line analysis technology and concurrently analyzes difficulty, props up
It holds the data source for carrying out self-dispersing data set and supports while this data source is analyzed.
To realize above-mentioned technical purpose and the technique effect, the invention is realized by the following technical scheme:
Concurrent data analysis method based on Analysis Service pond and dynamic virtual data set, which is characterized in that including as follows
Step:
Step 1, creation analysis service pool:
101, the configuration information in Analysis Service pond is read；
102, the number of concurrent that setting Analysis Service pond is supported；
103, the configuration information of connection creation dummy data set is serviced for each analysis；
After step 2, client receive the analysis request of user, is connected to Analysis Service pond application Analysis Service and obtain one
The Analysis Service of a free time connects: where when there are multiple analysis requests, each analysis request is successively from Analysis Service pond
Obtain Analysis Service connection；
Step 3, client, which connect to service to on-line analysis by Analysis Service, sends analysis request；
Step 4, on-line analysis service are analyzed and processed:
401, the target data ranges that parsing user's request will be handled；
402, it is created as the dynamic virtual data set specified in this service configuration；
403, anolytic sentence is converted by user's request to submit and carry out on-line analysis；
404, analysis result is returned into user by reference format.
It is preferred that judging whether user's request is disposably to handle, if so, Analysis Service connection is returned after step 404
Return Analysis Service pond.
It is preferred that the data format output and input is JSON format in step 4.
It is preferred that, by the input in parsing customer analysis request, storing spy in conjunction with the data of place system in step 402
The dummy data set of target is analyzed in sign, dynamic creation.
The beneficial effects of the present invention are:
Multiple Analysis Service connections are created previously according to the number of concurrent of configuration setting, when there are multiple analysis requests, often
A request successively obtains Analysis Service connection, concurrent Develop Data analysis from Analysis Service pond.Each analysis service is connected to
Before analysis starts, first according to the content creating dummy data set of request, secondly obtained on this data set by parser
As a result, returning to requesting client after result is finally formatted as reference format.Realize to discrete data set it is online simultaneously
Analytic function is sent out, on-line analysis tool is allowed to support to carry out concurrent data point using the enterprise of different data storage scheme
Analysis has achieved the effect that while having analyzed to obtain multiple analysis condition results, saved the time for successively analyzing waiting, improve enterprise
The running efficiency of industry provides more supports for decision of the senior level.
Detailed description of the invention
Fig. 1 is the process signal of the concurrent data analysis method the present invention is based on Analysis Service pond and dynamic virtual data set
Figure；
Fig. 2 is the flow chart in present invention creation Analysis Service pond；
Fig. 3 is the flow chart of on-line analysis service of the present invention.
Specific embodiment
Technical solution of the present invention is described in further detail with specific embodiment with reference to the accompanying drawing, so that ability
The technical staff in domain can better understand the present invention and can be practiced, but illustrated embodiment is not as to limit of the invention
It is fixed.
Concurrent data analysis method based on Analysis Service pond and dynamic virtual data set, the flow chart of entire analysis method
As shown in Figure 1, comparing with traditional method for carrying out data analysis based on specific set of data, the method for the present invention introduces analysis clothes
Business pond and dummy data set, solve concurrent problem analysis and data distribution discontinuous problem respectively.
Specifically comprise the following steps:
Step 1, creation analysis service pool, as shown in Figure 2:
101, the configuration information (configuration file or the configuration source that can be obtained online) in Analysis Service pond is read, comprising: obtain
The information such as database-driven, database link address, log-on message, configuration file catalogue and configuration file name format.
102, the number of concurrent that setting Analysis Service pond is supported, i.e., carry out the setting in Analysis Service pond according to configuration information, if
Surely the number of concurrent (i.e. the quantity of Analysis Service connection) supported.
103, the configuration information of connection creation dummy data set is serviced for each analysis: creation is identical with number of concurrent to match
Confidence breath (file or configuration source), each configuration information include following information: with the one-to-one virtual data set name of the configuration,
The model structure for needing to analyze and relational operator etc..
The Analysis Service pond for supporting concurrently to access is realized by pond technology, and concurrent analysis request is successively from Analysis Service
Analysis Service connection is obtained in pond, is connected by Analysis Service and completes analysis, realizes concurrently dividing online to discrete data set
Analyse function.Analysis Service pool technology creates multiple Analysis Services according to the number of concurrent of configuration setting and connects, when the multiple analyses of appearance
When request, each request successively obtains Analysis Service connection, concurrent Develop Data analysis from Analysis Service pond.
For example, Java, which can be used, edits Analysis Service pond interface, as shown in table 1:
1 Analysis Service pond interface table of table
| Function name | Function |
| instance | Create Analysis Service pond |
| initParams | Initialize service pool parameter |
| getConnection | Obtain service chaining |
| getConnectionCount | Obtain quantity of service |
| returnConnection | Give back service chaining |
| clear | Clear up service pool |
After step 2, client receive the analysis request of user, is connected to Analysis Service pond application Analysis Service and obtain one
The Analysis Service of a free time connects: where when there are multiple analysis requests, each analysis request is successively from Analysis Service pond
Obtain Analysis Service connection.
Step 3, client, which connect to service to on-line analysis by Analysis Service, sends analysis request.
Step 4, on-line analysis service are analyzed and processed, as shown in Figure 3:
401, the target data ranges that parsing user's request will be handled, for example, point in parsing user's request
The information such as target, condition are analysed, target data ranges are therefrom obtained.
402, it is created as the dynamic virtual data set specified in this service configuration: preferably, passing through parsing customer analysis request
In input, in conjunction with the data storage features of place system, therefrom extract creation dummy data set element (such as period,
Table list etc.), the dummy data set of target is analyzed according to element dynamic creation.
For the feature of system data storage, target data ranges are resolved to the creation of creation this analysis service data collection
Sentence, dynamic creation are the dummy data set specified in this service configuration.Such as dividing depositing for table as unit of time interval
Storage system parses the data table name for including according to the start and end time in input and the modeling rule in data source
Claim, then destination virtual data set is created comprising the pictorial representation of these tables by creation.
403, anolytic sentence is converted by user's request to submit and carry out on-line analysis.
404, analysis result is returned into user by reference format.
Analysis Service is connected to before analysis starts, and first according to the content creating dummy data set of request, is secondly counted herein
According on collection by parser obtain as a result, result is finally formatted as reference format after return to requesting client, this hair
It is bright to be directed to data analytic definition following input, output data format, wherein using with JSON(JavaScript Object
Notation, a kind of data interchange format of lightweight) structure data organization form, support customized extension, it is easy to spread
It uses.
Input format:
{ " Id ": " BeginTime ": " EndTime ": " Alg " }
Wherein, Id(unique encodings), the BeginTime(time started), the EndTime(end time) be can be selected in parameter,
Alg(counts destination name) it is mandatory parameter, such as:
{ " 123456 ": " 2015-01-01 00:00:00 ": " 2015-01-15 00:00:00 ": [" group_
Generating_capacity ", " group_radiance "] }
Output format:
{ " Success ": " Data " }
Wherein, Success indicate access as a result, Data indicate result data, such as:
" Success ": true, " Data ": [{ " Object ": " new energy power station 7 ", " MaxLoss ": " 900 " },
{ " Object ": " new energy power station 8 ", " MaxLoss ": " 196 " }, " Oobject ": " new energy power station 0 ", " MaxLoss ":
“10”}]}
After step 404, judge whether user's request is disposably to handle, if so, Analysis Service connection is returned to
Analysis Service pond, i.e. release connection.
Multiple Analysis Service connections are created previously according to the number of concurrent of configuration setting, when there are multiple analysis requests, often
A request successively obtains Analysis Service connection, concurrent Develop Data analysis from Analysis Service pond.Each analysis service is connected to
Before analysis starts, first according to the content creating dummy data set of request, secondly obtained on this data set by parser
As a result, returning to requesting client after result is finally formatted as reference format.Realize to discrete data set it is online simultaneously
Analytic function is sent out, on-line analysis tool is allowed to support to carry out concurrent data point using the enterprise of different data storage scheme
Analysis has achieved the effect that while having analyzed to obtain multiple analysis condition results, saved the time for successively analyzing waiting, improve enterprise
The running efficiency of industry provides more supports for decision of the senior level.
The above is only a preferred embodiment of the present invention, is not intended to limit the scope of the invention, all to utilize this hair
Equivalent structure made by bright specification and accompanying drawing content perhaps equivalent process transformation or be directly or indirectly used in other correlation
Technical field, be included within the scope of the present invention.
Claims (3)
1. the concurrent data analysis method based on Analysis Service pond and dynamic virtual data set, which is characterized in that including walking as follows
It is rapid:
Step 1, creation analysis service pool:
101, the configuration information in Analysis Service pond is read；
102, the number of concurrent that setting Analysis Service pond is supported；
103, the configuration information of connection creation dummy data set is serviced for each analysis；
After step 2, client receive the analysis request of user, is connected to Analysis Service pond application Analysis Service and obtain a sky
Not busy Analysis Service connection: where when there are multiple analysis requests, each analysis request is successively obtained from Analysis Service pond
Analysis Service connection；
Step 3, client, which connect to service to on-line analysis by Analysis Service, sends analysis request；
Step 4, on-line analysis service are analyzed and processed:
401, the target data ranges that parsing user's request will be handled；
402, it is created as the dynamic virtual data set specified in this service configuration；
403, anolytic sentence is converted by user's request to submit and carry out on-line analysis；
404, analysis result is returned into user by reference format；
In step 402, by the input in parsing customer analysis request, in conjunction with the data storage features of place system, dynamic is created
Build the dummy data set of analysis target.
2. the concurrent data analysis method according to claim 1 based on Analysis Service pond and dynamic virtual data set,
It is characterized in that after step 404, judging whether user's request is disposably to handle, if so, Analysis Service connection is returned to
Analysis Service pond.
3. the concurrent data analysis method according to claim 1 based on Analysis Service pond and dynamic virtual data set,
It is characterized in that, in step 4, the data format output and input is JSON format.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610928392.7A CN106446279B (en) | 2016-10-31 | 2016-10-31 | Concurrent data analysis method based on Analysis Service pond and dynamic virtual data set |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610928392.7A CN106446279B (en) | 2016-10-31 | 2016-10-31 | Concurrent data analysis method based on Analysis Service pond and dynamic virtual data set |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106446279A CN106446279A (en) | 2017-02-22 |
| CN106446279B true CN106446279B (en) | 2019-09-27 |
Family
ID=58177252
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610928392.7A Active CN106446279B (en) | 2016-10-31 | 2016-10-31 | Concurrent data analysis method based on Analysis Service pond and dynamic virtual data set |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106446279B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103440302A (en) * | 2013-08-21 | 2013-12-11 | 广东电网公司电力调度控制中心 | Real-time data exchange method and system |
| CN103970527A (en) * | 2013-01-28 | 2014-08-06 | 国际商业机器公司 | Assistive Overlay For Report Generation |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9405914B2 (en) * | 2011-05-10 | 2016-08-02 | Thales Canada Inc. | Data analysis system |
-
2016
- 2016-10-31 CN CN201610928392.7A patent/CN106446279B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103970527A (en) * | 2013-01-28 | 2014-08-06 | 国际商业机器公司 | Assistive Overlay For Report Generation |
| CN103440302A (en) * | 2013-08-21 | 2013-12-11 | 广东电网公司电力调度控制中心 | Real-time data exchange method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106446279A (en) | 2017-02-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110717319B (en) | Self-service report generation method, device, computing equipment and system | |
| WO2018036272A1 (en) | News content pushing method, electronic device, and computer readable storage medium | |
| CN109299320B (en) | Information interaction method and device, computer equipment and storage medium | |
| JP2023506362A (en) | DOCUMENT AUDIT METHOD, APPARATUS, SYSTEM, DEVICE AND STORAGE MEDIUM | |
| CN102521354B (en) | Auditing and testing method and auditing and testing device for data base protocol | |
| CN103778251B (en) | SPARQL parallel query method towards extensive RDF graph data | |
| WO2021082499A1 (en) | Resource annotation management system | |
| JP5624674B2 (en) | How to improve queries for searching databases | |
| CN103246963B (en) | Based on the staffs training system of Internet of Things | |
| CN105574052A (en) | Database query method and apparatus | |
| CN105302906A (en) | Information labeling method and apparatus | |
| CN102411540A (en) | Automatic management system of workflow-based common software testing process | |
| CN103905482B (en) | Method, push server and the system of pushed information | |
| CN115471283B (en) | Advertisement batch delivery method, device, equipment and storage medium | |
| CN104834730B (en) | data analysis system and method | |
| CN101430660A (en) | Pressure model analysis method based on TPS in software performance test | |
| CN110929007A (en) | Electric power marketing knowledge system platform and application method | |
| CN111610281A (en) | Cloud platform framework based on gas chromatography-mass spectrometry library identification and operation method thereof | |
| TW201833730A (en) | Information interaction method and device | |
| CN101645073A (en) | Method for guiding prior database file into embedded type database | |
| CN108600313B (en) | Tourism product release system, method and system docking device | |
| CN106446279B (en) | Concurrent data analysis method based on Analysis Service pond and dynamic virtual data set | |
| CN110909072B (en) | Data table establishment method, device and equipment | |
| CN112052248A (en) | Audit big data processing method and system | |
| CN111027093A (en) | Access right control method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |