CN102567353B - Data analysis device and data analysis method based on dynamically expandable analysis factors - Google Patents
Data analysis device and data analysis method based on dynamically expandable analysis factors Download PDFInfo
- Publication number
- CN102567353B CN102567353B CN201010599831.7A CN201010599831A CN102567353B CN 102567353 B CN102567353 B CN 102567353B CN 201010599831 A CN201010599831 A CN 201010599831A CN 102567353 B CN102567353 B CN 102567353B
- Authority
- CN
- China
- Prior art keywords
- analysis
- data
- factor
- analysis factor
- configuration
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
The invention provides a data analysis device and a data analysis method based on dynamically expandable analysis factors. The data analysis device based on the dynamically expandable analysis factors comprises a configuration module, a data extraction module, an analysis factor calculation module, a data analysis and processing module and an analysis result output module. By the aid of the data analysis device and the data analysis method based on the dynamically expandable analysis factors, expansion is easy, instantaneity and accuracy can be guaranteed, and systematic operating efficiency and performance can be enhanced.
Description
Technical field
The present invention relates to data analysis set-up and method, more specifically, relate to the data analysis set-up based on the analysis factor of dynamic extending and method.
Background technology
At present, along with the becoming increasingly abundant of class of business of the growing of information data processing demands and different field, the data analysis based on the analysis factor (each Essential Elements Of Analysis namely used in data analysis) of dynamic extending becomes more and more important.
Usually, existing based on the data analysing method of analysis factor and the ultimate principle of device as follows: from data source extract raw data; Also build data model and calculate each analysis factor simultaneously according to the rule preset (described rule pre-determines based on each embody rule) based on described raw data; Calculate analysis result based on described analysis factor and according to rule (described rule based on each application demand pre-determine) the creation analysis model preset; Send described analysis result to application server and be used for subsequent treatment.
But, due to becoming increasingly abundant along with class of business, in data analysis, need the analysis factor used to get more and more, the change simultaneously based on the data analysis rule of embody rule demand is also more and more frequent, thus needs often to adjust dynamically analysis factor and data analysis rule.Therefore, there are the following problems for above-mentioned existing technical scheme: when needing extensive diagnostic because of the period of the day from 11 p.m. to 1 a.m (during the analysis factor that namely use of data analysis rule needs is newly-increased), if this analysis factor is not within the scope of the analysis factor preset, also cannot obtain through simple operation according to the analysis factor preset, then must revise system code in artificially, thus the requirement of system to real-time can not be met; Meanwhile, which is consuming time more, and thus maintenance cost raises, and due to process loaded down with trivial details and be easy to make mistakes, therefore reduce work efficiency and the performance of whole system; In addition, when data analysis rule needs to change, too must artificially amendment system code.
Therefore, there is following demand: provide a kind of and be easy to expand, real-time and accuracy can be guaranteed and data analysis set-up and the method for the analysis factor based on dynamic extending of system works efficiency and performance can be improved.
Summary of the invention
In order to solve the defect existing for above-mentioned prior art, the present invention proposes a kind of data analysis set-up and method of the analysis factor based on dynamic extending.
The object of the invention is to be achieved through the following technical solutions:
Based on a data analysis set-up for the analysis factor of dynamic extending, the data analysis set-up of the described analysis factor based on dynamic extending comprises:
Configuration module, described configuration module is used for arranging described configuration file according to the input of user;
Data extraction module, described data extraction module is used for extracting described raw data from data source;
Analysis factor computing module, described analysis factor computing module is used for building data model based on described raw data and described configuration file and calculating the value of at least one analysis factor;
Data Analysis Services module, described Data Analysis Services module is used for based on the value of at least one analysis factor described and described configuration file creation analysis model and calculates analysis result;
Analysis result output module, described analysis result output module is used for described analysis result to be sent to application server.
In scheme disclosed above, preferably, described configuration file comprises the configuration data of analysis factor, the configuration data definition raw data of described analysis factor and the mapping relations of analysis factor.
In scheme disclosed above, preferably, the configuration data of described analysis factor is the form of extend markup language.
In scheme disclosed above, preferably, the configuration data of described analysis factor comprises at least one in following key element: raw data mark, statistical measures, statistics dimension, statistical and filtercondition.
In scheme disclosed above, preferably, described configuration file comprises the configuration data of analysis rule, the configuration data defined analysis factor of described analysis rule and the mapping relations of analytical model.
In scheme disclosed above, preferably, the configuration data of described analysis rule is the form of extend markup language.
In scheme disclosed above, preferably, described analysis factor computing module by analysis factor data-interface by the value transmit of at least one analysis factor described in calculating to described Data Analysis Services module.
In scheme disclosed above, preferably, described analysis factor data-interface adopts Hash mapping table (MAP) structure, and the output of wherein said analysis factor data-interface is the key-value (KEY-VALUE) of the analysis factor calculated.
In scheme disclosed above, preferably, at least one analysis factor described is the key element in described data model.
In scheme disclosed above, preferably, described analysis factor computing module comprises analysis factor statistic unit further, and described analysis factor statistic unit is used for carrying out statistical computation to the value of at least one analysis factor described.
In scheme disclosed above, preferably, described analysis factor statistic unit comprises at least one in following statistical calculation mode: sue for peace (Sum), get minimum value (Min), get maximal value (Max).
In scheme disclosed above, preferably, described analytical model is the set of Logic judgment rule of being mutually related.
Object of the present invention is also achieved through the following technical solutions:
Based on a data analysing method for the analysis factor of dynamic extending, the data analysing method of the described analysis factor based on dynamic extending comprises the steps:
(A1) configuration file is set according to the input of user:
(A2) raw data is extracted from data source;
(A3) build data model based on described raw data and described configuration file and calculate the value of at least one analysis factor;
(A4) analysis result is calculated based on the value of at least one analysis factor described and described configuration file creation analysis model;
(A5) described analysis result is sent to application server.
In scheme disclosed above, preferably, described configuration file comprises the configuration data of analysis factor, the configuration data definition raw data of described analysis factor and the mapping relations of analysis factor.
In scheme disclosed above, preferably, the configuration data of described analysis factor is the form of extend markup language.
In scheme disclosed above, preferably, the configuration data of described analysis factor comprises at least one in following key element: raw data mark, statistical measures, statistics dimension, statistical and filtercondition.
In scheme disclosed above, preferably, described configuration file comprises the configuration data of analysis rule, the configuration data defined analysis factor of described analysis rule and the mapping relations of analytical model.
In scheme disclosed above, preferably, the configuration data of described analysis rule is the form of extend markup language.
In scheme disclosed above, preferably, the value of at least one analysis factor described in described method is calculated by the transmission of analysis factor data-interface.
In scheme disclosed above, preferably, described analysis factor data-interface adopts Hash mapping table (MAP) structure, and the output of wherein said analysis factor data-interface is the key-value (KEY-VALUE) of the analysis factor calculated.
In scheme disclosed above, preferably, at least one analysis factor described is the key element in described data model.
In scheme disclosed above, preferably, described step (A3) comprises further:
(B1) statistical computation is carried out to the value of at least one analysis factor described.
In scheme disclosed above, preferably, described statistical computation comprises at least one in following statistical calculation mode: sue for peace (Sum), get minimum value (Min), get maximal value (Max).
In scheme disclosed above, preferably, described analytical model is the set of Logic judgment rule of being mutually related.
Operational order automatically generating device and the method tool of structure based query language disclosed in this invention have the following advantages: be easy to extensive diagnostic Summing Factor analysis rule; Real-time and accuracy can be guaranteed; System works efficiency and performance can be improved.
Accompanying drawing explanation
By reference to the accompanying drawings, technical characteristic of the present invention and advantage will be understood better by those skilled in the art, wherein:
Fig. 1 is according to an embodiment of the invention based on the structural drawing of the data analysis set-up of the analysis factor of dynamic extending;
Fig. 2 is according to an embodiment of the invention based on the process flow diagram of the data analysing method of the analysis factor of dynamic extending;
Embodiment
Fig. 1 is according to an embodiment of the invention based on the structural drawing of the data analysis set-up of the analysis factor of dynamic extending.As shown in Figure 1, the data analysis set-up 1 of the analysis factor based on dynamic extending disclosed in this invention is for carrying out data analysis based on the analysis rule in raw data and configuration file.As shown in Figure 1, described data analysis set-up 1 comprises configuration module 2, data extraction module 3, analysis factor computing module 4, Data Analysis Services module 5 and analysis result output module 6.Wherein, described configuration module 2 is for arranging described configuration file according to the input of user.Described data extraction module 3 is for extracting described raw data from data source.Described analysis factor computing module 4 is for building data model based on described raw data and described configuration file and calculating the value of at least one analysis factor.Described Data Analysis Services module 5 is for calculating analysis result based on the value of at least one analysis factor described and described configuration file creation analysis model.Described analysis result output module 6 is for being sent to application server (such as safety detection server) by described analysis result.
Preferably, in data analysis set-up disclosed in this invention, described configuration file comprises the configuration data of analysis factor, the configuration data definition raw data of described analysis factor and the mapping relations (i.e. logical relation) of analysis factor.
Preferably, the configuration data of described analysis factor is the form of XML (extend markup language).
Preferably, the configuration data of described analysis factor comprises at least one in following key element: raw data mark, statistical measures, statistics dimension, statistical and filtercondition.
Preferably, in data analysis set-up disclosed in this invention, described configuration file comprises the configuration data of analysis rule, the mapping relations (i.e. logical relation) of the configuration data defined analysis factor of described analysis rule and analytical model (i.e. business rule).
Preferably, the configuration data of described analysis rule is the form of XML (extend markup language).
Preferably, in data analysis set-up disclosed in this invention, the value transmit of at least one analysis factor described in calculating is given described Data Analysis Services module 5 by analysis factor data-interface by described analysis factor computing module 4.Wherein, preferably, described analysis factor data-interface adopts Hash mapping table (MAP) structure, and namely the output of described analysis factor data-interface is the key-value (KEY-VALUE) of the analysis factor calculated.
Preferably, in data analysis set-up disclosed in this invention, at least one analysis factor described is the key element in described data model (i.e. business model).
Preferably, in data analysis set-up disclosed in this invention, described analysis factor computing module 4 comprises analysis factor statistic unit further, for carrying out statistical computation to the value of at least one analysis factor described.Described analysis factor statistic unit can comprise at least one in following statistical calculation mode: sue for peace (Sum), get minimum value (Min), get maximal value (Max).
Preferably, in data analysis set-up disclosed in this invention, described analytical model is the set of Logic judgment rule of being mutually related.
As shown in Figure 1, exemplarily, the basic functional principle of data analysis set-up disclosed in this invention is as follows: arrange configuration file according to user's input by configuration module 2; Described data extraction module 3 extracts raw data from data source; Described analysis factor computing module 4 builds data model based on the configuration data (i.e. user setting business model) of the analysis factor in described configuration file and described raw data and calculates the value of at least one analysis factor; The value transmit of at least one analysis factor described in calculating is given described Data Analysis Services module 5 by analysis factor data-interface by described analysis factor computing module 4; Described Data Analysis Services module 5 is based on the configuration data (i.e. the analysis rule of regulation engine of user's setting) of the analysis rule in described configuration file and the value creation analysis model of at least one analysis factor described and calculate analysis result; Described analysis result is sent to application server by described analysis result output module 6.
Exemplarily, it is mutual that data analysis set-up disclosed in this invention is applied to safety information, such as System of Financial Risk Management.Described raw data can be such as transaction attribute information, merchant information etc.Described application server can be safety detection server.
Fig. 2 is according to an embodiment of the invention based on the process flow diagram of the data analysing method of the analysis factor of dynamic extending.As shown in Figure 2, the data analysing method of the analysis factor based on dynamic extending disclosed in this invention comprises the steps: that (A1) arranges configuration file according to the input of user: (A2) extracts raw data from data source; (A3) build data model based on described raw data and described configuration file and calculate the value of at least one analysis factor; (A4) analysis result is calculated based on the value of at least one analysis factor described and described configuration file creation analysis model; (A5) described analysis result is sent to application server (such as safety detection server).
Preferably, in data analysing method disclosed in this invention, described configuration file comprises the configuration data of analysis factor, the configuration data definition raw data of described analysis factor and the mapping relations (i.e. logical relation) of analysis factor.
Preferably, the configuration data of described analysis factor is the form of XML (extend markup language).
Preferably, the configuration data of described analysis factor comprises at least one in following key element: raw data mark, statistical measures, statistics dimension, statistical and filtercondition.
Preferably, in data analysing method disclosed in this invention, described configuration file comprises the configuration data of analysis rule, the mapping relations (i.e. logical relation) of the configuration data defined analysis factor of described analysis rule and analytical model (i.e. business rule).
Preferably, the configuration data of described analysis rule is the form of XML (extend markup language).
Preferably, in data analysing method disclosed in this invention, the value of at least one analysis factor described in being calculated by the transmission of analysis factor data-interface.Wherein, preferably, described analysis factor data-interface adopts Hash mapping table (MAP) structure, and namely the output of described analysis factor data-interface is the key-value (KEY-VALUE) of the analysis factor calculated.
Preferably, in data analysing method disclosed in this invention, at least one analysis factor described is the key element in described data model (i.e. business model).
Preferably, in data analysing method disclosed in this invention, described step (A3) comprises further: (B1) carries out statistical computation to the value of at least one analysis factor described.Described statistical computation comprises at least one in following statistical calculation mode: sue for peace (Sum), get minimum value (Min), get maximal value (Max).
Preferably, in data analysing method disclosed in this invention, described analytical model is the set of Logic judgment rule of being mutually related.
Although the present invention is described by above-mentioned preferred implementation, its way of realization is not limited to above-mentioned embodiment.Should be realized that: when not departing from purport of the present invention and scope, those skilled in the art can make different changes and amendment to the present invention.
Claims (20)
1., based on a data analysis set-up for the analysis factor of dynamic extending, the data analysis set-up of the described analysis factor based on dynamic extending comprises:
Configuration module, described configuration module is used for arranging configuration file according to the input of user;
Data extraction module, described data extraction module is used for extracting raw data from data source;
Analysis factor computing module, described analysis factor computing module is used for building data model based on described raw data and described configuration file and calculating the value of at least one analysis factor;
Data Analysis Services module, described Data Analysis Services module is used for based on the value of at least one analysis factor described and described configuration file creation analysis model and calculates analysis result;
Analysis result output module, described analysis result output module is used for described analysis result to be sent to application server;
Wherein, the value transmit of at least one analysis factor described in calculating is given described Data Analysis Services module by analysis factor data-interface by described analysis factor computing module, and described analysis factor data-interface adopts Hash mapping list structure, and the output of wherein said analysis factor data-interface is the key-value of the analysis factor calculated.
2. the data analysis set-up of the analysis factor based on dynamic extending according to claim 1, it is characterized in that, described configuration file comprises the configuration data of analysis factor, the configuration data definition raw data of described analysis factor and the mapping relations of analysis factor.
3. the data analysis set-up of the analysis factor based on dynamic extending according to claim 2, is characterized in that, the configuration data of described analysis factor is the form of extend markup language.
4. the data analysis set-up of the analysis factor based on dynamic extending according to claim 3, it is characterized in that, the configuration data of described analysis factor comprises at least one in following key element: raw data mark, statistical measures, statistics dimension, statistical and filtercondition.
5. the data analysis set-up of the analysis factor based on dynamic extending according to claim 4, it is characterized in that, described configuration file comprises the configuration data of analysis rule, the configuration data defined analysis factor of described analysis rule and the mapping relations of analytical model.
6. the data analysis set-up of the analysis factor based on dynamic extending according to claim 5, is characterized in that, the configuration data of described analysis rule is the form of extend markup language.
7. the data analysis set-up of the analysis factor based on dynamic extending according to claim 1, is characterized in that, at least one analysis factor described is the key element in described data model.
8. the data analysis set-up of the analysis factor based on dynamic extending according to claim 7, it is characterized in that, described analysis factor computing module comprises analysis factor statistic unit further, and described analysis factor statistic unit is used for carrying out statistical computation to the value of at least one analysis factor described.
9. the data analysis set-up of the analysis factor based on dynamic extending according to claim 8, is characterized in that, described analysis factor statistic unit comprises at least one in following statistical calculation mode: sue for peace, get minimum value, get maximal value.
10. the data analysis set-up of the analysis factor based on dynamic extending according to claim 9, is characterized in that, described analytical model is the set of Logic judgment rule of being mutually related.
11. 1 kinds of data analysing methods based on the analysis factor of dynamic extending, the data analysing method of the described analysis factor based on dynamic extending comprises the steps:
(A1) configuration file is set according to the input of user:
(A2) raw data is extracted from data source;
(A3) build data model based on described raw data and described configuration file and calculate the value of at least one analysis factor;
(A4) analysis result is calculated based on the value of at least one analysis factor described and described configuration file creation analysis model;
(A5) described analysis result is sent to application server;
Wherein, the value of at least one analysis factor described in described method is calculated by the transmission of analysis factor data-interface, and described analysis factor data-interface adopts Hash mapping list structure, and the output of wherein said analysis factor data-interface is the key-value of the analysis factor calculated.
The data analysing method of 12. analysis factors based on dynamic extending according to claim 11, it is characterized in that, described configuration file comprises the configuration data of analysis factor, the configuration data definition raw data of described analysis factor and the mapping relations of analysis factor.
The data analysing method of 13. analysis factors based on dynamic extending according to claim 12, is characterized in that, the configuration data of described analysis factor is the form of extend markup language.
The data analysing method of 14. analysis factors based on dynamic extending according to claim 13, it is characterized in that, the configuration data of described analysis factor comprises at least one in following key element: raw data mark, statistical measures, statistics dimension, statistical and filtercondition.
The data analysing method of 15. analysis factors based on dynamic extending according to claim 14, it is characterized in that, described configuration file comprises the configuration data of analysis rule, the configuration data defined analysis factor of described analysis rule and the mapping relations of analytical model.
The data analysing method of 16. analysis factors based on dynamic extending according to claim 15, is characterized in that, the configuration data of described analysis rule is the form of extend markup language.
The data analysing method of 17. analysis factors based on dynamic extending according to claim 11, is characterized in that, at least one analysis factor described is the key element in described data model.
The data analysing method of 18. analysis factors based on dynamic extending according to claim 17, is characterized in that, described step (A3) comprises further:
(B1) statistical computation is carried out to the value of at least one analysis factor described.
The data analysing method of 19. analysis factors based on dynamic extending according to claim 18, is characterized in that, described statistical computation comprises at least one in following statistical calculation mode: sue for peace, get minimum value, get maximal value.
The data analysing method of 20. analysis factors based on dynamic extending according to claim 19, is characterized in that, described analytical model is the set of Logic judgment rule of being mutually related.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201010599831.7A CN102567353B (en) | 2010-12-17 | 2010-12-17 | Data analysis device and data analysis method based on dynamically expandable analysis factors |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201010599831.7A CN102567353B (en) | 2010-12-17 | 2010-12-17 | Data analysis device and data analysis method based on dynamically expandable analysis factors |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102567353A CN102567353A (en) | 2012-07-11 |
| CN102567353B true CN102567353B (en) | 2015-02-18 |
Family
ID=46412793
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201010599831.7A Active CN102567353B (en) | 2010-12-17 | 2010-12-17 | Data analysis device and data analysis method based on dynamically expandable analysis factors |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102567353B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102541811B (en) * | 2010-12-27 | 2015-02-18 | 中国银联股份有限公司 | On-demand computing-based data analysis device and method for analysis factors |
| CN110827070B (en) * | 2019-10-30 | 2023-05-23 | 神州数码融信软件有限公司 | User growth calculation method and device based on dynamic expansion factors |
-
2010
- 2010-12-17 CN CN201010599831.7A patent/CN102567353B/en active Active
Non-Patent Citations (2)
| Title |
|---|
| 基于J2EE和多维数据模型的金融数据分析系统;王凯等;《中国优秀硕士学位论文全文数据库》;20051231;全文 * |
| 沈臻等.面向电信企业的商业智能分析系统.《微计算机信息(管控一体化)》.2008,第24卷(第3-3期),第23页第1.2节-第24页左栏10行,第24页左栏第2节"用户定制功能的实现",第25页左栏"结束语"部分. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102567353A (en) | 2012-07-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108647330B (en) | 3D lightweight conversion method based on BIM model file | |
| CN110348441B (en) | Value-added tax invoice identification method and device, computer equipment and storage medium | |
| CN103399851B (en) | Method and system for analyzing and predicting performance of structured query language (SQL) scrip | |
| CN105224631B (en) | The method built the system of the open cloud of industry and work out XBRL financial statement | |
| CN102541811B (en) | On-demand computing-based data analysis device and method for analysis factors | |
| WO2009094290A3 (en) | System and method of business model management | |
| CN103064664A (en) | Hadoop parameter automatic optimization method and system based on performance pre-evaluation | |
| CN109299074B (en) | Data verification method and system based on templated database view | |
| CN109063362B (en) | Avionics software interface control file design management system | |
| CN108228726B (en) | Incremental transaction content acquisition method and storage medium for distribution network red and black images | |
| CN114035793A (en) | Page generation method, page generation device, equipment and storage medium | |
| CN102567353B (en) | Data analysis device and data analysis method based on dynamically expandable analysis factors | |
| CN104268713A (en) | Performance assessment computing method and system | |
| CN103257861B (en) | A kind of method of automatic generation bios code and device | |
| CN102968305B (en) | Logical process method, device and evaluation system | |
| CN111008189B (en) | Dynamic data model construction method | |
| CN106713516A (en) | Method for quickly making mirror image of oVirt cloud platform computing node | |
| CN114064012B (en) | Dynamic and static combined interface code generation method and system and electronic equipment | |
| CN104933119A (en) | Big data management method | |
| CN116155689A (en) | ClickHouse-based high-availability Kong gateway log analysis method and system | |
| CN105205168A (en) | Exposure system based on Redis database and operation method thereof | |
| KR101449725B1 (en) | Apparatus and method for converting pdf document | |
| CN104182522A (en) | Secondary indexing method and device on basis of circulation bitmap model | |
| CN103077284A (en) | Method and system for automatically generating auxiliary code file by using general text template | |
| CN203911987U (en) | Data processing system based on cloud computing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |