CN102521246A - Cloud data warehouse system - Google Patents
Cloud data warehouse system Download PDFInfo
- Publication number
- CN102521246A CN102521246A CN2011103580170A CN201110358017A CN102521246A CN 102521246 A CN102521246 A CN 102521246A CN 2011103580170 A CN2011103580170 A CN 2011103580170A CN 201110358017 A CN201110358017 A CN 201110358017A CN 102521246 A CN102521246 A CN 102521246A
- Authority
- CN
- China
- Prior art keywords
- data
- module
- mining
- main control
- hbase database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004458 analytical method Methods 0.000 claims abstract description 51
- 238000007418 data mining Methods 0.000 claims abstract description 39
- 238000004422 calculation algorithm Methods 0.000 claims description 25
- 238000013507 mapping Methods 0.000 claims description 14
- 238000010586 diagram Methods 0.000 claims description 8
- 238000005065 mining Methods 0.000 claims description 6
- 238000012098 association analyses Methods 0.000 claims description 3
- 238000010224 classification analysis Methods 0.000 claims description 3
- 230000001960 triggered effect Effects 0.000 claims description 3
- 230000005540 biological transmission Effects 0.000 claims description 2
- 238000000605 extraction Methods 0.000 claims description 2
- 238000004364 calculation method Methods 0.000 abstract description 3
- 238000012545 processing Methods 0.000 abstract description 3
- 238000000034 method Methods 0.000 description 9
- 238000005516 engineering process Methods 0.000 description 8
- 239000000284 extract Substances 0.000 description 7
- 230000005611 electricity Effects 0.000 description 5
- 230000010354 integration Effects 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000007621 cluster analysis Methods 0.000 description 2
- 241001310793 Podium Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000002354 daily effect Effects 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 230000001747 exhibiting effect Effects 0.000 description 1
- 238000000547 structure data Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 230000005612 types of electricity Effects 0.000 description 1
Images
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a cloud data warehouse system for supporting mass data processing and providing a data mining service, a multidimensional analysis service and a data presentation service. The cloud data warehouse system mainly comprises a main control module, a data loading module and a data mining module, wherein the main control module is used for sending instructions to other application modules and controlling the flow direction of data stream; the data loading module is used for loading data from an external database; and the data mining module is used for carrying out data mining calculation on the data. The cloud data warehouse system further comprises a multidimensional analysis module for carrying out multidimensional analysis on the data and a data presentation module for presenting the data by means of making reports.
Description
Technical field
The present invention relates to the data warehouse technology field, particularly a kind of cloud data warehouse.
Background technology
Along with continuing to increase of process information amount, enterprise needs multi-angle to handle magnanimity information and therefrom obtains the information of supporting decision-making.Operational data storehouse towards issued transaction seems unable to do what one wishes at this moment, and the data warehouse technology of the integrated mass data of subject-oriented produces thus.Data warehouse be a subject-oriented, integrated, metastable, the reflection historical variations data acquisition; Be used to support management decision; Data warehouse can get up the data integration of a plurality of systems, carries out data analysis according to different themes then, and analysis result is used for supporting decision-making.
The cloud computing technology is a kind of novel commercial podium, has brought new Information Service Mode.Cloud computing is the recent development of computation schemas such as Distributed Calculation, parallel processing, grid computing.Cloud computing through resources such as various interconnected calculating, storage, data, application are effectively integrated realize multi-level virtual and abstract.
The instrument of common structure data warehouse has IBM Cognos and SAP B0 etc.But above-mentioned both adopts centralized single node pattern to make up data warehouse, and the extendability of such data warehouse is poor.Along with the scale of present business data sharply enlarges, some has reached the rank of TB, and above-mentioned traditional data warehouse can not satisfy the use needs all the more.
In addition, existing data warehouse concentrates on the multidimensional analysis aspect mostly to the analysis of data, and is limited to supporting based on the Knowledge Discovery of data mining technology, can not satisfy enterprise excavates valuable information from mass data needs.
Summary of the invention
In view of this, fundamental purpose of the present invention provides the cloud data warehouse, possesses good extendability, and in the data pick-up that the traditional data warehouse is provided, multidimensional analysis service, supports the data mining service.Satisfy the application demand of present stage.
Cloud data warehouse according to the invention, concrete technical scheme is following:
A kind of cloud data warehouse; It is characterized in that; Said system cloud data warehouse is structured on Hadoop platform and the operating system software; Built-in HBase database of Hadoop platform and data warehouse instrument Hive, the structure of cloud data warehouse comprises that main control module, data insmod and data-mining module:
The main control module, the module that the timer-triggered scheduler service function embodies in the operating system, the main control module insmods and data-mining module transmission instruction, the flow direction of control data stream to data;
Data insmod, and an end connects external data base, and an end is got in touch the HBase database, and are connected with the main control module, be responsible for data in the external data base are written into the HBase database, or data in the HBase database are written into external data base;
Data-mining module is connected to the HBase database, and is connected with the main control module, is responsible for integrating data among the HBase, and data mining algorithm is provided, the rule and the pattern of calculating and mining data.
Preferably, said system further comprises:
The multidimensional analysis module is connected to the data warehouse instrument Hive in the HBase database, and is connected with the main control module, and the data warehouse instrument Hive that is used for calling HBase carries out the multidimensional analysis service.
Preferably, said system further comprises:
The data display module is connected to the HBase database, and is connected with the main control module, is used for the data of cloud data warehouse are passed through the embodied of form or synoptic diagram.
Said data insmod and specifically are used for:
Data insmod and accept the instruction that is written into data of main control module; Extracted data from external data base; According to the required formatting parameter of HBase database; And be the required form of HBase database according to the parameter that is provided with data-switching, the data that will meet form are written into the HBase database.
Said data-mining module specifically is used for:
Data-mining module is accepted the instruction that the main control module is carried out data mining analysis; Begin to integrate the data that to carry out data mining analysis in the HBase database; Write in the table; Seek the data mining algorithm in the data-mining module again,, obtain the rule and the pattern of data carrying out computing and analyze operation result through the data of integrating.
Said searching data mining algorithm specifically comprises:
Association analysis algorithm, forecast analysis algorithm, cluster algorithm, classification analysis algorithm, outlier analytical algorithm or the algorithm combination in any of writing voluntarily.
Said multidimensional analysis module specifically is used for:
After the multidimensional analysis module is accepted the main control module and is carried out the instruction of multidimensional analysis; Instrument Hive in order data warehouse is bivariate table data among the Hive with the data map that the needs of storing based on row in the HBase database carry out multidimensional analysis again; Mapping ruler is set simultaneously sends to data warehouse instrument Hive; Hive receive orders with mapping ruler after, connect the HBase database, extracted data is also accomplished the mapping of data.
Said data display module specifically is used for:
The data display module is accepted the instruction of main control modules exhibit data, and extraction needs the data of displaying from the HBase database, and the pattern of data display is set, again with the embodied of data through form or synoptic diagram.
Can know that through above technical scheme the beneficial effect that the present invention exists is: make up data warehouse based on the cloud computing technology, make data warehouse have natural good extendability.Then through main control module controls data insmod, data-mining module, multidimensional analysis module and data display module support that the data pick-up of data warehouse is written into, multidimensional analysis, data mining and the multinomial service of report making.
Description of drawings
Fig. 1: cloud data warehouse overall architecture synoptic diagram.
Fig. 2: power consumption classification synoptic diagram.
Embodiment
The invention provides a kind ofly, will combine accompanying drawing that technical scheme of the present invention is carried out complete description below based on cloud computing technique construction data warehouse.And described embodiment only is part embodiment among the present invention.Based on the embodiment among the present invention, those of ordinary skills are not passing through other embodiments that draw under the creative work situation, belong to the scope of the present invention's protection equally.
With reference to Fig. 1, the structure of data warehouse according to the invention comprises:
The Hadoop ecosystem of Hadoop and correlator item design thereof, the Hadoop ecosystem provide a basic platform of using the cloud computing technology.In the overall architecture of cloud data warehouse, the HBase database is implemented on the Hadoop, and Hive is based on the data warehouse instrument of Hadoop.Hadoop has realized that distributed file system HDFS and MapReduce calculate framework, and HDFS makes that Hadoop has that extendability is good, safety, economic dispatch advantage, thus the natural succession of the cloud data warehouse of framework on Hadoop these characteristics.MapReduce adopts mobile computing but not the thought of mobile data is carried out distributed, parallel computation to distributed storage in the mass data of HDFS, has improved the speed of calculating greatly.
The data that go out based on the Hadoop ecosystem exploitation among the present invention insmod, and data-mining module and data display module all are based on the HBase database and make up, and are connected with HBase, operate to the data among the HBase.The multidimensional analysis module makes up based on data warehouse instrument Hive, and according to concrete service needed the Hive module is called.
The main control module of system is actually the operating system software that exists based on system and forms, and the timer-triggered scheduler service abstraction that operating system software is provided is that module embodies.The main control module is sent instruction to each module, controls the operation of each module and the trend of data stream.
Data insmod; Be Extract TransformLoad; Be called for short ETL; It provides the interface between multiple isomeric data such as traditional relational database data, Document type data and the cloud data warehouse, supports multiple isomeric data being written in the cloud data warehouse, and being written in relevant database of the data in the cloud data warehouse.ETL module one end is connected with external relations type database, and an end connects the HBase database.
The concrete steps that the ETL module is written into data are following:
1, the connection parameter of input external relations type database comprises: IP address, Service name, user name, password;
2, input ETL parameter comprises: the structure of target data, transformation rule, degree of parallelism, exception handling among data source, major key, delta field, the HBase;
3, ETL module extracted data from external relations type database, and from the Map function of MapReduce calculating framework, extract the SQL statement that is used for translation data;
4, the Reduce phase data is changed and is written into, and according to the ETL parameter of setting and the SQL statement in the Map function, will become target data structure from the aerial data-switching that extracts of external relations type data, each field during the corresponding HBase of acquisition shows; Data after will changing then write HBase;
If it is unusual that the transfer process of 5 certain bar record takes place, according to the exception handling that defines, this record is abandoned, and write and abandon daily record, continue next bar record of conversion then, to all record conversions be written into completion.
The realization of ETL module has made full use of the MapReduce distributed parallel calculating framework that Hadoop provides, and has realized the parallelization of ETL process; The ETL module is written into efficient when relevant database is written into data to the cloud data warehouse, having adopted the incrementization technology thereby improved data greatly simultaneously.
The multidimensional analysis module, promptly On-Line Analytical Processing abbreviates the OLAP module as, and this module connects the data warehouse instrument Hive among the HBase, and the multidimensional analysis function is accomplished in the work of being responsible for calling data warehouse instrument Hive.
The concrete course of work of OLAP module is following:
The OLAP module is at first sent instruction to Hive, and indication Hive extracts the data that need multidimensional analysis among the HBase, and data are mapped as the two-dimensional table format among the Hive based on the row formats stored in HBase, and the OLAP module is that Hive formulates mapping ruler simultaneously.The instruction that Hive receives mapping (enum) data can be connected the HBase database afterwards with mapping ruler, and extracted data is also accomplished mapping process.The OLAP module is carried out the multidimensional analysis type of data as required afterwards, and indication Hive sets up a form that stores analysis result in HBase, and indication Hive also is mapped as a blank bivariate table with this form in Hive.Data warehouse instrument Hive carries out multidimensional analysis to data in the bivariate table, and the data after analyzing are write blank bivariate table.After analyzing completion, the bivariate table of data converted into again among the HBase and is stored among the HBase based on the row formats stored after OLAP module indication Hive will write and analyze.
Data-mining module; Be Data Mining, be abbreviated as the DM module, this module provides the data mining algorithm of plurality of classes; Support that the user carries out the Knowledge Discovery analysis of plurality of classes such as association analysis, forecast analysis, cluster analysis, classification analysis, outlier analysis in the cloud data warehouse; Calculate framework through MapReduce and realize the data computation analysis,, support user's decision-making for the user provides more valuable knowledge.
The concrete steps of DM module are:
Data integration is at first created new table in HBase, with the data integration that need carry out mining analysis among the HBase to before in the table created; Seek the data mining algorithm in the DM module again,, select corresponding algorithm to carry out computing, realize dissimilar analyses such as classification, cluster, prediction, find the rule and the pattern of data to the mining analysis of various objectives.
Below provide the specific embodiment explanation to the practical application in the network system:
With reference to shown in Figure 2, the analysis of domestic consumer's electricity consumption behavior is the basis with each electricity consumer electricity consumption data hourly, and the clustering algorithm in the DM module of use cloud data warehouse carries out cluster analysis to it.The result who obtains according to algorithm is divided into five types of A, B, C, D, E with electricity consumer, and the electricity consumption rule of every type of electricity consumer has nothing in common with each other, and can find out the difference between them intuitively from the average power consumption rule of every type of 24 hours every day of user.
This shows that category-A user whole day power consumption is very low always, this type user possibly belong to vacant room user, and very low power consumption comes from line loss; Category-B user power utilization amount begins to rise from 6:00, and daytime, power consumption kept certain level, and evening, power consumption rose, but the comparison that downtrending occurs early, and this type user possibly be the family that the elderly lives; C class user has tangible crest and trough, and daytime, power consumption was very low, and evening, peak of power consumption was higher than category-B, and downtrending occur also late than category-B, this type user possibly be the family that the working clan lives; The D class is the comprehensive of two types of B, C basically, and this type user possibly be the elderly and working clan blended family; E class user whole day power consumption is very high always, and this type user possibly be used as commercial use to residential building.
The data display module, this module links to each other with HBase, is used for representing the form of cloud Data Warehouse through form to the user intuitively.The data that the data display module represents come from HBase, and its data exhibiting form is rich and varied, except traditional tabulation, crosstab, also support curve map, histogram, pie chart figuresization to represent form.
Data display module flow process is following:
From HBase, extract the data that will show, the data display pattern is set, each parameter of curve map is set, transverse axis sign, longitudinal axis sign, curve form, color etc. are showed data at last.
The deal with data flow process of data warehouse according to the invention is following:
1, the ETL module is accepted the instruction of main control module, from relevant database, is written into data to the HBase database;
2, main control module is selected data are carried out data mining analysis or multidimensional analysis, selects data mining analysis to get into step 3, selects multidimensional analysis to get into step 5;
3, the DM module is received the instruction of main control module, to carrying out data integration to a table of data mining among the HBase, seeks the mining algorithm that self has according to instruction, and the utilization mining algorithm carries out computing to data;
4, the DM module data that will pass through computing are returned HBase and are stored, and whether need make form or synoptic diagram by main control module judgment data, if do not need, flow process finishes, if need then get into step 8;
5, the OLAP module is received the instruction of main control module; Indication Hive extracts the data that need multidimensional analysis among the HBase; And data are mapped as the two-dimensional table format among the Hive based on the row formats stored in HBase; The OLAP module is that Hive formulates mapping ruler simultaneously, and the instruction that Hive receives mapping (enum) data can be connected the HBase database afterwards with mapping ruler, and extracted data is also accomplished mapping process;
6, the OLAP module is carried out the multidimensional analysis type of data as required; Indication Hive sets up a form that stores analysis result in HBase; And indication Hive also is mapped as a blank bivariate table with this form in Hive; Data warehouse instrument Hive carries out multidimensional analysis to data in the bivariate table, and the data after analyzing are write blank bivariate table;
7, the bivariate table of data converted into again among the HBase and is stored among the HBase based on the row formats stored after OLAP module indication Hive will write and analyze; Whether main control module judgment data need make form or synoptic diagram; If do not need, flow process finishes, if need then get into step 8;
8, the data display module receives the instruction of main control modules exhibit data, from HBase, extracts the data that will show, and the data display pattern is set, and each parameter of curve map is set, and transverse axis sign, longitudinal axis sign, curve form, color etc. are showed data at last.
In sum, the invention provides a kind of data warehouse based on the cloud computing technological architecture.Characteristics of the present invention are, have good extendability and, can carry out polytype data, services such as multidimensional analysis and data mining.Satisfied the demand in the present business.
The above only is the preferred embodiments of the invention; Should be pointed out that for those skilled in the art, under the prerequisite that does not break away from the principle of the invention; Can also make some improvement and retouching, these improvement and retouching also should be regarded as protection scope of the present invention.
Claims (8)
1. cloud data warehouse; It is characterized in that; Said system cloud data warehouse is structured on Hadoop platform and the operating system software; Built-in HBase database of Hadoop platform and data warehouse instrument Hive, the structure of cloud data warehouse comprises that main control module, data insmod and data-mining module:
The main control module, the module that the timer-triggered scheduler service function embodies in the operating system, the main control module insmods and data-mining module transmission instruction, the flow direction of control data stream to data;
Data insmod, and an end connects external data base, and an end is got in touch the HBase database, and are connected with the main control module, be responsible for data in the external data base are written into the HBase database, or data in the HBase database are written into external data base;
Data-mining module is connected to the HBase database, and is connected with the main control module, is responsible for integrating data among the HBase, and data mining algorithm is provided, the rule and the pattern of calculating and mining data.
2. according to claim 1 system, it is characterized in that said system further comprises:
The multidimensional analysis module is connected to the data warehouse instrument Hive in the HBase database, and is connected with the main control module, and the data warehouse instrument Hive that is used for calling HBase carries out the multidimensional analysis service.
3. according to the said system of claim 1, it is characterized in that said system further comprises:
The data display module is connected to the HBase database, and is connected with the main control module, is used for the data of cloud data warehouse are passed through the embodied of form or synoptic diagram.
4. according to the said system of claim 1, it is characterized in that said data insmod and specifically are used for:
Data insmod and accept the instruction that is written into data of main control module; Extracted data from external data base; According to the required formatting parameter of HBase database; And be the required form of HBase database according to the parameter that is provided with data-switching, the data that will meet form are written into the HBase database.
5. according to the said system of claim 1, it is characterized in that said data-mining module specifically is used for:
Data-mining module is accepted the instruction that the main control module is carried out data mining analysis; Begin to integrate the data that to carry out data mining analysis in the HBase database; Write in the table; Seek the data mining algorithm in the data-mining module again,, obtain the rule and the pattern of data carrying out computing and analyze operation result through the data of integrating.
6. according to the said system of claim 5, it is characterized in that said searching data mining algorithm specifically comprises:
Association analysis algorithm, forecast analysis algorithm, cluster algorithm, classification analysis algorithm, outlier analytical algorithm or the algorithm combination in any of writing voluntarily.
7. according to the said system of claim 2, it is characterized in that said multidimensional analysis module specifically is used for:
After the multidimensional analysis module is accepted the main control module and is carried out the instruction of multidimensional analysis; Instrument Hive in order data warehouse is bivariate table data among the Hive with the data map that the needs of storing based on row in the HBase database carry out multidimensional analysis again; Mapping ruler is set simultaneously sends to data warehouse instrument Hive; Hive receive orders with mapping ruler after, connect the HBase database, extracted data is also accomplished the mapping of data.
8. according to the said decorum of claim 3, it is characterized in that said data display module specifically is used for:
The data display module is accepted the instruction of main control modules exhibit data, and extraction needs the data of displaying from the HBase database, and the pattern of data display is set, again with the embodied of data through form or synoptic diagram.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011103580170A CN102521246A (en) | 2011-11-11 | 2011-11-11 | Cloud data warehouse system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011103580170A CN102521246A (en) | 2011-11-11 | 2011-11-11 | Cloud data warehouse system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102521246A true CN102521246A (en) | 2012-06-27 |
Family
ID=46292168
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2011103580170A Pending CN102521246A (en) | 2011-11-11 | 2011-11-11 | Cloud data warehouse system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102521246A (en) |
Cited By (30)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102880709A (en) * | 2012-09-28 | 2013-01-16 | 用友软件股份有限公司 | Data warehouse management system and data warehouse management method |
| CN103136335A (en) * | 2013-01-31 | 2013-06-05 | 北京千分点信息科技有限公司 | Data control method based on data platforms |
| CN103198138A (en) * | 2013-04-16 | 2013-07-10 | 北京科技大学 | Large-scale hot continuous rolling data scheme customizing system based on cloud computing |
| CN103399925A (en) * | 2013-08-05 | 2013-11-20 | 河海大学 | Rainfall multidimensional analysis system based on hydrologic data and implementation method of rainfall multidimensional analysis system |
| CN103425762A (en) * | 2013-08-05 | 2013-12-04 | 南京邮电大学 | Telecom operator mass data processing method based on Hadoop platform |
| CN103617211A (en) * | 2013-11-20 | 2014-03-05 | 浪潮电子信息产业股份有限公司 | HBase loaded data importing method |
| CN103729453A (en) * | 2014-01-02 | 2014-04-16 | 浪潮电子信息产业股份有限公司 | HBase table conjunctive query optimization method |
| CN103886487A (en) * | 2014-03-28 | 2014-06-25 | 焦点科技股份有限公司 | Individualized recommendation method and system based on distributed B2B platform |
| CN103955502A (en) * | 2014-04-24 | 2014-07-30 | 科技谷（厦门）信息技术有限公司 | Visualized on-line analytical processing (OLAP) application realizing method and system |
| CN104123367A (en) * | 2014-07-24 | 2014-10-29 | 中国农业银行股份有限公司 | Data migration method and system from non-product-factory pattern to product factory pattern |
| CN104298760A (en) * | 2014-10-23 | 2015-01-21 | 北京京东尚科信息技术有限公司 | Data processing method and data processing device applied to data warehouse |
| CN104461551A (en) * | 2014-12-16 | 2015-03-25 | 芜湖乐锐思信息咨询有限公司 | Parallel data processing based big data processing system |
| CN104573071A (en) * | 2015-01-26 | 2015-04-29 | 湖南大学 | Intelligent school situation analysis system and method based on megadata technology |
| CN104573074A (en) * | 2015-01-27 | 2015-04-29 | 广东帝弘数据技术有限公司 | High-speed calculating and analyzing method based on hospital data |
| CN104699802A (en) * | 2015-03-20 | 2015-06-10 | 浪潮集团有限公司 | Visualized analysis method based on industry data |
| CN104715013A (en) * | 2015-01-26 | 2015-06-17 | 南京邮电大学 | Hadoop-based user health data analysis method and system |
| CN104915793A (en) * | 2015-06-30 | 2015-09-16 | 北京西塔网络科技股份有限公司 | Public information intelligent analysis platform based on big data analysis and mining |
| CN105205085A (en) * | 2014-06-30 | 2015-12-30 | 中兴通讯股份有限公司 | Multi-dimensional analysis method and device for mass data |
| CN105302831A (en) * | 2014-07-18 | 2016-02-03 | 上海星红桉数据科技有限公司 | High-speed calculation analysis method based on mass user behavior data |
| CN105426478A (en) * | 2015-11-18 | 2016-03-23 | 四川长虹电器股份有限公司 | Method for user behavior analysis |
| CN105760505A (en) * | 2016-02-23 | 2016-07-13 | 浪潮软件集团有限公司 | Hive-based historical data analysis and archiving method |
| CN105824892A (en) * | 2016-03-11 | 2016-08-03 | 广东电网有限责任公司电力科学研究院 | Method for synchronizing and processing data by data pool |
| CN106021285A (en) * | 2016-04-29 | 2016-10-12 | 武汉佰钧成技术有限责任公司 | Method for incremental extraction and analysis of mass data based on Hadoop platform |
| CN106227862A (en) * | 2016-07-29 | 2016-12-14 | 浪潮软件集团有限公司 | E-commerce data integration method based on distribution |
| CN106228000A (en) * | 2016-07-18 | 2016-12-14 | 北京千安哲信息技术有限公司 | Over-treatment detecting system and method |
| CN108280230A (en) * | 2018-02-27 | 2018-07-13 | 北京中关村科金技术有限公司 | A kind of method, apparatus, equipment and the storage medium of analysis data |
| CN109002440A (en) * | 2017-06-06 | 2018-12-14 | 北京京东尚科信息技术有限公司 | Method, apparatus and system for big data multidimensional analysis |
| CN109195175A (en) * | 2018-09-03 | 2019-01-11 | 郑州云海信息技术有限公司 | A kind of mobile wireless network optimization method based on cloud computing |
| CN111160865A (en) * | 2019-12-31 | 2020-05-15 | 新奥数能科技有限公司 | Workflow management method and device |
| CN111290350A (en) * | 2019-12-23 | 2020-06-16 | 南京中赢生态农业科技发展有限公司 | Agricultural aquaculture intelligent monitoring system based on Internet of things |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101996250A (en) * | 2010-11-15 | 2011-03-30 | 中国科学院计算技术研究所 | Hadoop-based mass stream data storage and query method and system |
| CN102236867A (en) * | 2011-08-15 | 2011-11-09 | 悠易互通（北京）广告有限公司 | Cloud computing-based audience behavioral analysis advertisement targeting system |
-
2011
- 2011-11-11 CN CN2011103580170A patent/CN102521246A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101996250A (en) * | 2010-11-15 | 2011-03-30 | 中国科学院计算技术研究所 | Hadoop-based mass stream data storage and query method and system |
| CN102236867A (en) * | 2011-08-15 | 2011-11-09 | 悠易互通（北京）广告有限公司 | Cloud computing-based audience behavioral analysis advertisement targeting system |
Cited By (36)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102880709A (en) * | 2012-09-28 | 2013-01-16 | 用友软件股份有限公司 | Data warehouse management system and data warehouse management method |
| CN103136335A (en) * | 2013-01-31 | 2013-06-05 | 北京千分点信息科技有限公司 | Data control method based on data platforms |
| CN103136335B (en) * | 2013-01-31 | 2016-06-29 | 北京百分点信息科技有限公司 | A kind of data control method based on data platform |
| CN103198138A (en) * | 2013-04-16 | 2013-07-10 | 北京科技大学 | Large-scale hot continuous rolling data scheme customizing system based on cloud computing |
| CN103399925A (en) * | 2013-08-05 | 2013-11-20 | 河海大学 | Rainfall multidimensional analysis system based on hydrologic data and implementation method of rainfall multidimensional analysis system |
| CN103425762A (en) * | 2013-08-05 | 2013-12-04 | 南京邮电大学 | Telecom operator mass data processing method based on Hadoop platform |
| CN103617211A (en) * | 2013-11-20 | 2014-03-05 | 浪潮电子信息产业股份有限公司 | HBase loaded data importing method |
| CN103729453A (en) * | 2014-01-02 | 2014-04-16 | 浪潮电子信息产业股份有限公司 | HBase table conjunctive query optimization method |
| CN103886487A (en) * | 2014-03-28 | 2014-06-25 | 焦点科技股份有限公司 | Individualized recommendation method and system based on distributed B2B platform |
| CN103886487B (en) * | 2014-03-28 | 2016-01-27 | 焦点科技股份有限公司 | Based on personalized recommendation method and the system of distributed B2B platform |
| CN103955502A (en) * | 2014-04-24 | 2014-07-30 | 科技谷（厦门）信息技术有限公司 | Visualized on-line analytical processing (OLAP) application realizing method and system |
| CN103955502B (en) * | 2014-04-24 | 2017-07-28 | 科技谷（厦门）信息技术有限公司 | A kind of visualization OLAP application realization method and system |
| CN105205085A (en) * | 2014-06-30 | 2015-12-30 | 中兴通讯股份有限公司 | Multi-dimensional analysis method and device for mass data |
| CN105302831A (en) * | 2014-07-18 | 2016-02-03 | 上海星红桉数据科技有限公司 | High-speed calculation analysis method based on mass user behavior data |
| CN104123367A (en) * | 2014-07-24 | 2014-10-29 | 中国农业银行股份有限公司 | Data migration method and system from non-product-factory pattern to product factory pattern |
| CN104123367B (en) * | 2014-07-24 | 2017-06-30 | 中国农业银行股份有限公司 | Data migration method and system of the non-product factory mode to product factory mode |
| CN104298760A (en) * | 2014-10-23 | 2015-01-21 | 北京京东尚科信息技术有限公司 | Data processing method and data processing device applied to data warehouse |
| CN104298760B (en) * | 2014-10-23 | 2019-02-05 | 北京京东尚科信息技术有限公司 | A kind of data processing method and data processing equipment applied to data warehouse |
| CN104461551A (en) * | 2014-12-16 | 2015-03-25 | 芜湖乐锐思信息咨询有限公司 | Parallel data processing based big data processing system |
| CN104573071A (en) * | 2015-01-26 | 2015-04-29 | 湖南大学 | Intelligent school situation analysis system and method based on megadata technology |
| CN104715013A (en) * | 2015-01-26 | 2015-06-17 | 南京邮电大学 | Hadoop-based user health data analysis method and system |
| CN104573074A (en) * | 2015-01-27 | 2015-04-29 | 广东帝弘数据技术有限公司 | High-speed calculating and analyzing method based on hospital data |
| CN104699802A (en) * | 2015-03-20 | 2015-06-10 | 浪潮集团有限公司 | Visualized analysis method based on industry data |
| CN104915793A (en) * | 2015-06-30 | 2015-09-16 | 北京西塔网络科技股份有限公司 | Public information intelligent analysis platform based on big data analysis and mining |
| CN105426478A (en) * | 2015-11-18 | 2016-03-23 | 四川长虹电器股份有限公司 | Method for user behavior analysis |
| CN105760505A (en) * | 2016-02-23 | 2016-07-13 | 浪潮软件集团有限公司 | Hive-based historical data analysis and archiving method |
| CN105824892A (en) * | 2016-03-11 | 2016-08-03 | 广东电网有限责任公司电力科学研究院 | Method for synchronizing and processing data by data pool |
| CN106021285A (en) * | 2016-04-29 | 2016-10-12 | 武汉佰钧成技术有限责任公司 | Method for incremental extraction and analysis of mass data based on Hadoop platform |
| CN106228000A (en) * | 2016-07-18 | 2016-12-14 | 北京千安哲信息技术有限公司 | Over-treatment detecting system and method |
| CN106227862A (en) * | 2016-07-29 | 2016-12-14 | 浪潮软件集团有限公司 | E-commerce data integration method based on distribution |
| CN109002440A (en) * | 2017-06-06 | 2018-12-14 | 北京京东尚科信息技术有限公司 | Method, apparatus and system for big data multidimensional analysis |
| CN108280230A (en) * | 2018-02-27 | 2018-07-13 | 北京中关村科金技术有限公司 | A kind of method, apparatus, equipment and the storage medium of analysis data |
| CN109195175A (en) * | 2018-09-03 | 2019-01-11 | 郑州云海信息技术有限公司 | A kind of mobile wireless network optimization method based on cloud computing |
| CN109195175B (en) * | 2018-09-03 | 2021-12-21 | 郑州云海信息技术有限公司 | Mobile wireless network optimization method based on cloud computing |
| CN111290350A (en) * | 2019-12-23 | 2020-06-16 | 南京中赢生态农业科技发展有限公司 | Agricultural aquaculture intelligent monitoring system based on Internet of things |
| CN111160865A (en) * | 2019-12-31 | 2020-05-15 | 新奥数能科技有限公司 | Workflow management method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102521246A (en) | Cloud data warehouse system | |
| CN104820670B (en) | A kind of acquisition of power information big data and storage method | |
| CN102467570B (en) | Connection query system and method for distributed data warehouse | |
| CN103064875B (en) | A kind of spatial service data distributed enquiring method | |
| CN102722531B (en) | Query method based on regional bitmap indexes in cloud environment | |
| CN102012912B (en) | Management method for unstructured data based on cloud computing environment | |
| CN106339509A (en) | Power grid operation data sharing system based on large data technology | |
| CN104573071A (en) | Intelligent school situation analysis system and method based on megadata technology | |
| Liang et al. | Express supervision system based on NodeJS and MongoDB | |
| CN103430144A (en) | Data source analytics | |
| CN107895046B (en) | Heterogeneous data integration platform | |
| Yi et al. | Evolutional model of tourism efficiency based on the DEA method: A case study of cities in Guangdong Province, China | |
| CN104123346A (en) | Structural data searching method | |
| CN102521307A (en) | Parallel query processing method for share-nothing database cluster in cloud computing environment | |
| CN103019728A (en) | Effective complex report parsing engine and parsing method thereof | |
| CN105930446A (en) | Telecommunication customer tag generation method based on Hadoop distributed technology | |
| CN112632025A (en) | Power grid enterprise management decision support application system based on PAAS platform | |
| CN107590181A (en) | A kind of intelligent analysis system of big data | |
| CN101452485A (en) | Method and device for generating multidimensional cubic based on relational database | |
| CN104699857A (en) | Big data storage method based on knowledge engineering | |
| CN103646051A (en) | Big-data parallel processing system and method based on column storage | |
| CN105183809A (en) | Cloud platform data query method | |
| CN102779138A (en) | Hard disk access method of real time data | |
| CN102654878B (en) | Data operation method for distributed database in embedded system and board card | |
| CN105159971A (en) | Cloud platform data retrieval method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| ASS | Succession or assignment of patent right |
Owner name: STATE ELECTRIC NET CROP. Free format text: FORMER OWNER: STATE GRID INFORMATION + TELECOMMUNICATION CO., LTD. Effective date: 20120727 Owner name: STATE GRID INFORMATION + TELECOMMUNICATION CO., LT Free format text: FORMER OWNER: BEIJING GUODIANTONG NETWORK TECHNOLOGY CO., LTD. Effective date: 20120727 |
|
| C10 | Entry into substantive examination | ||
| C41 | Transfer of patent application or patent right or utility model | ||
| COR | Change of bibliographic data |
Free format text: CORRECT: ADDRESS; FROM: 100761 XUANWU, BEIJING TO: 100031 XICHENG, BEIJING |
|
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20120727 Address after: 100031 Xicheng District West Chang'an Avenue, No. 86, Beijing Applicant after: State Grid Corporation of China Co-applicant after: State Grid Information & Telecommunication Co.,Ltd. Co-applicant after: BEIJING GUODIANTONG NETWORK TECHNOLOGY Co.,Ltd. Address before: 100761 Beijing Xuanwu District city two, Baiguang Road Applicant before: State Grid Information & Telecommunication Co.,Ltd. Co-applicant before: BEIJING GUODIANTONG NETWORK TECHNOLOGY Co.,Ltd. |
|
| C12 | Rejection of a patent application after its publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20120627 |