CN105956932A - Distribution and utilization data fusion method and system - Google Patents
Distribution and utilization data fusion method and system Download PDFInfo
- Publication number
- CN105956932A CN105956932A CN201610287063.9A CN201610287063A CN105956932A CN 105956932 A CN105956932 A CN 105956932A CN 201610287063 A CN201610287063 A CN 201610287063A CN 105956932 A CN105956932 A CN 105956932A
- Authority
- CN
- China
- Prior art keywords
- data
- adapted electricity
- electricity data
- distributed system
- keyword
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
- G06Q50/06—Electricity, gas or water supply
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/258—Data format conversion from or to a database
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/80—Information retrieval; Database structures therefor; File system structures therefor of semi-structured data, e.g. markup language structured data such as SGML, XML or HTML
Abstract
The invention relates to a distribution and utilization data fusion method and system. The method comprises the steps of extracting distribution and utilization data from an intelligent distribution and utilization grid, and storing the distribution and utilization data in a distributed system; converting the format of the distribution and utilization data into a set format of the distributed system; obtaining category fields of converted distribution and utilization data, and combining distribution and utilization data of a same category field; and obtaining key words of different types of distribution and utilization data, and converting the types of the key words into a set type. The method can realize distribution and utilization data fusion, allow fused data to possess a uniform format, and key words of a same type, and is in favor of improving fused data reliability and robustness.
Description
Technical field
The present invention relates to technical field of power systems, particularly relate to a kind of adapted electricity data fusion method and be
System.
Background technology
In the information age, flow of information carried the energy of " data " become the innovation of actuation techniques ability,
The important force that management mode is changed, social value promotes.Focus on 5 key features (a large amount of Volume,
High speed Velocity, various Variety, true Veracity, be worth Value), " big data " concept once
Propose just to obtain huge development, in the Internet, ecommerce, medicine, the field such as advertisement obtain
Extensively application, obtains highly significant effect and value.At intelligence adapted electrical domain, surpass hundred million intelligence at present
If ammeter all beams back data with every 15 minutes one time equifrequent value, the data total amount that every day produces just can reach
Tens PB (petabyte), exceed the data storage of many electronic business enterprise.Intelligent grid and electric power
Market is all in the urgent need to excavating by magnanimity adapted electricity data value, it is achieved user's degree of depth participate in electricity consumption interactive with
Price responds.
Data integration is the primary stage of data fusion, and this stage needs the underlying issue solved to be adapted TV university
It is data pick-up problem that key technical problem is merged in data integration.Adapted electrical network exists substantial amounts of electricity data and
Non-electrical data, structure is the most irregular, and is dynamically change.For in adapted TV university data
System isomery, syntactic metacharacter, Semantic Heterogeneous problem, use based on XML (extensible markup language) and basis
The heterogeneous data integrating method of body.For system and this two classes problem of syntactic metacharacter, utilize the advantage of middleware
Bottom data is packed, to form external unified interface, thus reaches the conforming mesh of user operation
's.And for the problem of Semantic Heterogeneous, need to use the corresponding semantic model of technique construction of body, thus
Form the different expression-forms to the identical concept and carry out the basic skills of normalizing.The isomeric data of magnanimity is integrated
After be still inevitably present the phenomenons such as the information collision of instance-level, redundancy.Fusion is the height of data integration
In the level stage, fusing stage is different from the key character of data integration and is, by Data Integration, cleaning, to produce
New knowledge.The input of fused layer comes from the result that data set stratification returns, and uses domain knowledge and fusion
The object information of data integration is analyzed, clears up, integrates and obtain fusion results by rule.
In intelligence adapted electrical network, traditional data fusion scheme is typically by the adapted of the most various form inequalities
Electricity data store to same memory space, to realize the fusion of above-mentioned multiple adapted electricity data, but, on
State the adapted electricity data format differences after fusion relatively big, easily affect the employing that it preserved or is correlated with
Reliability.
Summary of the invention
Based on this, it is necessary to easily affect the technology of the adapted electricity data reliability after fusion for traditional scheme
Problem, it is provided that a kind of adapted electricity data fusion method and system.
A kind of adapted electricity data fusion method, comprises the steps:
From intelligence adapted electrical network extraction adapted electricity data, and described adapted electricity data are stored to distributed system；
The form of described adapted electricity data is converted to the setting form of distributed system；
Obtain the classification field of the adapted electricity data after form conversion, by adapted electricity data identical for classification field
Merge；
Obtain the keyword of different classes of adapted electricity data, be converted to the type of described keyword set type.
A kind of adapted electricity data fusion system, including:
Described adapted electricity data for extracting adapted electricity data from intelligence adapted electrical network, and are deposited by abstraction module
Storage is to distributed system；
Modular converter, for being converted to the setting form of distributed system by the form of described adapted electricity data；
Merger module, the classification field of the adapted electricity data after obtaining form conversion, by classification field phase
Same adapted electricity data merge；
Integrate module, for obtaining the keyword of different classes of adapted electricity data, by the type of described keyword
Be converted to set type.
Above-mentioned adapted electricity data fusion method and system, from the adapted electricity number that intelligence adapted electrical network extraction is to be fused
According to rear, described adapted electricity data stored to distributed system, and carries out corresponding form conversion, by form
After conversion, the adapted electricity data that classification field is identical merge, and different classes of adapted electricity Data Integration is
The adapted electricity data that key word type is identical, thus realize the fusion of above-mentioned adapted electricity data, after making fusion
Data have unified form, and the keyword with type, the reliability of data after being conducive to raising to merge
And robustness.
Accompanying drawing explanation
Fig. 1 is the adapted electricity data fusion method flow chart of an embodiment；
Fig. 2 is the Sqoop structural representation of an embodiment；
Fig. 3 is the Flume NG structural representation of an embodiment；
Fig. 4 is the adapted electricity data fusion method flow chart of an embodiment；
Fig. 5 is the adapted electricity data fusion system structural representation of an embodiment.
Detailed description of the invention
Below in conjunction with the accompanying drawings the adapted electricity data fusion method of the present invention and the detailed description of the invention of system are made in detail
Thin description.
The adapted electricity data fusion method flow chart of an embodiment it is shown, including as follows with reference to Fig. 1, Fig. 1
Step:
Described adapted electricity data from intelligence adapted electrical network extraction adapted electricity data, and are stored to distributed by S10
System；
Above-mentioned adapted electricity data are the power dispatching data in intelligence adapted electrical network and electricity consumption data etc., and it can be from intelligence
Can adapted electrical network Relational database in obtain, generally can include structural data, semi-structured data and
Unstructured data.Said structure data, semi-structured data and unstructured data have different coming
Source, structure and form, merge it, can be to the adapted electricity of above-mentioned separate sources, structure and form
Data are managed collectively, and are effectively increased the stability of adapted electricity data.Can be according to adapted electricity data
Type utilizes different extraction tools to extract the adapted electricity data of each type.
The adapted electricity data of extraction are stored to distributed system, it is possible to use the conversion of distributed system, deposit
Adapted electricity data are processed by the instruments such as storage accordingly, advantageously ensure that above-mentioned adapted electricity data are converted
Stability in the process such as journey.
In one embodiment, above-mentioned adapted electricity data can include structural data, semi-structured data and
Unstructured data.
Said structure data can include the adapted telecommunications being stored in intelligence each data base of adapted electrical network
Breath, facility information, asset data etc..Above-mentioned semi-structured data can include knowing of intelligence adapted electrical network
Know the various material documents in storehouse, the form of above-mentioned material document can include word form, pdf form,
And text document form corresponding to service hotline recording file after structuring processes etc..Above-mentioned non-
Structural data can include all kinds of monitor video and not yet carry out the service hotline recording that structuring processes
Deng.
The extraction of above-mentioned semi-structured data can use the methods such as LogStash, above-mentioned LogStash to be a
Logging tools.Said structure data and semi-structured data can be (distributed, towards row with Hbase
PostgreSQL database) method stores accordingly, and unstructured data utilizes HDFS method directly to deposit
Storage.
In one embodiment, the process of above-mentioned extraction adapted electricity data may include that
Use Sqoop method drawing-out structure data；
Use Flume NG method extraction semi-structured data；
Use Kettle method extraction unstructured data.
Above-mentioned Sqoop is the tool for transmitting between data base, can be by adapted electricity by above-mentioned Sqoop method
After structural data in net is directed into corresponding data base, carry out associated extraction.
The massive logs data in different pieces of information source can efficiently be collected, be polymerized, move by above-mentioned Flume NG
Dynamic, finally store in the centralization data-storage system of distributed system.
Above-mentioned Kettle method can extract and manage the unstructured data from disparate databases.
As an embodiment, the step of above-mentioned employing Sqoop method drawing-out structure data may include that
Read the list structure of structural data, generate Sqoop according to described list structure and run class, by described
Sqoop runs class packing, obtains jar bag (executable file bag), described jar bag is submitted to Hadoop
(data warehouse can be built)；
Perform the mapper type of mapreduce task and perform the parallel task number of mapreduce；Above-mentioned
Mapreduce is a kind of programming model, for the concurrent operation of large-scale dataset (more than 1TB)；
Performed mapreduce task by Hadoop, structural data carried out cutting, record cutting scope,
Create RecordReader and from data base, read data, creating Map task and in the way of reading line by line
Drawing-out structure data from the relevant database of structural data.
During above-mentioned employing Sqoop method drawing-out structure data, it is also possible to preservation said structure is set
Change pattern of the input and the output format of the data base of data, corresponding to ensure during structural data reads
Database data input or the fairness of output.The structure of Sqoop can be as in figure 2 it is shown, it be a opening
The instrument in source, is mainly used in entering between Hadoop (Hive) and traditional data base (mysql, postgresql etc.)
The transmission of row data, can be by a relevant database (such as: MySQL, Oracle, Postgres etc.)
In data lead in the HDFS entering Hadoop, it is also possible to the data of HDFS are led entering relational data
In storehouse.Sqoop is mainly by carrying out data interaction between JDBC and relevant database, as long as therefore supporting
The data base of JDBC, can carry out the interactive operations such as data extraction by Sqoop to it.For other
Data base, it is also possible to by the way of increasing JDBC and driving or increase intermediary interface so that it is support and Sqoop
Carry out data interaction.The occupation mode of Sqoop is more succinct, its internal integration Hive, Hbase and Oozie,
Support by the way of map-reduce, transmit data, thus concurrent characteristic and failure tolerance be provided, it is possible to
Realize the extraction process of structural data well.
In one embodiment, the step of above-mentioned employing Flume NG method extraction semi-structured data includes:
By the Source assembly of Flume NG, extraction event is sent to Channel assembly, and passes to
Sink assembly；
Sink assembly gathers semi-structured data, and sends described semi-structured data to HDFS (Hadoop
Distributed file system) on cluster.
In the present embodiment, realize can adopting in the extraction process of semi-structured data by Flume NG method
Take the method that multiple Agent (software of energy autonomic activities or hardware entities) writes HDFS, be respectively each
Individual Agent arranges the Data Source end of input.The present embodiment uses Flume NG method to extract semi-structured number
According to, the extraction efficiency of semi-structured data can be improved.
The structure of above-mentioned Flume NG can as it is shown on figure 3, its be Cloudera provide one distributed,
Reliably, available system, the magnanimity semi-structured data in different pieces of information source can efficiently be collected by it,
Polymerization, mobile, finally store in a centralization data-storage system, be particularly well-suited to various daily record,
The extraction work of the semi-structured data of text class.
The framework of Flume NG mainly has a following key concept:
Event: one data cell, with an optional message header；
Flow:Event arrives the abstract of the migration of point of destination from source point；
Client: operation is positioned at the Event at source point, sends it to Flume Agent；
Agent: one independent Flume process, comprises assembly Source, Channel, Sink；
Source: be used for consuming the Event being delivered to this assembly；
One interim storage of Channel: transfer Event, preserves what Source component passes came
Event；
Sink: read from Channel and remove Event, is delivered to Event in Flow Pipeline
Next Agent；
External system produces semi-structured data, directly by the Source assembly of the Agent of Flume by thing
Part (such as log lines) is sent to middle interim Channel assembly, is ultimately transferred to Sink assembly, Sink group
Part can directly store the data on HDFS cluster.
In one embodiment, the step of above-mentioned employing Kettle method extraction unstructured data may include that
Create HTTP file；
Utilize described HTTP file that unstructured data is write HDFS.
Above-mentioned Kettle is a ETL (data warehouse technology) instrument increased income, can Window, Linux,
Run in the operating systems such as Unix, data pick-up efficient stable.Its purpose of design is that various data are put into one
Rise, then export with the form desired by a kind of user, go for the extraction of various structural data,
Also the extraction work for all kinds of unstructured datas is supported.Kettle allows management from the number in different pieces of information source
According to, by providing a patterned user environment to describe the real needs of user.Kettle supports two kinds
Script file, conversion (transformation) and work (job), transformation completes for data
Basis conversion, job then completes the control of whole workflow.Kettle can perform following operating system life
Order and operation: Ping main frame, write daily record, send mail, obtain mail from POP Server and be saved in this
Ground, comparison document folder, file, create, replicate, move, delete, compressed file, obtain from HTTP
Or upper transmitting file, operating delay wait.During the extraction work of above-mentioned unstructured data, it is possible to use Kettle
Obtain file and the mobile function created, thus realize unstructured data is write HDFS.
In one embodiment, the above-mentioned step described adapted electricity data stored to distributed system can be wrapped
Include:
Hbase method is utilized to store described structural data and semi-structured data to distributed system；
HDFS method is utilized to store described unstructured data to distributed system.
The present embodiment can ensure that all types of adapted electricity data store the order to distributed system.
S20, is converted to the setting form of distributed system by the form of described adapted electricity data；
Above-mentioned steps can use based on data transfer methods such as selecting, separate, merge, convert or collect
Adapted electricity data are carried out corresponding form conversion.Above-mentioned setting form can be that distributed system easily identifies
Data form, such as, the form such as text formatting, SequenceFile or AvroDataFile.Above-mentioned lattice
Adapted electricity data after formula conversion can include adapted electrical information and characterize the classification field of data category, profit
Use above-mentioned classification field, it is possible to achieve the classification of adapted electricity data.
In one embodiment, the form of described adapted electricity data is converted to the setting form of distributed system
Step may include that
Described adapted electricity data are implanted the SQL statement that distributed system is embedded；
Be converted to the form of described SQL statement set form；
Adapted electricity data are extracted from the SQL statement after conversion.
Adapted electricity data are implanted the SQL statement that distributed system is embedded by the present embodiment, to realize corresponding lattice
Formula is changed, it is ensured that adapted electricity data stability in form transformation process.
S30, obtains the classification field of the adapted electricity data after form conversion, by adapted electricity identical for classification field
Data merge；
Above-mentioned classification field is the field characterizing adapted electricity data category, and it is the adapted electricity number after form conversion
According to ingredient, adapted electricity data identical for classification field are merged, will the identical adapted of classification
Electricity aggregation of data, to same memory element, makes above-mentioned generic adapted electricity data can pass through same entrance
Inquire about accordingly or utilize.
S40, obtains the keyword of different classes of adapted electricity data, is converted to the type of described keyword set
Type.
Be converted to the key word type of different classes of adapted electricity data set type, make above-mentioned adapted electricity data
It is fused to similar key data.Above-mentioned similar key data is the data that key word type is identical；Will not
Generic adapted electricity Data Integration is similar key data, it is possible to achieve the fusion of corresponding adapted electricity data,
The adapted electricity data after fusion are made to have same or like data form and the consistent keyword of type,
It is made to possess higher stability and robustness, it is easy to be queried or obtain accordingly.
The adapted electricity data fusion method that the present invention provides, from the adapted electricity that intelligence adapted electrical network extraction is to be fused
After data, described adapted electricity data are stored to distributed system, and carries out corresponding form conversion, by lattice
After formula conversion, the adapted electricity data that classification field is identical merge, different classes of adapted electricity Data Integration
For the adapted electricity data that key word type is identical, thus realize the fusion of above-mentioned adapted electricity data, after making fusion
Data there is unified form, and the keyword with type, be conducive to improving merge after data reliable
Property and robustness.
In one embodiment, the keyword of above-mentioned acquisition different classes of adapted electricity data, by described keyword
Type be converted to set type step after can also include:
Keyword corresponding to different classes of adapted electricity data and described adapted electricity data is preserved to data
Storehouse；
The fusion of adapted electricity data is carried out in the database according to keyword.
The present embodiment, the adapted electricity data after merging preserve the data base to HDFS distributed system,
The most above-mentioned data base realizes the fusion of different types of adapted electricity data further, has utilization by above-mentioned number
According to related keyword, the adapted electricity data after merging are inquired about according to storehouse, can improve adapted electricity data
Search efficiency.
In one embodiment, the flow chart of above-mentioned adapted electricity data fusion method can be joined as shown in Figure 4
Examine Fig. 4, first can utilize extraction tool Sqoop, Flume NG and Kettle that adapted electricity data are carried out
Extraction, stores the adapted electricity data acquisition of extraction with based on distributed storage and object storage mode, enters
And use and based on the data transfer method selecting, separate, merge, convert and collecting, the data of storage are carried out
Data are changed, and are converted to the setting form of distributed system with the form by adapted electricity data, then by after conversion
Data carry out subregion merger process, specifically can be divided into dsc data district and real time data district, then to hot number
Integrate accordingly according to the adapted electricity data in district and real time data district, it is achieved the fusion of above-mentioned adapted electricity data.
With reference to shown in Fig. 5, Fig. 5 is the adapted electricity data fusion system structural representation of an embodiment, bag
Include:
Abstraction module 10, for extracting adapted electricity data from intelligence adapted electrical network, and by described adapted electricity data
Store to distributed system；
Modular converter 20, for being converted to the setting form of distributed system by the form of described adapted electricity data；
Merger module 30, the classification field of the adapted electricity data after obtaining form conversion, by classification field
Identical adapted electricity data merge；
Integrate module 40, for obtaining the keyword of different classes of adapted electricity data, by the class of described keyword
Type is converted to set type.
The adapted electricity data fusion method one that the adapted electricity data fusion system that the present invention provides provides with the present invention
One is corresponding, and technical characteristic and beneficial effect thereof that the embodiment in described adapted electricity data fusion method illustrates are equal
Be applicable to the embodiment of adapted electricity data fusion system, hereby give notice that.
Each technical characteristic of embodiment described above can combine arbitrarily, for making description succinct, the most right
The all possible combination of each technical characteristic in above-described embodiment is all described, but, if these skills
There is not contradiction in the combination of art feature, is all considered to be the scope that this specification is recorded.
Embodiment described above only have expressed the several embodiments of the present invention, and it describes more concrete and detailed,
But can not therefore be construed as limiting the scope of the patent.It should be pointed out that, for this area
For those of ordinary skill, without departing from the inventive concept of the premise, it is also possible to make some deformation and change
Entering, these broadly fall into protection scope of the present invention.Therefore, the protection domain of patent of the present invention should be with appended power
Profit requires to be as the criterion.
Claims (10)
1. an adapted electricity data fusion method, it is characterised in that comprise the steps:
From intelligence adapted electrical network extraction adapted electricity data, and described adapted electricity data are stored to distributed system；
The form of described adapted electricity data is converted to the setting form of distributed system；
Obtain the classification field of the adapted electricity data after form conversion, by adapted electricity data identical for classification field
Merge；
Obtain the keyword of different classes of adapted electricity data, be converted to the type of described keyword set type.
Adapted electricity data fusion method the most according to claim 1, it is characterised in that described acquisition is not
The keyword of generic adapted electricity data, goes back after the type of described keyword is converted to the step of setting type
Including:
Keyword corresponding to different classes of adapted electricity data and described adapted electricity data is preserved to data
Storehouse；
The fusion of adapted electricity data is carried out in the database according to keyword.
Adapted electricity data fusion method the most according to claim 1, it is characterised in that described adapted electricity
Data include structural data, semi-structured data and unstructured data.
Adapted electricity data fusion method the most according to claim 3, it is characterised in that described extraction is joined
The process of electricity consumption data includes:
Use Sqoop method drawing-out structure data；
Use Flume NG method extraction semi-structured data；
Use Kettle method extraction unstructured data.
Adapted electricity data fusion method the most according to claim 4, it is characterised in that described employing
The step of Sqoop method drawing-out structure data includes:
Read the list structure of structural data, generate Sqoop according to described list structure and run class, by described
Sqoop runs class packing, obtains jar bag, described jar bag is submitted to Hadoop；
Perform the mapper type of mapreduce task and perform the parallel task number of mapreduce；
Performed mapreduce task by Hadoop, structural data carried out cutting, record cutting scope,
Create RecordReader and from data base, read data, creating Map task and in the way of reading line by line
Drawing-out structure data from the relevant database of structural data.
Adapted electricity data fusion method the most according to claim 4, it is characterised in that described employing
The step of Flume NG method extraction semi-structured data includes:
By the Source assembly of Flume NG, extraction event is sent to Channel assembly, and passes to
Sink assembly；
Sink assembly gathers semi-structured data, and is sent to HDFS cluster by described semi-structured data.
Adapted electricity data fusion method the most according to claim 4, it is characterised in that described employing
The step of Kettle method extraction unstructured data includes:
Create HTTP file；
Utilize described HTTP file that unstructured data is write HDFS.
Adapted electricity data fusion method the most according to claim 3, it is characterised in that described by described
The step that adapted electricity data store to distributed system includes:
Hbase method is utilized to store described structural data and semi-structured data to distributed system；
HDFS method is utilized to store described unstructured data to distributed system.
Adapted electricity data fusion method the most according to claim 1, it is characterised in that described by described
The form of adapted electricity data is converted to the step setting form of distributed system and includes:
Described adapted electricity data are implanted the SQL statement that distributed system is embedded；
Be converted to the form of described SQL statement set form；
Adapted electricity data are extracted from the SQL statement after conversion.
10. an adapted electricity data fusion system, it is characterised in that including:
Described adapted electricity data for extracting adapted electricity data from intelligence adapted electrical network, and are deposited by abstraction module
Storage is to distributed system；
Modular converter, for being converted to the setting form of distributed system by the form of described adapted electricity data；
Merger module, the classification field of the adapted electricity data after obtaining form conversion, by classification field phase
Same adapted electricity data merge；
Integrate module, for obtaining the keyword of different classes of adapted electricity data, by the type of described keyword
Be converted to set type.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610287063.9A CN105956932A (en) | 2016-04-29 | 2016-04-29 | Distribution and utilization data fusion method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610287063.9A CN105956932A (en) | 2016-04-29 | 2016-04-29 | Distribution and utilization data fusion method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105956932A true CN105956932A (en) | 2016-09-21 |
Family
ID=56913461
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610287063.9A Pending CN105956932A (en) | 2016-04-29 | 2016-04-29 | Distribution and utilization data fusion method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105956932A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106776903A (en) * | 2016-11-30 | 2017-05-31 | 国网重庆市电力公司电力科学研究院 | A kind of big data shared system and method that auxiliary tone is sought suitable for intelligent grid |
| CN108763583A (en) * | 2018-06-11 | 2018-11-06 | 山东汇贸电子口岸有限公司 | A kind of microblog hot topic extracting method and system based on keyword search |
| CN109995856A (en) * | 2019-03-21 | 2019-07-09 | 国电南瑞科技股份有限公司 | A kind of grid operation data wide area collects method and system |
| CN110990351A (en) * | 2019-12-05 | 2020-04-10 | 南方电网数字电网研究院有限公司 | Unstructured data acquisition method, device and system and computer equipment |
| CN111949612A (en) * | 2020-07-31 | 2020-11-17 | 广西美立方工程咨询有限公司 | Unstructured data storage middleware system based on hadoop and use method thereof |
| CN112347071A (en) * | 2020-12-06 | 2021-02-09 | 国网山东省电力公司电力科学研究院 | Power distribution network cloud platform data fusion method and power distribution network cloud platform |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101452450A (en) * | 2007-11-30 | 2009-06-10 | 上海市电力公司 | Multiple source data conversion service method and apparatus thereof |
| CN104992379A (en) * | 2015-07-10 | 2015-10-21 | 贵州广思信息网络有限公司 | Dynamic text exchange format management method of big data of power industry |
| CN105205085A (en) * | 2014-06-30 | 2015-12-30 | 中兴通讯股份有限公司 | Multi-dimensional analysis method and device for mass data |
| CN105279234A (en) * | 2015-09-24 | 2016-01-27 | 国网山西省电力公司大同供电公司 | Cross-platform data acquisition and conversion method of state overhaul data of electric transmission and transformation equipment |
-
2016
- 2016-04-29 CN CN201610287063.9A patent/CN105956932A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101452450A (en) * | 2007-11-30 | 2009-06-10 | 上海市电力公司 | Multiple source data conversion service method and apparatus thereof |
| CN105205085A (en) * | 2014-06-30 | 2015-12-30 | 中兴通讯股份有限公司 | Multi-dimensional analysis method and device for mass data |
| CN104992379A (en) * | 2015-07-10 | 2015-10-21 | 贵州广思信息网络有限公司 | Dynamic text exchange format management method of big data of power industry |
| CN105279234A (en) * | 2015-09-24 | 2016-01-27 | 国网山西省电力公司大同供电公司 | Cross-platform data acquisition and conversion method of state overhaul data of electric transmission and transformation equipment |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106776903A (en) * | 2016-11-30 | 2017-05-31 | 国网重庆市电力公司电力科学研究院 | A kind of big data shared system and method that auxiliary tone is sought suitable for intelligent grid |
| CN108763583A (en) * | 2018-06-11 | 2018-11-06 | 山东汇贸电子口岸有限公司 | A kind of microblog hot topic extracting method and system based on keyword search |
| CN109995856A (en) * | 2019-03-21 | 2019-07-09 | 国电南瑞科技股份有限公司 | A kind of grid operation data wide area collects method and system |
| CN110990351A (en) * | 2019-12-05 | 2020-04-10 | 南方电网数字电网研究院有限公司 | Unstructured data acquisition method, device and system and computer equipment |
| CN110990351B (en) * | 2019-12-05 | 2020-09-04 | 南方电网数字电网研究院有限公司 | Unstructured data acquisition method, device and system and computer equipment |
| CN111949612A (en) * | 2020-07-31 | 2020-11-17 | 广西美立方工程咨询有限公司 | Unstructured data storage middleware system based on hadoop and use method thereof |
| CN111949612B (en) * | 2020-07-31 | 2023-02-28 | 广西美立方工程咨询有限公司 | Unstructured data storage middleware system based on hadoop and use method thereof |
| CN112347071A (en) * | 2020-12-06 | 2021-02-09 | 国网山东省电力公司电力科学研究院 | Power distribution network cloud platform data fusion method and power distribution network cloud platform |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104820670B (en) | A kind of acquisition of power information big data and storage method | |
| CN105956932A (en) | Distribution and utilization data fusion method and system | |
| Casado et al. | Emerging trends and technologies in big data processing | |
| CN109446344B (en) | Intelligent analysis report automatic generation system based on big data | |
| CN102999537B (en) | System and method for data migration | |
| CN106227800B (en) | Storage method and management system for highly-associated big data | |
| CN104160394B (en) | Scalable analysis platform for semi-structured data | |
| Grolinger et al. | Knowledge as a service framework for disaster data management | |
| CN105183834B (en) | A kind of traffic big data semantic applications method of servicing based on ontology library | |
| CN109272155A (en) | A kind of corporate behavior analysis system based on big data | |
| US20150095303A1 (en) | Knowledge Graph Generator Enabled by Diagonal Search | |
| CN108964996B (en) | Urban and rural integrated information grid system and information sharing method based on same | |
| CN105956015A (en) | Service platform integration method based on big data | |
| CN108268565B (en) | Method and system for processing user browsing behavior data based on data warehouse | |
| CN106202536A (en) | Global metadata standardized platform system based on XBRL and construction method thereof | |
| CN102917009B (en) | A kind of stock certificate data collection based on cloud computing technology and storage means and system | |
| Mohammed et al. | A review of big data environment and its related technologies | |
| CN106095796A (en) | Distributed data storage method, Apparatus and system | |
| CN110275920A (en) | Data query method, apparatus, electronic equipment and computer readable storage medium | |
| CN106951552A (en) | A kind of user behavior data processing method based on Hadoop | |
| CN107247799A (en) | Data processing method, system and its modeling method of compatible a variety of big data storages | |
| CN103310025A (en) | Unstructured-data description method and device | |
| CN102902813A (en) | Log collection system | |
| CN103631922A (en) | Hadoop cluster-based large-scale Web information extraction method and system | |
| CN107103064A (en) | Data statistical approach and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160921 |
|
| RJ01 | Rejection of invention patent application after publication |