CN105824945A - Method for collecting global energy Internet technology resource data - Google Patents
Method for collecting global energy Internet technology resource data Download PDFInfo
- Publication number
- CN105824945A CN105824945A CN201610161855.1A CN201610161855A CN105824945A CN 105824945 A CN105824945 A CN 105824945A CN 201610161855 A CN201610161855 A CN 201610161855A CN 105824945 A CN105824945 A CN 105824945A
- Authority
- CN
- China
- Prior art keywords
- data
- global energy
- energy internet
- automatically
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000005516 engineering process Methods 0.000 title claims abstract description 42
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000004458 analytical method Methods 0.000 claims abstract description 20
- 238000013500 data storage Methods 0.000 claims abstract description 18
- 238000013480 data collection Methods 0.000 claims abstract description 16
- 238000003860 storage Methods 0.000 claims abstract description 9
- 230000005611 electricity Effects 0.000 claims description 24
- 238000009826 distribution Methods 0.000 claims description 19
- 230000000007 visual effect Effects 0.000 claims description 11
- 238000000605 extraction Methods 0.000 claims description 9
- 230000006870 function Effects 0.000 claims description 9
- 238000012544 monitoring process Methods 0.000 claims description 9
- 238000012423 maintenance Methods 0.000 claims description 8
- VNWKTOKETHGBQD-UHFFFAOYSA-N methane Chemical compound C VNWKTOKETHGBQD-UHFFFAOYSA-N 0.000 claims description 8
- 238000012800 visualization Methods 0.000 claims description 8
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 claims description 8
- 238000011156 evaluation Methods 0.000 claims description 7
- 238000007726 management method Methods 0.000 claims description 7
- 238000004519 manufacturing process Methods 0.000 claims description 7
- 238000004891 communication Methods 0.000 claims description 6
- 239000000284 extract Substances 0.000 claims description 6
- 210000000352 storage cell Anatomy 0.000 claims description 6
- 238000007405 data analysis Methods 0.000 claims description 4
- 238000011161 development Methods 0.000 claims description 4
- 238000004146 energy storage Methods 0.000 claims description 4
- 230000003993 interaction Effects 0.000 claims description 4
- 239000003345 natural gas Substances 0.000 claims description 4
- 238000003058 natural language processing Methods 0.000 claims description 4
- 238000001556 precipitation Methods 0.000 claims description 4
- 238000012545 processing Methods 0.000 claims description 4
- 230000002195 synergetic effect Effects 0.000 claims description 4
- 239000003034 coal gas Substances 0.000 claims description 3
- 238000013075 data extraction Methods 0.000 claims description 3
- 238000010276 construction Methods 0.000 abstract description 4
- 238000004364 calculation method Methods 0.000 abstract 1
- 238000013439 planning Methods 0.000 abstract 1
- 238000013481 data capture Methods 0.000 description 3
- 239000003245 coal Substances 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/285—Clustering or classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Economics (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Health & Medical Sciences (AREA)
- Marketing (AREA)
- Entrepreneurship & Innovation (AREA)
- Public Health (AREA)
- Game Theory and Decision Science (AREA)
- Educational Administration (AREA)
- Quality & Reliability (AREA)
- Operations Research (AREA)
- Water Supply & Treatment (AREA)
- General Health & Medical Sciences (AREA)
- Primary Health Care (AREA)
- Development Economics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention provides a method for collecting global energy Internet technology resource data. The method comprises the following steps: constructing a collection system to classify the global energy Internet technology resource data; acquiring the global energy Internet technology resource data in different ways; performing global energy Internet technology resource data storage and search based on a Hadoop distributed platform and Oracle full-text search specific to the characteristics of the global energy Internet technology resource data. By adopting the method, comprehensive, effective and accurate data collection through a global energy Internet technology is realized, and a basis is laid for the analysis, calculation, planning and assisted decision of global energy Internet construction based on multi-source information; moreover, the method has the advantages of various flexible data acquisition ways, abundant data types, inclusion of a large amount of information, rapidness and safety in storage and rapidness in access.
Description
Technical field
The present invention relates to Regulation field, be specifically related to a kind of global energy Internet technology resource data collection method.
Background technology
The geographical weather environment span that global energy the Internet relates to is big, design field is numerous, comprise data class many and dispersion, exist collection difficulty, analyze loaded down with trivial details problem.
There is presently no its research of a complete comprehensive support, the global energy Internet technology resource data collection method of integrated multi-specialized realm information.Global energy the Internet is exactly " extra-high voltage grid+intelligent grid+clean energy resource ", with intelligent grid coordinates collection of data method as reference.Intelligent grid mostly only considered power industry related data when data collection, and data acquiring mode is the most single, and data class is relatively fewer, lacks the thinking of globalization data collection mode of thinking.
Summary of the invention
In view of this, a kind of global energy Internet technology resource data collection method that the present invention provides, the method achieve comprehensive, effectively and accurately for global energy Internet technology carry out data collection, for the analysis based on multi-source information of global energy Internet Construction, calculate, plan and aid decision lays the foundation, and its data acquiring mode is many and flexible, data class enriches, it is many to comprise information, storage quick and safe and accessing rapidly.
It is an object of the invention to be achieved through the following technical solutions:
A kind of global energy Internet technology resource data collection method, described method comprises the steps:
Step 1. sets up the data gathering system of global energy Internet technology resource, and described data gathering system includes data storage cell, monitoring unit, data center, visual presentation platform, analysis and evaluation unit, specialized computing unit, data maintenance unit and the data-interface being in communication with each other；
Described global energy Internet technology resource data, according to the source of global energy internet data, is classified by step 2.；
Step 3. obtains described global energy Internet technology resource data；
Step 4., based on Hadoop distributed platform and Oracle full-text search, sets up global energy internet data storage and retrieval structural system.
Preferably, the described data storage cell in described step 1 includes oracle database and Hadoop distributed file system；
Described monitoring unit is interface monitoring terminal；
Described data center is global energy Internet data center, and provides data retrieval for oracle database, and stores based on Hadoop distributed file system and calculate；
Described visual presentation platform includes visual human-computer interaction interface；
Described analysis and evaluation unit data analysis based on index system establishment is applied with appraisal procedure；
Described specialized computing unit calculates based on described Visualization Platform；
Described data maintenance unit is for being managed described data and safeguarding；
Described data-interface includes that data user-machine interface, web interface data obtain data acquisition interface in interface and power industry automatically.
Preferably, described step 2 includes:
2-1., according to the source of global energy internet data, carries out a subseries to described global energy Internet technology resource data, obtains a subseries number play staff；Wherein, a described subseries number play staff includes geographic information data, meteorological data, resource data, electricity transaction class data, technical capability data and basic data；
2-2. carries out secondary classification to each data in a described subseries number play staff, including:
Described geographic information data includes longitude and the energy distributed intelligence of the distribution in latitude, mountains and rivers, river and lake, water energy, wind energy and solar energy；
Described meteorological data includes temperature, wind-force and precipitation data；
What described resource data included wind, light, water, coal and natural gas can source distribution, cost and can development reserves information；
Described electricity transaction class data include market quotes, exchange hand, conclusion of the business electricity price, load type, electric pressure, date and exchange rate information；
Described technical capability data include power supply class technical capability data and electrical network class technical capability data；
Described basic data includes countries population, GDP and tertiary industry GDP accounting information；
Described power supply class technical capability data include the generating set type of wind-powered electricity generation and photovoltaic energy, installed capacity and energy storage parameter；Described electrical network class technical capability data include grid equipment parameter, capacity of trunk and load data.
Preferably, the mode obtaining described global energy Internet technology resource data in described step 3 includes:
User, according to self-demand, carries out web data and automatically searches for and obtain；
Obtain power industry expert data；Wherein, described expert data includes electric power enterprise production run data, electric power enterprise operation data, Management of Electrical Enterprise data, Urban Data, achievement data and thematic data；
Automatically extract data message in text, and according to the Type division of described data, it is achieved data based on character analysis function obtain automatically.
Preferably, described user, according to self-demand, carries out web data and automatically searches for and obtain, including:
A. user formulates according to self-demand and downloads rule；
B. user downloads rule according to described, determines download period and system running frequency, carries out web data and automatically search for and obtain.
Preferably, described step b includes:
B-1. from targeted website, obtain the more new data of service end in real time, during the navigation of the most described webpage auto-browsing, mixed processing html text and JavaScript script, in the page, obtain hyperlink, complete web data and automatically search for；
B-2. user downloads rule according to described, determines download period and system running frequency, automatically obtains more new data and stored to locally stored catalogue by described more new data；Complete web data automatically to obtain.
Preferably, if the described page in described step b-1 is the list data page, the most described step b-1 also includes:
C. user selects table field information to put mode in storage with list data；
D. record user selects and timing selects according to described user, the data loading that will update in the described list data page.
Preferably, described electric power enterprise production run data in described acquisition power industry expert data include generated energy, power distribution network main equipment and voltage stability data, wherein, described power distribution network main equipment includes high-tension line, main transformer, medium-voltage line and distribution transformer；
Described electric power enterprise operation data includes pricing, electricity sales amount and Electricity customers data；
Described Management of Electrical Enterprise data include ERP, unified platform and synergetic office work data；
Described Urban Data includes the population in city, geographical position and air quality data.
Preferably, data message in text is automatically extracted, and according to the Type division of described data described in, it is achieved data based on character analysis function obtain automatically, including:
E. for target URL, use extraction model based on natural language processing, automatically carry out the extraction of text message；
F. in the locally stored hard disk of described data extraction obtained；
G. according to Text Classification based on naive Bayesian, data are classified automatically, and according to probability belonging to the technical resource data message type of the information of calculating, described information is divided into geodata information, weather information or energy information.
Preferably, the data base in described global energy internet data storage and retrieval structural system in described step 4 is relevant database, and described global energy internet data storage and retrieval structural system includes that the information collection module being in communication with each other, index module, text cluster module, classified index module, index merge module, enquiry module and visualization model
From above-mentioned technical scheme it can be seen that the invention provides a kind of global energy Internet technology resource data collection method, the method by building collection system,；Global energy Internet technology resource data is classified；Obtain described global energy Internet technology resource data in a different manner；For global energy Internet technology resource data feature, it is taken based on the global energy internet data storage and retrieval of Hadoop distributed platform and Oracle full-text search.The present invention propose method achieve comprehensive, effectively and accurately for global energy Internet technology carry out data collection, for the analysis based on multi-source information of global energy Internet Construction, calculate, plan and aid decision lays the foundation, and its data acquiring mode is many and flexible, data class enriches, it is many to comprise information, storage quick and safe and accessing rapidly.
With immediate prior art ratio, the technical scheme that the present invention provides has a following excellent effect:
1, in technical scheme provided by the present invention, for the analysis based on multi-source information of global energy Internet Construction, calculate, plan and aid decision lays the foundation.
2, technical scheme provided by the present invention, support based on data center's hardware platform, classify from data, obtain, store the collection carrying out global energy Internet technology resource data in terms of three, it is achieved that carry out data collection for global energy Internet technology comprehensively, effectively and accurately.
3, technical scheme provided by the present invention, data acquiring mode is many and flexible, data class enriches, it is many to comprise information, store quick and safe and access rapid.
4, the technical scheme that the present invention provides, is widely used, has significant Social benefit and economic benefit.
Accompanying drawing explanation
Fig. 1 is the flow chart of a kind of global energy Internet technology resource data collection method of the present invention；
Fig. 2 be the present invention method of data capture in the schematic flow sheet of step 2；
Fig. 3 be the present invention method of data capture in user in step 3 according to self-demand, carry out the schematic flow sheet that web data is automatically searched for and obtained；
Fig. 4 be the present invention method of data capture in step 3 automatically extracts data message in text, and according to the Type division of described data, it is achieved the schematic flow sheet that data based on character analysis function obtain automatically；
Fig. 5 is the global energy Internet data center hardware structure figure in the concrete application examples of the present invention；
Fig. 6 is the global energy Internet data center data base _ ER illustraton of model in the concrete application examples of the present invention；
Fig. 7 is the global energy internet data index structure in the concrete application examples of the present invention.
Detailed description of the invention
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is clearly and completely described, it is clear that described embodiment is only a part of embodiment of the present invention rather than whole embodiments.Based on embodiments of the invention, the every other embodiment that those of ordinary skill in the art are obtained under not making creative work premise, broadly fall into the scope of protection of the invention.
As it is shown in figure 1, the present invention provides a kind of global energy Internet technology resource data collection method, method comprises the steps:
Step 1. sets up the data gathering system of global energy Internet technology resource, and data gathering system includes data storage cell, monitoring unit, data center, visual presentation platform, analysis and evaluation unit, specialized computing unit, data maintenance unit and the data-interface being in communication with each other；
Global energy Internet technology resource data, according to the source of global energy internet data, is classified by step 2.；
Step 3. obtains global energy Internet technology resource data；
Step 4., based on Hadoop distributed platform and Oracle full-text search, sets up global energy internet data storage and retrieval structural system.
Wherein, the data storage cell in step 1 includes oracle database and Hadoop distributed file system；
Monitoring unit is interface monitoring terminal；
Data center is global energy Internet data center, and provides data retrieval for oracle database, and stores based on Hadoop distributed file system and calculate；
Visual presentation platform includes visual human-computer interaction interface；
Analysis and evaluation unit data analysis based on index system establishment is applied with appraisal procedure；
Specialized computing unit calculates based on Visualization Platform；
Data maintenance unit is for being managed data and safeguarding；
Data-interface includes that data user-machine interface, web interface data obtain data acquisition interface in interface and power industry automatically.
Wherein, the mode obtaining global energy Internet technology resource data in step 3 includes:
User, according to self-demand, carries out web data and automatically searches for and obtain；
Obtain power industry expert data；Wherein, expert data includes electric power enterprise production run data, electric power enterprise operation data, Management of Electrical Enterprise data, Urban Data, achievement data and thematic data；
Automatically extract data message in text, and according to the Type division of data, it is achieved data based on character analysis function obtain automatically.
As in figure 2 it is shown, step 2 includes:
2-1., according to the source of global energy internet data, carries out a subseries to global energy Internet technology resource data, obtains a subseries number play staff；Wherein, a subseries number play staff includes geographic information data, meteorological data, resource data, electricity transaction class data, technical capability data and basic data；
2-2. carries out secondary classification to each data in a subseries number play staff, including:
Geographic information data includes longitude and the energy distributed intelligence of the distribution in latitude, mountains and rivers, river and lake, water energy, wind energy and solar energy；
Meteorological data includes temperature, wind-force and precipitation data；
What resource data included wind, light, water, coal and natural gas can source distribution, cost and can development reserves information；
Electricity transaction class data include market quotes, exchange hand, conclusion of the business electricity price, load type, electric pressure, date and exchange rate information；
Technical capability data include power supply class technical capability data and electrical network class technical capability data；
Basic data includes countries population, GDP and tertiary industry GDP accounting information.
Wherein, the power supply class technical capability data in 2-2 include the generating set type of wind-powered electricity generation and photovoltaic energy, installed capacity and energy storage parameter；Electrical network class technical capability data include grid equipment parameter, capacity of trunk and load data.
As it is shown on figure 3, the user in step 3 is according to self-demand, carries out web data and automatically search for and include with acquisition:
A. user formulates according to self-demand and downloads rule；
B. user is according to downloading rule, determines download period and system running frequency, carries out web data and automatically search for and obtain.
Wherein, step b includes:
B-1. from targeted website, obtain the more new data of service end in real time, i.e. during the navigation of webpage auto-browsing, mixed processing html text and JavaScript script, in the page, obtain hyperlink, complete web data and automatically search for；
B-2. user is according to downloading rule, determines download period and system running frequency, automatically obtains more new data and stored to locally stored catalogue by more new data；Complete web data automatically to obtain.
Wherein, if the page in step b-1 is the list data page, then step b-1 also includes:
C. user selects table field information to put mode in storage with list data；
D. record user selects and timing selects according to user, the data loading that will update in the list data page.
Wherein, the electric power enterprise production run data in step 4 include generated energy, power distribution network main equipment and voltage stability data, and wherein, power distribution network main equipment includes high-tension line, main transformer, medium-voltage line and distribution transformer；
Electric power enterprise operation data includes pricing, electricity sales amount and Electricity customers data；
Management of Electrical Enterprise data include ERP, unified platform and synergetic office work data；
Urban Data includes the population in city, geographical position and air quality data.
As shown in Figure 4, step 3 automatically extracts data message in text, and according to the Type division of data, it is achieved data based on character analysis function automatically obtain and include:
E. for target URL, use extraction model based on natural language processing, automatically carry out the extraction of text message；
F. in the locally stored hard disk of data extraction obtained；
G. according to Text Classification based on naive Bayesian, data are classified automatically, and according to probability belonging to the technical resource data message type of the information of calculating, information is divided into geodata information, weather information or energy information.
Wherein, the data base in global energy internet data storage and retrieval structural system in step 4 is relevant database, and global energy internet data storage and retrieval structural system includes that the information collection module being in communication with each other, index module, text cluster module, classified index module, index merge module, enquiry module and visualization model.
The present invention provides the concrete application examples of a kind of global energy Internet technology resource data collection method, as follows:
1) data center's hardware structure is as shown in Figure 5:
Data-interface: include that data man machine interface, web interface data obtain and data acquisition in power industry automatically.
Data: global energy internet data, give oracle database and carry out data retrieval, store based on Hadoop distributed file system and calculate.
Visual presentation: visual human-computer interaction interface
Analysis and evaluation: data analysis based on index system establishment is applied with appraisal procedure.
Specialized calculating: professional computing function based on Visualization Platform.
Data maintenance: data management and maintenance.
2) data classification principle is established.According to the difference in global energy internet data source, data can be categorized as geographic information data, meteorological data, resource data, electricity transaction class data, technical capability data and basic data.Geographic information data mainly includes longitude, latitude, the distributed intelligence of the primary energy such as mountains and rivers, river, the distribution in lake, water energy, wind energy, solar energy.
Meteorological data mainly includes temperature, wind-force, precipitation etc..Resource data include the primary energy distributions such as wind, light, water, coal, natural gas, cost, can the information such as development reserves.
Electricity transaction class data mainly include each market quotes, exchange hand, conclusion of the business electricity price, load type, electric pressure, the information such as date and the exchange rate；The technical capability packet data containing two aspects: power supply class technical capability data, electrical network class technical capability data.
Power supply class technical capability data mainly include wind-powered electricity generation, the generating set type of photovoltaic equal energy source, installed capacity, energy storage parameter etc.；Electrical network class technical capability data mainly include the data such as grid equipment parameter, capacity of trunk, load；Basic data includes the information such as countries population, GDP, tertiary industry GDP accounting.As shown in Figure 6.
3) need for user, carry out web data and automatically search for and obtain.
Specifically referring to, user according to demand, oneself formulates and downloads rule, including single data download period and automated system operation frequency etc., obtains the data that up-to-date service end pushes in real time from targeted website, stores in local storage catalogue.
Technically can be divided into two steps, webpage auto-browsing navigates, the automatic acquisition of more new data.In terms of the auto-browsing navigation of webpage, html text and JavaScript script are made mixed processing, in the page, intactly crawls contained hyperlink, in terms of the automatic acquisition of more new data, formulated by user above oneself and download rule, determine download period and system running frequency.For the list data page, selecting table field information to put mode in storage with list data, program can be recorded user and select, and the most periodically selects according to user, the data loading that will update in this page.
Above two steps achieve the automatic acquisition of webpage more new data.
4) in power industry, relevant speciality data are obtained.
Data center has expert data Acquisition channel in industry, can safety from abundant data resource obtain global energy Internet technology resource related information.Expert data includes electric power enterprise production run data, the data in terms of generated energy, power distribution network main equipment (including high-tension line, main transformer, medium-voltage line and distribution transformer etc.), voltage stability etc.；Electric power enterprise operation data, data in terms of pricing, electricity sales amount, Electricity customers etc.；Management of Electrical Enterprise data, the data in terms of ERP, unified platform, synergetic office work etc..Secondly, the achievement data such as macroeconomy, meteorological data or thematic data, the population in domestic and international multiple cities, geographical position, the data such as air quality are also contained in power industry data repository.
5) by automatically extracting text data information, and the method that data are classified, it is achieved data based on character analysis function obtain automatically.
For target URL, use extraction model based on natural language processing, automatically carry out the extraction of text message,.In the locally stored hard disk of data that extraction obtains, use Text Classification based on naive Bayesian that data are classified automatically, belong to the probability of any class technical resource data message by calculating certain information, be geodata information by information classification, weather information, energy information etc..
6) global energy internet data storage and retrieval structural models based on Hadoop distributed platform Yu Oracle full-text search.
It is various that Hadoop distributed file storage system can process structure type, and renewal speed is fast, mass historical data carries out off-line analysis and processes the global energy Internet technology resource data strong with interactivity.
The global energy internet information of multi-source heterogeneous source set is supported in Oracle full-text search, keeps the verity of legacy data largely.
The system uses relevant database.The structure of system mainly includes that information collection module, index module, text cluster module, classified index module, index merge module, enquiry module and visualization model etc., as shown in Figure 7.
Above example is only in order to illustrate that technical scheme is not intended to limit; although the present invention being described in detail with reference to above-described embodiment; the detailed description of the invention of the present invention still can be modified or equivalent by those of ordinary skill in the field; and these are without departing from any amendment of spirit and scope of the invention or equivalent, within the claims of its present invention all awaited the reply in application.
Claims (10)
1. a global energy Internet technology resource data collection method, it is characterised in that described method comprises the steps:
Step 1. sets up the data gathering system of global energy Internet technology resource, and described data gathering system includes data storage cell, monitoring unit, data center, visual presentation platform, analysis and evaluation unit, specialized computing unit, data maintenance unit and the data-interface being in communication with each other；
Described global energy Internet technology resource data, according to the source of global energy internet data, is classified by step 2.；
Step 3. obtains described global energy Internet technology resource data；
Step 4., based on Hadoop distributed platform and Oracle full-text search, sets up global energy internet data storage and retrieval structural system.
2. the method for claim 1, it is characterised in that the described data storage cell in described step 1 includes oracle database and Hadoop distributed file system；
Described monitoring unit is interface monitoring terminal；
Described data center is global energy Internet data center, and provides data retrieval for oracle database, and stores based on Hadoop distributed file system and calculate；
Described visual presentation platform includes visual human-computer interaction interface；
Described analysis and evaluation unit data analysis based on index system establishment is applied with appraisal procedure；
Described specialized computing unit calculates based on described Visualization Platform；
Described data maintenance unit is for being managed described data and safeguarding；
Described data-interface includes that data user-machine interface, web interface data obtain data acquisition interface in interface and power industry automatically.
3. the method for claim 1, it is characterised in that described step 2 includes:
2-1., according to the source of global energy internet data, carries out a subseries to described global energy Internet technology resource data, obtains a subseries number play staff；Wherein, a described subseries number play staff includes geographic information data, meteorological data, resource data, electricity transaction class data, technical capability data and basic data；
2-2. carries out secondary classification to each data in a described subseries number play staff, including:
Described geographic information data includes longitude and the energy distributed intelligence of the distribution in latitude, mountains and rivers, river and lake, water energy, wind energy and solar energy；
Described meteorological data includes temperature, wind-force and precipitation data；
What described resource data included wind, light, water, coal and natural gas can source distribution, cost and can development reserves information；
Described electricity transaction class data include market quotes, exchange hand, conclusion of the business electricity price, load type, electric pressure, date and exchange rate information；
Described technical capability data include power supply class technical capability data and electrical network class technical capability data；
Described basic data includes countries population, GDP and tertiary industry GDP accounting information；
Described power supply class technical capability data include the generating set type of wind-powered electricity generation and photovoltaic energy, installed capacity and energy storage parameter；Described electrical network class technical capability data include grid equipment parameter, capacity of trunk and load data.
4. the method for claim 1, it is characterised in that the mode obtaining described global energy Internet technology resource data in described step 3 includes:
User, according to self-demand, carries out web data and automatically searches for and obtain；
Obtain power industry expert data；Wherein, described expert data includes electric power enterprise production run data, electric power enterprise operation data, Management of Electrical Enterprise data, Urban Data, achievement data and thematic data；
Automatically extract data message in text, and according to the Type division of described data, it is achieved data based on character analysis function obtain automatically.
5. method as claimed in claim 4, it is characterised in that described user, according to self-demand, carries out web data and automatically searches for and obtain, including:
A. user formulates according to self-demand and downloads rule；
B. user downloads rule according to described, determines download period and system running frequency, carries out web data and automatically search for and obtain.
6. method as claimed in claim 5, it is characterised in that described step b includes:
B-1. from targeted website, obtain the more new data of service end in real time, during the navigation of the most described webpage auto-browsing, mixed processing html text and JavaScript script, in the page, obtain hyperlink, complete web data and automatically search for；
B-2. user downloads rule according to described, determines download period and system running frequency, automatically obtains more new data and stored to locally stored catalogue by described more new data；Complete web data automatically to obtain.
7. method as claimed in claim 6, it is characterised in that if the described page in described step b-1 is the list data page, also include in the most described step b-1:
C. user selects table field information to put mode in storage with list data；
D. record user selects and timing selects according to described user, the data loading that will update in the described list data page.
8. method as claimed in claim 4, it is characterized in that, described electric power enterprise production run data in described acquisition power industry expert data include generated energy, power distribution network main equipment and voltage stability data, wherein, described power distribution network main equipment includes high-tension line, main transformer, medium-voltage line and distribution transformer；
Described electric power enterprise operation data includes pricing, electricity sales amount and Electricity customers data；
Described Management of Electrical Enterprise data include ERP, unified platform and synergetic office work data；
Described Urban Data includes the population in city, geographical position and air quality data.
9. method as claimed in claim 4, it is characterised in that described in automatically extract data message in text, and according to the Type division of described data, it is achieved data based on character analysis function obtain automatically, including:
E. for target URL, use extraction model based on natural language processing, automatically carry out the extraction of text message；
F. in the locally stored hard disk of described data extraction obtained；
G. according to Text Classification based on naive Bayesian, data are classified automatically, and according to probability belonging to the technical resource data message type of the information of calculating, described information is divided into geodata information, weather information or energy information.
10. the method for claim 1, it is characterized in that, the data base in described global energy internet data storage and retrieval structural system in described step 4 is relevant database, and described global energy internet data storage and retrieval structural system includes that the information collection module being in communication with each other, index module, text cluster module, classified index module, index merge module, enquiry module and visualization model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610161855.1A CN105824945A (en) | 2016-03-21 | 2016-03-21 | Method for collecting global energy Internet technology resource data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610161855.1A CN105824945A (en) | 2016-03-21 | 2016-03-21 | Method for collecting global energy Internet technology resource data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105824945A true CN105824945A (en) | 2016-08-03 |
Family
ID=56524866
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610161855.1A Pending CN105824945A (en) | 2016-03-21 | 2016-03-21 | Method for collecting global energy Internet technology resource data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105824945A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108694220A (en) * | 2017-04-12 | 2018-10-23 | 普天信息技术有限公司 | A kind of air quality index acquisition methods and device |
| CN109214435A (en) * | 2018-08-21 | 2019-01-15 | 北京睦合达信息技术股份有限公司 | A kind of data classification method and device |
| CN109857819A (en) * | 2018-11-20 | 2019-06-07 | 国网能源研究院有限公司 | A kind of global energy geography multimedia interactive display systems |
| CN114638558A (en) * | 2022-05-19 | 2022-06-17 | 天津市普迅电力信息技术有限公司 | Data set classification method for operation accident analysis of comprehensive energy system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104036025A (en) * | 2014-06-27 | 2014-09-10 | 蓝盾信息安全技术有限公司 | Distribution-base mass log collection system |
| US20140292533A1 (en) * | 2011-04-22 | 2014-10-02 | Expanergy, Llc | Universal energy internet of things apparatus and methods |
| CN104820670A (en) * | 2015-03-13 | 2015-08-05 | 国家电网公司 | Method for acquiring and storing big data of power information |
| CN104881424A (en) * | 2015-03-13 | 2015-09-02 | 国家电网公司 | Regular expression-based acquisition, storage and analysis method of power big data |
| CN105119750A (en) * | 2015-09-08 | 2015-12-02 | 南京联成科技发展有限公司 | Distributed information security operation and maintenance management platform based on massive data |
-
2016
- 2016-03-21 CN CN201610161855.1A patent/CN105824945A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140292533A1 (en) * | 2011-04-22 | 2014-10-02 | Expanergy, Llc | Universal energy internet of things apparatus and methods |
| CN104036025A (en) * | 2014-06-27 | 2014-09-10 | 蓝盾信息安全技术有限公司 | Distribution-base mass log collection system |
| CN104820670A (en) * | 2015-03-13 | 2015-08-05 | 国家电网公司 | Method for acquiring and storing big data of power information |
| CN104881424A (en) * | 2015-03-13 | 2015-09-02 | 国家电网公司 | Regular expression-based acquisition, storage and analysis method of power big data |
| CN105119750A (en) * | 2015-09-08 | 2015-12-02 | 南京联成科技发展有限公司 | Distributed information security operation and maintenance management platform based on massive data |
Non-Patent Citations (2)
| Title |
|---|
| 罗学礼 等: ""电力企业的非结构化数据检索研究"", 《计算机与数字工程》 * |
| 蒲天骄 等: ""基于主动配电网的城市能源互联网体系架构及其关键技术"", 《中国电机工程学报》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108694220A (en) * | 2017-04-12 | 2018-10-23 | 普天信息技术有限公司 | A kind of air quality index acquisition methods and device |
| CN109214435A (en) * | 2018-08-21 | 2019-01-15 | 北京睦合达信息技术股份有限公司 | A kind of data classification method and device |
| CN109857819A (en) * | 2018-11-20 | 2019-06-07 | 国网能源研究院有限公司 | A kind of global energy geography multimedia interactive display systems |
| CN109857819B (en) * | 2018-11-20 | 2021-02-05 | 国网能源研究院有限公司 | Global energy geography multimedia interactive display system |
| CN114638558A (en) * | 2022-05-19 | 2022-06-17 | 天津市普迅电力信息技术有限公司 | Data set classification method for operation accident analysis of comprehensive energy system |
| CN114638558B (en) * | 2022-05-19 | 2022-08-23 | 天津市普迅电力信息技术有限公司 | Data set classification method for operation accident analysis of comprehensive energy system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Daneshvar Rouyendegh et al. | Using intuitionistic fuzzy TOPSIS in site selection of wind power plants in Turkey | |
| Nielsen et al. | GIS based analysis of future district heating potential in Denmark | |
| CN109635127B (en) | Power equipment portrait knowledge map construction method based on big data technology | |
| JP6310662B2 (en) | Utilities management analysis via social network data | |
| Lorenzoni et al. | Classification and modeling of load profiles of isolated mini-grids in developing countries: A data-driven approach | |
| CN105824945A (en) | Method for collecting global energy Internet technology resource data | |
| Ayodele et al. | A statistical analysis of wind distribution and wind power potential in the coastal region of South Africa | |
| Xiao et al. | Research on an optimal site selection model for desert photovoltaic power plants based on analytic hierarchy process and geographic information system | |
| Almutairi | Determining the appropriate location for renewable hydrogen development using multi‐criteria decision‐making approaches | |
| Shahraki Shahdabadi et al. | Using multi-criteria decision-making methods to select the best location for the construction of a biomass power plant in Iran | |
| Ali et al. | Generating open-source datasets for power distribution network using openstreetmaps | |
| DeLucia et al. | Energy planning for developing countries: a study of Bangladesh | |
| Park et al. | Analysis on trends and future signs of smart grids | |
| CN113902583A (en) | Distribution network side operation and maintenance method and system using low-voltage network equipment data | |
| Al-Yahyai et al. | Wind resource assessment using numerical weather prediction models and multi-criteria decision making technique: case study (Masirah Island, Oman) | |
| CN117333032A (en) | Management method and system for canal city weather safety monitoring and forecasting service | |
| Dutta | Data mining and graph theory focused solutions to smart grid challenges | |
| Hu et al. | Big data analysis for the hydropower development potential of ASEAN-8 based on the hydropower digital planning model | |
| Wang et al. | Research on tariff recovery risks assessment method based on electrical user portrait technology | |
| Mutale et al. | Economic feasibility of onshore wind energy potential for electricity generation in Zambia | |
| Yin et al. | Estimating power plant generation in the global power plant database | |
| Trofimov et al. | Output Forms for Calculation Results in the Computing & Geo-Information System | |
| Sawyer | Meeting Future Electricity Needs in the East African Community: Mapping Renewable Energy Potential | |
| Voronkin et al. | Economic efficiency of power stations using renewable energy sources | |
| Abudureyimu et al. | Off-shore wind power potential evaluation and economy analysis of entire Japan using GIS technology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160803 |