CN111737483A - Construction method of big data knowledge graph of smart power grid - Google Patents
Construction method of big data knowledge graph of smart power grid Download PDFInfo
- Publication number
- CN111737483A CN111737483A CN202010375821.9A CN202010375821A CN111737483A CN 111737483 A CN111737483 A CN 111737483A CN 202010375821 A CN202010375821 A CN 202010375821A CN 111737483 A CN111737483 A CN 111737483A
- Authority
- CN
- China
- Prior art keywords
- power grid
- knowledge graph
- data
- big data
- established model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
- G06Q50/06—Electricity, gas or water supply
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Health & Medical Sciences (AREA)
- Economics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Water Supply & Treatment (AREA)
- Human Resources & Organizations (AREA)
- Life Sciences & Earth Sciences (AREA)
- Quality & Reliability (AREA)
- Public Health (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Animal Behavior & Ethology (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Supply And Distribution Of Alternating Current (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The embodiment of the application provides a construction method of a big data knowledge graph of an intelligent power grid, and the construction method comprises the following steps: acquiring power grid equipment information and the connection relation between the power grid equipment from the power grid equipment by Neo4 j; based on a historical topology database, carrying out data cleaning on the obtained power grid equipment information; modeling the cleaned data according to the connection relation between the power grid devices; and carrying out topology search analysis on the established model to obtain a topological relation knowledge graph between target devices. The method is beneficial to quickly constructing different knowledge models according to business requirements, discovering hidden values in the information of the electric power big data, and realizing efficient analysis on the panoramic data of cross-region, cross-time and cross-space information resources.
Description
Technical Field
The invention belongs to the field of map construction, and particularly relates to a construction method of a big data knowledge map of an intelligent power grid.
Background
The power network is large and complex in structure, and the query operation speed and the performance of the traditional database are very slow and poor. The knowledge map technology can obviously improve the effectiveness of knowledge retrieval, so that the retrieval result is more comprehensive and accurate, the query intention of a user can be systematically understood, and the accurate answer is directly returned instead of a large number of search results. The power grid knowledge intelligent retrieval system is developed on the basis of the power grid knowledge map. For example, in a power grid system, if it is desired to know whether a fault of one device affects a certain key device, if a query operation is performed in a plurality of tables by using a conventional relational database, a relational path between two devices needs to be known in advance, editing difficulty of query statements is large, and query speed is extremely slow due to differences in data structures.
Disclosure of Invention
In order to overcome the defects and shortcomings in the prior art, the invention provides the construction method of the smart grid big data knowledge graph, which is beneficial to quickly constructing different knowledge models according to business requirements, finding the hidden value in the information of the electric power big data and realizing the efficient analysis of the panoramic data of cross-region, cross-time and cross-space information resources.
The embodiment of the application provides a construction method of a big data knowledge graph of an intelligent power grid, and the construction method comprises the following steps:
acquiring power grid equipment information and the connection relation between the power grid equipment from the power grid equipment by Neo4 j;
based on a historical topology database, carrying out data cleaning on the obtained power grid equipment information;
modeling the cleaned data according to the connection relation between the power grid devices;
and carrying out topology search analysis on the established model to obtain a topological relation knowledge graph between target devices.
Optionally, the performing topology search analysis on the established model includes:
and carrying out topology search analysis on the established model based on a search algorithm.
Optionally, the search algorithm includes:
graph traversal algorithms, spanning tree algorithms, and path search algorithms.
Optionally, the performing topology search analysis on the established model to obtain a knowledge graph of a topological relation between target devices includes:
determining an initial storage node Vs where a target device is located;
traversing the nodes Vt adjacent to the initial storage node Vs in the established model;
traversing nodes Vn adjacent to the nodes Vt in the established model;
and if the target storage node V is traversed in the traversing process, outputting a path between the initial storage node Vs and the target storage node V according to the traversing process.
The technical scheme provided by the invention has the beneficial effects that:
the method is beneficial to quickly constructing different knowledge models according to business requirements, discovering hidden values in the information of the electric power big data, and realizing efficient analysis on the panoramic data of cross-region, cross-time and cross-space information resources.
Drawings
In order to more clearly illustrate the technical solution of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on the drawings without creative efforts.
FIG. 1 is a schematic flow chart diagram of a method for constructing a big data knowledge graph of a smart grid according to the present application;
FIG. 2 is an algorithm flow diagram of a breadth-first search algorithm as set forth in the present application;
FIG. 3(a) is a schematic flow chart of a method for constructing a big data knowledge graph of a smart grid according to the present application;
fig. 3(b) is a schematic flow chart of a method for constructing a big data knowledge graph of a smart grid according to the present application.
Detailed Description
To make the structure and advantages of the present invention clearer, the structure of the present invention will be further described with reference to the accompanying drawings.
Example one
The embodiment of the application provides a construction method of a big data knowledge graph of a smart grid, and as shown in fig. 1, the construction method comprises the following steps:
11. acquiring power grid equipment information and the connection relation between the power grid equipment from the power grid equipment by Neo4 j;
12. based on a historical topology database, carrying out data cleaning on the obtained power grid equipment information;
13. modeling the cleaned data according to the connection relation between the power grid devices;
14. and carrying out topology search analysis on the established model to obtain a topological relation knowledge graph between target devices.
The power network is large and complex in structure, and the query operation speed and the performance of the traditional database are very slow and poor. The knowledge map technology can obviously improve the effectiveness of knowledge retrieval, so that the retrieval result is more comprehensive and accurate, the query intention of a user can be systematically understood, and the accurate answer is directly returned instead of a large number of search results. For example, in a power grid system, if it is desired to know whether a fault of one device affects a certain key device, if a query operation is performed in a plurality of tables by using a conventional relational database, a relational path between two devices needs to be known in advance, editing difficulty of query statements is large, and query speed is extremely slow due to differences in data structures.
For example, in a power grid system, if it is desired to know whether a fault of one device affects a certain key device, if a query operation is performed in a plurality of tables by using a conventional relational database, a relational path between two devices needs to be known in advance, editing difficulty of query statements is large, and query speed is extremely slow due to differences in data structures. A series of tests are carried out on the basis of power grid data of a certain power-saving company, model relations among all systems are integrated on the basis of power grid resource model data, a set of power grid knowledge model based on a knowledge graph is established, and some advanced analysis services can be provided on the model according to a graph theory algorithm.
In the knowledge graph, by using the characteristics of a Neo4j graph data structure, a breadth-first search method is adopted for searching, the time complexity of traversing a data network is only O (n), and only two devices with unknown relations need to be input, and the relation paths and other nodes existing on the paths can be returned. Given two nodes V _ s and V _ c, the patent needs to find the shortest path from V _ s to V _ e, and the algorithm flow chart of the breadth-first search algorithm is shown in fig. 2.
The power grid system contains thousands of power devices and elements, and constitutes a large and complex power network. The Zhejiang electric power company develops data middle station construction work in 2016, and as long as 2018 for 1 month, the total number of the electric networks in the Zhejiang province is counted as follows: the main network 217 ten thousand, the distribution network 2568 ten thousand, 1 hundred million 153 sets of low-voltage equipment and 1 hundred million 2983 sets in total. The total data storage amount is 560.48TB, wherein the total data storage amount is structured data 209.5TB, unstructured data 254.86TB and real-time measurement data 72.02 TB. A method for constructing a big data knowledge graph of an intelligent power grid is designed and implemented, a large amount of equipment information of a power grid domain and the relation between the equipment information and the relation are integrated, and various services such as network visualization, equipment and relation retrieval, power grid topology analysis and the like are carried out on the basis of the knowledge graph of the power grid domain.
(I) power grid resource model service integration display
The system automatically imports each resource model service and performs equipment association relation analysis and display;
providing an efficient data retrieval function for each service;
various implicit structural relations among the devices can be efficiently analyzed;
(II) clustering analysis of power grid resource data
The correlations between the data objects are dynamically fitted. The clustering method does not need to deeply recognize data and give clustering number in advance, and realizes association degree analysis by calculating the similarity of various integrated electric power data objects;
calculating various graph theory algorithms, similarity (converting a clustering problem of data into a modularization problem of a network), and node strength (in a spatial clustering process, the greater the similarity among data objects is, the more obvious the importance of the node is, and the larger cohesion is in a local range);
the test results are shown in table 1, the time performance of the knowledge graph using the method is superior to that of the traditional relational database, on some tasks, the retrieval records of the knowledge graph are more than those of the relational database, and even some retrieval tasks can not be completed through the relational database. This is because the data structures of the two layers are different, and the knowledge map layer stored in Neo4j is a high-performance map engine with all the characteristics of a mature database, and the structured data is stored in the network instead of a table, so that the defect that the traditional relational database is not good at processing the relational network is overcome. For those records that cannot be retrieved by the relational database and the tasks that cannot be completed, the relationship between the device nodes is too complex or the relationship path is too long, so that the database has insufficient capability to handle, which also reflects the problem that the relational database is not good at handling the relational network. It is emphasized that all data in the database service is statistics of queries with known relationship paths, which otherwise would consume more time.
TABLE 1 time Performance comparison Table
As the number of provincial power grid equipment reaches hundreds of millions, the efficiency and timeliness of data migration are important indexes of an evaluation model. In the process of actually establishing the power grid knowledge graph, the method tests the efficiency of dumping data by using the traditional LOAD-CSV algorithm and the Neo4j-Import method provided by the method for the node number with different orders of magnitude.
The LOAD-CSV and the Neo4j-Import are two data Import modes provided by Neo4j and are suitable for different application scenarios. It can be known from fig. 3(a) that as the number of nodes increases, the LOAD-CSV method duration increases from 1.579s to 534.505s, while the Neo4 j-inport method duration increases from 1.582s to 15.463 s. After comparison, the Neo4j-Import method is found to be significantly more efficient than the LOAD-CSV during the data Import phase. Observing fig. 3(b) shows that the time gap and the data amount of the two methods are positively correlated, and the time gap increases from the initial-0.003 s to 5519.042s, where-0.003 s is the program test error. For actual grid equipment data, the information amount of grid equipment nodes is larger than that in an experiment, and a relationship between the nodes needs to be established, so that an efficient data importing method is very important. Neo4j-Import is selected in this patent to complete the data Import work of Neo4j graph database. And after the importing work is finished, the construction of the power grid equipment knowledge graph is finished. The electric power knowledge graph built by the method has no error data, semi-automatic construction of the knowledge graph can be realized, a large amount of manpower resources can be saved, and the working efficiency is improved.
The sequence numbers in the above embodiments are merely for description, and do not represent the sequence of the assembly or the use of the components.
The above description is only exemplary of the present invention and should not be taken as limiting the invention, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (4)
1. The construction method of the big data knowledge graph of the smart power grid is characterized by comprising the following steps:
acquiring power grid equipment information and the connection relation between the power grid equipment from the power grid equipment by Neo4 j;
based on a historical topology database, carrying out data cleaning on the obtained power grid equipment information;
modeling the cleaned data according to the connection relation between the power grid devices;
and carrying out topology search analysis on the established model to obtain a topological relation knowledge graph between target devices.
2. The method for constructing the big data knowledge graph of the smart grid according to claim 1, wherein the topology search analysis of the established model comprises:
and carrying out topology search analysis on the established model based on a search algorithm.
3. The construction method of the smart grid big data knowledge graph according to claim 1, wherein the search algorithm comprises:
graph traversal algorithms, spanning tree algorithms, and path search algorithms.
4. The method for constructing the big data knowledge graph of the smart grid according to claim 1, wherein the step of performing topology search analysis on the established model to obtain the topological relation knowledge graph between the target devices comprises the following steps:
determining an initial storage node Vs where a target device is located;
traversing the nodes Vt adjacent to the initial storage node Vs in the established model;
traversing nodes Vn adjacent to the nodes Vt in the established model;
and if the target storage node V is traversed in the traversing process, outputting a path between the initial storage node Vs and the target storage node V according to the traversing process.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010375821.9A CN111737483A (en) | 2020-05-04 | 2020-05-04 | Construction method of big data knowledge graph of smart power grid |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010375821.9A CN111737483A (en) | 2020-05-04 | 2020-05-04 | Construction method of big data knowledge graph of smart power grid |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111737483A true CN111737483A (en) | 2020-10-02 |
Family
ID=72646964
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010375821.9A Pending CN111737483A (en) | 2020-05-04 | 2020-05-04 | Construction method of big data knowledge graph of smart power grid |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111737483A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112418758A (en) * | 2020-11-17 | 2021-02-26 | 国网电子商务有限公司 | Method and system for intelligently recommending carriers to shippers |
| CN113344638A (en) * | 2021-06-29 | 2021-09-03 | 云南电网有限责任公司信息中心 | Hypergraph-based power grid user group portrait construction method and device |
| CN113704489A (en) * | 2021-08-17 | 2021-11-26 | 深圳供电局有限公司 | Power grid dispatching knowledge graph forming method and system |
| CN114996974A (en) * | 2022-07-18 | 2022-09-02 | 南方电网科学研究院有限责任公司 | Power grid topology analysis method based on knowledge graph |
| CN117009548A (en) * | 2023-08-02 | 2023-11-07 | 广东立升科技有限公司 | Knowledge graph supervision system based on secret equipment maintenance |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018082364A1 (en) * | 2016-11-02 | 2018-05-11 | 全球能源互联网研究院有限公司 | Method and device for analyzing power-off range of power distribution network, and computer storage medium |
| CN108228724A (en) * | 2017-12-11 | 2018-06-29 | 厦门亿力吉奥信息科技有限公司 | Power grid GIS topology analyzing method and storage medium based on chart database |
| CN109840270A (en) * | 2018-12-23 | 2019-06-04 | 国网浙江省电力有限公司 | A kind of grid equipment approaches to IM based on Neo4j |
| CN110704577A (en) * | 2019-10-10 | 2020-01-17 | 国家电网公司华中分部 | Method and system for searching power grid scheduling data |
| CN110727741A (en) * | 2019-09-29 | 2020-01-24 | 全球能源互联网研究院有限公司 | Knowledge graph construction method and system of power system |
-
2020
- 2020-05-04 CN CN202010375821.9A patent/CN111737483A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018082364A1 (en) * | 2016-11-02 | 2018-05-11 | 全球能源互联网研究院有限公司 | Method and device for analyzing power-off range of power distribution network, and computer storage medium |
| CN108228724A (en) * | 2017-12-11 | 2018-06-29 | 厦门亿力吉奥信息科技有限公司 | Power grid GIS topology analyzing method and storage medium based on chart database |
| CN109840270A (en) * | 2018-12-23 | 2019-06-04 | 国网浙江省电力有限公司 | A kind of grid equipment approaches to IM based on Neo4j |
| CN110727741A (en) * | 2019-09-29 | 2020-01-24 | 全球能源互联网研究院有限公司 | Knowledge graph construction method and system of power system |
| CN110704577A (en) * | 2019-10-10 | 2020-01-17 | 国家电网公司华中分部 | Method and system for searching power grid scheduling data |
Non-Patent Citations (1)
| Title |
|---|
| 马义松 等: "基于Neo4j的电力大数据建模及分析", 《电工电能新技术》, vol. 35, no. 02, pages 24 - 30 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112418758A (en) * | 2020-11-17 | 2021-02-26 | 国网电子商务有限公司 | Method and system for intelligently recommending carriers to shippers |
| CN113344638A (en) * | 2021-06-29 | 2021-09-03 | 云南电网有限责任公司信息中心 | Hypergraph-based power grid user group portrait construction method and device |
| CN113344638B (en) * | 2021-06-29 | 2022-05-24 | 云南电网有限责任公司信息中心 | Power grid user group portrait construction method and device based on hypergraph |
| CN113704489A (en) * | 2021-08-17 | 2021-11-26 | 深圳供电局有限公司 | Power grid dispatching knowledge graph forming method and system |
| CN114996974A (en) * | 2022-07-18 | 2022-09-02 | 南方电网科学研究院有限责任公司 | Power grid topology analysis method based on knowledge graph |
| CN117009548A (en) * | 2023-08-02 | 2023-11-07 | 广东立升科技有限公司 | Knowledge graph supervision system based on secret equipment maintenance |
| CN117009548B (en) * | 2023-08-02 | 2023-12-26 | 广东立升科技有限公司 | Knowledge graph supervision system based on secret equipment maintenance |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111737483A (en) | Construction method of big data knowledge graph of smart power grid | |
| CN109669934B (en) | Data warehouse system suitable for electric power customer service and construction method thereof | |
| CN109582667A (en) | A kind of multiple database mixing storage method and system based on power regulation big data | |
| JP4397978B2 (en) | Binding ordering method using concentration | |
| CN106503243B (en) | Electric power big data querying method based on HBase secondary index | |
| CN108228724A (en) | Power grid GIS topology analyzing method and storage medium based on chart database | |
| CN104700190A (en) | Method and device for matching item and professionals | |
| CN107832876A (en) | Subregion peak load Forecasting Methodology based on MapReduce frameworks | |
| Zhang et al. | Role of fractal dimension in random walks on scale-free networks | |
| CN109325062A (en) | A kind of data dependence method for digging and system based on distributed computing | |
| Mo et al. | Network simplification and k-terminal reliability evaluation of sensor-cloud systems | |
| CN116662412B (en) | Data mining method for big data of power grid distribution and utilization | |
| CN106528849B (en) | Complete history record-oriented graph query overhead method | |
| Ye et al. | Mining resource community and resource role network from event logs | |
| CN114691958A (en) | Community retrieval method based on user geographical location diversity | |
| CN111159178B (en) | Data map path navigation method based on big data SQL analysis | |
| Imawan et al. | Road traffic analytic query processing based on a timeline modeling | |
| CN109446294B (en) | Parallel mutual subspace Skyline query method | |
| Abul-Basher et al. | TGDB: towards a benchmark for graph databases | |
| CN111259062A (en) | Method and device capable of ensuring sequence of result sets of full-table query statements of distributed database | |
| CN112257950A (en) | Trade path configuration method applied to power market and computer-readable storage medium | |
| CN110517009A (en) | Real-time common layer building method, device and server | |
| Wang et al. | Fast Cycle Structure Detection for Power Grids Based on Graph Computing | |
| CN105302921B (en) | Map class date storage method and system | |
| Nakanishi et al. | Approaching the interconnection of heterogeneous knowledge bases on a knowledge grid |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |