CN111209288A - Data processing system and method for unified power grid space-time information service - Google Patents
Data processing system and method for unified power grid space-time information service Download PDFInfo
- Publication number
- CN111209288A CN111209288A CN201911359443.9A CN201911359443A CN111209288A CN 111209288 A CN111209288 A CN 111209288A CN 201911359443 A CN201911359443 A CN 201911359443A CN 111209288 A CN111209288 A CN 111209288A
- Authority
- CN
- China
- Prior art keywords
- data
- processing
- data processing
- subprogram
- power grid
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012545 processing Methods 0.000 title claims abstract description 140
- 238000000034 method Methods 0.000 title claims abstract description 16
- 230000008569 process Effects 0.000 claims abstract description 9
- 238000005192 partition Methods 0.000 claims abstract description 8
- 238000003780 insertion Methods 0.000 claims description 19
- 230000037431 insertion Effects 0.000 claims description 19
- 230000002123 temporal effect Effects 0.000 claims description 16
- 238000003672 processing method Methods 0.000 claims description 10
- 238000010276 construction Methods 0.000 claims description 8
- 230000004048 modification Effects 0.000 claims description 5
- 238000012986 modification Methods 0.000 claims description 5
- 238000012217 deletion Methods 0.000 claims description 4
- 230000037430 deletion Effects 0.000 claims description 4
- 230000008859 change Effects 0.000 claims description 3
- 230000007246 mechanism Effects 0.000 abstract description 3
- 238000011160 research Methods 0.000 description 3
- 238000013500 data storage Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000005215 recombination Methods 0.000 description 1
- 230000006798 recombination Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2379—Updates performed during online database operations; commit processing
- G06F16/2386—Bulk updating operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data processing system and a method for unified service of power grid temporal-spatial information, which comprises a power grid GIS platform Oracle database, a data reading module and a data processing module, the data reading module reads the data of the Oracle database of the power grid GIS platform as required, the data processing module processes the data which needs to be read by the data reading module, the main program is responsible for polling a required processing table, recording synchronization success and failure logs and a database log table for recording the latest STATE ID of each device synchronization, and then the main program starts the subprocess, and triggering an increment updating mechanism, wherein the subprogram is used for synchronizing a single device type graph table or an attribute table of a single partition, and receiving command line parameter partition ID, starting STATE ID, ending STATE ID and table name, so that the accuracy and the efficiency of unified service data processing of the time-space information of the power grid are guaranteed.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a data processing system and a data processing method for unified power grid spatio-temporal information service.
Background
Based on the existing research achievements such as a digital power grid, a time-space GIS and the like, open-source and commercial time-space basic software is fused, unified time-space data storage logic is researched, and lossless format conversion of different geographic information data is realized; the method comprises the steps of researching and establishing a unified time-space reference of a power grid and a unified time reference of unified multi-scale basic geographic information data by adopting a CGCS2000 geodetic coordinate system authorized and organized by the current State mapping administration, and ensuring the consistency of space and time among different scale data; the method is characterized in that the functional position of the power grid is taken as a reference, the relevant fusion of production, GIS, scheduling, metering, marketing, meteorological service domain data and the like is realized, a unified time-space data storage model of the power grid is constructed by data expansion, addition of space, time and attribute identification and data recombination on the basis of an established basic geographic information database, a time-space data retrieval and query mechanism is established, and the time-space display application effect is improved.
The unified space-time service specification of the power grid is formulated according to the OGC specification, decoupling, reconstruction and multiplexing of the existing functions are completed according to the specification, key technical researches of a space data distributed cache technology, a second-level real-time reconstruction technology of space-time topology, a space data pre-fetching mode, a tile data organization method based on space-time clustering, multi-space-time multi-version local graph management service and power grid space-time information analysis are completed, and the technical bottleneck of the space-time unified service is solved. Finally, the space-time service transformation of the existing space service is realized, and a unified, standard and complete space-time service basic platform is formed.
In order to ensure data processing of the power grid space-time information unified service, exploring a proper power grid space-time information unified service data processing method becomes a research subject which is urgently needed to be solved at present, and therefore a data processing system and method of the power grid space-time information unified service are provided.
Disclosure of Invention
The technical problem to be solved by the invention is to overcome the defects of the prior art and provide a data processing system and a data processing method for unified service of time-space information of a power grid.
In order to achieve the purpose, the invention provides the following technical scheme:
the utility model provides a data processing system of unified service of electric wire netting space-time information, includes electric wire netting GIS platform Oracle database, data reading module, data processing module, data reading module reads the data of electric wire netting GIS platform Oracle database as required, data processing module handles the data that data reading module needs read.
Preferably, the data reading module is a power grid GIS user.
Preferably, the data processing module is composed of a main program and a subprogram.
A data processing method for unified power grid spatio-temporal information service comprises the following steps:
step 1: firstly, a data reading module reads data in an Oracle database of a power grid GIS platform, the data reading module transmits the data to a data processing module, and the data processing module processes the data;
step 2: when the data processing module processes data, firstly, a main program prompts an access user to input a partition list, then, processing type initialization and updating are input, when an updating processing type is selected, updating time needs to be input, finally, the main program starts a background thread, polling is carried out until the updating time, when the initialization processing type is selected, the main program firstly obtains a table name, then obtains the latest stste ID from an Oracle database, and finally, starts a subprogram;
and step 3: the subprogram starts to work, firstly the subprogram is initialized, a file is configured, command line parameters are received, a latest device command which is cleaned and installed in a target library is executed in the target library, then construction query is carried out, table field query and query SQL splicing are respectively carried out in the construction query, then core processing is carried out on the data, when the core processing is carried out, traversal recording is carried out on the data firstly, then record change is carried out on the data, namely new creation, modification and deletion processing are carried out on the data, then batch insertion and batch update processing are carried out on the data, traversal recording is carried out on the data, and finally the subprogram work is finished.
Preferably, when the subprogram in step 3 needs to newly create the data, the subprogram firstly performs spatial index field processing, switching device center point processing, record insertion data processing, temporal field data processing, VER flag processing on the data, and performs batch insertion and batch update processing after the data is newly created.
Preferably, when the subprogram in step 3 needs to modify the data, the subprogram performs spatial index field processing, switching device center point processing, record insertion data processing, temporal field data processing, VER FLAGS processing, record update data processing, and temporal field processing on the data, and performs batch insertion and batch update processing after the data modification processing.
Preferably, in step 3, when the subprogram needs to delete the data, the subprogram firstly performs VERFLAGS processing, record update data processing, and temporal field processing on the data, and performs batch update processing after the data deletion processing.
Compared with the prior art, the invention has the beneficial effects that:
the invention relates to a data processing system and a data processing method for a power grid time-space information unified service, which are characterized in that a main program is responsible for polling a required processing table, recording a synchronization success and failure log and a database log table to record the latest STATE ID of each device synchronization, then the main program starts a subprogram and is triggered by an increment updating mechanism, the subprogram is used for synchronizing a single device type graph table or an attribute table of a single partition and receiving command line parameter partition IDs, starting STATE IDs, ending STATE IDs and table names, and therefore the accuracy and the efficiency of the power grid time-space information unified service data processing are guaranteed.
Drawings
FIG. 1 is a schematic structural view of the present invention;
FIG. 2 is a schematic structural diagram of a main program of the present invention;
FIG. 3 is a schematic diagram of the subroutine of the present invention.
Reference numbers in the figures: 1. a power grid GIS platform Oracle database; 2. a data reading module; 3. a data processing module; 4. a main program; 5. and a subroutine.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1-3, the present invention provides a technical solution:
a data processing system for unified service of power grid temporal-spatial information comprises a power grid GIS platform Oracle database 1, a data reading module 2 and a data processing module 3, wherein the data reading module 2 reads data of the power grid GIS platform Oracle database 1 as required, the data processing module 3 processes the data which needs to be read by the data reading module 2, the data reading module 2 is a power grid GIS user, and the data processing module 3 is composed of a main program 4 and a subprogram 5.
A data processing method for unified power grid spatio-temporal information service comprises the following steps:
step 1: firstly, a data reading module 2 reads data in an Oracle database 1 of a power grid GIS platform, the data reading module 2 transmits the data to a data processing module 3, and the data processing module 3 processes the data;
step 2: when the data processing module 3 processes data, firstly, the main program 4 prompts an access user to input a partition list, then, the processing type is input for initialization and updating, when the updating processing type is selected, the updating time needs to be input, finally, the main program 4 starts a background thread, polling is carried out until the updating time, when the initialization processing type is selected, the main program 4 firstly obtains a table name, then obtains the latest stste ID from an Oracle database, and finally, starts the subprogram 5;
and step 3: the subprogram 5 starts to work, firstly the subprogram 5 is initialized, a file is configured, command line parameters are received, a latest device command which is cleaned and installed in a target library is executed in the target library, then construction query is carried out, table field query and query SQL splicing are respectively carried out in the construction query, then core processing is carried out on data, during the core processing, traversal recording is carried out on the data firstly, then record change is carried out on the data, namely new creation, modification and deletion processing is carried out on the data, then batch insertion and batch update processing are carried out on the data, traversal recording is carried out on the data, and finally the subprogram work is finished; when the data needs to be newly built, the subprogram 5 firstly performs spatial index field processing, switching equipment central point processing, record insertion data processing, temporal field data processing and VER FLAGS processing on the data, and performs batch insertion and batch update processing after the data is newly built; step 3, when the subprogram 5 needs to modify the data, the subprogram 5 firstly performs spatial index field processing, switching equipment central point processing, record insertion data processing, temporal field data processing, VER FLAGS processing, record update data processing and temporal field processing on the data, and performs batch insertion and batch update processing after the data is modified; in step 3, when the subprogram 5 needs to delete the data, the subprogram 5 firstly performs VER flag gs processing, record update data processing and temporal field processing on the data, and performs batch update processing after the data is deleted.
The working principle is as follows: when the data reading module 2 reads data in an Oracle database 1 of a power grid GIS platform, the data reading module 2 transmits the data to a data processing module 3, when the data processing module 3 processes the data, firstly a main program 4 prompts an access user to input a partition list, then processing type initialization and updating are input, when the updating processing type is selected, updating time needs to be input, finally the main program 4 starts a background thread, polling is carried out until the updating time, when the initialization processing type is selected, the main program 4 firstly obtains a table name, then obtains the latest stste ID from the Oracle database, and finally starts a subprogram 5; when the subprogram 5 works, firstly, initializing, configuring files and receiving command line parameters, simultaneously executing a command of cleaning and latest equipment installed in a target library, then, carrying out construction query, wherein the construction query comprises table field query and query SQL splicing, and then, carrying out core processing on data; when data needs to be newly built, the subprogram 5 firstly performs spatial index field processing, switching equipment central point processing, record insertion data processing, temporal field data processing and VERFLAGS processing on the data, and performs batch insertion and batch update processing after the data is newly built; when data needs to be modified, the subprogram 5 firstly performs spatial index field processing, switching device central point processing, record insertion data processing, temporal field data processing, VER FLAGS processing, record update data processing and temporal field processing on the data, and performs batch insertion and batch update processing after the data is modified; when the subprogram 5 needs to delete the data, the subprogram 5 firstly performs VER FLAGS processing, record updating data processing and temporal field processing on the data, and performs batch updating processing after the data is deleted.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (7)
1. The utility model provides a data processing system of unified service of electric wire netting space-time information, its characterized in that, includes electric wire netting GIS platform Oracle database (1), data reading module (2), data processing module (3), data reading module (2) reads the data of electric wire netting GIS platform Oracle database (1) as required, data processing module (3) are handled the data that data reading module (2) need read.
2. The data processing system for unified services of grid spatio-temporal information according to claim 1, characterized in that: and the data reading module (2) is a power grid GIS user.
3. The data processing system for unified services of grid spatio-temporal information according to claim 1, characterized in that: the data processing module (3) is composed of a main program (4) and a subprogram (5).
4. A data processing method for unified power grid spatio-temporal information service is characterized by comprising the following steps:
step 1: firstly, a data reading module (2) reads data in an Oracle database (1) of a power grid GIS platform, the data reading module (2) transmits the data to a data processing module (3), and the data processing module (3) processes the data;
step 2: when the data processing module (3) processes data, firstly, a main program (4) prompts an access user to input a partition list, then, processing type initialization and updating are input, when an updating processing type is selected, updating time needs to be input, finally, the main program (4) starts a background thread, polling is carried out until the updating time, when the initialization processing type is selected, the main program (4) firstly obtains a table name, then obtains the latest stste ID from an Oracle database, and finally, starts a subprogram (5);
and step 3: the subprogram (5) starts to work, firstly the subprogram (5) is initialized, a file is configured, command line parameters are received, meanwhile, a latest device command which is cleaned and installed in a target library is executed in the target library, then construction query is carried out, table field query and query SQL splicing are respectively carried out in the construction query, then core processing is carried out on the data, during the core processing, traversal recording is carried out on the data, then record change is carried out on the data, namely, new creation, modification and deletion processing is carried out on the data, then batch insertion and batch update processing are carried out on the data, traversal recording is carried out on the data, and finally the subprogram work is finished.
5. The data processing method of the unified power grid spatio-temporal information service as claimed in claim 4, wherein: and 3, when the subprogram (5) needs to newly create the data, the subprogram (5) firstly performs spatial index field processing, switching equipment center point processing, record insertion data processing, temporal field data processing and VER FLAGS processing on the data, and performs batch insertion and batch update processing after the data is newly created.
6. The data processing method of the unified power grid spatio-temporal information service as claimed in claim 4, wherein: and 3, when the subprogram (5) needs to modify the data, the subprogram (5) firstly performs spatial index field processing, switching equipment center point processing, record insertion data processing, temporal field data processing, VER FLAGS processing, record updating data processing and temporal field processing on the data, and performs batch insertion and batch updating processing after the data is modified.
7. The data processing method for the unified power grid spatio-temporal information service as claimed in claim 1, wherein: and 3, when the subprogram (5) needs to delete the data, the subprogram (5) firstly performs VER FLAGS processing, record updating data processing and temporal field processing on the data, and performs batch updating processing after the data is deleted.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911359443.9A CN111209288A (en) | 2019-12-25 | 2019-12-25 | Data processing system and method for unified power grid space-time information service |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911359443.9A CN111209288A (en) | 2019-12-25 | 2019-12-25 | Data processing system and method for unified power grid space-time information service |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111209288A true CN111209288A (en) | 2020-05-29 |
Family
ID=70786342
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911359443.9A Pending CN111209288A (en) | 2019-12-25 | 2019-12-25 | Data processing system and method for unified power grid space-time information service |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111209288A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113422827A (en) * | 2021-06-23 | 2021-09-21 | 南方电网数字电网研究院有限公司 | Real-time synchronization integration system of comprehensive data of power grid based on streaming technology |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130326388A1 (en) * | 2012-05-31 | 2013-12-05 | State Grid Corporation Of China | Power grid visualization system and method based on three-dimensional gis technology |
| CN103631945A (en) * | 2013-12-10 | 2014-03-12 | 国家电网公司 | Power network GIS platform based on local buffer technology |
| CN108763256A (en) * | 2018-03-30 | 2018-11-06 | 云南电网有限责任公司信息中心 | A kind of electric power GIS Temporal Model construction method based on Dike-pond modes |
-
2019
- 2019-12-25 CN CN201911359443.9A patent/CN111209288A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130326388A1 (en) * | 2012-05-31 | 2013-12-05 | State Grid Corporation Of China | Power grid visualization system and method based on three-dimensional gis technology |
| CN103631945A (en) * | 2013-12-10 | 2014-03-12 | 国家电网公司 | Power network GIS platform based on local buffer technology |
| CN108763256A (en) * | 2018-03-30 | 2018-11-06 | 云南电网有限责任公司信息中心 | A kind of electric power GIS Temporal Model construction method based on Dike-pond modes |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113422827A (en) * | 2021-06-23 | 2021-09-21 | 南方电网数字电网研究院有限公司 | Real-time synchronization integration system of comprehensive data of power grid based on streaming technology |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Abuhashim et al. | Smart contract designs on blockchain applications | |
| CN107038162B (en) | Real-time data query method and system based on database log | |
| CN102023983B (en) | Managing method of statistical space-time database | |
| CN104160394B (en) | Scalable analysis platform for semi-structured data | |
| CN106951694B (en) | Adaptive modeling method for online monitoring system of secondary equipment of power system | |
| CN109726174A (en) | Data archiving method, system, equipment and storage medium | |
| CN109241161A (en) | A kind of meteorological data management method | |
| WO2023087673A1 (en) | Hierarchical data retrieval method and apparatus, and device | |
| CN109840270A (en) | A kind of grid equipment approaches to IM based on Neo4j | |
| CN110196885B (en) | Cloud distributed real-time database system | |
| CN103020204A (en) | Method and system for carrying out multi-dimensional regional inquiry on distribution type sequence table | |
| CN107943831B (en) | HBase-based power grid historical data centralized storage method | |
| CN108595664A (en) | A kind of agricultural data monitoring method under hadoop environment | |
| CN102855332A (en) | Graphic configuration management database based on graphic database | |
| CN113434557B (en) | Method, device, equipment and storage medium for querying range of label data | |
| CN104035956A (en) | Time-series data storage method based on distributive column storage | |
| CN112286941A (en) | Big data synchronization method and device based on Binlog + HBase + Hive | |
| CN104572856A (en) | Converged storage method of service source data | |
| CN110941617A (en) | Power grid GIS data storage method based on version control | |
| CN111209288A (en) | Data processing system and method for unified power grid space-time information service | |
| CN106326040A (en) | Method and device for managing snapshot metadata | |
| CN114398567A (en) | Multi-time-state power grid topological graph data management method and device and computer equipment | |
| CN102724301B (en) | Cloud database system and method and equipment for reading and writing cloud data | |
| CN111427964A (en) | Industrial cloud data storage model for running timestamp | |
| CN112698819A (en) | Method, device and storage medium for designing tree-oriented object programming program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200529 |