CN104111945A - Storage management method and system for massive power-quality data - Google Patents

Storage management method and system for massive power-quality data Download PDF

Info

Publication number
CN104111945A
CN104111945A CN201310137243.5A CN201310137243A CN104111945A CN 104111945 A CN104111945 A CN 104111945A CN 201310137243 A CN201310137243 A CN 201310137243A CN 104111945 A CN104111945 A CN 104111945A
Authority
CN
China
Prior art keywords
data
communication server
management method
oracle database
oracle
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201310137243.5A
Other languages
Chinese (zh)
Inventor
夏锐
唐新红
熊耀明
李权添
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
LINBO TECHNOLOGY GROUP Co Ltd
Original Assignee
LINBO TECHNOLOGY GROUP Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by LINBO TECHNOLOGY GROUP Co Ltd filed Critical LINBO TECHNOLOGY GROUP Co Ltd
Priority to CN201310137243.5A priority Critical patent/CN104111945A/en
Publication of CN104111945A publication Critical patent/CN104111945A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/06Energy or water supply

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Economics (AREA)
  • General Physics & Mathematics (AREA)
  • Water Supply & Treatment (AREA)
  • General Health & Medical Sciences (AREA)
  • Primary Health Care (AREA)
  • Strategic Management (AREA)
  • Tourism & Hospitality (AREA)
  • Human Resources & Organizations (AREA)
  • General Business, Economics & Management (AREA)
  • Marketing (AREA)
  • Public Health (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Supply And Distribution Of Alternating Current (AREA)

Abstract

The invention discloses a storage management method and system for massive power-quality data. The storage management method includes steps of (1) acquiring original monitoring data via a power quality monitoring terminal; (2) converting and compressing the original monitoring data via a communication server; (3) storing compressed data blocks into an ORACLE database; (4) reading the data blocks in the ORACLE database and displaying the same in a memory table manner after decompressing and analyzing when application services need to access the monitoring data in the ORACLE database, accessing the memory table via PL/SQL (program language/structured query language) query statements or the stored procedures, and finally realizing transparent access to the storage data in the ORACLE database by the application services. The storage management method and system has the advantages that the problem that access to the massive power-quality data occupies too many storage spaces and is poor in performance in the prior art is solved, storage space can be saved by more than 70%, and hardware cost is saved for users. Meanwhile, access response performance of the application services is improved, and the storage management method and system is the massive-data storage technology efficient and easy to use.

Description

A kind of memory management method of magnanimity power quality data and system
Technical field
The present invention discloses a kind of memory management method of data, particularly a kind of memory management method of magnanimity power quality data and system, be mainly used in quality of power supply Mass Data Management field, the Mass Data Management of general-purpose system also had to great practicality simultaneously.
Background technology
The quality of power supply is to weigh a major criterion of electrical network electric energy transmitting quality, and to the stabilization of power grids and safety and economic operation, accuracy and the serviceable life of line loss and residential electricity consumption and electric energy measuring equipment have a direct impact.In order to detect the quality of power supply of reflection electrical network transmission, nearly ten years, there is automatically monitoring the electric energy quality monitoring terminal equipment of power network line state.These equipment are except self showing in real time the function of current monitoring point power grid quality state, Real-time Alarm, communication interface flexibly is also provided, can carry out communication with outer computer, the data upload that oneself is monitored is stored in external computer system, thereby can provide the functions such as more friendly powerful the whole network comprehensive statistics analysis, alarm, historic state inquiry, further strengthened the convenience of data monitoring, practicality, has promoted the development of electric power event.
In recent years, the quantity of being disposed in network system by its power quality monitoring device is more and more, the electrical network department rank of paying close attention to the quality of power supply is also more and more higher, by original counties and cities' one-level, rise to province, and even the whole network is carried out unified electric energy quality monitoring management, cause the data volume of needs storage increasing, taking 1000 monitoring points as example, the data volume of 1 year just at least exceedes 1T, this not only needs the input of high disk array storage hardware equipment, make data maintenance get up more and more difficult simultaneously, system responses performance is more and more slower, bring very big inconvenience to routine work.
Summary of the invention
Drop into more for the above-mentioned power quality monitoring device of the prior art of mentioning, shortcoming difficult in maintenance, the invention provides a kind of memory management method and system of new magnanimity power quality data, it is by carrying out Binary Conversion compression storage to original electric energy quality monitoring data, and use ORACLE memory table to realize the transparent access to database, relate at present quality of power supply mass data access to take storage space excessive to solve, the problem that access performance is not good, can save more than 70% data space, in saving hardware cost for user, also improved the access response performance of applied business, it is a kind of mass data storage administrative skill of ease-of-use and high efficiency.
The technical scheme that the present invention solves its technical matters employing is: a kind of memory management method of magnanimity power quality data, this memory management method comprises the steps:
(1), electric energy quality monitoring terminal gathers primary monitoring data;
(2), primary monitoring data is carried out conversion compression process by communication server;
(3), the data block after compression stores ORACLE database into;
(4), in the time that application service need to be accessed the Monitoring Data in ORACLE database, data block in ORACLE database is read after decompress(ion) is resolved and is presented with the form of memory table, by PL/SQL query statement or storing process access memory table, thereby realize application service to storing the transparent access of data in ORACLE.
A storage management system for magnanimity power quality data, this system comprises electric energy quality monitoring terminal, communication server, ORACLE database and application service access modules, electric energy quality monitoring terminal is used for gathering primary monitoring data; Communication server is used for receiving primary monitoring data, and it is carried out to conversion compression process; ORACLE database is used for storing data block; Application service access modules is used for the data block of ORACLE database to read, and presents with the form of memory table after decompress(ion) is resolved.
The technical scheme that the present invention solves its technical matters employing further comprises:
When described primary monitoring data is carried out conversion compression process by communication server, convert primary monitoring data to binary data, recompress into GZIP form.
The primary monitoring data that described electric energy quality monitor collection is come, deposits in volatile data base, and ephemeral data stock only has data on the same day, empties data task on non-same day in 0 execution every day.
Described communication server is provided with more than one, forms communication server group, each communication server equally loaded.
When described data decompression conversion process, data block is from ORACLE database is read, use the practical bag of UTL_COMPRESS of ORACLE to carry out GZIP decompress(ion), then by PL/SQL program segment, binary data blocks is resolved to ORACLE object set, being converted to after object set, is memory table by using TABLE construction of function, for PL/SQL storing process or SQL statement transparent access.
Described communication server is provided with more than one, forms communication server group, each communication server equally loaded.
When described communication server carries out conversion compression process to primary monitoring data, convert primary monitoring data to binary data, recompress into GZIP form.
The invention has the beneficial effects as follows: with the comparison of existing non-conversion Condensed Storage Technique, tool of the present invention has the following advantages: 1. data storage capacity can reduce more than 70%, reduced expensive storage hardware expense expense.2. the performance that has improved data query statistics, more obvious to relating to the statistical query of long period, the query statistic response time can improve more than 5 times.3. database volume diminishes, and has improved the convenience of database daily servicing.
Below in conjunction with the drawings and specific embodiments, the present invention will be further described.
Brief description of the drawings
Fig. 1 is patent working scheme entirety realization flow figure of the present invention.
Fig. 2 is patent data Transpression realization flow figure of the present invention.
Fig. 3 is patent data decompress(ion) conversion realization flow figure of the present invention.
Embodiment
The present embodiment is the preferred embodiment for the present invention, and other all its principles are identical with the present embodiment or approximate with basic structure, all within protection domain of the present invention.
The present invention is by the raw data of the quality of power supply of every day is converted to individual data piece, and it is compressed to storage, and then solution data capacity causes too greatly the problem that storage hardware expense is too high, data access performance is low.
Please refer to accompanying drawing 1, the present invention is mainly a kind of memory management method of magnanimity power quality data, and it comprises the steps:
(1), electric energy quality monitoring terminal gathers primary monitoring data, electric energy quality monitoring terminal is installed and is distributed in the monitored circuit of electrical network everywhere, it uploads Monitoring Data after receiving the data acquisition request of communication server;
(2), primary monitoring data is carried out conversion compression process by communication server, in the present embodiment, communication server receives the data of multiple electric energy quality monitoring terminals, when electric energy quality monitoring terminal is more than 500 time, for the good serviceability of keeping system, should use multiple communication servers, form communication server group, each monitoring communication server equally loaded wherein.Please refer to accompanying drawing 2, in the present embodiment, in the time that data-switching compression is processed, the primary monitoring data that electric energy quality monitor collection is come, deposit in volatile data base (in order to ensure that ephemeral data stock only has data on the same day, empty data task on non-same day in 0 execution every day), binary data modular converter is only by log-on data Transpression warehouse-in task clocked flip, and the management and running time of task and operation interval can configure.Twice Runtime interval is shorter, and ORACLE database data and volatile data base latest data are shorter time delay, but can increase the load of server; Twice Runtime interval is longer, can reduce the load of server, but ORACLE database and volatile data base latest data time delay also can be elongated.Parameter value when concrete application can be determined according to the requirement of the data delay degree of the performance of server and business department's permission.While carrying out the conversion of data, data all day of every kind of data category of each equipment are converted to individual data piece, after data-switching completes, recompress into GZIP form, deposit ORACLE database in;
The form of data block refers to following table:
Table one is voltage deviation data block format form, data block is recorded in position 0 reference time, take four bytes, form is YYYYMMDD, then be to record quantity, account for three bytes, then be every and record number, wherein, every record comprises: time, A phase voltage effective value, B phase voltage effective value and C phase voltage effective value, in the present embodiment, time is side-play amount between reference time, occupies three bytes, to calculate second, A phase voltage effective value, B phase voltage effective value and C phase voltage effective value respectively occupy three bytes
Voltage effective value unit: 1/10000V.
Table one: voltage deviation data block format
Table two is unbalanced data block format form, data block is recorded in position 0 reference time, take four bytes, form is YYYYMMDD, then be to record quantity, account for three bytes, then be every and record number, wherein, every record comprises: the time, voltage zero-sequence component, voltage positive-sequence component, voltage negative sequence component, current zero sequence component, electric current positive-sequence component and electric current negative sequence component, in the present embodiment, time is side-play amount between reference time, occupy three bytes, to calculate second, voltage zero-sequence component, voltage positive-sequence component, voltage negative sequence component, current zero sequence component, electric current positive-sequence component and electric current negative sequence component respectively occupy three bytes, each component of voltage unit: 1/10000V, each current component unit: 1/10000A.
Table two: unbalanced data block format
Table three is frequency data block format form, and data block is recorded in position 0 reference time, takies four bytes, form is YYYYMMDD, is then to record quantity, accounts for three bytes, then be every and record number, wherein, every record comprises: time and frequency values, in the present embodiment, time is side-play amount between reference time, occupy three bytes, to calculate second, frequency values unit: 1/10000Hz.
Table three: frequency data block format
Table four is flickering data block format form, and data block is recorded in position 0 reference time, takies four bytes, form is YYYYMMDD, is then to record quantity, accounts for three bytes, then be every and record number, wherein, every record comprises: time, the flickering of A phase, the flickering of B phase and the flickering of C phase, in the present embodiment, time is side-play amount between reference time, occupies three bytes, to calculate second, the flickering of A phase, the flickering of B phase and the flickering of C phase respectively occupy two bytes, flickering value unit: 1/1000.
Table five is harmonic data block format (voltage, the amplitude of electric current, phase place) form, data block is recorded in position 0 reference time, take four bytes, form is YYYYMMDD, then be to record quantity, account for three bytes, then whether there is higher hamonic wave, account for a byte, then be every and record number, wherein, every record comprises: the time, A is 50 subharmonic mutually, B is 50 subharmonic and C 50 subharmonic mutually mutually, in the present embodiment, time is side-play amount between reference time, occupy three bytes, to calculate second, A is 50 subharmonic mutually, B phase 50 subharmonic and the C mutually every subharmonic of 50 subharmonic occupy two bytes, voltage unit: 1/10000V, current unit: 1/10000A, phase unit: 1/100, if there is no higher hamonic wave, these byte positions do not exist.
Table four: flickering data block format
Table five: harmonic data block format (amplitude of voltage, electric current, phase place)
(3), the data block after compression stores ORACLE database into;
(4), in the time that application service need to be accessed the Monitoring Data in ORACLE, binary data blocks in ORACLE is read after decompress(ion) is resolved and is presented with the form of memory table, by PL/SQL query statement or storing process access memory table, thereby realize application service to storing the transparent access of data in ORACLE.In the time of data decompression conversion process, data block is from ORACLE database is read, use the practical bag of UTL_COMPRESS of ORACLE to carry out GZIP decompress(ion), then by PL/SQL program segment, binary data blocks is resolved to ORACLE object set, be converted to after object set, be memory table by using TABLE construction of function, thereby can supply PL/SQL storing process or SQL statement transparent access.
The present invention provides a kind of storage management system of magnanimity power quality data simultaneously, this system mainly comprises electric energy quality monitoring terminal, communication server, ORACLE database and application service access modules, wherein, electric energy quality monitoring terminal is used for gathering primary monitoring data; Communication server is used for receiving primary monitoring data, and it is carried out to conversion compression process, when communication server carries out conversion compression process to primary monitoring data, convert primary monitoring data to binary data, recompress into GZIP form, in the present embodiment, communication server is provided with more than one, form communication server group, each communication server equally loaded; ORACLE database is used for storing data block; Application service access modules is used for the data block of ORACLE database to read, and presents with the form of memory table after decompress(ion) is resolved.
The above; it is only typical case study on implementation of the present invention; not the present invention is done to any pro forma restriction, those skilled in the art utilize the technology contents of above-mentioned announcement to make a little simple modification, equivalent variations or modification, all drop in protection scope of the present invention.

Claims (8)

1. a memory management method for magnanimity power quality data, is characterized in that: described memory management method comprises the steps:
(1), electric energy quality monitoring terminal gathers primary monitoring data;
(2), primary monitoring data is carried out conversion compression process by communication server;
(3), the data block after compression stores ORACLE database into;
(4), in the time that application service need to be accessed the Monitoring Data in ORACLE database, data block in ORACLE database is read after decompress(ion) is resolved and is presented with the form of memory table, by PL/SQL query statement or storing process access memory table, thereby realize application service to storing the transparent access of data in ORACLE.
2. the memory management method of magnanimity power quality data according to claim 1, it is characterized in that: when described primary monitoring data is carried out conversion compression process by communication server, convert primary monitoring data to binary data, recompress into GZIP form.
3. the memory management method of magnanimity power quality data according to claim 1, it is characterized in that: the primary monitoring data that described electric energy quality monitor collection is come, deposit in volatile data base, ephemeral data stock only has data on the same day, empties data task on non-same day in 0 execution every day.
4. the memory management method of magnanimity power quality data according to claim 1, is characterized in that: described communication server is provided with more than one, forms communication server group, each communication server equally loaded.
5. the memory management method of magnanimity power quality data according to claim 1, it is characterized in that: when described data decompression conversion process, data block is from ORACLE database is read, use the practical bag of UTL_COMPRESS of ORACLE to carry out GZIP decompress(ion), then by PL/SQL program segment, binary data blocks is resolved to ORACLE object set, be converted to after object set, be memory table by using TABLE construction of function, for PL/SQL storing process or SQL statement transparent access.
6. a storage management system for magnanimity power quality data, is characterized in that: described system comprises electric energy quality monitoring terminal, communication server, ORACLE database and application service access modules,
Electric energy quality monitoring terminal is used for gathering primary monitoring data;
Communication server is used for receiving primary monitoring data, and it is carried out to conversion compression process;
ORACLE database is used for storing data block;
Application service access modules is used for the data block of ORACLE database to read, and presents with the form of memory table after decompress(ion) is resolved.
7. the storage management system of magnanimity power quality data according to claim 6, is characterized in that: described communication server is provided with more than one, forms communication server group, each communication server equally loaded.
8. the storage management system of magnanimity power quality data according to claim 6, it is characterized in that: when described communication server carries out conversion compression process to primary monitoring data, convert primary monitoring data to binary data, recompress into GZIP form.
CN201310137243.5A 2013-04-19 2013-04-19 Storage management method and system for massive power-quality data Pending CN104111945A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201310137243.5A CN104111945A (en) 2013-04-19 2013-04-19 Storage management method and system for massive power-quality data

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201310137243.5A CN104111945A (en) 2013-04-19 2013-04-19 Storage management method and system for massive power-quality data

Publications (1)

Publication Number Publication Date
CN104111945A true CN104111945A (en) 2014-10-22

Family

ID=51708740

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201310137243.5A Pending CN104111945A (en) 2013-04-19 2013-04-19 Storage management method and system for massive power-quality data

Country Status (1)

Country Link
CN (1) CN104111945A (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105302915A (en) * 2015-12-23 2016-02-03 西安美林数据技术股份有限公司 High-performance data processing system based on memory calculation
CN105894396A (en) * 2016-04-22 2016-08-24 威胜集团有限公司 Metering data storage method for electric energy metering device
CN106250517A (en) * 2016-08-02 2016-12-21 广东电网有限责任公司中山供电局 The storage system and method for regional power grid power consumption data
CN106569063A (en) * 2016-11-02 2017-04-19 海南电力技术研究院 Power quality sampling data processing method and power quality sampling data processing system

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100094803A1 (en) * 2006-12-08 2010-04-15 Satoshi Yamakawa Data management apparatus, method and program
CN102053202A (en) * 2009-11-10 2011-05-11 北京博电新力电力系统仪器有限公司 Smart grid-oriented power quality monitoring system and method
US20110246420A1 (en) * 2008-12-23 2011-10-06 Fenglai Wang Database system based on web application and data management method thereof
CN202735418U (en) * 2012-01-20 2013-02-13 北京特域科技有限公司 Power quality monitoring system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100094803A1 (en) * 2006-12-08 2010-04-15 Satoshi Yamakawa Data management apparatus, method and program
US20110246420A1 (en) * 2008-12-23 2011-10-06 Fenglai Wang Database system based on web application and data management method thereof
CN102053202A (en) * 2009-11-10 2011-05-11 北京博电新力电力系统仪器有限公司 Smart grid-oriented power quality monitoring system and method
CN202735418U (en) * 2012-01-20 2013-02-13 北京特域科技有限公司 Power quality monitoring system

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105302915A (en) * 2015-12-23 2016-02-03 西安美林数据技术股份有限公司 High-performance data processing system based on memory calculation
CN105302915B (en) * 2015-12-23 2019-04-09 美林数据技术股份有限公司 The high-performance data processing system calculated based on memory
CN105894396A (en) * 2016-04-22 2016-08-24 威胜集团有限公司 Metering data storage method for electric energy metering device
CN105894396B (en) * 2016-04-22 2020-05-12 威胜集团有限公司 Metering data storage method of electric energy metering device
CN106250517A (en) * 2016-08-02 2016-12-21 广东电网有限责任公司中山供电局 The storage system and method for regional power grid power consumption data
CN106569063A (en) * 2016-11-02 2017-04-19 海南电力技术研究院 Power quality sampling data processing method and power quality sampling data processing system

Similar Documents

Publication Publication Date Title
CN110022226B (en) Object-oriented data acquisition system and acquisition method
CN101587152B (en) Power quality monitoring system
CN104317800A (en) Hybrid storage system and method for mass intelligent power utilization data
CN1967620A (en) Online visible energy consumption audit management system
CN102254024A (en) Mass data processing system and method
CN103955509A (en) Quick search method for massive electric power metering data
CN102856982B (en) Enterprise energy consumption monitoring and management system
CN105939376A (en) Smart electric meter management system and method based on cloud computing
CN104111945A (en) Storage management method and system for massive power-quality data
CN105513328A (en) Electric meter system based on Wifi cloud services
CN201859537U (en) Multifunctional bandwidth power carrier concentrator
CN114446037A (en) Carbon emission monitoring system and method based on Internet of things technology
CN102929244B (en) A kind of community energy management system and management method
CN205384728U (en) Intelligence concentrator data acquisition device
CN106647467B (en) A kind of electric power data information cloud comprehensive monitoring system
CN102841595A (en) Multi-stage resource scheduling management system
CN110636107A (en) Power grid data interaction system based on 5g communication and interaction method thereof
CN204965718U (en) Photovoltaic intelligent monitoring system
CN110555583A (en) method for uniformly processing wide-area operation data of intelligent power grid dispatching control system
CN203084729U (en) Power quality monitoring system based on distributed infrastructure (Hadoop)
CN103018582A (en) Power quality monitoring system
CN103427491A (en) Automatic collecting and monitoring system for intelligent power grid data
CN106546814A (en) A kind of intelligent low pressure electric quantity monitoring system
CN202362901U (en) Acquisition terminal of special transformer
CN204835712U (en) Electric wire netting electric quantity acquisition platform

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20141022