Summary of the invention
For the deficiencies in the prior art, the invention provides a kind of quality of power supply mass data storage and read method, the present invention is applicable to power quality data storage and the read method of real time database management, power quality data storage mode based on key-value pair, be applicable to data file management and the memory management pattern of quality of power supply real-time data base, be applicable to magnanimity power quality data access management mode.
The object of the invention is to adopt following technical proposals to realize:
Mass data storage and a read method, its improvements are, described method comprises
(1) quality of power supply mass data is adopted to the storage of key-value pair form;
(2) key-value pair is stored hereof in the mode of serializing;
(3) real time data is write fashionablely, locates this data of monitoring point district, by the respective storage areas of pointer position monitor data according to device id;
(4) according to the index classification code that will obtain data, read the corresponding data in track.
Preferably, described step (1) comprises in key-value pair data, and the corresponding value of each key, using power quality monitoring device ID as key; In the data field of measured value value of being put into of this monitoring of equipment index.
Further, the measured value of described monitoring of equipment index comprises voltage deviation, frequency departure, three-phase imbalance, voltage fluctuation and flicker, harmonic wave and a harmonic wave.
Further, in described each value district, the measured value of monitoring is analyzed and encodes by index classification, by calculating side-play amount rapidly locating piece.
Preferably, described step (3) pointer comprises and take 200 milliseconds, 3 seconds, 1 minute and 10 minutes four memory blocks that are data granularity.
Preferably, described step (3) comprises that described real time data data writing is with time stamp, and the difference of the moment point of described time stamp and data field stem is less than 10 minutes, directly by the mistiming, is obtained the skew of data field, can be directly by data write memory, the converging operationJu Hecaozuo that log-on data makes progress simultaneously;
Described real time data data writing is with time stamp, and the difference of the moment point of described time stamp and data field stem is more than or equal to 10 minutes, starts background thread ,Jiang Ci segment data district writing in files.
Preferably, described step (4) comprises that the storing time intervals of data divided by 2 hours, certain the track position on the unit respective file head that each is divided out, and the data block in file is directly pointed in described groove position.
Further, described file is the form of catalogue per year, reads the file in a plurality of catalogues for the period of straddling over year.
Compared with the prior art, beneficial effect of the present invention is:
The present invention is directed to electric energy quality monitoring system, proposed data model, storage administration and mass data access mode based on key-value pair.
In the present invention, key-value pair storage mode cuts into data block by each data file and stores, and to data access in database, is all to using data block as least unit, has improved system access speed.
In the present invention, data file management and access mode have taken into full account the feature of power quality data many indexs, many granularities, and power quality data application demand.
The present invention is applied to electric energy quality monitoring system, can realize storage and the high speed Concurrency Access of large-scale data.
Embodiment
Below in conjunction with accompanying drawing, the specific embodiment of the present invention is described in further detail.
1, as shown in Figure 1, the present invention adopts the storage of key-value pair (key-value) form by power quality data, utilizes key-value pair that each data of monitoring point is encapsulated as to index node, by the form of tree or list, organizes index node, improves data query speed.
Wherein, in key-value pair data, the corresponding value (value) of each key (key).Key is ID, using power quality monitoring device ID as key; The measured value of all monitoring indexes of this equipment (as, voltage deviation, frequency departure, three-phase imbalance, voltage fluctuation and flicker, harmonic wave, a harmonic wave etc.) all in the data field of the value of being put into.In Yi Gezhi district, Monitoring Data is analyzed and encodes by index classification, by calculating side-play amount rapidly locating piece.
Because the operation in monoblock region can only be carried out to value in the real-time storehouse of main flow, the data centralization of all indexs in monitoring point is put into a key-value pair, EMS memory occupation is relatively large, for fear of reading continually monoblock value region, in internal memory, set up the buffer zone of currency, when only district on duty is updated to afterbody, just start the write operation to database.
2, key-value pair is stored hereof in the mode of serializing.Utilize internal memory and universal document system, by calculating fast the method for side-play amount, meet application demand.
As shown in Figure 2, file management mode, comprises file directory, file structure, data structure.File is organization directory per year, and the annual data in each monitoring point are stored according to data granularity classification, and data granularity can be divided into 200 milliseconds, 3 seconds, 1 minute, 10 minutes and 2 hours, and specific design is according to practical situations.The inside of each file, arranged 1 annual data for unit divides ,Liang hour data district by 2 hours according to time sequencing.In two hour data districts, comprise this data field type codes (200 milliseconds, 3 seconds, 1 minute, 10 minutes or 2 hours), data length and each data constantly.Each data constantly, comprises the measured value of each power quality index and the quality of value.
As shown in Figure 3, memory management adopts binary tree mode to manage, and by the lock mechanism of operating system, controls the Concurrency Access to this binary tree, by Thread Pool Technology, completes the operation that in tree, data are write to file.In binary tree, in each node, usining device id as key assignments, is that 200 milliseconds, 3 seconds, 1 minute and 10 minutes granularity datas are divided independently storage area.
3, the data in internal memory tree data structure storage current ten minutes, using the ID of monitoring equipment as key assignments, by pointer, locate the data field of this monitoring point, comprise four pointers, point to respectively and take 200 milliseconds, 3 seconds, 1 minute and 10 minutes four memory blocks that are data granularity.
Wherein, when real time data, write fashionablely, first find the corresponding memory node of monitoring equipment, then find corresponding data field according to data granularity.The data that real time data writes are with time stamp, if the difference of the moment point of this time stamp and data field stem is less than 10 minutes, directly by the mistiming, obtained the skew of data field, can be directly by data write memory, and the converging operationJu Hecaozuo that log-on data makes progress simultaneously.
For example: when writing the data in 15 200ms cycles, start converging operationJu Hecaozuo, generate the data in 3 second cycle; Similarly, then by 3 number of seconds according to cycle polymerization in 1 minute, the like.
Wherein, when real time data, write fashionablely, the data that real time data writes are with time stamp, if the difference of the moment point of this time stamp and data field stem is more than or equal to 10 minutes, start background thread ,Jiang Ci segment data district writing in files.200 milliseconds, 3 seconds, 1 minute, 10 minutes and the storage of 2 hour data file sectional lists, for ease of directly writing data in these 5 file pointers, deposit in the pointer of opening after these 5 files in the memory node of this monitoring equipment.File header be take 2 hours each tracks that are unit for 1 year for recording, and in groove, numerical value is for recording the skew of this unit data field constantly.Find corresponding groove position according to the start time of this batch data, by the afterbody of data writing in files, the skew of this segment data is recorded in corresponding groove position simultaneously.
4, being read as of historical data:
The storing time intervals of historical data was divided by 2 hours, certain the track position on the unit respective file head that each is divided out, and the data block in file is directly pointed in these groove positions.According to the index classification code that will obtain data, can read soon the corresponding data in track data field pointed, as shown in Figure 4 again.File is the form of catalogue per year, is equivalent to read the file in a plurality of catalogues for the period of straddling over year.
The present invention adopts service-oriented technology to provide service interface for client software, so that the secondary development of application program, and the expansion of market demand.Interface used can be encapsulated as two kinds, C bank interface and Web service interface, and C bank interface adopts CORBA service bus, and Web service interface adopts Apache or Web Logical etc.
For guaranteeing that device id has the whole network uniqueness, the URI of IEC61970CIM can be mapped as to device id, and adopt unique mapping ruler.The coding of all kinds of power quality indexs also should have the whole network uniqueness.Adopt service-oriented mode, the user who is the whole network by access service interface provides unified coding.
Finally should be noted that: above embodiment is only in order to illustrate that technical scheme of the present invention is not intended to limit, although the present invention is had been described in detail with reference to above-described embodiment, those of ordinary skill in the field are to be understood that: still can modify or be equal to replacement the specific embodiment of the present invention, and do not depart from any modification of spirit and scope of the invention or be equal to replacement, it all should be encompassed in the middle of claim scope of the present invention.