CN105138278B - A kind of the naked of intelligent substation network message takes inventory method for storing - Google Patents
A kind of the naked of intelligent substation network message takes inventory method for storing Download PDFInfo
- Publication number
- CN105138278B CN105138278B CN201510445719.0A CN201510445719A CN105138278B CN 105138278 B CN105138278 B CN 105138278B CN 201510445719 A CN201510445719 A CN 201510445719A CN 105138278 B CN105138278 B CN 105138278B
- Authority
- CN
- China
- Prior art keywords
- network message
- data block
- message
- network
- type
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention proposes a kind of the naked of intelligent substation network message and takes inventory method for storing, comprises the following steps：Hard disk partition is several data blocks by hard disk partition step；Data block function partiting step；It is respectively the data block that all types of network message distribution matches with its occupied space by network message using its own type as partitioning standards；Memory Allocation step is that the network message of each type distributes a memory；Network message storing step.The balanced safety and stabilization for utilizing hard disk, being conducive to hard-disk system of this storage method.Network message is stored separately by type, reduces the traversal time of later retrieval.Independent memory space design, fault-tolerance and security get a promotion.
Description
Technical field
The invention belongs to technical field of data storage, specifically, are related to a kind of naked disk of intelligent substation network message
Storage method.
Background technology
The network message of intelligent substation generates when being 7 × 24 small, therefore corresponding analysis processing device must in substation
The pattern of uninterrupted operation must be in.In addition, the on-line storage flow of network message is very big, and store function requirement is relatively simple
It is single.Under the conditions of current technology, the storage medium of this mass data is necessarily dependent upon hard disk.In intelligent substation, because
Using networking and total digitalization, electrical equipment is connected with network, and the interaction of electrical equipment is transferred using network message, and most often
There are five the network message types seen：
1) SV network messages, high-frequency uninterruptedly generate, more than account for overall network message 95%；
2) GOOSE network message, low frequency is intermittent to be generated, and network message amount interior running environment of being stood is affected, about
Account for about the 2% of overall network message；
3) MMS network messages, low frequency is intermittent to be generated, network message amount by interior automatic operating environment of standing influenced compared with
Greatly, by accounting for the 2% of overall network message；
4) PTP network messages, low frequency is intermittent to be generated, for pair when, account for the 0.1% of overall network message；
5) other network messages, shake hands between network network message, random network message etc., network message amount are stood interior network
Environment is affected, and accounts for the 0.5% of overall network message.
At present, the various monitoring devices in intelligent substation have following several features when handling network message storage：Mirror
In the magnanimity of network message, inevitably using hard disk as storage medium, and using operating system acquiescence control hard disk
Mode, storage strategy be according to network message reach time, carry out sequential storage；There are two types of storage modes, and one kind is solid
It fixes time.Such as 10 minutes, memory buffer continuously filled 10 minutes network message data, and carried out within every 10 minutes a hard disk and write
Enter operation, while empty memory buffer；Another is fixed size.Such as 10 Mbytes, memory buffer continuously fills 10
Mbytes network message data, and every 10 minutes carry out a hard disk write operation, while empty memory buffer.
The drawbacks of aforesaid way, has：
1st, because depending on the file system of operating system and hard disk used so that network message is actually written into opportunity
It is uncertain, it can not directly judge whether this network message write operation succeeds；Also can once be read by the way that data will be write
Go out verification, but the cost so done is very big；Consequence caused by write-in opportunity does not know is, once system power failure, can not provide true
The data of hard disk are written in fact, and subsequent processing can not carry out；
2nd, the position of write-in can not determine, easily some magnetic tracks of generation hard disk and sector are erasable by high intensity, and remaining
Magnetic track and sector are left unused for a long time, are not only caused the utilization of resources uneven, are also easy to cause hard disk corruptions unavailable, and then are influenced entire
The safety and stabilization of hard-disk system；
3rd, the sequential storage mode that network message uses, network message type hybrid are not distinguished, cause to retrieve together
For PPT when the rare network message of types, cost is the same with the mass network message of retrieval SV types, the big drop of efficiency；
4th, set time formula writes hard disk method and does not account for changes in flow rate, when interior power failure of such as standing is with overhaul of the equipments, may not have
There is any network message to generate, will at this time generate a series of fragment file that length are 0；
5th, fixed size formula writes hard disk method and does not also account for changes in flow rate, will once network message flow is very low in station
Cause even a couple of days a few hours that can not all generate the data of size needed for buffer, at this time in the event of power outage, then
Buffer data are all lost immediately, and security and real-time can not be protected；
6th, using file as basic unit of storage, every time retrieve special time period, particular type of network message when, be required for by
One opens a series of corresponding documents, reads file content to ordered retrieval, cost time are longer again after memory buffer.
The content of the invention
Writing position can not determine that network message is pressed during the present invention to be solved existing substation network packet storage
The sequencing of arrival carries out random storage, it has not been convenient to later retrieval and the wasting skill of irrational utilization memory space
Art problem, it is proposed that a kind of the naked of intelligent substation network message takes inventory method for storing, can solve the above problems.
A kind of the naked of intelligent substation network message takes inventory method for storing, comprises the following steps：
Hard disk partition step, is several data blocks by hard disk partition, and sequence number is numbered with for each data block order；
Data block function partiting step, it is meta data block to select one of data block, for first number of storage file block
According to other data blocks are message data block, for the network message data of storage file block；
It is empty according to shared by volume forecasting different type network message by network message using its own type as partitioning standards
Between size, be respectively the message data block that all types of network messages distribution match with its occupied space；
Memory Allocation step is that the network message of each type distributes a memory；
Network message storing step, comprises the following steps：
(1), network message is written into first to be write into the memory corresponding to its type according to its type, is described
Network message to be written distributes a blocks of files sequence number, and the parameter information of blocks of files corresponding to the blocks of files sequence number is extracted,
Form the metadata of this document block；
(2), the message data block of the type distribution for the network message to be written is found, by first number of the blocks of files
According to write-in into meta data block, the blocks of files is write sequentially in time into the message data block distributed.
Further, in (1), network message is written into first according to it the network message storing step the step of
Type write to in the memory corresponding to its type, which enters after memory is accumulated with pcap forms, memory fills out
Man Hou, first carries out the compression of a gzip form, and adds gzip and cyclic redundancy check, constitutes the gzip_ of gzip forms
buffer。
Further, the network message storing step the step of in (1), blocks of files corresponding to the blocks of files sequence number
Parameter information include at least offset of the blocks of files corresponding to this document block sequence number in entire hard disk, length, network ID, network
Type of message, the timestamp of first frame network message, the timestamp information of last frame network message.
Further, the network message storing step the step of in (2), blocks of files is write sequentially in time to institute
During the message data block of distribution, after the message data block distributed when one fills up, if the type network message institute
The message data block quota of distribution is not finished, and directly application occupies next distributed message data block, and otherwise, covering is most
The message data block of old distribution.
Further, in the hard disk partition step, the capacity of the memory block is using 1G bytes as base unit, each number
It is differed according to the capacity of block for 1~nG, wherein n is positive integer.
Advantageous effect using the invention described above technical solution is：1st, this storage method is according to different type network message
Characteristic distributes data block, distributes number more message data block for big flow network message, mutually should be small capaciated flow network message
Distribute number less message data block, equilibrium is not in that some magnetic tracks of hard disk and sector are wiped by high intensity using hard disk
It writes, and long-term leave unused of remaining magnetic track and sector causes the not available problem of hard disk corruptions, is conducive to the general safety of hard-disk system
And stabilization.
2nd, network message is stored separately by type, reduces the traversal time of later retrieval and the number of access hard disk,
For a small amount of network message, effect is especially apparent.
3rd, independent memory space design, once substation's particular network accident, only influences specific among carrying out
The storage of type network message, remaining type network message is unaffected, and fault-tolerance and security get a promotion.
4th, the real-time tracking formula maintenance mechanism of metadata+data block is established, is preserved respectively in fixed position, it can be rapid
Writing position of the network message in hard disk is positioned, even if power down occurs, recovers correct writing position after also restarting in next time.
Description of the drawings
Fig. 1 is that a kind of naked embodiment hard disk for taking inventory method for storing of intelligent substation network message proposed by the invention is drawn
Separation structure schematic diagram；
Fig. 2 is a kind of naked embodiment flow for taking inventory method for storing of intelligent substation network message proposed by the invention
Figure.
Specific embodiment
The principle and features of the present invention will be described below with reference to the accompanying drawings, and the given examples are served only to explain the present invention, and
It is non-to be used to limit the scope of the present invention.
Embodiment one, the present embodiment proposes a kind of the naked of intelligent substation network message and takes inventory method for storing, including following
Step：
Hard disk partition is several numbers as shown in Figure 1, being hard-disc storage data structure schematic diagram by hard disk partition step
It is respectively BLOCK1, BLOCK2 ... according to block BLOCK, each memory block BLOCK includes several blocks of files F_M, is respectively F_
M1, F_M2 ... are numbered with sequence number, such as 0,1,2,3 ... for each data block and file block sequence.
Data block function partiting step, in each data block, fixed to choose the data block that wherein number is 0 be meta data block,
For the metadata of storage file block, remainder data block is used for the network message data of storage file block, is message data block；It is logical
It crosses and divides data block according to function, realize metadata and stored respectively with network message data, only need to remember meta data block
Position, you can be realized as quick-searching to blocks of files by retrieving metadata.
It is empty according to shared by volume forecasting different type network message by network message using its own type as partitioning standards
Between size, be respectively the number of data blocks that all types of network messages distribution match with its occupied space；By network message
It is stored separately by type, in later retrieval, can targetedly retrieve to store such according to the type of network message
The data block of type reduces the traversal time of later retrieval and the number of access hard disk, for a small amount of network message, effect
It is especially apparent.In addition, according to the characteristic of different type network message distribute data block, for big flow network message distribute number compared with
More message data blocks mutually should be small capaciated flow network message and distribute small numbers of message data block, as SV network messages can be with
More message data blocks are distributed, and PTP network messages can distribute less message data block, realize the conjunction of memory space
Reason utilizes, equilibrium using hard disk, be not in hard disk some magnetic tracks and sector it is erasable by high intensity, and remaining magnetic track and sector
Long-term leave unused causes the not available problem of hard disk corruptions, is conducive to the safety and stabilization of hard-disk system entirety.
Wherein, in the hard disk partition step, the capacity of the data block is using 1G bytes as base unit, each data block
Capacity differ for 1~nG, wherein n is integer.It is differed in size by divide the capacity of message data block, equally facilitated
Can be balanced when being distributed for different types of network message, it avoids when being divided into impartial amount of capacity, if the distribution of each data block
Capacity is too big, causes to cause the wasting of resources during storage few network message of occupied space, if the capacity of each memory block distribution is too small,
When can cause the storage big network message of occupied space again, distribute that the quantity of memory block is excessive for it, increase that retrieval takes asks
Topic.
By being data block, blocks of files by hard disk partition in this method, it is respectively used to store different types of network message
Or the file block message of different content, certain types of network message can only occupy the message data block in default quota, shape
Into independent memory space, a message data block must not store the network message of more than one type, if data volume is bigger
The network message of type can distribute the message data block of greater number, by using independent after its occupied space is estimated
Memory space designs, once substation's particular network accident, only influences the storage of particular type of network message, remaining type network
Message is unaffected, and fault-tolerance and security get a promotion.
Memory Allocation step is that the network message of each type distributes a memory；
Network message storing step, as shown in Fig. 2, comprising the following steps：
S1, network message is written into first is write according to its type into the memory corresponding to its type, be described
Network message to be written distributes a blocks of files sequence number, and the parameter information of blocks of files corresponding to the blocks of files sequence number is extracted,
Form the metadata of this document block；Network message is stored in the form of blocks of files, each blocks of files has unique blocks of files sequence
Number, later data is facilitated to search, is reduced to the abrasion of hard disk traversal time and reduction to hard disk.
S2, the data block distributed for the type of the network message to be written is found, the metadata of the blocks of files is write
Enter in the data block that number is 0, the blocks of files is write sequentially in time into the remainder data block distributed.
As a preferred embodiment, in the step S1 of the network message storing step, net is written into first
Network message is write according to its type into the memory corresponding to its type, which enters after memory with pcap forms
Accumulation after memory fills up, first carries out the compression of a gzip form, and adds gzip and cyclic redundancy check, constitutes gzip lattice
The gzip_buffer of formula.Data encoding based on standardized format, the later-stage secondary for being conducive to data are used, and have bigger
Open and versatility.
Wherein, in the step S1 of the network message storing step, the ginseng of blocks of files corresponding to the blocks of files sequence number
Number information include at least offset of the blocks of files corresponding to this document block sequence number in entire hard disk, length, network ID, network message
Type, the timestamp of first frame network message, the timestamp information of last frame network message.By the way that more than parameter information is carried
Metadata information is taken into, writing position of the network message in hard disk can be positioned rapidly, it, also can be in weight next time even if power down occurs
Recover correct writing position after opening.
In the step S2 of the network message storing step, blocks of files is write sequentially in time to currently being distributed
During message data block, after the message data block distributed when one fills up, if what the type network message was distributed
Message data block quota is not finished, and directly application occupies next message data block ready for its distribution, and otherwise, covering is most
The message data block of old distribution.Storage mode is according to first in first out, Xun Huan storage.
According to the network message that this method stores, single network message or the network message file of some period are being inquired about
When, by way of secondary index, first according to finding the message data block containing objective network message in type, then in first number
According to the data block containing objective network message is searched in block, in a manner of " leapfroging ", achieve the purpose that quick-searching.
The foregoing is merely presently preferred embodiments of the present invention, is not intended to limit the invention, it is all the present invention spirit and
Within principle, any modifications, equivalent replacements and improvements are made should all be included in the protection scope of the present invention.
Claims (4)
1. a kind of the naked of intelligent substation network message takes inventory method for storing, which is characterized in that comprises the following steps：
Hard disk partition step, is several data blocks by hard disk partition, and sequence number is numbered with for each data block order；
Data block function partiting step, it is meta data block to select one of data block, for the metadata of storage file block,
His data block is message data block, for the network message data of storage file block；
By network message using its own type as partitioning standards, according to space shared by volume forecasting different type network message
Size is respectively the message data block that all types of network message distribution matches with its occupied space；
Memory Allocation step is that the network message of each type distributes a memory；
Network message storing step, comprises the following steps：
(1), it is written into network message first to be write into the memory corresponding to its type according to its type, to be described to be written
Enter network message and distribute a blocks of files sequence number, the parameter information of blocks of files corresponding to the blocks of files sequence number is extracted, formed
The metadata of this document block；
(2), the message data block of the type distribution for the network message to be written is found, the metadata of the blocks of files is write
Enter into meta data block, the blocks of files is write sequentially in time into the message data block distributed；
The network message storing step the step of in (1), be written into first network message according to its type write to
In memory corresponding to its type, which enters after memory is accumulated with pcap forms, after memory fills up, first carries out one
The compression of secondary gzip forms, and gzip and cyclic redundancy check are added, constitute the gzip_buffer of gzip forms.
2. the naked of intelligent substation network message according to claim 1 takes inventory method for storing, which is characterized in that in the net
In the step of network packet storage step (1), the parameter information of blocks of files corresponding to the blocks of files sequence number includes at least this document
Offset of the blocks of files corresponding to block sequence number in entire hard disk, length, network ID, network message type, first frame network message
Timestamp, the timestamp information of last frame network message.
3. the naked of intelligent substation network message according to claim 2 takes inventory method for storing, which is characterized in that in the net
In the step of network packet storage step (2), during blocks of files is write sequentially in time to the message data block distributed,
After the message data block distributed when one fills up, if the message data block quota that the type network message is distributed is not used
Complete, directly application occupies next distributed message data block, otherwise, covers the message data block of oldest distribution.
4. naked according to claim 1-3 any one of them intelligent substation network messages takes inventory method for storing, which is characterized in that
In the hard disk partition step, for the capacity of the data block using 1G bytes as base unit, the capacity of each data block is 1~nG
It differs, wherein n is positive integer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510445719.0A CN105138278B (en) | 2015-07-27 | 2015-07-27 | A kind of the naked of intelligent substation network message takes inventory method for storing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510445719.0A CN105138278B (en) | 2015-07-27 | 2015-07-27 | A kind of the naked of intelligent substation network message takes inventory method for storing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105138278A CN105138278A (en) | 2015-12-09 |
| CN105138278B true CN105138278B (en) | 2018-05-25 |
Family
ID=54723639
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510445719.0A Active CN105138278B (en) | 2015-07-27 | 2015-07-27 | A kind of the naked of intelligent substation network message takes inventory method for storing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105138278B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107302505B (en) * | 2017-06-22 | 2019-10-29 | 迈普通信技术股份有限公司 | Manage the method and device of caching |
| CN110932942B (en) * | 2019-12-17 | 2021-08-10 | 山东鲁软数字科技有限公司智慧能源分公司 | Intelligent substation network message recording and analyzing method and system |
| CN111143306B (en) * | 2019-12-17 | 2023-10-31 | 山东鲁软数字科技有限公司智慧能源分公司 | Message optimizing storage method and system for intelligent station network division device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1855881A (en) * | 2005-04-28 | 2006-11-01 | 华为技术有限公司 | Method for dynamically sharing space of memory |
| CN104063187A (en) * | 2014-07-08 | 2014-09-24 | 珠海安联锐视科技股份有限公司 | Hard disk file system suitable for security application and operation method thereof |
| CN104092581A (en) * | 2014-06-30 | 2014-10-08 | 国家电网公司 | Quick message processing method in message recording and analyzing system of intelligent substation |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5268787B2 (en) * | 2009-06-04 | 2013-08-21 | キヤノン株式会社 | Information processing apparatus, control method therefor, and program |
-
2015
- 2015-07-27 CN CN201510445719.0A patent/CN105138278B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1855881A (en) * | 2005-04-28 | 2006-11-01 | 华为技术有限公司 | Method for dynamically sharing space of memory |
| CN104092581A (en) * | 2014-06-30 | 2014-10-08 | 国家电网公司 | Quick message processing method in message recording and analyzing system of intelligent substation |
| CN104063187A (en) * | 2014-07-08 | 2014-09-24 | 珠海安联锐视科技股份有限公司 | Hard disk file system suitable for security application and operation method thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105138278A (en) | 2015-12-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103425663B (en) | Embedded system data base establishment table, deposit record, the method and apparatus for deleting record | |
| CN104731896B (en) | A kind of data processing method and system | |
| CN108197181A (en) | A kind of compression and storage method of time series data, electronic equipment and storage medium | |
| CN104216835B (en) | A kind of method and device for realizing internal memory fusion | |
| CN104362757B (en) | Transformer substation on-line monitoring system based on IEC61850 | |
| CN107239382A (en) | The log processing method and system of a kind of container application | |
| CN107168657A (en) | It is a kind of that cache design method is layered based on the virtual disk that distributed block is stored | |
| CN105138278B (en) | A kind of the naked of intelligent substation network message takes inventory method for storing | |
| CN103870393B (en) | cache management method and system | |
| CN104750859A (en) | Network storing method | |
| CN108399050B (en) | Data processing method and device | |
| CN102158550B (en) | IEC61850-based power quality transient data transmission method | |
| CN109582213A (en) | Data reconstruction method and device, data-storage system | |
| CN106033324A (en) | Data storage method and device | |
| CN101944119A (en) | Real-time event management method for intelligent electronic equipment | |
| CN105389376A (en) | Contiguous block based non-fragmentation multimedia data storage method and system | |
| CN110442646A (en) | A kind of ceph data simultaneous module main side write performance optimization system and method | |
| CN105100716A (en) | Safe memory cell used for network video monitoring and system thereof | |
| CN110908590A (en) | Distributed storage method and system for transformer substation data | |
| CN101414244A (en) | A kind of methods, devices and systems of processing data under network environment | |
| CN107818106B (en) | Big data offline calculation data quality verification method and device | |
| CN204706021U (en) | A kind of memory node improving ease for maintenance | |
| CN115408149A (en) | Time sequence storage engine memory design and distribution method and device | |
| CN109471860B (en) | Large-scale charging pile data processing method and device for electric vehicle charging network | |
| CN102937956A (en) | Method and device for storing real-time messages in intelligent substation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 650011 Tuo Dong Road, Kunming, Kunming, Yunnan Co-patentee after: Wuhan Huadian CIS Technology Co., Ltd. Patentee after: Yunnan Electric Power Dispatch Control Center Co-patentee after: Hubei Industry University Address before: 650011 Tuo Dong Road, Kunming, Kunming, Yunnan Co-patentee before: Wuhan Shuncheng Technology Co., Ltd. Patentee before: Yunnan Electric Power Dispatch Control Center Co-patentee before: Hubei Industry University |