Support the distributed memory real-time storage device and method of magnanimity acquisition terminal
Technical field
The present invention relates to computer realm, particularly a kind of distributed memory real-time storage device and method of supporting the magnanimity acquisition terminal.
Background technology
The power information collection of current China has covered and has all specially become large users and part low pressure residents, and by share information such as file data with information system, has realized functions such as arrearage power failure, forward purchasing electric control.Construction by national intelligent grid expects the construction object that " all standing, full collection, control in full " realized at " 12 " end substantially, makes that residential electricity consumption is more intelligent, hommization is more served in power supply.
Along with the continuous propelling of " all standing " process, acquisition terminal quantity increases rapidly, is example with the Shandong Province, realizes that expectation terminal quantity in " all standing " back will reach 2,400,000.The rapid growth of terminal quantity has brought the image data of magnanimity, and how handling these data efficiently becomes problem demanding prompt solution.In the power information gatherer process, the magnanimity message that send on the acquisition terminal need become readable basic business data after resolving, carry out power business on the basis of business datum.In the power business, the statistics of single terminal power is the basis of other business realizing, and the realization of this business need be mated terminal image data and terminal archives one by one.
Current terminal archives generally are stored in the disk, under the little situation of terminal file data amount, can satisfy business demand substantially.But after the arriving of magnanimity acquisition terminal epoch, since the relative low speed of I/O operation, the real-time that this storage mode will influence the archives storage greatly, upgrade and inquire about.The reading and writing data speed of internal memory exceeds several magnitude than disk, the terminal archives is stored in the internal memory can effectively promotes performance.But, because terminal file data amount is big, if leave in the internal memory of same server, certainly will be very high to the performance requirement of server, existing server is difficult to satisfy this performance requirement; Even satisfy reluctantly, the storage of terminal archives also can take a large amount of internal memories, influences image data task handling property; Simultaneously, this storage mode is difficult to adapt to the sustainable growth of terminal scale.For guaranteeing reading speed and the integrality of power information acquisition system terminal archives, this paper has proposed a kind of distributed real-time memory storage device.
Summary of the invention
The present invention is directed to the deficiencies in the prior art, a kind of distributed memory real-time storage device and method of supporting the magnanimity acquisition terminal is provided.
For achieving the above object, the present invention adopts following technical scheme:
A kind of distributed memory real-time storage device of supporting the magnanimity acquisition terminal comprises main control server and at least one data server, described main control server and data server communication,
Described main control server, it is used for from system database reading terminals archive information table, and the terminal archive information table that reads is divided into a plurality of archives pieces, and each archives piece is assigned to the data server of having jurisdiction over, in described main control server internal memory stored record the situation of cutting apart of terminal archive information table and the archives piece metadata of archives piece storage condition, and provide data server present load status inquiry and the comparing function of having jurisdiction over;
Described data server, be used for receiving the archives piece that main control server distributes, form with a plurality of archives pieces is stored in terminal archive information table in the internal memory, and provides query function towards electric weight calculating task module, and data server quantity arranges a plurality of and can realize dynamically increase and decrease.
Described main control server also comprises with lower module:
The archives distribution module is used for from system database reading terminals archive information table, is divided into a plurality of archives pieces and is assigned to the data server of having jurisdiction over;
Archives metadata store module is used for the archives piece metadata that the memory file distribution module forms, and the archives piece position enquiring based on terminal number is provided;
The load comparison module is used for the inquiry institute current loading condition of the data server of having jurisdiction over, and compares the load size of the data server of having jurisdiction on this basis, draws load minimal data server.
Described data server also comprises with lower module:
Module for managing files is for the archives piece that receives the archives module assigns;
The archives memory module is used for the real data of memory file piece, and accepts the inquiry of document query module;
The document query module is used for providing the load information of data server and the inquiry that realizes terminal archive information table, and provides query function towards electric weight calculating task module.
Can copy mutually between the module for managing files of described each data server.
The document query module of described load comparison module energy data query server.
After described load comparison module inquires the server of load minimum, send to electric weight and calculate task module.
A kind of storage means of supporting the distributed memory real-time storage device of magnanimity acquisition terminal may further comprise the steps:
(1) the archives distribution module of main control server is according to the size of the actual conditions configuration profile piece of terminal archive information table, from terminal archive information table, read archive information by terminal number from little extremely big order, when the terminal archive information that reads in reached the configuration size, packing formed an archives piece; Therefore the archives table of archives piece metadata is cut apart only needs record first and last to deposit the terminal number of this archives piece in the situation, reduced the shared internal memory of archives piece metadata;
(2) for guaranteeing the high reliability of file data, for each archives piece n(n being set is natural number, and n 〉=3) individual copy; The allocation scheme of archives distribution module is an archives piece to be assigned to n different data server store, and guarantees to be positioned on the different frames in its n the different data server;
(3) archives piece metadata is write archives metadata store module, the real data of archives is write in the data server of distribution;
(4) the archives piece of the reception of the module for managing files in data server archives module assigns deposits the archives memory module in, the terminal file data writes first data server archives memory module in the mode of streamline from the archives distribution module, copy to second from first, copy to the 3rd from second, wheel pushes away successively, up to copying to n from n-1.
When the archives piece in the described step (4) write data server, the archives distribution module continued the data in the receiving terminal archive information table; The archives distribution module regularly reads acquisition terminal archive information table from system database, repeat above storing process, realizes the synchronous of archive information.
After the archives storage of step (4) was finished, the document query that calculates task module towards electric weight calculated task module by electric weight and initiates, and may further comprise the steps:
(1) at first electric weight calculates task module and archives piece metadata store module communication, and query task relates to archives piece and the archives piece position under the terminal; Terminal number is positioned between the numbering area of certain archives piece, then returns the residing data server information of this archives piece; Forward step (2) to;
(2) with data server information input load comparison module, by with the document query module communication of data server, inquiry is the loading condition of these data servers relatively, calculates task module to electric weight and returns load minimal data server info;
(3) electric weight calculates task module and initiates inquiry to this server, and the document query module is accepted query statement, carries out matching inquiry in the archives memory module, calculates task module to electric weight and returns the terminal archive information, finishes document query.
The invention has the beneficial effects as follows:
1. the present invention is stored in the terminal archives in the internal memory, than traditional being stored in the disk, and the speed that has improved the storages of terminal archives, inquiry greatly and upgraded.
2. the present invention is based on master-slave architecture and make up, can realize the dynamic increase and decrease of data server according to demand, improved the extensibility of device, made things convenient for the management of archives pieces.
3. the terminal archives are formed the archives piece by the number order packing, reduced archives piece metadata size, reduced and stored required main control server internal memory.
4. the present invention provides many copies mechanism to the terminal archives, guarantees the reliability of storage, has reduced the requirement to hardware.
Description of drawings
Fig. 1 is a kind of distributed real-time memory storage device figure at power information acquisition system terminal archives;
Among the figure 1, main control server, 2, data server, 101, the archives distribution module, 102, archives metadata store module, 103, the load comparison module, 201, module for managing files, 202, the archives memory module, 203, the document query module.
Embodiment
Below in conjunction with drawings and Examples the present invention is further specified:
Fig. 1, be based on a kind of distributed real-time memory storage device at power information acquisition system terminal archives that the present invention realizes, this device is applied to extensive power information collection and processing platform, provide a kind of memory mechanism in order to the storage terminal archive information, and provide a kind of inquiry mechanism to realize the quick coupling of terminal archives and terminal image data, can improve the reliability of archive information storage, accelerate terminal document query speed.
This device comprises: main control server 1 and data server 2; Described main control server 1 comprises archives distribution module 101, archives metadata store module 102 and load comparison module 103; Described data server 2 comprises module for managing files 201, archives memory module 202 and document query module 203.
Main control server 1, be used for from system database reading terminals archive information table, be divided into less archives piece and be assigned to the data server of having jurisdiction over 2, in main control server 1 internal memory stored record the archives table cut apart the archives piece metadata of situation and archives piece storage condition, and provide the data server of having jurisdiction over 2 present load status inquiry and comparing functions.
Data server 2, be used for receiving the archives piece that main control server 1 distributes, form with a plurality of archives pieces is stored in the terminal archive information in the internal memory, and provides query function towards electric weight calculating task module, and number of servers can arrange a plurality of and can realize dynamic increase and decrease.
The archives distribution module 101 of described main control server 1 is used for from system database reading terminals archive information table, is divided into less archives piece and is assigned to the data server of having jurisdiction over 2.Archives distribution module regularly (configurable) reads acquisition terminal archive information table from system database, realize the synchronous of archive information.Size according to the configurable archives piece of the actual conditions of terminal archive information.Read archive information by the terminal number order from terminal archive information table, when the terminal archive information that reads in reached the configuration size, packing formed an archives piece.For guaranteeing the high reliability of file data, each archives piece arranges three copies.The distribution mechanism of archives distribution module is an archives piece to be assigned to three different data servers 2 store, and guarantee wherein two servers to be positioned at same frame and the 3rd be positioned on the different frames.The terminal archives metadata of record archives table being cut apart situation and archives piece storage condition writes archives metadata store module, terminal archives real data is write in the data server 2 of distribution.When this archives piece write data server 2, the archives distribution module continued the data in the receiving terminal archive information table.
Archives metadata store module 102 is used for the archives piece metadata that the memory file distribution module forms, and the matching inquiry based on terminal number is provided.In archives when storage, because the storage characteristics of archives in the archives piece, only need record first and last to deposit the terminal number of this archives piece in, reduced the shared internal memory of archives metadata.During document query, if terminal number is positioned between the numbering area of certain archives piece, then return residing data server 2 information of this archives piece.
Load comparison module 103, be used for inquiry the current loading condition of the data server of having jurisdiction over 2, relatively draw the lighter server of load on this basis.During document query, obtain archives piece data server 2 information of living in from archives metadata store module, inquiry is the loading condition of these data servers 2 relatively, returns the server info of load minimum.
The module for managing files 201 of described data server 2 is for the archives piece that receives the archives module assigns.During the archives storage, the terminal file data writes first data server 2 archives memory modules in the mode of streamline from the archives distribution module, copies to second from first, copies to the 3rd from second.
Archives memory module 202 is used for memory file piece real data, and accepts the inquiry of document query module 203.During document query, accept the inquiry of document query module 203, return electric weight and calculate task module terminal archive information.
Document query module 203 for the load information that data server 2 is provided, realizes the inquiry of terminal archives.Accept the inquiry of load comparison module 103, return the loading condition of notebook data server 2.Receive the query statement that electric weight calculates task module, in archives memory module 202, carry out matching inquiry.
A kind of storage means of supporting the distributed memory real-time storage device of magnanimity acquisition terminal may further comprise the steps:
(1) the archives distribution module 101 of main control server 1 is according to the size of the actual conditions configuration profile piece of terminal archive information table, from terminal archive information table, read archive information by terminal number from little extremely big order, when the terminal archive information that reads in reached the configuration size, packing formed an archives piece; Therefore the archives table of archives piece metadata is cut apart only needs record first and last to deposit the terminal number of this archives piece in the situation, reduced the shared internal memory of archives piece metadata;
(2) for guaranteeing the high reliability of file data, for each archives piece arranges n copy; The allocation scheme of archives distribution module is an archives piece to be assigned to n different data server 2 store, and guarantees to be positioned on the different frames in its n the different data server 2, and wherein n is natural number, and n 〉=3;
(3) archives piece metadata is write archives metadata store module 102, the real data of archives is write in the data server 2 of distribution;
(4) the archives piece of the reception of the module for managing files in the data server 2 archives module assigns deposits archives memory module 202 in, the terminal file data writes first data server 2 archives memory modules 202 in the mode of streamline from the archives distribution module, copy to second from first, copy to the 3rd from second, wheel pushes away successively, up to copying to n from n-1.
When the archives piece in the described step (4) write data server 2, archives distribution module 101 continued the data in the receiving terminal archive information tables; Archives distribution module 101 regularly reads acquisition terminal archive information table from system database, repeat above storing process, realizes the synchronous of archive information.
After the archives storage of step (4) was finished, the document query that calculates task module towards electric weight calculated task module by electric weight and initiates, and may further comprise the steps:
(1) at first electric weight calculates task module and archives piece metadata store module communication, and query task relates to archives piece and the archives piece position under the terminal; Terminal number is positioned between the numbering area of certain archives piece, then returns residing data server 2 information of this archives piece; Forward step (2) to;
(2) with data server 2 information input load comparison modules 103, by communicating by letter with the document query module 203 of data server 2, inquiry is the loading condition of these data servers 2 relatively, calculates task module to electric weight and returns load minimal data server 2 information;
(3) electric weight calculates task module and initiates inquiry to this server, and document query module 203 is accepted query statement, carries out matching inquiry in archives memory module 202, calculates task module to electric weight and returns the terminal archive information, finishes document query.