Support the distributed memory real-time storage device and method of magnanimity acquisition terminal
Technical field
The present invention relates to computer realm, particularly to a kind of distributed memory real-time storage device supporting magnanimity acquisition terminal and
Method.
Background technology
The power information collection of current China has covered and has the most specially become large user and part low pressure resident, and by with informationization
System shares the information such as file data, it is achieved that the functions such as arrearage has a power failure, forward purchasing electric control.By the construction of country's intelligent grid,
Expect the basic construction object realizing " all standing, full collection, control in full " in " 12 " end so that residential electricity consumption is more
Intelligent, electric service more hommization.
Along with the continuous propelling of " all standing " process, acquisition terminal quantity increases rapidly, as a example by Shandong Province, it is achieved " entirely cover
Lid " afterwards anticipated terminal quantity be up to 2,400,000.The growth rapidly of terminal quantity brings the collection data of magnanimity, the most efficiently
These data of process become problem demanding prompt solution.In power information gatherer process, the magnanimity message that acquisition terminal send needs
After resolving, become readable basic business data, on the basis of business datum, carry out power business.In power business, single
The statistics of individual terminal power is the basis of other business realizing, and the realization of this business needs data acquisition of terminal and terminal shelves
Case is mated one by one.
Present terminal archives are generally stored in disk, and in the case of terminal file data volume is little, the business that can substantially meet needs
Ask.But after the magnanimity acquisition terminal epoch arrive, due to the relative low speeds of I/O operation, this storage mode is by strong influence
Archives storage, the real-time updating and inquiring about.The reading and writing data speed relatively disk of internal memory is several orders of magnitude higher, by terminal file
Being stored in internal memory can effective improving performance.But, owing to terminal file data volume is big, if leaving same server in
In internal memory, certainly will be the highest to the performance requirement of server, existing server is difficult to meet this performance requirement;Even if meeting reluctantly,
The storage of terminal file also can take a large amount of internal memory, and impact gathers data task process performance;Meanwhile, this storage mode is difficult to
Adapt to the sustainable growth of terminal scale.For ensureing reading speed and the integrity of power information acquisition system terminal file, carry herein
Go out a kind of distributed real-time storage device.
Summary of the invention
The present invention is directed to the deficiencies in the prior art, it is provided that a kind of distributed memory real-time storage device supporting magnanimity acquisition terminal and
Method.
For achieving the above object, the present invention adopts the following technical scheme that
A kind of distributed memory real-time storage device supporting magnanimity acquisition terminal, takes including main control server and at least one data
Business device, described main control server and data server communication,
Described main control server, it is for reading terminals archive information table from system database, and the terminal shelves that will be read
Case information table is divided into multiple archives block, and each archives block is assigned to administrative data server, at described main control server
In internal memory, storage have recorded the segmentation situation of terminal file information table and the archives block metadata of archives block storage condition, and provides institute
Linchpin data server current load situation inquiry and comparing function;
Described data server, for receiving the archives block of main control server distribution, with the form of multiple archives blocks by terminal shelves
Case information table is stored in internal memory, and calculates task module offer query function towards electricity, and data server quantity arranges multiple
And dynamically increase and decrease can be realized.
Described main control server also includes with lower module:
Archives distribution module, for reading terminals archive information table from system database, is divided into multiple archives block and divides
It is fitted on administrative data server;
Archives metadata memory module, for storing the archives block metadata that archives distribution module is formed, it is provided that based on terminal number
Archives block position enquiring;
Load comparison module, for inquiring about the loading condition that administrative data server is current, the most administrative data take
The load of business device, draws the data server that load is minimum.
Described data server also includes with lower module:
Module for managing files, for the archives block of receiving archives distribution module assignment;
Archives storage module, for storing the real data of archives block, and accepts the inquiry of document query module;
Document query module, for providing the load information of data server and realizing the inquiry of terminal file information table, and face
Calculate task module to electricity and query function is provided.
Can mutually replicate between the module for managing files of described each data server.
Described load comparison module can inquire about the document query module of data server.
After the server that described duty factor is minimum compared with module polls to load, it is sent to electricity and calculates task module.
A kind of storage method of distributed memory real-time storage device supporting magnanimity acquisition terminal, comprises the following steps:
(1) archives of main control server distribute the module size according to the practical situation configuration profile block of terminal file information table, from
Terminal file information table reads archive information by terminal number from little to big order, when the terminal file information read in reaches to join
When putting size, it is packed into an archives block;Therefore the archives table segmentation situation of archives block metadata only need to record first and
Last is stored in the terminal number of this archives block, reduces the internal memory shared by archives block metadata;
(2) for ensureing the high reliability of file data, arranging n(n for each archives block is natural number, and n >=3) individual copy;
The method of salary distribution of archives distribution module is an archives block to be assigned to n different data server store, and ensures it
N different data server is positioned in different frames;
(3) archives block metadata is write archives metadata memory module, the real data of archives is write the data, services of distribution
In device;
(4) the archives block of the distribution of the module for managing files receiving archives in data server module assignment is stored in archives storage module,
Terminal file data are in pipelined fashion from archives distribution module first data server archives storage module of write, from first
Individual copying to second, copy to the 3rd from second, wheel pushing away successively, until copying to n from (n-1)th.
While archives block write data server in described step (4), archives distribution module continues to terminal file letter
Data in breath table;Archives distribution module periodically reads acquisition terminal archive information table from system database, repeats stored above
Process, it is achieved the synchronization of archive information.
After the archives storage of step (4) completes, the document query calculating task module towards electricity is to be calculated task mould by electricity
Block is initiated, and comprises the following steps:
(1) first electricity calculating task module communicates with archives block metadata memory module, and query task relates to the shelves belonging to terminal
Case block and archives block position;Terminal number is positioned at the numbering interval of certain archives block, then return these data residing for archives block clothes
Business device information;Forward step (2) to;
(2) by data server information input load comparison module, communicated by the document query module with data server, look into
Ask the loading condition comparing these data servers, calculate task module to electricity and return the data server information that load is minimum;
(3) electricity calculates task module and initiates inquiry to this server, and document query module accepts query statement, at archives storage
Module carries out matching inquiry, calculates task module to electricity and return terminal file information, complete document query.
The invention has the beneficial effects as follows:
1. terminal file is stored in internal memory by the present invention, is stored in disk compared to traditional, substantially increases terminal file
Storage, the speed inquired about and update.
2. the present invention builds based on master-slave architecture, can realize the dynamic increase and decrease of data server according to demand, improve device
Extensibility, facilitates the management of archives block.
3. terminal file order by number is packed into archives block, reduces archives block metadata size, decrease storage required
Main control server internal memory.
4. the present invention provides many copies mechanism to terminal file, it is ensured that the reliability of storage, reduces the requirement to hardware.
Accompanying drawing explanation
Fig. 1 is that a kind of distributed real-time for power information acquisition system terminal file stores installation drawing;
In figure 1, main control server, 2, data server, 101, archives distribution module, 102, archives metadata memory module,
103, load comparison module, 201, module for managing files, 202, archives storage module, 203, document query module.
Detailed description of the invention
With embodiment, the present invention is further described below in conjunction with the accompanying drawings:
Fig. 1, is that a kind of distributed real-time for power information acquisition system terminal file realized based on the present invention stores
Device, this device is applied to extensive power information collection and processing platform, it is provided that a kind of memory mechanism is in order to store terminal file
Information, and provide a kind of inquiry mechanism to realize the Rapid matching of terminal file and data acquisition of terminal, archive information storage can be improved
Reliability, increasing terminal document query speed.
This device includes: main control server 1 and data server 2;Described main control server 1 include archives distribution module 101,
Archives metadata memory module 102 and load comparison module 103;Described data server 2 includes module for managing files 201, shelves
Case memory module 202 and document query module 203.
Main control server 1, for reading terminals archive information table from system database, is divided into less archives block also
Being assigned to administrative data server 2, in main control server 1 internal memory, storage have recorded archives table segmentation situation and the storage of archives block
The archives block metadata of situation, and the inquiry of administrative data server 2 current load situation and comparing function are provided.
Data server 2, for receiving the archives block of main control server 1 distribution, with the form of multiple archives blocks by terminal file
Information is stored in internal memory, and calculates task module towards electricity and provide query function, and number of servers can arrange multiple and can be real
The most dynamically increase and decrease.
The archives distribution module 101 of described main control server 1, for reading terminals archive information table from system database, will
It is divided into less archives block and is assigned to administrative data server 2.Archives distribution module periodically (can configure) is from system number
According to storehouse is read acquisition terminal archive information table, it is achieved the synchronization of archive information.Practical situation according to terminal file information can be joined
Put the size of archives block.Archive information is read by terminal number order, when the terminal file letter read in from terminal file information table
When breath reaches to configure size, it is packed into an archives block.For ensureing the high reliability of file data, each archives block arranges three
Individual copy.The distribution mechanism of archives distribution module is an archives block to be assigned to three different data servers 2 store,
And ensure that two of which server is positioned at same frame and the 3rd is positioned in different frames.Will record archives table segmentation situation
And the terminal file metadata write archives metadata memory module of archives block storage condition, by the write point of terminal file real data
In the data server 2 joined.While this archives block write data server 2, archives distribution module continues to terminal shelves
Data in case information table.
Archives metadata memory module 102, for storing the archives block metadata that archives distribution module is formed, it is provided that based on terminal
The matching inquiry of numbering.During archives storage, due to the storage characteristics of archives block Archives, only need to record first and last
It is stored in the terminal number of this archives block, reduces the internal memory shared by archives metadata.During document query, if terminal number is positioned at
In the numbering interval of certain archives block, then return this data server 2 information residing for archives block.
Load comparison module 103, for inquiring about the loading condition that administrative data server 2 is current, compares on this basis and draws
The server of light load.During document query, obtain data server 2 information residing for archives block from archives metadata memory module,
The loading condition of these data servers 2 is compared in inquiry, returns the server info that load is minimum.
The module for managing files 201 of described data server 2, for the archives block of receiving archives distribution module assignment.Archives are deposited
Chu Shi, terminal file data write first data server 2 archives storage module from archives distribution module in pipelined fashion,
Copy to second from first, copy to the 3rd from second.
Archives storage module 202, is used for storing archives block real data, and accepts the inquiry of document query module 203.Archives
During inquiry, accept the inquiry of document query module 203, return electricity and calculate task module terminal file information.
Document query module 203, for providing the load information of data server 2, it is achieved the inquiry of terminal file.Accept negative
Carry the inquiry of comparison module 103, return the loading condition of notebook data server 2.The inquiry receiving electricity calculating task module refers to
Order, carries out matching inquiry in archives storage module 202.
A kind of storage method of distributed memory real-time storage device supporting magnanimity acquisition terminal, comprises the following steps:
(1) archives of main control server 1 distribute the big of the module 101 practical situation configuration profile block according to terminal file information table
Little, from terminal file information table, read archive information by terminal number from little to big order, when the terminal file information read in
When reaching to configure size, it is packed into an archives block;Therefore the archives table segmentation situation of archives block metadata only need to record the
One and last be stored in the terminal number of this archives block, reduce the internal memory shared by archives block metadata;
(2) for ensureing the high reliability of file data, n copy is set for each archives block;The distribution side of archives distribution module
Formula is an archives block to be assigned to n different data server 2 store, and ensures its n different data, services
Being positioned in device 2 in different frames, wherein n is natural number, and n >=3;
(3) archives block metadata is write archives metadata memory module 102, the real data of archives is write the data of distribution
In server 2;
(4) the archives block of the distribution of the module for managing files receiving archives in data server 2 module assignment is stored in archives storage module
202, terminal file data are in pipelined fashion from archives distribution module first data server 2 archives storage module of write
202, copy to second from first, copy to the 3rd from second, wheel pushes away successively, until copying to n from (n-1)th
Individual.
While archives block write data server 2 in described step (4), archives distribution module 101 continues to terminal
Data in archive information table;Archives distribution module 101 periodically reads acquisition terminal archive information table, weight from system database
Multiple process stored above, it is achieved the synchronization of archive information.
After the archives storage of step (4) completes, the document query calculating task module towards electricity is to be calculated task mould by electricity
Block is initiated, and comprises the following steps:
(1) first electricity calculating task module communicates with archives block metadata memory module, and query task relates to the shelves belonging to terminal
Case block and archives block position;Terminal number is positioned at the numbering interval of certain archives block, then return these data residing for archives block clothes
Business device 2 information;Forward step (2) to;
(2) by data server 2 information input load comparison module 103, by the document query module with data server 2
203 communications, inquiry is compared the loading condition of these data servers 2, is calculated task module to electricity and return the number that load is minimum
According to server 2 information;
(3) electricity calculates task module and initiates inquiry to this server, and document query module 203 accepts query statement, at archives
Memory module 202 carries out matching inquiry, calculates task module to electricity and return terminal file information, complete document query.