CN104090987B - A kind of history data store and indexing means - Google Patents
A kind of history data store and indexing means Download PDFInfo
- Publication number
- CN104090987B CN104090987B CN201410364895.7A CN201410364895A CN104090987B CN 104090987 B CN104090987 B CN 104090987B CN 201410364895 A CN201410364895 A CN 201410364895A CN 104090987 B CN104090987 B CN 104090987B
- Authority
- CN
- China
- Prior art keywords
- data
- interim
- file
- archive file
- index
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2477—Temporal data queries
Abstract
The invention discloses a kind of storage of historical data and indexing means, belong to real time historical database field.The present invention gives a kind of storage method of simple efficient historical data and a kind of efficient Indexing Mechanism according to the characteristics of industrial historical data storage format.On the one hand the storage method of historical data arranges flow by compressed cache historical data and optimization data filing and reduces magnetic disc i/o as far as possible, on the other hand being distributed in Each point in time by the magnetic disc i/o load balancing during data filing is arranged, so as to ensure to meet the memory requirement of current historical data, while the storage method has the characteristic of dynamic expansion；Indexing Mechanism can realize the quick access to historical data, while can be unaffected by access of the dynamic modification index file system for data when database progress dynamic expanding.
Description
Technical field
The invention belongs to real time historical database technical field, more particularly, to a kind of history data store and index
Method.
Background technology
Industrial information has become the theme in 21 century global industry field, in recent years IT application process start with
The impetus of a kind of depth and range joint development deepens continuously.With the rise of Internet of Things, big quantity sensor and intelligent terminal
It is widely used in traditional industry field, further having promoted the development process of industrial information.
In traditional industrial process stream, real-time data base has been widely used.It is mainly used in some data volumes compared with
The real-time monitoring of small industrial production system and data persistence, the industry being related to include electric power, oil, metallurgy, chemical industry etc..
Real time historical database describes creation data using file data point as base unit.The file data point pair of one database
A physical data source in inductive sensing device, such as temperature of the voltage of a certain bar power transmission line, some test point etc..Archives number
The data that strong point is collected are discrete datas, all with timestamp.From collection in worksite to production process data pass through a system
Finally it is stored in after column processing among real time historical database.Such data are generally in follow-up process backtracking or data point
Used during analysis.
With the deep development of industrial information, policymaker is to the storage efficiency of historical data and data access speed
It is required that more and more higher, therefore it is more prominent the problem of historical data access efficiency deficiency existing for current real time historical database.
Higher and higher demand is required in face of historical data access efficiency, there is no complete available embodiment both to meet to work as at present
Preceding data access requirement, ensure that system has dynamic scalability to tackle the need of the data storage of sustainable growth and processing again
Ask.
The content of the invention
For the disadvantages described above or Improvement requirement of prior art, the present invention provides storage and the index side of a kind of historical data
Method, according to the characteristics of industrial data storage format, give a kind of storage method and one of simple efficient history data file
The efficient Indexing Mechanism of kind, disclosure satisfy that the efficient access requirement of current historical data, while can ensure with dynamic expansion
Characteristic.On the one hand the storage method arranges flow by magnetic disc i/o by compressed cache historical data and optimization data filing
Reduce as far as possible, Each point in time is on the other hand distributed to by the magnetic disc i/o load balancing during filing is arranged
On, so as to ensure to meet the memory requirement of current historical data, while the storage method has the characteristic of dynamic expansion.The index
Mechanism can realize the quick access to historical data, while can be changed when database progress dynamic expanding by dynamic
Index file make it that access of the system for data is unaffected.
To achieve the above object, according to one aspect of the present invention, there is provided a kind of storage method of historical data, bag
Include：
Step 1 reads a historical data insertion from level cache and asked, the numbering of insertion request bag point containing archives,
The data value of the archives point and the timestamp of the data value, the archives point refer to the time series data of a data source；
Step 2 carries out lossy compression method using Lossy Compression Algorithm to the data value in the insertion request, if institute
State data value not to be compressed out, then the data value and the timestamp are write into the archives point corresponding to L2 cache
In Secondary buffer in write state, in the L2 cache, each archives point corresponds to two Secondary buffers, wherein
When one Secondary buffer is in write state, another is at read states, when the data value of archives point and described
When timestamp writes the L2 cache, it is first determined the Secondary buffer of write state is currently at corresponding to the archives point, so
Specific writing position is determined according to the buffering area head of Secondary buffer afterwards, then completes writing for the data value and the timestamp
Enter, final updating is currently at the buffering area header of the Secondary buffer of write state；
If the Secondary buffer that step 3 is currently at write state has been write the timing of full or its timer and reached,
The reading and writing states of described two Secondary buffers corresponding to the archives point is exchanged, and by the Secondary buffer in read states
The filing historical data stored writes interim archive file, and the interim archive file only stores the filing history number of today
According to specific writing position is numbered according to archives point and inquired about in filing index file temporarily, and after completing write-in, renewal is interim again
File the writing position information of index file；
Step 4 travels through all files point, by its filing history number in interim archive file whenever entering new one day
According to formal archive file is all write, the afterbody of the formal archive file is always write every time, after completing write-in, then will write-in
Starting position recorded filing index file；Before interim archive file is arranged, first interim filing rope was created for new one day
Quotation part and interim archive file；After arrangement finishes, the interim filing index file arranged and interim archive file are deleted.
It is another aspect of this invention to provide that a kind of indexing means of historical data are provided, including：
Step 1 obtains an inquiry request, and it includes archives point numbering, the initial time T of inquiryBEGINAnd the end time
TEND；
The query time section included in the inquiry request is divided into N number of same natural day by step 2 by natural day
Subquery task, wherein N represent the natural day number that the query time section includes；
Step 3 obtains a subquery task being not carried out；
Step 4 according to " year-month-day " information of the initial time of the subquery task, judge initial time whether be
Today, step 5 is if it is performed, otherwise perform step 7；
Archives point numbering of the step 5 in the subquery task calculates the interim indexing units of archives point in today
Corresponding offset address in interim index file, calculation formula are as follows：
Interim indexing units offset address=sizeof (interim indexing units) × (archives point numbering -1)
Wherein, sizeof (interim indexing units) represents the length of the interim indexing units of each archives point；According to described
The offset address of interim indexing units reads the interim indexing units of the archives point from the interim index file of today, so as to
Obtain the first address and current data block first address of first data block；
Step 6 travels through whole data block chained list, in order read block from the first data BOB(beginning of block) of the archives point
In data field and be cached to compressed data set Data, also read and the two level of write state is currently at corresponding to the archives point delays
The compressed data rushed in area is simultaneously cached to compressed data set Data, continues executing with step 9；
" year-month-day " the acquisition of information date index file for the initial time that step 7 includes according to the subquery task
Index block, formal archive file storage location and filename, corresponding filing index are further obtained from the index block
Date index data item corresponding to file storage catalogue and filename and Query Dates, further indexed according to the date
Offset address and archives point numbering of the filing index data in corresponding filing index file in data item, obtains Query Dates
The index data of this day archives point, including described this day of Query Dates data in corresponding formal archive file
Offset address and length；
The index information that step 8 obtains according to the step 7, the archives are read out from corresponding formal archive file
Compressed data set Data corresponding to this day of point；
Step 9 solves according to the data update cycle of the archives point to the data in the compressed data set Data
Compression, obtain the daylong decompression data acquisition system DeData of the in chronological sequence tactic archives point；
If the initial time of the step 10 subquery task is equal to the end time, represent to inquire about a special time stamp
Data, then the data D that timestamp is equal to initial time is searched in the decompression data set DeDataA, if there is then returning
Return data DA, the result set of sky is otherwise returned, if the initial time is not equal to the end time, represents one spy of inquiry
The data for section of fixing time, then it will meet that timestamp is more than or equal to the initial time and small in the decompression data set DeData
Subquery task result set is in chronological sequence sequentially inserted into the data equal to the end time condition, by i-th of son
Query task TiResult data collection obtained by inquiry is designated as RSi, wherein, i span is 1~N；
Step 11 judges whether that all equal executeds of subquery task finish, if not then continuing executing with the step 3,
Otherwise step 12 is performed；
Step 12 is by the result data collection RS obtained by all subquery tasksiMerged according to time order and function order,
Archives point to be checked is obtained in the initial time TBEGINTo the end time TENDAll historical datas in period,
Finally the final result collection after merging is returned.
In general, by the contemplated above technical scheme of the present invention compared with prior art, have below beneficial to effect
Fruit：
First, being used alternatingly using two groups of L2 caches of A, B, can improve the caching capabilities of data, prevent data from losing
Lose；
Secondly, by designing interim archive file and interim index file, in view of its file structure is simple, can subtract significantly
Disk operating number during few data filing, so as to improve the filing ability of data；
Again, by designing interim archive file method for sorting and tightening the formal archive file of formula, can greatly reduce
The memory space of formal archive file；
Finally, index and file by the date and index, can substantially reduce in formal archive file in query process
Disk operating number, so as to improve search efficiency.
Brief description of the drawings
Fig. 1 is the general illustration of historical data persistent storage process in the embodiment of the present invention；
Fig. 2 is the structural representation of L2 cache of the embodiment of the present invention；
Fig. 3 is interim archive file and interim indexed file structure schematic diagram in the embodiment of the present invention；
Fig. 4 is formal archive file, date index file and the structural representation for filing index file in the embodiment of the present invention
Figure；
Fig. 5 is data persistence storage method flow chart in the embodiment of the present invention；
Fig. 6 is the data in L2 cache to be filed to storage temporarily to the side in interim archive file in the embodiment of the present invention
Method flow chart；
Fig. 7 is that interim archive file is arranged into the method flow diagram to formal archive file in the embodiment of the present invention；
Fig. 8 be it is provided in an embodiment of the present invention index one archives point at any time in section historical data method flow
Figure.
Embodiment
In order to make the purpose , technical scheme and advantage of the present invention be clearer, it is right below in conjunction with drawings and Examples
The present invention is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, and
It is not used in the restriction present invention.As long as in addition, technical characteristic involved in each embodiment of invention described below
Conflict can is not formed each other to be mutually combined.
Because the requirement that industrial historical data stores for precision is not very high, therefore it is not necessarily to all deposit all data
Storage to it, it is necessary to be compressed.The number of write magnetic disk when in order to reduce historical data persistent storage, it is necessary to will first go through
History data are first cached, so as to disposably by the historical data write-in data in magnetic disk file of a period of time.
Fig. 1 show the general illustration of historical data persistent storage process in the embodiment of the present invention.Of the invention real
Apply in example, historical data mainly experienced from being cached to interim archive file, then from interim archive file to formal archive file
Process.System first initializes, including creates level cache, L2 cache and Miscellaneous Documents.Wherein, level cache is using advanced
The structure of first dequeue, its big I are set according to archives point quantity and its renewal speed, and calculation formula is as follows：
Cache1Size=Num (archives point) × sizeof (data insertion request)
Wherein, the archives point transformation included in Num (archives point) expression system, this upper limit are limited to operation of the present invention
Hardware platform disposal ability；Sizeof (data insertion request) represents the memory space needed for a data insertion request,
Preferable parameter is 64 bytes.The structure of L2 cache is described in detail with size below in connection with Fig. 2.Miscellaneous Documents
Establishment is described in detail below in connection with Fig. 5.
Data comprising archives point are write level cache with inserting RQ cycle by external data source module, wherein, shelves
Case point refers to the time series data of a data source, and it is numbered since 1, and the data of each archives point can be floating numbers, whole
Figurate number or Boolean type number.External data source module can be continuously to level cache write-in data insertion request, the present invention
System also can synchronously read data insertion request constantly, one by one from level cache in embodiment, then in insertion request
Data carry out lossy compression method, using swinging door compression algorithm in the embodiment of the present invention, but not limited to this algorithm, equivalent to only
By flex point data or some accidental datas deposit L2 cache.The data of a period of time can be cached in L2 cache, when two
When level caching timer reaches or L2 cache writes full, system will be by the interim filing of data write-in in L2 cache
In file.Interim archive file only stores the data of today, waits until after having crossed today, understands in the time-division in morning of second day,
After the data for ensuring the previous day are all written with temporary file, temporary file can be arranged and write formal archive file by system.
Fig. 2 show the structural representation of L2 cache of the embodiment of the present invention.L2 cache is divided into the slow of multiple archives points
Area is rushed, each archives point possesses two groups of Secondary buffers of A, B, and it is read states that two Secondary buffers, which are always to maintain one, separately
One is write state, when the cycle of the filing timer of one of Secondary buffer reaches or the Secondary buffer writes full,
Then exchange the read-write state of two Secondary buffers.Each Secondary buffer includes buffering area head and data, buffering area head bag
Flag bit containing read-write state, the timestamp of first data in Secondary buffer, the timestamp of latest data and space-consuming
Size.Wherein, read-write state flag bit is used to identify the current state of the Secondary buffer；The timestamp of first data represents
The timestamp of first data preserved in the Secondary buffer, millisecond is accurate to, each timestamp occupies 9 bytes, its middle age
Occupy 2 bytes, the moon, day, when, minute, second occupy 1 byte respectively, millisecond occupies 2 bytes；The timestamp of latest data then represents most
The timestamp (even if this data is reduce by compression algorithm) of new data；Space-consuming size have recorded current level 2 buffering
The size of data storage content in area.The timestamp and data value of the packet of Secondary buffer point containing map file.
The size of each group of Secondary buffer of each archives point is calculated according to equation below：
Cache2Size=Items × [sizeof (data value)+sizeof (timestamp)]
Wherein, Items represents the data item number of system maximum storage, and preferable parameter is 256；Sizeof (data value)
Memory space needed for each data value is represented, is together decided on by the data type and operating system addressing bit number of data value, is operated
System Addresses digit is generally 32 or 64；Sizeof (timestamp) represents memory space needed for timestamp, and timestamp includes
" during year-month-day:Point:Second milliseconds ", are in embodiments of the present invention 9 bytes.
Fig. 3 show the interim archive file of the embodiment of the present invention and the structural representation of interim index file.Wherein face
When archive file name shaped like tData_YYYY-MM-DD.dat, such as tData_2014-3-19.dat；Interim index file name shape
Such as tIndex_YYYY-MM-DD.index, such as tIndex_2014-3-19.index.Interim archive file include file header and
Filing data area, file header include：State flag bit and file size, state flag bit illustrate that the interim archive file is currently located
In what state, in embodiments of the present invention, particular state includes idle, filing and arrangement etc.；File size includes filing number
According to area and the size of file header.As shown in figure 3, it is that each archives point is assigned with an interim Index List in interim index file
Member, the first address and current data block first address of the first data block of archives point are store in interim indexing units.Wherein, it is first
The first address of individual data block refers to offset address of the first data block in interim archive file；Current data block first address refers to
Be it is current just filing data data block data available for writing offset address of the region in interim archive file.Temporarily
The filing data area of archive file is made up of multiple archives point data blocks, corresponding one section of the archives point of each archives point data block
Historical archiving data in period, each archives point can include multiple archives point data blocks.Archives point data block includes three
Individual part：Data field size, Next pointers and data field.Wherein, the data of data field storage archives point, data field size are number
According to storage size shared by data in area；Next pointers are that next archives point data block offsets ground in interim archive file
Location, so as to form an archives point data block chained list.
System can pre-allocate one piece of data block to deposit for each archives point when initialization in interim archive file
The archives point historical data of one day is stored up, now data field size is initialized as 0, Next pointers and is initialized as address blank.Meanwhile often
The first address and last data block first address of first data block of the individual archives point in interim indexing units are initialized as
Data block is pre-allocated in the offset address in interim archive file.
When the data in L2 cache are written into interim archive file, then the byte number according to each writing data blocks is needed
Update the data block and write offset address.And if current data block is fully written, system can be somebody's turn to do at interim archive file end
Archives point one data block of additional distribution, and new distribution data block is being faced under the Next pointer records with this current data block again
When archive file in offset address.Now, the first address of the first data block in interim indexing units is kept constant, and current
Data block first address will be updated to offset address of the new distribution data block in interim archive file.
Interim archive file and interim index file belong to temporary file, the data in the interim archive file of yesterday
After arrangement, the temporary file of yesterday can be all deleted, while will also create the temporary file of tomorrow, be constantly present system
The interim archive file and interim index file of today and tomorrow, so that daily temporary file can accomplish that seamless switching makes
With.
Fig. 4 show formal archive file in the embodiment of the present invention, date index file and the structure for filing index file
Relation between schematic diagram, and these files.Wherein formal archive file name represents text shaped like hisdata_N.dat, wherein N
Part sequence number, such as first formal archive file is hisdata_1.dat；Date index file has and only one, its file
Name is shaped like time.index；It is corresponding with formal archive file to file index file, its filename shaped like hisdata_N.index,
Wherein N is consistent with the sequence number in formal archive file name.To ensure to present invention can be suitably applied to different file, each formally return
The storage cap of files can be configured according to actual file system, such as：Formal filing text in FAT32 file system
The storage cap of part is no more than 4GB, and new technology file system storage cap can then be set to the value more than 4GB, and the specific upper limit is also
It can be modified according to practical application request, but preferable formal archive file storage cap is not lower than 2GB.When one just
Formula archive file reaches the storage cap of setting, then creates a new formal archive file and filing index file, its sequence number
All will be from increasing 1.In embodiments of the present invention, formal archive file as shown in figure 4, since file header each archives of sequential storage
Put the data of every day.
Date index file is made up of multiple index blocks, the corresponding formal archive file of each index block.Each index
Start Date (include year, month, day), Close Date of the block comprising institute's data storage in corresponding formal archive file (include
Year, month, day), the number of days of data storage, file corresponding to formal archive file storage catalogue and filename, formal archive file
Index file storage catalogue and filename, and date index data item.Wherein, the quantity of date index data item and storage
The number of days of data is equal.Each date index data item includes date and its filing index data and files index file corresponding
In offset address.
Filing index file correspond to formal archive file, and the number of days of data storage is equal to filing index in formal archive file
File the number of days of index data in file.Filing index file is made up of multiple filing index data blocks, each files index number
The index information of all files point data of one day in formal archive file is corresponded to according to block.File each archives in index data block
The index data of point contains offset address and data length of the archives point data of one day in corresponding formal archive file
Degree.
Fig. 5 show the flow chart of data persistence storage method in the embodiment of the present invention, specifically includes following steps：
In step 501, system initialization, level cache is created, and the maximum archives points amount in system is by upper
The text mode creates one group of L2 cache for each archives point；
In step 502, the interim archive file before today or today is checked for, if it is performs step
Rapid 503, otherwise perform step 504；
In step 503, there will be interim archive file sorting and file to formal archive file, and delete after completion
Except the interim archive file and interim index file；
In step 504, date index file is checked for, if there is no then according to foregoing date index file
The newly-built date index file of naming rule；Formal archive file is checked for, if there is no then according to foregoing
Newly-built one empty formal archive file of naming rule and its corresponding filing rope of formal archive file and filing index file
Quotation part, and index block corresponding to the afterbody insertion in date index file；According to foregoing interim archive file and interim index
The naming rule of file distinguishes interim archive file and the interim index file of newly-built today and tomorrow；
In step 505, historical data insertion request is read from level cache, archives point numbering in analysis request and
Timestamp corresponding to data；
In step 506, if the data time stamp in insertion request is currently at the two of write state earlier than the archives point
The latest data timestamp preserved in level caching, then insertion request is abandoned, step 505 is re-executed, otherwise performs step
507；
In step 507, the newest number in the buffering area head for the Secondary buffer that write state will be in corresponding to archives point
It is the timestamp of this data according to update of time stamp, and performs Lossy Compression Algorithm, such as swinging door compression algorithm, but be not limited to
This algorithm；
In step 508, judge whether data are compressed out, be then to perform step 509, otherwise perform step 510；
In step 509, judged to file whether the cycle has arrived according to the current time in system, the filing cycle can be by actually should
With requiring to determine, the embodiment of the present invention preferably filed the cycle for 10 minutes, if it is performs step 512, otherwise performs step
Rapid 505；
In step 510, the Secondary buffer of write state will be in corresponding to data and its timestamp write-in archives point, and
The latest data timestamp in buffering area head is updated, and has been accounted for byte number renewal of the space-consuming plus this write-in data
Use space；
In step 511, taken in the buffering area head of the Secondary buffer in write state according to corresponding to archives point
Space, judges whether the Secondary buffer has been write completely, or judges whether the filing cycle has arrived according to the current time in system, such as
Fruit is then to perform step 512, otherwise performs step 505；
In step 512, two groups of Secondary buffer read-write states of A, B of archives point are exchanged, and reading shape will be replaced by
The data that the Secondary buffer of state is stored write interim archive file and reinitialize the Secondary buffer；
In step 513, inspection system current time, judge whether to enter new one day, if it is perform step
514, otherwise perform step 505；
In the step 514, by the interim archive file sorting and file of yesterday to formal archive file, yesterday is deleted after completing
It interim archive file and interim index file, and the interim archive file of tomorrow and interim index file are created, so as to every
It temporary file can accomplish that seamless switching uses.
The data conversion storage that Fig. 6 is shown in the embodiment of the present invention by an archives point in L2 cache is literary to interim filing
Flow chart in part, specifically includes following steps：
In step 601, according to the state flag bit of the interim archive file of today, judge whether interim archive file is located
In idle condition, step 602 is if it is performed, otherwise performs step 603；
In step 602, the state flag bit of the interim archive file of today is updated to write state；
In step 603, an archives point numbering is obtained, and is read and is somebody's turn to do from interim index file according to archives point numbering
The interim indexing units of archives point, using the current data block first address in interim indexing units, read from interim archive file
The data field size of current data block is taken, and the address information for obtaining its L2 cache is numbered according to archives point；
In step 604, in the buffering area head of the Secondary buffer in read states according to corresponding to the archives point
Space-consuming, the data field size of current data block and the upper limit of data block, judge that can current data block accommodate this and be in
All data in the Secondary buffer of read states, if can if perform step 606, otherwise perform step 605；
In step 605, a data block is newly created in the afterbody of interim archive file, preferable data block size is
16K bytes；The Next pointers of current data block are pointed into offset address of the data block newly created in interim archive file, and
Current data block first address in the interim indexing units of the archives point in interim index file is updated to newly create data
Offset address of the block in interim archive file；
In step 606, the data write-in current data block this being in the Secondary buffer of read states, is in this
The data field size of the renewal current data block of space-consuming in the buffering area head of the Secondary buffer of read states；This is in
The timestamp of first data, the timestamp of latest data in the Secondary buffer of read states and space-consuming size are just
Beginning turns to 0, continues executing with step 603 until all archives points all handle completion.
Fig. 7 show and the method flow diagram that interim archive file forms formal archive file is arranged in the embodiment of the present invention,
Specifically include following steps：
In step 701, system monitors whether current time has been arrived interim archive file data preparation write-in just in real time
The time threshold T of formula archive fileOrderIf time threshold T is arrivedOrderStep 702 is then performed, otherwise continues monitoring in real time
Current time, in embodiments of the present invention, preferable time threshold TOrderFor 15 minutes, to ensure the number in all L2 caches
According to the interim archive file for being all written into yesterday；
In a step 702, the compressed data capacity sum of all files point in the interim archive file of yesterday is scanned, according to
Formal archive file maxsize judges whether current formal archive file can accommodate, and is then to perform step 705, otherwise holds
Row step 703；
In step 703, the number in the interim archive file of yesterday is arranged according to the order of archives point numbering from small to large
According to the data of each archives point corresponding data block in interim archive file are integrated together sequentially in time, are somebody's turn to do
The archives point compressed data set of one day；
In step 704, one by one according to archives point numbering order from small to large by the compressed data set of each archives point
Current formal archive file is written to, meanwhile, one filing index data block of increase, record in corresponding filing index file
Offset address and its length of the compressed data set of each archives point in current formal archive file, then, are indexed on the date
Tail of file increases a date index data item, records the filing index data block newly increased in currently filing index file
Offset address and its corresponding date；
In step 705, empty according to newly-built one of the naming rule of foregoing formal archive file and filing index file
Formal archive file and its corresponding filing index file, and index block corresponding to the afterbody insertion in date index file, will
The formal archive file newly created writes a dataset into newly-built archive file and update date rope as current formal archive file
Quotation part and filing index file；
In step 706, when yesterday interim archive file arrange finish after, can by the interim archive file of yesterday with
And interim index file is deleted, and the interim archive file of newly-built tomorrow and interim index file are to ensure daily temporary file
Accomplishing seamless switching when day data stores.
Due to historical data in formal archive file the compressed data of same archives point on the same day in the presence of together,
Therefore the indexing means of the embodiment of the present invention can also may be used with the historical data in certain period in the archives point one day of fast positioning one
To index the historical data at specific moment.
Fig. 8 show it is provided in an embodiment of the present invention index one archives point at any time in section historical data method
Flow chart, specifically include following steps：
In step 801, an inquiry request is obtained, it includes archives point numbering, the initial time T of inquiryBEGINAnd knot
Beam time TEND, initial time TBEGINWith end time TENDInclude " during year-month-day:Point:Second milliseconds "；
In step 802, the query time section included in inquiry request is divided into N number of same natural day by natural day
Subquery task, wherein N represents the natural day number that the query time section that includes includes in inquiry request, and each subquery is appointed
The beginning and ending time of business is in same natural day；Use TiI-th of subquery task is represented, i span is 1~N, its
Neutron interrogation task T1Beginning and ending time be initial time TBEGINTo TBEGINPlace nature day 23 when 59 divide 59 seconds 999 milliseconds,
Subquery task TNBeginning and ending time be end time TENDPlace nature day 0 when 0 divide 0 second 0 millisecond to TEND, and subquery is appointed
Be engaged in T2, T3..., TN-1Represent a complete time of the tundish containing natural day, i.e., in same natural day 0 when 0 divide 0 second 0 milli
Second to when 23 59 divide 59 seconds 999 milliseconds；
In step 803, a subquery task being not carried out is obtained；
In step 804, according to " year-month-day " information of the initial time of subquery task, the subquery task is judged
Initial time whether be today, if it is perform step 805, otherwise perform step 807；
In step 805, in subquery task archives point numbering calculate the archives point interim indexing units exist
Corresponding offset address in the interim index file of today, calculation formula are as follows：
Interim indexing units offset address=sizeof (interim indexing units) × (archives point numbering -1)
Wherein, sizeof (interim indexing units) represents the length of the interim indexing units of each archives point, in the present invention
Preferable parameter is 16 bytes in embodiment；Read according to the offset address of interim indexing units from the interim index file of today
The interim indexing units of the archives point are taken, so as to obtain the first address of first data block and current data block first address；
In step 806, from the first data BOB(beginning of block) of the archives point, whole data block chained list is traveled through, is read in order
Data field in data block is simultaneously cached to compressed data set Data, also reads and is currently at the two of write state corresponding to the archives point
Level buffering area in compressed data and be cached to compressed data set Data, continue executing with step 809；
In step 807, " year-month-day " acquisition of information date index of the initial time included according to subquery task
The index block of file, formal archive file storage location and filename, corresponding filing index are further obtained from index block
Date index data item corresponding to file storage catalogue and filename and Query Dates, further according to date index data
Filing index data in the corresponding offset address filed in index file and archives point numbering, obtain Query Dates this
The index data of its archives point, including offset address of the data in corresponding formal archive file of this day of Query Dates with
And length；
In step 808, the index information obtained according to step 807, this is read out from corresponding formal archive file
Compressed data set Data corresponding to this day of archives point；
In step 809, according to the data update cycle of the archives point, the data in compressed data set Data are solved
Compression, obtain the daylong decompression data acquisition system DeData of in chronological sequence tactic archives point；
In step 810, if the initial time of subquery task is equal to the end time, one special time of inquiry is represented
The data of stamp, then the data D that timestamp is equal to initial time is searched in data set DeData is decompressedA, if there is then returning
Data DA, otherwise return to empty result set；If initial time is not equal to the end time, one special time period of inquiry is represented
Data, then it will meet that timestamp is more than or equal to initial time and is less than or equal to end time condition in decompression data set DeData
Data be in chronological sequence sequentially inserted into subquery task result set；By i-th of subquery task TiObtained by inquiry
Result data collection is designated as RSi；
In step 811, judge whether that all equal executeds of subquery task finish, if not then continuing executing with step
803, otherwise perform step 812；
In step 812, by N number of period, the result data collection RS obtained by all subquery tasksiAccording to the time
Sequencing merges, that is, obtains archives point to be checked in initial time TBEGINTo end time TENDIt is all in period
Historical data, finally the final result collection after merging is returned.
As it will be easily appreciated by one skilled in the art that the foregoing is merely illustrative of the preferred embodiments of the present invention, not to
The limitation present invention, all any modification, equivalent and improvement made within the spirit and principles of the invention etc., all should be included
Within protection scope of the present invention.
Claims (5)
- A kind of 1. historical data storage method, it is characterised in that including：Step 1 reads a historical data insertion request from level cache, and insertion request bag point containing archives is numbered, be described The data value of archives point and the timestamp of the data value, the archives point refer to the time series data of a data source；Step 2 carries out lossy compression method using Lossy Compression Algorithm to the data value in the insertion request, if the number It is not compressed out according to value, then the data value and the timestamp are write into the archives point is being in corresponding to L2 cache In the Secondary buffer of write state, in the L2 cache, each archives point corresponds to two Secondary buffers, one of them When Secondary buffer is in write state, another is at read states, the data value and the time when archives point When stamp writes the L2 cache, it is first determined the Secondary buffer of write state, Ran Hougen are currently at corresponding to the archives point Specific writing position is determined according to the buffering area head of Secondary buffer, then completes the write-in of the data value and the timestamp, Final updating is currently at the buffering area header of the Secondary buffer of write state；Wherein, each Secondary buffer includes buffering area head and data, and the buffering area head includes read-write state flag bit, institute The timestamp of first data in Secondary buffer, the timestamp of latest data and space-consuming size are stated, wherein, the reading Write state flag bit is used to identify the current state of the Secondary buffer；The timestamp of first data represents described two The timestamp of first data preserved in level buffering area；The timestamp of the latest data then represents the time of latest data Stamp；The size of space-consuming have recorded in the Secondary buffer size of data storage content；The level 2 buffering The timestamp and data value of the packet in area point containing map file；If the Secondary buffer that step 3 is currently at write state has been write the timing of full or its timer and reached, should The reading and writing state of described two Secondary buffers corresponding to archives point is exchanged, and the Secondary buffer in read states is deposited The filing historical data of storage writes interim archive file, and the interim archive file only stores the filing historical data of today, tool The writing position of body is numbered in filing index file inquiry temporarily, after completing write-in, the interim filing of renewal again according to archives point The writing position information of index file；Wherein, the interim archive file includes file header and filing data area, the file header Comprising the state flag bit and file size, the state flag bit illustrates what shape the interim archive file is currently at State, the file size include the size of the filing data area and the file header；In the interim filing index file An interim indexing units are distributed for each archives point, the first data block of archives point is store in the interim indexing units First address and current data block first address, wherein, the first address of first data block refers to that the first data block is faced described When archive file in offset address, current data block first address refers to current just available for writing in the data block of filing data Offset address of the region of data in the interim archive file；Step 4 travels through all files point whenever entering new one day, and its filing historical data in interim archive file is complete Portion writes formal archive file, always writes the afterbody of the formal archive file every time, completes after writing, then write-in is started Position recorded filing index file；Before interim archive file is arranged, first interim filing index text was created for new one day Part and interim archive file；After arrangement finishes, the interim filing index file arranged and interim archive file are deleted；Wherein, before the step 1, in addition to initialization operation, the initialization operation specifically include following sub-step：(0-1) creates the level cache and the L2 cache, and creates Secondary buffer described in one group for each archives point, Its size is calculated according to equation below：Cache2Size=Items × [sizeof (data value)+sizeof (timestamp)]Wherein, Items represents the data item number of system maximum storage；Sizeof (data value) represents to deposit needed for each data value Store up space；Sizeof (timestamp) represents memory space needed for each timestamp；(0-2) checks for the interim archive file before today or today, if it is performs step (0-3), no Then perform step (0-4)；(0-3) there will be interim archive file sorting and file to formal archive file, and delete after completion described interim Archive file and interim index file；(0-4) checks for date index file, if there is no a then newly-built date index file；Check whether and deposit In formal archive file, if there is no then newly-built one empty formal archive file and its corresponding filing index file, and Index block corresponding to afterbody insertion in the date index file；The interim archive file of newly-built today and tomorrow and face respectively When index file.
- 2. the method as described in claim 1, it is characterised in that the step 3 specifically includes following sub-step：Whether (3-1) judges the interim archive file in the free time according to the state flag bit of the interim archive file of today State, step (3-2) is if it is performed, otherwise perform step (3-3)；The state flag bit of the interim archive file is updated to write state by (3-2)；(3-3) obtains an archives point numbering, and according to the reading from the interim index file of archives point numbering The interim indexing units of archives point, using the current data block first address in the interim indexing units, from the interim filing The data field size of the current data block is read in file, and the address for obtaining its L2 cache is numbered according to the archives point Information；Space-consuming in (3-4) buffering area head of the Secondary buffer in read states according to corresponding to the archives point is big The upper limit of small, current data block data field size and data block, it is described in reading shape to judge that can current data block accommodate All data in the Secondary buffer of state, if can if perform step (3-6), otherwise perform step (3-5)；(3-5) newly creates a data block in the afterbody of the interim archive file, by the Next pointers of the current data block Offset address of the data block newly created in interim archive file is pointed to, and by the archives point in the interim index file In the interim indexing units in current data block first address be updated to the new establishment data block in the interim filing Offset address in file；Data in the Secondary buffer in read states are write current data block by (3-6), and read states are in described Secondary buffer buffering area head in the size of space-consuming renewal current data block data field size, will described in The timestamp of first data, the timestamp of the latest data in Secondary buffer in read states and it is described Space-consuming size is initialized as 0, continues executing with step (3-3) until all archives points all handle completion.
- 3. the method as described in claim 1, it is characterised in that the filing data area of the interim archive file is by multiple Archives point data block forms, the Historical archiving data in the corresponding archives point certain time of each archives point data block, often Individual archives point includes multiple archives point data blocks, and the archives point data block includes data field size, Next pointers and data field, Wherein, the data of the data field storage archives point, the data field size are memory space shared by data in the data field Size, the Next pointers are next archives point data block offset address in the interim archive file, so as to form one Individual archives point data block chained list.
- 4. method as claimed in claim 2, it is characterised in that the step 4 specifically includes following sub-step：(4-1) system monitors whether current time has been arrived the formal archive file of interim archive file data preparation write-in in real time Time threshold TOrderIf reach the time threshold TOrderStep (4-2) is then performed, when otherwise continuing to monitor current in real time Between；The compressed data capacity sum of all files point in the interim archive file of (4-2) scanning yesterday, according to formal filing text Part maxsize judges whether current formal archive file can accommodate, and is then to perform step (4-5), otherwise performs step (4-3)；The order that (4-3) is numbered from small to large according to archives point arranges the data in the interim archive file of yesterday, by each shelves Case point data of corresponding data block in the interim archive file are integrated together sequentially in time, obtain archives point one day Compressed data set；The compressed data set of each archives point is written to currently just by (4-4) according to the order of archives point numbering from small to large one by one Formula archive file, meanwhile, one filing index data block of increase, records each archives point in corresponding filing index file Offset address and its length of the compressed data set in current formal archive file, then, in the increase of date index file afterbody One date index data item, record newly increase filing index data block currently filing index file in offset address and Its corresponding date, wherein, the date index file is made up of multiple index blocks, the corresponding one formal filing of each index block File；(4-5) newly-built one empty formal archive file and its corresponding filing index file, and in the date index file Afterbody insertion corresponding to index block, using the formal archive file newly created as currently formal archive file, data set is write Enter newly-built archive file and update the date index file and the filing index file；(4-6), can be by the interim archive file of yesterday and interim index after the interim archive file of yesterday is arranged and finished File is deleted, and the interim archive file of newly-built tomorrow and interim index file are to ensure daily temporary file across day data Accomplish seamless switching when storage.
- A kind of 5. historical data indexing means based on historical data storage method described in claim 1, it is characterised in that bag Include：Step 1 obtains an inquiry request, and it includes archives point numbering, the initial time T of inquiryBEGINWith end time TEND；Step 2 looks into the son that the query time section included in the inquiry request is divided into N number of same natural day by natural day Inquiry task, wherein N represent the natural day number that the query time section includes；Step 3 obtains a subquery task being not carried out；Step 4 judges whether initial time is today according to " year-month-day " information of the initial time of the subquery task, If it is step 5 is performed, otherwise performs step 7；The interim indexing units of archives point numbering calculating archives point of the step 5 in the subquery task are interim today Corresponding offset address, calculation formula are as follows in index file：Interim indexing units offset address=sizeof (interim indexing units) × (archives point numbering -1)Wherein, sizeof (interim indexing units) represents the length of the interim indexing units of each archives point；According to described interim The offset address of indexing units reads the interim indexing units of the archives point from the interim index file of today, so as to obtain The first address and current data block first address of first data block；Step 6 travels through whole data block chained list, in order in read block from the first data BOB(beginning of block) of the archives point Data field is simultaneously cached to compressed data set Data, also reads the Secondary buffer that write state is currently at corresponding to the archives point In compressed data and be cached to compressed data set Data, continue executing with step 9；The rope of " year-month-day " the acquisition of information date index file for the initial time that step 7 includes according to the subquery task Draw block, formal archive file storage location and filename, corresponding filing index file are further obtained from the index block Date index data item corresponding to storage catalogue and filename and Query Dates, further according to the date index data Filing index data in the corresponding offset address filed in index file and archives point numbering, obtain Query Dates this The index data of its archives point, include skew of the data in corresponding formal archive file of described this day of Query Dates Address and length；The index information that step 8 obtains according to the step 7, read out from corresponding formal archive file the archives point this Compressed data set Data corresponding to one day；Step 9 was decompressed to the data in the compressed data set Data according to the data update cycle of the archives point, Obtain the daylong decompression data acquisition system DeData of the in chronological sequence tactic archives point；If the initial time of the step 10 subquery task is equal to the end time, represent to inquire about the number of a special time stamp According to then lookup timestamp is equal to the data D of initial time in the decompression data set DeDataA, if there is then returning to number According to DA, empty result set is otherwise returned to, if the initial time is not equal to the end time, when representing that inquiry one is specific Between section data, then will meet in the decompression data set DeData timestamp be more than or equal to the initial time and less than etc. Subquery task result set is in chronological sequence sequentially inserted into the data of the end time condition, by i-th of subquery Task TiResult data collection obtained by inquiry is designated as RSi, wherein, i span is 1~N；Step 11 judges whether that all equal executeds of subquery task finish, if not the step 3 is then continued executing with, otherwise Perform step 12；Step 12 is by the result data collection RS obtained by all subquery tasksiMerge, that is, obtain according to time order and function order Archives point to be checked is in the initial time TBEGINTo the end time TENDAll historical datas in period, finally will Final result collection after merging returns.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410364895.7A CN104090987B (en) | 2014-07-28 | 2014-07-28 | A kind of history data store and indexing means |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410364895.7A CN104090987B (en) | 2014-07-28 | 2014-07-28 | A kind of history data store and indexing means |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104090987A CN104090987A (en) | 2014-10-08 |
| CN104090987B true CN104090987B (en) | 2018-03-13 |
Family
ID=51638703
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410364895.7A Active CN104090987B (en) | 2014-07-28 | 2014-07-28 | A kind of history data store and indexing means |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104090987B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110837592A (en) * | 2019-11-08 | 2020-02-25 | 京东数字科技控股有限公司 | Method, apparatus and computer readable storage medium for data archiving |
Families Citing this family (38)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104408041B (en) * | 2014-10-13 | 2018-03-30 | 深圳市德传技术有限公司 | A kind of store method of gps data |
| CN105739912A (en) * | 2014-12-11 | 2016-07-06 | 北京北方微电子基地设备工艺研究中心有限责任公司 | Method and system for storing real-time data in semiconductor device |
| CN104537030B (en) * | 2014-12-22 | 2018-03-02 | 华为技术有限公司 | A kind of business datum querying method and device, Database Systems |
| CN105792163B (en) * | 2014-12-24 | 2019-12-03 | 联芯科技有限公司 | Frequency sub-band search association table updating method and device, small region search method and device |
| CN104679821B (en) * | 2014-12-30 | 2018-12-21 | 深圳市海蕴新能源有限公司 | data access processing method and device |
| US9892053B2 (en) * | 2015-03-24 | 2018-02-13 | Intel Corporation | Compaction for memory hierarchies |
| CN104699816B (en) * | 2015-03-25 | 2017-12-12 | 成都华迈通信技术有限公司 | A kind of data processing method and device |
| CN104881481B (en) * | 2015-06-03 | 2018-05-01 | 广东省机电设备招标有限公司 | A kind of method and device for accessing magnanimity time series data |
| CN106557272B (en) * | 2015-09-30 | 2019-07-30 | 中国科学院软件研究所 | A kind of efficient sensor historic data archiving method |
| CN105426128A (en) * | 2015-11-16 | 2016-03-23 | 浪潮集团有限公司 | Index maintenance method and device |
| CN106815227A (en) * | 2015-11-27 | 2017-06-09 | 阿里巴巴集团控股有限公司 | A kind of method and apparatus of storage information |
| CN105786975B (en) * | 2016-02-04 | 2018-12-28 | 北京中盛益华科技有限公司 | Massive video storage method for quick-searching |
| CN106021369A (en) * | 2016-05-11 | 2016-10-12 | 上海佳依佳信息科技有限公司 | Method and equipment for data change management |
| CN106095828B (en) * | 2016-06-01 | 2019-03-29 | 积成电子股份有限公司 | A kind of curve data storage scheme based on power information acquisition terminal |
| CN107562764A (en) * | 2016-07-01 | 2018-01-09 | 中国铁路信息技术中心 | A kind of car flow of railway data slicer method and apparatus |
| CN108614818B (en) * | 2016-12-09 | 2020-12-25 | 杭州海康威视数字技术股份有限公司 | Data storage, updating and query method and device |
| CN106855890B (en) * | 2017-01-09 | 2020-07-28 | 深圳巨杉数据库软件有限公司 | Method for realizing final consistency full-text retrieval of high-performance database |
| CN107273403A (en) * | 2017-04-24 | 2017-10-20 | 西安电子科技大学 | A kind of history data collection and the method for storage |
| CN109118609B (en) * | 2017-06-26 | 2020-11-13 | 深圳市道通智能航空技术有限公司 | Unmanned aerial vehicle flight log recording method and device, storage medium and computer equipment |
| CN107894875B (en) * | 2017-11-20 | 2020-12-04 | 深圳市德传技术有限公司 | RTU data storage method |
| CN108008431A (en) * | 2017-11-28 | 2018-05-08 | 佛山市安尔康姆航空科技有限公司 | A kind of unmanned vehicle GPS location recording method |
| CN108920553B (en) * | 2018-06-19 | 2020-10-30 | 西安电子科技大学 | Data recording method for airborne multi-sensor platform |
| CN110704174A (en) * | 2018-07-09 | 2020-01-17 | 中国移动通信有限公司研究院 | Memory release method and device, electronic equipment and storage medium |
| JP7106656B2 (en) * | 2018-09-07 | 2022-07-26 | 株式会社東芝 | DATABASE DEVICE, PROGRAM AND DATA PROCESSING METHOD |
| CN111352935B (en) * | 2019-07-29 | 2021-05-18 | 创新先进技术有限公司 | Index creating method, device and equipment in block chain type account book |
| CN110765074B (en) * | 2019-09-20 | 2022-03-01 | 国网山东省电力公司青岛供电公司 | Method and system for quickly accessing electric load curve data of acquisition terminal |
| CN112685329A (en) * | 2019-10-17 | 2021-04-20 | 云丁网络技术(北京)有限公司 | Method for processing data and related device |
| CN111061736B (en) * | 2019-12-16 | 2023-05-09 | 北京盛世华人供应链管理有限公司 | Multi-index cache big data overtemperature judging method in logistics temperature control industry |
| CN111881138A (en) * | 2020-07-29 | 2020-11-03 | 北京微步在线科技有限公司 | Data storage method and device |
| CN112214493A (en) * | 2020-10-20 | 2021-01-12 | 北京九思易自动化软件有限公司 | Historical record storage method and device, electronic equipment and storage medium |
| CN112702418A (en) * | 2020-12-21 | 2021-04-23 | 潍柴动力股份有限公司 | Double-cache data downloading control method and device and vehicle |
| CN114116795B (en) * | 2021-11-01 | 2022-12-20 | 北京力控元通科技有限公司 | Data storage and query method, device, storage medium and electronic equipment |
| CN115658619B (en) * | 2022-11-17 | 2023-03-28 | 北京星辰天合科技股份有限公司 | Process processing method and device, processor and electronic equipment |
| CN116301579B (en) * | 2022-12-06 | 2024-01-12 | 无锡市海鹰加科海洋技术有限责任公司 | File access system for self-contained measuring equipment and access method thereof |
| CN116204684A (en) * | 2023-02-01 | 2023-06-02 | 浙江正泰仪器仪表有限责任公司 | Storage method, device, equipment and medium of electric energy meter |
| CN116541348B (en) * | 2023-03-22 | 2023-09-26 | 河北热点科技股份有限公司 | Intelligent data storage method and terminal query integrated machine |
| CN116340367B (en) * | 2023-05-30 | 2023-08-08 | 杭州芯翼科技有限公司 | Data query method, device, equipment and storage medium |
| CN116383165B (en) * | 2023-06-03 | 2023-08-11 | 浙江档安科技有限公司 | Adaptive writing archive storage method |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102646130A (en) * | 2012-03-12 | 2012-08-22 | 华中科技大学 | Method for storing and indexing mass historical data |
| CN103853499A (en) * | 2012-11-30 | 2014-06-11 | 上海可鲁系统软件有限公司 | Multi-source data storing and reading method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8090751B2 (en) * | 2002-02-05 | 2012-01-03 | Hewlett-Packard Development Company, L.P. | Operational data store |
-
2014
- 2014-07-28 CN CN201410364895.7A patent/CN104090987B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102646130A (en) * | 2012-03-12 | 2012-08-22 | 华中科技大学 | Method for storing and indexing mass historical data |
| CN103853499A (en) * | 2012-11-30 | 2014-06-11 | 上海可鲁系统软件有限公司 | Multi-source data storing and reading method |
Non-Patent Citations (1)
| Title |
|---|
| 海量历史数据库系统研究;王章;《中国优秀硕士学位论文全文数据库 信息科技辑》;20130715(第7期);第18-28、33-45页 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110837592A (en) * | 2019-11-08 | 2020-02-25 | 京东数字科技控股有限公司 | Method, apparatus and computer readable storage medium for data archiving |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104090987A (en) | 2014-10-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104090987B (en) | A kind of history data store and indexing means | |
| CN102890722B (en) | Indexing method applied to time sequence historical database | |
| CN103605805B (en) | A kind of storage method of magnanimity time series data | |
| CN105930103B (en) | A kind of correcting and eleting codes covering write method of distributed storage CEPH | |
| CN107423422B (en) | Spatial data distributed storage and search method and system based on grid | |
| EP2454691B1 (en) | Database storage architecture | |
| CN102646130B (en) | Method for storing and indexing mass historical data | |
| CN103020204B (en) | A kind of method and its system carrying out multi-dimensional interval query to distributed sequence list | |
| CN103412803B (en) | The method and device that data are recovered | |
| CN102999519B (en) | Read-write method and system for database | |
| CN107870970B (en) | A kind of data store query method and system | |
| CN104077405B (en) | Time sequential type data access method | |
| CN106716409A (en) | Method and system for adaptively building and updating column store database from row store database based on query demands | |
| CN106682042B (en) | A kind of relation data caching and querying method and device | |
| CN106844555A (en) | A kind of time series data storage method for power network WAMS systems | |
| CN110309233A (en) | Method, apparatus, server and the storage medium of data storage | |
| CN107391600A (en) | Method and apparatus for accessing time series data in internal memory | |
| CN102521306A (en) | Application method for data storage system | |
| CN104765782B (en) | A kind of index order update method and device | |
| CN104156432A (en) | File access method | |
| Lu et al. | TridentKV: A read-Optimized LSM-tree based KV store via adaptive indexing and space-efficient partitioning | |
| CN107273403A (en) | A kind of history data collection and the method for storage | |
| YiChuan et al. | Research of real-time data warehouse storage strategy based on multi-level caches | |
| CN104731716A (en) | Data storage method | |
| CN104731779A (en) | Real-time file system data organization and management method facing real-time databases |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |