CN105653613A - Data storage method and device applied to intelligent grain depot - Google Patents
Data storage method and device applied to intelligent grain depot Download PDFInfo
- Publication number
- CN105653613A CN105653613A CN201510989224.4A CN201510989224A CN105653613A CN 105653613 A CN105653613 A CN 105653613A CN 201510989224 A CN201510989224 A CN 201510989224A CN 105653613 A CN105653613 A CN 105653613A
- Authority
- CN
- China
- Prior art keywords
- index page
- item
- page
- record block
- record
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000013500 data storage Methods 0.000 title claims abstract description 14
- 238000007906 compression Methods 0.000 claims description 62
- 230000006835 compression Effects 0.000 claims description 61
- 230000003068 static effect Effects 0.000 claims description 13
- 230000001066 destructive effect Effects 0.000 claims description 8
- 238000012544 monitoring process Methods 0.000 description 2
- 230000006837 decompression Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2272—Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
- G06Q50/02—Agriculture; Fishing; Mining
Abstract
The invention discloses a data storage method and device applied to an intelligent grain depot. The method comprises the following steps: dividing an archive according to pages, storing an index page distribution strategy and a record block distribution strategy under a root directory of the an archive, wherein a record block consists of a plurality of pages; when a storage request which carries a bit number ID (Identity), historical data and the bit number of a timestamp is received, inquiring a preset index page distribution strategy and record block distribution strategy under the root directory of the archive; according to the record block distribution strategy, searching a target record block corresponding to the bit number ID from the idle area of the archive, filling historical data into the target record block; according to the index page distribution strategy, searching the target index page corresponding to the bit number ID; and recording the index information and the timestamp of the target record block in the target index page. Since the data storage method does not need to record the corresponding relationship of each bit number ID and the index information of the record block in the archive, the space of the archive is saved.
Description
Technical field
The application relates to real-time database in processing industry archiving techniques field, more particularly, it relates to a kind of date storage method being applied to intelligence grain depot and device.
Background technology
Along with the development of process industry, real-time data base needs the data volume stored also increasing. For intelligence grain depot, all need to store in real-time data base for intelligence the multiparameter monitoring the state of foodstuff data of grain depot, multiparameter grain feelings alert data etc. Here data volume increase is mainly reflected in: the point requirement of the item of storage increasingly comes many, it is necessary to process the history data store of hundreds of thousands item simultaneously; The data time span of storage requires also more and more longer, it is necessary to store the historical data of more than 2 years. Wherein, item is the least unit entity of data storage, is generally made up of q (Quality Codes), t (timestamp), v (historical data).
Existing data base achieves mode and adopts dynamic memory mode, namely the storage for certain item asks, randomly choose the memory area of free time to store the historical data of item, then in certain fixed position achieved, the corresponding relation between item ID and memory area index information is recorded, in order to subsequent query.
It is understood that when the item that stored number is numerous, it is necessary to recording the corresponding relation between each item ID and memory area index information in the archive of data base, this will take a big chunk space of archive, reduces database purchase ability.
Summary of the invention
In view of this, this application provides a kind of date storage method being applied to intelligence grain depot and device, for solve prior art need to record in achieving item ID and store this item historical data memory area index information between corresponding relation, thus the problem reducing database purchase ability.
To achieve these goals, it is proposed that scheme as follows:
A kind of date storage method being applied to intelligence grain depot, including:
Receiving the storage request of item, described storage request includes item ID, historical data and timestamp;
Preset index page allocation strategy, record block allocation strategy is inquired about under the root achieved, wherein, described index page allocation strategy defines item ID and manipulative indexing page position, described record block allocation strategy defines item ID and corresponding record block position, described archive has divided according to page in advance, and described record block is made up of some pages;
According to described record block allocation strategy, search and described target record block corresponding for item ID;
Described historical data is filled into described target record block;
According to described index page allocation strategy, search and described target index page corresponding for item ID;
By the index information of described target record block and described timestamp record at described target index page.
Preferably, described described historical data is filled into described target record block, including:
Described historical data is carried out real non-destructive compression, obtains Real Time Compression data;
Described Real Time Compression data are filled in the record page of buffer memory;
Judge whether the occupied state of described record page reaches preset occupied state, if so, then described record page is carried out static text compression, obtain compression blocks;
Described compression blocks is filled in described target record block.
Preferably, also include:
Judge whether described target index page is filled with;
If so, according to described index page allocation strategy, distribute new index page for described item, and the index information of new index page is added in a upper index page.
Preferably, also include:
Judging whether the number of the index page corresponding with described item reaches threshold value, wherein, the index page corresponding with described item is as the one-level index page of described item;
If so, it is that described item creates secondary index page, described secondary index page records the index information of the one-level index page of rheme number to some extent.
Preferably, also include:
The first one-level index page of the secondary index page of described item Yu described item is carried out place-exchange.
A kind of data storage device being applied to intelligence grain depot, including:
Storage request reception unit, for receiving the storage request of item, described storage request includes item ID, historical data and timestamp;
Root query unit, for inquiring about preset index page allocation strategy, record block allocation strategy under the root achieved, wherein, described index page allocation strategy defines item ID and manipulative indexing page position, described record block allocation strategy defines item ID and corresponding record block position, described archive has divided according to page in advance, and described record block is made up of some pages;
Target record block searches unit, for according to described record block allocation strategy, searching and described target record block corresponding for item ID;
Historical data fills unit, for described historical data is filled into described target record block;
Target index page searches unit, for according to described index page allocation strategy, searching and described target index page corresponding for item ID;
Target index page record unit, for by the index information of described target record block and described timestamp record at described target index page.
Preferably, described historical data filling unit includes:
Lossless compression unit, for described historical data is carried out real non-destructive compression, obtains Real Time Compression data;
Real Time Compression data fill unit, for being filled in the record page of buffer memory by described Real Time Compression data;
Record page condition adjudgement unit, for judging whether the occupied state of described record page reaches preset occupied state;
Static Compression unit, for when the judged result of described record page condition adjudgement unit is for being, carrying out static text compression by described record page, obtain compression blocks;
Compression blocks fills unit, for being filled in described target record block by described compression blocks.
Preferably, also include:
Target index page judging unit, is used for judging whether described target index page is filled with;
New index page allocation units, for when the judged result of described target index page judging unit is for being, according to described index page allocation strategy, distributing new index page for described item, and be added in a upper index page by the index information of new index page.
Preferably, also include:
Index page number judging unit, for judging whether the number of the index page corresponding with described item reaches threshold value, wherein, the index page corresponding with described item is as the one-level index page of described item;
Secondary index page creating unit, for when the judged result of described index page number judging unit is for being, creating secondary index page for described item, record the index information of the one-level index page of rheme number to some extent in described secondary index page.
Preferably, also include:
Index page place-exchange unit, for carrying out place-exchange by the secondary index page of described item and the first one-level index page of described item.
Can be seen that from above-mentioned technical scheme, what the embodiment of the present application provided is applied to the date storage method of intelligence grain depot, in advance archive is divided according to page, and prestored index page allocation strategy and record block allocation strategy under the root achieved, wherein, index page allocation strategy defines item ID and manipulative indexing page position, and record block allocation strategy defines item ID and corresponding record block position, and record block is made up of some pages. When the storage receiving the item carrying item ID, historical data and timestamp is asked, preset index page allocation strategy and record block allocation strategy is inquired about under the root achieved, according to record block allocation strategy, the target record block corresponding with item ID is searched from the free area achieved, historical data is filled into target record block, according to index page allocation strategy, search the target index page corresponding with item ID, by the index information of target record block and timestamp record at target index page. The date storage method of the application, prespecified index page allocation strategy and record block allocation strategy, determine record block and index page according to above-mentioned two strategy during storage item, store historical data in record block, index page stores index information and the timestamp of record block, according to item ID and index page allocation strategy and record block allocation strategy, historical data corresponding to item ID can be inquired about during subsequent query item historical data. Owing to the application need not record the corresponding relation between the index information of each item ID and record block in archive, save the space of archive, improve the storage capacity of data base.
Accompanying drawing explanation
In order to be illustrated more clearly that the embodiment of the present application or technical scheme of the prior art, the accompanying drawing used required in embodiment or description of the prior art will be briefly described below, apparently, accompanying drawing in the following describes is only embodiments herein, for those of ordinary skill in the art, under the premise not paying creative work, it is also possible to obtain other accompanying drawing according to the accompanying drawing provided.
Fig. 1 is the disclosed a kind of archiving structure schematic diagram of the embodiment of the present application;
Fig. 2 is a kind of date storage method flow chart being applied to intelligence grain depot disclosed in the embodiment of the present application;
Fig. 3 is the date storage method flow chart that another kind disclosed in the embodiment of the present application is applied to intelligence grain depot;
Fig. 4 is another date storage method flow chart being applied to intelligence grain depot disclosed in the embodiment of the present application;
Fig. 5 is another date storage method flow chart being applied to intelligence grain depot disclosed in the embodiment of the present application;
Fig. 6 is a kind of data storage device structural representation being applied to intelligence grain depot disclosed in the embodiment of the present application.
Detailed description of the invention
Below in conjunction with the accompanying drawing in the embodiment of the present application, the technical scheme in the embodiment of the present application is clearly and completely described, it is clear that described embodiment is only some embodiments of the present application, rather than whole embodiments. Based on the embodiment in the application, the every other embodiment that those of ordinary skill in the art obtain under not making creative work premise, broadly fall into the scope of the application protection.
Before introducing the application scheme, first introduce archive. Archive has been divided for minimum unit by the application with page, and specifies root page, index page and record block, wherein, root page, index page size can be one page, the size of record block can be multipage, such as 4 pages.
The application presets index page allocation strategy and record block allocation strategy, and the two strategy has been preserved under the root directory. Record block is for storing the historical data of item, and index page is used for storing the index information of record block and the timestamp of item.
For index page allocation strategy and record block allocation strategy, it can have multiple, as shown in Figure 1, the application can set that the first index page of item is positioned at and achieves the page that the page number is identical with described item, namely stores index page corresponding to each item ID backward successively from achieving head (removing root page). Simultaneously, for record block, it according to the allocation strategy contrary with index page, can store forward from achieving afterbody successively, such as item ID be 1 record block be positioned at archive last record block, item ID be 2 record block be positioned at archive penultimate record block.
Certainly, above-mentioned merely illustrating a kind of optional allocation strategy, in addition the application can also set the allocation strategy of other index page and record block.
It is a kind of date storage method flow chart being applied to intelligence grain depot disclosed in the embodiment of the present application referring to Fig. 2, Fig. 2.
As in figure 2 it is shown, the method includes:
Step S200, receive item storage request;
Wherein, described storage request includes item ID, historical data and timestamp. Further, can also carrying Quality Codes in request, Quality Codes represents a kind of state of item.
For the historical data of item, its needs preserve for a long time, it is therefore desirable to be stored in archive, and the history trendline time of the timestamp of log history data, timestamp namely historical data.
The historical data of item can be the intelligence multiparameter monitoring the state of foodstuff data of grain depot, multiparameter grain feelings alert data etc.
Step S210, under the root achieved, inquire about preset index page allocation strategy, record block allocation strategy;
Wherein, described index page allocation strategy defines item ID and manipulative indexing page position, and described record block allocation strategy defines item ID and corresponding record block position, and described archive has divided according to page in advance, and described record block is made up of some pages.
Step S220, according to described record block allocation strategy, search and described target record block corresponding for item ID;
Specifically, if before the storage request of this item, having stored the historical data of identical item ID in archive, then the target record block found in this step is likely to stored have historical data. If before the storage request of this item, not storing the historical data of identical item ID in archive, then the target record block that this finds is idle block.
Step S230, described historical data is filled into described target record block;
Step S240, according to described index page allocation strategy, search and described target index page corresponding for item ID;
Similar with above-mentioned target record block, if before the storage request of this item, having stored the historical data of identical item ID in archive, then the target index page found in this step is likely to stored have index information and timestamp. If before the storage request of this item, not storing the historical data of identical item ID in archive, then the target index page that this finds is idle index page.
Step S250, by the index information of described target record block and described timestamp record at described target index page.
Specifically, the index information of target record block can include target record block positional information in achieving.
By the index information of record object record block in the index page corresponding with item ID, during subsequent query item historical data, by inputting item ID, find and corresponding for the item ID index page inputted according to preset index page allocation strategy, and then according to the index information of record block of record in index page and timestamp, find the record block of correspondence, therefrom read historical data.
What the embodiment of the present application provided is applied to the date storage method of intelligence grain depot, prespecified index page allocation strategy and record block allocation strategy, determine record block and index page according to above-mentioned two strategy during storage item, store historical data in record block, index page stores index information and the timestamp of record block, according to item ID and index page allocation strategy and record block allocation strategy, historical data corresponding to item ID can be inquired about during subsequent query item historical data. Owing to the application need not record the corresponding relation between the index information of each item ID and record block in archive, save the space of archive, improve the storage capacity of data base.
It is the date storage method flow chart that another kind disclosed in the embodiment of the present application is applied to intelligence grain depot referring to Fig. 3, Fig. 3.
As it is shown on figure 3, the method includes:
Step S300, receive item storage request;
Wherein, described storage request includes item ID, historical data and timestamp. Further, can also carrying Quality Codes in request, Quality Codes represents a kind of state of item.
For the historical data of item, its needs preserve for a long time, it is therefore desirable to be stored in archive, and the history trendline time of the timestamp of log history data, timestamp namely historical data.
Step S310, under the root achieved, inquire about preset index page allocation strategy, record block allocation strategy;
Wherein, described index page allocation strategy defines item ID and manipulative indexing page position, and described record block allocation strategy defines item ID and corresponding record block position, and described archive has divided according to page in advance, and described record block is made up of some pages.
Step S320, according to described record block allocation strategy, search and described target record block corresponding for item ID;
Specifically, if before the storage request of this item, having stored the historical data of identical item ID in archive, then the target record block found in this step is likely to stored have historical data. If before the storage request of this item, not storing the historical data of identical item ID in archive, then the target record block that this finds is idle block.
Step S330, described historical data is carried out real non-destructive compression, obtain Real Time Compression data;
Specifically, real non-destructive compression is a kind of lossless compressiong being specifically designed to real time data, and its compression and decompression speed are very fast.
Step S340, described Real Time Compression data are filled in the record page of buffer memory;
Specifically, the application is cached with record page outside achieving. The Real Time Compression data obtained through real non-destructive compression are filled in record page.
Step S350, judge whether the occupied state of described record page reaches preset occupied state, if so, then perform step S360;
Specifically, the occupied state of record page can be whether record page fills completely.
Step S360, described record page is carried out static text compression, obtain compression blocks;
When the occupied state determining record page reaches threshold value occupied state, record page is carried out static text compression, obtains compression blocks. Static text compression has higher compression ratio, by static text compression, the data in record page is compressed further.
Step S370, described compression blocks is filled in described target record block;
Specifically, the compression blocks obtained after record page being carried out static text compression is filled in the target record block in archive.
Step S380, according to described index page allocation strategy, search and described target index page corresponding for item ID;
Similar with above-mentioned target record block, if before the storage request of this item, having stored the historical data of identical item ID in archive, then the target index page found in this step is likely to stored have index information and timestamp. If before the storage request of this item, not storing the historical data of identical item ID in archive, then the target index page that this finds is idle index page.
Step S390, by the index information of described target record block and described timestamp record at described target index page.
Compared to a upper embodiment, the process that historical data is filled in the present embodiment target record block further is described. When filling historical data in target record block, historical data is carried out two layer compression process and obtains compression blocks, greatly reduce history data store space, in the target record block in finally compression blocks write being achieved, save the memory space of archive.
It is another date storage method flow chart being applied to intelligence grain depot disclosed in the embodiment of the present application referring to Fig. 4, Fig. 4.
As shown in Figure 4, the method includes:
Step S400, receive item storage request;
Wherein, described storage request includes item ID, historical data and timestamp. Further, can also carrying Quality Codes in request, Quality Codes represents a kind of state of item.
For the historical data of item, its needs preserve for a long time, it is therefore desirable to be stored in archive, and the history trendline time of the timestamp of log history data, timestamp namely historical data.
Step S410, under the root achieved, inquire about preset index page allocation strategy, record block allocation strategy;
Wherein, described index page allocation strategy defines item ID and manipulative indexing page position, and described record block allocation strategy defines item ID and corresponding record block position, and described archive has divided according to page in advance, and described record block is made up of some pages.
Step S420, according to described record block allocation strategy, search and described target record block corresponding for item ID;
Specifically, if before the storage request of this item, having stored the historical data of identical item ID in archive, then the target record block found in this step is likely to stored have historical data. If before the storage request of this item, not storing the historical data of identical item ID in archive, then the target record block that this finds is idle block.
Step S430, described historical data is filled into described target record block;
Step S440, according to described index page allocation strategy, search and described target index page corresponding for item ID;
Similar with above-mentioned target record block, if before the storage request of this item, having stored the historical data of identical item ID in archive, then the target index page found in this step is likely to stored have index information and timestamp. If before the storage request of this item, not storing the historical data of identical item ID in archive, then the target index page that this finds is idle index page.
Step S450, by the index information of described target record block and described timestamp record at described target index page;
Step S460, judge whether described target index page is filled with, if so, perform step S470;
Specifically, if be filled with a certain target index page corresponding for item ID, then the process of the new index page of following establishment is performed.
Step S470, according to described index page allocation strategy, for the index page that the distribution of described item is new, and the index information of new index page is added in a upper index page.
Specifically, newly assigned index page is blank index page, and the index information of this index page will recorded in an index page. A upper index page described here, refers to the index page that history the last time is the distribution of described item.
By being added to by the index information of new index page in a upper index page, follow-up this index information that can pass through when data query searches the index page of association.
Optionally, above-described embodiment has been described above the allocation strategy of the first index page of item. Further, for the follow-up new index page distributed for described item, it can according to from achieving the mode that afterbody is sequentially allocated forward.
Further, if for the number of index page of some item distribution too much time, the considerable time will be expended during subsequent query. In order to solve this problem, itself first set the index page corresponding with item of the various embodiments described above introduction as one-level index page, namely one-level index page stores index information and the timestamp of record block. On this basis, the application is further introduced into the secondary index page of item, and secondary index page is for storing the index information of each one-level index page of item.
It is another date storage method flow chart being applied to intelligence grain depot disclosed in the embodiment of the present application referring to Fig. 5, Fig. 5.
As it is shown in figure 5, the method includes:
Step S500, receive item storage request;
Wherein, described storage request includes item ID, historical data and timestamp.Further, can also carrying Quality Codes in request, Quality Codes represents a kind of state of item.
For the historical data of item, its needs preserve for a long time, it is therefore desirable to be stored in archive, and the history trendline time of the timestamp of log history data, timestamp namely historical data.
Step S510, under the root achieved, inquire about preset index page allocation strategy, record block allocation strategy;
Wherein, described index page allocation strategy defines item ID and manipulative indexing page position, and described record block allocation strategy defines item ID and corresponding record block position, and described archive has divided according to page in advance, and described record block is made up of some pages.
Step S520, according to described record block allocation strategy, search and described target record block corresponding for item ID;
Specifically, if before the storage request of this item, having stored the historical data of identical item ID in archive, then the target record block found in this step is likely to stored have historical data. If before the storage request of this item, not storing the historical data of identical item ID in archive, then the target record block that this finds is idle block.
Step S530, described historical data is filled into described target record block;
Step S540, according to described index page allocation strategy, search and described target index page corresponding for item ID;
Similar with above-mentioned target record block, if before the storage request of this item, having stored the historical data of identical item ID in archive, then the target index page found in this step is likely to stored have index information and timestamp. If before the storage request of this item, not storing the historical data of identical item ID in archive, then the target index page that this finds is idle index page.
Step S550, by the index information of described target record block and described timestamp record at described target index page;
Step S560, judge whether described target index page is filled with, if so, perform step S470;
Specifically, if be filled with a certain target index page corresponding for item ID, then the process of the new index page of following establishment is performed.
Step S570, according to described index page allocation strategy, for the index page that the distribution of described item is new, and the index information of new index page is added in a upper index page;
Step S580, judging whether the number of the index page corresponding with described item reaches threshold value, wherein, the index page corresponding with described item is as the one-level index page of described item; If so, step S590 is performed;
Step S590, for described item create secondary index page, described secondary index page records the index information of the one-level index page of rheme number to some extent.
The application, when the number determining one-level index page that item is corresponding exceedes threshold value, creates the secondary index page of item further, is write in secondary index page by the index information of the one-level index page of item. During the historical data of follow-up lookup item, it is possible to by traveling through the secondary index page of item, find the index information of all one-level index pages of item, and then find the one-level index page of correspondence according to these index informations, be greatly improved efficiency data query.
Optionally, owing to the first one-level index page of item is positioned at the head position of archive, therefore to be easy to inquiry, on the basis of the above, the first one-level index page of the secondary index page of described item Yu described item can also be carried out place-exchange by the application further.That is, the secondary index page of item to be adjusted the head position achieved, so easily facilitate when data query and directly find secondary index page.
The data storage device below the embodiment of the present application provided is described, and data storage device described below and above-described date storage method can mutually to should refer to.
It is a kind of data storage device structural representation being applied to intelligence grain depot disclosed in the embodiment of the present application referring to Fig. 6, Fig. 6.
As shown in Figure 6, this device includes:
Storage request reception unit 61, for receiving the storage request of item, described storage request includes item ID, historical data and timestamp;
Root query unit 62, for inquiring about preset index page allocation strategy, record block allocation strategy under the root achieved, wherein, described index page allocation strategy defines item ID and manipulative indexing page position, described record block allocation strategy defines item ID and corresponding record block position, described archive has divided according to page in advance, and described record block is made up of some pages;
Target record block searches unit 63, for according to described record block allocation strategy, searching and described target record block corresponding for item ID;
Historical data fills unit 64, for described historical data is filled into described target record block;
Target index page searches unit 65, for according to described index page allocation strategy, searching and described target index page corresponding for item ID;
Target index page record unit 66, for by the index information of described target record block and described timestamp record at described target index page.
What the embodiment of the present application provided is applied to the data storage device of intelligence grain depot, prespecified index page allocation strategy and record block allocation strategy, determine record block and index page according to above-mentioned two strategy during storage item, store historical data in record block, index page stores index information and the timestamp of record block, according to item ID and index page allocation strategy and record block allocation strategy, historical data corresponding to item ID can be inquired about during subsequent query item historical data. Owing to the application need not record the corresponding relation between the index information of each item ID and record block in archive, save the space of archive, improve the storage capacity of data base.
Optionally, the embodiment of the present application also discloses historical data and fills the optional structure of one of unit, and historical data is filled unit and be may include that
Lossless compression unit, for described historical data is carried out real non-destructive compression, obtains Real Time Compression data;
Real Time Compression data fill unit, for being filled in the record page of buffer memory by described Real Time Compression data;
Record page condition adjudgement unit, for judging whether the occupied state of described record page reaches preset occupied state;
Static Compression unit, for when the judged result of described record page condition adjudgement unit is for being, carrying out static text compression by described record page, obtain compression blocks;
Compression blocks fills unit, for being filled in described target record block by described compression blocks.
Optionally, on above-described embodiment basis, the data storage device being applied to intelligence grain depot of the application can further include:
Target index page judging unit, is used for judging whether described target index page is filled with;
New index page allocation units, for when the judged result of described target index page judging unit is for being, according to described index page allocation strategy, distributing new index page for described item, and be added in a upper index page by the index information of new index page.
Optionally, on above-described embodiment basis, the data storage device being applied to intelligence grain depot of the application can further include:
Index page number judging unit, for judging whether the number of the index page corresponding with described item reaches threshold value, wherein, the index page corresponding with described item is as the one-level index page of described item;
Secondary index page creating unit, for when the judged result of described index page number judging unit is for being, creating secondary index page for described item, record the index information of the one-level index page of rheme number to some extent in described secondary index page.
Optionally, on above-described embodiment basis, the data storage device being applied to intelligence grain depot of the application can further include:
Index page place-exchange unit, for carrying out place-exchange by the secondary index page of described item and the first one-level index page of described item.
Finally, it can further be stated that, in this article, the relational terms of such as first and second or the like is used merely to separate an entity or operation with another entity or operating space, and not necessarily requires or imply the relation that there is any this reality between these entities or operation or sequentially. And, term " includes ", " comprising " or its any other variant are intended to comprising of nonexcludability, so that include the process of a series of key element, method, article or equipment not only include those key elements, but also include other key elements being not expressly set out, or also include the key element intrinsic for this process, method, article or equipment. When there is no more restriction, statement " including ... " key element limited, it is not excluded that there is also other identical element in including the process of described key element, method, article or equipment.
In this specification, each embodiment adopts the mode gone forward one by one to describe, and what each embodiment stressed is the difference with other embodiments, between each embodiment identical similar portion mutually referring to.
Described above to the disclosed embodiments, makes professional and technical personnel in the field be capable of or uses the application. The multiple amendment of these embodiments be will be apparent from for those skilled in the art, and generic principles defined herein when without departing from spirit herein or scope, can realize in other embodiments. Therefore, the application is not intended to be limited to the embodiments shown herein, and is to fit to the widest scope consistent with principles disclosed herein and features of novelty.
Claims (10)
1. the date storage method being applied to intelligence grain depot, it is characterised in that including:
Receiving the storage request of item, described storage request includes item ID, historical data and timestamp;
Preset index page allocation strategy, record block allocation strategy is inquired about under the root achieved, wherein, described index page allocation strategy defines item ID and manipulative indexing page position, described record block allocation strategy defines item ID and corresponding record block position, described archive has divided according to page in advance, and described record block is made up of some pages;
According to described record block allocation strategy, search and described target record block corresponding for item ID;
Described historical data is filled into described target record block;
According to described index page allocation strategy, search and described target index page corresponding for item ID;
By the index information of described target record block and described timestamp record at described target index page.
2. method according to claim 1, it is characterised in that described described historical data is filled into described target record block, including:
Described historical data is carried out real non-destructive compression, obtains Real Time Compression data;
Described Real Time Compression data are filled in the record page of buffer memory;
Judge whether the occupied state of described record page reaches preset occupied state, if so, then described record page is carried out static text compression, obtain compression blocks;
Described compression blocks is filled in described target record block.
3. method according to claim 1 and 2, it is characterised in that also include:
Judge whether described target index page is filled with;
If so, according to described index page allocation strategy, distribute new index page for described item, and the index information of new index page is added in a upper index page.
4. method according to claim 3, it is characterised in that also include:
Judging whether the number of the index page corresponding with described item reaches threshold value, wherein, the index page corresponding with described item is as the one-level index page of described item;
If so, it is that described item creates secondary index page, described secondary index page records the index information of the one-level index page of rheme number to some extent.
5. method according to claim 4, it is characterised in that also include:
The first one-level index page of the secondary index page of described item Yu described item is carried out place-exchange.
6. the data storage device being applied to intelligence grain depot, it is characterised in that including:
Storage request reception unit, for receiving the storage request of item, described storage request includes item ID, historical data and timestamp;
Root query unit, for inquiring about preset index page allocation strategy, record block allocation strategy under the root achieved, wherein, described index page allocation strategy defines item ID and manipulative indexing page position, described record block allocation strategy defines item ID and corresponding record block position, described archive has divided according to page in advance, and described record block is made up of some pages;
Target record block searches unit, for according to described record block allocation strategy, searching and described target record block corresponding for item ID;
Historical data fills unit, for described historical data is filled into described target record block;
Target index page searches unit, for according to described index page allocation strategy, searching and described target index page corresponding for item ID;
Target index page record unit, for by the index information of described target record block and described timestamp record at described target index page.
7. device according to claim 6, it is characterised in that described historical data is filled unit and included:
Lossless compression unit, for described historical data is carried out real non-destructive compression, obtains Real Time Compression data;
Real Time Compression data fill unit, for being filled in the record page of buffer memory by described Real Time Compression data;
Record page condition adjudgement unit, for judging whether the occupied state of described record page reaches preset occupied state;
Static Compression unit, for when the judged result of described record page condition adjudgement unit is for being, carrying out static text compression by described record page, obtain compression blocks;
Compression blocks fills unit, for being filled in described target record block by described compression blocks.
8. the device according to claim 6 or 7, it is characterised in that also include:
Target index page judging unit, is used for judging whether described target index page is filled with;
New index page allocation units, for when the judged result of described target index page judging unit is for being, according to described index page allocation strategy, distributing new index page for described item, and be added in a upper index page by the index information of new index page.
9. device according to claim 8, it is characterised in that also include:
Index page number judging unit, for judging whether the number of the index page corresponding with described item reaches threshold value, wherein, the index page corresponding with described item is as the one-level index page of described item;
Secondary index page creating unit, for when the judged result of described index page number judging unit is for being, creating secondary index page for described item, record the index information of the one-level index page of rheme number to some extent in described secondary index page.
10. device according to claim 9, it is characterised in that also include:
Index page place-exchange unit, for carrying out place-exchange by the secondary index page of described item and the first one-level index page of described item.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510989224.4A CN105653613A (en) | 2015-12-24 | 2015-12-24 | Data storage method and device applied to intelligent grain depot |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510989224.4A CN105653613A (en) | 2015-12-24 | 2015-12-24 | Data storage method and device applied to intelligent grain depot |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105653613A true CN105653613A (en) | 2016-06-08 |
Family
ID=56476848
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510989224.4A Pending CN105653613A (en) | 2015-12-24 | 2015-12-24 | Data storage method and device applied to intelligent grain depot |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105653613A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101169628A (en) * | 2007-11-14 | 2008-04-30 | 中控科技集团有限公司 | Data storage method and device |
| CN102446183A (en) * | 2010-10-12 | 2012-05-09 | 上海可鲁系统软件有限公司 | Industrial data storage method |
| CN103853499A (en) * | 2012-11-30 | 2014-06-11 | 上海可鲁系统软件有限公司 | Multi-source data storing and reading method |
| CN103914449A (en) * | 2012-12-29 | 2014-07-09 | 上海可鲁系统软件有限公司 | Multisource time series data compression storage method |
-
2015
- 2015-12-24 CN CN201510989224.4A patent/CN105653613A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101169628A (en) * | 2007-11-14 | 2008-04-30 | 中控科技集团有限公司 | Data storage method and device |
| CN102446183A (en) * | 2010-10-12 | 2012-05-09 | 上海可鲁系统软件有限公司 | Industrial data storage method |
| CN103853499A (en) * | 2012-11-30 | 2014-06-11 | 上海可鲁系统软件有限公司 | Multi-source data storing and reading method |
| CN103914449A (en) * | 2012-12-29 | 2014-07-09 | 上海可鲁系统软件有限公司 | Multisource time series data compression storage method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| TWI603211B (en) | Construction of inverted index system based on Lucene, data processing method and device | |
| CN102467572B (en) | Data block inquiring method for supporting data de-duplication program | |
| US7650326B2 (en) | Fast aggregation of compressed data using full table scans | |
| CN101656094B (en) | Data storage method and storage device | |
| US20150261466A1 (en) | Device and method for storing data in distributed storage system | |
| CN102799598A (en) | Data recovery method for deleting repeated data | |
| CN103635887B (en) | Data caching method and storage system | |
| CN104715039A (en) | Column-based storage and research method and equipment based on hard disk and internal storage | |
| CN103984640A (en) | Method and device for realizing data prefetching | |
| CN103744913A (en) | Database retrieval method based on search engine technology | |
| CN103500224B (en) | A kind of method for writing data and device, method for reading data and device | |
| CN102708183A (en) | Method and device for data compression | |
| CN108446399B (en) | Dynamic storage optimization method for structured massive real-time data | |
| JP4233564B2 (en) | Data processing apparatus, data processing program and recording medium | |
| CN112579714A (en) | Data query method, device, equipment and machine-readable storage medium | |
| CN111611250A (en) | Data storage device, data query method, data query device, server and storage medium | |
| CN105159987A (en) | Data storage and query method and apparatus | |
| CN101770474A (en) | History searching record-based searching method and device | |
| CN101770504A (en) | Data storage method, data reading method, data storage equipment and data reading equipment | |
| CN105653613A (en) | Data storage method and device applied to intelligent grain depot | |
| CN106909623B (en) | A kind of data set and date storage method for supporting efficient mass data to analyze and retrieve | |
| CN109753505B (en) | Method and system for creating temporary storage unit in big data storage system | |
| US6654867B2 (en) | Method and system to pre-fetch compressed memory blocks using pointers | |
| CN108647243B (en) | Industrial big data storage method based on time series | |
| WO2024021491A1 (en) | Data slicing method, apparatus and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C41 | Transfer of patent application or patent right or utility model | ||
| CB03 | Change of inventor or designer information |
Inventor after: Chen Ting Inventor after: Yao Jie Inventor after: Zheng Leiluo Inventor after: Ruan Zhijian Inventor after: Li Dewen Inventor before: Chen Ting Inventor before: Zheng Leiluo Inventor before: Ruan Zhijian Inventor before: Li Dewen |
|
| COR | Change of bibliographic data | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20160930 Address after: High tech Zone technology two road 710075 Shaanxi city of Xi'an province 01 Plaza No. 72 floor C101 Applicant after: Central control technology (Xi'an) Co., Ltd. Address before: 310053 Hangzhou Province, Binjiang District Province, No. six and No. 309 Road, the center of science and Technology Park (high tech Zone) () Applicant before: Zhongkong Technology Co., Ltd., Zhejiang |
|
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160608 |
|
| RJ01 | Rejection of invention patent application after publication |