CN102779180B - The operation processing method of data-storage system, data-storage system - Google Patents
The operation processing method of data-storage system, data-storage system Download PDFInfo
- Publication number
- CN102779180B CN102779180B CN201210222998.0A CN201210222998A CN102779180B CN 102779180 B CN102779180 B CN 102779180B CN 201210222998 A CN201210222998 A CN 201210222998A CN 102779180 B CN102779180 B CN 102779180B
- Authority
- CN
- China
- Prior art keywords
- data
- block
- index
- identification
- write
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the present invention provides a kind of operation processing method of data-storage system, data-storage system.The memory disk that the method comprises data-storage system comprises Hash table bucket and index block, receives data manipulation process request message, carries Data Identification in data manipulation process request message; From Hash table, corresponding index block is obtained according to Data Identification, the mapping relations of Data Identification and index block are stored in Hash table, store index record in each index block, in index record, include the mapping relations of the data block of Data Identification and storage Data Identification institute identification data; The mark of the data block storing Data Identification institute identification data is obtained from the index record of index block; Corresponding data manipulation process is carried out to the data block of the mark correspondence of data block.The embodiment of the present invention can by arranging Hash table bucket and index block in memory disk, make the method for operating of this system simplify further, and then can the access performance of raising disk to a certain degree.
Description
Technical field
The present invention relates to technical field of data storage, particularly relate to a kind of operation processing method of data-storage system, data-storage system.
Background technology
Network-caching (Web Cache) needs to carry out buffer memory to hot spot data, and the webserver (Web Server) needs mass data storage, and a large amount of Web page and video file also need to carry out high-performance storage.And a ring of the key index of Consumer's Experience web access is data storage fast, index, reading.
In currently available technology, the 3rd extended file system (Third extended file system, Ext3) is file system general, common in linux system, and it can be utilized to store key word (Key) data of magnanimity.Wherein, the storage organization of Ext3 catalogue is similar to tree-shaped, and the index technology of Ext3 catalogue determines this file is placed on which page node of Htree according to the hash value of each directory entry title.The Htree structure of each catalogue file comprises index block or directory entry block, and each index entry comprises the logical pointer of a hash key value and a sensing directory entry block, and directory entry block comprises normal directory entry (directory entry) list.Ext3 supports one-level or second-level directory index block, when root (root) index block is full, will become secondary index from one-level index.Therefore, just be provided with the mode of indirect index to the mapping of the data in magnetic disk block of deposit data file data by Ext3 file: an indirect index, first find index block, then by index data in index block, find the block number storing data, just can navigate to actual DSB data store block; Secondary indirect index, first finds secondary indirect index block, therefrom obtains an index block, obtains the block number of index block, finally by the block number of this index block, then obtain the data of index block, thus navigate to the position of data in this index block.During locating file, travel through all records under it, and compare with the filename that will search in catalogue like this, when title is identical, then search successfully.
But, if file directory is very dark or file a lot (as more than 1000) under catalogue, during access file will through the catalogue of multiple level read and matching ratio under last catalogue comparatively, access performance can be very poor.Therefore, time in prior art by form of catalogue access disk, there is the defect of access performance difference.
Summary of the invention
The embodiment of the present invention provides a kind of operation processing method of data-storage system, data-storage system, can improve disk access performance.
The embodiment of the present invention provides a kind of operation processing method of data-storage system on the one hand, and the memory disk of data-storage system comprises Hash table, data block and index block, and method comprises:
Receive data manipulation process request message, in data manipulation process request message, carry Data Identification;
From Hash table, corresponding index block is obtained according to Data Identification, the mapping relations of Data Identification and index block are stored in Hash table, store index record in each index block, in index record, include the mapping relations of the data block of Data Identification and storage Data Identification institute identification data;
The mark of the data block storing Data Identification institute identification data is obtained from the index record of index block;
Corresponding data manipulation process is carried out to the data block of the mark correspondence of data block.
The embodiment of the present invention provides a kind of data-storage system on the other hand, the memory disk of data-storage system comprises Hash table, index block and data block, the mapping relations of Data Identification and index block are stored in Hash table, index record is stored in each index block, include the mapping relations that Data Identification and data table memory show the data block of institute's identification data in index record, system also comprises:
Interface unit, for receiving data manipulation process request message, carries Data Identification in data manipulation process request message;
First acquiring unit, is connected with interface unit, for obtaining corresponding index block from Hash table according to Data Identification, and sends to second acquisition unit;
Second acquisition unit, is connected with the first acquiring unit, for obtaining the mark of the data block storing Data Identification institute identification data from the index record of index block, and sends to operational processes unit;
Operational processes unit, is connected with second acquisition unit, and the data block for the mark correspondence to data block carries out corresponding data manipulation process.
In the above embodiment of the present invention, by comprising Hash table bucket, data block and index block at the memory disk of data-storage system, thus make the method for operating of this system for first receiving data manipulation process request message, carry Data Identification in data manipulation process request message; From the Hash table Hash table bucket, corresponding index block is obtained again according to Data Identification, the mapping relations of Data Identification and index block are stored in Hash table, store index record in each index block, in index record, include the mapping relations of the data block of Data Identification and storage Data Identification institute identification data; Then from the index record of index block, obtain the mark of the data block storing Data Identification institute identification data.Access disk relative in prior art by the form of catalogue, effectively can improve access speed, improve the access performance of disk.
Accompanying drawing explanation
In order to be illustrated more clearly in the embodiment of the present invention or technical scheme of the prior art, be briefly described to the accompanying drawing used required in embodiment or description of the prior art below, apparently, accompanying drawing in the following describes is some embodiments of the present invention, for those of ordinary skill in the art, under the prerequisite not paying creative work, other accompanying drawing can also be obtained according to these accompanying drawings.
The process flow diagram of the operation processing method of a kind of data-storage system that Fig. 1 provides for the embodiment of the present invention;
The process flow diagram of the method for writing data of a kind of data-storage system that Fig. 2 provides for the embodiment of the present invention;
The schematic diagram of the data cover writing in files of a kind of data-storage system that Fig. 3 provides for the embodiment of the present invention;
The schematic diagram of the data cover wiring method of a kind of data-storage system that Fig. 4 provides for the embodiment of the present invention;
The process flow diagram of the method for reading data of a kind of data-storage system that Fig. 5 provides for the embodiment of the present invention;
Fig. 6 A is the schematic diagram one that in the embodiment of the present invention, data block reads;
Fig. 6 B is the schematic diagram two that in the embodiment of the present invention, data block reads;
The process flow diagram of the data-erasure method of a kind of data-storage system that Fig. 7 provides for the embodiment of the present invention;
The framework schematic diagram of a kind of data-storage system that Fig. 8 provides for the embodiment of the present invention;
The framework schematic diagram of the another kind of data-storage system that Fig. 9 provides for the embodiment of the present invention;
A kind of data-storage system operation chart that Figure 10 provides for the embodiment of the present invention;
The magnetic disk figure of a kind of data-storage system that Figure 11 provides for the embodiment of the present invention;
The structural representation of processing unit is write in a kind of data-storage system that Figure 12 provides for the embodiment of the present invention.
Embodiment
For making the object of the embodiment of the present invention, technical scheme and advantage clearly, below in conjunction with the accompanying drawing in the embodiment of the present invention, technical scheme in the embodiment of the present invention is clearly and completely described, obviously, described embodiment is the present invention's part embodiment, instead of whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art, not making the every other embodiment obtained under creative work prerequisite, belong to the scope of protection of the invention.
The process flow diagram of the operation processing method of a kind of data-storage system that Fig. 1 provides for the embodiment of the present invention.As shown in Figure 1, the memory disk of data-storage system comprises Hash table bucket, data block and index block, and the operation processing method of this data-storage system comprises:
Step 101, reception data manipulation process request message, carry Data Identification in data manipulation process request message.
The Data Identification carried in data manipulation process request message in the present embodiment is the unique identifications of data in memory disk, can represent with key word key, and also can be expressed as Hash Round Robin data partition id, this is not restricted.
Corresponding index block is obtained in step 102, the Hash table that arranges from Hash table bucket according to Data Identification, the mapping relations of Data Identification and index block are stored in Hash table, store index record in each index block, in index record, include the mapping relations of the data block of Data Identification and storage Data Identification institute identification data.
Hash table is stored in Hash table bucket in the mode showing bucket, and Hash table stores the information of index block, the information needed when namely searching index block.Specifically, Hash table, also referred to as hash table, is a kind of data structure directly conducted interviews according to Data Identification.That is, visiting record by Data Identification being mapped to a position in Hash table, to accelerate the speed of searching, wherein, the address of index block in the record of this position, can be included.Hash table be by Hash calculation by larger deposit data in relatively little space, and the space of Hash table is relatively little, can produce on position that two different Data Identifications are mapped to same Hash table and produce conflict (collision), open chain method can be adopted to solve, specifically, be exactly from the disk management module of bitmap region, apply for the storage space that a free block is shown as new Hash, and existing Hash watch chain is received new Hash table, wherein, all index records in free block are all on conflict chain, conflict chain can be the form of chained list, and each node physics is discontinuous, also can be continuous print.The conflict solving Hash table in the present embodiment is not limited to this, can by other data structure as B tree removes Organization Conflicts record, other method that can solve the conflict of Hash table that every those skilled in the art use.
Needs illustrate, according to key word key application hash function HashFun1, that is: hash value index=HashFunc1 (key), the address of corresponding index block can be obtained, if do not find corresponding index block address in the index block just searched, the index block that can continue to inquire about backward until find, or returns and inquires about unsuccessfully.Index block saves the index information of all records of this hash value total, and when the space of an index block stores index record not, then can produce conflict, and the present embodiment uses open chain method to manage conflict.From the disk management module of bitmap region, specifically apply for that a free block is as new index block, and existing index block and new index block are linked.When disk space is enough, unlimited many index records can be stored by the mode of this unidirectional series connection.
Fig. 6 A is the schematic diagram one that in the embodiment of the present invention, data block reads.Fig. 6 B is the schematic diagram two that in the embodiment of the present invention, data block reads.As shown in Figure 6 A and 6 B, index block can comprise as lower area: head zone, series connection pointer, minimax id, posting field, it should be noted that posting field can have multiple, such as posting field 1 is until posting field n, the region that in the present embodiment, index block comprises not as limit, index block can comprise above-mentioned zone several, all or on this basis increase other region again.Wherein, head zone preserves the essential information of this index block, such as: the size of single record, total record number, actual record number and the reference position be recorded in index block, the present embodiment is not limited with cited essential information.Series connection pointer, preserves the information of next index block.Minimax id, refers to the position range of the index record obtained after key word Key application hash function HashFunc2 at this index block.Wherein, minimax id is that index location uses, and can determine index record fast whether in this index block.Posting field preserves index records all in this index block, mainly comprise data in the Hash Round Robin data partition id of data block, data in the scope of data block, namely positional information, the data block number of index block, also the Hash Round Robin data partition id of this index record or the out of Memory corresponding to key word (key) can be included, as the information of creation-time, modification time, erasing time or user.
Step 103, from the index record of index block, obtain the mark of the data block storing Data Identification institute identification data.
That is, from the index record of index block, search the mark of the data block storing Data Identification, the mark of data block can be the address of data block.Index record can be searched according to the mode of certain data structure storage, and as sort algorithm, binary tree, what use certain algorithm searches the number of times that can reduce and mate in search procedure, thus accelerates inquiry.Because an index block can preserve a lot of index record, if apply certain algorithm to search coupling to index record, when inserting, deleting or read data, the frequent operation to data block can be reduced.
Step 104, corresponding data manipulation process is carried out to the data block of the mark correspondence of data block.
Carry out corresponding data manipulation process to data block in the present embodiment, can comprise: read data, write data and delete data, wherein, write data type can include: new write, covering are write and added and write.When reading data with write data manipulation parallel processing, by the judgement to write type, can judge whether write data operate with reading data in same data block, if not, parallel work-flow can read data and service data, improve read-write concurrency performance.
The present embodiment is by being divided into the region comprising Hash table bucket, data block and index block by the memory disk of data-storage system, the method of operating of this system can be simplified, first receive data manipulation process request message, in data manipulation process request message, carry Data Identification; From the Hash table Hash table bucket, corresponding index block is obtained again according to Data Identification, the mapping relations of Data Identification and index block are stored in Hash table, store index record in each index block, in index record, include the mapping relations of the data block of Data Identification and storage Data Identification institute identification data; Then from the index record of index block, obtain the mark of the data block storing Data Identification institute identification data; Thus the step of this system operation methods is simplified, and then can the access performance of raising disk to a certain degree.
The present embodiment, optionally, data-storage system can also comprise buffer zone, and the data of partial data block are mainly arranged in internal memory by cache management, can directly access from internal memory when conducting interviews, convenient access, thus improves access speed.After reception data manipulation process request message, first from the Hash table of Hash bucket, the address of corresponding index block is obtained in the buffer according to Data Identification, concrete steps are identical with step 102 in above-described embodiment, no longer illustrate at this, distinguish from above-described embodiment step 102 and be that the Hash table bucket present position corresponding with Data Identification is different, the Hash table bucket of the present embodiment is positioned at buffer zone, that is, be obtain corresponding index block information according to Data Identification from the Hash table of the Hash table bucket being arranged in buffer zone in the present embodiment.Then, the mark of the data block storing Data Identification institute identification data is obtained from the index record of index block, no longer illustrate at this with the same section of above-described embodiment step 103, difference is that index block is arranged in buffer zone, specifically, be the mark obtaining the data block storing Data Identification institute identification data from the index record of the index block being arranged in buffer zone.Partial data block is arranged in the buffer, can conveniently access, and for solid state hard disc (solid state disk, SSD), number of operations too much can affect its serviceable life, such as, when write operation, first can be written in buffer zone, after certain number of times, be synchronized to disk again, which reduce the number of times writing disk, also can improve the life-span of disk simultaneously.
The process flow diagram of the method for writing data of a kind of data-storage system that Fig. 2 provides for the embodiment of the present invention; As shown in Figure 2, the method for writing data of data-storage system comprises: receive data manipulation process request message, wherein, data manipulation process request message can be write data requests message, off-set value (offset) and data message (data) is also carried in write data requests message, wherein, the memory range that can also obtain data with existing in data block from the index record of index block can also be included in write data requests message, corresponding data manipulation process is carried out to the data block of the mark correspondence of data block.Corresponding data manipulation process judges according to off-set value the write type that write data requests message is corresponding, and write type comprises covering and writes, adds and write or newly write.Wherein, covering is write and is referred at data with existing region write new data; Add to write and refer to write data after data with existing region; Other situation is then new write.The method for writing data concrete steps of data-storage system comprise:
Step 201, reception write data requests message.
Wherein, off-set value (offset) and data message data, key word (key) can be carried in write data requests message, write the size (size) of data, as write (key, offset, size, data), or only carry data message, be namely expressed as write (data).
Step 202, from the Hash table of Hash table bucket, obtain corresponding index block according to Data Identification, namely, according to key word key application hash function HashFun1, that is: index=HashFunc1 (key), the Hash Round Robin data partition of corresponding index block can be obtained.
Step 203, from the index record of index block, obtain the mark of the data block storing Data Identification institute identification data, can according to key word key application hash function HashFun2, the Hash Round Robin data partition id of a generation index record.Needs illustrate, when designing Hash table, can according to Message Digest Algorithm 5 (Message Digest Algorithm, MD5) or Secure Hash Algorithm (SHA1), the probability that the Hash Round Robin data partition id of the index record that application hash function HashFunc2 is obtained does not conflict or conflicts is smaller.
Step 204, the Hash Round Robin data partition id of newly-generated index record to be mated with index record, that is, the Hash Round Robin data partition id of the index record obtained by application hash function HashFunc2 in index record mates with index record, sort algorithm or binary tree can be adopted, be not limited in index record the method for searching data block identifier at this.Enter step 205 if the match is successful, otherwise it fails to match, return failed message.
Step 205, judge according to off-set value the write type that write data requests message is corresponding.Can according to side-play amount (offset) within the data block the length range (range) of data situation judge write type.Specifically comprise:
If among the length range (range) of side-play amount (offset) data within the data block, then the write type that write request message is corresponding is covering when writing, and enters step 2051;
If after the length range (range) of side-play amount (offset) data within the data block, then the write type that write request message is corresponding is write for adding, and enters step 206;
Other is then new write, enters step 206.
Judge that write type is covering when writing, data message be written to corresponding data block and comprise step:
Step 2051, the version number generating a new data block and data block address id, according to new version number and key word key, application HashFun2, generates the new data block address id that a preparation carries out covering write operation.
Whether step 2052, the data of searching this data block address id exist, if have data in the data block of this data block address id, then return step 2051, regeneration new version number, and according to new version number and key word key, application HashFun2 generates new data block address id, until do not store any data in the data block of searching this data block address id; If existed without any data in the id data block of this address, then enter next step 2053;
Step 2053, to the new data block of disk management module application one, then by original block copy in new data block, wherein, disk management module is positioned at bitmap region.
Step 2054, original data block is removed to recovery zone, wherein, the recovery zone for storing deleted data block is arranged in memory disk, also includes the bitmap for distributing data block in memory disk.
The mark of step 2055, return data block, the mark of data block can include new version number.
Step 206, data message to be written in corresponding data block according to write type.
The present embodiment obtains corresponding index block according to the Data Identification of write data requests message from Hash table, then from the index record of index block, obtain the mark of the data block storing Data Identification institute identification data, again by data writing data blocks, wherein wiring method comprises: covering is write, newly write and add and write.In the method for writing data of this data-storage system, can just find corresponding index block by once searching, thus the method for operating of this system can be simplified, and then the access performance of disk can be improved.
The schematic diagram of the data cover writing in files of a kind of data-storage system that Fig. 3 provides for the embodiment of the present invention; The schematic diagram of the data cover wiring method of a kind of data-storage system that Fig. 4 provides for the embodiment of the present invention.As shown in Figure 3 and Figure 4, original comprises four data block block1, block2, block3 and block4, during by the former data of new data cover data block block3, first according to the Hash Round Robin data partition id matched in index record, search in index record and need to carry out covering the data block write, the data block number namely in the position range of all data blocks of index record storage in match index record.Change in units of whole data block, the data block Block3 ' that application one is new in disk management module, by original block copy in new data block Block3 ', new data is write again to data block Block3 ', and generate new version number, legacy data block block3 is moved to recovery zone simultaneously.The covering literary style of such enforcement, can not change remainder data block, and be kept in application layer by newly-generated version number, for application layer, can continue to use capped data, i.e. delayed deletion function.When rewriting the partial data of legacy data block block3, copy-on-write technique can meet following 4 points, one) capped original can normally use; Two) new writen data can not affect the operation of the data block to original file; Three) covering in the operation write, to needing, the data change of change is little, can not cause deterioration write performance; Four) only need application to need the data block of change, space hold is few.And concerning user, comprise block1,2,4 and new data block block3 ' to the reading of new file, comparatively speaking, new data block block3 ' is new application, can reduce taking and writing expense of data space.And legacy data block3 in the past also can continue accessed.
The process flow diagram of the method for reading data of a kind of data-storage system that Fig. 5 provides for the embodiment of the present invention.As shown in Fig. 5, Fig. 6 A and Fig. 6 B, data manipulation process request message is read data request message, i.e. read (key, offset, size), then data method comprises:
Step 501, reception read data request message.
Wherein, this message carries off-set value (offset), and the size of data of key word (key) and expectation reading, (size), as read (key, offset, size).
Step 502, from the Hash table of Hash table bucket, obtain corresponding index block according to Data Identification, that is, according to key word key application hash function HashFun1, that is: cryptographic hash index=HashFunc1 (key), the Hash Round Robin data partition of corresponding index block can be obtained, wherein, key word key is unique Data Identification in disk.
Step 503, from the index record of index block, obtain the mark of the data block storing Data Identification institute identification data, can according to key word key application hash function HashFun2, generate the Hash Round Robin data partition id of a new index record.
Step 504, judge the matching result of the Hash Round Robin data partition id of index record in index record with newly-generated, sort algorithm, binary tree can be adopted with the matching process of newly-generated new mark id, be not limited in index record the method for searching data block identifier at this.
Enter step 507 if the match is successful, otherwise enter step 505.
Step 505, enter recovery area and search.When not getting the mark of the data block storing Data Identification institute identification data from the index record of index block, from recovery zone, obtain the mark of the data block storing Data Identification institute identification data according to Data Identification.Recovery area records deleted object information, specifically comprises the mark of index and the storage data block of object, can use in follow-up reading.Recovery zone can be forced when Insufficient disk space to reclaim, and recovery zone can use least recently used algorithm (Least Recently Used, LRU) to judge to need the data block of complete deletion.
Step 506, judge the Hash Round Robin data partition id matching result of index record in recovery zone with newly-generated if the result of searching is mated with the Hash Round Robin data partition id of newly-generated index record, then to enter step 507; Otherwise return failure information.
Step 507, reading data, that is, read the data of the data block of mating with the Hash Round Robin data partition id of new index record, in recovery zone, Data Identification institute identification data is read from the data block of the mark correspondence of data block, also from the index record of index block, Data Identification institute identification data can be read from the data block of the mark correspondence of data block.
Step 508, to return success.
The present embodiment obtains corresponding index block according to the Data Identification of data read request message from Hash table, from the index record of index block, then obtains the mark of the data block storing Data Identification institute identification data, then operate accordingly data block.In the method for reading data of this data-storage system, can once search the index block just finding needs, thus the method for operating of this system can be simplified, and then can the access performance of raising disk to a certain degree.
The process flow diagram of the data-erasure method of a kind of data-storage system that Fig. 7 provides for the embodiment of the present invention; As shown in Figure 7, the step 601-step 604 of the data-erasure method of data-storage system is substantially identical with the step 501-step 504 in above-described embodiment, same section no longer illustrates, difference is, the data manipulation process request message that step 601 receives deletes data request information, wherein, this message carries data message (key), wherein, the key unique identification that to be key word be also in memory disk, deletes request of data and can be expressed as delete (key).Judge the matching result of new mark id in index record with newly-generated, enter step 605 if the match is successful, otherwise return failure information.
The result that step 605, basis are searched, removes to recovery zone by this data block.
Step 606, to return success.
The present embodiment obtains corresponding index block according to the Data Identification of data deletion request message from Hash table, from the index record of index block, then obtains the mark of the data block storing Data Identification institute identification data, then data block is removed to recovery zone.In the data-erasure method of this data-storage system, can once search the index block just finding needs, thus the method for operating of this system can be simplified, and then can the access performance of raising disk to a certain degree.
The configuration diagram of a kind of data-storage system that Fig. 8 provides for the embodiment of the present invention.As shown in Figure 8, the memory disk of data-storage system comprises Hash table bucket, index block and data block, wherein, the mapping relations of Data Identification and index block are stored in the Hash table arranged in Hash table bucket, store index record in each index block, in index record, include the mapping relations that Data Identification and data table memory show the data block of institute's identification data.Index block, preserves the map information of key word Key to data block, can provide the information of locator data block; Data block: the storage area of data.The minimum administrative unit of memory disk is data block, and the size of data block affects the access performance of memory disk.Therefore can the size of setting data block when formaing memory disk, the setting of this value size can specifically according to the mean size of storage object, the number of storage object and memory disk total volume, the size of Hash table bucket and data block is adjusted, to obtain a good performance.Such as solid state hard disc, the setting of data block size can be the integral multiple size of minimum erase units.In general, the loss of searching matching times and less management space that the size demand fulfillment of data block is less, in the present embodiment, data block is minimum is 4Kbytes, but not as limit.
The data-storage system of the present embodiment also comprises: interface unit 101, first acquiring unit 102, second acquisition unit 103 and operational processes unit 104.
Interface unit 101, for receiving write data requests message, data manipulation process request message can comprise, data read operation, data write operation and data deletion action, and accepting to carry Data Identification in the data manipulation process request message fallen, this Data Identification is information unique in data-storage system, other not identical therewith information.
First acquiring unit 102, be connected with interface unit 101, for obtaining corresponding index block from Hash table according to Data Identification, Hash table preserves the mapping relations of Data Identification and index block, sends to second acquisition unit 103 from Hash table after obtaining the information of index block.
Second acquisition unit 103, be connected with the first acquiring unit 102, for obtaining the mark of the data block storing Data Identification institute identification data from the index record of index block, wherein, index record is stored in each index block, include the mapping relations of the data block of Data Identification and storage Data Identification institute identification data in index record, obtain the mark of data block from index block after, send to operational processes unit 104.
Operational processes unit 104, is connected with second acquisition unit 103, and the data block for the mark correspondence to data block carries out corresponding data manipulation process.
The memory disk of the present embodiment data-storage system can comprise Hash table bucket, index block and data block, and this system also comprises interface unit 101, first acquiring unit 102, second acquisition unit 103 and operating unit 104.And then can by the Hash table arranged in Hash table bucket in the method for operating of this system, once search and just can find the index block corresponding with data request information, thus the method for operating of this system can be simplified, and then can the access performance of raising disk to a certain degree.
On the basis of technique scheme, optionally, data-storage system also comprises a basic and important module buffer zone, and specifically, buffer zone the data of partial data block is arranged on internal memory, to improve access speed.Magnetic disk and storage Organization of Data directly affect the setting of data in buffer zone.When interface unit 101 receives write data requests message, first by this Message Transmission in the Hash table in buffer zone, specifically, the first acquiring unit 102 obtains corresponding index block according to the Data Identification carried in write data requests message from the Hash table being arranged in buffer zone.Obtain the address of index block from Hash table after, second acquisition unit 103 obtains the mark of the data block storing Data Identification institute identification data from the index record of the index block being arranged in buffer zone.
The configuration diagram of the another kind of data-storage system that Fig. 9 provides for the embodiment of the present invention.As shown in Figure 9, on the basis of technique scheme, optionally, data-storage system can also comprise:
3rd acquiring unit 105, be connected with second acquisition unit 103, for being read data request message at data manipulation process request message, and second acquisition unit 103 when not getting the mark of the data block storing Data Identification institute identification data from the index record of index block, the mark of the data block storing Data Identification institute identification data can be obtained according to Data Identification from recovery zone, then the mark obtaining data block from index block is sent to operational processes unit 104; Specifically, operational processes unit 104 may be used for reading Data Identification institute identification data the data block of the mark correspondence of the data block obtained from second acquisition unit 103 or the 3rd acquiring unit 105.
Needs illustrate, the memory disk of the present embodiment is raw device disk, also referred to as raw partition (raw partition) equipment, that one is not through format, the block device that can not be read by file system in Unix/Linux operating system, data manipulation in raw device disk without the buffering of file system, directly can be operated it by application program.Raw device disk as an equipment manage by operating system.According to disk drive, system can carry out the practical operation of data.Interface unit 101 can provide and outside operation-interface, includes basic operation and bookkeeping, and wherein, basic operation has the deletion of the reading of data, the write of data and data, and bookkeeping can include format and load.
A kind of data-storage system operation chart that Figure 10 provides for the embodiment of the present invention.This operation of the present embodiment can be the deletion of the reading of data, the write of data and data as shown in Figure 10, and this is not restricted:
1, the file system opening raw device is called by interface unit;
2, reading request of data is sent to raw device management;
3, raw device management receives data are mated with to buffer data, and judges whether that the match is successful, directly returns from buffer zone, if it fails to match, enter step 4 if the match is successful;
4, raw device management sends request to disk drive, and calls the operation of file system;
5, disk drive locates concrete Disk Locality, and carries out write operation to the data disk of this position.
Needs illustrate, data read operation is identical with Fig. 9 flow process, and the step of data deletion action is identical with the step 1-step 3 in Figure 10 flow process, illustrates no longer one by one at this.
The magnetic disk figure of a kind of data-storage system that Figure 11 provides for the embodiment of the present invention.As shown in figure 11, memory disk can comprise: superblock, Hash show bucket, recovery zone, bitmap region, index block region, data block region.Wherein, superblock records the essential information of this memory disk, quantity, hash table size, the index block size of data block in size as total in: memory disk, service condition, memory disk, recovery zone size, bitmap block size, and the information such as index record size and index record number in index block.Bitmap region records the service condition of all data blocks, has namely used and untapped situation.Such as: for the disk of solid state hard disc, the record of bitmap region has influence on the allocation order of disk, also the serviceable life to disk is directly had influence on, therefore when accessing disk, can according to the order of disk block, according to order-assigned from front to back, can ensure that the write number of times of each disk block is more balanced like this, thus extend the life-span of solid state hard disc.For hard disk drive (Hard Disk Drive, HDD), because the impact of mechanical magnetic head, disk block can be distributed, to extend the life-span of hard disk drive in continuity ground.All the other regions illustrate in the above-described embodiments, and this is no longer going to repeat them.In the present embodiment, actual performance can be considered, and when disk initialization according to disk space, total record number, design the suitable size of hash table, the size of data block, to obtain best access performance.
Based on the above embodiments, the one or more combination in the admin instance of this disk region, superblock, Hash table, recovery zone, bitmap region, index block region, data block region in the buffer zone of data-storage system, can be included.Wherein, the admin instance of this disk region provides raw device supervisory routine to use.The data of the data block of having deleted can be kept at recovery zone by recovery zone, use for delayed deletion module.In the startup optimization process of raw device management, delayed deletion module comes into force, the data of having deleted can be accessed, but after restarting, this partial data can permanent delet, therefore recovery zone can be arranged on buffer zone, the present embodiment is not limited to this, also it can be arranged in disk.Wherein, the specific descriptions in superblock, Hash table, bitmap region, index block region are identical with the disk structure in the non-buffered district in above-described embodiment, no longer illustrate at this.The admin instance of this disk region, superblock, whole Hash can be shown data, bitmap, index block and data block partly and reside in buffer zone.Select the method residing in the partial data block of buffer zone, the algorithm of least recently used algorithm or other intelligent cache memory can be passed through.
At data-storage system, when carrying out read data operation, if the data needing to read directly can be chosen in the buffer, then direct from buffer zone reading data.When carrying out write data manipulation, if the data block needing write can be found in buffer zone, then data of writing direct in the buffer, and or the operation of replacing synchronous according to display by user again by the data syn-chronization of change in disk.Placed in the buffer by the data block of will often access, can access speed be accelerated.
The structural representation of processing unit is write in a kind of data-storage system that Figure 12 provides for the embodiment of the present invention.As shown in figure 12, when interface unit reception data manipulation process request message is write data requests message, off-set value (offset) and data message (data) can be carried in write data requests message, write data requests message can also comprise size of data size (size) and key word (key), from Hash table, corresponding index block is obtained according to data message, the memory range of data with existing in data block is obtained again from the index record of index block, wherein, the data of the length range (range) of data are carried in the information of data block, operational processes unit can according to the side-play amount (offset) in data request information within the data block data length range (range) judge write type, wherein operational processes unit comprises:
First judgment sub-unit 1041, for judging the write type that write data requests message is corresponding according to off-set value, write type comprises covering and writes, adds and write or newly write; Specifically, if among the scope (range) of side-play amount (offset) data length within the data block, then the write type that write request message is corresponding is that covering is when writing; If after the scope (range) of side-play amount (offset) data within the data block, then the write type that write request message is corresponding is write for adding; Other is then new write.
Write operation subelement 1043, for being written to data message in corresponding data block according to write type.Concrete grammar is identical with the method for above-described embodiment Fig. 3, no longer illustrates at this.
On the basis of technique scheme, optionally, size of data value is also carried in the write data requests message that interface unit receives, data-storage system also includes the second judging unit 1042, during for judging to determine that data to be written are in the space of data block according to off-set value and size of data value, return the failed mistake of write.That is, second judging unit 1042 can judge to determine that data to be written are in the locus of data block according to off-set value and size of data value, if when off-set value and size of data value exceed the spatial dimension of data block, then do not write any data, and return the failed mistake of write.If when off-set value and size of data value are in the spatial dimension of data block, then carry out write operation.
In the present embodiment, memory disk can also include the bitmap for distributing data block, and for storing the recovery zone of deleted data block, the write type that write request message is corresponding be covering write time, write operation subelement 1043 is specifically for removing to recovery zone by data block; From the disk management module of bitmap region, apply for new data block, and generate the mark of data block; Data message is written in new data block.The concrete methods of realizing of the present embodiment is identical with above-described embodiment step 2051-step 2054, and this is no longer going to repeat them.
On the basis of above-described embodiment, pre-fetch function can be increased.When reading the data, the mode indicating object that can show according to access module or fetch interface, reading disk block is to buffer zone in advance, when again reading request of data, directly can read from buffer zone, and then put forward the access high-performance of disk.
The data manipulation process request message of the present embodiment data-storage system is write data requests message, off-set value and data message is also carried in write data requests message, from the index record of index block, also obtain the memory range of data with existing in data block, operational processes unit includes the first judgment sub-unit 1041, write operation subelement 1043 and the second judging unit 104.Can by the Hash table arranged in Hash table bucket when the write data manipulation of this system, once search and just can find the index block corresponding with data request information, thus the method for operating of this system can be simplified, and then can the access performance of raising disk to a certain degree.
Based on above-described embodiment, by comprising Hash table bucket, data block and index block at the memory disk of data-storage system, thus make the method for operating of this system for first receiving data manipulation process request message, carry Data Identification in data manipulation process request message; From the Hash table Hash table bucket, corresponding index block is obtained again according to Data Identification, the mapping relations of Data Identification and index block are stored in Hash table, store index record in each index block, in index record, include the mapping relations of the data block of Data Identification and storage Data Identification institute identification data; Then from the index record of index block, obtain the mark of the data block storing Data Identification institute identification data; And then can the access performance of raising disk to a certain degree.
One of ordinary skill in the art will appreciate that: all or part of step realizing above-mentioned each embodiment of the method can have been come by the hardware that programmed instruction is relevant.Aforesaid program can be stored in a computer read/write memory medium.This program, when performing, performs the step comprising above-mentioned each embodiment of the method; And aforesaid storage medium comprises: ROM, RAM, magnetic disc or CD etc. various can be program code stored medium.
Last it is noted that above each embodiment is only in order to illustrate technical scheme of the present invention, be not intended to limit; Although with reference to foregoing embodiments to invention has been detailed description, those of ordinary skill in the art is to be understood that: it still can be modified to the technical scheme described in foregoing embodiments, or carries out equivalent replacement to wherein some or all of technical characteristic; And these amendments or replacement, do not make the essence of appropriate technical solution depart from the scope of various embodiments of the present invention technical scheme.
Claims (10)
1. an operation processing method for data-storage system, is characterized in that, the memory disk of described data-storage system comprises Hash table bucket, data block and index block, and described method comprises:
Receive data manipulation process request message, in described data manipulation process request message, carry Data Identification;
According to obtaining corresponding index block in the Hash table that described Data Identification is arranged from described Hash table bucket, the mapping relations of Data Identification and described index block are stored in described Hash table, store index record in each index block, in described index record, include the mapping relations of the data block of described Data Identification and storage described Data Identification institute identification data;
The mark of the data block storing described Data Identification institute identification data is obtained from the index record of described index block;
Corresponding data manipulation process is carried out to the data block of the mark correspondence of described data block;
Described data manipulation process request message is write data requests message, off-set value and data message is also carried in write data request message, from the index record of index block, also obtain the memory range of data with existing in described data block, the data block of the described mark correspondence to data block is carried out corresponding data manipulation process and is comprised:
Judge that the write type that write data request message is corresponding, said write type comprise covering and write, add and write or newly write according to described off-set value;
According to said write type, described data message is written in corresponding data block;
Described memory disk also includes the bitmap for distributing data block, and for storing the recovery zone of deleted data block, the write type that said write request message is corresponding be covering write time, describedly described data message is written to corresponding data block according to write type and comprises:
Described data block is removed to recovery zone;
From bitmap, apply for new data block, and generate the mark of described data block;
Described data message is written in new data block.
2. the operation processing method of data-storage system according to claim 1, is characterized in that, described data-storage system also comprises buffer zone, describedly from Hash table, obtains corresponding index block according to Data Identification and is specially:
Corresponding index block is obtained from the Hash table being arranged in buffer zone according to described Data Identification;
The described mark obtaining the data block storing described Data Identification institute identification data from the index record of index block is specially:
The mark of the data block storing described Data Identification institute identification data is obtained from the index record of the index block being arranged in buffer zone.
3. the operation processing method of data-storage system according to claim 1 and 2, is characterized in that, described data manipulation process request message is read data request message, and described method also comprises:
When not getting the mark of the data block storing described Data Identification institute identification data from the index record of described index block, from recovery zone, obtain the mark of the data block storing described Data Identification institute identification data according to described Data Identification;
The data block of the described mark correspondence to described data block is carried out corresponding data manipulation process and is comprised:
Described Data Identification institute identification data is read from the data block of the mark correspondence of described data block.
4. a data-storage system, it is characterized in that, the memory disk of described data-storage system comprises Hash table bucket, index block and data block, the mapping relations of Data Identification and described index block are stored in the Hash table arranged in described Hash table bucket, index record is stored in each index block, include the mapping relations of the data block of described Data Identification and storage described data representation institute identification data in described index record, described system also comprises:
Interface unit, for receiving data manipulation process request message, carries Data Identification in described data manipulation process request message;
First acquiring unit, is connected with described interface unit, for obtaining corresponding index block from Hash table according to described Data Identification, and sends to second acquisition unit;
Second acquisition unit, is connected with described first acquiring unit, for obtaining the mark of the data block storing described Data Identification institute identification data in the index record from described index block, and sends to described operational processes unit;
Operational processes unit, is connected with described second acquisition unit, and the data block for the mark correspondence to described data block carries out corresponding data manipulation process;
Described data manipulation process request message is write data requests message, off-set value and data message is also carried in write data request message, from the index record of index block, also obtain the memory range of data with existing in described data block, described operational processes unit comprises:
First judgment sub-unit, for judging that according to described off-set value the write type that write data request message is corresponding, said write type comprise covering and write, add and write or newly write;
Write operation subelement, for being written in corresponding data block according to said write type by described data message;
Described memory disk also includes the bitmap for distributing data block, and for storing the recovery zone of deleted data block, the write type that said write request message is corresponding be covering write time, described write operation subelement is specifically for removing to recovery zone by described data block; From bitmap, apply for new data block, and generate the mark of described data block; Described data message is written in new data block.
5. data-storage system according to claim 4, is characterized in that, described system also comprises buffer zone, and described first acquiring unit is specifically for obtaining corresponding index block according to described Data Identification from the Hash table being arranged in buffer zone;
Described second acquisition unit is specifically for obtaining the mark of the data block storing described Data Identification institute identification data from the index record of the index block being arranged in buffer zone.
6. data-storage system according to claim 4, is characterized in that, also carries size of data value in write data request message, and described system also comprises;
Second judging unit, during for judging to determine that data to be written are in the space of described data block according to described off-set value and described size of data value, returns the failed mistake of write.
7. the data-storage system according to claim 4 or 5, is characterized in that, also comprises:
3rd acquiring unit, be connected with described second acquisition unit, for being read data request message at described data manipulation process request message, and second acquisition unit when not getting the mark of the data block storing described Data Identification institute identification data from the index record of described index block, from recovery zone, obtains the mark of the data block of storage described Data Identification institute identification data according to described Data Identification;
Described Data Identification institute identification data is read in the data block of described operational processes unit specifically for the mark correspondence of the data block from second acquisition unit or the acquisition of the 3rd acquiring unit.
8. the data-storage system according to claim 4 or 5, is characterized in that, described memory disk also comprises:
Superblock, for storing the one or more combination in the number of index record in the size of index record in the size of described memory disk, the using state information of described memory disk, Hash table size, data block size, recovery zone size, bitmap size, number of data blocks, index block, index block.
9. the data-storage system according to claim 4 or 5, is characterized in that, described memory disk is raw device disk.
10. data-storage system according to claim 5, is characterized in that, also comprises the one or more combination in superblock information, Hash table data, recovery zone, bitmap region, index block region, data block region in described buffer zone.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210222998.0A CN102779180B (en) | 2012-06-29 | 2012-06-29 | The operation processing method of data-storage system, data-storage system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210222998.0A CN102779180B (en) | 2012-06-29 | 2012-06-29 | The operation processing method of data-storage system, data-storage system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102779180A CN102779180A (en) | 2012-11-14 |
| CN102779180B true CN102779180B (en) | 2015-09-09 |
Family
ID=47124092
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210222998.0A Expired - Fee Related CN102779180B (en) | 2012-06-29 | 2012-06-29 | The operation processing method of data-storage system, data-storage system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102779180B (en) |
Families Citing this family (55)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103856516B (en) * | 2012-12-03 | 2018-05-25 | 腾讯科技（深圳）有限公司 | Data storage, read method and data storage, reading device |
| CN103226965B (en) * | 2013-03-29 | 2015-09-09 | 浙江大学 | Based on the audio/video data access method of time bitmap |
| CN104298681B (en) * | 2013-07-18 | 2018-06-19 | 华为技术有限公司 | A kind of date storage method and device |
| CN103500092B (en) * | 2013-09-23 | 2017-01-18 | 华为技术有限公司 | Judgment method for WORM appending writing and storage device |
| CN104462141B (en) * | 2013-09-24 | 2018-05-22 | 中国移动通信集团重庆有限公司 | Method, system and the storage engines device of a kind of data storage and inquiry |
| CN103544259B (en) * | 2013-10-16 | 2017-01-18 | 国家计算机网络与信息安全管理中心 | Aggregating sorting TopK inquiry processing method and system |
| CN103500227B (en) * | 2013-10-23 | 2017-02-22 | 腾讯科技（深圳）有限公司 | Video frame storage management method, related device and related communication system |
| CN103699337B (en) * | 2013-12-04 | 2017-01-18 | 华为技术有限公司 | Writing control method and system based on independent redundant array of independent disks (RAID) |
| CN103593486B (en) * | 2013-12-06 | 2017-11-21 | 国家电网公司 | A kind of quality of power supply mass data storage and read method |
| CN103631959A (en) * | 2013-12-17 | 2014-03-12 | 江苏名通信息科技有限公司 | Method for supporting million user data sub-meter based on Hash algorithm |
| CN105183532B (en) * | 2014-04-29 | 2019-02-26 | 哈尔滨工程大学 | One kind is based on interface function performance optimization method in Dalvik virtual machine JNI mechanism |
| CN104035809B (en) * | 2014-04-29 | 2017-11-17 | 哈尔滨工程大学 | One kind is based on interface function performance optimization method in Dalvik virtual machine JNI mechanism |
| CN104636185B (en) * | 2015-01-27 | 2018-03-02 | 华为技术有限公司 | Business contexts management method, physical host, PCIE device and migration management equipment |
| US10394786B2 (en) | 2015-04-20 | 2019-08-27 | Futurewei Technologies, Inc. | Serialization scheme for storing data and lightweight indices on devices with append-only bands |
| CN105022627B (en) * | 2015-06-02 | 2018-04-06 | 上海斐讯数据通信技术有限公司 | A kind of communication means in android system between View |
| KR20170045806A (en) * | 2015-10-20 | 2017-04-28 | 삼성전자주식회사 | Semiconductor memory device and method of operating the same |
| CN106815260B (en) * | 2015-12-01 | 2021-05-04 | 阿里巴巴集团控股有限公司 | Index establishing method and equipment |
| CN106940700B (en) * | 2016-01-05 | 2021-11-12 | 南京中兴新软件有限责任公司 | Method, device and system for overwriting distributed file system |
| CN105574212B (en) * | 2016-02-24 | 2019-04-26 | 北京大学 | A kind of image search method of more index disk hash data structures |
| CN107277087B (en) * | 2016-04-06 | 2020-09-04 | 阿里巴巴集团控股有限公司 | Data processing method and device |
| CN105847102B (en) * | 2016-04-29 | 2020-11-24 | 珠海格力智能装备技术研究院有限公司 | Method, equipment and system for realizing field bus communication |
| CN106874348B (en) * | 2016-12-26 | 2020-06-16 | 贵州白山云科技股份有限公司 | File storage and index method and device and file reading method |
| CN107315543B (en) * | 2017-07-03 | 2020-10-09 | 合肥联宝信息技术有限公司 | Method and device for reading and writing data in EC (embedded control) memory |
| CN109254962B (en) * | 2017-07-06 | 2020-10-16 | 中国移动通信集团浙江有限公司 | Index optimization method and device based on T-tree and storage medium |
| CN110168529B (en) * | 2017-08-03 | 2022-05-13 | 华为云计算技术有限公司 | Data storage method, device and storage medium |
| CN109408278B (en) * | 2017-08-15 | 2021-06-15 | 李剑春 | Data processing method and device |
| CN107704202B (en) * | 2017-09-18 | 2021-09-07 | 北京京东尚科信息技术有限公司 | Method and device for quickly reading and writing data |
| CN108255958B (en) * | 2017-12-21 | 2022-05-03 | 百度在线网络技术（北京）有限公司 | Data query method, device and storage medium |
| CN109960463B (en) * | 2017-12-26 | 2021-11-02 | 中兴通讯股份有限公司 | Disk management method, device and storage medium |
| CN108255432A (en) * | 2018-01-12 | 2018-07-06 | 郑州云海信息技术有限公司 | Write operation control method, system, device and storage medium based on bedding storage |
| CN108710472A (en) * | 2018-04-27 | 2018-10-26 | 北京大学深圳研究生院 | For the abrasion equilibrium management method and distributed memory system inside storage unit |
| CN110597802A (en) * | 2018-05-24 | 2019-12-20 | 北京京东尚科信息技术有限公司 | Message processing method and device |
| CN109032529A (en) * | 2018-08-21 | 2018-12-18 | 郑州云海信息技术有限公司 | A kind of hard disk initialization method and system |
| CN110866063B (en) * | 2018-08-27 | 2023-10-31 | 阿里云计算有限公司 | Data tracking processing method and device |
| CN109542860B (en) * | 2018-10-25 | 2023-07-07 | 平安科技（深圳）有限公司 | Service data management method based on HDFS and terminal equipment |
| CN109634932B (en) * | 2018-11-30 | 2021-03-23 | 北京瑞卓喜投科技发展有限公司 | Intelligent contract storage method and storage system |
| CN111435286B (en) * | 2019-01-14 | 2023-12-05 | 深圳市茁壮网络股份有限公司 | Data storage method, device and system |
| CN110427364A (en) * | 2019-06-21 | 2019-11-08 | 北京奇艺世纪科技有限公司 | A kind of data processing method, device, electronic equipment and storage medium |
| CN112241336A (en) * | 2019-07-19 | 2021-01-19 | 伊姆西Ip控股有限责任公司 | Method, apparatus and computer program product for backing up data |
| CN110569398A (en) * | 2019-09-09 | 2019-12-13 | 广州虎牙科技有限公司 | data processing method and device, computer equipment and readable storage medium |
| CN111190537B (en) * | 2019-12-10 | 2023-08-25 | 优刻得科技股份有限公司 | Method and system for managing sequential storage disk in additional writing scene |
| CN111190930B (en) * | 2019-12-29 | 2023-11-28 | 杭州拓深科技有限公司 | Fire-fighting equipment historical data positioning system and positioning method integrating hash and block |
| CN111309261A (en) * | 2020-02-16 | 2020-06-19 | 西安奥卡云数据科技有限公司 | Physical data position mapping method on single node in distributed storage system |
| CN111309772A (en) * | 2020-03-23 | 2020-06-19 | 优刻得科技股份有限公司 | Data operation management method and hardware processing module |
| CN111767436B (en) * | 2020-06-23 | 2023-11-10 | 北京思特奇信息技术股份有限公司 | HASH index data storage and reading method and system |
| CN111797102A (en) * | 2020-07-06 | 2020-10-20 | 亿度慧达教育科技（北京）有限公司 | Data writing method and device and electronic equipment |
| CN114063884A (en) * | 2020-07-31 | 2022-02-18 | 伊姆西Ip控股有限责任公司 | Partitioning method, apparatus and computer program product for extended storage system |
| CN112162950B (en) * | 2020-09-11 | 2022-11-15 | 杭州涂鸦信息技术有限公司 | Data processing method and device based on file system and computer equipment |
| CN112667858A (en) * | 2020-12-25 | 2021-04-16 | 深圳创新科技术有限公司 | Method for storing data by adopting HASH chain and data writing and reading methods |
| CN113377709B (en) * | 2021-06-08 | 2022-07-19 | 北京计算机技术及应用研究所 | Hash-based HDFS back-end storage system |
| CN113703667A (en) * | 2021-07-14 | 2021-11-26 | 深圳市有为信息技术发展有限公司 | File system processing method and device for storing data in real time, vehicle-mounted terminal and commercial vehicle |
| CN113934725A (en) * | 2021-09-02 | 2022-01-14 | 远景智能国际私人投资有限公司 | Data acquisition method and device and Internet of things |
| CN114662160B (en) * | 2022-05-25 | 2022-08-23 | 成都易我科技开发有限责任公司 | Digital summarization method, system and digital summarization method in network transmission |
| CN115617707B (en) * | 2022-12-07 | 2023-03-10 | 摩尔线程智能科技(北京)有限责任公司 | Device and method for managing storage space, computing equipment and chip |
| CN116304390B (en) * | 2023-04-13 | 2024-02-13 | 北京基调网络股份有限公司 | Time sequence data processing method and device, storage medium and electronic equipment |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101551807A (en) * | 2009-05-07 | 2009-10-07 | 山东中创软件商用中间件股份有限公司 | Multilevel index technology for file database |
| CN101782922A (en) * | 2009-12-29 | 2010-07-21 | 山东山大鸥玛软件有限公司 | Multi-level bucket hashing index method for searching mass data |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101968795B (en) * | 2010-09-03 | 2012-05-09 | 清华大学 | Cache method for file system with changeable data block length |
| CN102314503A (en) * | 2011-09-01 | 2012-01-11 | 浪潮(北京)电子信息产业有限公司 | Indexing method |
-
2012
- 2012-06-29 CN CN201210222998.0A patent/CN102779180B/en not_active Expired - Fee Related
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101551807A (en) * | 2009-05-07 | 2009-10-07 | 山东中创软件商用中间件股份有限公司 | Multilevel index technology for file database |
| CN101782922A (en) * | 2009-12-29 | 2010-07-21 | 山东山大鸥玛软件有限公司 | Multi-level bucket hashing index method for searching mass data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102779180A (en) | 2012-11-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102779180B (en) | The operation processing method of data-storage system, data-storage system | |
| US11799959B2 (en) | Data processing method, apparatus, and system | |
| US8250033B1 (en) | Replication of a data set using differential snapshots | |
| US9043334B2 (en) | Method and system for accessing files on a storage system | |
| CN103080910B (en) | Storage system | |
| US8306950B2 (en) | Managing data access requests after persistent snapshots | |
| US8924664B2 (en) | Logical object deletion | |
| US20130332656A1 (en) | File system for maintaining data versions in solid state memory | |
| CN104346357A (en) | File accessing method and system for embedded terminal | |
| CN102084360A (en) | Apparatus, system, and method for validating that a correct data segment is read from a data storage device | |
| US20150149500A1 (en) | Multi-level lookup architecture to facilitate failure recovery | |
| CN110888837B (en) | Object storage small file merging method and device | |
| CN110134551B (en) | Continuous data protection method and device | |
| KR20110046118A (en) | Adaptive logging apparatus and method | |
| JP4825719B2 (en) | Fast file attribute search | |
| CN110352410A (en) | Track the access module and preextraction index node of index node | |
| US20170286444A1 (en) | Region-integrated data deduplication implementing a multi-lifetime duplicate finder | |
| US8151053B2 (en) | Hierarchical storage control apparatus, hierarchical storage control system, hierarchical storage control method, and program for controlling storage apparatus having hierarchical structure | |
| CN107562898A (en) | A kind of method that recycle bin is created based on KUX operating systems | |
| CN109325023B (en) | Data processing method and device | |
| CN112597074B (en) | Data processing method and device | |
| CN113821476B (en) | Data processing method and device | |
| CN113050891B (en) | Method and device for protecting deduplication data | |
| KR100704621B1 (en) | Method and apparatus for guaranteeing data integrity of portable storage device | |
| WO2021189312A1 (en) | Meta server crash recovery in object storage system using enhanced meta structure |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20150909 Termination date: 20180629 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |