CN102779180A - Operation processing method of data storage system and data storage system - Google Patents
Operation processing method of data storage system and data storage system Download PDFInfo
- Publication number
- CN102779180A CN102779180A CN2012102229980A CN201210222998A CN102779180A CN 102779180 A CN102779180 A CN 102779180A CN 2012102229980 A CN2012102229980 A CN 2012102229980A CN 201210222998 A CN201210222998 A CN 201210222998A CN 102779180 A CN102779180 A CN 102779180A
- Authority
- CN
- China
- Prior art keywords
- data
- block
- index
- identification
- write
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
An embodiment of the invention provides an operation processing method of a data storage system and the data storage system. The operation processing method includes that a storage magnetic disk of the data storage system comprises a hash table bucket and index blocks and receives data operation processing request message which carries data identification, the corresponding index block is obtained from a harsh table according to the data identification, mapping relation of the data identification and the index block is stored in the hash table, and each index block stores an index record which includes the mapping relation of the data identification and a data block storing data identified by the data identification. Identification of the data block storing the data identified by the data identification is obtained from the index record of each index block, and corresponding data operation processing is conducted on the data block that the identification of the data block corresponds to. By means of the method, the harsh table bucket and the index blocks can be stored in the storage magnetic disk to enable the operation method of the system to be further simplified. Visit performance of the magnetic disk can be further improved to some extent.
Description
Technical field
The present invention relates to technical field of data storage, relate in particular to a kind of operation processing method of data-storage system, data-storage system.
Background technology
Network-caching (Web Cache) need carry out buffer memory to hot spot data, and the webserver (Web Server) need be to mass data storage, and a large amount of Web webpages and video file also need carry out the high-performance storage.And a ring of the key index of user experience web access is fast data storage, index, reads.
In the prior art, (Third extended file system Ext3) is file system general, common in the linux system to the 3rd extended file system, can utilize key word (Key) data of its storage magnanimity at present.Wherein, the storage organization of Ext3 catalogue is similar to tree-shaped, and the index technology of Ext3 catalogue is to confirm according to the hash value of each directory entry title this file is placed on which page node of Htree.The Htree structure of each catalogue file comprises index block or directory entry piece, and each index entry comprises a hash key value and a logical pointer that points to the directory entry piece, and the directory entry piece comprises normal directory entry (directory entry) tabulation.Ext3 supports one-level or second-level directory index block, when root (root) index block is full, will become secondary index by the one-level index.Therefore; Just had the mode of indirect index through the Ext3 file to the mapping of the data in magnetic disk piece of deposit data file data: an indirect index, find index block earlier, again through index data in the index block; Find the piece number of storage data, just can navigate to actual DSB data store block; The secondary indirect index finds secondary indirect index piece earlier, therefrom obtains index block one time, in this index block, obtains the piece number of index block, and the piece through this index block number obtains the data of index block again, thereby navigates to the position of data at last.In catalogue, during locating file, travel through all records under it like this, and with the filename that will search relatively, when title is identical, then search successfully.
Yet, if file directory is very dark or catalogue under file a lot (as above 1000), the catalogue that will pass through a plurality of levels during access file read and last catalogue under matching ratio, the access performance meeting is very poor.Therefore, when passing through form of catalogue visit disk in the prior art, there is the defective of access performance difference.
Summary of the invention
The embodiment of the invention provides a kind of operation processing method of data-storage system, and data-storage system can improve the disk access performance.
The embodiment of the invention provides a kind of operation processing method of data-storage system on the one hand, and the memory disk of data-storage system comprises Hash table, data block and index block, and method comprises:
Receive data manipulation and handle request message, data manipulation is handled in the request message and is carried Data Identification;
From Hash table, obtain corresponding index block according to Data Identification; Store the mapping relations of Data Identification and index block in the Hash table; Store index record in each index block, include the mapping relations of Data Identification and the data block of storing Data Identification institute identification data in the index record;
From the index record of index block, obtain the sign of the data block of storage Data Identification institute identification data;
The corresponding data block of the sign of data block is carried out corresponding data manipulation to be handled.
The embodiment of the invention provides a kind of data-storage system on the other hand; The memory disk of data-storage system comprises Hash table, index block and data block; Store the mapping relations of Data Identification and index block in the Hash table; Store index record in each index block, include the mapping relations that Data Identification and data table memory show the data block of institute's identification data in the index record, system also comprises:
Interface unit is used to receive data manipulation and handles request message, and data manipulation is handled in the request message and carried Data Identification;
First acquiring unit is connected with interface unit, is used for obtaining corresponding index block according to Data Identification from Hash table, and sends to second acquisition unit;
Second acquisition unit is connected with first acquiring unit, is used for obtaining from the index record of index block the sign of the data block of storage Data Identification institute identification data, and sends to the operational processes unit;
The operational processes unit is connected with second acquisition unit, is used for that the corresponding data block of the sign of data block is carried out corresponding data manipulation and handles.
In the above embodiment of the present invention; Comprise Hash table bucket, data block and index block through memory disk at data-storage system; Thereby make the method for operating of this system handle request message at first receiving data manipulation, data manipulation is handled in the request message and is carried Data Identification; Again according to obtaining corresponding index block the Hash table of Data Identification in the Hash table bucket; Store the mapping relations of Data Identification and index block in the Hash table; Store index record in each index block, include the mapping relations of Data Identification and the data block of storing Data Identification institute identification data in the index record; Then from the index record of index block, obtain the sign of the data block of storage Data Identification institute identification data.With respect to visiting disk through the form of catalogue in the prior art, can effectively improve access speed, improve the access performance of disk.
Description of drawings
In order to be illustrated more clearly in the embodiment of the invention or technical scheme of the prior art; To do one to the accompanying drawing of required use in embodiment or the description of the Prior Art below introduces simply; Obviously, the accompanying drawing in describing below is some embodiments of the present invention, for those of ordinary skills; Under the prerequisite of not paying creative work property, can also obtain other accompanying drawing according to these accompanying drawings.
The process flow diagram of the operation processing method of a kind of data-storage system that Fig. 1 provides for the embodiment of the invention;
The process flow diagram of the method for writing data of a kind of data-storage system that Fig. 2 provides for the embodiment of the invention;
Fig. 3 covers the synoptic diagram that writes file for the data of a kind of data-storage system that the embodiment of the invention provides;
The data of a kind of data-storage system that Fig. 4 provides for the embodiment of the invention cover the synoptic diagram of wiring method;
The process flow diagram of the method for reading data of a kind of data-storage system that Fig. 5 provides for the embodiment of the invention;
Fig. 6 A is the synoptic diagram one that data block reads in the embodiment of the invention;
Fig. 6 B is the synoptic diagram two that data block reads in the embodiment of the invention;
The process flow diagram of the data-erasure method of a kind of data-storage system that Fig. 7 provides for the embodiment of the invention;
The framework synoptic diagram of a kind of data-storage system that Fig. 8 provides for the embodiment of the invention;
The framework synoptic diagram of the another kind of data-storage system that Fig. 9 provides for the embodiment of the invention;
A kind of data-storage system operation chart that Figure 10 provides for the embodiment of the invention;
The magnetic disk figure of a kind of data-storage system that Figure 11 provides for the embodiment of the invention;
Write the structural representation of processing unit in a kind of data-storage system that Figure 12 provides for the embodiment of the invention.
Embodiment
For the purpose, technical scheme and the advantage that make the embodiment of the invention clearer; To combine the accompanying drawing in the embodiment of the invention below; Technical scheme in the embodiment of the invention is carried out clear, intactly description; Obviously, described embodiment is the present invention's part embodiment, rather than whole embodiment.Based on the embodiment among the present invention, those of ordinary skills are not making the every other embodiment that is obtained under the creative work prerequisite, all belong to the scope of the present invention's protection.
The process flow diagram of the operation processing method of a kind of data-storage system that Fig. 1 provides for the embodiment of the invention.As shown in Figure 1, the memory disk of data-storage system comprises Hash table bucket, data block and index block, and the operation processing method of this data-storage system comprises:
Request message is handled in step 101, reception data manipulation, and data manipulation is handled in the request message and carried Data Identification.
The Data Identification that carries in the data manipulation processing request message in the present embodiment is the unique identifications of data in memory disk, can represent with key word key, and also can be to be expressed as Hash address id, do not limit at this.
Obtain corresponding index block step 102, the Hash table that in the Hash table bucket, is provided with according to Data Identification; Store the mapping relations of Data Identification and index block in the Hash table; Store index record in each index block, include the mapping relations of Data Identification and the data block of storing Data Identification institute identification data in the index record.
Hash table is stored in the Hash table bucket with the mode of table bucket, and Hash table stores the information of index block, the information that needs when promptly searching index block.Particularly, Hash table is also referred to as hash table, is a kind of data structure of directly conducting interviews of coming according to Data Identification.That is to say that a position visits record in the Hash table through being mapped to Data Identification,, wherein, can include the address of index block in the record of this position to accelerate searching speed.Hash table be through hash calculation will be bigger deposit data in less relatively space, and the space of Hash table is less relatively, can produce two different Data Identifications and be mapped on the position of same Hash table and produce conflict (collision); Can adopt the open chain method to solve, particularly, be exactly the storage space that free block of application is shown as new Hash from the disk management module of bitmap region; And will have the Hash watch chain now and receive new Hash table; Wherein, all on the conflict chain, the conflict chain can be the form of chained list to all index records in the free block; And each node physics is discontinuous, also can be continuous.The conflict that solves Hash table in the present embodiment is not limited to this, can remove tissue conflict record with other data structure such as B tree, and other method of the conflict that can solve Hash table that every those skilled in the art use all can.
Need explain; Use hash function HashFun1 according to key word key; That is: hash value index=HashFunc1 (key) can obtain the address of corresponding index block, if in the index block of just searching, do not find corresponding index block address; Can continue to inquire about backward index block until finding, perhaps return the inquiry failure.Index block has been preserved the index information of all records of total this hash value, and the space of an index block then can produce conflict when storing index record inadequately, and present embodiment uses the open chain method to manage conflict.Particularly be that free block of application and will have index block now and new index block is linked as new index block from the disk management module of bitmap region.Under the enough situation of disk space, can store unlimited many index records through the mode of this unidirectional series connection.
Fig. 6 A is the synoptic diagram one that data block reads in the embodiment of the invention.Fig. 6 B is the synoptic diagram two that data block reads in the embodiment of the invention.Shown in Fig. 6 A and Fig. 6 B; Index block can comprise like lower area: head zone, series connection pointer, minimax id, posting field; It is a plurality of to need to prove that posting field can have; For example posting field 1 is up to posting field n, and the zone that index block comprises in the present embodiment is as limit, index block can comprise above-mentioned zone several, all or on this basis increase other zone again.Wherein, head zone is preserved the essential information of this index block, and for example: the size of single record, total record number, actual record number and be recorded in the reference position in the index block, present embodiment does not exceed with cited essential information.The series connection pointer, the information of preserving next index block.Minimax id is meant key word Key is used the index record that obtains behind the hash function HashFunc2 position range at this index block.Wherein, minimax id is that the index location uses, and can confirm fast that index record is whether in this index block.Posting field is preserved index records all in this index block; Mainly comprise data in the Hash address of data block id, data in the scope of data block; The just positional information of index block, data block number; The Hash address id or the pairing out of Memory of key word (key) that also can include this index record are like creation-time, modification time, deletion time or user's information.
That is to say that from the index record of index block, search the sign of the data block that stores Data Identification, the sign of data block can be the address of data block.Index record can be searched according to the mode of certain data structure storage, like sort algorithm, binary tree, uses searching of certain algorithm can reduce the number of times that in search procedure, matees, thereby quickens inquiry.Because an index block can be preserved a lot of index records, if use certain algorithm index record is searched coupling, can when insertion, deletion or reading of data, reduce frequent operation to data block.
In the present embodiment data block is carried out corresponding data manipulation and handle, can comprise: reading of data, write data and deleted data, wherein, writing data type can include: newly write, cover and write and append and write.In reading of data when writing the data manipulation parallel processing; Through writing data and whether reading of data operates in same data block to writing the judgement of type, can judging, if not; Can parallel work-flow reading of data and service data, improve the concurrent performance of read-write.
Present embodiment is divided into the zone that comprises Hash table bucket, data block and index block through the memory disk with data-storage system; Can simplify the method for operating of this system; At first receive data manipulation and handle request message, data manipulation is handled in the request message and is carried Data Identification; Again according to obtaining corresponding index block the Hash table of Data Identification in the Hash table bucket; Store the mapping relations of Data Identification and index block in the Hash table; Store index record in each index block, include the mapping relations of Data Identification and the data block of storing Data Identification institute identification data in the index record; Then from the index record of index block, obtain the sign of the data block of storage Data Identification institute identification data; Thereby the step of this system operation methods is simplified, and then the access performance of raising disk that can be to a certain degree.
Present embodiment, optional, data-storage system can also comprise buffer zone, cache management mainly is that the data with the partial data piece are arranged in the internal memory, when conducting interviews, can directly from internal memory, visit, convenient visit, thus improve access speed.After receiving data manipulation processing request message; At first in buffer zone, from the Hash table of hash, obtain the address of corresponding index block according to Data Identification; Step 102 is identical in concrete steps and the foregoing description; Explain no longer that at this be different with the corresponding Hash table bucket present position of Data Identification with the foregoing description step 102 difference, the Hash table bucket of present embodiment is positioned at buffer zone; That is to say, be to obtain corresponding index block information according to Data Identification from the Hash table of the Hash table bucket that is arranged in buffer zone in the present embodiment.Then; From the index record of index block, obtain the sign of the data block of storage Data Identification institute identification data; No longer explain at this with the same section of the foregoing description step 103; Difference is that index block is arranged in buffer zone, particularly, is the sign of obtaining the data block of storage Data Identification institute identification data from the index record of the index block that is arranged in buffer zone.The partial data piece is arranged in the buffer zone, can conveniently visits, and for solid state hard disc (solid state disk; SSD), number of operations too much can influence its serviceable life, for example when write operation; Can be written in buffer zone earlier; Behind certain number of times, be synchronized to disk again, reduced the number of times of writing disk like this, the life-span that also can improve disk simultaneously.
The process flow diagram of the method for writing data of a kind of data-storage system that Fig. 2 provides for the embodiment of the invention; As shown in Figure 2; The method for writing data of data-storage system comprises: receive data manipulation and handle request message; Wherein, Request message is handled in data manipulation can be write data requests message, also carries off-set value (offset) and data message (data) in the write data requests message, wherein; Can also include the memory range that from the index record of index block, can also obtain data with existing in the data block in the write data requests message, the corresponding data block of the sign of data block carried out corresponding data manipulation handle.Corresponding data manipulation handle be according to off-set value judge write data requests message corresponding write type, write type and comprise covering and write, append and write or newly write.Wherein, cover to write and be meant in the data with existing zone and write new data; Append to write and be meant after the data with existing zone and write data; Other situation then is newly to write.The method for writing data concrete steps of data-storage system comprise:
Wherein, can carry the size (size) of off-set value (offset) and data message data, key word (key), write data in the write data requests message, like write (key; Offset, size, data); Or only carry data message, promptly be expressed as write (data).
If among the length range (range) of side-play amount (offset) data in data block, then writing the corresponding type that writes of request message is to cover when writing, and gets into step 2051;
If the length range (range) of side-play amount (offset) data in data block is afterwards, then write the corresponding type that writes of request message and write for appending, get into step 206;
Other then is newly to write, and gets into step 206.
Judge that writing type is to cover when writing, data message be written to comprise step in the corresponding data block:
The version number and the data block address id of step 2051, a new data block of generation according to new version number and key word key, use HashFun2, generate the new data block address id that a preparation covers write operation.
Whether step 2052, the data of searching this data block address id exist; If in the data block of this data block address id data are arranged; Then return step 2051, new version number of regeneration, and according to new version number and key word key; Use HashFun2 and generate new data block address id, in the data block of searching this data block address id, do not store any data; If have no data to exist in the id data block of this address, then get into next step 2053;
The sign of step 2055, return data piece, the sign of data block can include new version number.
Present embodiment obtains corresponding index block according to the Data Identification of write data requests message from Hash table; Then from the index record of index block, obtain the sign of the data block of storage Data Identification institute identification data; With in the data writing data blocks, wherein wiring method comprises again: cover and write, newly write and append and write.In the method for writing data of this data-storage system, can just find corresponding index block through once searching, thereby can simplify the method for operating of this system, and then can improve the access performance of disk.
Fig. 3 covers the synoptic diagram that writes file for the data of a kind of data-storage system that the embodiment of the invention provides; The data of a kind of data-storage system that Fig. 4 provides for the embodiment of the invention cover the synoptic diagram of wiring method.Like Fig. 3 and shown in Figure 4; Original comprises four data block block1, block2, block3 and block4; During with the former data of new data cover data piece block3; At first, in index record, search and to cover the data block of writing, just the data block number in the match index record in the position range of all data blocks that index record is stored according to the Hash address id that in index record, matches.With the whole data block is that unit changes; New data block Block3 ' of application in the disk management module; Original block copy is arrived among the new data block Block3 '; Write new data to data block Block3 ' again, and generate new version number, simultaneously legacy data piece block3 is moved to the recovery zone.The covering literary style of implementing like this can not changed the remainder data piece, and newly-generated version number is kept in the application layer, as far as application layer, can continue to use the data that are capped, i.e. the delayed deletion function.When the partial data of legacy data piece block3 was rewritten, the copy-on-write technology can satisfy following 4 points, one) original that is capped can normally use; Two) new writen data can not influence the operation to the data block of original file; Three) in the operation that covering is write, to the data change of needs changes seldom, can not cause the deterioration write performance; Four) only need application to need the data block of change, space hold is few.And concerning the user, reading of new file comprised block1,2,4 and new data block block3 ', and comparatively speaking, new data block block3 ' is new application, that can reduce data space takies and writes expense.And legacy data block3 in the past also can continue to be visited.
The process flow diagram of the method for reading data of a kind of data-storage system that Fig. 5 provides for the embodiment of the invention.Shown in Fig. 5, Fig. 6 A and Fig. 6 B, it is read data request message that request message is handled in data manipulation, promptly read (size), then data method comprises for key, offset:
Wherein, this message carries off-set value (offset), key word (key) and the expectation size of data, (size) that read, as read (key, offset, size).
Successfully get into step 507 if mate, otherwise get into step 505.
Present embodiment obtains corresponding index block according to the Data Identification of data read request message from Hash table, then from the index record of index block, obtain the sign of the data block of storage Data Identification institute identification data, data block is operated accordingly again.In the method for reading data of this data-storage system, can once search and just find the index block that needs, thereby can simplify the method for operating of this system, and then the access performance of raising disk that can be to a certain degree.
The process flow diagram of the data-erasure method of a kind of data-storage system that Fig. 7 provides for the embodiment of the invention; As shown in Figure 7, the step 601-step 604 of the data-erasure method of data-storage system is basic identical with the step 501-step 504 in the foregoing description, and same section is no longer explained; Difference is that it is the deleted data request message that request message is handled in the data manipulation that step 601 received, wherein; This message carries data message (key); Wherein, key is that key word also is the unique identification in the memory disk, and the deleted data request can be expressed as delete (key).Judgement in index record with the matching result of newly-generated new sign id, successfully get into step 605 if mate, otherwise return failure information.
The result that step 605, basis are searched removes to the recovery zone with this data block.
Present embodiment obtains corresponding index block according to the Data Identification of data deletion request message from Hash table, then from the index record of index block, obtain the sign of the data block of storage Data Identification institute identification data, again data block is removed to the recovery zone.In the data-erasure method of this data-storage system, can once search and just find the index block that needs, thereby can simplify the method for operating of this system, and then the access performance of raising disk that can be to a certain degree.
The configuration diagram of a kind of data-storage system that Fig. 8 provides for the embodiment of the invention.As shown in Figure 8; The memory disk of data-storage system comprises Hash table bucket, index block and data block; Wherein, Store the mapping relations of Data Identification and index block in the Hash table that is provided with in the Hash table bucket, store index record in each index block, include the mapping relations that Data Identification and data table memory show the data block of institute's identification data in the index record.Index block is preserved the map information of key word Key to data block, and the information of locator data piece can be provided; Data block: the storage area of data.The minimum administrative unit of memory disk is a data block, the access performance of the affects memory disk of data block.Therefore size that can the setting data piece when the format memory disk; The setting of this value size can be specifically according to the mean size of storage object, the number and the memory disk total volume of storage object; Size to Hash table bucket and data block is adjusted, to obtain a preferable performance.Solid state hard disc for example, the setting of data block size can be the integral multiple sizes of minimum erase units.In general, the size of data block need satisfy less loss of searching matching times and less management space, and the data block minimum is 4Kbytes in the present embodiment, but not as limit.
The data-storage system of present embodiment also comprises: interface unit 101, first acquiring unit 102, second acquisition unit 103 and operational processes unit 104.
First acquiring unit 102; Be connected with interface unit 101; Be used for obtaining corresponding index block according to Data Identification from Hash table, Hash table is preserved the mapping relations of Data Identification and index block, obtains from Hash table sending to second acquisition unit 103 after the information of index block.
The memory disk of present embodiment data-storage system can comprise Hash table bucket, index block and data block, and this system also comprises interface unit 101, first acquiring unit 102, second acquisition unit 103 and operating unit 104.And then can be on the method for operating of this system through the Hash table that is provided with in the Hash table bucket; Once search and just can find and the corresponding index block of request of data information; Thereby can simplify the method for operating of this system, and then the access performance of raising disk that can be to a certain degree.
On the basis of technique scheme, optional, data-storage system also comprises a basic and important module buffer zone, and particularly, buffer zone is that the data with the partial data piece are arranged on internal memory, to improve access speed.Magnetic disk and storage data organization directly affect the setting of data in the buffer zone.When interface unit 101 receives write data requests message; At first this message is delivered in the Hash table in the buffer zone; Particularly, first acquiring unit 102 obtains corresponding index block according to the Data Identification that carries in the write data requests message from the Hash table that is arranged in buffer zone.Obtain the address of index block from Hash table after, second acquisition unit 103 obtains the sign of the data block of storage Data Identification institute identification data from the index record of the index block that is arranged in buffer zone.
The configuration diagram of the another kind of data-storage system that Fig. 9 provides for the embodiment of the invention.As shown in Figure 9, optional on the basis of technique scheme, data-storage system can also comprise:
The 3rd acquiring unit 105; Be connected with second acquisition unit 103; Being used for handling request message in data manipulation is read data request message; And when second acquisition unit 103 does not get access to the sign of data block of storage Data Identification institute identification data from the index record of index block, can from the recovery zone, obtain the sign of the data block of storage Data Identification institute identification data, will send to operational processes unit 104 from the sign that index block obtains data block again according to Data Identification; Particularly, the corresponding data block reading of data sign institute identification data of sign of operational processes unit 104 data block that can be used for obtaining from second acquisition unit 103 or the 3rd acquiring unit 105.
Need explain; The memory disk of present embodiment is the raw device disk; Being also referred to as raw partition (raw partition) equipment, is a kind of not through format, the block device that in Unix/Linux operating system, can not read through file system; Data manipulation in the raw device disk can directly be operated it by application program without the buffering of file system.The raw device disk is to be managed by operating system as an equipment.According to disk drive, system can carry out the practical operation of data.Interface unit 101 can provide and outside operation-interface, includes basic operation and bookkeeping, and wherein, basic operation has the deletion with data that writes of the reading of data, data, and bookkeeping can include format and load.
A kind of data-storage system operation chart that Figure 10 provides for the embodiment of the invention.This operation of present embodiment shown in figure 10 can be the deletion with data that writes of the reading of data, data, does not limit at this:
1, calls the file system of opening raw device through interface unit;
2, send the reading of data request to the raw device management;
3, raw device management receive data with to the buffer data coupling, and judge whether mate successfully, directly return if mate successfully from buffer zone, fail entering step 4 if mate;
4, the raw device management is sent request to disk drive, and the operation of calling file system;
5, disk drive is located concrete disk position, and the data disk of this position is carried out write operation.
Need explain that data read operation is identical with Fig. 9 flow process, the step of data deletion operation is identical with the step 1-step 3 in Figure 10 flow process, in this explanation no longer one by one.
The magnetic disk figure of a kind of data-storage system that Figure 11 provides for the embodiment of the invention.Shown in figure 11, memory disk can comprise: superblock, Hash show bucket, recovery zone, bitmap region, index block is regional, data block is regional.Wherein, Superblock records the essential information of this memory disk; Quantity, hash table size, index block size like data block in the total size of: memory disk, operating position, the memory disk; The recovery zone is big or small, bitmap block is big or small, and information such as size of the index record in the index block and index record number.Bitmap region records the operating position of all data blocks, has promptly used and untapped situation.For example: for the disk of solid state hard disc; The record of bitmap region has influence on the allocation order of disk, also directly has influence on the serviceable life to disk, therefore when the visit disk; Can be according to the order of disk block; According in the past backward order assignment, can guarantee like this each disk block to write indegree relatively more balanced, thereby prolong the life-span of solid state hard disc.(Hard Disk Drive HDD), because the influence of mechanical magnetic head, can distribute disk block in continuity ground, to prolong the life-span of hard disk drive for hard disk drive.All the other zones are explained in the above-described embodiments, are given unnecessary details no longer one by one at this.In the present embodiment, can consider actual performance, and when disk initialization according to disk space, total record number, design the size of suitable hash table, the size of data block, to obtain the access performance of the best.
Based on the above embodiments, can include admin instance, superblock, Hash table, recovery zone, bitmap region, index block zone, the data block of this disk region one or more combination in regional in the buffer zone of data-storage system.Wherein, the admin instance of this disk region provides the raw device supervisory routine to use.The recovery zone can be kept at the recovery zone with the data of the data block of having deleted, and uses for the delayed deletion module.The delayed deletion module comes into force in the startup operational process of raw device management; Can visit the data of having deleted; Yet after restarting, this partial data can permanent delet, therefore can the recovery zone be arranged on buffer zone; Present embodiment is not limited to this, also can it be arranged in the disk.Wherein, the disk structure of the non-buffer zone in the specific descriptions in superblock, Hash table, bitmap region, index block zone and the foregoing description is identical, no longer explains at this.Can admin instance, superblock, whole Hash table data, bitmap, index block and the partial data piece of this disk region be resided in buffer zone.Selection resides in the method for the partial data piece of buffer zone, can be through the algorithm of LRU or other intelligent high-speed memory buffer.
At data-storage system, when carrying out the read data operation, if can directly in buffer zone, choose the data that need read, then directly from the buffer zone reading of data.When writing data manipulation,, then write direct data in buffer zone if can find the data block that need write at buffer zone, and by the user according to display synchronization or the replacement operation again with the change data sync in disk.Be placed in the buffer zone through the data block that will often visit, can accelerate access speed.
Write the structural representation of processing unit in a kind of data-storage system that Figure 12 provides for the embodiment of the invention.Shown in figure 12; When interface unit reception data manipulation processing request message is write data requests message; Can carry off-set value (offset) and data message (data) in the write data requests message; Write data requests message can also comprise size of data size (size) and key word (key), from Hash table, obtains corresponding index block according to data message, from the index record of index block, obtains the memory range of data with existing in the data block again; Wherein, Carry the data of the length range (range) of data in the information of data block, the operational processes unit can write type according to length range (range) judgement of the side-play amount in the data request information (offset) data in data block, and wherein the operational processes unit comprises:
First judgment sub-unit 1041, be used for according to off-set value judge write data requests message corresponding write type, write type and comprise covering and write, append and write or newly write; Particularly, if among the scope (range) of side-play amount (offset) data length in data block, then writing the corresponding type that writes of request message is to cover when writing; If the scope (range) of side-play amount (offset) data in data block afterwards, then write the corresponding type that writes of request message and write for appending; Other then is newly to write.
Write operation subelement 1043 is used for according to writing type data message being written to corresponding data block.Concrete grammar is identical with the method for the foregoing description Fig. 3, no longer explains at this.
On the basis of technique scheme; Optional; Also carry the size of data value in the write data requests message that interface unit receives; Data-storage system also includes second judging unit 1042, is used for judging according to off-set value and size of data value returning and writing the failure mistake when confirming that data to be written are in the space of data block.That is to say; Second judging unit 1042 can judge that definite data to be written are in the locus of data block according to off-set value and size of data value; If when off-set value and size of data value surpass the spatial dimension of data block, then do not write any data, and return and write the failure mistake.If off-set value and size of data value in the spatial dimension of data block the time, are then carried out write operation.
In the present embodiment; Memory disk can also include the bitmap that is used for the distribute data piece; And the recovery zone that is used to store the data block of being deleted, write request message corresponding write type and write for covering the time, write operation subelement 1043 specifically is used for data block is removed to the recovery zone; From the disk management module of bitmap region, apply for new data block, and generate the sign of data block; Data message is written in the new data block.The concrete implementation method of present embodiment is identical with the foregoing description step 2051-step 2054, gives unnecessary details no longer one by one at this.
On the basis of the foregoing description, can increase pre-fetch function.When reading of data, the mode that indicates object that can show according to access module or fetch interface, the reading disk piece is to buffer zone in advance, when reading of data request once more, can directly from buffer zone, read, and then put forward the visit high-performance of disk.
It is write data requests message that request message is handled in the data manipulation of present embodiment data-storage system; Also carry off-set value and data message in the write data requests message; From the index record of index block, also obtain the memory range of data with existing in the data block, the operational processes unit includes first judgment sub-unit 1041, write operation subelement 1043 and second judging unit 104.This system write data manipulation the time Hash table that is provided with in can be through the Hash table bucket; Once search and just can find and the corresponding index block of request of data information; Thereby can simplify the method for operating of this system, and then the access performance of raising disk that can be to a certain degree.
Based on the foregoing description; Comprise Hash table bucket, data block and index block through memory disk at data-storage system; Thereby make the method for operating of this system handle request message at first receiving data manipulation, data manipulation is handled in the request message and is carried Data Identification; Again according to obtaining corresponding index block the Hash table of Data Identification in the Hash table bucket; Store the mapping relations of Data Identification and index block in the Hash table; Store index record in each index block, include the mapping relations of Data Identification and the data block of storing Data Identification institute identification data in the index record; Then from the index record of index block, obtain the sign of the data block of storage Data Identification institute identification data; And then the access performance of raising disk that can be to a certain degree.
One of ordinary skill in the art will appreciate that: all or part of step that realizes above-mentioned each method embodiment can be accomplished through the relevant hardware of programmed instruction.Aforesaid program can be stored in the computer read/write memory medium.This program the step that comprises above-mentioned each method embodiment when carrying out; And aforesaid storage medium comprises: various media that can be program code stored such as ROM, RAM, magnetic disc or CD.
What should explain at last is: above each embodiment is only in order to explaining technical scheme of the present invention, but not to its restriction; Although the present invention has been carried out detailed explanation with reference to aforementioned each embodiment; Those of ordinary skill in the art is to be understood that: it still can be made amendment to the technical scheme that aforementioned each embodiment put down in writing, perhaps to wherein part or all technical characteristic are equal to replacement; And these are revised or replacement, do not make the scope of the essence disengaging various embodiments of the present invention technical scheme of relevant art scheme.
Claims (16)
1. the operation processing method of a data-storage system is characterized in that, the memory disk of said data-storage system comprises Hash table bucket, data block and index block, and said method comprises:
Receive data manipulation and handle request message, said data manipulation is handled in the request message and is carried Data Identification;
Obtain corresponding index block the Hash table that in said Hash table bucket, is provided with according to said Data Identification; Store the mapping relations of Data Identification and said index block in the said Hash table; Store index record in each index block, include the mapping relations of the data block of said Data Identification and the said Data Identification of storage institute identification data in the said index record;
From the index record of said index block, obtain the sign of the data block of the said Data Identification of storage institute identification data;
The corresponding data block of the sign of said data block is carried out corresponding data manipulation to be handled.
2. the operation processing method of data-storage system according to claim 1 is characterized in that, said data-storage system also comprises buffer zone, saidly from Hash table, obtains corresponding index block according to Data Identification and is specially:
Obtain corresponding index block according to said Data Identification from the Hash table that is arranged in buffer zone;
The said sign of from the index record of index block, obtaining the data block of the said Data Identification of storage institute identification data is specially:
Obtain the sign of the data block of the said Data Identification of storage institute identification data from the index record of the index block that is arranged in buffer zone.
3. the operation processing method of data-storage system according to claim 1 and 2; It is characterized in that; It is write data requests message that request message is handled in said data manipulation; Also carry off-set value and data message in the write data request message, from the index record of index block, also obtain the memory range of data with existing in the said data block, the corresponding data block of said sign to data block is carried out corresponding data manipulation and is handled and comprise:
According to the type that writes of said off-set value judgement write data request message correspondence, the said write type comprises covering writes, appends and write or newly write;
According to the said write type said data message is written in the corresponding data block.
4. the operation processing method of data-storage system according to claim 3; It is characterized in that; Said memory disk also includes the bitmap that is used for the distribute data piece; And the recovery zone that is used to store the data block of being deleted, the said write request message corresponding write type and write for covering the time, said basis writes type and said data message is written in the corresponding data block comprises:
Said data block is removed to the recovery zone;
From bitmap, apply for new data block, and generate the sign of said data block;
Said data message is written in the new data block.
5. the operation processing method of data-storage system according to claim 1 and 2; It is characterized in that; Said memory disk also includes the recovery zone that is used to store the data block of being deleted; It is write data requests message that request message is handled in said data manipulation, saidly the corresponding data block of the sign of said data block is carried out corresponding data manipulation handles and comprises:
Said data block is removed to the recovery zone.
6. the operation processing method of data-storage system according to claim 1 and 2 is characterized in that, it is read data request message that request message is handled in said data manipulation, and said method also comprises:
When the sign of the data block that from the index record of said index block, does not get access to the said Data Identification of storage institute identification data, from the recovery zone, obtain the sign of the data block of the said Data Identification of storage institute identification data according to said Data Identification;
The corresponding data block of said sign to said data block is carried out corresponding data manipulation and is handled and comprise:
From the corresponding data block of the sign of said data block, read said Data Identification institute identification data.
7. data-storage system; It is characterized in that; The memory disk of said data-storage system comprises Hash table bucket, index block and data block, stores the mapping relations of Data Identification and said index block in the Hash table that is provided with in the said Hash table bucket, stores index record in each index block; Include the mapping relations of the data block of said Data Identification and the said data representation of storage institute identification data in the said index record, said system also comprises:
Interface unit is used to receive data manipulation and handles request message, and said data manipulation is handled in the request message and carried Data Identification;
First acquiring unit is connected with said interface unit, is used for obtaining corresponding index block according to said Data Identification from Hash table, and sends to said second acquisition unit;
Second acquisition unit is connected with said first acquiring unit, is used for obtaining from the index record of said index block the sign of the data block of the said Data Identification of storage institute identification data, and sends to said operational processes unit;
The operational processes unit is connected with said second acquisition unit, is used for that the corresponding data block of the sign of said data block is carried out corresponding data manipulation and handles.
8. data-storage system according to claim 7 is characterized in that said system also comprises buffer zone, and said first acquiring unit specifically is used for obtaining corresponding index block according to said Data Identification from the Hash table that is arranged in buffer zone;
Said second acquisition unit specifically is used for obtaining from the index record of the index block that is arranged in buffer zone the sign of the data block of the said Data Identification of storage institute identification data.
9. according to claim 7 or 8 described data-storage systems; It is characterized in that; It is write data requests message that request message is handled in said data manipulation; Also carry off-set value and data message in the write data request message, from the index record of index block, also obtain the memory range of data with existing in the said data block, said operational processes unit comprises:
First judgment sub-unit is used for the type that writes according to said off-set value judgement write data request message correspondence, and the said write type comprises covering writes, appends and write or newly write;
The write operation subelement is used for according to the said write type said data message being written to corresponding data block.
10. data-storage system according to claim 9 is characterized in that, also carries the size of data value in the write data request message, and said system also comprises;
Second judging unit is used for judging according to said off-set value and said size of data value and returns and write the failure mistake when confirming that data to be written are in the space of said data block.
11. data-storage system according to claim 10; It is characterized in that; Said memory disk also includes the bitmap that is used for the distribute data piece; And the recovery zone that is used to store the data block of being deleted, the said write request message corresponding write type and write for covering the time, said write operation subelement specifically is used for said data block is removed to the recovery zone; From bitmap, apply for new data block, and generate the sign of said data block; Said data message is written in the new data block.
12. according to claim 7 or 8 described data-storage systems; It is characterized in that; Said memory disk also includes the recovery zone that is used to store the data block of being deleted; It is write data requests message that request message is handled in said data manipulation, and said operational processes unit specifically is used for said data block is removed to the recovery zone.
13. according to claim 7 or 8 described data-storage systems, it is characterized in that, also comprise:
The 3rd acquiring unit; Be connected with said second acquisition unit; Being used for handling request message in said data manipulation is read data request message; And when second acquisition unit does not get access to the sign of data block of the said Data Identification of storage institute identification data from the index record of said index block, from the recovery zone, obtain the sign of the data block of the said Data Identification of storage institute identification data according to said Data Identification;
The corresponding data block of sign of the data block that said operational processes unit specifically is used for obtaining from second acquisition unit or the 3rd acquiring unit reads said Data Identification institute identification data.
14., it is characterized in that said memory disk also comprises according to claim 7 or 8 described data-storage systems:
Superblock, be used for storing the size of said memory disk, said memory disk user mode information, Hash table size, data block is big or small, the recovery zone is big or small, one or multinomial combination in the size of bitmap size, number of data blocks, index block index record, index block in the number of index record.
15., it is characterized in that said memory disk is the raw device disk according to claim 7 or 8 described data-storage systems.
16. according to claim 7 or 8 described data-storage systems, it is characterized in that, also comprise superblock information, Hash table data, recovery zone, bitmap region, index block zone, the data block one or more combination in regional in the said buffer zone.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210222998.0A CN102779180B (en) | 2012-06-29 | 2012-06-29 | The operation processing method of data-storage system, data-storage system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210222998.0A CN102779180B (en) | 2012-06-29 | 2012-06-29 | The operation processing method of data-storage system, data-storage system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102779180A true CN102779180A (en) | 2012-11-14 |
| CN102779180B CN102779180B (en) | 2015-09-09 |
Family
ID=47124092
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210222998.0A Expired - Fee Related CN102779180B (en) | 2012-06-29 | 2012-06-29 | The operation processing method of data-storage system, data-storage system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102779180B (en) |
Cited By (53)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103226965A (en) * | 2013-03-29 | 2013-07-31 | 浙江大学 | Time-bitmap-based audio/video data access method |
| CN103500092A (en) * | 2013-09-23 | 2014-01-08 | 华为技术有限公司 | Judgment method for WORM appending writing and storage device |
| CN103500227A (en) * | 2013-10-23 | 2014-01-08 | 腾讯科技（深圳）有限公司 | Video frame storage management method, related device and related communication system |
| CN103544259A (en) * | 2013-10-16 | 2014-01-29 | 国家计算机网络与信息安全管理中心 | Aggregating sorting TopK inquiry processing method and system |
| CN103593486A (en) * | 2013-12-06 | 2014-02-19 | 国家电网公司 | Method for storing and reading mass data of power quality |
| CN103631959A (en) * | 2013-12-17 | 2014-03-12 | 江苏名通信息科技有限公司 | Method for supporting million user data sub-meter based on Hash algorithm |
| CN103699337A (en) * | 2013-12-04 | 2014-04-02 | 华为技术有限公司 | Writing control method and system based on independent redundant array of independent disks (RAID) |
| CN103856516A (en) * | 2012-12-03 | 2014-06-11 | 腾讯科技（深圳）有限公司 | Data storage and reading method and data storage and reading device |
| CN104035809A (en) * | 2014-04-29 | 2014-09-10 | 哈尔滨工程大学 | Method for optimizing performance of interface functions in JNI mechanism based on Dalvik virtual machine |
| CN104298681A (en) * | 2013-07-18 | 2015-01-21 | 华为技术有限公司 | Data storage method and device |
| CN104462141A (en) * | 2013-09-24 | 2015-03-25 | 中国移动通信集团重庆有限公司 | Data storage and query method and system and storage engine device |
| CN104636185A (en) * | 2015-01-27 | 2015-05-20 | 华为技术有限公司 | Service context management method, physical host, PCIE equipment and migration management equipment |
| CN105183532A (en) * | 2014-04-29 | 2015-12-23 | 哈尔滨工程大学 | Interface function performance optimization method in JNI mechanism based on Dalvik virtual machine |
| CN105574212A (en) * | 2016-02-24 | 2016-05-11 | 北京大学 | Image retrieval method for multi-index disk Hash structure |
| CN105847102A (en) * | 2016-04-29 | 2016-08-10 | 珠海格力电器股份有限公司 | Method, device and system realizing field bus communication |
| WO2016169456A1 (en) * | 2015-04-20 | 2016-10-27 | Huawei Technologies Co., Ltd. | Serialization scheme for storing data and lightweight indices on devices with append-only bands |
| WO2016192328A1 (en) * | 2015-06-02 | 2016-12-08 | 上海斐讯数据通信技术有限公司 | Method for communicating between views in android system |
| CN106815260A (en) * | 2015-12-01 | 2017-06-09 | 阿里巴巴集团控股有限公司 | A kind of index establishing method and equipment |
| CN106874348A (en) * | 2016-12-26 | 2017-06-20 | 贵州白山云科技有限公司 | File is stored and the method for indexing means, device and reading file |
| CN106940700A (en) * | 2016-01-05 | 2017-07-11 | 南京中兴新软件有限责任公司 | The covering write method of distributed file system, apparatus and system |
| CN106997783A (en) * | 2015-10-20 | 2017-08-01 | 三星电子株式会社 | The method of semiconductor storage and operation semiconductor storage |
| CN107277087A (en) * | 2016-04-06 | 2017-10-20 | 阿里巴巴集团控股有限公司 | Data processing method and device |
| CN107315543A (en) * | 2017-07-03 | 2017-11-03 | 合肥联宝信息技术有限公司 | The reading/writing method and device of data in a kind of memory to EC |
| CN107704202A (en) * | 2017-09-18 | 2018-02-16 | 北京京东尚科信息技术有限公司 | A kind of method and apparatus of data fast reading and writing |
| CN108255958A (en) * | 2017-12-21 | 2018-07-06 | 百度在线网络技术（北京）有限公司 | Data query method, apparatus and storage medium |
| CN108255432A (en) * | 2018-01-12 | 2018-07-06 | 郑州云海信息技术有限公司 | Write operation control method, system, device and storage medium based on bedding storage |
| CN108710472A (en) * | 2018-04-27 | 2018-10-26 | 北京大学深圳研究生院 | For the abrasion equilibrium management method and distributed memory system inside storage unit |
| CN109032529A (en) * | 2018-08-21 | 2018-12-18 | 郑州云海信息技术有限公司 | A kind of hard disk initialization method and system |
| CN109254962A (en) * | 2017-07-06 | 2019-01-22 | 中国移动通信集团浙江有限公司 | A kind of optimiged index method and device based on T- tree |
| WO2019024060A1 (en) * | 2017-08-03 | 2019-02-07 | 华为技术有限公司 | Data storage method and device, and storage medium |
| CN109408278A (en) * | 2017-08-15 | 2019-03-01 | 李剑春 | Data processing method and device |
| CN109542860A (en) * | 2018-10-25 | 2019-03-29 | 平安科技（深圳）有限公司 | Service data management method, terminal device based on HDFS |
| CN109634932A (en) * | 2018-11-30 | 2019-04-16 | 北京瑞卓喜投科技发展有限公司 | A kind of intelligence contract storage method and storage system |
| CN109960463A (en) * | 2017-12-26 | 2019-07-02 | 中兴通讯股份有限公司 | A kind of disk management method, device and storage medium |
| CN110427364A (en) * | 2019-06-21 | 2019-11-08 | 北京奇艺世纪科技有限公司 | A kind of data processing method, device, electronic equipment and storage medium |
| CN110569398A (en) * | 2019-09-09 | 2019-12-13 | 广州虎牙科技有限公司 | data processing method and device, computer equipment and readable storage medium |
| CN110866063A (en) * | 2018-08-27 | 2020-03-06 | 阿里巴巴集团控股有限公司 | Data tracking processing method and device |
| CN111190930A (en) * | 2019-12-29 | 2020-05-22 | 杭州拓深科技有限公司 | Fire fighting equipment historical data positioning system and positioning method integrating Hash and blocking |
| CN111190537A (en) * | 2019-12-10 | 2020-05-22 | 优刻得科技股份有限公司 | Method and system for managing sequential storage disks in write-addition scene |
| CN111309261A (en) * | 2020-02-16 | 2020-06-19 | 西安奥卡云数据科技有限公司 | Physical data position mapping method on single node in distributed storage system |
| CN111309772A (en) * | 2020-03-23 | 2020-06-19 | 优刻得科技股份有限公司 | Data operation management method and hardware processing module |
| CN111435286A (en) * | 2019-01-14 | 2020-07-21 | 深圳市茁壮网络股份有限公司 | Data storage method, device and system |
| CN111767436A (en) * | 2020-06-23 | 2020-10-13 | 北京思特奇信息技术股份有限公司 | HASH index data storage and reading method and system |
| CN111797102A (en) * | 2020-07-06 | 2020-10-20 | 亿度慧达教育科技（北京）有限公司 | Data writing method and device and electronic equipment |
| CN112162950A (en) * | 2020-09-11 | 2021-01-01 | 杭州涂鸦信息技术有限公司 | Data processing method and device based on file system and computer equipment |
| CN112241336A (en) * | 2019-07-19 | 2021-01-19 | 伊姆西Ip控股有限责任公司 | Method, apparatus and computer program product for backing up data |
| CN112667858A (en) * | 2020-12-25 | 2021-04-16 | 深圳创新科技术有限公司 | Method for storing data by adopting HASH chain and data writing and reading methods |
| CN113377709A (en) * | 2021-06-08 | 2021-09-10 | 北京计算机技术及应用研究所 | Hash-based HDFS (Hadoop distributed File System) back-end storage system |
| CN113703667A (en) * | 2021-07-14 | 2021-11-26 | 深圳市有为信息技术发展有限公司 | File system processing method and device for storing data in real time, vehicle-mounted terminal and commercial vehicle |
| CN114063884A (en) * | 2020-07-31 | 2022-02-18 | 伊姆西Ip控股有限责任公司 | Partitioning method, apparatus and computer program product for extended storage system |
| CN114662160A (en) * | 2022-05-25 | 2022-06-24 | 成都易我科技开发有限责任公司 | Digital summarization method, system and digital summarization method in network transmission |
| CN115617707A (en) * | 2022-12-07 | 2023-01-17 | 摩尔线程智能科技(北京)有限责任公司 | Device and method for managing storage space, computing equipment and chip |
| CN116304390A (en) * | 2023-04-13 | 2023-06-23 | 北京基调网络股份有限公司 | Time sequence data processing method and device, storage medium and electronic equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101551807A (en) * | 2009-05-07 | 2009-10-07 | 山东中创软件商用中间件股份有限公司 | Multilevel index technology for file database |
| CN101782922A (en) * | 2009-12-29 | 2010-07-21 | 山东山大鸥玛软件有限公司 | Multi-level bucket hashing index method for searching mass data |
| CN101968795A (en) * | 2010-09-03 | 2011-02-09 | 清华大学 | Cache method for file system with changeable data block length |
| CN102314503A (en) * | 2011-09-01 | 2012-01-11 | 浪潮(北京)电子信息产业有限公司 | Indexing method |
-
2012
- 2012-06-29 CN CN201210222998.0A patent/CN102779180B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101551807A (en) * | 2009-05-07 | 2009-10-07 | 山东中创软件商用中间件股份有限公司 | Multilevel index technology for file database |
| CN101782922A (en) * | 2009-12-29 | 2010-07-21 | 山东山大鸥玛软件有限公司 | Multi-level bucket hashing index method for searching mass data |
| CN101968795A (en) * | 2010-09-03 | 2011-02-09 | 清华大学 | Cache method for file system with changeable data block length |
| CN102314503A (en) * | 2011-09-01 | 2012-01-11 | 浪潮(北京)电子信息产业有限公司 | Indexing method |
Cited By (83)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103856516B (en) * | 2012-12-03 | 2018-05-25 | 腾讯科技（深圳）有限公司 | Data storage, read method and data storage, reading device |
| CN103856516A (en) * | 2012-12-03 | 2014-06-11 | 腾讯科技（深圳）有限公司 | Data storage and reading method and data storage and reading device |
| CN103226965A (en) * | 2013-03-29 | 2013-07-31 | 浙江大学 | Time-bitmap-based audio/video data access method |
| CN103226965B (en) * | 2013-03-29 | 2015-09-09 | 浙江大学 | Based on the audio/video data access method of time bitmap |
| CN104298681A (en) * | 2013-07-18 | 2015-01-21 | 华为技术有限公司 | Data storage method and device |
| CN104298681B (en) * | 2013-07-18 | 2018-06-19 | 华为技术有限公司 | A kind of date storage method and device |
| CN103500092A (en) * | 2013-09-23 | 2014-01-08 | 华为技术有限公司 | Judgment method for WORM appending writing and storage device |
| CN103500092B (en) * | 2013-09-23 | 2017-01-18 | 华为技术有限公司 | Judgment method for WORM appending writing and storage device |
| CN104462141A (en) * | 2013-09-24 | 2015-03-25 | 中国移动通信集团重庆有限公司 | Data storage and query method and system and storage engine device |
| CN104462141B (en) * | 2013-09-24 | 2018-05-22 | 中国移动通信集团重庆有限公司 | Method, system and the storage engines device of a kind of data storage and inquiry |
| CN103544259B (en) * | 2013-10-16 | 2017-01-18 | 国家计算机网络与信息安全管理中心 | Aggregating sorting TopK inquiry processing method and system |
| CN103544259A (en) * | 2013-10-16 | 2014-01-29 | 国家计算机网络与信息安全管理中心 | Aggregating sorting TopK inquiry processing method and system |
| CN103500227A (en) * | 2013-10-23 | 2014-01-08 | 腾讯科技（深圳）有限公司 | Video frame storage management method, related device and related communication system |
| WO2015058617A1 (en) * | 2013-10-23 | 2015-04-30 | 腾讯科技（深圳）有限公司 | Video frame storage management method, video server and communication system |
| CN103500227B (en) * | 2013-10-23 | 2017-02-22 | 腾讯科技（深圳）有限公司 | Video frame storage management method, related device and related communication system |
| CN103699337A (en) * | 2013-12-04 | 2014-04-02 | 华为技术有限公司 | Writing control method and system based on independent redundant array of independent disks (RAID) |
| CN103593486A (en) * | 2013-12-06 | 2014-02-19 | 国家电网公司 | Method for storing and reading mass data of power quality |
| CN103593486B (en) * | 2013-12-06 | 2017-11-21 | 国家电网公司 | A kind of quality of power supply mass data storage and read method |
| CN103631959A (en) * | 2013-12-17 | 2014-03-12 | 江苏名通信息科技有限公司 | Method for supporting million user data sub-meter based on Hash algorithm |
| CN105183532A (en) * | 2014-04-29 | 2015-12-23 | 哈尔滨工程大学 | Interface function performance optimization method in JNI mechanism based on Dalvik virtual machine |
| CN104035809A (en) * | 2014-04-29 | 2014-09-10 | 哈尔滨工程大学 | Method for optimizing performance of interface functions in JNI mechanism based on Dalvik virtual machine |
| CN104035809B (en) * | 2014-04-29 | 2017-11-17 | 哈尔滨工程大学 | One kind is based on interface function performance optimization method in Dalvik virtual machine JNI mechanism |
| CN105183532B (en) * | 2014-04-29 | 2019-02-26 | 哈尔滨工程大学 | One kind is based on interface function performance optimization method in Dalvik virtual machine JNI mechanism |
| CN104636185A (en) * | 2015-01-27 | 2015-05-20 | 华为技术有限公司 | Service context management method, physical host, PCIE equipment and migration management equipment |
| CN104636185B (en) * | 2015-01-27 | 2018-03-02 | 华为技术有限公司 | Business contexts management method, physical host, PCIE device and migration management equipment |
| WO2016169456A1 (en) * | 2015-04-20 | 2016-10-27 | Huawei Technologies Co., Ltd. | Serialization scheme for storing data and lightweight indices on devices with append-only bands |
| US10394786B2 (en) | 2015-04-20 | 2019-08-27 | Futurewei Technologies, Inc. | Serialization scheme for storing data and lightweight indices on devices with append-only bands |
| WO2016192328A1 (en) * | 2015-06-02 | 2016-12-08 | 上海斐讯数据通信技术有限公司 | Method for communicating between views in android system |
| CN106997783A (en) * | 2015-10-20 | 2017-08-01 | 三星电子株式会社 | The method of semiconductor storage and operation semiconductor storage |
| CN106997783B (en) * | 2015-10-20 | 2021-12-28 | 三星电子株式会社 | Semiconductor memory device and method of operating semiconductor memory device |
| CN106815260A (en) * | 2015-12-01 | 2017-06-09 | 阿里巴巴集团控股有限公司 | A kind of index establishing method and equipment |
| CN106815260B (en) * | 2015-12-01 | 2021-05-04 | 阿里巴巴集团控股有限公司 | Index establishing method and equipment |
| CN106940700A (en) * | 2016-01-05 | 2017-07-11 | 南京中兴新软件有限责任公司 | The covering write method of distributed file system, apparatus and system |
| CN106940700B (en) * | 2016-01-05 | 2021-11-12 | 南京中兴新软件有限责任公司 | Method, device and system for overwriting distributed file system |
| CN105574212A (en) * | 2016-02-24 | 2016-05-11 | 北京大学 | Image retrieval method for multi-index disk Hash structure |
| CN105574212B (en) * | 2016-02-24 | 2019-04-26 | 北京大学 | A kind of image search method of more index disk hash data structures |
| CN107277087A (en) * | 2016-04-06 | 2017-10-20 | 阿里巴巴集团控股有限公司 | Data processing method and device |
| CN107277087B (en) * | 2016-04-06 | 2020-09-04 | 阿里巴巴集团控股有限公司 | Data processing method and device |
| CN105847102A (en) * | 2016-04-29 | 2016-08-10 | 珠海格力电器股份有限公司 | Method, device and system realizing field bus communication |
| CN105847102B (en) * | 2016-04-29 | 2020-11-24 | 珠海格力智能装备技术研究院有限公司 | Method, equipment and system for realizing field bus communication |
| CN106874348A (en) * | 2016-12-26 | 2017-06-20 | 贵州白山云科技有限公司 | File is stored and the method for indexing means, device and reading file |
| CN107315543B (en) * | 2017-07-03 | 2020-10-09 | 合肥联宝信息技术有限公司 | Method and device for reading and writing data in EC (embedded control) memory |
| CN107315543A (en) * | 2017-07-03 | 2017-11-03 | 合肥联宝信息技术有限公司 | The reading/writing method and device of data in a kind of memory to EC |
| CN109254962A (en) * | 2017-07-06 | 2019-01-22 | 中国移动通信集团浙江有限公司 | A kind of optimiged index method and device based on T- tree |
| US11249969B2 (en) | 2017-08-03 | 2022-02-15 | Huawei Techologies Co., Ltd. | Data storage method and apparatus, and storage medium |
| WO2019024060A1 (en) * | 2017-08-03 | 2019-02-07 | 华为技术有限公司 | Data storage method and device, and storage medium |
| CN109408278A (en) * | 2017-08-15 | 2019-03-01 | 李剑春 | Data processing method and device |
| CN109408278B (en) * | 2017-08-15 | 2021-06-15 | 李剑春 | Data processing method and device |
| CN107704202A (en) * | 2017-09-18 | 2018-02-16 | 北京京东尚科信息技术有限公司 | A kind of method and apparatus of data fast reading and writing |
| CN108255958A (en) * | 2017-12-21 | 2018-07-06 | 百度在线网络技术（北京）有限公司 | Data query method, apparatus and storage medium |
| CN109960463B (en) * | 2017-12-26 | 2021-11-02 | 中兴通讯股份有限公司 | Disk management method, device and storage medium |
| CN109960463A (en) * | 2017-12-26 | 2019-07-02 | 中兴通讯股份有限公司 | A kind of disk management method, device and storage medium |
| CN108255432A (en) * | 2018-01-12 | 2018-07-06 | 郑州云海信息技术有限公司 | Write operation control method, system, device and storage medium based on bedding storage |
| CN108710472A (en) * | 2018-04-27 | 2018-10-26 | 北京大学深圳研究生院 | For the abrasion equilibrium management method and distributed memory system inside storage unit |
| CN109032529A (en) * | 2018-08-21 | 2018-12-18 | 郑州云海信息技术有限公司 | A kind of hard disk initialization method and system |
| CN110866063A (en) * | 2018-08-27 | 2020-03-06 | 阿里巴巴集团控股有限公司 | Data tracking processing method and device |
| CN110866063B (en) * | 2018-08-27 | 2023-10-31 | 阿里云计算有限公司 | Data tracking processing method and device |
| CN109542860B (en) * | 2018-10-25 | 2023-07-07 | 平安科技（深圳）有限公司 | Service data management method based on HDFS and terminal equipment |
| CN109542860A (en) * | 2018-10-25 | 2019-03-29 | 平安科技（深圳）有限公司 | Service data management method, terminal device based on HDFS |
| CN109634932A (en) * | 2018-11-30 | 2019-04-16 | 北京瑞卓喜投科技发展有限公司 | A kind of intelligence contract storage method and storage system |
| CN111435286A (en) * | 2019-01-14 | 2020-07-21 | 深圳市茁壮网络股份有限公司 | Data storage method, device and system |
| CN111435286B (en) * | 2019-01-14 | 2023-12-05 | 深圳市茁壮网络股份有限公司 | Data storage method, device and system |
| CN110427364A (en) * | 2019-06-21 | 2019-11-08 | 北京奇艺世纪科技有限公司 | A kind of data processing method, device, electronic equipment and storage medium |
| CN112241336A (en) * | 2019-07-19 | 2021-01-19 | 伊姆西Ip控股有限责任公司 | Method, apparatus and computer program product for backing up data |
| CN110569398A (en) * | 2019-09-09 | 2019-12-13 | 广州虎牙科技有限公司 | data processing method and device, computer equipment and readable storage medium |
| CN111190537B (en) * | 2019-12-10 | 2023-08-25 | 优刻得科技股份有限公司 | Method and system for managing sequential storage disk in additional writing scene |
| CN111190537A (en) * | 2019-12-10 | 2020-05-22 | 优刻得科技股份有限公司 | Method and system for managing sequential storage disks in write-addition scene |
| CN111190930B (en) * | 2019-12-29 | 2023-11-28 | 杭州拓深科技有限公司 | Fire-fighting equipment historical data positioning system and positioning method integrating hash and block |
| CN111190930A (en) * | 2019-12-29 | 2020-05-22 | 杭州拓深科技有限公司 | Fire fighting equipment historical data positioning system and positioning method integrating Hash and blocking |
| CN111309261A (en) * | 2020-02-16 | 2020-06-19 | 西安奥卡云数据科技有限公司 | Physical data position mapping method on single node in distributed storage system |
| CN111309772A (en) * | 2020-03-23 | 2020-06-19 | 优刻得科技股份有限公司 | Data operation management method and hardware processing module |
| CN111767436A (en) * | 2020-06-23 | 2020-10-13 | 北京思特奇信息技术股份有限公司 | HASH index data storage and reading method and system |
| CN111767436B (en) * | 2020-06-23 | 2023-11-10 | 北京思特奇信息技术股份有限公司 | HASH index data storage and reading method and system |
| CN111797102A (en) * | 2020-07-06 | 2020-10-20 | 亿度慧达教育科技（北京）有限公司 | Data writing method and device and electronic equipment |
| CN114063884A (en) * | 2020-07-31 | 2022-02-18 | 伊姆西Ip控股有限责任公司 | Partitioning method, apparatus and computer program product for extended storage system |
| CN112162950A (en) * | 2020-09-11 | 2021-01-01 | 杭州涂鸦信息技术有限公司 | Data processing method and device based on file system and computer equipment |
| CN112667858A (en) * | 2020-12-25 | 2021-04-16 | 深圳创新科技术有限公司 | Method for storing data by adopting HASH chain and data writing and reading methods |
| CN113377709A (en) * | 2021-06-08 | 2021-09-10 | 北京计算机技术及应用研究所 | Hash-based HDFS (Hadoop distributed File System) back-end storage system |
| CN113703667A (en) * | 2021-07-14 | 2021-11-26 | 深圳市有为信息技术发展有限公司 | File system processing method and device for storing data in real time, vehicle-mounted terminal and commercial vehicle |
| CN114662160A (en) * | 2022-05-25 | 2022-06-24 | 成都易我科技开发有限责任公司 | Digital summarization method, system and digital summarization method in network transmission |
| CN115617707A (en) * | 2022-12-07 | 2023-01-17 | 摩尔线程智能科技(北京)有限责任公司 | Device and method for managing storage space, computing equipment and chip |
| CN116304390A (en) * | 2023-04-13 | 2023-06-23 | 北京基调网络股份有限公司 | Time sequence data processing method and device, storage medium and electronic equipment |
| CN116304390B (en) * | 2023-04-13 | 2024-02-13 | 北京基调网络股份有限公司 | Time sequence data processing method and device, storage medium and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102779180B (en) | 2015-09-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102779180A (en) | Operation processing method of data storage system and data storage system | |
| US11799959B2 (en) | Data processing method, apparatus, and system | |
| CN104395904B (en) | Efficient data object storage and retrieval | |
| US8799601B1 (en) | Techniques for managing deduplication based on recently written extents | |
| US8250033B1 (en) | Replication of a data set using differential snapshots | |
| US9043334B2 (en) | Method and system for accessing files on a storage system | |
| US8924664B2 (en) | Logical object deletion | |
| US20140297603A1 (en) | Method and apparatus for deduplication of replicated file | |
| WO2016086819A1 (en) | Method and apparatus for writing data into shingled magnetic record smr hard disk | |
| CN102084360A (en) | Apparatus, system, and method for validating that a correct data segment is read from a data storage device | |
| CN104346357A (en) | File accessing method and system for embedded terminal | |
| KR20110039417A (en) | Apparatus, system, and method for efficient mapping of virtual and physical addresses | |
| CN104408111A (en) | Method and device for deleting duplicate data | |
| CN104424219A (en) | Method and equipment of managing data documents | |
| CN103186617A (en) | Data storage method and device | |
| US10423580B2 (en) | Storage and compression of an aggregation file | |
| US10095706B2 (en) | Data access system and data access method | |
| US20150302021A1 (en) | Storage system | |
| CN110352410A (en) | Track the access module and preextraction index node of index node | |
| JP2019028954A (en) | Storage control apparatus, program, and deduplication method | |
| CN104615459A (en) | MoCA equipment parameter configuration method and device | |
| US11016884B2 (en) | Virtual block redirection clean-up | |
| CN102063473B (en) | Method for expanding storage space on line | |
| US20120078922A1 (en) | Data reorganization | |
| CN104298467A (en) | p2p (peer to peer) cache file management method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20150909 Termination date: 20180629 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |