CN101644996A - Storage method of index data and storage control device - Google Patents
Storage method of index data and storage control device Download PDFInfo
- Publication number
- CN101644996A CN101644996A CN200910176946A CN200910176946A CN101644996A CN 101644996 A CN101644996 A CN 101644996A CN 200910176946 A CN200910176946 A CN 200910176946A CN 200910176946 A CN200910176946 A CN 200910176946A CN 101644996 A CN101644996 A CN 101644996A
- Authority
- CN
- China
- Prior art keywords
- index data
- index
- memory
- spatial cache
- disk
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a storage method of index data and a storage control device; the method comprises the following steps: index data of a disk of a coder, which is needed to be stored to a storage device, is cached; when the quantity of the cached index data reaches preset threshold, or a trigger event is detected, the cached index data is stored to the disk. By adopting the invention, the efficiency on reading-in the index data and video data to the disk can be improved.
Description
Technical field
The present invention relates to communication field, relate in particular to a kind of storage means and memory control device of index data.
Background technology
In the disk that existing memory device uses, data storage is on the disc of disk, and data write adds that by head arm the motion of disc itself locatees concrete position.At present, the development of exact instrument has made the rotating speed of disk rise to per minute commentaries on classics up to ten thousand, but compares still very low with circuit transmission speed in the computing machine.The magnetic head of hard disk moves to the required time of card specific track from initial position, is called the average seek time of hard disk, is the important parameter that influences the hard disk internal transfer rate.The average seek time of the disk that uses is about 7ms at present.
On the other hand, the transmission speed of disk itself is than faster, and the speed of disk interface has surpassed 100MBps already, and the PS read or write speed of disk can reach more than the 50MBps.The order read-write is meant that the address of the access object of disk is continuous, and such read-write has significantly reduced seek time.Comparatively speaking, random read-write is meant that the address of the access object of disk is discontinuous, and disk will consume in a large number and spend on the seek time.In reality test, when data block was very little, the performance of random read-write was general not as good as 1/10 of sequential read write performance, even during very big data block, the performance of random read-write is general not as good as 1/2 of sequential read write performance.
In the storage of the supervisory system of most existing realizes, for more rapidly and efficiently the video recording data are carried out query and search, general way is for the video recording data increase index, in stored video data, stores corresponding index entry; When recording a video inquiry,, can navigate to video data accurately fast by search index information.
The file memory format that media data and index data are stored simultaneously can be expressed as shown in Figure 1, and wherein, SUPER DATA (super data) back is MAIN INDEX (an one-level index), and size is 128K.One-level index block back is the data cell of a series of 256M size, is used to preserve data.The 256M Byte data cell that act as the retrieval back of one-level index block.In the data cell of each 256M, comprise SECOND INDEX (secondary index) and data block again, retrieve the data block that each I frame group data comprises by secondary index.
When disk is carried out write operation, video data with the storage unit of 16K Byte as minimum, is carried out IO operation, each I frame group is made up of the data block of a plurality of 16KB, write an I frame group after, write index operation.Each I frame group in secondary index item location.When the remaining space of the data cell of current 256M is not enough to store an I frame group, can gets the data cell of next 256M and store, also will write the one-level index this moment when writing secondary index, each 256M data block in one-level index entry location.
For the storage space of each scrambler channel allocation all is continuous.Because the singularity of monitoring business, writing all of video data writes in proper order, and this sequential write in continuous space is most effective.But in existing scheme, to the retrieval of video data, distributing the fixedly index block of block fixed size again in the storage space continuously during for convenience of playing back videos.Make each scrambler passage when writing data in memory device, every order writes I frame group data, just magnetic head must be moved to and write corresponding index entry in the index block, original sequential write has been become at random write.Thisly whenever write the realization that I frame group data are just write an index block, introduced the magnetic head seek time, feature based on the hard disk of describing in the background technology, seek time is the important parameter that influences the hard disk internal transfer rate, write at random a large amount of consumption of disk have been spent on the seek time, influenced the performance of write data.
Improve the exponential decline relation that is based on the memory device readwrite performance with IO read-write random rates, WriteMode at random of the prior art is particularly serious to the memory property influence under the low code check of monitoring business store video stream.
Storage flow such as a 2M bps code check writes corresponding SAN resource, and with the most frequently used setting, an I frame batch total was calculated in 1 second, and during write data, every 16KB does an IO operation, and that has write an I frame group, need carry out 128 IO operations.After having write I frame group, once write the IO operation of index.So the IO that stores under 2M bps code check storage flow read-write random rates is 0.78%.
By same computing method, the storage flow IO of 1Mbps code check read-write random rates is 1.6%; The storage flow IO read-write random rates of 512Kbps code check is 3.1%; The storage flow IO read-write random rates of 256Kbps code check is 6.2%.
As seen, the storage flow code check is low more, and random rates is high more.Through experiment, the disk read-write performance is exponential decline with the raising of IO read-write random rates, as shown in Figure 2.
Therefore, in existing scheme, an I frame of every sequential write group is write an index entry in the index block of arriving, and makes to write continuously to have become at random and writes.When this stores in low code check storage flow, produce very large IO read-write random rates, introduced the magnetic head seek time, influenced the efficient of hard disk internal data transfer, influenced the performance of write data.
Summary of the invention
The invention provides a kind of storage means and memory control device of index data, be used for improving the efficient that writes index data and video data to disk.
The invention provides a kind of storage means of index data, comprising:
The index data that scrambler need be stored into the disk of memory device carries out buffer memory;
When the quantity of the index data of buffer memory arrives preset threshold value or detects trigger event, with the index datastore in the described buffer memory in described disk.
Wherein, also comprise: divide a plurality of index data spatial caches in the position of the described index data of buffer memory, the size of described index data spatial cache is determined by the size of each described index data, the index of the corresponding LUN resource of described each index data.
Wherein, the described index data that scrambler need be stored into the disk of memory device carries out buffer memory and comprises:
Need store the index data of the LUN resource in the disk of memory device for scrambler into, be the index data spatial cache of a correspondence of described each LUN resources allocation, with the index data spatial cache of described index datastore to described distribution; And store corresponding relation between each LUN resource and each the index data spatial cache.
Wherein, described index datastore in the buffer memory is comprised in described disk:
For each index data spatial cache of having stored index data, obtain the index data of storing in the described index data spatial cache, and the corresponding relation between described LUN resource and the index data spatial cache; With the index datastore zone of described index datastore in the LUN resource corresponding with described index data spatial cache.
Wherein, when the cache location of described index data is the internal memory of described memory device, if described memory device receives the query requests to index data, then described memory device is searched the index data that relates in the described query requests in the index data spatial cache of described internal memory.
Wherein, described when detecting trigger event when the cache location of described index data is the flash memory of described scrambler, the index datastore in the described buffer memory is comprised in described disk:
When described memory device receives the query requests of index data, trigger described scrambler with the index datastore in the flash memory in described disk, described memory device is searched the index data that relates in the described query requests in described disk.
The present invention also provides a kind of memory control device of index data, comprising:
The index data buffer unit, the index data that is used for scrambler need be stored into the disk of memory device carries out buffer memory;
Index data unloading unit is used for when the quantity of the index data of described index data buffer unit buffer memory arrives preset threshold value or detects trigger event, with the index datastore in the described buffer memory in described disk.
Wherein, also comprise:
The spatial cache division unit, be used for dividing a plurality of index data spatial caches in the position of the described index data of buffer memory, the size of described index data spatial cache is determined by the size of each described index data, the index of the corresponding LUN resource of described each index data.
Wherein, described index data buffer unit, specifically be used for: the index data of LUN resource that need store the disk of memory device for scrambler into, in the index data spatial cache that described spatial cache division unit is divided, be the index data spatial cache of a described correspondence of described each LUN resources allocation, with the index data spatial cache of described index datastore to described distribution; And store corresponding relation between each LUN resource and each the index data spatial cache.
Wherein, described index data unloading unit, specifically be used for: at the index data spatial cache of described spatial cache division unit division, stored the index data spatial cache of index data for each, obtain the index data of storing in the described index data spatial cache, and the corresponding relation between described LUN resource and the index data spatial cache; With the index datastore zone of described index datastore in the LUN resource corresponding with described index data spatial cache.
Wherein, when the cache location of described index data was the internal memory of described memory device, described memory control device was arranged in described memory device, and described memory control device also comprises:
The query requests processing unit when being used to receive the query requests to index data, is searched the index data that relates in the described query requests in the index data spatial cache of described internal memory.
Wherein, when the cache location of described index data was the flash memory of described scrambler, described memory control device was arranged in described scrambler,
Described memory device comprises: the query requests processing unit, when being used to receive the query requests to index data, trigger described memory control device with the index datastore in the flash memory of described scrambler in the disk of described memory device, and in described disk, search the index data that relates in the described query requests;
The index data unloading unit of described memory control device specifically is used for: according to the triggering of described memory device, with the index datastore in the flash memory of described scrambler in the disk of described memory device.
Compared with prior art, the present invention has the following advantages:
The application of the invention, the index data that scrambler need be stored into the disk of memory device carries out buffer memory; When the quantity of the index data of buffer memory arrives preset threshold value or detects trigger event, with the index datastore in the described buffer memory in described disk, thereby the disk read-write random rates when having reduced the video data storage has improved the efficient that writes index data and video data in disk.
Description of drawings
In order to be illustrated more clearly in the embodiment of the invention or technical scheme of the prior art, to do to introduce simply to the accompanying drawing of required use in embodiment or the description of the Prior Art below, apparently, accompanying drawing in describing below only is some embodiments of the present invention, for those of ordinary skills, under the prerequisite of not paying creative work, can also obtain other accompanying drawing according to these accompanying drawings.
Fig. 1 is the file memory format synoptic diagram that the media data that provides in the prior art and index data are stored simultaneously;
Fig. 2 is that the disk read-write performance improves the synoptic diagram that descends with IO read-write random rates in the prior art;
Fig. 3 is the storage means process flow diagram of the index data that provides among the present invention;
Fig. 4 is that the disk address that uses among the present invention distributes synoptic diagram;
Fig. 5 is the form synoptic diagram of the secondary index that uses among the present invention;
Fig. 6 is the piecemeal synoptic diagram of index data spatial cache in the internal memory of the memory device that uses among the present invention;
Fig. 7 is the synoptic diagram that the index data in the internal memory and media data is write disk that uses among the present invention;
Fig. 8 is that the mutual synoptic diagram of signaling is inquired about in the video recording that uses among the present invention;
Fig. 9 is the structural representation of the memory control device that uses among the present invention.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the invention, the technical scheme in the embodiment of the invention is clearly and completely described, obviously, described embodiment only is the present invention's part embodiment, rather than whole embodiment.Based on the embodiment among the present invention, those of ordinary skills belong to the scope of protection of the invention not making the every other embodiment that is obtained under the creative work prerequisite.
In the prior art, with the monitor video data storage when the hard disk, just write an index data after whenever writing an I frame group, make data storage to the storage mode of hard disk for writing at random.In the storage means of index data provided by the invention, the subtend hard disk writes the time of index data and revises, the index data that needs is write hard disk carries out buffer memory in advance, when the index data of buffer memory has arrived some or detected trigger event when taking place, the index data of buffer memory is written in the hard disk.So just can effectively reduce the IO read-write random rates of monitor video data storage, improve the disk read-write performance.
Concrete, the storage means of index data provided by the invention as shown in Figure 3, comprising:
Step s301, the index data that scrambler need be stored into the disk of memory device carry out buffer memory;
Step s302, when the quantity of the index data of buffer memory arrives preset threshold value or detects trigger event, with the index datastore in the buffer memory in disk.
For example, after the quantity of the index data of buffer memory arrives 100, again property with 100 index data one-time writes of buffer memory in hard disk.These 100 index data correspondences 100 I frame groups, also promptly realized writing in proper order 100 I frame group data after, the zone of the storage index block of magnetic head tracking in the disk also writes index data.By this flow process, make IO read-write random rates be reduced to original 1%.Storage flow IO read-write random rates with the 256Kbps code check is an example, random rates is 6.2% in the existing techniques in realizing, and after using method provided by the invention, IO read-write random rates has been reduced to 0.062%, greatly reduce IO read-write random rates like this, improved the disk read-write performance greatly.
For example, be example with address block shown in Figure 4, in this sufficient address piece, wherein 0x111101～0x112100 address belongs to the zone of storing index data, and 0x112101～0x888899 address belongs to the zone of storage I frame group.
In the prior art scheme, every I frame group data (occupying 4 addresses with I frame group data is example) of writing in disk are then write index data one time in disk, write address is 0x112101 in proper order, 0x112102,0x112103,0x112104,0x111101,0x112105,0x112106,0x112107,0x112108,0x111102,0x112109...... above-mentioned writing mode need be finished tracking 5 times.
When using method provided by the invention, the index data that needs are written to disk carries out behind the buffer memory index data of whole buffer memorys is written to disk together again, and write address is in proper order: 0x112101,0x112102,0x112103,0x112104,0x112105,0x112106,0x112107,0x112108,0x111101,0x111102,0x112109.......In the mode of two index write-onces of above-mentioned buffer memory, only need finish tracking 3 times, reduced seek time, improved the storage write performance.
Be elaborated below in conjunction with the storage means of different application scenarioss to the index data that provides among the present invention.
In the application scenarios one of the present invention, in the internal memory of memory device, distribute a block space (hereinafter referred to as the index data spatial cache), be used for the index data that buffer memory need store the disk of memory device into.In the prior art, store the media data in the disk of memory device into for needs, usually also in the internal memory of memory device, distribute a block space (hereinafter referred to as the media data spatial cache), be used for media data is carried out buffer memory, by buffer memory to media data, when carrying out the reading of media data, can directly from buffer memory, read, improve reading speed.It is that the index data in the index data spatial cache is write disk that index data in the index data spatial cache arrives certain quantity.
For index data spatial cache and media data spatial cache, because in the video monitoring system, index data is wanted much less with respect to video data, therefore make with the index information in the index data spatial cache write disk number of times, than the number of times that the media data in the media data spatial cache is written to disk lack many, reduced the disk read-write random rates when video data is stored.
With 100 index datas of buffer memory in the index data spatial cache is example, and according to prior art, the total size of the secondary index of each 256M data block correspondence is 256KB, and is shown in Figure 5 under the secondary index form:
The secondary index sign has been described current last index entry numbering that writes, and the position of last I frame group of storing in the current 256M data block.This secondary index is identified at the renewal that writes a laggard line data of secondary index item.
Each secondary index item accounts for 16 bytes, represents the information of I frame group data.Whenever write I frame group data, just increase a secondary index item.
In the prior art, when media data is write the data block of a new 256MB, also need upgrade the one-level index, but just can write the one-level index one time owing to need the individual IO operation in 16386 ((256*1024-256)/16), therefore it is very little that the IO that causes writes random rates, therefore only consider in the method for the present invention secondary index is carried out buffer memory, buffer memory one-level index also can use method same among the present invention to realize if desired.
The Cache block size of the required distribution of 100 secondary index items of buffer memory is 1.6K, adds the secondary index sign that 16 bytes repeat to write, and distributes 2KB therefore for the index Cache of the LUN resource of an encoder stores passage correspondence.
A memory device can allow 1024 LUN resources, therefore for a memory device, distributes the space of 2MB for the index data spatial cache in internal memory.Can be for the storage of No. 1024 scrambler passages.The piecemeal of index Cache is illustrated in fig. 6 shown below.
Simultaneously, be positioned at the allocation table of index data spatial cache, be used to write down the corresponding relation of each index data spatial cache and LUN resource.The LUN resource identifies with LUN UUID (Universally UniqueIdentifier, general unique identifier).The form of this allocation table can be as shown in table 1:
Table 1. index data spatial cache allocation table
| Sequence number | Index data spatial cache address | ??LUN?UUID |
| 1 | ??0～2k | ??LUN1-UUID：xxxxxxxxx |
| 2 | ??2～4k | ??LUN2-UUID：xxxxxxxxx |
| 3 | ??4～6k | ??Null |
| ...... | ??...... | ??...... |
| 1024 | ??2046～2048k | ??LUN1024-UUID：xxxxxxxxx |
Index data spatial cache allocation table is a list item with the index data spatial cache address of 2K size, before not formulating any storage plan, promptly an index data spatial cache unallocated before, " LUN UUID " that this table is corresponding all is null.
According to the priority of the scrambler passage being formulated storage plan, promptly the LUN priority of distributing is distributed from small to large and is the index data spatial cache of null LUN UUID number.After storage plan is formulated successfully, fill in LUN UUID number in the index data spatial cache respective items of in index data spatial cache allocation table, distributing.After the storage plan deletion, in index data spatial cache allocation table, delete corresponding LUN UUID number, reset to null.
When scrambler passage 1 write data to the LUN1 of correspondence resource, it was in the piecemeal of 0～2k that secondary index item and up-to-date secondary index sign are write index Cache address in advance.And media data is write among the Cache that provides in the prior art.
For media data, from the media data spatial cache, write in the disk according to existing techniques in realizing.Only divide block writes to count down at 100 o'clock for secondary index information, just write in the disk at the index data spatial cache.This process as shown in Figure 7.
Among the present invention, memory device carries power-off protection, can guarantee that the data in the buffer memory can not lost because of outage within a certain period of time, can after recovering normally the data in the buffer memory be written in the disk.Therefore, because the power-off protection function of memory device self, make the problem that causes index information to be lost because of abnormal power-down to avoid.
In existing supervisory system,, can improve efficient to the retrieval and inquisition of storage video recording with media data and index data stores synchronized.After the order that receives read data, preferentially read index data or media data in the internal memory, if do not have the index data or the media data of coupling, index data in the reading disk or media data again in the internal memory.
Therefore, this scheme one index entry is stored in the indexed cache, even when recording a video inquiry, index entry is not written in the disk index block from the index data spatial cache, memory device is read data from this index data spatial cache earlier also, do not influence the reading of index, do not influence the professional normal execution of retrieval playback video recording.
When using the method that application scenarios one of the present invention provides, during from the index data spatial cache toward the disk write data, make IO as much as possible be written in the continuous data block, the index block number of times of writing that causes IO at random to write is reduced, thereby improve the performance that IO writes.
In the application scenarios two of the present invention, use the flash memory flash of scrambler itself to store index data in advance, regular again or Event triggered is written in the disk, reduces the IO random rates of disk write data.
Be example still with 100 index entries of buffer memory, similar according to passage allocation index metadata cache space in flash memory flash that scrambler self is supported to the method that provides in the above-mentioned application scenarios one, be each channel allocation 2KB space.
If scrambler is supported hyperchannel, then in flash memory flash, set up the index data spatial cache allocation table that is similar in the application scenarios one, come recording indexes corresponding codes device passage and corresponding LUN resource, be used for writing fashionable the differentiation to memory device.This table of definition is index-LUN resource mapping table in the scheme two.By LUN Initiator and LUN Target composite marker LUN resource.As shown in table 2:
Table 2 index data spatial cache index-LUN resource mapping table
| Sequence number | Index data spatial cache index address | ??LUN?Initiator | ??LUN?Target |
| 1 | ??0～2k | ??ECInitiator：1918 | ??ECTarget：1918:23691:1245921330 |
| 2 | ??2～4k | ??ECInitiator：1920 | ??ECTarget：1920:23693:1145691440 |
| 3 | ??4～6k | ??Null | ?? |
| 4 | ??6～8k | ??Null | ??Null |
| ...... | ??...... | ??...... |
Scrambler not with the index entry write storage device, but is recorded among the flash memory flash that scrambler self distributes to respective channel after video data is write memory device, and flash memory flash is written to index entry in the memory device regularly or after the Event triggered again.
Regularly with the mode of index write storage device, the mode that regularly index is write disk with buffer memory in the application scenarios one is the same for flash memory flash, can write operation count to 100 time by flash memory after, so with in the index entry write storage device of preserving in the flash memory.
By the mode of Event triggered with the index data write storage device of buffer memory in the flash memory, be to consider in the prior art, must first search index data to the inquiry of the media data stored in the memory device.Therefore client to memory device when initiating the data query request, management server at first sends the triggering signaling that the index data of buffer memory is write disk to scrambler, after scrambler writes the index data of buffer memory disk and writes successfully, management server reinforms the index information after the data management server inquiry is upgraded, realization inquires up-to-date index information, and then can inquire nearest video data video file.Signalling exchange process flow diagram in the above-mentioned flow process as shown in Figure 8.
When using the method that application scenarios two of the present invention provides, with scrambler flash memory flash storage index information, regularly or Event triggered be written in the memory device, the index block number of times of writing that causes IO at random to write is reduced, thereby improves the performance that IO writes.Because flash memory flash belongs to the hardware read-write,, after recovery is normal, the data in the flash memory are written in the disk so the back interior index data of flash memory that can guarantee equally to cut off the power supply is not lost.
The present invention also provides a kind of memory control device of index data, as shown in Figure 9, comprising:
Index data buffer unit 10, the index data that is used for scrambler need be stored into the disk of memory device carries out buffer memory;
Index data unloading unit 20 is used for when the quantity of the index data of described index data buffer unit buffer memory arrives preset threshold value or detects trigger event, with the index datastore in the described buffer memory in described disk.
Wherein, also comprise:
Spatial cache division unit 30 is used for dividing a plurality of index data spatial caches in the position of buffer memory index data, and the size of each index data spatial cache is determined by the size of each index data, the index of the corresponding LUN resource of each index data.
For above-mentioned index data buffer unit 10, specifically be used for: the index data of LUN resource that need store the disk of memory device for scrambler into, in the index data spatial cache that spatial cache division unit 30 is divided, be index data spatial cache of each LUN resources allocation, afterwards with the index data spatial cache of index datastore to distribution; And store corresponding relation between each LUN resource and each the index data spatial cache.
For above-mentioned index data unloading unit 20, specifically be used for: at the index data spatial cache of spatial cache division unit 30 divisions, stored the index data spatial cache of index data for each, obtain the index data of storing in the index data spatial cache, and the corresponding relation between LUN resource and the index data spatial cache; With the index datastore the obtained index datastore zone in the LUN resource corresponding with this index data spatial cache.
When the cache location of index data was the internal memory of memory device, memory control device provided by the invention was arranged in memory device, and this memory control device can also comprise:
The query requests processing unit when being used to receive the query requests to index data, is searched the index data that relates in this query requests in the index data spatial cache of the internal memory of memory device.
When the cache location of index data was the flash memory of scrambler, memory control device provided by the invention was arranged in described scrambler, under this situation:
Comprise the query requests processing unit in the memory device, when being used to receive the query requests to index data, trigger memory control device provided by the invention with the index datastore in the flash memory of scrambler in the disk of memory device, and in described disk, search the index data that relates in this query requests;
Index data unloading unit for memory control device provided by the invention specifically is used for the triggering according to memory device, with the index datastore in the flash memory of scrambler in the disk of memory device.
The application of the invention can improve the efficient that writes index data and video data in disk.
Through the above description of the embodiments, those skilled in the art can be well understood to the present invention and can realize by hardware, also can realize by the mode that software adds necessary general hardware platform.Based on such understanding, technical scheme of the present invention can embody with the form of software product, it (can be CD-ROM that this software product can be stored in a non-volatile memory medium, USB flash disk, portable hard drive etc.) in, comprise some instructions with so that computer equipment (can be personal computer, server, the perhaps network equipment etc.) carry out the described method of each embodiment of the present invention.
It will be appreciated by those skilled in the art that accompanying drawing is the synoptic diagram of a preferred embodiment, unit in the accompanying drawing or flow process might not be that enforcement the present invention is necessary.
It will be appreciated by those skilled in the art that the unit in the device among the embodiment can be distributed in the device of embodiment according to the embodiment description, also can carry out respective change and be arranged in the one or more devices that are different from present embodiment.A unit can be merged in the unit of the foregoing description, also can further split into a plurality of subelements.
The invention described above embodiment sequence number is not represented the quality of embodiment just to description.
Claims (12)
1, a kind of storage means of index data is characterized in that, comprising:
The index data that scrambler need be stored into the disk of memory device carries out buffer memory;
When the quantity of the index data of buffer memory arrives preset threshold value or detects trigger event, with the index datastore in the described buffer memory in described disk.
2, the method for claim 1, it is characterized in that, also comprise: divide a plurality of index data spatial caches in the position of the described index data of buffer memory, the size of described index data spatial cache is determined by the size of each described index data, the index of the corresponding LUN resource of described each index data.
3, method as claimed in claim 2 is characterized in that, the described index data that scrambler need be stored into the disk of memory device carries out buffer memory and comprises:
Need store the index data of the LUN resource in the disk of memory device for scrambler into, be the index data spatial cache of a correspondence of described each LUN resources allocation, with the index data spatial cache of described index datastore to described distribution; And store corresponding relation between each LUN resource and each the index data spatial cache.
4, method as claimed in claim 3 is characterized in that, described index datastore in the buffer memory is comprised in described disk:
For each index data spatial cache of having stored index data, obtain the index data of storing in the described index data spatial cache, and the corresponding relation between described LUN resource and the index data spatial cache; With the index datastore zone of described index datastore in the LUN resource corresponding with described index data spatial cache.
5, as each described method in the claim 1 to 4, it is characterized in that, when the cache location of described index data is the internal memory of described memory device, if described memory device receives the query requests to index data, then described memory device is searched the index data that relates in the described query requests in the index data spatial cache of described internal memory.
6, as each described method in the claim 1 to 4, it is characterized in that, described when detecting trigger event when the cache location of described index data is the flash memory of described scrambler, the index datastore in the described buffer memory is comprised in described disk:
When described memory device receives the query requests of index data, trigger described scrambler with the index datastore in the flash memory in described disk, described memory device is searched the index data that relates in the described query requests in described disk.
7, a kind of memory control device of index data is characterized in that, comprising:
The index data buffer unit, the index data that is used for scrambler need be stored into the disk of memory device carries out buffer memory;
Index data unloading unit is used for when the quantity of the index data of described index data buffer unit buffer memory arrives preset threshold value or detects trigger event, with the index datastore in the described buffer memory in described disk.
8, device as claimed in claim 7 is characterized in that, also comprises:
The spatial cache division unit, be used for dividing a plurality of index data spatial caches in the position of the described index data of buffer memory, the size of described index data spatial cache is determined by the size of each described index data, the index of the corresponding LUN resource of described each index data.
9, device as claimed in claim 8, it is characterized in that, described index data buffer unit, specifically be used for: the index data of LUN resource that need store the disk of memory device for scrambler into, in the index data spatial cache that described spatial cache division unit is divided, be the index data spatial cache of a described correspondence of described each LUN resources allocation, with the index data spatial cache of described index datastore to described distribution; And store corresponding relation between each LUN resource and each the index data spatial cache.
10, device as claimed in claim 9, it is characterized in that, described index data unloading unit, specifically be used for: at the index data spatial cache of described spatial cache division unit division, stored the index data spatial cache of index data for each, obtain the index data of storing in the described index data spatial cache, and the corresponding relation between described LUN resource and the index data spatial cache; With the index datastore zone of described index datastore in the LUN resource corresponding with described index data spatial cache.
As each described device in the claim 7 to 10, it is characterized in that 11, when the cache location of described index data was the internal memory of described memory device, described memory control device was arranged in described memory device, described memory control device also comprises:
The query requests processing unit when being used to receive the query requests to index data, is searched the index data that relates in the described query requests in the index data spatial cache of described internal memory.
As each described device in the claim 7 to 10, it is characterized in that 12, when the cache location of described index data was the flash memory of described scrambler, described memory control device was arranged in described scrambler,
Described memory device comprises: the query requests processing unit, when being used to receive the query requests to index data, trigger described memory control device with the index datastore in the flash memory of described scrambler in the disk of described memory device, and in described disk, search the index data that relates in the described query requests;
The index data unloading unit of described memory control device specifically is used for: according to the triggering of described memory device, with the index datastore in the flash memory of described scrambler in the disk of described memory device.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN200910176946A CN101644996A (en) | 2009-09-25 | 2009-09-25 | Storage method of index data and storage control device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN200910176946A CN101644996A (en) | 2009-09-25 | 2009-09-25 | Storage method of index data and storage control device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN101644996A true CN101644996A (en) | 2010-02-10 |
Family
ID=41656897
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN200910176946A Pending CN101644996A (en) | 2009-09-25 | 2009-09-25 | Storage method of index data and storage control device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101644996A (en) |
Cited By (26)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102207981A (en) * | 2011-07-13 | 2011-10-05 | 华为软件技术有限公司 | Method and system for managing file |
| CN102375694A (en) * | 2010-08-20 | 2012-03-14 | 杭州华三通信技术有限公司 | Data processing method and device |
| CN102387425A (en) * | 2010-08-30 | 2012-03-21 | 中兴通讯股份有限公司 | Caching device and method |
| CN102843396A (en) * | 2011-06-22 | 2012-12-26 | 中兴通讯股份有限公司 | Data writing and reading method and device in distributed caching system |
| CN102968422A (en) * | 2011-08-31 | 2013-03-13 | 中国航天科工集团第二研究院七○六所 | System and method for controlling streaming data storage |
| CN103037203A (en) * | 2012-12-11 | 2013-04-10 | 浙江宇视科技有限公司 | Method and device for index recovery based on block storage |
| CN104156385A (en) * | 2014-06-24 | 2014-11-19 | 北京京东尚科信息技术有限公司 | Method and device for processing time series data |
| CN104702874A (en) * | 2015-03-05 | 2015-06-10 | 安徽清新互联信息科技有限公司 | Storing method for file monitored by video |
| CN105094705A (en) * | 2015-07-27 | 2015-11-25 | 武汉兴图新科电子股份有限公司 | Method for optimizing disk storage strategy |
| CN106101596A (en) * | 2016-08-15 | 2016-11-09 | Tcl集团股份有限公司 | A kind of video storage method and device |
| CN106131473A (en) * | 2016-06-23 | 2016-11-16 | 深圳英飞拓科技股份有限公司 | The date storage method of video monitoring system and device |
| CN106294603A (en) * | 2016-07-29 | 2017-01-04 | 北京奇虎科技有限公司 | File memory method and device |
| WO2018014711A1 (en) * | 2016-07-19 | 2018-01-25 | 杭州海康威视数字技术股份有限公司 | Method and device for playing video |
| CN102737068B (en) * | 2011-04-15 | 2018-06-05 | 北京百度网讯科技有限公司 | A kind of method and apparatus for being used to carry out retrieval data cache management |
| CN108647278A (en) * | 2018-05-03 | 2018-10-12 | 中北大学 | A kind of file management method and system |
| CN108664577A (en) * | 2018-05-03 | 2018-10-16 | 中北大学 | A kind of file management method and system based on the free areas FLASH |
| CN108737853A (en) * | 2017-04-20 | 2018-11-02 | 腾讯科技（深圳）有限公司 | A kind of the drop code processing method and server of data file |
| CN109582230A (en) * | 2018-11-20 | 2019-04-05 | 深圳松诺技术有限公司 | A kind of data access method and access system based on matrix form index |
| CN109783321A (en) * | 2019-01-24 | 2019-05-21 | 深圳市景阳信息技术有限公司 | Monitoring data management method, device, terminal device |
| CN110764706A (en) * | 2019-10-25 | 2020-02-07 | 普联技术有限公司 | Storage system, data management method, and storage medium |
| CN111722789A (en) * | 2019-03-20 | 2020-09-29 | 点序科技股份有限公司 | Memory management method and memory storage device |
| CN112540731A (en) * | 2020-12-22 | 2021-03-23 | 北京百度网讯科技有限公司 | Data additional writing method, device, equipment, medium and program product |
| CN112954244A (en) * | 2021-01-19 | 2021-06-11 | 普联技术有限公司 | Method, device and equipment for realizing storage of monitoring video and storage medium |
| CN112988618A (en) * | 2021-02-20 | 2021-06-18 | 山东英信计算机技术有限公司 | Garbage recycling method, device and equipment for solid state disk and storage medium |
| CN113282582A (en) * | 2021-05-21 | 2021-08-20 | 海南超船电子商务有限公司 | Efficient storage method and system for ship position data |
| CN114327278A (en) * | 2021-12-30 | 2022-04-12 | 北京百度网讯科技有限公司 | Data additional writing method, device, equipment and storage medium |
-
2009
- 2009-09-25 CN CN200910176946A patent/CN101644996A/en active Pending
Cited By (42)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102375694A (en) * | 2010-08-20 | 2012-03-14 | 杭州华三通信技术有限公司 | Data processing method and device |
| CN102387425A (en) * | 2010-08-30 | 2012-03-21 | 中兴通讯股份有限公司 | Caching device and method |
| CN102737068B (en) * | 2011-04-15 | 2018-06-05 | 北京百度网讯科技有限公司 | A kind of method and apparatus for being used to carry out retrieval data cache management |
| CN102843396A (en) * | 2011-06-22 | 2012-12-26 | 中兴通讯股份有限公司 | Data writing and reading method and device in distributed caching system |
| CN102843396B (en) * | 2011-06-22 | 2018-03-13 | 中兴通讯股份有限公司 | Data write-in and read method and device in a kind of distributed cache system |
| CN102207981A (en) * | 2011-07-13 | 2011-10-05 | 华为软件技术有限公司 | Method and system for managing file |
| CN102968422B (en) * | 2011-08-31 | 2015-06-17 | 中国航天科工集团第二研究院七0六所 | System and method for controlling streaming data storage |
| CN102968422A (en) * | 2011-08-31 | 2013-03-13 | 中国航天科工集团第二研究院七○六所 | System and method for controlling streaming data storage |
| CN103037203B (en) * | 2012-12-11 | 2016-06-08 | 浙江宇视科技有限公司 | A kind of index restoration methods based on block storage and device |
| CN103037203A (en) * | 2012-12-11 | 2013-04-10 | 浙江宇视科技有限公司 | Method and device for index recovery based on block storage |
| CN104156385A (en) * | 2014-06-24 | 2014-11-19 | 北京京东尚科信息技术有限公司 | Method and device for processing time series data |
| CN104702874A (en) * | 2015-03-05 | 2015-06-10 | 安徽清新互联信息科技有限公司 | Storing method for file monitored by video |
| CN105094705A (en) * | 2015-07-27 | 2015-11-25 | 武汉兴图新科电子股份有限公司 | Method for optimizing disk storage strategy |
| CN106131473A (en) * | 2016-06-23 | 2016-11-16 | 深圳英飞拓科技股份有限公司 | The date storage method of video monitoring system and device |
| CN106131473B (en) * | 2016-06-23 | 2019-03-12 | 深圳英飞拓科技股份有限公司 | The date storage method and device of video monitoring system |
| WO2018014711A1 (en) * | 2016-07-19 | 2018-01-25 | 杭州海康威视数字技术股份有限公司 | Method and device for playing video |
| CN107635109A (en) * | 2016-07-19 | 2018-01-26 | 杭州海康威视数字技术股份有限公司 | One kind video recording player method and device |
| CN106294603A (en) * | 2016-07-29 | 2017-01-04 | 北京奇虎科技有限公司 | File memory method and device |
| CN106294603B (en) * | 2016-07-29 | 2019-07-23 | 北京奇虎科技有限公司 | File memory method and device |
| CN106101596A (en) * | 2016-08-15 | 2016-11-09 | Tcl集团股份有限公司 | A kind of video storage method and device |
| CN108737853A (en) * | 2017-04-20 | 2018-11-02 | 腾讯科技（深圳）有限公司 | A kind of the drop code processing method and server of data file |
| US11444998B2 (en) | 2017-04-20 | 2022-09-13 | Tencent Technology (Shenzhen) Company Limited | Bit rate reduction processing method for data file, and server |
| CN108737853B (en) * | 2017-04-20 | 2021-07-13 | 腾讯科技（深圳）有限公司 | Data file code reduction processing method and server |
| CN108647278B (en) * | 2018-05-03 | 2021-07-02 | 中北大学 | File management method and system |
| CN108664577A (en) * | 2018-05-03 | 2018-10-16 | 中北大学 | A kind of file management method and system based on the free areas FLASH |
| CN108647278A (en) * | 2018-05-03 | 2018-10-12 | 中北大学 | A kind of file management method and system |
| CN109582230A (en) * | 2018-11-20 | 2019-04-05 | 深圳松诺技术有限公司 | A kind of data access method and access system based on matrix form index |
| CN109582230B (en) * | 2018-11-20 | 2022-04-08 | 深圳松诺技术有限公司 | Matrix index-based data access method and access system |
| CN109783321B (en) * | 2019-01-24 | 2022-09-23 | 深圳市景阳信息技术有限公司 | Monitoring data management method and device and terminal equipment |
| CN109783321A (en) * | 2019-01-24 | 2019-05-21 | 深圳市景阳信息技术有限公司 | Monitoring data management method, device, terminal device |
| CN111722789B (en) * | 2019-03-20 | 2024-01-23 | 点序科技股份有限公司 | Memory management method and memory storage device |
| CN111722789A (en) * | 2019-03-20 | 2020-09-29 | 点序科技股份有限公司 | Memory management method and memory storage device |
| CN110764706A (en) * | 2019-10-25 | 2020-02-07 | 普联技术有限公司 | Storage system, data management method, and storage medium |
| CN110764706B (en) * | 2019-10-25 | 2023-10-03 | 普联技术有限公司 | Storage system, data management method, and storage medium |
| CN112540731B (en) * | 2020-12-22 | 2023-08-11 | 北京百度网讯科技有限公司 | Data append writing method, device, equipment, medium and program product |
| CN112540731A (en) * | 2020-12-22 | 2021-03-23 | 北京百度网讯科技有限公司 | Data additional writing method, device, equipment, medium and program product |
| CN112954244A (en) * | 2021-01-19 | 2021-06-11 | 普联技术有限公司 | Method, device and equipment for realizing storage of monitoring video and storage medium |
| CN112988618A (en) * | 2021-02-20 | 2021-06-18 | 山东英信计算机技术有限公司 | Garbage recycling method, device and equipment for solid state disk and storage medium |
| CN113282582A (en) * | 2021-05-21 | 2021-08-20 | 海南超船电子商务有限公司 | Efficient storage method and system for ship position data |
| CN113282582B (en) * | 2021-05-21 | 2023-06-20 | 海南超船电子商务有限公司 | Efficient storage method and system for ship position data |
| CN114327278A (en) * | 2021-12-30 | 2022-04-12 | 北京百度网讯科技有限公司 | Data additional writing method, device, equipment and storage medium |
| US11947842B2 (en) | 2021-12-30 | 2024-04-02 | Beijing Baidu Netcom Science Technology, Co., Ltd. | Method for writing data in append mode, device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101644996A (en) | Storage method of index data and storage control device | |
| US8782324B1 (en) | Techniques for managing placement of extents based on a history of active extents | |
| US9996466B2 (en) | Apparatus, system and method for caching compressed data | |
| US8909887B1 (en) | Selective defragmentation based on IO hot spots | |
| US8954699B1 (en) | Techniques for identifying IO hot spots using range-lock information | |
| US9128855B1 (en) | Flash cache partitioning | |
| US11429284B2 (en) | Data categorization based on invalidation velocities | |
| CN107924291B (en) | Storage system | |
| CN102713828B (en) | Disk cache method, device, and systemproviding Multi-device mirror images and stripe function | |
| CN103984648A (en) | Method to throttle rate of data caching for improved I/O performance | |
| US9612758B1 (en) | Performing a pre-warm-up procedure via intelligently forecasting as to when a host computer will access certain host data | |
| WO2018019119A1 (en) | Method and device for dynamic partial-parallel data layout for continuous data storage | |
| CN104346357A (en) | File accessing method and system for embedded terminal | |
| CN102609360A (en) | Data processing method, data processing device and data processing system | |
| CN103635887B (en) | Data caching method and storage system | |
| CN104794064A (en) | Cache management method based on region heat degree | |
| CN106569732B (en) | Data migration method and device | |
| CN103678523A (en) | Distributed cache data access method and device | |
| CN104620230A (en) | Method of managing memory | |
| US11461287B2 (en) | Managing a file system within multiple LUNS while different LUN level policies are applied to the LUNS | |
| CN111007991A (en) | Method for separating read-write request based on NVDIMM and computer thereof | |
| CN103399823A (en) | Method, equipment and system for storing service data | |
| US8402198B1 (en) | Mapping engine for a storage device | |
| CN105183374A (en) | Data read and write method and mainboard | |
| CN101566930B (en) | Virtual disk drive system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Open date: 20100210 |