CN102609360A - Data processing method, data processing device and data processing system - Google Patents
Data processing method, data processing device and data processing system Download PDFInfo
- Publication number
- CN102609360A CN102609360A CN2012100089393A CN201210008939A CN102609360A CN 102609360 A CN102609360 A CN 102609360A CN 2012100089393 A CN2012100089393 A CN 2012100089393A CN 201210008939 A CN201210008939 A CN 201210008939A CN 102609360 A CN102609360 A CN 102609360A
- Authority
- CN
- China
- Prior art keywords
- data
- data block
- block
- attribute
- memory block
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
An embodiment of the invention provides a data processing method, a data processing device and a data processing system. The data processing method includes: acquiring data writing requests comprising to-be-written data blocks containing heat attribute and compression attribute, writing the data blocks into corresponding target storage areas according to the heat attribute and the compression attribute, wherein the heat attribute includes a hot spot, a warm spot or a cold spot, the compression attribute includes a compressible attribute or non-compressible attribute. Accordingly, reading hit rate of data in cache or hot spot data disk is increased by utilizing data compression technology.
Description
Technical field
The present invention relates to computer realm, relate in particular to a kind of data processing method, Apparatus and system.
Background technology
How solving the IO bottleneck is the most popular topic in server field.The blast of information data is doubled and redoubled memory capacity, but the growth of storage and IO technology can't be caught up with the speed that information data amount increases, and causes the burden of IO serious all the more.The processor computing power that the compress technique utilization is more than needed both can reduce the capacity of data storage, can reduce the IO transmission bandwidth again; Caching technology (SSD buffer memory, Page Cache) is placed on speed memory module faster with hot spot data, reduces the IO bottleneck.
Operating system manufacturer provides the data compression function of file system and internal memory; Storage manufacturer provides the store compressed function; Database manufacturer provides the database compression function; Each manufacturer strives that separately compression function accomplishes transparently, avoids the influence to other functions of modules.But the function of Real Time Compression such as internal memory, file system is to different types of data piece Unified Treatment, and Real Time Compression compression algorithm ratio is not high, and these all cause present compress technique to have many defectives:
(1) from the internal memory to the file system to storage, each level provides transparent compression function separately, can't effectively cooperate.If open compression function simultaneously, possibly bring negative effect.
(2) the different compression algorithm is different to the different pieces of information compression efficiency, Unified Treatment, and performance boost is not obvious.
(3) buffer memory hot spot data real-time requires height, compresses the overhead that brings even might cause performance to descend.
Summary of the invention
Can't effectively cooperate each other to present various compression schemes; And hot spot data compresses the overhead problem of bringing; The embodiment of the invention combines compress technique and caching technology, and a kind of date storage method based on data compression is provided, and improves memory property.Concrete technical scheme is following:
The embodiment of the invention provides a kind of computing node; This computing node comprises hardware layer, operates in the operating system on the hardware layer and operates in the application layer on the operating system that said hardware layer comprises processor, memory controller, first storage array and second storage array; Said first storage array is rendered as two logical storage for operating system, is respectively first memory block and second memory block; Said second storage array is rendered as two logical storage to operating system, is respectively the 3rd memory block and the 4th memory block; The unpressed hot spot data of said first storage area stores, the warm spot data that said second storage area stores has been compressed, unpressed warm spot data of said the 3rd storage area stores and unpressed cold spot data, the cold spot data that said the 4th storage area stores has been compressed;
Said operating system is used for: the data that receive said application layer initiation write request; Said data write request and comprise data block to be written; Said data block has temperature attribute and packed attribute, and wherein said temperature attribute is focus, warm spot or cold spot, and said packed attribute is compressible or incompressible; Temperature attribute and packed attribute according to said data block; Send corresponding data to said processor and write instruction, by said processor said data are write instruction and send to said memory controller, said data write instruction and comprise the target memory block that said data block writes;
Said memory controller is used for: the data that receive said processor transmission write instruction, and write the target memory block that instruction writes said data block correspondence according to said data.
The embodiment of the invention also provides a kind of data processing method; This method is applied to computing node; Said computing node comprises first storage array and second storage array; Said first storage array is rendered as two logical storage for operating system, is respectively first memory block and second memory block; Said second storage array is rendered as two logical storage to operating system, is respectively the 3rd memory block and the 4th memory block, and this method comprises:
Obtain data and write request, said data write request and comprise data block to be written, and said data block comprises temperature attribute and packed attribute, and wherein said temperature attribute is focus, warm spot or cold spot, and said packed attribute is compressible or incompressible;
According to the temperature attribute and the packed attribute of said data block, said data block is write corresponding target memory block, said target memory block is the one or more memory blocks in first memory block, second memory block, the 3rd memory block and the 4th memory block.
The embodiment of the invention also provides a kind of computer system, and this system comprises the computing node that an embodiment of the invention provides at least.
In sum, computing node that the embodiment of the invention provides and computer system are utilized data compression technique, improve the hit rate of data at buffer memory or hot spot data dish; In addition; Temperature through to data is distinguished; For example the most frequent hot spot data of visit is not compressed, and visiting frequency warm spot data placed in the middle is carried out processed compressed, and the zone through storage area further being subdivided into stores compressed data with store the not zone of packed data; When utilizing compression to improve cache hit rate, some negative effects that data compression possibly bring have weakened; Once more, through increasing temperature attribute and packed attribute to data, between internal memory and the data storage, between buffer memory and the generic storage or can share the information of these data, realization collaborative work between hot spot data dish and the generic storage through information sharing.
Description of drawings
In order to be illustrated more clearly in the embodiment of the invention or technical scheme of the prior art; To do to introduce simply to the accompanying drawing of required use in embodiment or the description of the Prior Art below; Obviously, the accompanying drawing in describing below only is some embodiments of the present invention, for those of ordinary skills; Under the prerequisite of not paying creative work property, can also obtain other accompanying drawing according to these accompanying drawings.
The logical organization synoptic diagram of the computing node that Fig. 1 provides for the embodiment of the invention;
The storage area logical partitioning synoptic diagram of the computing node that Fig. 2 provides for the embodiment of the invention;
The logical organization synoptic diagram of the computer system that Fig. 3 provides for the embodiment of the invention;
The method flow synoptic diagram of a kind of data processing method that Fig. 4 provides for the embodiment of the invention;
The particular flow sheet of a kind of method for writing data that Fig. 5 provides for the embodiment of the invention;
The method flow synoptic diagram of the another kind of data processing method that Fig. 6 provides for the embodiment of the invention;
The particular flow sheet of the method for reading data that Fig. 7 provides for the embodiment of the invention;
The particular flow sheet of the data prefetching method that Fig. 8 provides for the embodiment of the invention;
The particular flow sheet of the data replacement method that Fig. 9 provides for the embodiment of the invention;
The particular flow sheet of the off-line data compression method that Figure 10 provides for the embodiment of the invention.
Embodiment
To combine the accompanying drawing in the embodiment of the invention below, the technical scheme in the embodiment of the invention is carried out clear, intactly description, obviously, described embodiment only is the present invention's part embodiment, rather than whole embodiment.Based on the embodiment among the present invention, those of ordinary skills are not making the every other embodiment that is obtained under the creative work prerequisite, all belong to the scope of the present invention's protection.
Please participate in Fig. 1, the structural representation of a kind of computing node that provides for the embodiment of the invention.As shown in Figure 1, this computing node comprises:
In this computing node; Operating system 20 is used for: the data that receive application layer 30 initiations write request, and said data write request and comprise data block to be written, and said data block has temperature attribute and packed attribute; Wherein said temperature attribute is focus, warm spot or cold spot; Said packed attribute is compressible or incompressible, according to the temperature attribute and the packed attribute of said data block, sends corresponding data to processor 11 and writes instruction; By processor 11 said data are write instruction and send to memory controller 12, said data write instruction and comprise the target memory block that said data block writes;
In a kind of implementation; The data that application layer 30 is initiated write the packed attribute of the data block that comprises in the request can be for compressible or incompressible; Promptly when packed attribute when being compressible; With same compression algorithm all data are compressed with corresponding and to be decompressed,, do not need data are compressed when packed attribute when being incompressible; Under another kind of implementation; When this packed attribute is compressible; Different data blocks can also have the different compression algorithm, promptly when the packed attribute of data block when being compressible, according to the concrete compression algorithm that the data block packed attribute points to this data block is carried out squeeze operation.
Further, operating system 20 also is used for when the packed attribute of said data block when being compressible the data compression instruction being sent to processor 11; Accordingly, processor 11 is used for: receive the data compression instruction that operating system 20 is sent, and said data block is carried out data compression step.
Particularly, operating system 20 is according to the temperature attribute and the packed attribute of said data block, sends corresponding data to said processor and writes instruction and can comprise following several kinds of situation:
If the temperature attribute of said data block is a focus, the data that said data block writes first memory block are write instruction to said processor transmission;
If the temperature attribute of said data block is a warm spot; And the packed attribute of said data block is compressible; Send the data compression instruction that said data block is compressed to said processor, and the data that the data block after will compressing writes second memory block write instruction;
If the temperature attribute of said data block is a warm spot, and the packed attribute of said data block is incompressible, sends the data that said data block is write the 3rd memory block to said processor and writes instruction;
If the temperature attribute of said data block is a cold spot; And the packed attribute of said data block is compressible; Send the data compression instruction that said data block is compressed to said processor, and the data that the data block after will compressing writes the 4th memory block write instruction;
If the temperature attribute of said data block is a cold spot, and the packed attribute of said data block is incompressible, sends the data that said data block is write the 3rd memory block to said processor and writes instruction.
In the computing node that the embodiment of the invention provides, first storage array can be solid-state hard disk SSD (Solid State Disk) array, and said second storage array is spinning disk HDD (Hard Disk Drive) array; Perhaps, said first storage array is the different SSD array of access performance with said second storage array.Wherein first storage array can be used as the high memory device of access performance, and this memory device can be done buffer memory, also can make the hot spot data dish.
The computing node that the embodiment of the invention provides can be a physical host; Specifically can be personal computer PC (Personal Computer), notebook computer laptop, workstation Workstation, server S erver, large scale computer Mainframe or supercomputer Supercomputer; Perhaps specifically can be mobile phone, intelligent terminal or the like, also can be virtual machine.
Except that said method, in the computing node that the embodiment of the invention provides, can also carry out operations such as corresponding data read, data replacement, data pre-fetching, off-line data compression.
In addition, need to prove that first and second just in order to distinguish storage array or memory block in the embodiment of the invention, and the meaning that limits of out-of-order.
See also Fig. 3, the logical organization synoptic diagram of a kind of computer system that provides for the embodiment of the invention, this system comprises a described computing node of the embodiment of the invention at least, can have communication between each computing node and connecting the connection of also can not communicating by letter.
In sum, computing node that the embodiment of the invention provides and computer system are utilized data compression technique, improve the hit rate of data at buffer memory or hot spot data dish, and support different data are taked the different compression algorithm, have improved the compression efficiency of data; In addition; Temperature through to data is distinguished; For example the most frequent hot spot data of visit is not compressed, and visiting frequency warm spot data placed in the middle is carried out processed compressed, and the zone through storage area further being subdivided into stores compressed data with store the not zone of packed data; When utilizing compression to improve cache hit rate, some negative effects that data compression possibly bring have weakened; Once more, through increasing temperature attribute and packed attribute to data, between internal memory and the data storage, between buffer memory and the generic storage or can share the information of these data, realization collaborative work between hot spot data dish and the generic storage through information sharing.
Please participate in Fig. 4, the method flow synoptic diagram of a kind of data processing method that provides for the embodiment of the invention.This method is applied to computing node, and said computing node comprises first storage array and second storage array, and said first storage array is rendered as two logical storage for operating system, is respectively first memory block and second memory block; Said second storage array is rendered as two logical storage to operating system, is respectively the 3rd memory block and the 4th memory block.Further, this method can be applied in the computing node that the embodiment of the invention provides.As shown in Figure 4, this method comprises:
S101, obtain data and write request, said data write request and comprise data block to be written, and said data block comprises temperature attribute and packed attribute, and wherein said temperature attribute is focus, warm spot or cold spot, and said packed attribute is compressible or incompressible; Said data write request and are generally initiated by application layer.
S102, according to the temperature attribute and the packed attribute of said data block, said data block is write corresponding target memory block, said target memory block is the one or more memory blocks in first memory block, second memory block, the 3rd memory block and the 4th memory block.
Particularly, this step S102 can comprise following several kinds of situation:
If the temperature attribute of said data block is a focus, said data block is write said first memory block;
If the temperature attribute of said data block is a warm spot, and the packed attribute of said data block is compressible, with writing said second memory block after the said data block compression;
If the temperature attribute of said data block is a warm spot, and the packed attribute of said data block is incompressible, and said data block is write said the 3rd memory block;
If the temperature attribute of said data block is a cold spot, and the packed attribute of said data block is compressible, with writing said the 4th memory block after the said data block compression;
If the temperature attribute of said data block is a cold spot, and the packed attribute of said data block is incompressible, and said data block is write said the 3rd memory block.
Should be understood that first storage array that the embodiment of the invention provided can be done buffer memory, also can make the hot spot data dish.When first storage array was made the hot spot data dish, second storage array was only stored the data except that the data of first storage array storage; When first storage array is done buffer memory, second storage array storage be the total data of whole system, comprise focus, warm spot and cold spot data.Therefore, when first storage array was buffer memory, said method write first storage array; Promptly the data of first memory block and second memory block can also write the corresponding memory block of second storage array; Therefore,, if the temperature attribute of said data block is a focus said data block is write after first memory block in the step of said method; Said method also comprises: if the packed attribute of said data block is compressible, with writing the 4th memory block after the said data block compression; If the packed attribute of said data block is incompressible, said data block is write the 3rd memory block.Accordingly; In the said method step if the temperature attribute of said data block is a warm spot; And the packed attribute of said data block is compressible, writes after second memory block after said data block is compressed, and said method also comprises: the data block after the said compression is write the 4th memory block.Should be understood that; Here said data block compression back is obtained the data block after the compression, carry out then data block after will compressing write the step of second memory block and will compress after data block write the step of the 4th memory block, the meaning of these two steps is that data block is promptly write buffer memory; Write the disk of storage total data again; The order of the two does not have special qualification, can carry out simultaneously, also can one formerly one after.In like manner abovementioned steps is also similar, repeats no more at this.
As shown in Figure 5, be the process flow diagram of a kind of implementation of the data processing method that provides of the embodiment of the invention:
401, obtain data and write request;
It can be that certain uses self-timing or not timing is initiated that these data write request, also can be caused by user's operation.
402, whether the temperature determined property data according to data are hot spot datas, if, change step 403, if not, change step 404;
403, data are write first memory block, data write operation finishes;
404, whether compressible according to the packed attribute judgment data of data, if incompressible, change step 406, if compressible, change step 405;
Whether the 405 temperature determined property data according to data are warm spot data, if, change step 407, if not, change step 408;
406 write the 3rd memory block with data, and data write operation finishes;
407 write second memory block after selecting corresponding compression algorithm with data compression according to the packed attribute of data, and data write operation finishes;
408 select corresponding compression algorithm according to the packed attribute of data, and with writing the 4th memory block after the data compression, data write operation finishes.
Here whether the packed attribute of data is not only can designation data compressible; The compression algorithm that can also designation data can take; For example packed attribute is that incompressible representative data can not be compressed, and packed attribute is LZW (Lempel-Ziv-Welch Encoding) compression algorithm, and then compressible the and compression algorithm that recommend of representative data is the LZW compression algorithm; Different like this data blocks just can be given the different compression algorithm; Thereby the characteristic according to data block is selected the proper compression compression algorithm, improves compression efficiency, and then improves memory property.
Need to prove; Fig. 5 is an example description, is a kind of implementation of the data processing method that provides of the embodiment of the invention, and those of ordinary skills expect other implementation in view of the above easily; The packed attribute of judgment data at first for example; With data qualification, the temperature attribute of judgment data then, the storage area of specified data etc.
See also Fig. 6, the method flow synoptic diagram of another data processing method that provides for the embodiment of the invention, as shown in Figure 6, this method comprises:
S201, obtain data read request, said data read request comprises data block sign to be read; This data read request generally is to be initiated by the application of application layer.
S202, be designated index with said data block to be read and from the metadata information that operating system is safeguarded, obtain the storage area of said data block to be read and the compressive state of said data block to be read, said compressive state is for compress or do not compress;
In a kind of implementation; Operating system can be safeguarded one or more metadata information tables; This metadata information table is owing to visit frequently can be stored in buffer memory or the hot spot data dish; This metadata information table comprises the metadata information of all data blocks, for example information such as the data temperature attribute of data block, packed attribute, compressive state attribute (having compressed or not compression), memory location.Wherein compression can be for compressible or incompressible, compressiblely can be specially the different compression algorithm again; The memory location can be first storage array or second storage array; Also can be specific to first or two or three or four memory blocks; Perhaps a certain more specifically sector; Perhaps all preserve to concrete a certain sector auxiliary information to little memory block from big storage array the memory location, and the embodiment of the invention is not done qualification to this.
S203, if the storage area of said data block to be read is first storage array, and the compressive state of said data block to be read is not compress, and reads said data block to be read from said first memory block;
S204, if the storage area of data block to be read is first storage array; And the compressive state of said data block to be read is to compress; With said data block decompression and move or be copied to first memory block, and read said data block from said first memory block;
S205, if the storage area of data block to be read is second storage array; And the compressive state of said data block to be read is not compress; With the migration of said data block to be read or copy to first memory block, and read said data block from said first memory block;
S206, if the storage area of data block to be read is second storage array; And the compressive state of said data block to be read is to compress; With said data block decompression to be read and move or be copied to first memory block, and read said data block from said first memory block.
This data processing method that Fig. 7 provides for the embodiment of the invention, the i.e. process flow diagram of a kind of implementation of method for reading data:
600, flow process begins;
601, obtain data read request, this data read request can be that a certain application self-timing or not timing are initiated, and also can operate triggering by the user;
602, whether judgment data is at first storage array, if execution in step 603 if not, is changeed step 605, first storage array is buffer memory or hot spot data dish in the embodiment of the invention, is used for storing the focus or the warm spot data of asking than frequentation;
603, whether judgment data is compressed, if execution in step 604 if not, is changeed step 608;
604, data are decompressed to first memory block from second memory block, the decompression algorithm of data can be chosen according to the packed attribute of data is corresponding here, also can adopt unified decompression algorithm;
605, whether judgment data is compressed, if execution in step 606 if not, is changeed step 607;
606, data are decompressed from the 4th memory block to first memory block;
607, data are moved or copy to first memory block from the 3rd memory block; Should be understood that; When first storage array is done buffer memory; Need the data of the 3rd memory block be copied to first memory block, when first storage array is made the hot spot data dish, can be with the data migtation of the 3rd memory block to first memory block;
608, from the first memory block reading of data;
609, data read operation finishes.
Need to prove that Fig. 7 only is a kind of implementation of the method for reading data that provides of the embodiment of the invention, those of ordinary skills draw other implementation alternative or that be equal in view of the above easily, and the present invention repeats no more at this.
The embodiment of the invention also provides the method flow synoptic diagram of another data processing method.This method can be regarded the data pre-fetching operation that operating system is initiated as; Soon regularly or irregularly will need the data of often visit to be prefetched to access speed memory block faster from the slow memory block of access speed; For example in the embodiment of the invention data are prefetched to first storage array (comprising first memory block and second memory block) of making buffer memory or making the hot spot data dish from first storage array (comprising the 3rd memory block and the 4th memory block) that the poor slightly HDD of access performance makes up, this first storage array can be implemented as SSD array at a high speed.This method comprises:
Obtain the data pre-fetching request, said data pre-fetching request comprises data block sign to be looked ahead;
Be designated index obtains data block said to be looked ahead from the metadata information that operating system is safeguarded compressive state with data block said to be looked ahead, said compressive state is for compress or not compression;
If the compressive state of data block said to be looked ahead is not compress, the data block that said data block sign is corresponding migration or copy to said first memory block from the 3rd memory block, and said data block is labeled as hot spot data;
If the compressive state of data block said to be looked ahead is to compress, when the clear area being arranged, with said data block decompression and move or copy to said first memory block, and said data block is labeled as hot spot data in said first memory block; When said first memory block does not have the clear area, with the migration of said data block or copy to said second memory block, and said data block is labeled as the warm spot data.
Need to prove it is whether first judgment data piece compresses that the embodiment of the invention is not intended to limit, still first judgment data piece is stored in that memory block.
Fig. 8 is the concrete realization flow figure of above-mentioned data processing method, and as shown in Figure 8, this method comprises:
801, data pre-fetching starts; Data pre-fetching request normally operating system is regular or irregular initiation, gets access to this data pre-fetching request log-on data prefetch operation afterwards.
802, whether judgment data is compressed.Pass through the compressive state information of query metadata information acquisition data in the embodiment of the invention.If execution in step 803 if not, is changeed step 804;
803, data are looked ahead to second memory block from the 4th memory block;
804, data are looked ahead to first memory block from the 3rd memory block;
805, judge whether first memory block is empty, if execution in step 806 if not, is changeed step 808;
806, data are contractd from the second memory block decompress(ion) read in first memory block;
807, mark the data as hot spot data;
808, mark the data as the warm spot data;
809, flow process finishes.
Need to prove; Below just illustrate; In the time of concrete the realization alternate manner can be arranged, for example before step 804 is read in data first memory block, can judge also whether first memory block has the free time, if idle execution in step 804 is arranged; If do not have a free time, can forgo data look ahead or reading in second memory block after the data compression.
Operate through data pre-fetching in the embodiment of the invention; Can will visit frequent relatively hot spot data be prefetched to the high storage array of access performance from the storage array of access performance difference; And the packed attribute according to data compresses data block in the process of looking ahead; Make more focus or warm spot data can be stored in the high storage array of access performance; Improve the hit rate that data read, according to the temperature attribute of data hot spot data is selected not compression memory simultaneously, in order to avoid the overhead that decompression brings influences the reading efficiency of data.
The embodiment of the invention also provides the method flow synoptic diagram of another data processing method.This method can be regarded the data replacement operation as; Promptly when the frequent data of visit in the high storage array of access performance become no longer frequently by visit; It is substituted into the low storage array of access performance from the high storage array of access performance; The replacement here can be that these data are taken out, and also can when taking out, replace with the frequent data of other visit.This method comprises:
Obtain the data replacement request, said data replacement request comprises data block sign to be replaced;
Be designated index obtains data block said to be replaced from the metadata information that operating system is safeguarded temperature attribute and packed attribute with data block said to be replaced;
If the temperature attribute of data block said to be replaced is a warm spot, and the packed attribute of data block said to be replaced is compressible, moves to second memory block with said data block compression and from first memory block;
If the temperature attribute of data block said to be replaced is a warm spot, and the packed attribute of data block said to be replaced is incompressible, and said data block is moved to the 3rd memory block from first memory block;
If the temperature attribute of data block said to be replaced is a cold spot, and the packed attribute of data block said to be replaced is compressible, and the 4th memory block is moved to from first memory block in said data block compression back;
If the temperature attribute of data block said to be replaced is a cold spot, and the packed attribute of data block said to be replaced is incompressible, and said data block is moved to the 3rd memory block from first memory block.
Fig. 9 is the particular flow sheet of above-mentioned data processing method, and is as shown in Figure 9, and this method comprises:
901, the data replacement operation begins.This operates, and normally operating system is regular or irregular initiation.
902, whether judgment data is the warm spot data, if execution in step 903 if not, is changeed step 906; Should be understood that; The data of the judgement here are the data of first memory block or second memory block normally; Why these data are stored in these two memory blocks is because their visit frequent degree are higher, but the visit frequent degree of data also is that the temperature of data can change; Therefore can carry out replacement operation to these data after a period of time, judge whether these data remain hot spot data.
903, judge whether these data are compressible, if, execution in step 904; If not, change step 905.
904, with replacing to second memory block from first memory block after this data compression.Here should be understood that data compression can be adopted unified compression algorithm, also can select the proper compression algorithm according to the packed attribute of data.
905, data are replaced to the 3rd memory block from first memory block.
906, whether judgment data is compressed, if, execution in step 907; If not, change step 908.
907, data are replaced to the 4th memory block from second memory block.
908, data are replaced to the 3rd memory block from first memory block.
909, the data replacement operation finishes.
Can find out from the data processing method that above-mentioned all embodiment provide; Carrying out that data read or data can be carried out online data compression or decompression according to demand when operation such as writing; And the data processing method that the embodiment of the invention provides is also supported the off-line data squeeze operation; Shown in figure 10, this example is the process of the data of the 3rd memory block being carried out the off-line compression, and this method comprises:
1001, off-line compression beginning.
1002, judged whether that data do not compress, if, execution in step 1003; If not, change step 1005.Need to prove that the embodiment of the invention is that hypothesis is all carried out processed compressed to the data of the 3rd memory block, the data of the 3rd memory block often are not compressed, and judge it is in order to guarantee more accurately not the redundant operation of packed data here.Certainly this determining step also can define according to actual needs separately, for example, if there is destination data to need compression, so directly these data compressed and get final product, does not need judgement.
1003, select compression algorithm, can select compression algorithm, also can take same compression algorithm all data according to the packed attribute of data.
1004, data are moved to the 4th memory block after the compression of the 3rd memory block, moving here can be migration, also can be to duplicate.
1005, the off-line squeeze operation finishes.
In sum, the data processing method that the embodiment of the invention provides is utilized data compression technique, improves the hit rate of data at buffer memory or hot spot data dish, and supports different data are taked the different compression algorithm, has improved the compression efficiency of data; In addition; Temperature through to data is distinguished; For example the most frequent hot spot data of visit is not compressed, and visiting frequency warm spot data placed in the middle is carried out processed compressed, and the zone through storage area further being subdivided into stores compressed data with store the not zone of packed data; When utilizing compression to improve cache hit rate, some negative effects that data compression possibly bring have weakened; Once more, through increasing temperature attribute and packed attribute to data, between internal memory and the data storage, between buffer memory and the generic storage or can share the information of these data, realization collaborative work between hot spot data dish and the generic storage through information sharing.
Claims (12)
1. computing node; It is characterized in that; Said computing node comprises hardware layer, operates in the operating system on the hardware layer and operates in the application layer on the operating system that said hardware layer comprises processor, memory controller, first storage array and second storage array; Said first storage array is rendered as two logical storage for operating system, is respectively first memory block and second memory block; Said second storage array is rendered as two logical storage to operating system, is respectively the 3rd memory block and the 4th memory block; The unpressed hot spot data of said first storage area stores, the warm spot data that said second storage area stores has been compressed, unpressed warm spot data of said the 3rd storage area stores and unpressed cold spot data, the cold spot data that said the 4th storage area stores has been compressed;
Said operating system is used for: the data that receive said application layer initiation write request; Said data write request and comprise data block to be written; Said data block has temperature attribute and packed attribute, and wherein said temperature attribute is focus, warm spot or cold spot, and said packed attribute is compressible or incompressible; Temperature attribute and packed attribute according to said data block; Send corresponding data to said processor and write instruction, by said processor said data are write instruction and send to said memory controller, said data write instruction and comprise the target memory block that said data block writes;
Said memory controller is used for: the data that receive said processor transmission write instruction, and write the target memory block that instruction writes said data block correspondence according to said data.
2. method according to claim 1 is characterized in that,
Said operating system also is used for: when the packed attribute of said data block when being compressible, the data compression instruction is sent to said processor;
Said processor is used for: receive the data compression instruction that said operating system is sent, and said data block is carried out data compression step.
3. method according to claim 2 is characterized in that, said temperature attribute and packed attribute according to said data block send corresponding data to said processor and write instruction, specifically comprise:
If the temperature attribute of said data block is a focus, the data that said data block writes first memory block are write instruction to said processor transmission;
If the temperature attribute of said data block is a warm spot; And the packed attribute of said data block is compressible; Send the data compression instruction that said data block is compressed to said processor, and the data that the data block after will compressing writes second memory block write instruction;
If the temperature attribute of said data block is a warm spot, and the packed attribute of said data block is incompressible, sends the data that said data block is write the 3rd memory block to said processor and writes instruction;
If the temperature attribute of said data block is a cold spot; And the packed attribute of said data block is compressible; Send the data compression instruction that said data block is compressed to said processor, and the data that the data block after will compressing writes the 4th memory block write instruction;
If the temperature attribute of said data block is a cold spot, and the packed attribute of said data block is incompressible, sends the data that said data block is write the 3rd memory block to said processor and writes instruction.
4. according to each described method of claim 1 to 3, it is characterized in that said first storage array is the solid-state hard disk SSD array, said second storage array is a spinning disk HDD array; Perhaps, said first storage array is the different SSD array of access performance with said second storage array.
5. data processing method; It is characterized in that said method is applied to computing node, said computing node comprises first storage array and second storage array; Said first storage array is rendered as two logical storage for operating system, is respectively first memory block and second memory block; Said second storage array is rendered as two logical storage to operating system, is respectively the 3rd memory block and the 4th memory block, and said method comprises:
Obtain data and write request, said data write request and comprise data block to be written, and said data block has temperature attribute and packed attribute, and wherein said temperature attribute is focus, warm spot or cold spot, and said packed attribute is compressible or incompressible;
According to the temperature attribute and the packed attribute of said data block, said data block is write corresponding target memory block, said target memory block is the one or more memory blocks in first memory block, second memory block, the 3rd memory block and the 4th memory block.
6. method according to claim 5 is characterized in that, said temperature attribute and packed attribute according to said data block write corresponding target memory block with said data block, comprising:
If the temperature attribute of said data block is a focus, said data block is write said first memory block;
If the temperature attribute of said data block is a warm spot, and the packed attribute of said data block is compressible, with writing said second memory block after the said data block compression;
If the temperature attribute of said data block is a warm spot, and the packed attribute of said data block is incompressible, and said data block is write said the 3rd memory block;
If the temperature attribute of said data block is a cold spot, and the packed attribute of said data block is compressible, with writing said the 4th memory block after the said data block compression;
If the temperature attribute of said data block is a cold spot, and the packed attribute of said data block is incompressible, and said data block is write said the 3rd memory block.
7. method according to claim 6 is characterized in that, if the temperature attribute of said said data block is a focus, and said data block is write after said first memory block, and said method also comprises:
If the packed attribute of said data block is compressible, with writing the 4th memory block after the said data block compression;
If the packed attribute of said data block is incompressible, said data block is write the 3rd memory block.
8. method according to claim 6 is characterized in that, if the temperature attribute of said said data block is a warm spot, and the packed attribute of said data block is compressible, and with writing after the said data block compression after second memory block, said method also comprises:
Data block after the said compression is write the 4th memory block.
9. method according to claim 5 is characterized in that, said method also comprises:
Obtain data read request, said data read request comprises data block sign to be read;
Be designated index with said data block to be read and from the metadata information that operating system is safeguarded, obtain the storage area of said data block to be read and the compressive state of said data block to be read, said compressive state is for compress or do not compress;
If the storage area of said data block to be read is first storage array, and the compressive state of said data block to be read is not compress, and reads said data block to be read from said first memory block;
If the storage area of data block to be read is first storage array, and the compressive state of said data block to be read is to compress, with said data block decompression and move or be copied to first memory block, and reads said data block from said first memory block;
If the storage area of data block to be read is second storage array, and the compressive state of said data block to be read is not compress, and with said data block to be read migration or copy to first memory block, and reads said data block from said first memory block;
If the storage area of data block to be read is second storage array; And the compressive state of said data block to be read is to compress; With said data block decompression to be read and move or be copied to first memory block, and read said data block from said first memory block.
10. method according to claim 5 is characterized in that, said method also comprises:
Obtain the data pre-fetching request, said data pre-fetching request comprises data block sign to be looked ahead;
Be designated index obtains data block said to be looked ahead from the metadata information that operating system is safeguarded compressive state with data block said to be looked ahead, said compressive state is for compress or not compression;
If the compressive state of data block said to be looked ahead is not compress, the data block that said data block sign is corresponding migration or copy to said first memory block from the 3rd memory block, and said data block is labeled as hot spot data;
If the compressive state of data block said to be looked ahead is to compress, when the clear area being arranged, with said data block decompression and move or copy to said first memory block, and said data block is labeled as hot spot data in said first memory block; When said first memory block does not have the clear area, with the migration of said data block or copy to said second memory block, and said data block is labeled as the warm spot data.
11. method according to claim 5 is characterized in that, said method also comprises:
Obtain the data replacement request, said data replacement request comprises data block sign to be replaced;
Be designated index obtains data block said to be replaced from the metadata information that operating system is safeguarded temperature attribute and packed attribute with data block said to be replaced;
If the temperature attribute of data block said to be replaced is a warm spot, and the packed attribute of data block said to be replaced is compressible, moves to second memory block with said data block compression and from first memory block;
If the temperature attribute of data block said to be replaced is a warm spot, and the packed attribute of data block said to be replaced is incompressible, and said data block is moved to the 3rd memory block from first memory block;
If the temperature attribute of data block said to be replaced is a cold spot, and the packed attribute of data block said to be replaced is compressible, and the 4th memory block is moved to from first memory block in said data block compression back;
If the temperature attribute of data block said to be replaced is a cold spot, and the packed attribute of data block said to be replaced is incompressible, and said data block is moved to the 3rd memory block from first memory block.
12. a computer system is characterized in that, comprising: at least one is like each described computing node of claim 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210008939.3A CN102609360B (en) | 2012-01-12 | 2012-01-12 | Data processing method, data processing device and data processing system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210008939.3A CN102609360B (en) | 2012-01-12 | 2012-01-12 | Data processing method, data processing device and data processing system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102609360A true CN102609360A (en) | 2012-07-25 |
| CN102609360B CN102609360B (en) | 2015-03-25 |
Family
ID=46526750
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210008939.3A Expired - Fee Related CN102609360B (en) | 2012-01-12 | 2012-01-12 | Data processing method, data processing device and data processing system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102609360B (en) |
Cited By (37)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102929783A (en) * | 2012-10-25 | 2013-02-13 | 华为技术有限公司 | Data storage method, device and system |
| CN103076995A (en) * | 2012-12-31 | 2013-05-01 | 深圳市硅格半导体有限公司 | NandFlash data processing method and device |
| CN103250143A (en) * | 2012-12-28 | 2013-08-14 | 华为技术有限公司 | Data storage method and storage device |
| CN103294167A (en) * | 2013-05-21 | 2013-09-11 | 暨南大学 | Data behavior based low-energy consumption cluster storage replication device and method |
| CN103631722A (en) * | 2012-08-29 | 2014-03-12 | 联想(北京)有限公司 | Data processing method and electronic equipment |
| CN103838680A (en) * | 2012-11-23 | 2014-06-04 | 北京四达时代软件技术股份有限公司 | Data caching method and device |
| CN104239231A (en) * | 2014-09-01 | 2014-12-24 | 上海爱数软件有限公司 | Method and device for accelerating preheating of second-level cache |
| CN104391661A (en) * | 2014-12-12 | 2015-03-04 | 西安三星电子研究有限公司 | Method and equipment for writing data into solid hard disk |
| CN104765571A (en) * | 2015-03-17 | 2015-07-08 | 深信服网络科技（深圳）有限公司 | Virtual data writing and reading method and system |
| CN105320669A (en) * | 2014-06-19 | 2016-02-10 | 腾讯科技（深圳）有限公司 | Method and device for data storage and method and device for data reading |
| CN105426113A (en) * | 2014-09-05 | 2016-03-23 | 群联电子股份有限公司 | Memory management method, memory storage apparatus and memory control circuit unit |
| CN105631035A (en) * | 2016-01-04 | 2016-06-01 | 北京百度网讯科技有限公司 | Data storage method and device |
| CN105988720A (en) * | 2015-02-09 | 2016-10-05 | 中国移动通信集团浙江有限公司 | Data storage device and method |
| CN106406759A (en) * | 2016-09-13 | 2017-02-15 | 郑州云海信息技术有限公司 | Data storage method and device |
| CN106557469A (en) * | 2015-09-24 | 2017-04-05 | 阿里巴巴集团控股有限公司 | The method and device of data in a kind of processing data warehouse |
| US9632927B2 (en) | 2014-09-25 | 2017-04-25 | International Business Machines Corporation | Reducing write amplification in solid-state drives by separating allocation of relocate writes from user writes |
| CN106909398A (en) * | 2017-03-10 | 2017-06-30 | 深圳创维－Rgb电子有限公司 | The compression method and device of a kind of executable file |
| US9760310B2 (en) | 2015-08-06 | 2017-09-12 | International Business Machines Corporation | High performance data storage management using dynamic compression |
| US9779021B2 (en) | 2014-12-19 | 2017-10-03 | International Business Machines Corporation | Non-volatile memory controller cache architecture with support for separation of data streams |
| US9886208B2 (en) | 2015-09-25 | 2018-02-06 | International Business Machines Corporation | Adaptive assignment of open logical erase blocks to data streams |
| CN107797758A (en) * | 2016-09-07 | 2018-03-13 | 腾讯科技（深圳）有限公司 | Date storage method, data access method and device |
| US10078582B2 (en) | 2014-12-10 | 2018-09-18 | International Business Machines Corporation | Non-volatile memory system having an increased effective number of supported heat levels |
| CN108984103A (en) * | 2017-06-02 | 2018-12-11 | 伊姆西Ip控股有限责任公司 | Method and apparatus for duplicate removal |
| CN109032965A (en) * | 2017-06-12 | 2018-12-18 | 华为技术有限公司 | A kind of method for reading data, host and storage equipment |
| CN109582245A (en) * | 2018-12-06 | 2019-04-05 | 联想(北京)有限公司 | Data processing method, device and equipment |
| CN109802684A (en) * | 2018-12-26 | 2019-05-24 | 华为技术有限公司 | The method and apparatus for carrying out data compression |
| CN110018879A (en) * | 2018-01-09 | 2019-07-16 | 阿里巴巴集团控股有限公司 | Delay loading method and device applied to distributed system |
| CN110879688A (en) * | 2019-10-25 | 2020-03-13 | 苏州浪潮智能科技有限公司 | Storage compression method and system |
| CN110908608A (en) * | 2019-11-22 | 2020-03-24 | 苏州浪潮智能科技有限公司 | Storage space saving method and system |
| WO2020073328A1 (en) * | 2018-10-12 | 2020-04-16 | 华为技术有限公司 | Data processing method and device |
| CN111562883A (en) * | 2020-04-26 | 2020-08-21 | 北京泽石科技有限公司 | Cache management system, method and device for solid state disk |
| CN111813340A (en) * | 2020-07-10 | 2020-10-23 | 杭州海康威视数字技术股份有限公司 | Instruction response method, system and device based on solid state disk and electronic equipment |
| CN112306974A (en) * | 2019-07-30 | 2021-02-02 | 深信服科技股份有限公司 | Data processing method, device, equipment and storage medium |
| US10924356B2 (en) | 2016-10-14 | 2021-02-16 | Tencent Technology (Shenzhen) Company Limited | Network service scheduling method and apparatus, storage medium, and program product |
| WO2021036329A1 (en) * | 2019-08-28 | 2021-03-04 | 华为技术有限公司 | Data storage method and apparatus |
| CN112487326A (en) * | 2020-11-27 | 2021-03-12 | 杭州安恒信息技术股份有限公司 | Data caching method, system, storage medium and equipment |
| CN112306974B (en) * | 2019-07-30 | 2024-10-22 | 深信服科技股份有限公司 | Data processing method, device, equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070266053A1 (en) * | 2005-12-22 | 2007-11-15 | Shlomo Ahal | Methods and apparatus for multiple point in time data access |
| CN101419588A (en) * | 2007-10-26 | 2009-04-29 | 康佳集团股份有限公司 | Chinese character compress encoding method |
| CN101907978A (en) * | 2010-07-27 | 2010-12-08 | 浙江大学 | Mixed storage system and storage method based on solid state disk and magnetic hard disk |
| US20110138115A1 (en) * | 2007-08-14 | 2011-06-09 | Samsung Electronics Co., Ltd. | Solid state memory (ssm), computer system including an ssm, and method of operating an ssm |
| CN102129472A (en) * | 2011-04-14 | 2011-07-20 | 上海红神信息技术有限公司 | Construction method for high-efficiency hybrid storage structure of semantic-orient search engine |
| CN102272742A (en) * | 2009-01-29 | 2011-12-07 | Lsi公司 | Allocate-on-write snapshot mechanism to provide dynamic storage tiering on-line data placement for volumes |
-
2012
- 2012-01-12 CN CN201210008939.3A patent/CN102609360B/en not_active Expired - Fee Related
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070266053A1 (en) * | 2005-12-22 | 2007-11-15 | Shlomo Ahal | Methods and apparatus for multiple point in time data access |
| US20110138115A1 (en) * | 2007-08-14 | 2011-06-09 | Samsung Electronics Co., Ltd. | Solid state memory (ssm), computer system including an ssm, and method of operating an ssm |
| CN101419588A (en) * | 2007-10-26 | 2009-04-29 | 康佳集团股份有限公司 | Chinese character compress encoding method |
| CN102272742A (en) * | 2009-01-29 | 2011-12-07 | Lsi公司 | Allocate-on-write snapshot mechanism to provide dynamic storage tiering on-line data placement for volumes |
| CN101907978A (en) * | 2010-07-27 | 2010-12-08 | 浙江大学 | Mixed storage system and storage method based on solid state disk and magnetic hard disk |
| CN102129472A (en) * | 2011-04-14 | 2011-07-20 | 上海红神信息技术有限公司 | Construction method for high-efficiency hybrid storage structure of semantic-orient search engine |
Cited By (63)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103631722A (en) * | 2012-08-29 | 2014-03-12 | 联想(北京)有限公司 | Data processing method and electronic equipment |
| CN102929783B (en) * | 2012-10-25 | 2015-12-02 | 华为技术有限公司 | The methods, devices and systems that a kind of data store |
| CN102929783A (en) * | 2012-10-25 | 2013-02-13 | 华为技术有限公司 | Data storage method, device and system |
| CN103838680A (en) * | 2012-11-23 | 2014-06-04 | 北京四达时代软件技术股份有限公司 | Data caching method and device |
| CN103838680B (en) * | 2012-11-23 | 2018-03-23 | 北京四达时代软件技术股份有限公司 | A kind of data cache method and device |
| CN103250143B (en) * | 2012-12-28 | 2015-03-11 | 华为技术有限公司 | Data storage method and storage device |
| US8838892B2 (en) | 2012-12-28 | 2014-09-16 | Huawei Technologies Co., Ltd. | Data storage method and storage device |
| CN103250143A (en) * | 2012-12-28 | 2013-08-14 | 华为技术有限公司 | Data storage method and storage device |
| CN103076995B (en) * | 2012-12-31 | 2015-11-25 | 深圳市硅格半导体有限公司 | The data processing method of NandFlash and device |
| CN103076995A (en) * | 2012-12-31 | 2013-05-01 | 深圳市硅格半导体有限公司 | NandFlash data processing method and device |
| CN103294167A (en) * | 2013-05-21 | 2013-09-11 | 暨南大学 | Data behavior based low-energy consumption cluster storage replication device and method |
| CN103294167B (en) * | 2013-05-21 | 2016-02-10 | 暨南大学 | A kind of low energy consumption cluster-based storage reproducing unit based on data behavior and method |
| CN105320669A (en) * | 2014-06-19 | 2016-02-10 | 腾讯科技（深圳）有限公司 | Method and device for data storage and method and device for data reading |
| CN104239231A (en) * | 2014-09-01 | 2014-12-24 | 上海爱数软件有限公司 | Method and device for accelerating preheating of second-level cache |
| CN104239231B (en) * | 2014-09-01 | 2018-01-30 | 上海爱数信息技术股份有限公司 | A kind of method and device for accelerating L2 cache preheating |
| CN105426113A (en) * | 2014-09-05 | 2016-03-23 | 群联电子股份有限公司 | Memory management method, memory storage apparatus and memory control circuit unit |
| CN105426113B (en) * | 2014-09-05 | 2018-11-02 | 群联电子股份有限公司 | Storage management method, memory storage apparatus and memorizer control circuit unit |
| US9632927B2 (en) | 2014-09-25 | 2017-04-25 | International Business Machines Corporation | Reducing write amplification in solid-state drives by separating allocation of relocate writes from user writes |
| US10162533B2 (en) | 2014-09-25 | 2018-12-25 | International Business Machines Corporation | Reducing write amplification in solid-state drives by separating allocation of relocate writes from user writes |
| US10579270B2 (en) | 2014-09-25 | 2020-03-03 | International Business Machines Corporation | Reducing write amplification in solid-state drives by separating allocation of relocate writes from user writes |
| US10078582B2 (en) | 2014-12-10 | 2018-09-18 | International Business Machines Corporation | Non-volatile memory system having an increased effective number of supported heat levels |
| US10831651B2 (en) | 2014-12-10 | 2020-11-10 | International Business Machines Corporation | Non-volatile memory system having an increased effective number of supported heat levels |
| CN104391661A (en) * | 2014-12-12 | 2015-03-04 | 西安三星电子研究有限公司 | Method and equipment for writing data into solid hard disk |
| US9779021B2 (en) | 2014-12-19 | 2017-10-03 | International Business Machines Corporation | Non-volatile memory controller cache architecture with support for separation of data streams |
| US11036637B2 (en) | 2014-12-19 | 2021-06-15 | International Business Machines Corporation | Non-volatile memory controller cache architecture with support for separation of data streams |
| US10387317B2 (en) | 2014-12-19 | 2019-08-20 | International Business Machines Corporation | Non-volatile memory controller cache architecture with support for separation of data streams |
| CN105988720A (en) * | 2015-02-09 | 2016-10-05 | 中国移动通信集团浙江有限公司 | Data storage device and method |
| CN104765571A (en) * | 2015-03-17 | 2015-07-08 | 深信服网络科技（深圳）有限公司 | Virtual data writing and reading method and system |
| US9760310B2 (en) | 2015-08-06 | 2017-09-12 | International Business Machines Corporation | High performance data storage management using dynamic compression |
| CN106557469A (en) * | 2015-09-24 | 2017-04-05 | 阿里巴巴集团控股有限公司 | The method and device of data in a kind of processing data warehouse |
| CN106557469B (en) * | 2015-09-24 | 2020-11-20 | 创新先进技术有限公司 | Method and device for processing data in data warehouse |
| US9886208B2 (en) | 2015-09-25 | 2018-02-06 | International Business Machines Corporation | Adaptive assignment of open logical erase blocks to data streams |
| US10613784B2 (en) | 2015-09-25 | 2020-04-07 | International Business Machines Corporation | Adaptive assignment of open logical erase blocks to data streams |
| CN105631035A (en) * | 2016-01-04 | 2016-06-01 | 北京百度网讯科技有限公司 | Data storage method and device |
| CN105631035B (en) * | 2016-01-04 | 2018-04-17 | 北京百度网讯科技有限公司 | Date storage method and device |
| US10649905B2 (en) | 2016-01-04 | 2020-05-12 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Method and apparatus for storing data |
| CN107797758B (en) * | 2016-09-07 | 2019-07-30 | 腾讯科技（深圳）有限公司 | Date storage method, data access method and device |
| CN107797758A (en) * | 2016-09-07 | 2018-03-13 | 腾讯科技（深圳）有限公司 | Date storage method, data access method and device |
| CN106406759B (en) * | 2016-09-13 | 2019-12-31 | 苏州浪潮智能科技有限公司 | Data storage method and device |
| CN106406759A (en) * | 2016-09-13 | 2017-02-15 | 郑州云海信息技术有限公司 | Data storage method and device |
| US10924356B2 (en) | 2016-10-14 | 2021-02-16 | Tencent Technology (Shenzhen) Company Limited | Network service scheduling method and apparatus, storage medium, and program product |
| CN106909398A (en) * | 2017-03-10 | 2017-06-30 | 深圳创维－Rgb电子有限公司 | The compression method and device of a kind of executable file |
| WO2018161791A1 (en) * | 2017-03-10 | 2018-09-13 | 深圳创维－Rgb电子有限公司 | Method and device for compressing executable file |
| US11461276B2 (en) | 2017-06-02 | 2022-10-04 | EMC IP Holding Company LLC | Method and device for deduplication |
| CN108984103A (en) * | 2017-06-02 | 2018-12-11 | 伊姆西Ip控股有限责任公司 | Method and apparatus for duplicate removal |
| CN109032965A (en) * | 2017-06-12 | 2018-12-18 | 华为技术有限公司 | A kind of method for reading data, host and storage equipment |
| CN109032965B (en) * | 2017-06-12 | 2021-01-12 | 华为技术有限公司 | Data reading method, host and storage device |
| CN110018879B (en) * | 2018-01-09 | 2023-06-09 | 阿里巴巴集团控股有限公司 | Delay loading method and device applied to distributed system |
| CN110018879A (en) * | 2018-01-09 | 2019-07-16 | 阿里巴巴集团控股有限公司 | Delay loading method and device applied to distributed system |
| WO2020073328A1 (en) * | 2018-10-12 | 2020-04-16 | 华为技术有限公司 | Data processing method and device |
| CN112673359B (en) * | 2018-10-12 | 2024-08-27 | 华为技术有限公司 | Data processing method and device |
| CN112673359A (en) * | 2018-10-12 | 2021-04-16 | 华为技术有限公司 | Data processing method and device |
| CN109582245A (en) * | 2018-12-06 | 2019-04-05 | 联想(北京)有限公司 | Data processing method, device and equipment |
| CN109802684A (en) * | 2018-12-26 | 2019-05-24 | 华为技术有限公司 | The method and apparatus for carrying out data compression |
| CN112306974B (en) * | 2019-07-30 | 2024-10-22 | 深信服科技股份有限公司 | Data processing method, device, equipment and storage medium |
| CN112306974A (en) * | 2019-07-30 | 2021-02-02 | 深信服科技股份有限公司 | Data processing method, device, equipment and storage medium |
| WO2021036329A1 (en) * | 2019-08-28 | 2021-03-04 | 华为技术有限公司 | Data storage method and apparatus |
| CN110879688A (en) * | 2019-10-25 | 2020-03-13 | 苏州浪潮智能科技有限公司 | Storage compression method and system |
| CN110908608A (en) * | 2019-11-22 | 2020-03-24 | 苏州浪潮智能科技有限公司 | Storage space saving method and system |
| CN111562883A (en) * | 2020-04-26 | 2020-08-21 | 北京泽石科技有限公司 | Cache management system, method and device for solid state disk |
| CN111813340A (en) * | 2020-07-10 | 2020-10-23 | 杭州海康威视数字技术股份有限公司 | Instruction response method, system and device based on solid state disk and electronic equipment |
| CN112487326B (en) * | 2020-11-27 | 2024-03-19 | 杭州安恒信息技术股份有限公司 | Data caching method, system, storage medium and equipment |
| CN112487326A (en) * | 2020-11-27 | 2021-03-12 | 杭州安恒信息技术股份有限公司 | Data caching method, system, storage medium and equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102609360B (en) | 2015-03-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102609360B (en) | Data processing method, data processing device and data processing system | |
| KR102556431B1 (en) | Solid state drive with heterogeneous nonvolatile memory types | |
| US10031675B1 (en) | Method and system for tiering data | |
| US9229653B2 (en) | Write spike performance enhancement in hybrid storage systems | |
| CN106716334B (en) | Efficient data movement within a file system volume | |
| US9792227B2 (en) | Heterogeneous unified memory | |
| US20120117328A1 (en) | Managing a Storage Cache Utilizing Externally Assigned Cache Priority Tags | |
| CN102023809A (en) | Storage system, method for reading data from storage system and method for writing data to storage system | |
| US11042324B2 (en) | Managing a raid group that uses storage devices of different types that provide different data storage characteristics | |
| Zhou et al. | I/O characteristics of smartphone applications and their implications for eMMC design | |
| CN105138292A (en) | Disk data reading method | |
| CN104267912A (en) | NAS (Network Attached Storage) accelerating method and system | |
| US8856439B2 (en) | Method and device for utilizing application-level prior knowledge for selectively storing data in higher performance media | |
| CN101645043A (en) | Methods for reading and writing data and memory device | |
| WO2018199794A1 (en) | Re-placing data within a mapped-raid environment | |
| CN104616680A (en) | Repeating data deleting system based on optical disc storage as well as data operating method and device | |
| CN103942161A (en) | Redundancy elimination system and method for read-only cache and redundancy elimination method for cache | |
| CN102981979A (en) | Method for improving speed of storage system data access | |
| CN109375868B (en) | Data storage method, scheduling device, system, equipment and storage medium | |
| US20220365720A1 (en) | Object management in tiered memory systems | |
| Lv et al. | Zonedstore: A concurrent zns-aware cache system for cloud data storage | |
| WO2021110570A1 (en) | Providing a dynamic random-access memory cache as second type memory | |
| CN112703481B (en) | Hybrid memory system interface | |
| US20220365705A1 (en) | Object management in tiered memory systems | |
| CN115344201A (en) | Data storage method, data query method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20150325 Termination date: 20180112 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |