CN106648955B - Compression method and related device - Google Patents
Compression method and related device Download PDFInfo
- Publication number
- CN106648955B CN106648955B CN201611006972.7A CN201611006972A CN106648955B CN 106648955 B CN106648955 B CN 106648955B CN 201611006972 A CN201611006972 A CN 201611006972A CN 106648955 B CN106648955 B CN 106648955B
- Authority
- CN
- China
- Prior art keywords
- result
- data
- original data
- compression
- decompression
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
- G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
- G06F11/1004—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's to protect a block of data words, e.g. CRC or checksum
Abstract
The invention relates to the technical field of computers, in particular to a compression technology. compression methods comprise the steps of obtaining original data, compressing the original data to obtain a compression result, decompressing the compression result to obtain a decompression result, checking whether the decompression result and the original data are or not, outputting the compression result if or not, and outputting the original data if not .
Description
Technical Field
The invention relates to the technical field of computers, in particular to a compression technology.
Background
Today, in the face of a data flood explosion, in the face of a dramatic expansion of data, a storage device (e.g., a disk array) needs to perform data compression to effectively achieve data capacity reduction: suppose the storage space is 1TB, but the data that the user needs to store is 1.3 TB. The storage device can compress the original data of the 1.3TB and store the compression result (data), so that the storage space can be saved, and the compression is transparent to the user; when data needs to be read, the compressed data needs to be decompressed first.
However, in the actual application process, the inventor finds that the data obtained after decompression may not be from the original data, and is externally represented as data damage, which further may cause an abnormality in the upper layer application, such as a downtime in the application of a virtual machine, a database, and the like.
Disclosure of Invention
The present invention is directed to a compression method and related apparatus to solve the above problems.
In order to achieve the purpose, the invention provides the following scheme:
, the embodiment of the present application provides compression methods, which are performed by a processor or a hardware accelerator card in a controller in a storage device in an interactive manner with other components, including obtaining raw data, compressing the raw data to obtain a compression result, decompressing the compression result to obtain a decompression result, checking whether the decompression result and the raw data are -fold, outputting the compression result if -fold, and outputting the raw data if -fold.
In possible designs, before compressing the original data or checking whether the decompression result and the original data are , the check data of the original data can be calculated, besides, before checking whether the decompression result and the original data are , the check data of the decompressed data can be calculated, and the step of checking whether the decompression result and the original data are can be specifically refined into that whether the check data of the original data and the check data of the decompression result are , if , the compression result is output, and if not , the original data is output, so that the problem that the data and the original data obtained after decompression are not can be better avoided.
In possible designs, the check data of the original data may include check codes (e.g., CRC codes) of the original data and at least kinds of lengths of the original data, and the check data of the decompression result may include check codes (e.g., CRC check codes) of the decompression result and at least kinds of lengths of the decompression result, for convenience of distinction, the check codes of the original data may be referred to as check codes, the length of the original data may be referred to as th length, the check codes of the decompression result may be referred to as second check codes, and the length of the decompression result may be referred to as second length, if the check data of the original data includes th check codes and th length, the check data of the decompression result may include only the second check codes, if the check data of the original data includes only check codes, the check data of the decompression result may include only the second check codes, and if the check data of the original data includes only th check codes, the hash value of the original data may be obtained by using the hash algorithm, so that the hash value of the hash algorithm may not provide hash value of the hash data.
In possible designs, the decompression result and the original data can also be compared directly byte by byte, which is relatively simple.
, the embodiment of the invention provides storage devices which have the function of realizing the storage device behavior in the practice of the method described above.
, embodiments of the present invention provide computer storage media storing computer software instructions for use by the storage devices described above, including programs designed to perform the aspects described above.
Compared with the prior art, according to the scheme provided by the invention, after the original data is compressed and before the original data is stored, the compression result is decompressed to obtain the decompression result, whether the decompression result is consistent with the original data by is checked, if the decompression result is consistent with the original data by , the compression result is output, namely, the compression result is stored, if the decompression result is not consistent with , the data obtained after the decompression and the original data are not consistent with , the compression result is abandoned, the original data is output, and the original data is stored.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive labor.
Fig. 1 is a schematic view of an application scenario provided in an embodiment of the present invention;
FIGS. 2a and 2b are exemplary block diagrams of a controller provided by an embodiment of the present invention;
fig. 3a and fig. 4 are exemplary flowcharts of a compression method according to an embodiment of the present invention;
FIG. 3b is an exemplary block diagram of a prior art compression method;
FIG. 5 is a diagram illustrating an exemplary structure of a compression device (especially a hardware accelerator card) according to an embodiment of the present invention;
fig. 6 is a schematic diagram illustrating a data flow inside a hardware accelerator card according to an embodiment of the present invention.
Detailed Description
The embodiment of the invention provides a compression method and related devices (a compression device, a storage device/system and a hardware acceleration data card).
The storage device/system may be a disk array or the like.
Fig. 1 depicts a block diagram of a storage device/system provided by an embodiment of the present invention, including at least controllers 101 and a hard disk enclosure 102.
The controller 101 may be an variety of computing devices, such as a server, desktop computer, etc. an operating system and other applications are installed on the controller 101. the controller 101 may receive input output (I/O) requests from an application host, store data carried in the I/O requests, and write the data to the hard disk enclosure 102. the hard disk enclosure 102 may then include or more hard disks.
In the application scenarios, controller 101 may be connected to an application host (not shown) via a SAN network.
The hard disk frame 102 may include or more hard disks, for example, 20 hard disks may be installed.
Fig. 2a is an exemplary diagram of configurations of the controller 101, and as shown in fig. 2a, the controller 101 may include a processor 201, a memory 202, a communication interface 203 (which may further include a front end interface card and a back end interface card ), and an input device 204 and an output device 205 connected via a bus.
The front-end interface card may be used to communicate with the application host, and the processor 201 of the controller 101 may receive various operation instructions of the application host through the front-end interface card.
The processor 201 of the controller 101 may be CPUs, or an asic (application Specific Integrated circuit), or ics configured to implement embodiments of the present invention.
More specifically, Memory 202 may comprise Memory, cache (e.g., cache206 in FIG. 2 b), wherein the Memory may comprise high-speed RAM Memory, and may further comprise non-volatile Memory (e.g., at least Disk drives).
And the back-end interface card can be used for communicating with the hard disk in the hard disk frame 102, and the processor 201 of the controller 101 can send data to the hard disk in the hard disk frame 102 through the back-end interface card for storage.
It should be noted that if the hard disk enclosure 102 and the controller 101 are installed together in enclosures, a backend interface card is integrated inside.
If the hard disk frame 102 and the controller 101 are separately housed in separate frames, the back end interface card may not be integrated with the other components at .
The compression apparatus may be applied to the storage device/system in the form of software or hardware.
In examples, when the compression apparatus is applied to a storage device/system in the form of software, the memory 202 may store a program for executing the technical solution of the present invention, and of course, the memory 202 may also store an operating system and other application programs.
In another examples, please refer to FIG. 2b, the compression apparatus can be applied to the above-mentioned storage device/system in the form of the hardware accelerator card 207. more specifically, the hardware accelerator card 207 can be installed on a slot of the controller 101 (control box).
Of course, there is also a cache in the hardware accelerator card 207.
In use, the processor 201 of the controller 101 may store raw data from the application host into the cache206, the cache206 may determine when to provide the raw data to the hardware acceleration card 207, and the hardware acceleration card 207 may perform the compression method provided by the present invention.
Based on the common aspects of the present invention described above, a detailed description will be given below of how the storage management of the present invention is performed .
Fig. 3a shows exemplary flows of the compression method provided by the embodiment of the present invention, so as to solve the problem that the data obtained after decompression may not be from the original data.
The method shown in fig. 3a is applied in the application scenario shown in fig. 1, and is implemented by the processor 201 of the controller 101 shown in fig. 2a interacting with other components, or by the hardware accelerator card 207 of the controller 101 shown in fig. 2b interacting with other components.
The process comprises the following steps:
part 300: raw data is acquired.
In examples, in the scenario of executing the compression method by the processor 201 of the controller 101, the part 300 may be executed by the processor 201 of the controller 101. specifically, the raw data sent by the application host may be received by the processor 201 of the controller 101 through the front-end interface card and stored into the cache 206. when the compression processing is needed, the processor 201 reads out the raw data in the cache206 again.
In another examples, in a scenario where the compression method is performed by the hardware accelerator card 207, after the processor 201 stores raw data from the application host into the cache206, the cache206 provides the raw data to the hardware accelerator card 207.
More specifically, the cache206 is a unit of memory page that provides the original data to the accelerator card, and the memory page storage capacity is 4kB (byte) per pages.
That is, in any case, the original data can be stored in the cache206 (memory).
It should be noted that when there is a large amount of raw data to be stored, a batch process may be performed, each time a portion of the raw data is acquired.
Part 301: and compressing the original data to obtain a compression result.
Different compression schemes may be used for compression, such as Huffman coding.
In the context of the compression method being executed by the processor 201 of the controller 101, in examples, the portion 301 may be executed by the processor 201 of the controller 101.
In another examples, in the scenario where the hardware accelerator card 207 executes the compression method, the portion 301 may be executed by the hardware accelerator card 207. the hardware accelerator card 207 may include a plurality of compression engines, for example, 10 compression engines, so that the compression process may be performed in parallel at the same time.
In fact, inside the accelerator card 207, it processes data on a memory page, and as mentioned above, the storage capacity of page memory is 4KB, and it can be times to process 16 page memory pages, assuming that times the accelerator card can process data of 64KB size at maximum.
Of course, the data on the 16 pages of memory may be compressed by multiple compression engines in parallel.
Part 302: and decompressing the compression result to obtain a decompression result.
The compression result may be decompressed using a decompression algorithm as opposed to a compression algorithm.
In the context of the compression method being executed by the processor 201 of the controller 101, in examples, portion 302 may be executed by the processor 201 of the controller 101.
In another examples, in the scenario where the hardware accelerator card 207 performs the compression method, the portion 302 may be performed by the hardware accelerator card 207. the hardware accelerator card 207 may include multiple decompression engines, for example, 3 decompression engines, so that the decompression processes may be performed simultaneously and in parallel.
303, checking whether the decompression result and the original data are , if , entering 304, otherwise, entering 305.
There are many verification ways, in examples, the decompression result and the original data can be directly compared byte by byte, if there is not result, it is determined that the two result is not result, otherwise, it is determined that the two result is result.
In another examples, a CRC (Cyclic Redundancy Check) Check, a hash Check, or the like may be employed.
In the context of the compression method being executed by the processor 201 of the controller 101, in examples, the portion 303 may be executed by the processor 201 of the controller 101.
In another examples, in the scenario where the compression method is performed by the hardware accelerator card 207, part 303 may be performed by the hardware accelerator card 207.
Part 304: and outputting a compression result.
If the check result is equal, it indicates that the compressed result is equal to the original data after decompression, and the decompressed data and the original data are not equal, so the compressed result is output.
Part 305: and outputting the original data.
If not , it indicates that the data obtained after decompression and the original data are not , the compression result is discarded, and the original data are output.
In examples, portions 304 and 305 may be performed by processor 201 of controller 101 in the scenario where the compression method is performed by processor 201 of controller 101 in this scenario, processor 201 of controller 101 may send raw data or compression results to hard disk box 102 for storage through a back-end interface card.
In another examples, in a scenario where the hardware accelerator card 207 executes the compression method, parts 304 and 305 may be executed by the hardware accelerator card 207. the hardware accelerator card 207 outputs the raw data or the compression result to the cache 206. subsequently, the cache206 sends the raw data or the compression result to the hard disk frame 102 for storage.
It should be noted that, in this embodiment, since the original data may be output finally, the compression result may also be output. Both the raw data and the compressed result are buffered in the hardware accelerator card 207 before being output.
Referring to fig. 3b, in the conventional compression method, after the original data is obtained, the original data is compressed and the compressed result is output, compared with the prior art, in the scheme provided by the present invention, after the original data is compressed and before the original data is stored, the compressed result is decompressed to obtain the decompressed result, and whether the decompressed result and the original data are or not is checked, if is satisfied, the compressed result is output, that is, the compressed result is stored, and if is not satisfied, it is stated that the data obtained after the decompression and the original data are not satisfied, the compressed result is discarded, the original data is output, and the original data is stored.
In the following, the technical solution of the present invention will be described in more detail by taking the hardware accelerator card 207 as an example to execute the compression method.
Fig. 4 shows interactive exemplary flows of the compression method provided by the embodiment of the present invention, the method shown in fig. 4 is applied in the application scenario shown in fig. 1, and is implemented by the hardware accelerator card 207 of the controller 101 shown in fig. 2b interacting with other components.
The interaction process comprises the following steps:
part 400: the hardware accelerator card 207 receives raw data from the cache 206.
Portion 400 is similar to portion 300, and will not be described in detail.
Part 401: the hardware accelerator card 207 calculates the verification data of the acquired raw data.
In examples, the check data may include a check code (e.g., a CRC check code) and a length of the original data.
The CRC (Cyclic Redundancy Check) algorithm is a hash functions of that generate short fixed-bit Check codes from specified data, and is used to detect or Check errors that may occur after data transmission or storage.
In other embodiments of the present invention, a hash algorithm may be further used to obtain a hash value of the original data as a check code of the original data.
The hash algorithm is a one-way abstract algorithm, fixed-length data is generated by any long data, and when a small amount of change occurs in input data, the output data can obviously change.
Of course, other algorithms may be used to calculate the check code for the original data.
For the sake of distinction, the (CRC) check code of the original data may be referred to as (CRC) check code, and the length of the original data may be referred to as length.
In another examples, the check data of the original data may also include only the (CRC) check code or the th length.
Part 402: the hardware accelerator card 207 compresses the original data to obtain a compression result.
And (403) part: the hardware accelerator card 207 decompresses the compression result to obtain a decompression result.
The portions 402-403 are similar to the portions 301-302, and are not described herein again.
Part 404: the hardware accelerator card 207 calculates the verification data of the decompression result.
In examples, the check data of the decompression result may include a check code (e.g., a CRC check code) and a length of the decompression result, which is times the check data of the original data.
In other embodiments of the present invention, a hash algorithm may be further used to obtain a hash value of the decompression result as the check code.
For the sake of distinction, the (CRC) check code of the decompression result may be referred to as a second (CRC) check code, and the length of the decompression result may be referred to as a second length.
Of course, in another examples, the check data of the decompression result may also include only the second (CRC) check code or the second length.
It should be noted that, if the check data of the original data includes the (CRC) check code and the length, the check data of the decompression result correspondingly includes the second (CRC) check code and the second length, if the check data of the original data includes only the (CRC) check code, the check data of the decompression result correspondingly includes only the second (CRC) check code, and if the check data of the original data includes only the length, the check data of the decompression result correspondingly includes only the second length.
Part 405, the hardware accelerator card 207 compares the check data of the original data with the check data of the decompression result to see if is true, if is true, then part 406 is entered, otherwise, part 407 is entered.
For the case that the check data of the original data includes (CRC) check code and length, and the check data of the decompression result correspondingly includes the second (CRC) check code and the second length, it is necessary to compare whether the (CRC) check code is equal to the second (CRC) check code, and whether the length is equal to the second length, which are both equal, to obtain the result of the check data .
For the case that the check data of the original data only includes the (CRC) check code or the length, and the check data of the decompression result correspondingly only includes the second (CRC) check code or the second length, the result of the check data can be obtained as long as the (CRC) check code is equal to the second (CRC) check code, or the length is equal to the second length.
406 part: the hardware accelerator card 207 outputs the compression result to the cache 206.
407 part: the hardware accelerator card 207 outputs the raw data to the cache 206.
The portions 406-407 are similar to the portions 304-305, and are not described in detail here.
Fig. 5 shows possible structural diagrams of the compression apparatus (especially, the hardware accelerator card) according to the above embodiments, which include a read/write module 501, a compression module 502, a decompression module 503, and a check module 504.
The compression module 502 may be configured to compress the original data obtained by the read-write module 501, so as to obtain a compression result; the compression module 502 may include a plurality of compression engines.
The decompression module 503 may be configured to decompress the compression result obtained by the compression module 502 to obtain a decompression result; the decompression module 503 may include a plurality of decompression engines.
The check module 504 can be used to check whether the decompression result obtained by the decompression module 503 matches with the original data obtained by the read/write module 501, if so, indicates that the read/write module 501 outputs the compression result, and if not, indicates that the read/write module 501 outputs the original data.
More specifically, referring to fig. 6, the read/write module 501 may be a DMA (Direct Memory Access) engine 5011.
The DMA engine 5011 transfers data using DMA technology.
The DMA engine 5011 further may include a DMA controller and internal memory (which may be RAM).
DMA technology allows hardware devices of different speeds to communicate without relying on a significant interrupt load of the CPU (referred to herein as processor 201 of controller 101).
DMA technology can copy data from address spaces to additional address spaces a typical example is to move blocks of external memory to faster memory areas within the chip.
Before DMA transfer, the CPU gives the DMA controller the bus control right, and after finishing DMA transfer, the DMA controller should give the CPU the bus control right again.
After the DMA controller obtains the bus control right, the CPU immediately hangs up or only executes internal operation, and the DMA controller outputs a read-write command to directly control the RAM and the I/O interface to carry out DMA transmission.
Under the control of the DMA controller, data transfer is directly performed between the RAM and the external device, and the participation of a CPU is not needed in the transfer process. Entry data (start position and data length of data) of data to be transferred is initially provided.
This is because it was mentioned above that the hardware accelerator card 207 internally handles pages of memory pages, each memory page being 4KB in size, but not all of the contents of the 4KB of memory are original data.
Of course, when the DMA engine outputs data (which may be referred to as write-back), it also needs to provide the entry data of the write-back data.
Entry Data may be input or output via a CMMDQ (Controller Memory Management Data Queue) module 505.
The CMMDQ module 505 may provide the DMA engine with the entry data for the raw data. When outputting the original data or the compression result, the CMMDQ module 5 may provide the table entry data of the output data to the outside.
Fig. 6 shows an internal data flow diagram of the hardware accelerator card 207. The method comprises the following steps:
part 601: the DMA engine 5011 receives the memory page from the cache 206.
More specifically, the memory pages are copied from the cache206 to the RAM inside the DMA engine 5011.
And the original data is stored in the memory page.
Part 602: the DMA engine 5011 obtains the entry data of the raw data from the CMMDQ module 505.
Part 603: the DMA engine 5011 provides the raw data to the compression module 502 based on the entry data for the raw data provided by the CMMDQ module 505.
The original data is stored in the memory page, the table entry data of the original data is used to describe the start position and the length of the original data, and after knowing the start position and the length, the original data can be read from the memory page and provided to the compression module 502.
Part 604: the compression module 502 compresses the original data to obtain a compression result.
The compression results are stored in the memory of the hardware accelerator card.
It was mentioned above that there are multiple compression engines that compress separately, and each compression engine can produce or more compressed data blocks, all of which constitute the compression result.
Part 605: the decompression module 503 decompresses the compression result to obtain a decompression result.
It should be noted that, after the compression module 502 obtains the compression result, its work is completed. How does decompression module 503 start decompression? Can be realized by an asynchronous queue: after compression is complete, the compression module 502 may place the number of compressed data blocks in the asynchronous queue.
As long as the asynchronous queue is not empty, the decompression module 503 will decompress the compressed data block according to the number in the asynchronous queue.
Part 606: the verification module 504 calculates verification data of the original data and compares the verification data with verification data of the decompression result to obtain a verification result.
For further details, see sections 401, 404 and 405, supra.
The check result may be or . in practice, may be represented by code 0, and may be represented by code 1.
Part 607: the check module 504 returns the check result to the CMMDQ module 505.
Part 608: the check module 504 returns the entry data of the original data or the compression result to the DMA engine 5011.
It should be noted that if the check result is yes, the check module 504 returns the table entry data of the compression result to the DMA engine 5011, and if not yes, the check module 504 returns the table entry data of the original data to the DMA engine 5011.
In other embodiments of the present invention, since the check module 504 returns the check result to the CMMDQ module 505, the CMMDQ module 505 may also return the original data or the table entry data of the compression result to the DMA engine 5011.
609: the DMA engine 5011 outputs original data or a compression result to the cache206 according to the table entry data; meanwhile, the CMMDQ module 505 outputs (to the cache 206) the original data or the table entry data of the compression result to the outside.
Specifically, if the result is verified, and the DMA engine 5011 receives the table entry data of the compression result, the memory page including the compression result is output, and the CMMDQ module 505 outputs the table entry data of the compression result.
On the contrary, if the verification result is not , the DMA engine 5011 receives the table entry data of the original data, and outputs the memory page containing the original data, and the CMMDQ module 505 outputs the table entry data of the original data.
In addition, the read/write module 501 or the DMA module 5011 may also execute the 300, 304, and 305 portions of the embodiment shown in FIG. 3a, and the 400, 406, and 407 portions of the embodiment shown in FIG. 4;
the compression module 502 may also perform the 301 portion of the embodiment shown in FIG. 3a, the 402 portion of the embodiment shown in FIG. 4;
the decompression module 503 may also perform part 302 of the embodiment shown in fig. 3a, part 403 of the embodiment shown in fig. 4;
the verification module 504 may also perform part 303 of the embodiment shown in fig. 3a, and parts 401, 404 and 405 of the embodiment shown in fig. 4.
The above description mainly introduces the scheme provided by the embodiment of the present invention from the perspective of interaction between various devices. It is understood that each device, such as the controller 101, the hardware accelerator card 207, etc., contains corresponding hardware structures and/or software modules for performing each function in order to realize the functions. Those of skill in the art will readily appreciate that the present invention can be implemented in hardware or a combination of hardware and computer software, with the exemplary elements and algorithm steps described in connection with the embodiments disclosed herein. Whether a function is performed as hardware or computer software drives hardware depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The above-mentioned embodiments, object, technical solutions and advantages of the present invention have been described in further , it should be understood that the above-mentioned embodiments are only illustrative of the present invention and are not intended to limit the scope of the present invention, and any modification, equivalent replacement, improvement, etc. made on the basis of the technical solutions of the present invention should be included in the scope of the present invention.
Claims (12)
1, A compression method, comprising:
acquiring original data;
compressing the original data to obtain a compression result;
decompressing the compression result to obtain a decompression result;
checking whether the decompression result and the original data result from ;
if is the same, outputting the compression result to be stored in a hard disk frame;
if not , the raw data is output for storage in the hard disk box.
2. The method of claim 1, further comprising, prior to compressing the original data or prior to verifying the decompression result from the original data, calculating verification data for the original data.
3. The method of claim 2, further comprising, prior to verifying the decompression result and the original data, computing verification data for the decompression result.
4. The method of claim 3, wherein said verifying said decompression result and said original data if resulted comprises:
comparing the verification data of the original data with the verification data of the decompression result to determine whether is the result.
5. The method as claimed in claim 4, wherein the check data of the original data includes Cyclic Redundancy Check (CRC) codes of the original data and at least kinds of lengths of the original data.
6. The method of claim 4, wherein the check data of the decompression result comprises at least CRC codes of the decompression result and the length of the decompression result.
7. The method of claim 1, wherein said verifying said decompression result and said original data if resulted comprises:
comparing the decompressed result with the original data on a byte-by-byte basis to determine whether is the result.
8. The method of any of claims 1-7 and , wherein the method is performed by a hardware accelerator card.
A compression apparatus of the type , comprising:
the compression module is used for compressing the original data acquired by the read-write module to obtain a compression result;
the decompression module is used for decompressing the compression result to obtain a decompression result;
and the checking module is used for checking whether the decompression result and the original data are , indicating the read-write module to output the compression result for storage in a hard disk frame if is , and indicating the read-write module to output the original data for storage in the hard disk frame if not is.
10. The apparatus of claim 9, wherein the verification module is further to:
calculating verification data of the original data before compressing the original data or before verifying whether a decompression result and the original data result are ;
before checking whether the decompression result and the original data result, check data of the decompression result is calculated.
11. The apparatus of claim 10, wherein in the checking whether the decompressed result is -wise consistent with the original data, the check module is to compare check data of the original data with check data of the decompressed result is -wise consistent.
Storage device of the kind of , characterized in that it comprises a compression apparatus according to claims 9-11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611006972.7A CN106648955B (en) | 2016-11-15 | 2016-11-15 | Compression method and related device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611006972.7A CN106648955B (en) | 2016-11-15 | 2016-11-15 | Compression method and related device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106648955A CN106648955A (en) | 2017-05-10 |
| CN106648955B true CN106648955B (en) | 2020-01-31 |
Family
ID=58805993
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611006972.7A Active CN106648955B (en) | 2016-11-15 | 2016-11-15 | Compression method and related device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106648955B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107688505A (en) * | 2017-08-15 | 2018-02-13 | 深圳前海信息技术有限公司 | Data verification method and device based on hardware circuit |
| CN108958966A (en) * | 2018-09-27 | 2018-12-07 | 合肥达博科技有限公司 | A kind of data guard method and device of lossless data compression |
| CN109587557B (en) | 2019-01-11 | 2022-03-08 | 京东方科技集团股份有限公司 | Data transmission method and device and display device |
| CN109922049B (en) * | 2019-02-02 | 2021-09-28 | 立旃（上海）科技有限公司 | Verification device and method based on block chain |
| CN113360314B (en) * | 2020-03-02 | 2023-07-18 | 芯原微电子（成都）有限公司 | Data processing method, system, equipment and computer readable storage medium |
| CN114077569B (en) * | 2020-08-18 | 2023-07-18 | 富泰华工业（深圳）有限公司 | Method and device for compressing data, and method and device for decompressing data |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1921019A (en) * | 2005-08-24 | 2007-02-28 | 英业达股份有限公司 | Method for testing storage by data compressing treatment |
| CN101478370A (en) * | 2009-01-20 | 2009-07-08 | 中兴通讯股份有限公司 | File compression method and apparatus based on file system |
| CN102129873A (en) * | 2011-03-29 | 2011-07-20 | 西安交通大学 | Data compression device and method for improving last-stage high-speed caching reliability of computer |
| CN103384884A (en) * | 2012-12-11 | 2013-11-06 | 华为技术有限公司 | File compression method and device, file decompression method and device, and server |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080240419A1 (en) * | 2007-03-30 | 2008-10-02 | Melanie Jean Sandberg | Apparatus, system, and method for testing data compression and data encryption circuitry |
-
2016
- 2016-11-15 CN CN201611006972.7A patent/CN106648955B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1921019A (en) * | 2005-08-24 | 2007-02-28 | 英业达股份有限公司 | Method for testing storage by data compressing treatment |
| CN101478370A (en) * | 2009-01-20 | 2009-07-08 | 中兴通讯股份有限公司 | File compression method and apparatus based on file system |
| CN102129873A (en) * | 2011-03-29 | 2011-07-20 | 西安交通大学 | Data compression device and method for improving last-stage high-speed caching reliability of computer |
| CN103384884A (en) * | 2012-12-11 | 2013-11-06 | 华为技术有限公司 | File compression method and device, file decompression method and device, and server |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106648955A (en) | 2017-05-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106648955B (en) | Compression method and related device | |
| US10963393B1 (en) | Storage system and a method for application aware processing | |
| US10114578B2 (en) | Solid state disk and data moving method | |
| US20190073132A1 (en) | Method and system for active persistent storage via a memory bus | |
| US9973210B1 (en) | Reduction of execution stalls of LZ4 decompression via parallelization | |
| US20090284400A1 (en) | Method and System for Reducing Required Storage During Decompression of a Compressed File | |
| WO2018072688A1 (en) | Quick loading method for kernel image file, and apparatus | |
| US10585604B2 (en) | Tool for selectively deploying inline compression | |
| CN111625181A (en) | Data processing method, redundant array controller of independent hard disk and data storage system | |
| US8856479B2 (en) | Implementing storage adapter performance optimization with hardware operations completion coalescence | |
| US10140033B2 (en) | Apparatus, system, and method for searching compressed data | |
| US11669466B2 (en) | Memory module data object processing systems and methods | |
| CN105528183A (en) | Data storage method and storage equipment | |
| US10795825B2 (en) | Compressing data for storage in cache memories in a hierarchy of cache memories | |
| Hu et al. | {QZFS}:{QAT} Accelerated Compression in File System for Application Agnostic and Cost Efficient Data Storage | |
| US11507292B2 (en) | System and method to utilize a composite block of data during compression of data blocks of fixed size | |
| US20170293452A1 (en) | Storage apparatus | |
| US10866941B2 (en) | Computer and database management method | |
| CN105354107A (en) | Data transmission method and system for NOR Flash | |
| US11573899B1 (en) | Transparent interleaving of compressed cache lines | |
| CN104571940A (en) | Data storage device and computing system with the same | |
| CN109445686B (en) | Storage disk and data access method | |
| US20230409226A1 (en) | Method and system for acceleration or offloading utilizing a multiple input data stream | |
| WO2024066753A1 (en) | Data compression method and related apparatus | |
| US10840943B1 (en) | System and method of data compression between backup server and storage |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20200416 Address after: 518129 Bantian HUAWEI headquarters office building, Longgang District, Guangdong, Shenzhen Patentee after: HUAWEI TECHNOLOGIES Co.,Ltd. Address before: 301, A building, room 3, building 301, foreshore Road, No. 310052, Binjiang District, Zhejiang, Hangzhou Patentee before: Huawei Technologies Co.,Ltd. |
|
| TR01 | Transfer of patent right |