WO2014089753A1 - Procédé de compression de fichiers, procédé de décompression de fichiers, dispositif et serveur - Google Patents

Procédé de compression de fichiers, procédé de décompression de fichiers, dispositif et serveur Download PDF

Info

Publication number
WO2014089753A1
WO2014089753A1 PCT/CN2012/086341 CN2012086341W WO2014089753A1 WO 2014089753 A1 WO2014089753 A1 WO 2014089753A1 CN 2012086341 W CN2012086341 W CN 2012086341W WO 2014089753 A1 WO2014089753 A1 WO 2014089753A1
Authority
WO
WIPO (PCT)
Prior art keywords
compressed
data blocks
data block
file
data
Prior art date
Application number
PCT/CN2012/086341
Other languages
English (en)
Chinese (zh)
Inventor
沈慧
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to CN201280003410.0A priority Critical patent/CN103384884B/zh
Priority to PCT/CN2012/086341 priority patent/WO2014089753A1/fr
Publication of WO2014089753A1 publication Critical patent/WO2014089753A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/17—Details of further file system functions
    • G06F16/174—Redundancy elimination performed by the file system
    • G06F16/1744—Redundancy elimination performed by the file system using compression, e.g. sparse files

Definitions

  • the present invention relates to the field of information technology, and in particular, to a file compression method, a file decompression method, an apparatus, and a server. Background technique
  • the file is first split into multiple data blocks, and then the split multiple data blocks are compressed in parallel, and then, each bit is compressed in units of bits.
  • the data blocks are combined into one compressed file.
  • the entire GZIP compressed file has only the starting address of the compressed data block, and there is no data block number of compressed data and the length of each data block.
  • the second compressed data block can be decompressed, that is, only one data block can be serially decompressed.
  • the embodiment of the invention provides a file compression method, a file decompression method, a device and a server, and the parallel decompression of data improves the speed and efficiency of decompression.
  • the first aspect provides a file compression method, including:
  • Length of the extended data content, number of data blocks, length of each compressed data block, and individual data The CRC value of the block is stored in the additional option;
  • the performing the parallel compression on the multiple data blocks comprises: performing parallel compression on the plurality of data blocks by using multiple compression engines.
  • the additional option further includes: SI1 and SI2, where the SI1 and SI2 represent additional options.
  • the ID of the extended data is a first aspect or the first possible implementation manner of the first aspect.
  • the second aspect provides a file decompression method, including:
  • the obtaining the length of each compressed data block in the compressed file, the number of data blocks, and the cyclic redundancy check CRC value of each data block specifically includes:
  • each compressed data block The length of each compressed data block, the number of data blocks, and the cyclic redundancy check CRC value of each data block are obtained from the additional options in the compressed file header extension extra option.
  • the performing, the decompressing the compressed data blocks in parallel includes:
  • the plurality of compressed data blocks are respectively decompressed in parallel by a plurality of decompression engines.
  • a third aspect provides a file compression apparatus, including:
  • a splitting unit configured to split the file into a plurality of data blocks, and count the number of the plurality of data blocks; the first calculating unit is configured to calculate the extended data content according to the number of the plurality of data blocks Length, the memory occupied by the additional options according to the length; a compression unit, configured to perform parallel compression on the plurality of data blocks to obtain a plurality of compressed data blocks; and a second calculating unit, configured to separately calculate each data block when the compression unit performs parallel compression on the plurality of data blocks Cyclic redundancy check CRC value;
  • a storage unit the length of the extended data content, the number of data blocks, the length of each compressed data block, and the CRC value of each data block are stored in an additional option;
  • Adding a unit configured to add the additional option to an extended extra option corresponding to a header in a compressed format
  • a merging unit configured to merge the plurality of compressed data blocks to obtain a compressed file after the adding unit adds the additional option to a location corresponding to a header in the GZIP format;
  • a sending unit configured to send the compressed file to the receiving end, so that the receiving end performs parallel decompression on the compressed file.
  • the fourth aspect provides a file decompression device, including:
  • an obtaining unit configured to obtain, according to an additional option of the compressed file header, a length of each compressed data block, a number of data blocks, and a cyclic redundancy check CRC value of each data block;
  • a dividing unit configured to block the compressed file according to the length of the compressed data block and the number of data blocks, to obtain each compressed data block;
  • a decompression unit configured to perform parallel decompression on the compressed data blocks to obtain corresponding data blocks
  • a calculating unit configured to calculate a CRC value of each of the decompressed data blocks when the decompressing unit performs parallel decompression on the compressed data blocks
  • a determining unit configured to determine whether a CRC value of each of the data blocks acquired by the acquiring unit is the same as a CRC value of each data block obtained by calculating the decompression
  • a determining unit configured to determine, when the determining unit determines that the CRC value is the same, that the data block is consistent with the original data block;
  • a merging unit configured to: when the determining unit determines that the data block is consistent with the original data block, merge the decompressed respective data blocks to obtain an original file.
  • the acquiring unit is specifically configured to obtain, according to an additional option in the compressed file header extension extra option, a length of each compressed data block, a number of data blocks, and Cyclic Redundancy Check CRC value for each data block.
  • the fifth aspect provides a server, including:
  • a processor configured to split a file to be compressed into a plurality of data blocks, and count the number of the plurality of data blocks; Calculating a length of the extended data content according to the number of the plurality of data blocks, and applying for an additional occupied memory according to the length;
  • a compression engine group comprising: a plurality of compression engines, configured to perform parallel compression on the plurality of data blocks to obtain a plurality of compressed data blocks;
  • the processor is further configured to calculate a cyclic redundancy check CRC value of each data block, and the length of the extended data content, the number of data blocks, the length of each compressed data block, and the CRC value of each data block. Storing in an additional option; and adding the additional option to the extended extra option corresponding to the header in the GZIP format, merging the plurality of compressed data blocks, obtaining a compressed file, and transmitting the compressed file to the receiving End, so that the receiving end performs parallel decompression on the compressed file.
  • a sixth aspect provides a server, including:
  • a processor configured to obtain, according to an additional option of the compressed file header, a length of each compressed data block, a number of data blocks, and a cyclic redundancy check CRC value of each data block; according to the length of the compressed data block The number of data blocks is divided into blocks, and each compressed data block is obtained;
  • a decompressing engine group configured to perform parallel decompression on the compressed data blocks to obtain corresponding data blocks
  • the processor is further configured to calculate a cyclic redundancy check CRC value of the respective data blocks obtained by decompression; if it is determined that the obtained CRC value of each data block and the CRC value of each data block obtained by decompression If the data block is the same as the original data block, the respective data blocks obtained by the decompression are combined to obtain the original file.
  • the length of each compressed data block and the Cyclic Redundancy Check value of each data block are added to the header information.
  • the compressed file may be decompressed in parallel according to the length of each compressed data block and the CRC value of each data block, thereby improving the speed of understanding compression. effectiveness.
  • FIG. 1 is a flowchart of a file compression method according to an embodiment of the present invention
  • FIG. 2 is a flowchart of a file decompression method according to an embodiment of the present invention
  • FIG. 3 is a schematic structural diagram of a file compression apparatus according to an embodiment of the present disclosure
  • FIG. 4 is a schematic structural diagram of a file decompressing apparatus according to an embodiment of the present invention.
  • FIG. 5 is a schematic structural diagram of a server according to an embodiment of the present disclosure.
  • FIG. 6 is a schematic structural diagram of another server according to an embodiment of the present disclosure.
  • FIG. 7 is a flowchart of an application example of a file compression method according to an embodiment of the present invention.
  • FIG. 8 is a flowchart of an application example of a file decompression method according to an embodiment of the present invention.
  • the technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. example. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.
  • FIG. 1 is a flowchart of a file compression method according to an embodiment of the present invention. The method includes: Step 101: split a file into multiple data blocks, and count the plurality of data blocks.
  • the server (X86 server, minimalist instruction set computer, IA-64 server, etc.) for files (such as UNIX system files, locally stored files or received files, or various formats running on the operating system)
  • files such as UNIX system files, locally stored files or received files, or various formats running on the operating system
  • the files especially the text files, etc.
  • splitting methods can be used. For different data blocks, different splitting methods can be used. For example, TMPGEnc can be used to split files in MPEG format. ASF can also be used. Tools split files in ASF or WMV format, and can also use AVI chop to split files in MPEG4 format and so on.
  • the method of splitting may be split according to a fixed number of bytes, or may be equally divided according to the size of the file, and may be arbitrarily split according to requirements, which is not limited in this embodiment.
  • the file in this embodiment refers to a file suitable for GZIP compression or decompression.
  • Step 102 Calculate a length of the extended data content according to the number of the plurality of data blocks, and apply for memory occupied by the additional option according to the length;
  • the length of the extended data content (XLEN, eXtra LENgth) is: The sum of the lengths of each data block, that is, the sum of the lengths of 10 data blocks. This embodiment defines the length occupied by all data blocks as the length of the extended data content.
  • the memory of 82bite can be applied through the malloc function.
  • the additional options are located in an extended option of the header of the compressed file, and the structure of the additional optional items is shown in Table 1 below.
  • Step 103 Perform parallel compression on the plurality of data blocks to obtain a plurality of compressed data blocks, and calculate a cyclic redundancy check CRC value of each data block;
  • the parallel compression needs to compress the multiple data blocks by using multiple compression engines.
  • parallel compression in the case of hardware compression, parallel compression is to use multiple compression engines to simultaneously multiple data.
  • Block compression in the case of software compression, parallel compression is the use of multi-threading technology to compress multiple data blocks while the central processing unit (CPU) has multiple physical cores.
  • the CRC check value of each data block is calculated.
  • One of the principles of the CRC algorithm is, but not limited to, the following: By means of polynomial division, the remaining numbers are check fields.
  • Polynomial division is used: The remainder is: 1010 (ie the check field is: 1010).
  • Step 104 Store the length of the extended data content, the number of data blocks, the length of each compressed data block, and the CRC value of each data block in an additional option;
  • the file is first divided into two data blocks, the number of statistical data blocks, and the length of the extended data content is obtained according to the number of data blocks; then, after compressing each data block, Obtaining the length of each compressed data block, and calculating a CRC value of each data block, and then, the length of the extended data content, the number of data blocks, the length of the first compressed data block, and the first data block a CRC value, a length of the second compressed data block, and a CRC value of the second data block, until the length of the second compressed data block and the CRC value of the second data block are sequentially added to the corresponding XLEN field in the additional option, respectively.
  • the additional options may further include identification information such as SI1 and SI2, wherein the SI1 and SI2 are IDs of the extended data content in the additional options.
  • the SI1 and the SI2 are identification information.
  • the XLEN is the length of the extended content, that is, the length from NUM to nCRC;
  • the NUM represents the number of data blocks owned by the compressed file
  • the 1 LEN, 1 CRC to NLEN, NCRC is used to indicate extended information, including length information of each compressed data block, and specifically includes: length of each compressed data block (block) after compression and CRC32 value of each data block before compression
  • the CRC32 is a data error check code.
  • the data is checked for error by comparing whether the original data and the CRC32 value of the compressed packet decompressed data are the same.
  • the structure of the additional options includes the specific contents as shown in Table 2:
  • the adaptive repair step 105 may also be performed as needed: adding the additional option to the extended extra option corresponding to the header in the compressed format. Merging the plurality of compressed data blocks to obtain a compressed file;
  • the extended extra option may include a source file name, a comment text, or a CRC 16 and the like in addition to the additional options.
  • This embodiment mainly expands the additional options. That is, the length of the extended data content, the number of data blocks, the length of each compressed data block, and the CRC value of each data block are increased in an additional option, so that the receiving end performs each data block according to the added information. Decompress in parallel.
  • Each of the independent GZIP compressed files includes a header, a data portion, and a trailer.
  • the header may include an extended extra option, and may also include: ID1 ID2, CM, FLG, MTIME, XFL, OS, where
  • the MTIME Indicates compression time, in UNIX format;
  • the XFL indicates compression mode
  • the FLG indicates an extended function identifier, and each BIT represents an additional data, and the specific corresponding content is represented in an extra, the extra includes: an additional option, an original file name, a comment text, and a CRC16.
  • the GZIP compressed file may also include the data part and the tail part. That is, currently, each independent GZIP compressed file is composed of the header, the data part and the tail part. composition.
  • the information of the head is as described above, and details are not described herein again.
  • the data portion includes one or more data blocks (this embodiment is one or more compressed data blocks, the same below), and the format of each data block includes BFINAL, BTYPE, and data DATA information.
  • the BFINAL bit occupies lbit, indicating whether it is the last data block. If the BFINAL bit is 1, it indicates the last data block.
  • BYTPE indicates the compression mode of the data, the compression type (2 bit), either static Huffman compression (01), dynamic Huffman compression (10) or uncompressed (00); DATA indicates compressed data, (for example, LZ77+huffman Encoding + binary tree characteristics, etc.).
  • the tail includes the 32-bit CRC value of the original file and the lower 32-bit value of the original data length, and the tail is mainly used to verify whether the decompressed file is consistent with the original file before compression.
  • Step 106 Send the compressed file to the receiving end, so that the receiving end performs parallel decompression on the compressed file.
  • the existing serial decompression may be used, or the parallel decompression provided by the embodiment of the present invention may be used (as shown in the following FIG. 2 embodiment). If serial decompression is used, it is necessary to use the content of the tail of the compressed file to verify whether the decompressed file is consistent with the original file before compression; and if the parallel decompression of the present application is used, it is not necessary to use the content of the tail. To verify whether the decompressed file is consistent with the original file before compression, but to verify the decompressed data block and the original data before compression according to the respective CRC values of the additional options in the extended extra option in the header. Whether the blocks are consistent.
  • the length of each compressed data block and the CRC value of each data block are added in an additional option of the header information by using a new field, so that when the receiving end decompresses, The compressed file can be decompressed in parallel based on this information, thereby improving the speed and efficiency of understanding compression.
  • FIG. 2 is a flowchart of a file decompression method according to an embodiment of the present invention, where the method includes:
  • Step 201 Obtain a length of each compressed data block in the compressed file, a number of data blocks, and a cyclic redundancy check CRC value of each data block;
  • the process of obtaining is as follows: The server obtains the length of each compressed data block, the number of data blocks, and the cyclic redundancy check CRC value of each data block from the additional options in the compressed file header extension extra option.
  • Step 202 Block the compressed file according to the length of the compressed data block and the number of data blocks, to obtain each compressed data block;
  • Step 203 performing parallel decompression on the compressed data blocks to obtain corresponding data blocks.
  • the server may input each compressed data block into a corresponding decompression engine, and respectively, by using multiple decompression engines.
  • the compressed data block is decompressed in parallel.
  • the process of parallel decompression is well known to those skilled in the art and will not be described herein.
  • Step 204 Calculate a CRC value of each of the data blocks obtained by decompression
  • Step 205 If the obtained CRC value of each data block and the decompressed data block are
  • Step 206 Combine the decompressed each of the data blocks to obtain an original file.
  • the server when decompressing, the server first obtains the length of each compressed data block and the CRC value of each data block from the compressed file, and decompresses the compressed file in parallel according to the information, and decompresses the compressed file. Data blocks can be checked for correctness by independent CRC values, which improves the speed and efficiency of compression.
  • the embodiment of the present invention further provides a file compression device, which is shown in FIG.
  • the device includes: a splitting unit 31, a first calculating unit 32, a compressing unit 33, and a second a calculating unit 34, a storage unit 35, an adding unit 36, and a merging unit 37, wherein the splitting unit 31 is configured to split the file into a plurality of data blocks, and count the number of the plurality of data blocks; , can be split according to the fixed number of bytes, can be evenly divided, and can be split as needed.
  • the first calculating unit 32 is configured to calculate a length of the extended data content according to the number of the plurality of data blocks, and apply for an additional occupied memory according to the length; the compressing unit 33 is configured to: The plurality of data blocks are compressed in parallel to obtain a plurality of compressed data blocks.
  • the plurality of data blocks may be compressed in parallel by a plurality of compression engines.
  • the second calculating unit 34 is configured to perform the compression on the compression unit 33.
  • the cyclic redundancy check CRC value of each data block is separately calculated;
  • the storage unit 35 is configured to use the length of the extended data content, the number of data blocks, and each compressed data block.
  • the length and the CRC value of each data block are stored in an additional option; the adding unit 36 is configured to add the additional option to the extended extra option corresponding to the header in the compressed format; the merging unit 37, And after the adding unit adds the additional option to a location corresponding to a header in the GZIP format, combining the plurality of compressed data blocks to obtain a compressed file.
  • the embodiment of the present invention further provides a file decompression device, which is shown in FIG. 4, and the device includes: an obtaining unit 41, a dividing unit 42, a decompressing unit 43, a calculating unit 44, and a determining unit 45. a determining unit 46, a merging unit 47, and a sending unit 48, wherein the obtaining unit 41 is configured to obtain, according to an additional option of the compressed file header, a length of each compressed data block, a number of data blocks, and each data block.
  • the cyclic redundancy check CRC value is specifically used to obtain the length of each compressed data block, the number of data blocks, and the cyclic redundancy check CRC of each data block from the additional options in the compressed file header extension extra option.
  • the dividing unit 42 is configured to block the compressed file according to the length of the compressed data block and the number of data blocks to obtain each compressed data block; and the decompressing unit 43 is configured to Each of the compressed data blocks is decompressed in parallel to obtain corresponding data blocks.
  • the calculating unit 44 is configured to perform, on the decompression unit, the respective compression numbers.
  • the determining unit 45 is configured to determine the CRC value of the respective data blocks acquired by the acquiring unit and calculate and decompress the obtained Whether the CRC value of each data block is the same; the determining unit 46, configured to determine, when the determining unit determines that the CRC value is the same, that the data block is consistent with the original data block; the merging unit 47 is configured to be in the determining unit When it is determined that the data block is consistent with the original data block, the respective data blocks obtained by the decompression are combined to obtain an original file; the sending unit 48 is configured to send the compressed file to the receiving end, so as to facilitate The receiving end performs parallel decompression on the compressed file.
  • the embodiment of the present invention further provides a server, which is shown in FIG. 5.
  • the server includes: a processor 51 and a compression engine group 52, wherein the processor 51 is configured to be compressed. Splitting the file into a plurality of data blocks, and counting the number of the plurality of data blocks; calculating the length of the extended data content according to the number of the plurality of data blocks, and applying the memory occupied by the additional options according to the length
  • the compression engine group 52 includes a plurality of compression engines for performing parallel compression on the plurality of data blocks to obtain a plurality of compressed data blocks.
  • the processor 51 is further configured to calculate a cyclic redundancy of each data block.
  • the embodiment of the present invention further provides another server, which is shown in FIG. 6.
  • the server includes: a processor 61 and a decompression engine group 62, wherein the processor 61 is configured to compress files. Obtaining, in an additional option of the header, a length of each compressed data block, a number of data blocks, and a cyclic redundancy check CRC value of each data block; according to the length of the compressed data block and the number of data blocks The compressed file is divided into blocks to obtain respective compressed data blocks.
  • the decompression engine group 62 is configured to perform parallel decompression on the compressed data blocks to obtain corresponding data blocks.
  • the processor 61 is further used to Calculating a cyclic redundancy check CRC value of the respective data blocks obtained by decompression; if it is determined that the obtained CRC value of each data block is the same as a CRC value of each data block obtained by decompression, the data block is The original data blocks are consistent; the respective data blocks obtained by the decompression are combined to obtain the original file.
  • the advantages of multi-core or multi-channel technology are exerted.
  • the length information of each block block and the CRC32 value of each block block original data block are stored in an additional option of the header extension option during compression, so that When decompressing, parallel decompression is performed according to the length information of each block block and the CRC32 value of the original block of each block block, thereby improving the speed and efficiency of understanding compression.
  • FIG. 7 is a flowchart of an application example of a file compression method according to an embodiment of the present invention. As shown in the figure, the compression mode mainly uses multiple compression engines of hardware or software to perform parallel on a block block. Compression, the entire compression process mainly includes:
  • the processor divides the original file into sub-data blocks, for example, splits the file into n sub-files, that is, sub-file 1, sub-file 2, and sub-file n, and counts the number of each sub-file, for example, n ;
  • the processor calculates the length of the extended data (XLEN) according to the number of subfiles (ie, n), and applies for storing the memory occupied by the extended data;
  • the processor transmits the respective sub-blocks to the corresponding respective compression engines (compression engine groups), and each of the compression engines performs parallel compression on the corresponding sub-files, and calculates a CRC32 value of the data block;
  • each compression engine After compressing each subfile into a compressed subfile, each compression engine stores the length of the compressed subfile and the CRC32 value of the atomic file into an additional option in the extended option, wherein the length of the subfile is in units of bits;
  • the processor adds additional options to the corresponding position of the compressed file header (ie, expandable options), and then merges the compressed subfiles to obtain a compressed file.
  • FIG. 8 is a flowchart of an application example of a file decompression method according to an embodiment of the present invention.
  • the decompression method mainly uses multiple decompression engines of hardware or software (ie, a solution).
  • Compression engine group Parallel decompression of a block block, the entire decompression process mainly includes:
  • the processor obtains the number of each block (ie, compressed subfile or compressed data block) and the length of each block from the additional options in the extended option in the compressed file, and according to the number of each block and the length of each block
  • the compressed file is divided into blocks to obtain individual blocks, such as block1 and block2 up to block n.
  • Each decompression engine decompresses each block in parallel and calculates the CRC value corresponding to each block.
  • the processor After the processor decompresses each block block into data blocks by each decompression engine, the processor reads the CRC value in the additional options corresponding to each block;
  • the processor compares the CRC value corresponding to each block calculated after decompression with the CRC32 value corresponding to each block read, and if the two are the same, it is confirmed that the data block is consistent with the original data block. 6. After all the blocks are decompressed, merge the decompressed data blocks to obtain the original file.
  • the present invention can be implemented by means of software plus a necessary general hardware platform, and of course, can also be through hardware, but in many cases, the former is a better implementation. the way.
  • the technical solution of the present invention which is essential or contributes to the prior art, may be embodied in the form of a software product, which may be stored in a storage medium such as a ROM/RAM or a disk. , an optical disk, etc., includes instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) to perform the methods described in various embodiments of the present invention or portions of the embodiments.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

La présente invention concerne un procédé de compression de fichiers, un procédé de décompression, un dispositif et un serveur. Le procédé de décompression comprend les étapes suivantes : l'acquisition de la longueur de chaque bloc de données compressé, du nombre de blocs de données et de la valeur de contrôle de redondance cyclique (CRC) de chaque bloc de données dans un fichier compressé ; en fonction de la longueur du bloc de données compressé et du nombre de blocs de données, le blocage du fichier compressé pour obtenir chaque bloc de données compressé ; l'exécution d'une décompression en parallèle sur chaque bloc de données compressé pour obtenir chaque bloc de données correspondant ; le calcul de la valeur CRC de chaque bloc de données obtenu par la décompression ; si la valeur CRC acquise de chaque bloc de données est la même que la valeur CRC de chaque bloc de données obtenu par la décompression, le bloc de données est compatible avec un bloc de données d'origine ; et la fusion de chaque bloc de données obtenu par la décompression afin d'obtenir un fichier d'origine. En fonction de la longueur acquise de chaque bloc de données compressé et de la valeur CRC de chaque bloc de données, la présente invention effectue une décompression en parallèle sur le fichier compressé, améliorant ainsi la vitesse et l'efficacité de la décompression.
PCT/CN2012/086341 2012-12-11 2012-12-11 Procédé de compression de fichiers, procédé de décompression de fichiers, dispositif et serveur WO2014089753A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201280003410.0A CN103384884B (zh) 2012-12-11 2012-12-11 一种文件压缩方法、文件解压缩方法、装置及服务器
PCT/CN2012/086341 WO2014089753A1 (fr) 2012-12-11 2012-12-11 Procédé de compression de fichiers, procédé de décompression de fichiers, dispositif et serveur

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2012/086341 WO2014089753A1 (fr) 2012-12-11 2012-12-11 Procédé de compression de fichiers, procédé de décompression de fichiers, dispositif et serveur

Publications (1)

Publication Number Publication Date
WO2014089753A1 true WO2014089753A1 (fr) 2014-06-19

Family

ID=49492140

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2012/086341 WO2014089753A1 (fr) 2012-12-11 2012-12-11 Procédé de compression de fichiers, procédé de décompression de fichiers, dispositif et serveur

Country Status (2)

Country Link
CN (1) CN103384884B (fr)
WO (1) WO2014089753A1 (fr)

Families Citing this family (44)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105740298A (zh) * 2014-12-12 2016-07-06 北京奇虎科技有限公司 文件处理方法、装置及服务端设备
CN105573785A (zh) * 2015-12-11 2016-05-11 青岛海信电器股份有限公司 一种差分包的制作方法及装置
CN108781213B (zh) * 2016-03-14 2020-08-14 华为技术有限公司 一种用于传输数据的方法、装置和系统
CN106021003B (zh) * 2016-05-05 2019-11-29 捷开通讯(深圳)有限公司 智能终端的修复方法、智能终端以及服务器
CN106126367B (zh) * 2016-06-28 2019-09-20 湖北锐世数字医学影像科技有限公司 一种文件的自校验方法及系统
CN107919935B (zh) * 2016-10-08 2022-04-15 中兴通讯股份有限公司 一种提高语音通信质量的方法及装置
CN107977233B (zh) 2016-10-19 2021-06-01 华为技术有限公司 内核镜像文件快速加载方法和装置
CN106503165A (zh) * 2016-10-31 2017-03-15 杭州华为数字技术有限公司 压缩、解压方法、装置以及设备
CN106648955B (zh) * 2016-11-15 2020-01-31 杭州华为数字技术有限公司 压缩方法及相关装置
WO2018201377A1 (fr) * 2017-05-04 2018-11-08 Nokia Technologies Oy Générateur de code détecteur d'erreurs et de code correcteur d'erreurs unifié
CN107967157B (zh) * 2017-08-17 2021-06-01 青岛海信移动通信技术股份有限公司 一种生成ota中间包过程中的数据处理方法及装置
CN108134609A (zh) * 2017-12-21 2018-06-08 深圳大学 一种通用数据gz格式的多线程压缩与解压方法及装置
WO2019119336A1 (fr) * 2017-12-21 2019-06-27 深圳大学 Procédés de compression et de décompression multifil dans un format de données générique gz, et dispositif
CN108446300B (zh) * 2018-01-26 2021-04-09 北京奇虎科技有限公司 数据信息的扫描方法及装置
CN108509642A (zh) * 2018-04-12 2018-09-07 郑州云海信息技术有限公司 压缩、解压gzip格式文件的方法、装置及存储介质
CN108520067A (zh) * 2018-04-12 2018-09-11 郑州云海信息技术有限公司 压缩、解压gzip格式文件的方法、装置及存储介质
CN110784225A (zh) 2018-07-31 2020-02-11 华为技术有限公司 数据压缩、解压方法及相关装置、电子设备、系统
CN108958966A (zh) * 2018-09-27 2018-12-07 合肥达博科技有限公司 一种无损数据压缩的数据保护方法和装置
CN109582653B (zh) * 2018-11-14 2020-12-08 网易（杭州）网络有限公司 文件的压缩、解压缩方法及设备
CN111290697B (zh) * 2018-12-07 2022-01-28 上海寒武纪信息科技有限公司 数据压缩方法、编码电路和运算装置
CN111382852B (zh) * 2018-12-28 2022-12-09 上海寒武纪信息科技有限公司 数据处理装置、方法、芯片及电子设备
CN111382853B (zh) * 2018-12-28 2022-12-09 上海寒武纪信息科技有限公司 数据处理装置、方法、芯片及电子设备
CN111294057A (zh) * 2018-12-07 2020-06-16 上海寒武纪信息科技有限公司 数据压缩方法、编码电路和运算装置
CN111381878A (zh) * 2018-12-28 2020-07-07 上海寒武纪信息科技有限公司 数据处理装置、方法、芯片及电子设备
CN111294056B (zh) * 2018-12-07 2022-03-29 上海寒武纪信息科技有限公司 数据解压方法和编码电路
CN111382856B (zh) * 2018-12-28 2022-12-09 上海寒武纪信息科技有限公司 数据处理装置、方法、芯片及电子设备
WO2020114283A1 (fr) * 2018-12-07 2020-06-11 上海寒武纪信息科技有限公司 Procédé et dispositif de traitement de données
CN109710581B (zh) * 2018-12-25 2023-05-30 四川巧夺天工信息安全智能设备有限公司 一种解压qcow镜像文件中压缩数据的方法
WO2020232682A1 (fr) * 2019-05-22 2020-11-26 深圳大学 Système et procédé de compression parallèle utilisant du matériel
CN110247666B (zh) * 2019-05-22 2023-08-18 深圳大学 一种硬件并行压缩的系统及方法
CN112099725A (zh) 2019-06-17 2020-12-18 华为技术有限公司 一种数据处理方法、装置及计算机可读存储介质
CN110727720B (zh) * 2019-10-21 2023-06-20 腾讯科技（深圳）有限公司 列表显示及查询方法、装置、存储介质和计算机设备
CN110808054B (zh) * 2019-11-04 2022-05-06 思必驰科技股份有限公司 多路音频的压缩与解压缩方法及系统
CN111258621B (zh) * 2019-11-19 2022-08-02 浙江瑞银电子有限公司 一种固件差异化升级方法
CN111723059B (zh) * 2020-05-25 2021-03-16 深圳市科楠科技开发有限公司 一种数据压缩方法、装置、终端设备及存储介质
CN112084158A (zh) * 2020-09-25 2020-12-15 北京百家科技集团有限公司 一种数据集文件压缩方法及装置
CN114697309A (zh) * 2020-12-25 2022-07-01 北京三快在线科技有限公司 文件传输方法、文件处理方法及装置
CN112866227A (zh) * 2021-01-13 2021-05-28 北京连山科技股份有限公司 一种文件授权保护方法和系统
CN112860646B (zh) * 2021-02-24 2022-12-02 上海泰宇信息技术股份有限公司 一种海量文件档案分布式聚合压缩与单一式抽取的方法
CN113225180A (zh) * 2021-04-29 2021-08-06 北京连山科技股份有限公司 一种通信密钥的保护方法和系统
CN114726924B (zh) * 2022-05-17 2022-10-21 北京安盟信息技术股份有限公司 一种基于tcp/ip协议栈特性提高网络带宽利用率方法
CN117795855A (zh) * 2022-05-27 2024-03-29 深圳华大基因科技服务有限公司 Fastq数据的处理方法及装置、电子设备和存储介质
CN115883839B (zh) * 2023-03-09 2023-06-06 湖北芯擎科技有限公司 一种图像校验方法、装置、设备及计算机可读存储介质
CN118036048B (zh) * 2024-04-11 2024-06-14 深圳至简天成科技有限公司 一种在线调试平台的数据持久化方法、装置及计算机介质

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102138170A (zh) * 2008-08-25 2011-07-27 索尼公司 数据转换器、数据转换方法和程序
CN102724500A (zh) * 2012-06-05 2012-10-10 沙基昌 一种视频数据的压缩/解压缩方法及系统
CN102740075A (zh) * 2012-06-05 2012-10-17 沙基昌 一种视频数据的压缩/解压缩方法及系统

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5686915A (en) * 1995-12-27 1997-11-11 Xerox Corporation Interleaved Huffman encoding and decoding method
CN102244518B (zh) * 2010-05-10 2016-01-20 百度在线网络技术（北京）有限公司 并行解压缩的硬件实现的系统及方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102138170A (zh) * 2008-08-25 2011-07-27 索尼公司 数据转换器、数据转换方法和程序
CN102724500A (zh) * 2012-06-05 2012-10-10 沙基昌 一种视频数据的压缩/解压缩方法及系统
CN102740075A (zh) * 2012-06-05 2012-10-17 沙基昌 一种视频数据的压缩/解压缩方法及系统

Also Published As

Publication number Publication date
CN103384884A (zh) 2013-11-06
CN103384884B (zh) 2016-11-16

Similar Documents

Publication Publication Date Title
WO2014089753A1 (fr) Procédé de compression de fichiers, procédé de décompression de fichiers, dispositif et serveur
US10680645B2 (en) System and method for data storage, transfer, synchronization, and security using codeword probability estimation
US10706018B2 (en) Bandwidth-efficient installation of software on target devices using reference code libraries
US10509582B2 (en) System and method for data storage, transfer, synchronization, and security
US11366790B2 (en) System and method for random-access manipulation of compacted data files
US11853550B2 (en) System and method for data compaction and security using multiple encoding algorithms
US11748309B2 (en) System and method for error-resilient data reduction
US20240028563A1 (en) System and method for securing high-speed intrachip communications
US20240069731A1 (en) System and method for data compaction and encryption of anonymized data records
US20230401173A1 (en) System and methods for secure deduplication of compacted data
JP6113816B1 (ja) 情報処理システム、情報処理装置、及びプログラム
WO2015176249A1 (fr) Procédé de transmission pour un disque dur d'interface ethernet sans fil, dispositif associé et système
US20230253980A1 (en) System and method for blockchain data compaction
US11609881B2 (en) System and method for computer data type identification
US20240080040A1 (en) System and method for data storage, transfer, synchronization, and security using automated model monitoring and training
WO2020264522A1 (fr) Stockage, transfert, synchronisation et sécurité de données à l'aide d'un codage récursif

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 12889982

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 12889982

Country of ref document: EP

Kind code of ref document: A1