CN116431585A - File compression method and device, and file decompression method and device - Google Patents
File compression method and device, and file decompression method and device Download PDFInfo
- Publication number
- CN116431585A CN116431585A CN202210002255.6A CN202210002255A CN116431585A CN 116431585 A CN116431585 A CN 116431585A CN 202210002255 A CN202210002255 A CN 202210002255A CN 116431585 A CN116431585 A CN 116431585A
- Authority
- CN
- China
- Prior art keywords
- file
- compressed
- data
- decompressed
- metadata
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 66
- 238000007906 compression Methods 0.000 title claims abstract description 49
- 230000006835 compression Effects 0.000 title claims abstract description 44
- 230000006837 decompression Effects 0.000 title claims description 36
- 238000005192 partition Methods 0.000 claims abstract description 21
- 238000004891 communication Methods 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 8
- 238000000638 solvent extraction Methods 0.000 claims description 4
- 230000005540 biological transmission Effects 0.000 abstract description 11
- 230000008569 process Effects 0.000 description 12
- 238000010586 diagram Methods 0.000 description 5
- 230000009471 action Effects 0.000 description 3
- 230000002093 peripheral effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/174—Redundancy elimination performed by the file system
- G06F16/1744—Redundancy elimination performed by the file system using compression, e.g. sparse files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/064—Management of blocks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Human Computer Interaction (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application relates to a method and a device for compressing a file, a method and a device for decompressing the file, wherein the method for compressing the file comprises the following steps: reading a file to be compressed in a file system according to a preset partition to obtain partition data; compressing the block data, and prefixing metadata on the compressed block data to obtain compressed data, wherein the metadata are used for representing the position of the compressed data in the file to be compressed; and assembling all the compressed data to obtain a compressed file. Through this application, all be carrying out whole compression to the file when handling big file transmission among the prior art has been solved, the lower problem of upper strata transmission efficiency is led to easily.
Description
Technical Field
The present disclosure relates to the field of data processing technologies, and in particular, to a method and an apparatus for compressing a file, and a method and an apparatus for decompressing a file.
Background
There are many compression and decompression algorithms for different scenes, which can provide a good balance when processing various data, or can provide a user with options of speed and compression ratio adjustment, etc. that can be optimized in a self-defining way. However, when dealing with large file transmission, the whole compression is performed on the file to ensure the integrity of the compressed file or single stream, and in the case of large file transmission, the compression mode is easy to become a bottleneck of upper layer transmission optimization, such as a definite two-stage process: file downloading and file decompression cannot be performed in parallel while ensuring file integrity.
Disclosure of Invention
The application provides a file compression method and device, a file decompression method and device, and aims to at least solve the problem that in the prior art, when large files are transmitted, the files are integrally compressed, so that the upper layer transmission efficiency is low easily.
In a first aspect, the present application provides a method for compressing a file, including: reading a file to be compressed in a file system according to a preset partition to obtain partition data; compressing the block data, and prefixing metadata on the compressed block data to obtain compressed data, wherein the metadata are used for representing the position of the compressed data in the file to be compressed; and assembling all the compressed data to obtain a compressed file.
In a second aspect, the present application provides a method for decompressing a file, including: determining a first storage space occupied by an original file corresponding to a file to be decompressed, wherein the original file is a file to be compressed of the file to be decompressed before compression, the file to be decompressed is obtained by assembling a plurality of compressed data, the compressed data are obtained by compressing block data, the block data are obtained by reading the file to be compressed according to preset blocks, and the compressed data prefix is provided with position metadata used for representing the compressed data in the file to be compressed; pre-distributing a second storage space corresponding to the first storage space in a file system; and decompressing the file to be decompressed in the second storage space based on the metadata to obtain the original file.
In a third aspect, there is provided a compression apparatus for a file, comprising: the first reading module is used for reading the file to be compressed in the file system according to the preset partitioning to obtain partitioning data; the compression module is used for compressing the block data, and prefix metadata is arranged on the compressed block data to obtain compressed data, wherein the metadata are used for representing the position of the compressed data in the file to be compressed; and the assembly module is used for assembling all the compressed data to obtain a compressed file.
In a fourth aspect, there is provided a decompression apparatus for a file, including: the device comprises a determining module, a compressing module and a judging module, wherein the determining module is used for determining a first storage space occupied by an original file corresponding to a file to be decompressed, wherein the original file is a file to be compressed of the file to be decompressed before compression, the file to be decompressed is obtained by assembling a plurality of compressed data, the compressed data is obtained by compressing block data, the block data is obtained by reading the file to be compressed according to a preset block, and a compressed data prefix is provided with position metadata used for representing the compressed data in the file to be compressed; the pre-allocation module is used for pre-allocating a second storage space corresponding to the first storage space in the file system; and the decompression module is used for decompressing the file to be decompressed in the second storage space based on the metadata to obtain the original file.
In a fifth aspect, there is provided a processing apparatus for a file, including: the apparatus according to the third aspect and the apparatus according to the fourth aspect.
In a sixth aspect, a distributed coordination server includes a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus;
a memory for storing a computer program;
a processor for implementing the steps of the method according to any embodiment of the first aspect or the steps of the method according to any embodiment of the second aspect when executing a program stored on a memory.
In a seventh aspect, a computer readable storage medium is provided, on which a computer program is stored which, when being executed by a processor, implements the steps of the method according to any embodiment of the first aspect, or implements the steps of the method according to any embodiment of the second aspect.
Compared with the related art, the method and the device for compressing files, the method and the device for decompressing files provided in the embodiment can read the files to be compressed according to preset blocks to obtain block data, further compress the block data, prefix metadata is added to the compressed block data to obtain compressed data, and the compressed files are assembled based on the compressed data to obtain the compressed files, that is, the compressed files are read and compressed based on the blocks and prefixed with metadata, so that the block data can be read and compressed first under the condition that the transmitted files are large, and the compressed data have metadata, so that unordered compression of the files can be realized, and corresponding compressed files can still be generated without affecting the decompression of the files under the condition that the integrity of the compressed files is not needed, thereby solving the problems that in the prior art, when the large files are transmitted, the files are integrally compressed, and the upper layer transmission efficiency is easy to be lower.
The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below to provide a more thorough understanding of the other features, objects, and advantages of the application.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
In order to more clearly illustrate the embodiments of the invention or the technical solutions of the prior art, the drawings which are used in the description of the embodiments or the prior art will be briefly described, and it will be obvious to a person skilled in the art that other drawings can be obtained from these drawings without inventive effort.
Fig. 1 is a flow chart of a method for compressing a file according to an embodiment of the present application;
fig. 2 is a flow chart of a method for decompressing a file according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of a compressing apparatus for a file according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of a decompression device for files according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of a document processing device according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
For the purposes of making the objects, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present application based on the embodiments herein.
First, related terms and prior art related to the embodiments of the present application are explained as follows:
compression: in order to reduce the volume of the file or save the time of data transmission, the mechanism of processing the file by a special algorithm to reduce the size of the data file can be generally classified into lossy compression and lossless compression, and the compression efficiency and speed depend on the regularity of the data of the file itself and the algorithm used. Wherein, the special algorithm can be: LZ4 (LZ 4 is a lossless compression algorithm with a compression rate of 400MB/s (0.16 bytes/cycle) per core, zstd (Zstandard for short), a fast real-time open source data compression program), gzip (short for several file compression programs, commonly referred to as implementations of GNU plans, where Gzip stands for GNU zip).
Decompression: and in the reverse process of compression, recovering the data of the original file from the compressed data through a special algorithm.
As shown in fig. 1, an embodiment of the present application provides a method for compressing a file, where the method includes the steps of:
102, reading a file to be compressed in a file system according to a preset partition to obtain partition data;
in this embodiment of the present application, the preset block is used to represent the size of the read block data, for example, if the preset block is 20M, then reading 20M data from the file to be compressed as the block data; or, if the preset block is 10M, reading 10M data from the file to be compressed as the block data. The above preset blocks are only examples, and the specific size of the preset blocks may be set correspondingly according to the actual situation.
therefore, in the embodiment of the application, each piece of compressed data has corresponding metadata, and because the metadata is used for representing the position of the compressed data in the file to be compressed, after the data is compressed in a blocking manner, the position of the decompressed data can be determined based on the metadata in the decompression process, so that the integrity of the file can be still ensured after the compression of the blocks is realized;
and 106, assembling all the compressed data to obtain a compressed file.
Through 102 to step 106 in the embodiment of the present application, the file to be compressed may be read according to the preset blocks to obtain the block data, further compress the block data, prefix metadata on the compressed block data to obtain the compressed data, and assemble the compressed data to obtain the compressed file based on the compressed data, that is, the compressed file is read and compressed based on the blocks, and the compressed data is prefixed with metadata, so when the transmitted file is larger, the block data may be read and compressed first, and the compressed data has metadata, so that disordered compression of the file may be realized, and when the integrity of the compressed file is not needed, the corresponding compressed file may still be generated without affecting decompression of the file, thereby solving the problem in the prior art that when the transmission of the large file is handled, the file is compressed integrally, and the upper layer transmission efficiency is low easily.
In an optional implementation manner of the embodiment of the present application, the method for reading, according to a preset partition, a file to be compressed in a file system and obtaining the partition data in the foregoing step 102 may further include at least one of the following:
step 11, reading a file to be compressed in a file system according to preset blocks corresponding to time periods to obtain block data, wherein different time periods correspond to different preset blocks;
for this step 11, different time periods correspond to different preset partitions, which may be in a specific example: the preset block corresponding to the first time period is 20M, the preset block corresponding to the second time period is 10M, the preset block corresponding to the third time period is 30M, in a specific application scene, if the first time period, the second time period and the third time period are continuous in time, the block data of 20M are read in the first time period, the block data of 10M are read in the second time period and the block data of 30M are read in the second time period and the block data of 30M are read in the third time period, in other application scenes, the block data of 10M are read in the first time period and the block data of 30M are read in the second time period and the block data of 30M are read in the third time period, and the read process is the block data of 20M are read in the first time period and the block data of 20M are read in the second first time period and the second time period.
That is, in the embodiment of the present application, the corresponding preset partition may be set according to the time requirement, for example, the current bandwidth is more left, and then the preset partition that may be set in the previous stage of the compressed file to be transmitted is larger, when the bandwidth is less left, the preset partition that may be set in the time period of subsequent reading may be smaller, so that bandwidth congestion may also be avoided without affecting the compression of the partition.
And step 12, reading the file to be compressed in the file system according to the preset blocks corresponding to the data types to obtain block data, wherein the data types of different data in the file to be compressed correspond to different preset blocks.
For this step 12, since the compression requirements of different data types are different, corresponding preset blocks can be set according to the data types, for example, the data types with higher integrity requirements can be set to be larger, the preset blocks can be compressed with a smaller amount of block data as much as possible, and the data types with lower integrity requirements can be set to be smaller, and the compression of the block data with a larger amount can be performed.
In still another optional implementation manner of the embodiment of the present application, the method for assembling all the compressed data related to the step 106 to obtain the compressed file may further include:
step 21, determining whether additional information associated with the file to be compressed exists;
and step 22, adding the additional information to the compressed data in the case that the additional information associated with the file to be compressed exists, and assembling all the compressed data to which the additional information is added to obtain the compressed file.
The additional information in the embodiment of the present application may be: the creation time of the original file, copyright information, copyright holder, user-defined information, etc., for example, in order to expand the file format, make service-related marks, the user can make special treatment according to the marks to obtain additional information, and for example, by means of the additional information, it is determined which marked block is needed, i.e. the data block without the additional information can be directly ignored.
In addition, in the embodiment of the application, the additional information can be patched into the compressed file in the form of TLV (Tag Length Value) by means of information coding.
Corresponding to the above method for compressing the file in fig. 1, the embodiment of the present application further provides a method for decompressing the file, as shown in fig. 2, including:
and step 206, decompressing the file to be decompressed in the second storage space based on the metadata to obtain the original file.
Through the steps 202 to 206, the corresponding second storage space can be pre-allocated in the file system according to the first storage space occupied by the original file corresponding to the file to be decompressed, so as to further read the second data of the blocks in the file to be decompressed and the corresponding metadata thereof, decompress the second data to the corresponding position of the second storage space based on the metadata, so as to generate the final original file, and the block decompression corresponds to the block compression, and the block decompression realizes the splitting of the file in the compression and decompression processes, thereby realizing unordered compression and decompression without affecting the final merging of the generated compressed file and the original file.
Based on the above example of the compression process in fig. 1, if the preset block is 20M, reading 20M data from the file to be compressed as the block data; or, if the preset block is 10M, reading 10M data from the file to be compressed as the block data; the decompression is performed according to a preset block, for example, the compressed block data is 20M, the 20M block data is decompressed in the decompression, and if the compressed block data is 10M, the 10M block data is decompressed in the decompression, that is, the inverse process corresponding to the compression process.
In an optional implementation manner of the embodiment of the present application, for the method for decompressing a file to be decompressed in the second storage space based on metadata referred to in step 206, the method for obtaining an original file may further include:
step 31, reading compressed data from a file to be decompressed, and decompressing the compressed data;
and step 32, writing the decompressed compressed data into a position corresponding to the metadata in the second storage space to obtain an original file.
Based on the above steps 31 and 32, in a specific decompression process, since metadata is prefixed in the compressed data in the compression process, it can be seen that the decompressed data can be written into a correct position based on the metadata although the data is decompressed by a block, and the merging and the generation of the original file are not affected.
In another optional implementation manner of the embodiment of the present application, before determining the first storage space occupied by the original file corresponding to the file to be decompressed, the method of the embodiment of the present application may further include:
and step 200, reading the additional information associated with the file to be compressed from the file to be decompressed.
The additional information may be creation time of the original file, copyright information, copyright holder, user-defined information, etc., for example, in order to expand the file format, make service-related marking, the user may make special treatment according to the marking to obtain the additional information, and for example, determine which marked block is needed through the additional information, i.e. the data block without the additional information may be directly ignored.
Corresponding to the above method for compressing files in fig. 1, the embodiment of the present application further provides a device for compressing files, as shown in fig. 3, where the device includes:
the first reading module 32 is configured to read a file to be compressed in the file system according to a preset partition to obtain partition data;
the compression module 34 is configured to compress the block data, prefix metadata on the compressed block data, and obtain compressed data, where the metadata is used to characterize a location of the compressed data in a file to be compressed;
and the assembling module 36 is used for assembling all the compressed data to obtain the compressed file.
According to the device, the file to be compressed can be read according to the preset blocks to obtain the block data, the block data are further compressed, metadata is prefixed on the compressed block data to obtain the compressed data, and the compressed file is assembled based on the compressed data to obtain the compressed file, that is to say, the compressed file is read and compressed based on the blocks, metadata is prefixed on the compressed data, so that the block data can be read and compressed first under the condition that the transmitted file is large, the compressed data has metadata, unordered compression of the file can be realized, the corresponding compressed file can still be generated without affecting the decompression of the file under the condition that the integrity of the compressed file is not needed, and the problem that the file is integrally compressed when the large file is transmitted in the prior art is solved, and the upper layer transmission efficiency is low is easy to be caused.
Optionally, the assembling module 36 in the embodiment of the present application may further include: a determining unit for determining whether additional information associated with the file to be compressed exists; an adding unit configured to add, in the case where there is additional information associated with a file to be compressed, the additional information to the compressed data; and the assembling unit is used for assembling all the compressed data added with the additional information to obtain the compressed file.
Optionally, the first reading module 32 in the embodiment of the present application includes at least one of the following: the first reading unit is used for reading the file to be compressed in the file system according to the preset blocks corresponding to the time periods to obtain block data, wherein different time periods correspond to different preset blocks; the second reading unit is used for reading the file to be compressed in the file system according to the preset blocks corresponding to the data types to obtain block data, wherein the data types of different data in the file to be compressed correspond to different preset blocks.
Corresponding to the above method for decompressing the file in fig. 2, the embodiment of the present application further provides a device for decompressing the file, as shown in fig. 4, where the device includes:
the determining module 42 is configured to determine a first storage space occupied by an original file corresponding to a file to be decompressed, where the original file is a file to be compressed of the file to be decompressed before compression, the file to be decompressed is obtained by assembling a plurality of compressed data, the compressed data is obtained by compressing block data, the block data is obtained by reading the file to be compressed according to a preset block, and a prefix of the compressed data is provided with position metadata for representing the compressed data in the file to be compressed;
a pre-allocation module 44, configured to pre-allocate a second storage space corresponding to the first storage space in the file system;
the decompression module 46 is configured to decompress the file to be decompressed in the second storage space based on the metadata, so as to obtain an original file.
By means of the device, the corresponding second storage space can be pre-allocated in the file system according to the first storage space occupied by the original file corresponding to the file to be decompressed, the second data of the blocks in the file to be decompressed and the corresponding metadata thereof are read, the second data are decompressed to the corresponding positions of the second storage space based on the metadata, so that the final original file is generated, the blocks are compressed and correspond to the blocks, the splitting of the file is achieved in the compression and decompression processes in the block decompression mode, and therefore unordered compression and decompression are achieved, and the compressed file and the original file which are finally combined and generated are not affected.
Based on the above example of the compression process in fig. 1, if the preset block is 20M, reading 20M data from the file to be compressed as the block data; or, if the preset block is 10M, reading 10M data from the file to be compressed as the block data; the decompression is performed according to a preset block, for example, the compressed block data is 20M, the 20M block data is decompressed in the decompression, and if the compressed block data is 10M, the 10M block data is decompressed in the decompression, that is, the inverse process corresponding to the compression process.
Optionally, the decompression module 46 in the embodiment of the present application may further include: a third reading unit for reading the compressed data from the file to be decompressed; the decompression unit is used for decompressing the compressed data; and the writing unit is used for writing the decompressed compressed data into a position corresponding to the metadata in the second storage space to obtain an original file.
Optionally, the apparatus of the embodiment of the present application may further include: and the second reading module is used for reading the additional information related to the file to be compressed from the file to be decompressed before determining the first storage space occupied by the original file corresponding to the file to be decompressed.
In an embodiment of the present application, there is also provided a device for processing a file, including: the compression device of the file in fig. 3 and the decompression device of the file in fig. 4 are specifically shown in fig. 5, where the compression device of the file includes:
the first reading module 32 is configured to read a file to be compressed in the file system according to a preset partition to obtain partition data;
the compression module 34 is configured to compress the block data, prefix metadata on the compressed block data, and obtain compressed data, where the metadata is used to characterize a location of the compressed data in a file to be compressed;
and the assembling module 36 is used for assembling all the compressed data to obtain the compressed file.
The decompression device of the file comprises:
the determining module 42 is configured to determine a first storage space occupied by an original file corresponding to a file to be decompressed, where the original file is a file to be compressed of the file to be decompressed before compression, the file to be decompressed is obtained by assembling a plurality of compressed data, the compressed data is obtained by compressing block data, the block data is obtained by reading the file to be compressed according to a preset block, and a prefix of the compressed data is provided with position metadata for representing the compressed data in the file to be compressed;
a pre-allocation module 44, configured to pre-allocate a second storage space corresponding to the first storage space in the file system;
the decompression module 46 is configured to decompress the file to be decompressed in the second storage space based on the metadata, so as to obtain an original file.
Fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application, and as shown in fig. 6, the embodiment of the present application provides an electronic device including a processor 61, a communication interface 62, a memory 63, and a communication bus 64, where the processor 61, the communication interface 62, and the memory 63 perform communication with each other through the communication bus 64,
a memory 63 for storing a computer program;
The technical effects brought by the implementation of the method steps in fig. 1 or fig. 2 by the processing in the server are consistent with those brought by the implementation of the access control method of the block storage device in fig. 1 or fig. 2 in the above embodiment, and are not described herein again.
The communication bus mentioned by the above terminal may be a peripheral component interconnect standard (Peripheral ComponentInterconnect, abbreviated as PCI) bus or an extended industry standard architecture (ExtendedIndustry StandardArchitecture, abbreviated as EISA) bus, etc. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, only one thick line is shown in fig. 6, but not only one bus or one type of bus.
The communication interface is used for communication between the terminal and other devices.
The memory may include random access memory (RandomAccess Memory, RAM) or non-volatile memory (non-volatile memory), such as at least one disk memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.
The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), etc.; but also digital signal processors (Digital Signal Processing, DSP for short), application specific integrated circuits (Application Specific Integrated Circuit, ASIC for short), field-programmable gate arrays (Field-Programmable Gate Array, FPGA for short) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
The present application also provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method for compressing a file or the steps of the method for decompressing a file provided in any one of the method embodiments described above.
In a further embodiment provided herein, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform the steps of the method of compressing a file or the method of decompressing a file as described in any of the above embodiments.
It should be noted that in this document, relational terms such as "first" and "second" and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The foregoing is only a specific embodiment of the invention to enable those skilled in the art to understand or practice the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (11)
1. A method of compressing a file, comprising:
reading a file to be compressed in a file system according to a preset partition to obtain partition data;
compressing the block data, and prefixing metadata on the compressed block data to obtain compressed data, wherein the metadata are used for representing the position of the compressed data in the file to be compressed;
and assembling all the compressed data to obtain a compressed file.
2. The method of claim 1, wherein assembling all of the compressed data to obtain a compressed file comprises:
determining whether additional information associated with the file to be compressed exists;
and adding the additional information into the compressed data when the additional information associated with the file to be compressed exists, and assembling all the compressed data to which the additional information is added to obtain the compressed file.
3. The method of claim 1, wherein the reading the file to be compressed in the file system according to the preset partition to obtain the partition data includes at least one of:
reading a file to be compressed in a file system according to preset blocks corresponding to time periods to obtain block data, wherein different time periods correspond to different preset blocks;
reading a file to be compressed in a file system according to preset blocks corresponding to data types to obtain block data, wherein the data types of different data in the file to be compressed correspond to different preset blocks.
4. A method for decompressing a file, comprising:
determining a first storage space occupied by an original file corresponding to a file to be decompressed, wherein the original file is a file to be compressed of the file to be decompressed before compression, the file to be decompressed is obtained by assembling a plurality of compressed data, the compressed data are obtained by compressing block data, the block data are obtained by reading the file to be compressed according to preset blocks, and the compressed data prefix is provided with position metadata used for representing the compressed data in the file to be compressed;
pre-distributing a second storage space corresponding to the first storage space in a file system;
and decompressing the file to be decompressed in the second storage space based on the metadata to obtain the original file.
5. The method of claim 4, wherein decompressing the file to be decompressed in the second storage space based on the metadata, the obtaining the original file comprises:
reading the compressed data from the file to be decompressed, and decompressing the compressed data;
and writing the decompressed compressed data into a position corresponding to the metadata in the second storage space to obtain the original file.
6. The method of claim 4, wherein prior to determining the first storage space occupied by the original file corresponding to the file to be decompressed, the method further comprises:
and reading the additional information associated with the file to be compressed from the file to be decompressed.
7. A document compression apparatus, comprising:
the first reading module is used for reading the file to be compressed in the file system according to the preset partitioning to obtain partitioning data;
the compression module is used for compressing the block data, and prefix metadata is arranged on the compressed block data to obtain compressed data, wherein the metadata are used for representing the position of the compressed data in the file to be compressed;
and the assembly module is used for assembling all the compressed data to obtain a compressed file.
8. A decompression apparatus for a document, comprising:
the device comprises a determining module, a compressing module and a judging module, wherein the determining module is used for determining a first storage space occupied by an original file corresponding to a file to be decompressed, wherein the original file is a file to be compressed of the file to be decompressed before compression, the file to be decompressed is obtained by assembling a plurality of compressed data, the compressed data is obtained by compressing block data, the block data is obtained by reading the file to be compressed according to a preset block, and a compressed data prefix is provided with position metadata used for representing the compressed data in the file to be compressed;
the pre-allocation module is used for pre-allocating a second storage space corresponding to the first storage space in the file system;
and the decompression module is used for decompressing the file to be decompressed in the second storage space based on the metadata to obtain the original file.
9. A document processing apparatus, comprising: a compression device of a document according to claim 7 and a decompression device of a document according to claim 8.
10. The electronic equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;
a memory for storing a computer program;
a processor for implementing the steps of the method of any one of claims 1-3 or the steps of the method of any one of claims 4-6 when executing a program stored on a memory.
11. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1-3 or the steps of the method of any of claims 4-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210002255.6A CN116431585A (en) | 2022-01-04 | 2022-01-04 | File compression method and device, and file decompression method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210002255.6A CN116431585A (en) | 2022-01-04 | 2022-01-04 | File compression method and device, and file decompression method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116431585A true CN116431585A (en) | 2023-07-14 |
Family
ID=87085975
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210002255.6A Pending CN116431585A (en) | 2022-01-04 | 2022-01-04 | File compression method and device, and file decompression method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116431585A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117312261A (en) * | 2023-11-29 | 2023-12-29 | 苏州元脑智能科技有限公司 | File compression encoding method, device storage medium and electronic equipment |
-
2022
- 2022-01-04 CN CN202210002255.6A patent/CN116431585A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117312261A (en) * | 2023-11-29 | 2023-12-29 | 苏州元脑智能科技有限公司 | File compression encoding method, device storage medium and electronic equipment |
| CN117312261B (en) * | 2023-11-29 | 2024-02-09 | 苏州元脑智能科技有限公司 | File compression encoding method, device storage medium and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108738006B (en) | Data transmission method and device based on Bluetooth | |
| CN110297944B (en) | Distributed XML data processing method and system | |
| CN108197324B (en) | Method and apparatus for storing data | |
| CN108053034B (en) | Model parameter processing method and device, electronic equipment and storage medium | |
| CN106685429B (en) | Integer compression method and device | |
| CN109062906B (en) | Translation method and device for program language resources | |
| CN114020272A (en) | Serialized encoding and decoding methods and devices and electronic equipment | |
| CN116431585A (en) | File compression method and device, and file decompression method and device | |
| CN115456781A (en) | Market data processing method and device, electronic equipment and storage medium | |
| CN110019347B (en) | Data processing method and device of block chain and terminal equipment | |
| CN106293542B (en) | Method and device for decompressing file | |
| CN111210826B (en) | Voice information processing method and device, storage medium and intelligent terminal | |
| CN112286706A (en) | Remote and rapid acquisition method for application information of android application and related equipment | |
| CN111857868A (en) | Security data reading and writing method, device, equipment and computer readable storage medium | |
| CN116865766A (en) | Waveform data compression method, system, equipment and medium | |
| CN106790466B (en) | Method, server and terminal equipment for pushing | |
| CN113641643A (en) | File writing method and device | |
| CN114791904A (en) | Persistent compression method and device for bloom filter | |
| CN103929404B (en) | Method for analyzing HTTP chunked code data | |
| CN109889608B (en) | Dynamic resource loading method and device, electronic equipment and storage medium | |
| CN112423104A (en) | Audio mixing method and system for multi-channel audio in live scene | |
| CN112100453B (en) | Character string distribution statistical method, system, equipment and computer storage medium | |
| CN109756550B (en) | Intelligent internet driving vehicle cloud communication method, device and system | |
| CN107689996B (en) | Data transmission method and device and terminal equipment | |
| CN113923209B (en) | Processing method for downloading batch data based on LevelDB |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |