CN109067405B - Data compression method, device, terminal and computer readable storage medium - Google Patents
Data compression method, device, terminal and computer readable storage medium Download PDFInfo
- Publication number
- CN109067405B CN109067405B CN201810844131.6A CN201810844131A CN109067405B CN 109067405 B CN109067405 B CN 109067405B CN 201810844131 A CN201810844131 A CN 201810844131A CN 109067405 B CN109067405 B CN 109067405B
- Authority
- CN
- China
- Prior art keywords
- data
- byte string
- compressed
- length
- mark byte
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
- H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
- H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
- H03M7/40—Conversion to or from variable length codes, e.g. Shannon-Fano code, Huffman code, Morse code
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
The application relates to the technical field of data compression, and discloses a data compression method, which comprises the following steps: for currently read-in 2 N Carrying out integer wavelet transformation on data to be compressed to obtain wavelet coefficients; the data to be compressed is data of which the data volatility is smaller than a threshold value; judging whether the wavelet coefficient is larger than a fluctuation threshold value; if the number of the wavelet coefficients is larger than the preset number, the wavelet coefficients are added to the tail of the compressed packet, the length of the coding mark byte string and the index bit are updated according to the byte length of the compressed packet, and the first preset numerical value is written into the corresponding bits in the coding mark byte string according to the updated length of the coding mark byte string and the updated index bit; and if the number of the bits in the code mark byte string is not larger than the preset number, adding the Huffman Hoffmann code corresponding to the wavelet coefficient to the tail of the compressed packet, updating the length of the code mark byte string and the index bit according to the length of the bytes of the compressed packet, and writing the second preset number into corresponding bits in the code mark byte string according to the updated length of the code mark byte string and the updated index bit. The method can realize data compression under an ultra-low computing capability system; the application also discloses a data compression device, a terminal and a computer readable storage medium, which have the beneficial effects.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a method and an apparatus for data compression, a terminal, and a computer-readable storage medium.
Background
Currently, most of the data compression algorithms use the principle of entropy coding and its variants, such as huffman coding, shannon coding, arithmetic compression algorithm. Or an algorithm that employs repetitive structure compression, such as the LZ77 compression algorithm. Wherein, a compression algorithm adopting an entropy coding principle needs to count data and construct a dictionary; the algorithm adopting the repetitive structure compression has high corresponding space complexity due to the logic complexity. The entropy coding is coding without losing any information according to the entropy principle in the coding process.
It can be seen that although the compression methods described above have a good compression effect, they have a high requirement on the computational performance of the processor, and therefore, most of them are used in systems with strong processor performance, such as PCs, servers, smartphones, cameras, and the like. In a device with only a microcontroller, the microcontroller has weak computing power and extremely limited memory space, so that the compression algorithm cannot be executed.
Disclosure of Invention
The application aims to provide a data compression method, a data compression device, a data compression terminal and a computer readable storage medium, which can realize data compression under an ultra-low computing capacity system.
In order to solve the above technical problem, the present application provides a method for data compression, where the method includes:
for currently read-in 2 N Carrying out integer wavelet transformation on data to be compressed to obtain wavelet coefficients;
judging whether the wavelet coefficient is larger than a fluctuation threshold value;
if the number of the wavelet coefficients is larger than the preset number, the wavelet coefficients are added to the tail of the compressed packet, and corresponding bits in the coding mark byte string are written with first preset numerical values;
and if the number is not larger than the preset value, adding the Huffman code corresponding to the wavelet coefficient to the tail of the compressed packet, and writing a second preset value into each corresponding bit in the code mark byte string.
Optionally, writing a first preset value into each corresponding bit in the encoding flag byte string includes:
updating the length of the coding mark byte string and the index bit according to the byte length of the compressed packet, and writing the first preset numerical value into corresponding bits in the coding mark byte string according to the updated length of the coding mark byte string and the index bit;
writing a second preset numerical value into each corresponding bit in the coding mark byte string, wherein the second preset numerical value comprises the following steps:
and updating the length of the coding mark byte string and the index bit according to the byte length of the compressed packet, and writing the second preset numerical value into corresponding bits in the coding mark byte string according to the updated length of the coding mark byte string and the index bit.
Optionally, for currently read-in 2 N Before integer wavelet transform is performed on the data to be compressed to obtain wavelet coefficients, the method further comprises the following steps:
initializing a compression header file; the compression header file comprises a Hoffmann tree and the coding mark byte string which are stored in a ROM, and the length of the coding mark byte string and the index bit which are stored in a memory.
Optionally, before initializing the compression header file, the method further includes:
and adding a data compression task, and setting the priority of the data compression task to be the lowest.
Optionally, before determining whether the wavelet coefficient is greater than a fluctuation threshold, the method further includes:
acquiring a preset number of sample data to be compressed;
determining the difference value between the maximum value and the minimum value of the sample data to be compressed;
and selecting a preset multiple of the difference value as the fluctuation threshold value.
The present application also discloses a data compression apparatus, comprising:
wavelet transform module for reading in currently read 2 N Carrying out integer wavelet transformation on data to be compressed to obtain wavelet coefficients;
the judging module is used for judging whether the wavelet coefficient is larger than a fluctuation threshold value or not;
the first execution module is used for adding the wavelet coefficient to the tail of the compressed packet if the wavelet coefficient is larger than the fluctuation threshold value, and writing a first preset numerical value into each corresponding bit in the coding mark byte string;
and the second execution module is used for adding the Huffman code corresponding to the wavelet coefficient to the tail part of the compressed packet if the fluctuation threshold value is not larger than the fluctuation threshold value, and writing a second preset numerical value into each corresponding bit in the code mark byte string.
Optionally, the apparatus further comprises:
the initialization module is used for initializing the compression header file; the compression header file comprises a Hoffmann tree and the coding mark byte string which are stored in a Read Only Memory (ROM), and the length of the coding mark byte string and the index bit which are stored in a memory.
Optionally, the apparatus further comprises:
the device comprises a fluctuation threshold value determining module, a compressing module and a compressing module, wherein the fluctuation threshold value determining module is used for acquiring sample data to be compressed in a preset quantity; determining the difference value between the maximum value and the minimum value of the sample data to be compressed; and selecting a preset multiple of the difference value as the fluctuation threshold value.
The application also discloses a terminal, including:
a memory for storing a computer program;
a processor for implementing the steps of the method of data compression as described above when executing the computer program. The present application also discloses a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the method of data compression as described above.
The data compression method provided by the application comprises the following steps: for currently read-in 2 N Carrying out integer wavelet transformation on data to be compressed to obtain wavelet coefficients; judging whether the wavelet coefficient is larger than a fluctuation threshold value or not; if the number of the bits is larger than the preset number, adding the wavelet coefficient to the tail of the compressed packet, and writing a first preset numerical value into corresponding bits in the coding mark byte string; and if the number of the bits is not larger than the preset value, adding Hoffmann codes corresponding to the wavelet coefficients to the tail of the compressed packet, and writing second preset values into corresponding bits in the code mark byte string.
Therefore, the method reduces the calculation complexity (wherein the calculation complexity of the wavelet transform and the Hoffmann code is smaller) by combining the Hoffmann code and the wavelet transform, and realizes data compression under an ultra-low computing capability system; the data to be compressed are distinguished and treated through a fluctuation threshold value, so that the compression ratio is improved; the application also provides a data compression device, a terminal and a computer readable storage medium, which have the beneficial effects and are not described herein again.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a flow chart of a method of data compression provided by an embodiment of the present application;
fig. 2 is a block diagram of a device for data compression according to an embodiment of the present disclosure.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making any creative effort belong to the protection scope of the present application.
In the prior art, both the method of compressing data by adopting an entropy coding principle and the method of compressing a repetitive structure are high in corresponding computational complexity, so that the methods can only be applied to processors with strong performance. Whereas in the case of a device having only a microprocessor, the above-described compression algorithm cannot be implemented. The data compression method provided by the embodiment has very low computational complexity and high compression ratio, and can realize data compression under an ultra-low computing capability system. Referring to fig. 1 in particular, fig. 1 is a flowchart of a data compression method according to an embodiment of the present disclosure; the method can comprise the following steps:
s100, for currently read 2 N And carrying out integer wavelet transformation on the data to be compressed to obtain a wavelet coefficient.
The present embodiment does not limit the type of the data to be compressed, and may be, for example, motion data, electrocardiographic data, image data, or the like. In order to increase the data compression ratio, the present embodiment may preferably select data with less fluctuation. Namely, when the fluctuation of the data needing to be compressed is smaller than the threshold value, the data is taken as the data to be compressed. It can be understood that the amount of small amplitude variation in the data is large, and the amount of large amplitude variation is small. For example, heart rate values acquired by a heart rate sensor, there may be small amplitude fluctuations within an interval, but heart rate values outside this interval are less likely to occur. I.e., the present embodiment has a very small compression rate for such data (e.g., sensor data).
Specifically, the main purpose of this step is to perform integer wavelet transform on currently acquired data to be compressed to obtain wavelet coefficients. That is, when compressing data to be compressed, the present embodiment needs to perform data compression on the data to be compressed in batches according to the number of the data to be compressed. E.g. 2, read in sequence for data to be compressed according to signal source N The data to be compressed is subjected to integer wavelet transform (i.e. each time 2 is read in) N One data to be compressed). This step is for the currently read-in 2 N And carrying out integer wavelet transformation on the data to be compressed. In this embodiment, the value of N is not limited, and may be determined according to the hardware computing capability. Because the computing power of the microprocessor is limited, if the value of N is too large, the computing complexity is increased, and the effect of data compression under an ultra-low computing power system is influenced; if the range of N is too small, the number of computations for the data to be compressed increases. Therefore, in order to ensure the calculation effect, it is preferable that N in the present embodiment may range from 8 to 10. Further, when the hardware processing capacity is low, in order to reduce the occupation of the processor memory by the data, the current 2 can be directly read from the ROM N And carrying out integer wavelet transformation on the data to be compressed. When the wavelet coefficient obtained currently is determined to be completely processed (namely the wavelet coefficient obtained currently is obtained currently)The number is all executed to the subsequent operation), the next 2 can be read in sequence N And carrying out integer wavelet transformation on the data to be compressed.
The wavelet coefficient is the coefficient of the wavelet basis function similar to the original signal. The wavelet transform is computationally inexpensive. The embodiment does not limit the specific integer wavelet transform process, and specifically refers to the calculation process of the integer wavelet transform in the prior art.
And S110, judging whether the wavelet coefficient is larger than a fluctuation threshold value.
The fluctuation threshold is used for distinguishing data to be compressed, and different compression actions are executed according to the distinguishing result. When the wavelet coefficient is larger than the fluctuation threshold, the data to be compressed corresponding to the wavelet coefficient does not need to be coded (namely, hoffmann coding is not needed), and when the wavelet coefficient is not larger than the fluctuation threshold, the data to be compressed corresponding to the wavelet coefficient needs to be coded (namely, hoffmann coding is needed). The fluctuation threshold is therefore related to the compression ratio of the data to be compressed.
Specifically, the specific value of the fluctuation threshold is not limited in this embodiment. For example, a user can set a fluctuation threshold value according to the fluctuation characteristics and experience of the data to be compressed; the user may also calculate the fluctuation threshold value through the fluctuation of sample data (which may be referred to as sample data to be compressed) corresponding to the data to be compressed.
Further, in order to increase the compression ratio, preferably, the method for determining the fluctuation threshold in this embodiment may include:
acquiring a preset amount of sample data to be compressed;
determining the difference value between the maximum value and the minimum value of the sample data to be compressed;
and selecting a preset multiple of the difference value as a fluctuation threshold value.
Specifically, the present embodiment does not limit the preset number. In general, the larger the preset number is, the higher the compression ratio corresponding to the calculated fluctuation threshold value is. The user can determine the corresponding numerical value of the preset quantity according to the type of the actual sample data to be compressed. The present embodiment also does not limit the specific value of the preset multiple, and the preset multiple is a value smaller than 1 in general. For example, the preset multiple has a value of 0.1.
In this embodiment, to improve the calculation efficiency, steps S110 to S130 may be performed once after obtaining a wavelet coefficient until current 2 N After all the wavelet coefficients obtained from the data to be compressed are executed in steps S110 to S130, step S100 can be executed again.
S120, if the number of the bits is larger than the preset number, adding the wavelet coefficient to the tail of the compressed packet, and writing a first preset numerical value into corresponding bits in the coding mark byte string; the bit of the coding mark byte string is sequentially corresponding to the byte strings in the compressed packet, and the first preset value represents that the byte strings in the compressed packet corresponding to the bit are uncoded data.
S130, if the number of the bits is not larger than the preset number, adding the Huffman Hoffmann codes corresponding to the wavelet coefficients to the tail of the compressed packet, and writing second preset numerical values into corresponding bits in the code mark byte string; and the second preset value represents that the byte string in the compressed packet corresponding to the bit is coded data.
In this embodiment, the Hoffmann code corresponding to the wavelet coefficient may be obtained from the Hoffmann number, that is, the Hoffmann code corresponding to the wavelet coefficient is found from the Hoffmann number. The Hoffmann coding algorithm does not need a large model, does not need iterative computation and the like, and is an algorithm with small computation amount. The Hoffmann tree (huffman tree) is a tree with the shortest weighted path length, and the nodes with larger weights are closer to the root. The setting of the Hoffmann tree can be determined according to the fluctuation threshold value and the kind of data to be compressed. This embodiment does not limit this.
In this embodiment, when the wavelet coefficient is greater than the fluctuation threshold, it indicates that the corresponding data has a sudden change, and the frequency of such data is very low, so that the corresponding Hoffmann code is very long, and therefore, in order to improve the calculation efficiency and reduce the calculation complexity, in this case, the wavelet coefficient may be directly added to the tail of the compression packet, and the Hoffmann code is not required. When the wavelet coefficient is not greater than the fluctuation threshold, it indicates that the corresponding data is very stable, and the frequency of occurrence of such data is very high, so the corresponding Hoffmann code is very short, and therefore, in order to improve the compression ratio under the condition of very small computational complexity, in this embodiment, the Hoffmann code corresponding to the wavelet coefficient may be added to the tail of the compression packet, and at this time, the Hoffmann code corresponding to the wavelet coefficient needs to be obtained.
In this embodiment, specific values of the first preset value and the second preset value are not limited, as long as whether the byte string in the compressed packet corresponding to each bit is encoded (that is, hoffmann encoding) can be determined by the value recorded on each bit in the encoding flag byte string. For example, when a value recorded on a bit in the encoding flag byte string is a first preset value, it is determined that the byte string in the compressed packet corresponding to the bit is data that is not encoded (i.e., hoffmann encoded). When the value recorded on a certain bit in the coding mark byte string is a second preset value, determining that the byte string in the compressed packet corresponding to the bit is data subjected to coding (namely Hoffmann coding).
Further, in order to improve the writing efficiency, and reduce the occupation of hardware storage resources as much as possible. Preferably, the present embodiment may record through a binary coded flag byte string (i.e. 01 string), specifically, the first preset value is 1 and the second preset value is 0, or the first preset value is 0 and the second preset value is 1. This results in a string of coded flag bytes of only 0 and 1.
Specifically, in this embodiment, the bits in the encoding flag byte string sequentially correspond to the byte strings in the compressed packet. I.e. one bit in the encoding flag byte string corresponds to one byte string in the compressed packet. For example, the first string of bytes in the compressed packet corresponds to the first bit in the encoding marker string of bytes. Thus, the user can determine which specific byte string in the compressed packet corresponds to each bit in the encoding flag byte string during decompression.
The present embodiment does not specifically limit how the bits in the encoding flag byte string are sequentially associated with the byte strings in the compressed packet. The bit in the encoding marker byte string and the byte string in the compressed packet can be recorded in sequence. For example, the corresponding byte length variation n after adding data to the current compressed packet may be determined (for example, the difference obtained by subtracting the length of the previous compressed packet from the length of the current compressed packet is divided by 8 to obtain the byte length variation n of the compressed packet), then n bits are added to the encoding flag byte string at the last recording position, and a corresponding value (a first preset value or a second preset value) is recorded at each added bit. Or updating the length of the encoding flag byte string (used for recording the current length of the encoding flag byte string) and the index bit (indicating that the last byte in the compressed packet uses the number of bits) according to the length of the current compressed packet data, wherein the corresponding updating mode can be that the integer part obtained by dividing the length of the compressed packet data by 8 is used as the number of bits in the encoding flag byte string (namely the number of bits in the encoding flag byte string), and the remainder is the numerical value of the index bit; and the position of the bit which needs to be recorded in the coding mark byte string at present can be determined according to the updated (namely the current) coding mark byte string length and the index bit, and then a corresponding numerical value (a first preset numerical value or a second preset numerical value) is recorded in the position.
Preferably, in order to improve the decompression reliability, in this embodiment, writing the first preset value in each corresponding bit of the encoding flag byte string may include:
updating the length of the coding mark byte string and the index bit according to the length of the bytes of the compressed packet, and writing a first preset numerical value in the corresponding bits of the coding mark byte string according to the updated length of the coding mark byte string and the index bit;
writing the second preset value in each corresponding bit of the encoding flag byte string may include:
and updating the length of the coding mark byte string and the index bit according to the length of the bytes of the compressed packet, and writing a second preset numerical value in the corresponding bits of the coding mark byte string according to the updated length of the coding mark byte string and the updated index bit.
Specifically, the specific position of the bit in the coded mark byte string to which the value needs to be written is determined according to the updated length of the coded mark byte string and the index bit, and then the required value (i.e., the first preset value or the second preset value) is written in the position. When a byte string has eight bits, the index bit is an index of 0 to 7, the 0 th bit is the lowest bit, and the 7 th bit is the highest bit.
Based on the technical scheme, the method for data compression provided by the embodiment of the application reduces the calculation complexity in a mode of combining Hoffmann coding and wavelet transformation, and realizes data compression under an ultra-low computing power system; and the data to be compressed are distinguished and treated through the fluctuation threshold value, so that the compression ratio is improved.
Further, before data compression, initialization of a compression header file is required, such as obtaining an initial Hoffmann tree and an encoding flag byte string. The embodiment does not limit the specific way of initializing the compression header file, nor the content in the compression header file. The user can determine the content of the compressed header file according to actual needs and perform corresponding initialization. Based on the foregoing embodiments, in order to further improve the utilization rate of hardware resources in this embodiment, when the compression header file includes the Hoffmann tree, the encoding flag byte string length, and the index bit, it is preferable that the Hoffmann tree and the encoding flag byte string included in the compression header file are stored in the ROM, and the encoding flag byte string length and the index bit are stored in the memory.
Among them, ROM is a short name of Read-only memory (ROM), and is a solid-state semiconductor memory capable of reading only data stored in advance. The property is that once the data is stored, it cannot be changed or deleted. It is commonly used in electronic or computer systems where the data is not required to be changed frequently and the data is not lost due to power down. A Memory (Memory) is also called an internal Memory and is used to temporarily store operation data in a CPU and data exchanged with an external Memory such as a hard disk. As long as the computer is in operation, the CPU transfers data to be operated to the memory for operation, and after the operation is completed, the CPU transmits the result out, and the operation of the memory also determines the stable operation of the computer.
Specifically, the Hoffmann tree and the encoding flag byte string are very important, and the Hoffmann tree does not need to be changed basically, so the Hoffmann tree and the encoding flag byte string are placed in the ROM, and the length of the encoding flag byte string and the index bit need to be updated and used frequently in the operation process, so the length of the encoding flag byte string and the index bit are placed in a memory with higher operation speed. Wherein, the length of the encoding mark byte string and the index bit can be realized by two registers.
Further, when the data compression method in this embodiment is applied to an ultra-low computing capability system, in order to ensure normal operation of the ultra-low computing capability system, in this embodiment, a data compression task may be added to the ultra-low computing capability system, and the priority of the data compression task is set to be the lowest.
Specifically, firstly, the chip (e.g., microprocessor) in the ultra-low computing power system is a chip with low power consumption and poor computing power, and the content with high priority in the ultra-low computing power system is more, and data compression is relatively not urgent, so that the chip can be set to have the lowest priority. Further, since it is necessary to perform operations such as data buffering when data compression is performed, it is also necessary to wait for a certain time, and it is not necessary to set the priority level high.
The following illustrates the data compression process under the ultra-low computing power system:
1. adding a data compression task and setting the priority of the data compression task to be the lowest.
2. Initializing a compressed file header: a Hoffmann tree is positioned in the ROM, and an encoding mark byte string is positioned in the ROM; an encoding tag byte string length and bit index are located in memory.
3. Sequentially fetching current 2 from ROM for signal source of data to be compressed N And carrying out integer wavelet transformation on the data to be compressed to obtain a wavelet coefficient.
4. And judging whether the wavelet coefficient is larger than a fluctuation threshold value or not.
5. If the wavelet coefficient is larger than the fluctuation threshold, adding the wavelet coefficient to the tail of the compressed packet, and marking in a coding mark byte string: sequentially adding N bits (called updating bits) in the coding mark byte string, setting 1,N as the byte length change number of the last compressed packet, if the updating bits reach the lowest bit of the used byte, dynamically increasing the length of the byte string, and updating the counting number of the length of the coding mark byte string.
6. If the wavelet coefficient is not larger than the fluctuation threshold, adding Hoffmann codes corresponding to the wavelet coefficients to the tail of the compressed packet, and marking in a code marking byte string: and sequentially increasing N bits in the coding mark byte string and setting the N bits as 0, wherein N is the length change number of the latest compressed packet.
The following describes apparatuses, terminals and computer-readable storage media for data compression according to embodiments of the present application, and the apparatuses, terminals and computer-readable storage media for data compression described below and the methods for data compression described above may be referred to correspondingly.
Referring to fig. 2, fig. 2 is a block diagram illustrating a data compression apparatus according to an embodiment of the present disclosure; the apparatus may include:
a judging module 200, configured to judge whether the wavelet coefficient is greater than a fluctuation threshold;
a first execution module 300, configured to add a wavelet coefficient to a tail of the compressed packet if the wavelet coefficient is greater than the fluctuation threshold, and write a first preset value into each corresponding bit in the coding flag byte string; the bit in the coding mark byte string sequentially corresponds to the byte string in the compressed packet, and the first preset value indicates that the byte string in the compressed packet corresponding to the bit is uncoded data;
a second execution module 400, configured to add the huffman Hoffmann code corresponding to the wavelet coefficient to the tail of the compressed packet if the huffman Hoffmann code is not greater than the fluctuation threshold, and write a second preset value into corresponding bits in the code tag byte string; and the second preset value represents that the byte string in the compressed packet corresponding to the bit is coded data.
Based on the above embodiments, the first execution module 300 may include:
the first writing unit is used for updating the length of the coding mark byte string and the index bit according to the byte length of the compressed packet and writing the first preset numerical value into corresponding bits in the coding mark byte string according to the updated length of the coding mark byte string and the updated index bit;
the second execution module 400 may include:
and the second writing unit is used for updating the length of the coding mark byte string and the index bit according to the byte length of the compressed packet and writing the second preset numerical value into corresponding bits in the coding mark byte string according to the updated length of the coding mark byte string and the updated index bit.
Based on the above embodiment, the apparatus may further include:
the initialization module is used for initializing the compression header file; the compression header file comprises a Hoffmann tree and the coding mark byte string which are stored in a ROM, and the length of the coding mark byte string and the index bit which are stored in a memory.
Based on any of the above embodiments, the apparatus may further include:
and the priority setting module is used for adding a data compression task and setting the priority of the data compression task to be the lowest.
Based on any of the above embodiments, the apparatus may further include:
the device comprises a fluctuation threshold value determining module, a compressing module and a compressing module, wherein the fluctuation threshold value determining module is used for acquiring sample data to be compressed in a preset quantity; determining the difference value between the maximum value and the minimum value of the sample data to be compressed; and selecting a preset multiple of the difference value as a fluctuation threshold value.
It should be noted that, based on any of the above embodiments, the apparatus may be implemented based on a programmable logic device, where the programmable logic device includes an FPGA, a CPLD, a single chip, and the like. These programmable logic devices may be provided in a microprocessor.
An embodiment of the present application further provides a terminal, including: a memory for storing a computer program; a processor for implementing the steps of the method of data compression as described in any of the embodiments above when the computer program is executed. E.g. when the processor is used for executing a computer programRealizing current read-in 2 N Carrying out integer wavelet transformation on data to be compressed to obtain wavelet coefficients; judging whether the wavelet coefficient is larger than a fluctuation threshold value; if the number of the bits is larger than the preset number, adding the wavelet coefficient to the tail of the compressed packet, and writing a first preset numerical value into corresponding bits in the coding mark byte string; the bit in the coding mark byte string sequentially corresponds to the byte string in the compressed packet, and the first preset value indicates that the byte string in the compressed packet corresponding to the bit is uncoded data; if the number is not larger than the preset value, adding Hoffmann codes corresponding to the wavelet coefficients to the tail of the compressed packet, and writing second preset values into corresponding bits in the code mark byte string; and the second preset value represents that the byte string in the compressed packet corresponding to the bit is coded data.
Specifically, the processor in this embodiment may be a microprocessor, and may also be a processor with higher performance used in a device such as a PC, a server, a smartphone, and a camera.
Based on the above embodiment, the terminal is specifically a wearable device. Such as smart watches, smart bracelets, smart hats, smart headsets, etc.
The present application further provides a computer-readable storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements the steps of the method for data compression according to any of the above embodiments. Implementing a currently read-in 2 if the computer program is executed by a processor N Carrying out integer wavelet transformation on data to be compressed to obtain wavelet coefficients; judging whether the wavelet coefficient is larger than a fluctuation threshold value; if the number of the bits is larger than the preset number, adding the wavelet coefficient to the tail of the compressed packet, and writing a first preset numerical value into corresponding bits in the coding mark byte string; the bit in the coding mark byte string sequentially corresponds to the byte string in the compressed packet, and the first preset value indicates that the byte string in the compressed packet corresponding to the bit is uncoded data; if the number of the bits is not larger than the preset number, adding Hoffmann codes corresponding to wavelet coefficients to the tail of the compressed packet, and writing corresponding bits in a code mark byte string into a second preset numerical value; and the second preset value represents that the byte string in the compressed packet corresponding to the bit is coded data.
The computer-readable storage medium may include: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
Those of skill would further appreciate that the various illustrative components and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the components and steps of the various examples have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The above detailed description is provided for a data compression apparatus, a terminal and a computer readable storage medium. The principles and embodiments of the present application are described herein using specific examples, which are only used to help understand the method and its core idea of the present application. It should be noted that, for those skilled in the art, it is possible to make several improvements and modifications to the present application without departing from the principle of the present application, and such improvements and modifications also fall within the scope of the claims of the present application.
Claims (9)
1. A method of data compression, the method comprising:
for currently read-in 2 N Carrying out integer wavelet transformation on data to be compressed to obtain wavelet coefficients; the data to be compressed is data of which the data volatility is smaller than a threshold value;
judging whether the wavelet coefficient is larger than a fluctuation threshold value;
if the wavelet coefficient is larger than the preset value, adding the wavelet coefficient to the tail of a compressed packet, updating the length of a coding mark byte string and an index bit according to the length of the bytes of the compressed packet, and writing a first preset value into corresponding bits in the coding mark byte string according to the updated length of the coding mark byte string and the updated index bit;
and if the number of the bits is not larger than the preset value, adding the Huffman Hoffmann code corresponding to the wavelet coefficient to the tail of the compressed packet, updating the length of the code mark byte string and the index bit according to the length of the bytes of the compressed packet, and writing a second preset value into corresponding bits in the code mark byte string according to the updated length of the code mark byte string and the updated index bit.
2. Method according to claim 1, characterized in that 2 for the current reading in N Before integer wavelet transform is performed on the data to be compressed to obtain wavelet coefficients, the method further comprises the following steps:
initializing a compression header file; the compression header file comprises a Hoffmann tree and the coding mark byte string which are stored in a ROM, and the length of the coding mark byte string and the index bit which are stored in a memory.
3. The method of claim 2, wherein prior to initializing the compression header file, further comprising:
and adding a data compression task, and setting the priority of the data compression task to be the lowest.
4. The method according to any one of claims 1-3, wherein before said determining whether said wavelet coefficients are greater than a fluctuation threshold, said method further comprises:
acquiring a preset amount of sample data to be compressed;
determining the difference value between the maximum value and the minimum value of the sample data to be compressed;
and selecting a preset multiple of the difference value as the fluctuation threshold value.
5. An apparatus for data compression, comprising:
wavelet transform module for reading in currently read-in 2 N Carrying out integer wavelet transformation on data to be compressed to obtain wavelet coefficients; the data to be compressed is data of which the data volatility is smaller than a threshold value;
the judging module is used for judging whether the wavelet coefficient is larger than a fluctuation threshold value or not;
the first execution module is used for adding the wavelet coefficient to the tail of a compressed packet if the wavelet coefficient is larger than the fluctuation threshold, updating the length of a coding mark byte string and an index bit according to the length of the byte of the compressed packet, and writing a first preset value into corresponding bits in the coding mark byte string according to the updated length of the coding mark byte string and the updated index bit;
and the second execution module is used for adding the Huffman code corresponding to the wavelet coefficient to the tail part of the compressed packet if the length of the compressed packet is not greater than the fluctuation threshold, updating the length of the coding mark byte string and the index bit according to the length of the byte of the compressed packet, and writing a second preset numerical value into corresponding bits in the coding mark byte string according to the updated length of the coding mark byte string and the index bit.
6. The apparatus of claim 5, further comprising:
the initialization module is used for initializing the compression header file; the compression header file comprises a Hoffmann tree and the coding mark byte string which are stored in a ROM, and the length of the coding mark byte string and the index bit which are stored in a memory.
7. The apparatus of claim 5 or 6, further comprising:
the device comprises a fluctuation threshold value determining module, a compressing module and a compressing module, wherein the fluctuation threshold value determining module is used for acquiring sample data to be compressed in a preset quantity; determining the difference value between the maximum value and the minimum value of the sample data to be compressed; and selecting a preset multiple of the difference value as the fluctuation threshold value.
8. A terminal, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the method of data compression as claimed in any one of claims 1 to 4 when executing the computer program.
9. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, which computer program, when being executed by a processor, carries out the steps of the method of data compression according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810844131.6A CN109067405B (en) | 2018-07-27 | 2018-07-27 | Data compression method, device, terminal and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810844131.6A CN109067405B (en) | 2018-07-27 | 2018-07-27 | Data compression method, device, terminal and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109067405A CN109067405A (en) | 2018-12-21 |
| CN109067405B true CN109067405B (en) | 2022-10-11 |
Family
ID=64835927
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810844131.6A Active CN109067405B (en) | 2018-07-27 | 2018-07-27 | Data compression method, device, terminal and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109067405B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110113618B (en) * | 2019-06-11 | 2021-09-03 | 苏州泓迅生物科技股份有限公司 | Image storage method, reading method, storage device and reading device |
| CN110446124B (en) * | 2019-08-19 | 2021-10-12 | 深圳市双翼科技股份有限公司 | Unit remote management method, storage medium and device based on optical network terminal |
| CN111181568A (en) * | 2020-01-10 | 2020-05-19 | 深圳花果公社商业服务有限公司 | Data compression device and method, data decompression device and method |
| CN112650751A (en) * | 2020-04-30 | 2021-04-13 | 北京金风慧能技术有限公司 | Data processing method and device, electronic equipment and data processing system |
| CN113542225B (en) * | 2021-06-17 | 2023-08-22 | 深圳市合广测控技术有限公司 | Data compression method and device, terminal equipment and storage medium |
| CN113672575A (en) * | 2021-07-16 | 2021-11-19 | 深圳智慧林网络科技有限公司 | Data compression method and device and storage medium |
| CN113630124B (en) * | 2021-08-10 | 2023-08-08 | 优刻得科技股份有限公司 | Method, system, equipment and medium for processing time sequence integer data |
| CN115173865B (en) * | 2022-03-04 | 2023-04-25 | 上海玫克生储能科技有限公司 | Battery data compression processing method for energy storage power station and electronic equipment |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5748786A (en) * | 1994-09-21 | 1998-05-05 | Ricoh Company, Ltd. | Apparatus for compression using reversible embedded wavelets |
| US5836003A (en) * | 1993-08-26 | 1998-11-10 | Visnet Ltd. | Methods and means for image and voice compression |
| CN1322442A (en) * | 1999-07-20 | 2001-11-14 | 皇家菲利浦电子有限公司 | Encoding method for compression of video sequence |
| CN1438767A (en) * | 2002-02-16 | 2003-08-27 | 三星电子株式会社 | Method and apparatus for changing compressed audio frequency signal using small wave packet |
| CN1471312A (en) * | 2002-07-26 | 2004-01-28 | 包头钢铁学院 | Selective medical image compression method |
| CN101038303A (en) * | 2007-02-07 | 2007-09-19 | 深圳市双合电脑系统股份有限公司 | Compression method for electric power waveform recording date |
| CN101924562A (en) * | 2010-06-24 | 2010-12-22 | 北京师范大学 | Compression-type coding scheme of curve vector data based on integer wavelet transformation |
| CN101984666A (en) * | 2010-11-19 | 2011-03-09 | 南京邮电大学 | Image lossless compression and decompression method based on lifting wavelet transform |
| CN105187071A (en) * | 2015-09-29 | 2015-12-23 | 东北大学 | Pipeline millisecond online data compound compression/decompression method |
| CN108322220A (en) * | 2018-02-08 | 2018-07-24 | 华为技术有限公司 | Decoding method, device and coding/decoding apparatus |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6873734B1 (en) * | 1994-09-21 | 2005-03-29 | Ricoh Company Ltd | Method and apparatus for compression using reversible wavelet transforms and an embedded codestream |
| CN101488214B (en) * | 2008-01-15 | 2011-07-27 | 中国科学院软件研究所 | Method for expanding anti-fake digital watermark and electronic seal information content |
| CN101448158B (en) * | 2008-12-19 | 2012-03-14 | 中国海洋大学 | Underwater video compression method |
| US9252807B2 (en) * | 2013-10-21 | 2016-02-02 | Globalfoundries Inc. | Efficient one-pass cache-aware compression |
-
2018
- 2018-07-27 CN CN201810844131.6A patent/CN109067405B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5836003A (en) * | 1993-08-26 | 1998-11-10 | Visnet Ltd. | Methods and means for image and voice compression |
| US5748786A (en) * | 1994-09-21 | 1998-05-05 | Ricoh Company, Ltd. | Apparatus for compression using reversible embedded wavelets |
| CN1322442A (en) * | 1999-07-20 | 2001-11-14 | 皇家菲利浦电子有限公司 | Encoding method for compression of video sequence |
| CN1438767A (en) * | 2002-02-16 | 2003-08-27 | 三星电子株式会社 | Method and apparatus for changing compressed audio frequency signal using small wave packet |
| CN1471312A (en) * | 2002-07-26 | 2004-01-28 | 包头钢铁学院 | Selective medical image compression method |
| CN101038303A (en) * | 2007-02-07 | 2007-09-19 | 深圳市双合电脑系统股份有限公司 | Compression method for electric power waveform recording date |
| CN101924562A (en) * | 2010-06-24 | 2010-12-22 | 北京师范大学 | Compression-type coding scheme of curve vector data based on integer wavelet transformation |
| CN101984666A (en) * | 2010-11-19 | 2011-03-09 | 南京邮电大学 | Image lossless compression and decompression method based on lifting wavelet transform |
| CN105187071A (en) * | 2015-09-29 | 2015-12-23 | 东北大学 | Pipeline millisecond online data compound compression/decompression method |
| CN108322220A (en) * | 2018-02-08 | 2018-07-24 | 华为技术有限公司 | Decoding method, device and coding/decoding apparatus |
Non-Patent Citations (2)
| Title |
|---|
| "Wavelet transform and Huffman coding based electrocardiogram compression algorithm: Application to telecardiology";S A Chouakri;《24th IUPAP Conference on Computational Physics》;20131231;1-16 * |
| "应用整数小波变换的LZ77电力数据压缩算法";许晓飞;《西安工程大学学报》;20180630;第32卷(第3期);337-342 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109067405A (en) | 2018-12-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109067405B (en) | Data compression method, device, terminal and computer readable storage medium | |
| CN109409518B (en) | Neural network model processing method and device and terminal | |
| US20210342692A1 (en) | Technologies for scaling deep learning training | |
| US10044370B1 (en) | Lossless binary compression in a memory constrained environment | |
| US9912353B1 (en) | Systems and methods for generating soft information in a flash device | |
| JP2021527376A (en) | Data compression | |
| US9100042B2 (en) | High throughput decoding of variable length data symbols | |
| CN108632624B (en) | Image data processing method and device, terminal equipment and readable storage medium | |
| CN104378119B (en) | The fast and lossless compression method of file system of embedded device data | |
| CN114065704A (en) | Data compression method, electronic device and computer program product | |
| CN108053034B (en) | Model parameter processing method and device, electronic equipment and storage medium | |
| US8878705B1 (en) | Variable bit-length reiterative lossless compression system and method | |
| US10103747B1 (en) | Lossless binary compression in a memory constrained environment | |
| CN116707532A (en) | Decompression method and device for compressed text, storage medium and electronic equipment | |
| CN112262578A (en) | Point cloud attribute encoding method and device and point cloud attribute decoding method and device | |
| US20230132037A1 (en) | Storage system and data processing method in storage system | |
| CN115765754A (en) | Data coding method and coded data comparison method | |
| CN107783990B (en) | Data compression method and terminal | |
| CN114282141A (en) | Processing method and device for compression format data, electronic equipment and readable storage medium | |
| US9583113B2 (en) | Audio compression using vector field normalization | |
| US9348535B1 (en) | Compression format designed for a very fast decompressor | |
| US10602178B1 (en) | Systems and methods for frame context selection | |
| US7538697B1 (en) | Heuristic modeling of adaptive compression escape sequence | |
| CN115001628B (en) | Data encoding method and device, data decoding method and device and data structure | |
| CN115934730B (en) | Data processing method and device, medium and computer equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20190625 Address after: 518057 Yuanzheng Industrial Park, 4012 North Fifth Avenue, Bantian Street, Longgang District, Shenzhen City, Guangdong Province Applicant after: LAUNCH TECH Co.,Ltd. Address before: 518000 Room 201, building A, No. 1, Qian Wan Road, Qianhai Shenzhen Hong Kong cooperation zone, Shenzhen, Guangdong (Shenzhen Qianhai business secretary Co., Ltd.) Applicant before: HESVIT HEALTH TECHNOLOGY Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |