WO2020015481A1 - 数据压缩方法及装置和计算机可读存储介质 - Google Patents

数据压缩方法及装置和计算机可读存储介质 Download PDF

Info

Publication number
WO2020015481A1
WO2020015481A1 PCT/CN2019/091011 CN2019091011W WO2020015481A1 WO 2020015481 A1 WO2020015481 A1 WO 2020015481A1 CN 2019091011 W CN2019091011 W CN 2019091011W WO 2020015481 A1 WO2020015481 A1 WO 2020015481A1
Authority
WO
WIPO (PCT)
Prior art keywords
values
value
codeword
encoding
data compression
Prior art date
Application number
PCT/CN2019/091011
Other languages
English (en)
French (fr)
Inventor
徐兵
张楠赓
Original Assignee
北京嘉楠捷思信息技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京嘉楠捷思信息技术有限公司 filed Critical 北京嘉楠捷思信息技术有限公司
Priority to US17/042,603 priority Critical patent/US11398832B2/en
Publication of WO2020015481A1 publication Critical patent/WO2020015481A1/zh
Priority to US17/844,857 priority patent/US11870467B2/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
    • H03M7/3066—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction by means of a mask or a bit-map
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
    • H03M7/40—Conversion to or from variable length codes, e.g. Shannon-Fano code, Huffman code, Morse code
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
    • H03M7/70—Type of the data to be coded, other than image and sound
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
    • H03M7/40—Conversion to or from variable length codes, e.g. Shannon-Fano code, Huffman code, Morse code
    • H03M7/4006—Conversion to or from arithmetic code

Definitions

  • the invention belongs to the field of data processing, and particularly relates to a data compression method and device, and a computer-readable storage medium.
  • a typical example is the neural network used in the field of artificial intelligence.
  • the depth of deep neural networks has become deeper and deeper.
  • the scale is getting larger and larger, and large-scale neural networks correspond to a large number of parameters.
  • the storage and transmission of these parameters has gradually become a problem that cannot be ignored, especially for edge computing.
  • Deep neural networks calculate directly on the terminal, so the terminal There are a large number of parameters stored on it.
  • the storage space of parameters has a large impact on the cost of the terminal neural network calculator (which can be a chip or other hardware / software form).
  • a major factor affecting the storage space of parameters is the maximum value of the parameter value after encoding. Codeword length. The longer the maximum codeword length, the larger the storage space required to store the code table.
  • Huffman code is a variable word length code, that is, the length of each codeword corresponding to the codeword is not necessarily the same, and it is usually the longest codeword.
  • the length will be longer than the original quantized encoding of the parameters.
  • the longest codeword length may be more than 20 bits and the shortest codeword length may be 3 bits.
  • the average codeword length will become shorter, for example, the average codeword length will become 12 bits. This improves the encoding efficiency compared to the original quantization encoding, but also brings certain difficulties to decoding.
  • the encoded code table needs to be passed to the decoder, and the decoder performs decoding by means of table lookup (or similar search tree), so for the original length
  • the traditional lossless compression algorithm is not optimal, but there is room for further optimization to reduce the maximum codeword length.
  • the probability distribution of the original or quantized values of the parameters is similar to the normal distribution, that is, the occurrence probability (or the number of occurrences) corresponding to each of the original values of the parameters is very different.
  • This provides the possibility for further compression of the parameter value encoding, that is, the maximum codeword length is further reduced compared to the traditional lossless compression encoding algorithm.
  • a data compression method which includes:
  • the predetermined threshold may be specifically determined by a person skilled in the art according to an actual application situation. Generally speaking, the larger the predetermined threshold is, the more the first set of values will be, and the more significant the effect of reducing the maximum codeword length will be, but the greater the impact on the overall coding efficiency will be. On the contrary, if the predetermined threshold is larger, If the value is smaller, the number of values in the first group will be smaller, and the effect of reducing the maximum codeword length will be smaller, but the effect on the overall coding efficiency may be smaller.
  • the purpose of pre-processing the first set of values and then encoding is to reduce the length of the codeword of the set of values.
  • pre-processing the first set of values may include: obtaining an original encoded codeword of each value in the first set of values.
  • encoding the second set of values and the preprocessed first set of values may include:
  • variable word length coding on the second set of values to obtain coded codewords corresponding to each value in the second set of values
  • the method may further include: forming a code table, the code table may include at least: each value in the first set of values and the corresponding combination codeword respectively; the second set of values Each value in the value and its corresponding coded or supplemented codeword respectively.
  • the method may further include: when decoding the input bit stream according to the code table, if the decoding result is the prefix, then pre-fetching the input bit stream from the prefix in the backward direction.
  • N bits be output as the original encoded codeword, where N may be the length of the original encoded codeword.
  • the length N of the original encoded codeword is preferably a natural number greater than 2. The larger N is, the more significant the advantages of the data compression method of the present invention are.
  • pre-processing the first set of values may include:
  • the pre-processing of the first set of values may include: calculating an average value of the appearance probability of each value in the first set of values, and calculating each value in the first set of values.
  • the occurrence probabilities of are set to be equal to each other and less than or equal to the average.
  • the preprocessing is intended to make the codeword lengths of the first set of values more uniform.
  • encoding the second set of values and the preprocessed first set of values includes:
  • variable word length coding on the second set of values and the preprocessed first set of values.
  • variable word length encoding may be a Shannon encoding, a Feno encoding, or a Huffman encoding.
  • a data compression device which includes an acquisition unit, a comparison unit, a preprocessing unit, and an encoding unit, wherein the acquisition unit is connected to the comparison unit, the comparison unit is connected to the preprocessing unit and the encoding unit, and The processing unit is connected to the coding unit and is characterized in that:
  • the obtaining unit is configured to obtain multiple values of a parameter and an appearance probability of each of the multiple values
  • the comparison unit is configured to compare the occurrence probability acquired by the acquisition unit with a predetermined threshold, and divide the plurality of values from the acquisition unit into a first group of occurrences whose occurrence probability is less than the predetermined threshold. A value, and a second set of values whose occurrence probability is greater than or equal to the predetermined threshold;
  • the pre-processing unit pre-processes the first set of values from the comparison unit.
  • the encoding unit encodes the second set of values from the comparison unit and the first set of values preprocessed by the preprocessing unit.
  • the predetermined threshold may be specifically determined by a person skilled in the art according to an actual application situation. Generally speaking, the larger the predetermined threshold is, the more the first set of values will be, and the more significant the effect of reducing the maximum codeword length will be, but the greater the impact on the overall coding efficiency will be. On the contrary, if the predetermined threshold is larger, If the value is smaller, the number of values in the first group will be smaller, and the effect of reducing the maximum codeword length will be smaller, but the effect on the overall coding efficiency may be smaller.
  • the purpose of pre-processing the first set of values and then encoding is to reduce the length of the codeword of the set of values.
  • the preprocessing includes: obtaining an original encoded codeword of each value in the first set of values.
  • the encoding performed by the encoding unit may include:
  • variable word length coding on the second set of values to obtain coded codewords corresponding to each value in the second set of values
  • the encoding unit is further configured to form a code table, and the code table at least includes: each value in the first set of values and the corresponding corresponding combination codeword; the second set of values Each value in the value and its corresponding coded or supplemented codeword respectively.
  • the pre-processing unit pre-processing the first set of values includes:
  • the first set of values can form a balanced binary tree.
  • the pre-processing unit may be further configured to:
  • the encoding unit may be further configured to perform variable word length encoding on the second set of values and the preprocessed first set of values together.
  • variable word length encoding is a Shannon encoding, a Fynol encoding, or a Huffman encoding.
  • a computer-readable storage medium having computer instructions stored thereon, characterized in that when the computer instructions are executed by a processor, the method as described above is implemented.
  • preprocessing is performed on a part of codewords with the lowest probability before encoding according to the present invention, so as to achieve the effect of reducing the maximum codeword length after encoding. Because the probability of the appearance of some of the parameters of the artificial neural network is much smaller than the values of other parts, the processing of this part of the value has little effect on the compression efficiency, but it will significantly reduce the complexity and implementation of decoding. Cost, especially the cost of memory space.
  • FIG. 1 is a schematic flowchart of a data compression method according to an embodiment of the present invention.
  • FIG. 2 is an example of a data compression method according to another embodiment of the present invention, in which (a) shows an example using a traditional Huffman coding, and (b) and (c) show a method using an embodiment of the present invention
  • FIG. 3 is another example of a data compression method according to another embodiment of the present invention, in which (a) shows an example using a traditional Huffman coding, and (b) and (c) show an example using the present invention. Examples of methods
  • FIG. 4 is an example of a data compression method according to yet another embodiment of the present invention, in which (a) shows an example using a conventional Huffman coding, and (b) shows an example using a method in an embodiment of the present invention;
  • FIG. 5 is a schematic diagram of a data compression device according to another embodiment of the present invention.
  • FIG. 6 shows a schematic diagram of a computer-readable storage medium according to an embodiment of the present invention.
  • FIG. 1 is a schematic flowchart of a data compression method according to an embodiment of the present invention.
  • the data compression method includes:
  • S101 Acquire multiple values of a parameter and an appearance probability of each of the multiple values
  • S102 Compare the occurrence probability with a predetermined threshold, where the value of the occurrence probability smaller than the predetermined threshold is the first set of values, and the value of the occurrence probability greater than or equal to the predetermined threshold is the first Two sets of values
  • S104 Encode the second set of values and the preprocessed first set of values.
  • the types of the parameters are not specifically limited, and may be various parameters, such as various parameters in an artificial neural network (such as parameters involved in convolution calculations), or parameters in other mathematical models or algorithms;
  • the degree of effectiveness of the effect achieved by the data compression method in the embodiment of the present invention is positively related to the magnitude of the difference in the probability distribution of the value of each parameter.
  • the parameters in the artificial neural network are taken as an example for description, but it is not limited to the parameters in the artificial neural network.
  • the parameter value processing is not performed in a unified manner as in the traditional method, but the parameter value is grouped, and the value of different groups is processed differently, so as to control and shorten the maximum codeword length. , Thereby reducing the size of the storage space used for the code table.
  • the larger the predetermined threshold is the larger the number of values in the first set of values will be, and the more significant the effect of reducing the maximum codeword length will be, but the more it may affect the overall coding efficiency.
  • the predetermined threshold value is smaller, the number of values in the first set of values will be smaller, and the effect of reducing the maximum codeword length will be smaller, but the effect on the overall coding efficiency may be smaller.
  • the purpose of preprocessing the first set of values and then encoding is to reduce the codeword length of the set of values.
  • FIG. 2 is an example of a data compression method according to another embodiment of the present invention, in which (a) shows an example using a traditional Huffman coding, and (b) and (c) show a method using an embodiment of the present invention
  • Fig. 3 is another example of a data compression method according to another embodiment of the present invention, in which (a) shows an example using a conventional Huffman encoding, and (b) and (c) show an example using the present invention Examples of methods in the examples.
  • the length N of the original encoding codeword of the parameter A before encoding is 3 bits (for example, the parameter A is quantized and encoded by 3 bits), and has eight values a 1 and a 2.
  • a 3 , a 4 , a 5 , a 6 , a 7 , and a 8 respectively, have the original encoding codewords 000, 001, 010, 011, 100, 101, 110, and 111, and the corresponding occurrence probability is 1 / 2, 1/4, 1/8, 1/16, 1/32, 1/64, 1/128, 1/128.
  • the shortest codeword length of the resulting code is only 1, that is, the codeword 1 corresponding to the value a1
  • the maximum codeword obtained is The length of the codeword reaches 7 bits, that is, the encoded codeword 0000001 corresponding to the value a 7 and the encoded codeword 0000000 corresponding to the value a 8 .
  • the predetermined threshold is 1/48, the values a 1 and a 2 are taken through step S102 described above. , A 3 , a 4 , a 5 , a 6 , a 7 , a 8 with probability 1/2, 1/4, 1/8, 1/16, 1/32, 1/64, 1/128, 1 / After comparing 128 with the predetermined threshold value, the eight values of parameter A are divided into two groups, that is, the first group values a 6 , a 7 , a 8 and the second group values a 1 , a 2 , a 3 , a 4 , a 5 .
  • the first group of values a 6 , a 7 , and a 8 are pre-processed to obtain the original encoded codewords 101, 110, and 111 of the first group of values a 6 , a 7 , and a 8 .
  • the second group of values a 1 , a 2 , a 3 , a 4 , and a 5 are subjected to variable word length coding, such as Huffman coding, to obtain the second group of values.
  • the shortest coded codeword 1 of the obtained coded codewords is 1 0 is added at the end as the supplementary codeword 10 of the value a 1 corresponding to the shortest encoded codeword, and the shortest encoded codeword 1 is supplemented with 1 as the prefix 11 at the end, and the prefix 11 and the obtained
  • the first set of values a 6 , a 7 , and a 8 are combined with the original encoded codewords 101, 110, and 111 to obtain 11101, 11110, and 11111, which are used as the first set of values a 6 , a 7 ,
  • the combined codeword of each value in a 8 is shown in (c) of FIG. 2.
  • the length of the longest coded codeword finally obtained in the embodiment of the present invention is 5, which is shorter than the length of the longest coded codeword corresponding to the conventional coding, which is 7.
  • the code table formed in the data compression method in this embodiment is shown in Table 1 below.
  • the code table may include at least: each value in the first set of values and their corresponding combined codewords; the second set of values Each value in and its corresponding coded codeword or complementary codeword.
  • Codeword Take value before encoding Codeword length 10 a 1 2 01 a 2 2 001 a 3 3 0001 a 4 4 0000 a 5 4 11101 a 6 5 11110 a 7 5 11111 a 8 5
  • the prefix may be a second set of values a 1, a 2, a 3 , a 4, a 5 obtained after encoding each codeword value in the codeword coded shortest 1 Supplement 1 at the end as the supplementary codeword 11 of the value a 1 corresponding to the shortest encoded codeword, and supplement the shortest encoded codeword 1 with 0 as the prefix 10 at the end, and use the prefix 10 with the obtained
  • the first set of values a 6 , a 7 , and a 8 are combined with the original encoded codewords 101, 110, and 111 to obtain 10101, 10110, and 10111, which are used as the first set of values a 6 , a 7 , and The combined codeword of each value in a 8 .
  • N is the length of the original encoded codeword.
  • the length N of the original encoded codeword is preferably a natural number greater than 2. The larger N is, the more significant the advantages of the data compression method of the present invention are.
  • the length N of the original encoding codeword of the parameter A before encoding is 4 bits (for example, the parameter A is quantized and encoded by 4 bits), and has 16 values a 1 , a 2 , a 3 , a 4 , a 5 , a 6 , a 7 , a 8 , a 9 , a 10 , a 11 , a 12 , a 13 , a 14 , a 15 , a 16 , each with the original encoding codeword 0000, 0001, 0010, 0011, 0100, 0101, 0110, 0111, 1000, 1001, 1010, 1011, 1100, 1101, 1110, 1111, the corresponding occurrence probability is 1/2, 1/4, 1/8, respectively.
  • the values a 1 and a 2 are taken through the above step S102.
  • a 3 , a 4 , a 5 , a 6 , a 7 , a 8 , a 9 , a 10 , a 11 , a 12 , a 13 , a 14 , a 15 , a 16 are compared with the predetermined threshold
  • the 16 values of parameter A are then divided into two groups, that is, the first group takes the values a 7 , a 8 , a 9 , a 10 , a 11 , a 12 , a 13 , a 14 , a 15 , a 16 and
  • the second group takes values a 1 , a 2 , a 3 , a 4 , a 5 , a 6 .
  • the second group of values a 1 , a 2 , a 3 , a 4 , a 5 , and a 6 is subjected to variable word length encoding, such as Huffman encoding, to obtain a second value.
  • group values a 1, a 2, a 3 , a 4, a 5, a 6 value corresponding to each encoded codeword 1,01,001,0001, 00001,00000 in addition, the resulting code word after
  • the shortest coded codeword 1 is supplemented with 0 at the end as the complementary codeword 10 of the value a 1 corresponding to the shortest coded codeword, and the shortest coded codeword 1 is supplemented with 1 as the prefix 11 at the end.
  • the codewords 0110, 0111, 1000, 1001, 1010, 1011, 1100, 1101, 1110, 1111 are combined to obtain 110110, 110111, 111000, 111001, 111010, 111011, 111100, 111101, 111110, 111111, which are taken as the first group respectively.
  • the combined codeword of each value in the value is shown in FIG. 3 (c). In this way, the length of the longest encoded codeword finally obtained in the embodiment of the present invention is 6, which is significantly reduced compared to the length of the longest encoded codeword 15 corresponding to the conventional encoding.
  • the code table formed in the data compression method in this embodiment is shown in Table 2 below.
  • the code table may include at least: each value in the first set of values and their corresponding combined codewords; the second set of values Each value in and its corresponding coded codeword or complementary codeword.
  • Codeword Take value before encoding Codeword length 10 a 1 2 01 a 2 2 001 a 3 3 0001 a 4 4 00001 a 5 5 00000 a 6 5 110110 a 7 6 110111 a 8 6 111000 a 9 6 111001 a 10 6 111010 a 11 6 111011 a 12 6 111100 a 13 6
  • FIG. 4 is an example of a data compression method according to still another embodiment of the present invention, in which (a) shows an example using a conventional Huffman coding, and (b) shows an example using a method in an embodiment of the present invention.
  • the length N of the original encoding codeword of the parameter A before encoding according to the present invention is 3 bits (for example, the parameter A is quantized and encoded by 3 bits), and has eight values a 1 , a 2 , a 3 , a 4 , a 5 , a 6 , a 7 , and a 8 , respectively, have the original encoding codewords 000, 001, 010, 011, 100, 101, 110, and 111, and the corresponding occurrence probabilities are 1 respectively. / 2, 1/4, 1/8, 1/16, 1/32, 1/64, 1/128, 1/128.
  • the predetermined threshold is 1/24
  • the values a 1 , a 2 , a 3 , and The probabilities of a 4 , a 5 , a 6 , a 7 , a 8 are 1/2, 1/4, 1/8, 1/16, 1/32, 1/64, 1/128, 1/128 and the schedule.
  • the eight values of parameter A are divided into two groups, that is, the first group takes the values a 5 , a 6 , a 7 , a 8 and the second group takes the values a 1 , a 2 , a 3 , a 4 .
  • the first group of values a 5 , a 6 , a 7 , and a 8 are pre-processed to reset the appearance probability of each of the values such that the first group of values a 5 , a 6 , a 7 , a
  • the sum of the occurrence probabilities of the values in 8 is not greater than the second smallest value of 1/8 in the second set of values a 1 , a 2 , a 3 , and a 4 , and the first set of values can form a balanced binary tree. This is In order to avoid that the lengths of the codewords obtained when the first set of values are re-encoded are not uniform and are too different from each other.
  • the pre-processing performed on the first group of values a 1 , a 2 , a 3 , and a 4 may include: calculating the values of each of the first group of values a 1 , a 2 , a 3 , and a 4
  • the average value of the appearance probability, and the appearance probability of each value in the first group of values a 1 , a 2 , a 3 , and a 4 is set to be equal to each other and less than or equal to the average value.
  • the lengths of the codewords obtained by encoding the values a 1 , a 2 , a 3 , and a 4 can be more uniform.
  • encode the second set of values a 5 , a 6 , a 7 , a 8 and the first set of values a 1 , a 2 , a 3 , and a 4 after the occurrence probability is reset for example, perform encoding Variable word length coding, as shown in Figure 4 (b).
  • the code table formed in the data compression method in this embodiment is shown in Table 3 below.
  • the code table may include at least: each value in the first set of values and their corresponding combined codewords; the second set of values Each value in and its corresponding coded codeword or complementary codeword.
  • Codeword Take value before encoding Codeword length 1 a 1 1 01 a 2 2 001 a 3 3 0001 a 4 4 000011 a 5 6 000010 a 6 6 000001 a 7 6 000000 a 8 6
  • the length of the longest coded codeword finally obtained in this example is 6, which is smaller than the length of the longest coded codeword corresponding to the conventional coding, which is 7.
  • variable word length encoding may be a Shannon encoding, a Ferno encoding, a Huffman encoding, or other encoding methods.
  • FIG. 5 a schematic diagram of a data compression apparatus 500 according to another embodiment of the present invention is shown.
  • the data compression device 500 includes: an obtaining unit 501, a comparing unit 502, a preprocessing unit 503, and an encoding unit 504, wherein the obtaining unit 501 is connected to the comparison unit 502, the comparison unit 502 is connected to the preprocessing unit 503 and the encoding unit 504, and preprocessing The unit 503 is connected to the encoding unit 504.
  • the obtaining unit 501 is configured to obtain multiple values of a parameter and an appearance probability of each of the multiple values
  • the comparing unit 502 is configured to compare the appearance probability obtained by the obtaining unit 501 with a predetermined threshold, and divide the multiple values from the obtaining unit 502 into a first group of values whose appearance probability is less than the predetermined threshold, and its appearance A second set of values with a probability greater than or equal to the predetermined threshold;
  • the pre-processing unit 503 pre-processes the first set of values from the comparison unit 502;
  • the encoding unit 504 encodes the second set of values from the comparison unit 502 and the first set of values preprocessed by the preprocessing unit 503.
  • the predetermined threshold may be specifically determined by a person skilled in the art according to an actual application situation. Generally speaking, the larger the predetermined threshold is, the more the first set of values will be, and the more significant the effect of reducing the maximum codeword length will be, but the greater the impact on the overall coding efficiency will be. On the contrary, if the predetermined threshold is larger, If the value is smaller, the number of values in the first group will be smaller, and the effect of reducing the maximum codeword length will be smaller, but the effect on the overall coding efficiency may be smaller.
  • the purpose of preprocessing the first set of values and then encoding is to reduce the codeword length of the set of values.
  • the preprocessing may include: obtaining an original encoded codeword of each value in the first set of values.
  • the encoding performed by the encoding unit 504 may include:
  • the original coded codeword combination of each value in the first set of values is used as the combined codeword of each value in the first set of values, respectively.
  • the encoding unit 504 may also be used to form a code table, where the code table includes at least: each value in the first set of values and the corresponding combination codeword respectively; the second set of values Each value of and its corresponding coded or supplemented codeword.
  • the pre-processing performed by the pre-processing unit 503 on the first set of values includes:
  • the occurrence probability of each value in the first group of values is set so that: the sum of the occurrence probability of each value in the first group of values is not greater than the next smallest value in the second group of values, and the first group of values Ability to form a balanced binary tree.
  • the pre-processing unit 503 may also be configured to:
  • the encoding unit 504 may be further configured to perform variable word length encoding on the second set of values and the preprocessed first set of values together.
  • variable word length encoding is Shannon encoding, Fynol encoding, Huffman encoding, or other encoding methods.
  • a computer-readable storage medium is also provided. As shown in FIG. 6, a schematic diagram of a computer-readable storage medium 600 according to an embodiment of the present invention is shown.
  • the computer-readable storage medium stores computer instructions, and the computer instructions are executed by a processor to implement Data compression method.
  • the computer-readable storage medium 600 may employ a portable compact disk read-only memory (CD-ROM).
  • CD-ROM portable compact disk read-only memory
  • the computer-readable storage medium 600 of the present invention is not limited thereto, and in this document, the computer-readable storage medium may be any tangible medium containing or storing computer instructions.
  • preprocessing is performed on a part of codewords with the lowest probability before encoding according to the present invention, so as to achieve the effect of reducing the maximum codeword length after encoding. Because the probability of the appearance of some of the parameters of the artificial neural network is much smaller than the values of other parts, the processing of this part of the value has little effect on the compression efficiency, but it will significantly reduce the complexity and implementation of decoding. Cost, especially the cost of memory space.
  • the units or modules described in the embodiments of the present disclosure may be implemented by software, and may also be implemented by hardware.
  • the embodiments can be implemented by means of software plus a necessary universal hardware platform, and of course, they can also be implemented by hardware.
  • the above-mentioned technical solution essentially or part that contributes to the existing technology can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic A disc, an optical disc, and the like include several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

一种数据压缩方法，其包括：获取参数的多个取值，以及所述多个取值中每一者的出现概率（S101）；将所述出现概率与预定阈值进行比较，其中所述出现概率小于所述预定阈值的取值为第一组取值，而所述出现概率大于或等于所述预定阈值的取值为第二组取值（S102）；对所述第一组取值进行预处理（S103）；以及对所述第二组取值和经预处理的第一组取值进行编码（S104）。利用该数据压缩方法，能够有利地降低最大码字长度，从而降低码表对储存空间的需求。

Description

数据压缩方法及装置和计算机可读存储介质 技术领域
本发明属于数据处理领域，具体涉及一种数据压缩方法和装置以及计算机可读存储介质。
背景技术
随着各种电子设备计算处理能力的不断增强，以及电子设备间通信的增多，在电子设备中存储的数据量以及在电子设备之间传输的数据量也不断上升。一个典型的例子就是人工智能领域中采用的神经网络，随着目前计算能力的增强和需要解决的问题越来越复杂，以及对神经网络性能的要求逐步提高，深度神经网络的深度越来越深、规模越来越大，大规模的神经网络对应着大量的参数，这些参数的存储和传输也逐渐成为一个不能忽视的问题，尤其是针对边缘计算，深度神经网络直接在终端上计算，因此终端上存储有大量参数，参数的存储空间对终端神经网络计算器(可以是芯片或者其他硬件/软件形式)的成本影响较大，而影响参数的存储空间的一个主要因素就是参数值编码后的最大码字长度，最大码字长度越长，则存储码表所需的存储空间就越大。
因此，存在对神经网络中需要存储和/或传输的数据(例如各种参数)进行压缩的方法。传统上，一种方式是减少神经网络中不必要的神经元连接，从而减少参数数量，但是这可能会降低神经网络的性能；另一种方式是对神经网络中的参数进行量化，对量化后的参数进一步采用一些无损压缩编码算法进行压缩编码，以提高参数的存储和传输效率，降低对存储空间的需求，但是该方式中，码表的大小取决于最大码字长度，最大码字长度越长，则所需存储空间越大。
例如，以采用霍夫曼编码作为无损压缩编码算法为例，霍夫曼码是一种可变字长码，即编码后每个码字对应码字长度不一定相同，通常最长的码字长度会比参数的原始量化编码要长，例如，量化后为16比特的参数经过霍夫曼编码之后，所得的最长码字长度可能为20比特以上，最短码字长度可能为3比特，但是平均码字长度会变短，例如平均码字长度变成12比特，这相比原始量化编码提高了编码效率，但是也给译码带来一定困难。
具体而言，在霍夫曼译码时，需要将编码后的码表传给译码器，译码器通过查表的方式(或者类似的查找树的方式)进行译码，那么对于本来长度较大的系数，例如16比特系数，至少需要存储2^16＝64k个表项，每个表项至少需要包括：16比特原始数据、编码后码字长度，如果不扩展则还需要包括编码后的码字。如果为了硬件实现查找表的方便，往往还需要将这个表按照最长码字长度进行扩展(即，将码表中所有编码后的码字高位对齐，不足的末尾补齐到最大码字长度，从而导致译码表所需空间过大)，以最长码字长度为20为例，那么就要存储2^20＝1M个表项。因此，现有技术中硬件实现译码时所需存储空间大。
但是实际上，对于神经网络参数而言，传统的无损压缩算法并非是最优的，而是存在进一步优化的空间，以降低最大码字长度。具体而言，通过对大量神经网络参数的分析发现，参数的原始取值或量化值的概率分布呈类似正态分布，即参数的各原始取值对应的出现概率(或出现次数)差异很大，这就为参数取值编码的进一步压缩提供了可能，即相比传统的无损压缩编码算法进一步降低最大码字长度。
发明内容
针对上面提到的码表大小由于编码后的最大码字长度过长而导致码表存储空间过大的问题，提出了一种数据压缩方法和装置，其能控制和缩短最大码字长度，从而降低用于码表的存储空间大小。
依据本发明的第一方面，提供了一种数据压缩方法，其包括：
获取参数的多个取值，以及所述多个取值中每一者的出现概率；
将所述出现概率与预定阈值进行比较，其中所述出现概率小于所述预定阈值的取值为第一组取值，而所述出现概率大于或等于所述预定阈值的取值为第二组取值；
对所述第一组取值进行预处理；以及
对所述第二组取值和经预处理的第一组取值进行编码。
在这里，预定阈值可以根据实际应用情况而由本领域技术人员具体确定。一般来说，预定阈值越大，则第一组取值的数量会越多，对最大码字长度的降低效果越显著，但是可能对总体编码效率的影响会越大；相反，如果预定阈值越小，则第一组取值的数量会越少，对最大码字长度的降低效果会较小，但是可能对总体编码效率的影响会越小。
此外，对第一组取值进行预处理然后进行编码的目的在于降低该组取值的编码码字长度。
优选地，对所述第一组取值进行预处理可以包括：获取所述第一组取值中各取值的原始编码码字。
优选地，对所述第二组取值和经预处理的第一组取值进行编码可以包括：
对所述第二组取值进行可变字长编码，得到所述第二组取值中各取值对应的编码后码字，以及
将所述可变字长编码所得的最短编码后码字在末尾补充0和1中的一者作为所述最短编码后码字对应的取值的补充后码字，并且将所述最短编码后码字在末尾补充0和1中的另一者作为前缀与所获取的所述第一组取值中各取值的原始编码码字组合分别作为所述第一组取值中的各取值的组合码字。
优选地，上述方法还可以包括：形成码表，所述码表可至少包括：所述第一组取值中的各取值及其分别对应的所述组合码字；所述第二组取值中的各取值及其分别对应的所述编码后码字或补充后码字。
优选地，所述方法还可以包括：在根据所述码表对输入比特流进行译码时， 如果译码结果为所述前缀，则在所述输入比特流中从所述前缀往后取预设N个比特作为所述原始编码码字进行输出，其中N可以是所述原始编码码字的长度。原始编码码字的长度N优选为大于2的自然数，N越大，本发明的数据压缩方法的优势更显著。
优选地，对所述第一组取值进行预处理可以包括：
将所述第一组取值中各取值的出现概率设置为使得：所述第一组取值中各取值的出现概率之和不大于所述第二组取值中的次小值，并且所述第一组取值能够形成平衡二叉树。更加优选地，对所述第一组取值进行预处理可以包括：计算所述第一组取值中各取值的出现概率的平均值，并且将所述第一组取值中各取值的出现概率设置为彼此相等且小于或等于所述平均值。在这里，所述预处理旨在使得对所述第一组取值编码后的码字长度更加均匀。
优选地，对所述第二组取值和经预处理的第一组取值进行编码包括：
对所述第二组取值和经预处理的第一组取值一起进行可变字长编码。
优选地，所述可变字长编码可以为香农编码、费诺编码或者霍夫曼编码。
依据本发明的第一方面，提供了一种数据压缩装置，包括获取单元、比较单元、预处理单元和编码单元，其中获取单元与比较单元连接，比较单元与预处理单元及编码单元连接，预处理单元与编码单元连接，其特征在于，
所述获取单元用于获取参数的多个取值，以及所述多个取值中每一者的出现概率；
所述比较单元用于将所述获取单元获取的出现概率与预定阈值进行比较，并且将来自所述获取单元的所述多个取值分为其出现概率小于所述预定阈值的第一组取值、以及其出现概率大于或等于所述预定阈值的第二组取值；
所述预处理单元对来自所述比较单元的所述第一组取值进行预处理；以及
所述编码单元对来自所述比较单元的所述第二组取值和经所述预处理单元预处理的第一组取值进行编码。
在这里，预定阈值可以根据实际应用情况而由本领域技术人员具体确定。一般来说，预定阈值越大，则第一组取值的数量会越多，对最大码字长度的降 低效果越显著，但是可能对总体编码效率的影响会越大；相反，如果预定阈值越小，则第一组取值的数量会越少，对最大码字长度的降低效果会较小，但是可能对总体编码效率的影响会越小。
此外，对第一组取值进行预处理然后进行编码的目的在于降低该组取值的编码码字长度。
优选地，所述预处理包括：获取所述第一组取值中各取值的原始编码码字。
优选地，所述编码单元进行的编码可以包括：
对所述第二组取值进行可变字长编码，得到所述第二组取值中各取值对应的编码后码字，以及
将所述可变字长编码所得的最短编码后码字在末尾补充0和1中的一者作为所述最短编码后码字对应的取值的补充后码字，并且将所述最短编码后码字在末尾补充0和1中的另一者作为前缀与所获取的所述第一组取值中各取值的原始编码码字组合分别作为所述第一组取值中的各取值的组合码字。
优选地，所述编码单元还用于形成码表，所述码表至少包括：所述第一组取值中的各取值及其分别对应的所述组合码字；所述第二组取值中的各取值及其分别对应的所述编码后码字或补充后码字。
优选地，所述预处理单元对所述第一组取值进行预处理包括：
将所述第一组取值中各取值的出现概率设置为使得：所述第一组取值中各取值的出现概率之和不大于所述第二组取值中的次小值，并且所述第一组取值能够形成平衡二叉树。
优选地，所述预处理单元还可以用于：
计算所述第一组取值中各取值的出现概率的平均值，并且将所述第一组取值中各取值的出现概率设置为彼此相等且小于或等于所述平均值。
优选地，所述编码单元还可以用于：对所述第二组取值和经预处理的第一组取值一起进行可变字长编码。
优选地，所述可变字长编码为香农编码、费诺编码或者霍夫曼编码。
根据本发明的又一方面，还提供了一种计算机可读存储介质，其上存储有 计算机指令，其特征在于，所述计算机指令被处理器执行时实现如上所述的方法。
利用以上方案，针对进行本发明的编码前概率最小的一部分码字进行预处理，从而达到降低最大编码后码字长度的效果。因为人工神经网络的参数中有一部分取值的出现概率是远小于其他部分的取值，因此针对这部分取值的处理对压缩效率影响不大，但却会显著降低译码的复杂度和实现代价，尤其是存储器空间的代价。
应当理解，上述说明仅是本发明技术方案的概述，以便能够更清楚地了解本发明的技术手段，从而可依照说明书的内容予以实施。为了让本发明的上述和其它目的、特征和优点能够更明显易懂，以下特举说明本发明的具体实施方式。
附图说明
通过阅读下文的示例性实施例的详细描述，本领域普通技术人员将明白本文所述的有点和益处以及其他优点和益处。附图仅用于示出示例性实施例的目的，而并不认为是对本发明的限制。而且在整个附图中，用相同的标号表示相同的部件。在附图中：
图1为根据本发明一实施例的数据压缩方法的流程示意图；
图2为根据本发明另一实施例的数据压缩方法的示例，其中(a)示出采用传统霍夫曼编码的示例，而(b)和(c)示出采用本发明实施例中的方法的示例；
图3为根据本发明另一实施例的数据压缩方法的另一示例，其中(a)示出采用传统霍夫曼编码的示例，而(b)和(c)示出采用本发明实施例中的方法的示例；
图4为根据本发明又一实施例的数据压缩方法的示例，其中(a)示出采用传统霍夫曼编码的示例，而(b)示出采用本发明实施例中的方法的示例；
图5为根据本发明又一实施例的数据压缩装置的示意图；
图6示出了根据本发明一实施例的计算机可读存储介质的示意图。
具体实施方式
下面将参照附图更详细地描述本公开的示例性实施例。虽然附图中显示了本公开的示例性实施例，然而应当理解，可以以各种形式实现本公开而不应被这里阐述的实施例所限制。相反，提供这些实施例是为了能够更透彻地理解本公开，并且能够将本公开的范围完整的传达给本领域的技术人员。
在本发明中，应理解，诸如“包括”或“具有”等术语旨在指示本说明书中所公开的特征、数字、步骤、行为、部件、部分或其组合的存在，并且不旨在排除一个或多个其他特征、数字、步骤、行为、部件、部分或其组合存在的可能性。
另外还需要说明的是，在不冲突的情况下，本发明中的实施例及实施例中的特征可以相互组合。下面将参考附图并结合实施例来详细说明本发明。
在图1中，示出了根据本发明一实施例的数据压缩方法的流程示意图。该数据压缩方法包括：
S101：获取参数的多个取值，以及所述多个取值中每一者的出现概率；
S102：将所述出现概率与预定阈值进行比较，其中所述出现概率小于所述预定阈值的取值为第一组取值，而所述出现概率大于或等于所述预定阈值的取值为第二组取值；
S103：对第一组取值进行预处理；以及
S104：对第二组取值和经预处理的第一组取值进行编码。
在这里，对所述参数的类型不作具体限定，其可以是各种参数，例如人工神经网络中的各种参数(例如卷积计算中涉及的参数)，或者其它数学模型或算法中的参数；本发明实施例中的数据压缩方法所达效果的有效程度高低，与各参数取值概率分布差异的大小呈正相关。本申请中以人工神经网络中的参数为 例进行说明，但是不限于人工神经网络中的参数。
本发明实施例中，对参数取值的处理并非如传统那样采用统一方式，而是对参数取值进行分组，并对不同组的取值进行不同的处理，从而实现控制和缩短最大码字长度，进而降低用于码表的存储空间大小的目的。
本领域技术人员可以理解的是，预定阈值越大，则第一组取值中的取值数量会越多，对最大码字长度的降低效果越显著，但是可能对总体编码效率的影响会越大；相反，如果预定阈值越小，则第一组取值中的取值数量会越少，对最大码字长度的降低效果会较小，但是可能对总体编码效率的影响会越小。
此外，对第一组取值进行预处理然后进行编码的目的在于降低该组取值的编码后码字长度。
下面结合图2和图3进一步描述本发明实施例。图2为根据本发明另一实施例的数据压缩方法的示例，其中(a)示出采用传统霍夫曼编码的示例，而(b)和(c)示出采用本发明实施例中的方法的示例；图3为根据本发明另一实施例的数据压缩方法的另一示例，其中(a)示出采用传统霍夫曼编码的示例，而(b)和(c)示出采用本发明实施例中的方法的示例。
如图2所示，在本示例中，参数A在编码前的原始编码码字长度N为3比特(例如，该参数A以3比特进行量化编码)，具有8个取值a 1、a 2、a 3、a 4、a 5、a 6、a 7、a 8，分别具有原始编码码字000、001、010、011、100、101、110、111，对应的出现概率为分别为1/2、1/4、1/8、1/16、1/32、1/64、1/128、1/128。如图中(a)所示，如果采用传统霍夫曼(Huffman)编码，尽管所得编码的最短码字长度仅为1，即取值a 1对应的编码后码字1，但是所得编码的最大码字长度达到了7比特，即取值a 7对应的编码后码字0000001和取值a 8对应的编码后码字0000000。
在采用本发明实施例中的数据压缩方法的情况下，如图2中(b)和(C)所示，假如预定阈值为1/48，则通过上述步骤S102将取值a 1、a 2、a 3、a 4、a 5、a 6、a 7、a 8的概率1/2、1/4、1/8、1/16、1/32、1/64、1/128、1/128与该预定阈值进行比较后将参数A的8个取值分为两组，即第一组取值a 6、a 7、a 8和第二 组取值a 1、a 2、a 3、a 4、a 5。接着对第一组取值a 6、a 7、a 8进行预处理以获取第一组取值a 6、a 7、a 8中各取值的原始编码码字101、110、111。同时，如图2中(b)所示，对第二组取值a 1、a 2、a 3、a 4、a 5进行可变字长编码如霍夫曼编码，得到第二组取值a 1、a 2、a 3、a 4、a 5中各取值对应的编码后码字1、01、001、0001、0000，此外，将所得编码后码字中的最短编码后码字1在末尾补充0作为所述最短编码后码字对应的取值a 1的补充后码字10，并且将所述最短编码后码字1在末尾补充1作为前缀11，将该前缀11与所获取的第一组取值a 6、a 7、a 8中各取值的原始编码码字101、110、111组合，得到11101、11110、11111，分别作为第一组取值a 6、a 7、a 8中的各取值的组合码字，如图2中(c)所示。这样，本发明实施例最终所得的最长编码后码字的长度为5，比传统编码对应的最长编码后码字的长度7减小了。
本实施例中的数据压缩方法中形成的码表如下表1所示，该码表可至少包括：第一组取值中的各取值及其分别对应的组合码字；第二组取值中的各取值及其分别对应的编码后码字或补充后码字。
表1
编码后码字 编码前取值 码字长度
10 a 1 2
01 a 2 2
001 a 3 3
0001 a 4 4
0000 a 5 4
11101 a 6 5
11110 a 7 5
11111 a 8 5
在上述获得前缀的方式中，当然也可以是将第二组取值a 1、a 2、a 3、a 4、a 5中各取值的所得编码后码字中的最短编码后码字1在末尾补充1作为所述最短编码后码字对应的取值a 1的补充后码字11，并且将所述最短编码后码字1在末 尾补充0作为前缀10，将该前缀10与所获取的第一组取值a 6、a 7、a 8中各取值的原始编码码字101、110、111组合，得到10101、10110、10111，分别作为第一组取值a 6、a 7、a 8中的各取值的组合码字。
在根据所得到的码表对输入比特流进行译码时，如果译码结果为所述前缀，则在输入比特流中从所述前缀往后取预设N个比特作为原始编码码字进行输出，其中N为所述原始编码码字的长度。原始编码码字长度N优选为大于2的自然数，N越大，本发明的数据压缩方法的优势更显著。
在另一示例中，如图3所示，参数A在编码前的原始编码码字长度N为4比特(例如，该参数A以4比特进行量化编码)，具有16个取值a 1、a 2、a 3、a 4、a 5、a 6、a 7、a 8、a 9、a 10、a 11、a 12、a 13、a 14、a 15、a 16，分别具有原始编码码字0000、0001、0010、0011、0100、0101、0110、0111、1000、1001、1010、1011、1100、1101、1110、1111，对应的出现概率为分别为1/2、1/4、1/8、1/16、1/32、1/64、1/128、1/256、1/512、1/1024、1/2048、1/4096、1/8192、1/16384、1/32768、1/32768。如图3中(a)所示，如果采用传统霍夫曼(Huffman)编码，尽管所得编码的最短码字长度仅为1，即取值a 1对应的编码后码字1，但是所得编码的最大码字长度达到了15比特，即取值a 15对应的编码后码字000000000000001和取值a 16对应的编码后码字000000000000000。
在采用本发明实施例中的数据压缩方法的情况下，如图3中(b)和(C)所示，假如预定阈值为1/72，则通过上述步骤S102将取值a 1、a 2、a 3、a 4、a 5、a 6、a 7、a 8、a 9、a 10、a 11、a 12、a 13、a 14、a 15、a 16的概率与该预定阈值进行比较后将参数A的16个取值分为两组，即第一组取值a 7、a 8、a 9、a 10、a 11、a 12、a 13、a 14、a 15、a 16和第二组取值a 1、a 2、a 3、a 4、a 5、a 6。接着对第一组取值a 7、a 8、a 9、a 10、a 11、a 12、a 13、a 14、a 15、a 16进行预处理以获取第一组取值a 7、a 8、a 9、a 10、a 11、a 12、a 13、a 14、a 15、a 16中各取值的原始编码码字0110、0111、1000、1001、1010、1011、1100、1101、1110、1111。同时，如图3中(b)所示，对第二组取值a 1、a 2、a 3、a 4、a 5、a 6进行可变字长编码如霍夫曼编码，得到第二组取值a 1、a 2、a 3、a 4、a 5、a 6中各取值对应的编码后码字1、01、001、0001、 00001、00000，此外，将所得编码后码字中的最短编码后码字1在末尾补充0作为所述最短编码后码字对应的取值a 1的补充后码字10，并且将所述最短编码后码字1在末尾补充1作为前缀11，将该前缀11与所获取的第一组取值a 7、a 8、a 9、a 10、a 11、a 12、a 13、a 14、a 15、a 16中各取值的原始编码码字0110、0111、1000、1001、1010、1011、1100、1101、1110、1111组合，得到110110、110111、111000、111001、111010、111011、111100、111101、111110、111111，分别作为第一组取值中的各取值的组合码字，如图3中(c)所示。这样，本发明实施例最终所得的最长编码后码字的长度为6，比传统编码对应的最长编码后码字的长度15显著减小了。
本实施例中的数据压缩方法中形成的码表如下表2所示，该码表可至少包括：第一组取值中的各取值及其分别对应的组合码字；第二组取值中的各取值及其分别对应的编码后码字或补充后码字。
表2
编码后码字 编码前取值 码字长度
10 a 1 2
01 a 2 2
001 a 3 3
0001 a 4 4
00001 a 5 5
00000 a 6 5
110110 a 7 6
110111 a 8 6
111000 a 9 6
111001 a 10 6
111010 a 11 6
111011 a 12 6
111100 a 13 6
111101 a 14 6
111110 a 15 6
111111 a 16 6
图4为根据本发明又一实施例的数据压缩方法的示例，其中(a)示出采用传统霍夫曼编码的示例，而(b)示出采用本发明实施例中的方法的示例。
如图4所示的示例中，参数A在根据本发明编码前的原始编码码字长度N为3比特(例如，该参数A以3比特进行量化编码)，具有8个取值a 1、a 2、a 3、a 4、a 5、a 6、a 7、a 8，分别具有原始编码码字000、001、010、011、100、101、110、111，对应的出现概率为分别为1/2、1/4、1/8、1/16、1/32、1/64、1/128、1/128。如图中(a)所示，如果采用传统霍夫曼编码，尽管所得编码的最短码字长度仅为1，即取值a 1对应的编码后码字1，但是所得编码的最大码字长度达到了7比特，即取值a 7对应的编码后码字0000001和a 8对应的编码后码字0000000。
在采用本发明实施例中的数据压缩方法的情况下，如图4中(b)所示，假如预定阈值为1/24，则通过上述步骤S102将取值a 1、a 2、a 3、a 4、a 5、a 6、a 7、a 8的概率1/2、1/4、1/8、1/16、1/32、1/64、1/128、1/128与该预定阈值进行比较后将参数A的8个取值分为两组，即第一组取值a 5、a 6、a 7、a 8和第二组取值a 1、a 2、a 3、a 4。接着对第一组取值a 5、a 6、a 7、a 8进行预处理以将其中各取值的出现概率重新设置为使得：第一组取值a 5、a 6、a 7、a 8中各取值的出现概率之和不大于第二组取值a 1、a 2、a 3、a 4中的次小值1/8，并且第一组取值能够形成平衡二叉树，这是为了避免对第一组取值重新编码时所得码字长度不够均匀而彼此差别太大。更加优选地，对第一组取值a 1、a 2、a 3、a 4进行的预处理可以包括：计算第一组取值a 1、a 2、a 3、a 4中各取值的出现概率的平均值，并且将第一组取值a 1、a 2、a 3、a 4中各取值的出现概率设置为彼此相等且小于或等于该平均值，这样，对第一组取值a 1、a 2、a 3、a 4编码后所得的码字长度能够更加均匀。
接下来，对第二组取值a 5、a 6、a 7、a 8和出现概率被重新设置后的第一组取值a 1、a 2、a 3、a 4一起进行编码，例如进行可变字长编码，如图4中(b)所示。 本实施例中的数据压缩方法中形成的码表如下表3所示，该码表可至少包括：第一组取值中的各取值及其分别对应的组合码字；第二组取值中的各取值及其分别对应的编码后码字或补充后码字。
表3
编码后码字 编码前取值 码字长度
1 a 1 1
01 a 2 2
001 a 3 3
0001 a 4 4
000011 a 5 6
000010 a 6 6
000001 a 7 6
000000 a 8 6
这样，本示例中最终所得的最长编码后码字的长度为6，比传统编码对应的最长编码后码字的长度7减小了。
这里的可变字长编码可以为香农编码、费诺编码或者霍夫曼编码，或者其它编码方式。
下面结合图5描述用于实现上述数据压缩方法的数据压缩装置。如图5所示，示出了根据本发明又一实施例的数据压缩装置500的示意图。该数据压缩装置500包括：获取单元501、比较单元502、预处理单元503和编码单元504，其中获取单元501与比较单元502连接，比较单元502与预处理单元503及编码单元504连接，预处理单元503与编码单元504连接。
其中，获取单元501用于获取参数的多个取值，以及所述多个取值中每一者的出现概率；
比较单元502用于将获取单元501获取的出现概率与预定阈值进行比较，并且将来自获取单元502的所述多个取值分为其出现概率小于预定阈值的第一组取值、以及其出现概率大于或等于所述预定阈值的第二组取值；
预处理单元503对来自比较单元502的第一组取值进行预处理；以及
编码单元504对来自比较单元502的第二组取值和经预处理单元503预处理的第一组取值进行编码。
在这里，预定阈值可以根据实际应用情况而由本领域技术人员具体确定。一般来说，预定阈值越大，则第一组取值的数量会越多，对最大码字长度的降低效果越显著，但是可能对总体编码效率的影响会越大；相反，如果预定阈值越小，则第一组取值的数量会越少，对最大码字长度的降低效果会较小，但是可能对总体编码效率的影响会越小。
此外，对第一组取值进行预处理然后进行编码的目的在于降低该组取值的编码后码字长度。
在一具体示例中，预处理可以包括：获取第一组取值中各取值的原始编码码字。
编码单元504进行的编码可以包括：
对第二组取值进行可变字长编码，得到第二组取值中各取值对应的编码后码字，以及将可变字长编码所得的最短编码后码字在末尾补充0和1中的一者作为所述最短编码后码字对应的取值的补充后码字，并且将所述最短编码后码字在末尾补充0和1中的另一者作为前缀与所获取的所述第一组取值中各取值的原始编码码字组合分别作为所述第一组取值中的各取值的组合码字。
此外，编码单元504还可以用于形成码表，该码表至少包括：所述第一组取值中的各取值及其分别对应的所述组合码字；所述第二组取值中的各取值及其分别对应的所述编码后码字或补充后码字。
在另一具体示例中，预处理单元503对第一组取值进行的预处理包括：
将第一组取值中各取值的出现概率设置为使得：第一组取值中各取值的出现概率之和不大于第二组取值中的次小值，并且第一组取值能够形成平衡二叉树。
或者，预处理单元503还可以用于：
计算第一组取值中各取值的出现概率的平均值，并且将第一组取值中各取 值的出现概率设置为彼此相等且小于或等于该平均值。
编码单元504还可以用于：对第二组取值和经预处理的第一组取值一起进行可变字长编码。
可变字长编码为香农编码、费诺编码或者霍夫曼编码，或者其它编码方式。
根据本发明的又一实施例，还提供了一种计算机可读存储介质。如图6所示，示出了根据本发明一实施例的计算机可读存储介质600的示意图，该计算机可读存储介质上存储有计算机指令，该计算机指令被处理器执行时实现如上所述的数据压缩方法。该计算机可读存储介质600可以采用便携式紧凑盘只读存储器(CD-ROM)。然而，本发明的计算机可读存储介质600不限于此，在本文件中，计算机可读存储介质可以是任何包含或存储计算机指令的有形介质。
利用以上方案，针对进行本发明的编码前概率最小的一部分码字进行预处理，从而达到降低最大编码后码字长度的效果。因为人工神经网络的参数中有一部分取值的出现概率是远小于其他部分的取值，因此针对这部分取值的处理对压缩效率影响不大，但却会显著降低译码的复杂度和实现代价，尤其是存储器空间的代价。
附图中的流程图和框图，图示了按照本公开各种实施例的方法、装置和计算机可读存储介质的可能实现的体系架构、功能和操作。应当注意，流程图中的每个方框所表示的步骤未必按照标号所示的顺序进行，有时可以基本并行地执行，有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的硬件来实现，或者可以用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的单元或模块可以通过软件的方式实现，也可以通过硬件的方式来实现。
通过以上对实施例的描述，本领域的技术人员可以清楚地了解到各实施例可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件。基于这样的理解，上述技术方案本质上或者说对现有技术做出贡献的部分可以以软 件产品的形式体现出来，该计算机软件产品可以存储在计算机可读存储介质中，如ROM/RAM、磁碟、光盘等，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行各个实施例或者实施例的某些部分所述的方法。
最后应说明的是：以上实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的精神和范围。

Claims (18)

  1. 一种数据压缩方法，其特征在于，包括：
    获取参数的多个取值，以及所述多个取值中每一者的出现概率；
    将所述出现概率与预定阈值进行比较，其中所述出现概率小于所述预定阈值的取值为第一组取值，而所述出现概率大于或等于所述预定阈值的取值为第二组取值；
    对所述第一组取值进行预处理；以及
    对所述第二组取值和经预处理的第一组取值进行编码。
  2. 如权利要求1所述的数据压缩方法，其特征在于，
    对所述第一组取值进行预处理包括：获取所述第一组取值中各取值的原始编码码字。
  3. 如权利要求2所述的数据压缩方法，其特征在于，
    对所述第二组取值和经预处理的第一组取值进行编码包括：
    对所述第二组取值进行可变字长编码，得到所述第二组取值中各取值对应的编码后码字，以及
    将所述可变字长编码所得的最短编码后码字在末尾补充0和1中的一者作为所述最短编码后码字对应的取值的补充后码字，并且将所述最短编码后码字在末尾补充0和1中的另一者作为前缀与所获取的所述第一组取值中各取值的原始编码码字组合分别作为所述第一组取值中的各取值的组合码字。
  4. 如权利要求3所述的数据压缩方法，其特征在于，还包括：
    形成码表，所述码表至少包括：所述第一组取值中的各取值及其分别对应的所述组合码字；所述第二组取值中的各取值及其分别对应的所述编码后码字或补充后码字。
  5. 如权利要求4所述的数据压缩方法，其特征在于，还包括：
    在根据所述码表对输入比特流进行译码时，如果译码结果为所述前缀，则在所述输入比特流中从所述前缀往后取预设N个比特作为所述原始编码码字进 行输出。
  6. 如权利要求1所述的数据压缩方法，其特征在于，对所述第一组取值进行预处理包括：
    将所述第一组取值中各取值的出现概率设置为使得：所述第一组取值中各取值的出现概率之和不大于所述第二组取值中的次小值，并且所述第一组取值能够形成平衡二叉树。
  7. 如权利要求6所述的数据压缩方法，其特征在于：
    计算所述第一组取值中各取值的出现概率的平均值，并且将所述第一组取值中各取值的出现概率设置为彼此相等且小于或等于所述平均值。
  8. 如权利要求6或7所述的数据压缩方法，其特征在于，对所述第二组取值和经预处理的第一组取值进行编码包括：
    对所述第二组取值和经预处理的第一组取值一起进行可变字长编码。
  9. 如权利要求3或8所述的数据压缩方法，其特征在于，所述可变字长编码为香农编码、费诺编码或者霍夫曼编码。
  10. 一种数据压缩装置，包括获取单元、比较单元、预处理单元和编码单元，其中获取单元与比较单元连接，比较单元与预处理单元及编码单元连接，预处理单元与编码单元连接，其特征在于，
    所述获取单元用于获取参数的多个取值，以及所述多个取值中每一者的出现概率；
    所述比较单元用于将所述获取单元获取的出现概率与预定阈值进行比较，并且将来自所述获取单元的所述多个取值分为其出现概率小于所述预定阈值的第一组取值、以及其出现概率大于或等于所述预定阈值的第二组取值；
    所述预处理单元对来自所述比较单元的所述第一组取值进行预处理；以及
    所述编码单元对来自所述比较单元的所述第二组取值和经所述预处理单元预处理的第一组取值进行编码。
  11. 如权利要求10所述的数据压缩装置，其特征在于，
    所述预处理包括：获取所述第一组取值中各取值的原始编码码字。
  12. 如权利要求11所述的数据压缩装置，其特征在于，
    所述编码单元进行的编码包括：
    对所述第二组取值进行可变字长编码，得到所述第二组取值中各取值对应的编码后码字，以及
    将所述可变字长编码所得的最短编码后码字在末尾补充0和1中的一者作为所述最短编码后码字对应的取值的补充后码字，并且将所述最短编码后码字在末尾补充0和1中的另一者作为前缀与所获取的所述第一组取值中各取值的原始编码码字组合分别作为所述第一组取值中的各取值的组合码字。
  13. 如权利要求12所述的数据压缩装置，其特征在于：
    所述编码单元还用于形成码表，所述码表至少包括：所述第一组取值中的各取值及其分别对应的所述组合码字；所述第二组取值中的各取值及其分别对应的所述编码后码字或补充后码字。
  14. 如权利要求10所述的数据压缩装置，其特征在于，所述预处理单元对所述第一组取值进行预处理包括：
    将所述第一组取值中各取值的出现概率设置为使得：所述第一组取值中各取值的出现概率之和不大于所述第二组取值中的次小值，并且所述第一组取值能够形成平衡二叉树。
  15. 如权利要求14所述的数据压缩装置，其特征在于，所述预处理单元还用于：
    计算所述第一组取值中各取值的出现概率的平均值，并且将所述第一组取值中各取值的出现概率设置为彼此相等且小于或等于所述平均值。
  16. 如权利要求14或15所述的数据压缩装置，其特征在于，所述编码单元用于：对所述第二组取值和经预处理的第一组取值一起进行可变字长编码。
  17. 如权利要12或16所述的数据压缩装置，其特征在于，所述可变字长编码为香农编码、费诺编码或者霍夫曼编码。
  18. 一种计算机可读存储介质，其上存储有计算机指令，其特征在于，所述计算机指令被处理器执行时实现如权利要求1-9中任一项所述的方法。
PCT/CN2019/091011 2018-07-19 2019-06-27 数据压缩方法及装置和计算机可读存储介质 WO2020015481A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US17/042,603 US11398832B2 (en) 2018-07-19 2019-06-27 Data compression method and apparatus, and computer readable storage medium
US17/844,857 US11870467B2 (en) 2018-07-19 2022-06-21 Data compression method and apparatus, and computer readable storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810794662.9 2018-07-19
CN201810794662.9A CN110739974B (zh) 2018-07-19 2018-07-19 数据压缩方法及装置和计算机可读存储介质

Related Child Applications (2)

Application Number Title Priority Date Filing Date
US17/042,603 A-371-Of-International US11398832B2 (en) 2018-07-19 2019-06-27 Data compression method and apparatus, and computer readable storage medium
US17/844,857 Continuation US11870467B2 (en) 2018-07-19 2022-06-21 Data compression method and apparatus, and computer readable storage medium

Publications (1)

Publication Number Publication Date
WO2020015481A1 true WO2020015481A1 (zh) 2020-01-23

Family

ID=69164970

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/091011 WO2020015481A1 (zh) 2018-07-19 2019-06-27 数据压缩方法及装置和计算机可读存储介质

Country Status (3)

Country Link
US (2) US11398832B2 (zh)
CN (1) CN110739974B (zh)
WO (1) WO2020015481A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110739974B (zh) * 2018-07-19 2023-11-10 嘉楠明芯(北京)科技有限公司 数据压缩方法及装置和计算机可读存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6140945A (en) * 1997-04-18 2000-10-31 Fuji Xerox Co., Ltd. Coding apparatus, decoding apparatus, coding-decoding apparatus and methods applied thereto
CN101501999A (zh) * 2006-06-19 2009-08-05 埃塞克斯Pa有限责任公司 数据编码
CN102164019A (zh) * 2010-12-31 2011-08-24 华为技术有限公司 信道状态信息压缩的方法及装置
CN102438145A (zh) * 2011-11-22 2012-05-02 广州中大电讯科技有限公司 一种基于Huffman编码的图片无损压缩方法

Family Cites Families (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5836003A (en) * 1993-08-26 1998-11-10 Visnet Ltd. Methods and means for image and voice compression
KR950010425B1 (ko) * 1993-09-11 1995-09-16 국방과학연구소 코드분류에 의한 병렬처리 가변장 부호 복호기
US5574449A (en) * 1995-02-24 1996-11-12 Intel Corporation Signal processing with hybrid variable-length and entropy encodidng
US6304676B1 (en) * 1996-10-03 2001-10-16 Mark A. Mathews Apparatus and method for successively refined competitive compression with redundant decompression
JP4160689B2 (ja) * 1998-11-10 2008-10-01 株式会社リコー 算術符号化装置および算術復号装置
US6535642B1 (en) * 1999-07-13 2003-03-18 Microsoft Corporation Approximate string matching system and process for lossless data compression
US7436329B2 (en) * 2003-04-17 2008-10-14 Droplet Technology, Inc. Multiple technique entropy coding system and method
US7016547B1 (en) * 2002-06-28 2006-03-21 Microsoft Corporation Adaptive entropy encoding/decoding for screen capture content
FR2903269B1 (fr) * 2006-06-28 2008-10-03 Assistance Tech Et Etude De Ma Dispositif et procede d'encodage de type cabac
EP2141815A1 (en) * 2008-07-01 2010-01-06 Deutsche Thomson OHG Method for a hybrid Golomb-Elias gamma coding
KR101456495B1 (ko) * 2008-08-28 2014-10-31 삼성전자주식회사 무손실 부호화/복호화 장치 및 방법
US8004431B2 (en) * 2008-12-09 2011-08-23 Qualcomm Incorporated Fast parsing of variable-to-fixed-length codes
US8862967B2 (en) * 2012-03-15 2014-10-14 Sandisk Technologies Inc. Statistical distribution based variable-bit error correction coding
US20170180757A1 (en) * 2015-12-18 2017-06-22 Blackberry Limited Binarizer selection for image and video coding
JP6869676B2 (ja) * 2016-09-27 2021-05-12 キヤノン株式会社 情報処理装置、情報処理方法及びプログラム
US10348336B2 (en) * 2017-06-06 2019-07-09 Tsofun Algorithm Ltd. System and method for early termination of decoding in a multi user equipment environment
CN110739974B (zh) * 2018-07-19 2023-11-10 嘉楠明芯(北京)科技有限公司 数据压缩方法及装置和计算机可读存储介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6140945A (en) * 1997-04-18 2000-10-31 Fuji Xerox Co., Ltd. Coding apparatus, decoding apparatus, coding-decoding apparatus and methods applied thereto
CN101501999A (zh) * 2006-06-19 2009-08-05 埃塞克斯Pa有限责任公司 数据编码
CN102164019A (zh) * 2010-12-31 2011-08-24 华为技术有限公司 信道状态信息压缩的方法及装置
CN102438145A (zh) * 2011-11-22 2012-05-02 广州中大电讯科技有限公司 一种基于Huffman编码的图片无损压缩方法

Also Published As

Publication number Publication date
US11398832B2 (en) 2022-07-26
CN110739974B (zh) 2023-11-10
US11870467B2 (en) 2024-01-09
US20210099187A1 (en) 2021-04-01
CN110739974A (zh) 2020-01-31
US20220329258A1 (en) 2022-10-13

Similar Documents

Publication Publication Date Title
CN109379598B (zh) 一种基于fpga实现的图像无损压缩方法
CN111093081B (zh) 一种图像无损压缩方法及系统
US20190370658A1 (en) Self-Tuning Incremental Model Compression Solution in Deep Neural Network with Guaranteed Accuracy Performance
CN110518917B (zh) 基于Huffman编码的LZW数据压缩方法及系统
US20210050027A1 (en) Coding and decoding of spectral peak positions
JP7123910B2 (ja) インデックスコーディング及びビットスケジューリングを備えた量子化器
CN107565970B (zh) 一种基于特征识别的混合无损压缩方法及装置
CN110649925B (zh) 划分的增益形状向量编码
CN111835364B (zh) 一种极化码的低复杂度神经bp译码方法
CN114726926B (zh) 一种用于拉普拉斯信源的自适应变长编码方法
RU2611249C1 (ru) Модификатор энтропии и способ его использования
WO2020015481A1 (zh) 数据压缩方法及装置和计算机可读存储介质
CN116915873B (zh) 基于物联网技术的高速电梯运行数据快速传输方法
US8754792B2 (en) System and method for fixed rate entropy coded scalar quantization
WO2008110116A1 (en) A implementing method and device for quantizing encoding and decoding a grid vector
KR20100009032A (ko) 데이터 무손실 압축 방법
CN103746701A (zh) 一种用于Rice无损数据压缩的快速编码选项选择方法
Martinez et al. Marlin: A high throughput variable-to-fixed codec using plurally parsable dictionaries
CN109831211B (zh) 使用控制符的半动态数据压缩算法
CN113096673B (zh) 基于生成对抗网络的语音处理方法及系统
Huang et al. Lossless compression algorithm for multi-source sensor data research
CN109698704B (zh) 比对型基因测序数据解压方法、系统及计算机可读介质
JP4883455B2 (ja) 符号化装置、符号化方法および符号化プログラム
Moon et al. Local Non-linear Quantization for Neural Network Compression in MPEG-NNR
Peric et al. Asymmetrical two-level scalar quantizer with extended Huffman coding for compression of Laplacian source

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19836985

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19836985

Country of ref document: EP

Kind code of ref document: A1