WO2023109019A1 - 一种基于deflate霍夫曼编码的动态码表生成装置和方法 - Google Patents

一种基于deflate霍夫曼编码的动态码表生成装置和方法 Download PDF

Info

Publication number
WO2023109019A1
WO2023109019A1 PCT/CN2022/095386 CN2022095386W WO2023109019A1 WO 2023109019 A1 WO2023109019 A1 WO 2023109019A1 CN 2022095386 W CN2022095386 W CN 2022095386W WO 2023109019 A1 WO2023109019 A1 WO 2023109019A1
Authority
WO
WIPO (PCT)
Prior art keywords
code
length
data
code length
circuit
Prior art date
Application number
PCT/CN2022/095386
Other languages
English (en)
French (fr)
Inventor
王振
刘宇豪
张永兴
王明明
Original Assignee
苏州浪潮智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 苏州浪潮智能科技有限公司 filed Critical 苏州浪潮智能科技有限公司
Publication of WO2023109019A1 publication Critical patent/WO2023109019A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
    • H03M7/40—Conversion to or from variable length codes, e.g. Shannon-Fano code, Huffman code, Morse code

Definitions

  • the present application relates to the field of computers, and more specifically relates to a device and method for generating a dynamic code table based on deflate Huffman coding.
  • Deflate is a lossless compression standard, which is a combination of LZ77 coding and Huffman coding.
  • the Huffman coding used in Deflate is "paradigm Huffman coding".
  • Paradigm Huffman coding was first proposed by Schwartz, which is a subset of Huffman coding. The central idea is to use some mandatory conventions to reconstruct the structure of the Huffman coding tree with only a small amount of data.
  • Huffman coding is the secondary compression of data after LZ77 compression, and further improves the compression rate. It is a variable-length coding for data processing by counting the number of occurrences of characters, and constructing a Huffman binary tree using the number of occurrences of characters. , encode from the leaf node of the tree to the root node, the more characters that appear, the shorter codes are replaced, and the characters that appear less frequently are replaced by longer codes, which can reduce the overall size of the data and achieve compression the goal of.
  • Huffman coding in Deflate format is divided into static Huffman coding and dynamic Huffman coding.
  • the key to dynamic Huffman coding is to generate corresponding dynamic coding code tables in real time through Huffman coding algorithms according to different data blocks.
  • the generation of the dynamic code table accounts for about 80% of the entire compression performance. Therefore, the efficient implementation of the dynamic code table is very important to the overall compression performance.
  • the generation of the Huffman coding dynamic code table is divided into code word length generation (CL) and code table (code table) generation.
  • CL code word length generation
  • code table code table
  • the traditional code table generation process takes about half of the dynamic code table generation time of Huffman coding.
  • the purpose of the embodiment of the present application is to propose a device and method for generating a dynamic code table based on deflate Huffman coding.
  • the time for generating the dynamic code table can be greatly reduced, and the time for generating the dynamic code table can be increased.
  • the versatility of the device can increase the flexibility of the device.
  • an aspect of the embodiments of the present application provides a dynamic code table generation device based on deflate Huffman coding, including:
  • a code length statistics circuit configured to calculate the number of code lengths of each code in the data to be encoded
  • a minimum encoding circuit configured to receive the data of the code length statistics circuit, and calculate the minimum encoding corresponding to each code length based on the number of code lengths of each code in the data to be encoded;
  • a code table generating circuit configured to receive data from the minimum encoding circuit, and generate a dynamic code table based on the minimum code corresponding to each code length in the data to be encoded, and store the dynamic code table in the register.
  • the control circuit is connected to the code length statistical circuit, the minimum encoding circuit and the code table generating circuit respectively, and the control circuit is configured to send an enable signal to the minimum encoding circuit after receiving the completion signal of the code length statistics circuit, and receive the completion signal of the minimum encoding circuit The enable signal is sent to the code table generation circuit after the signal.
  • the code length statistics circuit includes:
  • the comparator is configured to set each comparator to a fixed and different code length value as a comparison reference value, and the literal (text character to be compressed) and length (the length of the repeated text character) in the data to be encoded are respectively
  • the code length in the corresponding calculation interval is input to the comparator corresponding to the code length value, and each comparator marks the same and effective code length in parallel using a marking signal;
  • a statistical unit includes an adder, and the adder is configured to add the number of occurrences of each code length marked the same to obtain the number of code lengths of each code.
  • the minimum encoding circuit includes:
  • each data input unit is configured to input the number of code lengths corresponding to each code length
  • each adding unit the first input of each adding unit is connected to each data input unit;
  • the data input signal of each flip-flop is connected to the output terminal of the adding unit of the same level, the data output signal is connected to the second input terminal of the adding unit of the next level and outputs the minimum number of each code length.
  • the code table generation circuit includes:
  • the comparator is configured to set each comparator to a fixed and different code length value as a comparison reference value, respectively input the code length in the calculation interval corresponding to literal and length in the data to be encoded to the code length value corresponding in the comparator;
  • a counter the counter is configured such that the input terminal is connected to the output terminal of the comparator to count the number of occurrences of the code length value corresponding to each comparator;
  • the addition unit the first input of the addition unit inputs the minimum code of the code length, the second input is connected to the output of the counter, and the output outputs a code table of each code length.
  • the range of the code length is 1-15.
  • the calculation range corresponding to literal and length in the data to be encoded is 0-285.
  • Another aspect of the embodiments of the present application also provides a method for generating a dynamic code table based on deflate Huffman coding, the method comprising:
  • calculating the number of code lengths of each code in the data to be encoded includes:
  • the number of occurrences of each code sub-length in the statistical table is counted.
  • the range of the code length is 1-15.
  • Fig. 1 is a schematic diagram of a dynamic code table generation device based on deflate Huffman coding according to one or more embodiments of the present application;
  • FIG. 2 is a schematic diagram of a code length statistics circuit according to one or more embodiments of the present application
  • FIG. 3 is a schematic diagram of a hardware circuit of a statistical unit according to one or more embodiments of the present application.
  • FIG. 4 is a schematic diagram of a minimum encoding circuit according to one or more embodiments of the present application.
  • FIG. 5 is a schematic diagram of a code table generating circuit according to one or more embodiments of the present application.
  • Fig. 6 is a schematic flowchart of a method for generating a dynamic code table based on deflate Huffman coding according to one or more embodiments of the present application.
  • the first aspect of the embodiments of the present application proposes one or more embodiments of a device for generating a dynamic code table based on deflate Huffman coding.
  • Figure 1 shows a schematic diagram of the device.
  • the device may include:
  • a code length statistics circuit configured to calculate the number of code lengths of each code in the data to be encoded
  • a minimum encoding circuit configured to receive the data of the code length statistics circuit, and calculate the minimum encoding corresponding to each code length based on the number of code lengths of each code in the data to be encoded;
  • a code table generating circuit configured to receive data from the minimum encoding circuit, and generate a dynamic code table based on the minimum code corresponding to each code length in the data to be encoded, and store the dynamic code table in the register.
  • the device completely uses hardware circuit design, and the key circuit structure adopts parallel pipeline design, which greatly reduces the generation time of dynamic code table.
  • the device can be configured to select the number of adding circuit stages according to the frequency of the application system, which increases the versatility of the device.
  • the device is composed of a control circuit, a code length statistics circuit, a minimum encoding circuit and a code table generating circuit.
  • the entire calculation process of the device is controlled by the control circuit, and the minimum encoding circuit needs to wait for the code length statistics circuit to complete the calculation before starting to work.
  • the code table generating circuit needs to wait for the calculation of the minimum encoding circuit to complete and work.
  • the dynamic code table hardware device of Literal and length code word is exactly the same as the dynamic code table hardware device of distance (recurring text character and the distance between the text character that occurs for the first time) code word, can be reused mutually, through transmission Refer to the implementation.
  • the time for generating the dynamic code table can be greatly reduced, the versatility of the device can be increased, and the flexibility of the device can be increased.
  • the control circuit is connected to the code length statistical circuit, the minimum encoding circuit and the code table generating circuit respectively, and the control circuit is configured to send an enable signal to the minimum encoding circuit after receiving the completion signal of the code length statistics circuit, and receive the completion signal of the minimum encoding circuit The enable signal is sent to the code table generation circuit after the signal.
  • the control circuit includes CL_NUMBER unit, SMALL_CODE unit and CODE_TABLE unit.
  • the CL_NUMBER unit controls the code length statistics circuit to count the number of the same code length, which can support the interval code [0, 285] corresponding to the literal and length code words and the interval code [0, 29] corresponding to the distance code word and the travel code
  • the code length of the run-length head [0, 18] is used to count the number of the same code length
  • the SMALL_CODE unit controls the minimum encoding circuit to calculate the minimum encoding value of each code length
  • the CODE_TABLE unit controls the code table generation circuit to calculate the corresponding literal and length codewords
  • the code table of the interval code [0, 285] and the code table of the interval code [0, 29] corresponding to the distance code word.
  • the code length statistics circuit includes:
  • the comparator is configured to set each comparator to a fixed and different code length value as a comparison reference value, respectively input the code length in the calculation interval corresponding to literal and length in the data to be encoded to the code length value corresponding
  • each comparator uses a marking signal to mark the same and effective code length in parallel;
  • the statistical unit includes an adder, and the adder is configured to add the number of occurrences of each code length marked the same to obtain the number of code lengths of each code.
  • the code length statistics circuit mainly calculates the number of code lengths of literal, length codewords and distance codewords. This circuit adopts a parallel pipeline structure, which accelerates the speed of code length statistics and can be output in the fastest two clock cycles. statistical results.
  • the code lengths of the 286 codeword intervals (codeword interval range 0-285) of literal and length are judged, and the same and effective code lengths (code length range 1-15) are respectively marked in parallel with a marker signal.
  • each comparator corresponds to a code length range, and an array flag[0][i]-flag[14][i] is established.
  • One number is the code length range, the latter number is the value of the code sub-interval, that is, the value of 0-285, and the arrays are the flag signals with code length 1 to code length 15 respectively.
  • This process uses parallel circuits and can be completed in one clock cycle.
  • the statistic unit performs statistics in parallel according to the flag signals of each code length according to different code lengths, and obtains the number of different code lengths.
  • the statistics unit realizes the effective number statistics of 286 flag signals, and can choose to use a 1-stage addition circuit, a 3-stage addition circuit or a 4-stage addition circuit according to the configuration signal. According to the system frequency, configure the hardware design method using parallel addition circuits of different stages, calculate the number of 286 marker signals for each code length, and count the number of each code length.
  • the minimum encoding circuit includes:
  • each data input unit is configured to input the number of code lengths corresponding to each code length
  • each adding unit the first input of each adding unit is connected to each data input unit;
  • the minimum encoding circuit uses the pipeline method to calculate the minimum encoding value with a code length of 1 to 15. Since the minimum encoding value of each codeword length is affected by the previous codeword length, this circuit needs 15 clock cycles to calculate each code length. The minimum coded value for .
  • the minimum code with a code length of 1 is fixed at 0, so the circuit first calculates the minimum code with a code length of 2, and a 15-level calculation circuit is set in the circuit, corresponding to 15 code lengths, and the addition unit Input the number of the previous code length at the first input terminal, input the minimum code of the previous code length at the second input terminal, the combination of the two is the minimum code of the next code length, this process requires one clock cycle, each clock The cycle calculates the minimum code of each code length in turn and stores it.
  • the code table generation circuit includes:
  • the comparator is configured to set each comparator to a fixed and different code length value as a comparison reference value, respectively input the code length in the calculation interval corresponding to literal and length in the data to be encoded to the code length value corresponding in the comparator;
  • a counter the counter is configured such that the input terminal is connected to the output terminal of the comparator to count the number of occurrences of the code length value corresponding to each comparator;
  • the addition unit the first input of the addition unit inputs the minimum code of the code length, the second input is connected to the output of the counter, and the output outputs a code table of each code length.
  • the dynamic code table of Literal, length code word and Distance code word can be generated through the code table generation circuit.
  • Figure 5 shows the code table generation circuit. Starting from code word 0, the corresponding code length is judged at each clock cycle, and at the same time, the code length counter corresponding to the code length of 1 to 15 is counted up, indicating that the corresponding code length is How many times does it appear. Finally, the minimum code of the code length is added to the counter value to obtain the corresponding code word code value. In the next clock cycle, the encoding values from codeword 1 to codeword 285 are calculated sequentially according to this method, and finally the dynamic code table of Literal, length codeword and Distance codeword is obtained.
  • the range of the code length is 1-15.
  • the calculation range corresponding to literal and length in the data to be encoded is 0-285.
  • the time for generating the dynamic code table can be greatly reduced, the versatility of the device can be increased, and the flexibility of the device can be increased.
  • the second aspect of the embodiment of the present application proposes a method for generating a dynamic code table based on deflate Huffman coding, as shown in Figure 6, the method includes:
  • S1 calculates the number of code lengths of each code in the data to be encoded
  • S2 calculates the minimum encoding corresponding to each code length based on the number of code lengths of each code in the data to be encoded;
  • S3 generates a dynamic code table based on the minimum code corresponding to each code length in the data to be coded, and stores the dynamic code table in a register.
  • calculating the number of code lengths of each code in the data to be encoded includes:
  • the number of occurrences of each code sub-length in the statistical table is counted.
  • the range of the code length is 1-15.
  • the code length statistical circuit is configured to calculate the code length quantity of each code in the data to be encoded; the minimum encoding circuit, the minimum encoding The circuit is configured to receive the data of the code length statistical circuit, and calculate the minimum code corresponding to each code length based on the code length quantity of each code in the data to be encoded; the code table generation circuit, the code table generation circuit is configured to receive the data of the minimum code circuit, And the dynamic code table is generated based on the minimum code corresponding to each code length in the data to be encoded, and the technical scheme of storing the dynamic code table in the register can greatly reduce the time for generating the dynamic code table, increase the versatility of the device, and Increase the flexibility of the device.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)

Abstract

本申请提供了一种基于deflate霍夫曼编码的动态码表生成装置和方法，装置包括：码长统计电路，码长统计电路配置为计算待编码数据中各个码子的码长数量；最小编码电路，最小编码电路配置为接收码长统计电路的数据，并基于待编码数据中各个码子的码长数量计算各个码长对应的最小编码；码表生成电路，码表生成电路配置为接收最小编码电路的数据，并基于待编码数据中各个码长对应的最小编码生成动态码表，并将动态码表存储到寄存器中。

Description

一种基于deflate霍夫曼编码的动态码表生成装置和方法
相关申请的交叉引用
本申请要求于2021年12月17日提交中国专利局，申请号为202111551901.6，发明名称为“一种基于deflate霍夫曼编码的动态码表生成装置和方法”的中国专利申请的优先权和权益，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机领域，并且更具体地涉及一种基于deflate霍夫曼编码的动态码表生成装置和方法。
背景技术
随着时代的发展，人们使用和获取数据信息不再单纯的来自于书本，杂志，报刊等。同时伴随着移动互联，5G，社交网络，物联网，大数据，人工智能(AI)，云计算，安全监控等技术的成熟与发展，使得各种数据正迅速的膨胀并变大，呈爆炸性增长的趋势。5G正在加速普及，越来越多的设备和便捷的联网条件给存储市场带来了前所未有的机会。随着数据量的急剧增长，不断对数据存储系统提出挑战。在所有这些技术当中，云计算技术相当于人的大脑，提供着大容量的数据存储以及高效的计算方式，计算资源和存储资源也更加集中。但是同时使得产生的海量的数据指数级增长，对现有的存储设备带来了巨大的压力。如何能够高效的传输和存储海量数据，同时对CPU不造成过大负荷影响的问题成为关键。因此，高效安全的数据压缩技术成为降低存储成本，节省存储资源有效方法。同时如果将压缩和解压缩任务交给专用硬件电路处理，会大幅减少CPU的负荷，并且在提升存储性能方面非常可观。
Deflate是一种无损压缩标准，是由LZ77编码和霍夫曼编码组合的编码形式。Deflate中使用的霍夫曼编码就是“范式霍夫曼编码”，范式霍夫曼编码最早由Schwartz提出，它是霍夫曼编码的一个子集。其中心思想是使用某些强制的约定，仅通过很少的数据便能重构出霍夫曼编码树的结构。
霍夫曼编码是对数据经过LZ77压缩后的二次压缩，又进一步提高了压缩率，它是通过统计字符出现次数，对数据处理的可变长编码，利用字符出现的次数构造霍夫曼二叉树，从树的叶子节点到根节点进行编码，出现的越多的字符让短一些的码替换，同时出现次数少的字符被长一些的码替换掉，可以实现数据的总体大小减小，达到压缩的目 的。
Deflate格式的霍夫曼编码分为静态霍夫曼编码和动态霍夫曼编码，动态霍夫曼编码的关键就是根据不同的数据块通过霍夫曼编码算法实时产生对应的动态编码码表。然而动态码表的生成约占整个压缩性能的百分之八十，因此，动态码表的高效实现对整个压缩性能影响至关重要。其中，霍夫曼编码动态码表的生成分为码字长度生成(CL)和码表(code table)生成。然而，传统的码表生成过程占用了约一半的霍夫曼编码动态码表生成时间。
发明内容
有鉴于此，本申请实施例的目的在于提出一种基于deflate霍夫曼编码的动态码表生成装置和方法，通过使用本申请的技术方案，能够大幅减少了动态码表生成的时间，能够增加装置的通用性，能够增加装置的灵活性。
基于上述目的，本申请的实施例的一个方面提供了一种基于deflate霍夫曼编码的动态码表生成装置，包括：
码长统计电路，码长统计电路配置为计算待编码数据中各个码子的码长数量；
最小编码电路，最小编码电路配置为接收码长统计电路的数据，并基于待编码数据中各个码子的码长数量计算各个码长对应的最小编码；
码表生成电路，码表生成电路配置为接收最小编码电路的数据，并基于待编码数据中各个码长对应的最小编码生成动态码表，并将动态码表存储到寄存器中。
根据本申请的一或多个实施例，还包括：
控制电路，控制电路分别连接到码长统计电路、最小编码电路和码表生成电路，控制电路配置为接收码长统计电路的完成信号后向最小编码电路发送使能信号，接收最小编码电路的完成信号后向码表生成电路发送使能信号。
根据本申请的一或多个实施例，码长统计电路包括：
比较器，比较器配置为将每个比较器设定为固定且不同的码长值作为比较基准值，分别将待编码数据中literal(待压缩文本字符)和length(重复出现的文本字符长度)对应的计算区间中的码长输入到码长值对应的比较器中，各个比较器分别并行对相同并有效的码长使用标记信号进行标记；
统计单元，统计单元包括加法器，加法器配置为将标记相同的每个码长出现的个数进行相加以得到各个码子的码长数量。
根据本申请的一或多个实施例，最小编码电路包括：
数据输入单元，每个数据输入单元配置为输入每个码长对应的码长数量；
相加单元，每个相加单元的第一输入端连接到每个数据输入单元；
触发器，每个触发器的数据输入信号连接到同级相加单元的输出端，数据输出信号连接到下一级相加单元的第二输入端并输出每个码长的最小编号。
根据本申请的一或多个实施例，码表生成电路包括：
比较器，比较器配置为将每个比较器设定为固定且不同的码长值作为比较基准值，分别将待编码数据中literal和length对应的计算区间中的码长输入到码长值对应的比较器中；
计数器，计数器配置为输入端连接到比较器的输出端以统计每个比较器对应的码长值出现的次数；
相加单元，相加单元的第一输入端输入码长的最小编码，第二输入端连接到计数器的输出端，输出端输出每个码长的码表。
根据本申请的一或多个实施例，码长的区间范围为1-15。
根据本申请的一或多个实施例，待编码数据中literal和length对应的计算区间为0-285。
本申请的实施例的另一个方面，还提供了一种基于deflate霍夫曼编码的动态码表生成的方法，方法包括：
计算待编码数据中各个码子的码长数量；
基于待编码数据中各个码子的码长数量计算各个码长对应的最小编码；
基于待编码数据中各个码长对应的最小编码生成动态码表，并将动态码表存储到寄存器中。
根据本申请的一或多个实施例，计算待编码数据中各个码子的码长数量包括：
获取待编码的Literal值对应的Huffman码字长度和Length值对应的码字长度的统计表；
基于获取到的统计表统计每个码子长度在统计表中出现的次数。
根据本申请的一或多个实施例，码长的区间范围为1-15。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有 技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的实施例。
图1为根据本申请一或多个实施例的基于deflate霍夫曼编码的动态码表生成装置的示意图；
图2为根据本申请一或多个实施例的码长统计电路的示意图；
图3为根据本申请一或多个实施例的统计单元硬件电路的示意图；
图4为根据本申请一或多个实施例的最小编码电路的示意图；
图5为根据本申请一或多个实施例的码表生成电路的示意图；
图6为根据本申请一或多个实施例的基于deflate霍夫曼编码的动态码表生成方法的示意流程图。
具体实施方式
以下描述了本公开的实施例。然而，应该理解，所公开的实施例仅仅是示例，并且其他实施例可以采取各种替代形式。附图不一定按比例绘制；某些功能可能被夸大或最小化以显示特定部件的细节。因此，本文公开的具体结构和功能细节不应被解释为限制性的，而仅仅是作为用于教导本领域技术人员以各种方式使用本申请的代表性基础。如本领域普通技术人员将理解的，参考任何一个附图所示出和描述的各种特征可以与一个或多个其他附图中所示的特征组合以产生没有明确示出或描述的实施例。所示特征的组合为典型应用提供了代表性实施例。然而，与本公开的教导相一致的特征的各种组合和修改对于某些特定应用或实施方式可能是期望的。
基于上述目的，本申请的实施例的第一个方面，提出了一种基于deflate霍夫曼编码的动态码表生成装置的一或多个实施例。图1示出的是该装置的示意图。
如图1中所示，该装置可以包括：
码长统计电路，码长统计电路配置为计算待编码数据中各个码子的码长数量；
最小编码电路，最小编码电路配置为接收码长统计电路的数据，并基于待编码数据中各个码子的码长数量计算各个码长对应的最小编码；
码表生成电路，码表生成电路配置为接收最小编码电路的数据，并基于待编码数据中各个码长对应的最小编码生成动态码表，并将动态码表存储到寄存器中。
该装置完全使用硬件电路设计，关键电路结构采用并行流水设计，大幅减少了动态 码表生时间。同时此装置可以根据应用系统的频率，可配置选择加法电路级数，增加了该装置通用性。此装置由控制电路、码长统计电路、最小编码电路和码表生成电路组成。此装置的整个计算流程受控制电路控制，最小编码电路需要等待码长统计电路完成计算后开始工作。同样，码表生成电路需要等待最小编码电路计算完成后工作。等待码表生成电路计算完成后，将动态的码表锁存在寄存器中，同时此装置输出完成标志。Literal和length码字的动态码表硬件装置和distance(重复出现的文本字符和第一次出现的文本字符之间的距离)码字的动态码表硬件装置完全相同，可相互复用，通过传参实现。
通过本申请的一些技术方案，能够大幅减少了动态码表生成的时间，能够增加装置的通用性，能够增加装置的灵活性。
在本申请的一或多个优选实施例中，还包括：
控制电路，控制电路分别连接到码长统计电路、最小编码电路和码表生成电路，控制电路配置为接收码长统计电路的完成信号后向最小编码电路发送使能信号，接收最小编码电路的完成信号后向码表生成电路发送使能信号。
控制电路包括CL_NUMBER单元、SMALL_CODE单元和CODE_TABLE单元。CL_NUMBER单元控制码长统计电路进行各相同码长的数量统计，可支持对literal和length码字对应的区间码[0，285]和distance码字对应的区间码[0，29]和游成编码的游程头[0，18]的码长进行相同码长的数量统计，SMALL_CODE单元控制最小编码电路计算每个码长的最小编码值，CODE_TABLE单元控制码表生成电路计算literal和length码字对应的区间码[0，285]的码表和distance码字对应的区间码[0，29]的码表。
在本申请的一或多个优选实施例中，码长统计电路包括：
比较器，比较器配置为将每个比较器设定为固定且不同的码长值作为比较基准值，分别将待编码数据中literal和length对应的计算区间中的码长输入到码长值对应的比较器中，各个比较器分别并行对相同并有效的码长使用标记信号进行标记；
统计单元，统计单元包括加法器，加法器配置为将标记相同的每个码长出现的个数进行相加以得到各个码子的码长数量。如图2所示，码长统计电路主要计算literal、length码字和distance码字的各码长数量，此电路采用并行流水结构，加速了码长统计速度，最快两个时钟周期即可输出统计结果。对literal、length的286个码字区间(码字区间范围0～285)的码长进行判断，分别并行对相同并有效的码长(码长范围1-15)使用标记信号对其标记。如2图中所示，设置了0-14共15个比较器，每个比较器对应 一个码长范围，建立数组flag[0][i]-flag[14][i]，数组中的前一个数字为码长范围，后一个数字为码子区间的值，即0-285的值，数组分别为码长1到码长15的标志信号。这个过程使用并行电路，一个时钟周期即可完成。接下来统计单元根据各码长的标志信号，根据不同的码长并行对其进行统计，得到不同码长的数量。
如图3所示，统计单元实现了286个标志信号的有效个数统计，根据配置信号可以选择使用1级加法电路、3级加法电路或4级加法电路。可根据系统频率，配置采用不同级数并行加法电路的硬件设计方式，计算出每个码长286个标志信号的数量，统计出每个码长的数量。
在本申请的一或多个优选实施例中，最小编码电路包括：
数据输入单元，每个数据输入单元配置为输入每个码长对应的码长数量；
相加单元，每个相加单元的第一输入端连接到每个数据输入单元；
触发器，每个触发器的数据输入信号连接到同级相加单元的输出端，数据输出信号连接到下一级相加单元的第二输入端并输出每个码长的最小编号。最小编码电路采用流水方式计算码长为1到15最小编码值，由于每个码字长度的最小编码值受前一个码字长度影响，所以此电路需要15个时钟周期可以计算出每个码长的最小编码值。如图4所示，码长为1的最小编码固定为0，因此电路开始先计算码长为2的最小编码，该电路中设置了15级的计算电路，对应15个码长，相加单元的第一个输入端输入上一个码长的数量，第二输入端输入上一个码长的最小编码，两者组合即为下一个码长的最小编码，这个过程需要一个时钟周期，每个时钟周期依次计算出每个码长的最小编码并存储起来。
在本申请的一或多个优选实施例中，码表生成电路包括：
比较器，比较器配置为将每个比较器设定为固定且不同的码长值作为比较基准值，分别将待编码数据中literal和length对应的计算区间中的码长输入到码长值对应的比较器中；
计数器，计数器配置为输入端连接到比较器的输出端以统计每个比较器对应的码长值出现的次数；
相加单元，相加单元的第一输入端输入码长的最小编码，第二输入端连接到计数器的输出端，输出端输出每个码长的码表。当获得码长1到15的最小编码后，通过码表生成电路，即可生成Literal、length码字和Distance码字的动态码表。图5所示为 码表生成电路，从码字0开始，每个时钟周期判断其对应的码长，同时对应码长为1到15的码长计数器进行加一计数，表示相应的码长是第几次出现。最后用该码长的最小编码和计数器值相加，得到对应的码字编码值。下一个时钟周期依次按照此方法计算码字1到码字285的编码值，最终得到Literal、length码字和Distance码字的动态码表。
在本申请的一或多个优选实施例中，码长的区间范围为1-15。
在本申请的一或多个优选实施例中，待编码数据中literal和length对应的计算区间为0-285。
通过本申请的技术方案，能够大幅减少了动态码表生成的时间，能够增加装置的通用性，能够增加装置的灵活性。
基于上述目的，本申请的实施例的第二个方面，提出了一种基于deflate霍夫曼编码的动态码表生成的方法，如图6所示，方法包括：
S1计算待编码数据中各个码子的码长数量；
S2基于待编码数据中各个码子的码长数量计算各个码长对应的最小编码；
S3基于待编码数据中各个码长对应的最小编码生成动态码表，并将动态码表存储到寄存器中。
在本申请的一或多个优选实施例中，计算待编码数据中各个码子的码长数量包括：
获取待编码的Literal值对应的Huffman码字长度和Length值对应的码字长度的统计表；
基于获取到的统计表统计每个码子长度在统计表中出现的次数。
在本申请的一或多个优选实施例中，码长的区间范围为1-15。
在一些实施方案中，基于deflate霍夫曼编码的动态码表生成装置，通过设置码长统计电路，码长统计电路配置为计算待编码数据中各个码子的码长数量；最小编码电路，最小编码电路配置为接收码长统计电路的数据，并基于待编码数据中各个码子的码长数量计算各个码长对应的最小编码；码表生成电路，码表生成电路配置为接收最小编码电路的数据，并基于待编码数据中各个码长对应的最小编码生成动态码表，并将动态码表存储到寄存器中的技术方案，能够大幅减少了动态码表生成的时间，能够增加装置的通用性，能够增加装置的灵活性。
尽管已经示出和描述了本申请的实施例，对于本领域的普通技术人员而言，可以理解在不脱离本申请的原理和精神的情况下可以对这些实施例进行多种变化、修改、替换 和变型，本申请的范围由所附权利要求及其等同物限定。
上述实施例，特别是任何“优选”实施例是实现的可能示例，并且仅为了清楚地理解本申请的原理而提出。可以在不脱离本文所描述的技术的精神和原理的情况下对上述实施例进行许多变化和修改。所有修改旨在被包括在本公开的范围内并且由所附权利要求保护。

Claims (10)

  1. 一种基于deflate霍夫曼编码的动态码表生成装置，其特征在于，包括：
    码长统计电路，所述码长统计电路配置为计算待编码数据中各个码子的码长数量；
    最小编码电路，所述最小编码电路配置为接收所述码长统计电路的数据，并基于待编码数据中各个码子的码长数量计算各个码长对应的最小编码；以及
    码表生成电路，所述码表生成电路配置为接收所述最小编码电路的数据，并基于待编码数据中各个码长对应的最小编码生成动态码表，并将所述动态码表存储到寄存器中。
  2. 根据权利要求1所述的装置，其特征在于，还包括：
    控制电路，所述控制电路分别连接到所述码长统计电路、所述最小编码电路和所述码表生成电路，所述控制电路配置为接收所述码长统计电路的完成信号后向所述最小编码电路发送使能信号，并接收所述最小编码电路的完成信号后向所述码表生成电路发送使能信号。
  3. 根据权利要求1或2所述的装置，其特征在于，所述码长统计电路包括：
    比较器，所述比较器配置为将每个比较器设定为固定且不同的码长值作为比较基准值，分别将待编码数据中literal和length对应的计算区间中的码长输入到码长值对应的比较器中，各个比较器分别并行对相同并有效的码长使用标记信号进行标记；以及
    统计单元，所述统计单元包括加法器，所述加法器配置为将标记相同的每个码长出现的个数进行相加以得到各个码子的码长数量。
  4. 根据权利要求3所述的装置，其特征在于，所述的待编码数据中literal和length对应的计算区间为0–285。
  5. 根据权利要求1-4任一所述的装置，其特征在于，所述最小编码电路包括：
    数据输入单元，每个所述数据输入单元配置为输入每个码长对应的码长数量；
    相加单元，每个所述相加单元的第一输入端连接到每个所述数据输入单元；以及
    触发器，每个所述触发器的数据输入信号连接到同级所述相加单元的输出端，数据输出信号连接到下一级所述相加单元的第二输入端并输出每个码长的最小编号。
  6. 根据权利要求1-5任一所述的装置，其特征在于，所述码表生成电路包括：
    比较器，所述比较器配置为将每个比较器设定为固定且不同的码长值作为比较基准值，分别将待编码数据中literal和length对应的计算区间中的码长输入到码长值对应的比较器中；
    计数器，所述计数器配置为输入端连接到所述比较器的输出端以统计每个比较器对应的码长值出现的次数；以及
    相加单元，所述相加单元的第一输入端输入码长的最小编码，第二输入端连接到所述计数器的输出端，输出端输出每个码长的码表。
  7. 根据权利要求1-6任一所述的装置，其特征在于，码长的区间范围为1-15。
  8. 一种基于deflate霍夫曼编码的动态码表生成的方法，包括：
    计算待编码数据中各个码子的码长数量；
    基于待编码数据中各个码子的码长数量计算各个码长对应的最小编码；以及
    基于待编码数据中各个码长对应的最小编码生成动态码表，并将所述动态码表存储到寄存器中。
  9. 根据权利要求8所述的方法，其特征在于，所述的计算待编码数据中各个码子的码长数量，包括：
    获取待编码的Literal值对应的Huffman码字长度和Length值对应的码字长度的统计表；以及
    基于获取到的统计表统计每个码子长度在所述统计表中出现的次数。
  10. 根据权利要求8或9所述的方法，其特征在于，码长的区间范围为1-15。
PCT/CN2022/095386 2021-12-17 2022-05-26 一种基于deflate霍夫曼编码的动态码表生成装置和方法 WO2023109019A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111551901.6A CN113965207B (zh) 2021-12-17 2021-12-17 一种基于deflate霍夫曼编码的动态码表生成装置和方法
CN202111551901.6 2021-12-17

Publications (1)

Publication Number Publication Date
WO2023109019A1 true WO2023109019A1 (zh) 2023-06-22

Family

ID=79473239

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/095386 WO2023109019A1 (zh) 2021-12-17 2022-05-26 一种基于deflate霍夫曼编码的动态码表生成装置和方法

Country Status (2)

Country Link
CN (1) CN113965207B (zh)
WO (1) WO2023109019A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113965207B (zh) * 2021-12-17 2022-03-15 苏州浪潮智能科技有限公司 一种基于deflate霍夫曼编码的动态码表生成装置和方法
CN114401013B (zh) * 2022-03-25 2022-06-21 苏州浪潮智能科技有限公司 一种适用于deflate格式的Huffman压缩编码字符频率统计装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102545910A (zh) * 2010-12-30 2012-07-04 无锡华润矽科微电子有限公司 一种jpeg霍夫曼解码电路及其解码方法
CN104283568A (zh) * 2013-07-12 2015-01-14 中国科学院声学研究所 一种基于部分霍夫曼树的数据压缩编码方法
CN109889205A (zh) * 2019-04-03 2019-06-14 杭州嘉楠耘智信息科技有限公司 编码方法及系统、解码方法及系统、编解码方法及系统
CN113965207A (zh) * 2021-12-17 2022-01-21 苏州浪潮智能科技有限公司 一种基于deflate霍夫曼编码的动态码表生成装置和方法

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100209877B1 (ko) * 1994-11-26 1999-07-15 윤종용 복수개의 허프만부호테이블을 이용한 가변장부호화장치 및 복호화장치
CN100525450C (zh) * 2007-03-13 2009-08-05 北京中星微电子有限公司 一种实现霍夫曼译码的方法和装置
US8824569B2 (en) * 2011-12-07 2014-09-02 International Business Machines Corporation High bandwidth decompression of variable length encoded data streams
US9832287B2 (en) * 2014-03-28 2017-11-28 Comtech Ef Data Corp. System and method for dynamic Huffman decoding
US9337862B2 (en) * 2014-06-09 2016-05-10 Tidal Systems, Inc. VLSI efficient Huffman encoding apparatus and method
US9509336B1 (en) * 2015-05-11 2016-11-29 Via Alliance Semiconductor Co., Ltd. Hardware data compressor that pre-huffman encodes to decide whether to huffman encode a matched string or a back pointer thereto
US9515678B1 (en) * 2015-05-11 2016-12-06 Via Alliance Semiconductor Co., Ltd. Hardware data compressor that directly huffman encodes output tokens from LZ77 engine
US9584156B1 (en) * 2015-11-17 2017-02-28 International Business Mahcines Corporation Creating a dynamic Huffman table
US10693493B1 (en) * 2019-02-14 2020-06-23 International Business Machines Corporation Reducing latch count to save hardware area for dynamic Huffman table generation
CN111884660B (zh) * 2020-07-13 2022-06-17 山东云海国创云计算装备产业创新中心有限公司 一种哈夫曼编码设备

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102545910A (zh) * 2010-12-30 2012-07-04 无锡华润矽科微电子有限公司 一种jpeg霍夫曼解码电路及其解码方法
CN104283568A (zh) * 2013-07-12 2015-01-14 中国科学院声学研究所 一种基于部分霍夫曼树的数据压缩编码方法
CN109889205A (zh) * 2019-04-03 2019-06-14 杭州嘉楠耘智信息科技有限公司 编码方法及系统、解码方法及系统、编解码方法及系统
CN113965207A (zh) * 2021-12-17 2022-01-21 苏州浪潮智能科技有限公司 一种基于deflate霍夫曼编码的动态码表生成装置和方法

Also Published As

Publication number Publication date
CN113965207B (zh) 2022-03-15
CN113965207A (zh) 2022-01-21

Similar Documents

Publication Publication Date Title
WO2023109019A1 (zh) 一种基于deflate霍夫曼编码的动态码表生成装置和方法
CN106560010B (zh) Vlsi高效霍夫曼编码设备和方法
US10187081B1 (en) Dictionary preload for data compression
US11923875B2 (en) Sorting network-based dynamic Huffman encoding method, apparatus and device
WO2021027487A1 (zh) 一种编码方法及相关设备
CN101534124B (zh) 一种用于短小自然语言的压缩算法
CN115189696A (zh) 一种基于Huffman解码表的硬件压缩解压方法
CN116827354B (zh) 一种文件数据分布式存储管理系统
Sari et al. The implementation of timestamp, bitmap and rake algorithm on data compression and data transmission from iot to cloud
CN112332854A (zh) 霍夫曼编码的硬件实现方法、装置及存储介质
CN104682966B (zh) 列表数据的无损压缩方法
Ambadekar et al. Advanced data compression using J-bit Algorithm
Tiwari et al. Aggregated Deflate-RLE compression technique for body sensor network
Patel et al. Survey of lossless data compression algorithms
Ginzburg et al. Short Message Compression Scheme for Wireless Sensor Networks
CN113824449A (zh) 一种静态霍夫曼并行编码方法、系统、存储介质及设备
CN106559085A (zh) 一种范式哈夫曼解码方法及其装置
Huang et al. Lossless compression algorithm for multi-source sensor data research
CN102891730A (zh) 基于bcd码卫星短报文的编码方法和装置
Gagie On the value of multiple read/write streams for data compression
CN114401013B (zh) 一种适用于deflate格式的Huffman压缩编码字符频率统计装置
Ali et al. Efficient parallel compression and decompression for large XML files.
CN114665887B (zh) 一种基于整体压缩的json字符串数据压缩方法
Chakraborty et al. A Dictionary based Efficient Text Compression Technique using Replacement Strategy
Jianjun et al. Research and Improvement of Huffman Compression Method Based on Linear Linked Forest

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22905787

Country of ref document: EP

Kind code of ref document: A1