WO2021012278A1 - Procédé de traitement de données, système, codeur et décodeur - Google Patents

Procédé de traitement de données, système, codeur et décodeur Download PDF

Info

Publication number
WO2021012278A1
WO2021012278A1 PCT/CN2019/097771 CN2019097771W WO2021012278A1 WO 2021012278 A1 WO2021012278 A1 WO 2021012278A1 CN 2019097771 W CN2019097771 W CN 2019097771W WO 2021012278 A1 WO2021012278 A1 WO 2021012278A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
code stream
data block
residual
block
Prior art date
Application number
PCT/CN2019/097771
Other languages
English (en)
Chinese (zh)
Inventor
阮肇夏
赵文军
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to CN201980031021.0A priority Critical patent/CN112119593A/zh
Priority to PCT/CN2019/097771 priority patent/WO2021012278A1/fr
Publication of WO2021012278A1 publication Critical patent/WO2021012278A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
    • H03M7/3053—Block-companding PCM systems
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/423—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation characterised by memory arrangements
    • H04N19/426—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation characterised by memory arrangements using memory downsizing methods

Definitions

  • This application relates to the technical field of data processing, and in particular to a data processing method, system, encoder and decoder.
  • CNN Convolutional Neural Networks, convolutional neural network
  • weight data is the main parameter
  • the parameter scale is large.
  • the medium occupies high bandwidth, and requires high hardware computing and storage capabilities, which in turn limits the application of neural network models.
  • one of the objectives of the present invention is to provide a data processing method, system, encoder and decoder to achieve effective compression of the data to be encoded.
  • an embodiment of the present invention provides a data processing method, including:
  • an embodiment of the present invention provides a data processing method, including:
  • Parsing the data to be decoded determining a target encoding method corresponding to the data to be decoded among a plurality of preset first encoding methods, and determining a corresponding target decoding method according to the target encoding method;
  • the data to be decoded is decoded by the target decoding mode to generate decoded data.
  • an embodiment of the present invention provides a data processing method, including:
  • the residual value and the reference data are written into the coded stream data.
  • an embodiment of the present invention provides a data processing method, including:
  • an embodiment of the present invention provides a data processing method, which is applied to a configurator, a memory, and a processor connected in sequence, including:
  • the processor reads the compressed code stream in the memory and performs decompression processing on the compressed code stream to reduce the read bandwidth of the memory.
  • an embodiment of the present invention provides a data processing system, including: a configurator, a memory, and a processor connected in sequence;
  • the configurator is used to compress and encode a data block to be encoded formed by preset data to generate a compressed code stream, and write the compressed code stream into the memory, so as to reduce the write bandwidth to the memory;
  • the processor is configured to read the compressed code stream in the memory, and perform decompression processing on the compressed code stream, so as to reduce the read bandwidth of the memory.
  • an embodiment of the present invention provides a movable platform, and the movable platform includes:
  • a power system is provided in the body, and the power system is used to provide power to the movable platform; and, a data processing system as described in the sixth aspect.
  • an embodiment of the present invention provides an encoder, including: a selection circuit and an encoding circuit;
  • the selection circuit is configured to obtain the coding overhead of the data block to be coded by the multiple first coding modes, and determine the target coding mode with the smallest coding overhead from the multiple first coding modes;
  • the encoding circuit encodes the data block to be encoded by using the target encoding method.
  • an embodiment of the present invention provides an encoder, the encoder includes: a residual calculation circuit and an encoding circuit;
  • the residual calculation circuit is configured to use the data at a preset position in the data block to be encoded as reference data, and respectively calculate the residual value of every two adjacent data located on both sides of the reference data;
  • the coding circuit is configured to write the residual value and the reference data into the code stream data according to the code stream structure corresponding to the residual coding mode; wherein, the code stream structure corresponding to the residual coding mode Including: the reference data, the bit width occupied by the residual value, and the residual value.
  • an embodiment of the present invention provides a decoder, the decoder including: a parsing circuit and a decoding circuit;
  • the parsing circuit is configured to analyze the data to be decoded, determine a target encoding method corresponding to the data to be decoded among a plurality of preset first encoding methods, and determine a corresponding target decoding method according to the target encoding method;
  • the decoding circuit is used to decode the data to be decoded by the target decoding mode to generate decoded data.
  • an embodiment of the present invention provides a decoder, which includes: a parsing circuit and a decoding circuit;
  • the analysis circuit is used to analyze the data block to be decoded, and obtain the reference data and residual data in the data block to be decoded
  • the decoding circuit is used to perform a decrement operation or an accumulation operation on the reference data and the residual data to generate original data on both sides of the reference data.
  • a data processing method, system, encoder and decoder provided by various embodiments of the present invention
  • FIG. 1 is a schematic flowchart of a data processing method provided in an embodiment of the present invention
  • FIG. 2 is a schematic diagram of a code stream structure corresponding to a run-length encoding method provided in an embodiment of the present invention
  • Fig. 3 is a schematic diagram of a code stream structure corresponding to an original encoding method provided in an embodiment of the present invention
  • Fig. 5 is a schematic diagram of a code stream structure corresponding to a residual coding method provided in an embodiment of the present invention
  • FIG. 6 is a schematic flowchart of a method for performing data processing according to a hybrid encoding mode provided in an embodiment of the present invention
  • Fig. 7 is a schematic diagram of residual calculation provided in an embodiment of the present invention.
  • FIG. 8 is a schematic diagram of a code stream structure corresponding to a hybrid encoding method provided in an embodiment of the present invention.
  • FIG. 9 is a schematic flowchart of another method for performing data processing in a hybrid encoding manner provided in an embodiment of the present invention.
  • FIG. 10 is a schematic diagram of a code stream structure corresponding to a hybrid encoding method provided in an embodiment of the present invention.
  • Figure 11 is a schematic flow chart of a method for decoding data to be decoded provided in an embodiment of the present invention.
  • FIG. 12 is a schematic flowchart of a method for processing data to be coded provided in an embodiment of the present invention.
  • FIG. 13 is a schematic flowchart of a method for processing preset data provided in an embodiment of the present invention.
  • Figure 14 is a schematic diagram of a system for processing preset data provided in an embodiment of the present invention.
  • Figure 15 is a schematic diagram of a data processing system provided in an embodiment of the present invention.
  • Figure 16 is a schematic structural diagram of a movable platform provided in an embodiment of the present invention.
  • Figure 17 is a schematic structural diagram of an encoder provided in an embodiment of the present invention.
  • Figure 18 is a schematic diagram of the functional architecture of an encoder provided in an embodiment of the present invention.
  • FIG. 19 is a schematic structural diagram of a decoder provided in an embodiment of the present invention.
  • FIG. 20 is a schematic diagram of the functional architecture of a decoder provided in an embodiment of the present invention.
  • the embodiments of the present invention provide a data processing method, system, encoder and decoder.
  • Fig. 1 is a schematic flowchart of a data processing method provided in an embodiment of the present invention. Referring to Fig. 1, the method includes the following steps S1-S2:
  • the foregoing encoding overhead refers to the bit width occupied by the code stream data generated after encoding the data block to be encoded.
  • the aforementioned target encoding method with the smallest encoding overhead refers to an encoding method with the smallest data amount of the compressed code stream obtained after compression encoding of the data to be encoded among the preset multiple first encoding methods.
  • multiple first coding modes are set, and then the coding mode with the least coding overhead is selected from the multiple first coding modes for the data block to be coded for coding, thereby achieving the best compression effect.
  • the target encoding methods determined for different data blocks to be encoded may be different, and then encoding is performed by selecting a suitable encoding method with the best compression effect for the data blocks to be encoded, so that the compression obtained after compression The data volume of the code stream is the smallest.
  • the aforementioned data block to be encoded is obtained from an original data block, and the original data block contains data to be compressed and encoded.
  • the above method further includes the following steps A10-A20:
  • Step A10 Determine whether there is a redundant data block in the original data block, and if so, encode the redundant data block according to the second coding mode.
  • the aforementioned preset value may be determined according to the accuracy of compression coding.
  • the foregoing preset value may be set to 4.
  • the redundant data block in the original data block is 22222.
  • Step A20 Use the remaining data block from the original data block except the redundant data block as the data block to be encoded.
  • the redundant data block is encoded by the second encoding method, and the remaining data block of the original data block is removed from the redundant data block
  • data processing is performed according to the above-mentioned steps S1-S2 for the data block to be encoded.
  • the above-mentioned second encoding method includes: run-length encoding; and in this embodiment, the redundant data block is encoded in a run-length encoding manner.
  • encoding the redundant data block in a run-length encoding manner includes:
  • the redundant data block is written into the coded stream data according to the code stream structure corresponding to the run-length encoding; the code stream structure corresponding to the run-length encoding includes: the flag bit of the encoding mode, the start value and the run length .
  • the data of each row in the original data block is scanned row by row, and the redundant data block is further obtained after scanning a certain row.
  • the initial value and run length of the redundant data block taking the redundant data block as 22222 as an example, the initial value of the redundant data block is 2 and the run length is 5, the initial value of the redundant data block,
  • the run length is written into the code stream structure corresponding to the run length coding, and the code stream structure also contains a coding mode flag bit, which indicates that the coding mode adopted for the data of the code stream is run length coding. After the current line scan is completed, continue to scan the next line.
  • the flag bit 00 indicates that the coding mode corresponding to the code stream is the run-length coding mode, base represents the starting value, and len represents the run length.
  • the remaining blocks are first divided according to a specified size to obtain multiple data blocks, and the obtained multiple data blocks.
  • Each data block has the same size, and then each divided data block is regarded as a data block to be coded, and each data block to be coded is processed and coded according to the above steps S1-S2 respectively.
  • the remaining data blocks before dividing the remaining data blocks according to a specified size to obtain multiple data blocks, if the data amount of the data contained in the remaining data blocks does not satisfy an integer multiple of the foregoing specified size, It is not enough to divide the remaining data blocks into multiple data blocks of the specified size. At this time, the remaining data blocks are filled with data so that the amount of data contained in the remaining data blocks after data filling is an integer multiple of the specified size.
  • the data volume of each data block obtained after division is the specified size.
  • the above-mentioned specified size is 16 data.
  • M is an integer greater than or equal to 1. Then, the remaining data block is filled with data, so that the total data of the remaining data block after filling meets to be 16M.
  • the foregoing method of filling the remaining data blocks includes:
  • the remaining data block is filled with data by using the last data in the remaining data block; optionally, the remaining data block may be filled with data by using a predetermined data.
  • the remaining data blocks are divided into one data block according to each 16-bit data starting from the first data in the first place. After dividing one or more data blocks, if the last remaining data is: 0122254 Since the number of data bits of the remaining data is 7, the data at the end of the remaining data block is used for data filling, and the filled data is 0122254444444444. After data filling, all the data blocks divided by the remaining data block can be obtained. Then, the target encoding method is determined for each data block according to the above method, and the target encoding method is applied to encode each data block.
  • the remaining data blocks are filled with data so that the obtained data blocks have the same size, which is convenient for encoding the remaining data.
  • the above-mentioned multiple first encoding methods include: original encoding, residual encoding, and hybrid encoding; wherein, the hybrid encoding is after the data block to be encoded is divided into several data sub-blocks, Perform residual coding or original coding on each data block.
  • the original coding mode is used to encode the to-be-coded data block, which specifically includes the following step B10:
  • Step B10 Write the original data in the data block to be coded into the code stream data according to the code stream structure corresponding to the original coding mode.
  • the code stream structure corresponding to the above-mentioned original coding mode includes: the flag bit of the coding mode and original data.
  • the original encoding method includes: directly writing the original data in the data block to be encoded into the encoded bitstream data according to the code stream structure corresponding to the original encoding method.
  • the above flag bit 01 is used to indicate that the coding mode corresponding to the code stream structure is the original coding mode (RAW16), and the above p0, p1,, and p15 are waiting Encode the original data in the data block.
  • encoding the data block to be coded by the target coding mode includes the following step S20:
  • the data at a preset position in the data block to be encoded is used as the reference data, and the residual values of the data on both sides of the reference data are respectively calculated. This starts from the reference data and moves to both sides of the reference data. Compared with the prior art, the calculation method can improve the efficiency of calculation.
  • the data block to be encoded may also be further divided into multiple data sub-blocks, by setting one or more reference data in each data sub-block, and calculating the two sides of each reference data The residual value of two adjacent data, which can further improve the efficiency of calculation, and further improve the efficiency of compression coding.
  • the process of calculating the residual is shown in Fig. 4, taking 16 pieces of data contained in the data block to be coded as an example, the 16 pieces of data are respectively: p0, p1, p2, p3, p4,, p14, p15.
  • the data p8 at the center of the data block to be encoded as the reference data use the following array to record the residual values of pixels other than p8:
  • the residual values of two adjacent data are calculated on both sides respectively. Compared with the data in the initial position, the residual values are calculated, which can improve the calculation efficiency.
  • the following array can be used to record the residual values of pixels other than p8:
  • calculation direction can be adjusted as needed, and is not limited to the above two methods.
  • the residual value and the corresponding reference data are written into the code stream data according to the code stream structure corresponding to the residual coding method to obtain the corresponding code stream data; the residual coding method corresponds to
  • the code stream structure includes: the flag bit of the coding mode, the reference data, the bit width occupied by the residual value, and the residual value.
  • the code stream structure corresponding to the residual coding method is shown in FIG. 5.
  • the above-mentioned data segment to be coded contains 16 pieces of data as an example, and the flag bit 10 is used to characterize the code used in the code stream structure.
  • the method is the residual coding method (DPCM16), p8 is the reference data, Rb is the bit width occupied by the residual value, r0-r7 is the residual value, and r10-r15 is the residual value.
  • encoding the data block to be encoded by the hybrid encoding method specifically includes the following steps:
  • the method for dividing the data block to be coded may be to obtain a plurality of data sub-blocks with the same size evenly, or it may be divided into data sub-blocks with different sizes according to a set size.
  • the two-way calculation and coding method improves coding efficiency compared with the prior art.
  • the code stream structure corresponding to the foregoing hybrid encoding method includes: a flag bit of the encoding method, the reference data, the bit width occupied by the residual value, and the residual value.
  • multiple data sub-blocks are obtained by dividing the data block to be coded, and each data sub-block is coded in a residual coding manner.
  • the data contained in the data block to be encoded is: p0, p1, p2,, p14, p15
  • the data block to be encoded is divided into two data sub-blocks, which are respectively data blocks : P0p1, p7 and data block: p8p9,,, p15.
  • For two data sub-blocks select one data at a preset position as parameter data (respectively p4, p12 in the figure), and then calculate the residual values of two adjacent data on both sides of the reference data. And write the code stream data according to the coding structure of the hybrid coding.
  • the encoding structure corresponding to the hybrid encoding mode is shown in FIG. 8, where, in the hybrid encoding mode, the above-mentioned flag bit 10 is used to indicate that the encoding method adopted by the code stream structure is the residual encoding method (DPCM8_1&DPCM8_2), p4 and p12 are reference data respectively, Rb is the bit width occupied by the residual value, r0-r7 is the residual value, and r8-r14 is the residual value.
  • the above-mentioned flag bit 10 is used to indicate that the encoding method adopted by the code stream structure is the residual encoding method (DPCM8_1&DPCM8_2)
  • p4 and p12 are reference data respectively
  • Rb is the bit width occupied by the residual value
  • r0-r7 is the residual value
  • r8-r14 is the residual value.
  • the aforementioned target encoding method includes: a hybrid encoding method.
  • encoding the data block to be encoded by the hybrid encoding method specifically includes the following steps S21'-S23':
  • the method for dividing the data block to be coded may be to obtain a plurality of data sub-blocks with the same size evenly, or it may be divided into data sub-blocks with different sizes.
  • the first data sub-block is coded according to the above residual coding method, and then the obtained residual
  • the value and reference data are written into the coded code stream data according to the code stream structure corresponding to the hybrid coding mode.
  • the original data in the second data sub-block is written into the coded stream data directly according to the code stream structure corresponding to the mixed coding mode according to the original coding mode.
  • the hybrid coding mode when the hybrid coding mode is adopted, after the data block to be coded is divided to obtain multiple data sub-blocks, the first data sub-block is coded according to the residual coding mode, and the second data sub-block is Encode according to the original encoding method; the first data sub-block and the second data sub-block may be obtained by dividing all the data sub-blocks equally according to the number, and the number of the first data sub-block is the same as the number of the second data sub-block ; Of course, the number of first data sub-blocks may be different from the number of second data sub-blocks.
  • the code stream structure corresponding to the above-mentioned hybrid encoding method includes: the flag bit of the encoding method, the original data of the second data block, the reference data, the bit width occupied by the residual value, and the residual value.
  • an embodiment of the present invention provides a hybrid encoding method
  • the coding structure is shown in Figure 10.
  • the above-mentioned flag bit 11 is used to characterize that the coding mode corresponding to the bitstream structure is hybrid coding;
  • the bitstream structure of mixed coding includes not only the coding mode representing "hybrid coding", but also Characterize the coding mode corresponding to each sub-data block.
  • 1 bit can be used to mark whether the code stream of the corresponding sub-data block is original coding or residual coding, for example, 0 is used to represent the original coding, and 1 is used to represent the residual coding.
  • the code stream structure in this hybrid coding mode includes: (1) The code stream structure (RAW8_1&DPCM8_2) for the original coding and residual coding of the data sub-blocks.
  • the coding mode flag bit of the code stream structure is 00, where p0- p8 is the original data, p12 is the reference data, r8-r15 is the residual value, and R'b is the bit width occupied by the residual value;
  • Perform residual coding on data sub-blocks (DPCM8_1&DPCM8_2), the coding mode flag of this coding mode is 10; among them, p4 and p12 are reference
  • the encoding overhead calculated for the same data block to be encoded by each of the foregoing encoding methods can be referred to as follows:
  • the encoding overhead for the original encoding of the data block to be encoded is: 16*8bit; where 16 is the number of data, and the space occupied by each data is 8bit.
  • the coding overhead for residual coding of the data block to be coded is: 8bit+bitlen_16*15+3bit; among them, 8bit is the bit width occupied by the reference data, bitlen_16 is the bit width occupied by a residual value, and 3bit is the bitlen 16 Occupied bit width.
  • the coding overhead for dividing the data block to be coded into two data blocks for residual coding is: 1bit+8bit+bitlen8_1*7+3bit+1bit+8bit+bitlen8_2*7+3bit, where the calculation unit is 8 data, and 1bit is used
  • 8bit is the bit width occupied by the reference data
  • bitlen8_1 and bitlen8_2 are the bit width occupied by a residual value
  • 3bit is the bit width occupied by bitlen8_1 or bitlen8_2.
  • the encoding overhead of dividing the data block to be encoded into two data blocks for original encoding and residual encoding is: 1bit+8bit*8+1bit+8bit+bitlen8_2*7+3bit; among them, the first 8 data are original data, each The space occupied by the original data is 8bit, 1bit is used to mark whether the code stream is original encoding or residual encoding, the second 8bit is the bit width occupied by the reference data, and bitlen8_2 is the bit width occupied by a residual value. 3bit is the bit width occupied by bitlen8_2.
  • the coding overhead of dividing the data block to be coded into two data blocks for residual coding and original coding is: 1bit+8bit+bitlen8_1*7+3bit+1bit+8*8bit; for the definition of the calculation formula, please refer to the above coding overhead Calculation.
  • all the data blocks to be encoded contain 16-bit data as an example; but in practice, the data blocks to be encoded may also contain data such as 32 bits, 64 bits, etc.; The present invention does not limit this.
  • both the first encoding method and the second encoding method are lossless encoding, which can ensure the accuracy of the data to be encoded after compression and decompression.
  • the data processing method provided in the foregoing embodiment of the present invention can be applied to compress the weight data of a neural network, which can greatly reduce bandwidth overhead.
  • the embodiment of the present invention does not reduce the convergence speed of the network, nor does it need to fine-tune hyperparameters, and has high efficiency; in addition, the embodiment of the present invention does not cause a loss of the accuracy of the weight data, and ensures that the weight parameters will not introduce additional errors. And there is no need to retrain the network, and the system bandwidth can be reduced to a large extent. It also does not exist that after the weight data is compressed by the method provided in the foregoing embodiment of the present invention, the convergence speed of the network will not be reduced, and the efficiency is high. And it will not cause the loss of the accuracy of the weight data.
  • an embodiment of the present invention also provides a data processing method to decode the encoded data data (as data to be decoded).
  • an embodiment of the invention provides a data processing method, which includes the following steps S41-S42:
  • the above-mentioned data to be decoded includes the coding mode flag bit; the above-mentioned analysis of the data to be decoded can obtain the value of the flag bit contained in the decoded data, and then according to the preset mapping relationship between the value of the flag bit and the first coding mode,
  • the target encoding method corresponding to the data to be decoded is determined, and the corresponding target decoding method can be determined according to the target encoding method; for example, the target decoding method is the inverse operation of the target encoding method.
  • the flag bit 00 in the code stream structure is obtained, and the corresponding encoding method is determined to be run-length coding according to the flag bit 00. Then obtain the corresponding target decoding mode as run-length decoding.
  • the corresponding target decoding method can be determined, and the target decoding method may be the inverse operation of the encoding method.
  • the above-mentioned first encoding method includes one or more of run-length encoding, original data encoding, residual encoding, and hybrid encoding;
  • the hybrid coding is to divide a data block to be coded to generate multiple data sub-blocks, and perform residual coding or original data coding on each of the data blocks.
  • the above-mentioned residual coding is to use the data at the preset position in the data block to be coded as reference data, respectively calculate the residual value of each two adjacent data located on both sides of the reference data, and combine the residual value and the
  • the reference data is written into the coded code stream data according to the code stream structure corresponding to the residual coding mode.
  • the foregoing target decoding mode includes one or more of run-length decoding, original data copy, residual decoding, and hybrid decoding;
  • the above-mentioned hybrid decoding is to divide the data block to be decoded to generate multiple data sub-blocks.
  • the division method may be divided according to flag bits, and residual decoding or original data copy is performed on each data block. .
  • the above-mentioned residual decoding is to read the corresponding reference data, the bit length occupied by the residual value and the residual value from the bit stream structure of the data to be decoded, according to the bit length occupied by the reference data and the residual value Perform the inverse operation of the residual calculation with the residual value to generate decoded data.
  • using the residual decoding method to decode the data to be decoded includes: reading the corresponding reference data from the code stream structure of the data to be decoded and the residual value The bit length and residual value of, perform the inverse operation of residual calculation according to the reference data and residual value to obtain the original data.
  • using run-length decoding to decode the data to be decoded includes:
  • the start value and the run length are read from the code stream structure of the data to be decoded, and the data to be decoded is decoded and restored according to the start value and the run length to obtain decoded data.
  • using the original data copy mode to decode the data to be decoded includes:
  • the part of the data exceeding the preset data volume is discarded to generate The decoded data.
  • another embodiment of the present invention also provides a data processing method, including the following steps S51-S52:
  • the 16 data are: p0, p1, p2, p3, p4, ,,, p14, p15.
  • the reference data p8 Take the data p8 at the center of the data block to be encoded as the reference data, start with the reference data p8, calculate the residual values of the two adjacent data to both sides, and calculate the residuals relative to the data at the initial position Value, can improve the efficiency of calculation.
  • the intermediate data and residual data can be accumulated or decremented to both ends during decoding, which improves the parallelism of decoding.
  • the foregoing data block to be encoded may be obtained in the following two ways:
  • Manner 1 Divide the original data block according to a specified size to obtain multiple data sub-blocks, and use each of the data sub-blocks as the data block to be encoded.
  • Manner 2 Divide the original data block according to the specified size to obtain multiple data sub-blocks, and use the first data block of the multiple data blocks as the data block to be encoded.
  • part of the data sub-blocks (that is, the first data block) of the multiple data sub-blocks is used as the above-mentioned data block to be coded, and the data in the preset position in the data block to be coded is used as For reference data, calculate the residual values of every two adjacent data located on both sides of the reference data.
  • the first data block is used as the data block to be encoded
  • the data blocks other than the first data block among the multiple data sub-blocks obtained are based on the code stream structure corresponding to the original encoding method , Write the coded stream data;
  • the code stream structure corresponding to the original coding mode includes: the flag bit of the coding mode and the original data.
  • the above-mentioned residual value and reference data are written into the coded stream data according to the code stream structure corresponding to the residual coding mode, and the code stream structure corresponding to the residual coding mode includes: reference data, residual value And the bit width occupied by the residual value.
  • the two-way calculation method can improve the efficiency of calculation.
  • the data sub-block can also be further divided into multiple data blocks, by setting one or more reference data in each data block, and calculating the phases on both sides of each reference data.
  • the residual value of the adjacent two data can further improve the calculation efficiency and thus the coding efficiency.
  • the data block to be decoded when parsing the encoded data obtained by the above method as the data block to be decoded in an embodiment of the present invention, first obtain the reference data and residual data in the data block to be decoded; wherein, the data block to be decoded
  • the data block is encoded and generated; the reference data is data at a preset position in the original data; the residual data is the residual value of every two adjacent data located on both sides of the reference data during the encoding process.
  • the reference data and the residual data are decremented or accumulated to generate the original data on both sides of the reference data.
  • the data block to be encoded contains 16 data
  • the 16 data are: p0, p1, p2, p3, p4, ,, p14, p15.
  • the data p8 at the center of the data block to be encoded as the reference data use the following array to record the residual values of pixels other than p8:
  • p8 and diff[8] to diff[14] are used for decrement operations to obtain p9-p14 data respectively.
  • p8 and diff[0] to diff[7] are used for decrement operations.
  • the data of p7-p0 can be obtained separately.
  • decrement operations with r0 to r7 can respectively obtain data p7-p0
  • decrement operations with r9 to r15 can respectively obtain data p9-p14.
  • p8-diff[0] can get p7
  • p8-diff[0]-diff[1] can get p6, and other things are the same.
  • the following array can be used to record the residual values of pixels other than p8:
  • p8 and diff[8] to diff[14] are used for accumulation operation to obtain the data of p9-p14 respectively.
  • p8 and diff[0] to diff[7] are used for accumulation operation
  • the data of p7-p0 can be obtained separately.
  • the data of p7-p0 can be obtained by accumulating operations with r0 to r7
  • the data of p9-p14 can be obtained by accumulating operations with r9 to r15 respectively.
  • p8+diff[0] can get p7
  • p8+diff[0]+diff[1] can get p6, and the others are the same.
  • two-way simultaneous decoding operations can be performed on the data block to be decoded from both sides of the reference data, which has the positive effect of high decoding efficiency.
  • this embodiment provides a data processing method, which is applied to a configurator, a memory, and a processor connected in sequence, and the method includes the following steps S61-S62:
  • S61 Perform compression encoding on the data block to be encoded formed by the preset data by the configurator to generate a compressed code stream, and write the compressed code stream into the memory, so as to reduce the write bandwidth to the memory.
  • the above-mentioned configurator is provided with an encoder, and the configurator specifically compresses and encodes the data block to be encoded by the encoder to generate a code stream.
  • S62 Read the compressed code stream in the memory by the processor, and perform decompression processing on the compressed code stream to reduce the read bandwidth of the memory.
  • a decoder is provided on the aforementioned processor, and the processor specifically decompresses the aforementioned compressed code stream through the decoder.
  • the above-mentioned preset data is weight data based on a convolutional neural network; and in this embodiment, the configurator is used to allocate the weight data and compress and encode the weight data by an encoder.
  • the compressed code stream is written into the memory; the processor is used to read the compressed code stream corresponding to the weight data from the memory, decompress the compressed code stream through the decoder to generate weight data, and run the convolution based on the weight data Neural Networks.
  • the foregoing processor further includes a static random-access memory (Static Random-Access Memory, SRAM), which is used to read the compressed code stream corresponding to the weight data from the memory, so that the processor can pass through
  • SRAM static random-access memory
  • the decoder performs decompression processing on the compressed code stream in the SRAM to generate weight data, and runs the convolutional neural network based on the weight data.
  • FIG. 14 shows a schematic diagram of the relationship between the configurator, the memory and the processor; referring to FIG. 14, the above-mentioned configurator is connected to the memory, and the memory is connected to the processor.
  • the above-mentioned memory may be a double data rate synchronous dynamic random access memory (Double Data Rate Synchronous Dynamic Random Access Memory, DDR SDRAM); in this embodiment, an encoder is provided in the configurator, and the configurator is specifically coded
  • the processor compresses and encodes the data block to be encoded to generate a compressed code stream; the processor is provided with a decoder and a static random access memory, and the static random access memory is used to read the compressed code stream corresponding to the weight data from the memory.
  • a decoder is used to decompress the compressed code stream in the SRAM to generate weight data, and run a convolutional neural network based on the weight data.
  • the encoder in order to reduce the bandwidth overhead when the weight data is read from the memory to the static random access memory in the processor, the encoder decompresses the compressed weight data; accordingly, the configurator will Before the weight data is written into the memory, the encoder will compress the weight data.
  • This process uses encoders and decoders to losslessly compress and decompress the weight data, ensuring that the weight data does not introduce additional errors and does not require retraining the network. At the same time, the system bandwidth can be reduced to a large extent.
  • the encoder performs lossless compression processing on the weight data to ensure that the weight parameters do not introduce additional errors and do not need to retrain the network.
  • the specific process of the configurator performing compression encoding on the data block to be encoded to generate a compressed code stream includes: acquiring the encoding overhead of multiple first encoding methods for the data block to be encoded, In the encoding method, determine the target encoding method with the least encoding overhead; then use the target encoding method to compress and encode the data block to be encoded formed by the preset data to generate a compressed code stream; the preset data may be the weight of the convolutional neural network data.
  • the aforementioned encoding overhead is the bit width occupied by the code stream data generated after the data block to be encoded is encoded.
  • an encoding method with the least encoding overhead is selected for encoding, which can achieve effective compression of the data blocks to be encoded.
  • the above-mentioned configurator first determines whether there is a redundant data block in the original data block, and if so, the redundant data block is encoded according to the second encoding method.
  • the second encoding method includes: run-length encoding.
  • the specific configurator writes the redundant data block into the code stream data according to the code stream structure corresponding to the run-length encoding; the code stream corresponding to the run-length encoding
  • the structure includes: the flag bit of the encoding method, the starting value and the run length.
  • the remaining data blocks can be used as the data blocks to be encoded, and the encoding method with the least encoding overhead is selected for encoding.
  • the remaining data blocks may be divided according to a specified size to obtain multiple data blocks, each data block is used as the data block to be encoded, and the encoding method with the least encoding overhead is selected for encoding; in this embodiment, if the remaining data block If the amount of data does not satisfy an integer multiple of the specified size, then the remaining data blocks are first filled with data so that the size of each data block obtained after division is the specified size.
  • the above-mentioned configurator determines whether there is a redundant data block in the original data block in the following manner:
  • the configurator searches the original data block to see whether there is data that continuously appears and the number of repetitions is greater than a preset value, and if so, determines that there is a redundant data block, and uses the continuous data as the redundant data block; if not , It is determined that there is no redundant data block.
  • the above-mentioned multiple first encoding methods include: an original encoding method, a residual encoding method, and a hybrid encoding method; wherein the hybrid encoding method is to divide the data block to be encoded into several data After sub-blocks, residual coding or original coding is performed on each of the data blocks.
  • encoding the data block to be coded by the target coding mode includes:
  • the code stream structure corresponding to the original coding mode includes: a flag bit of the coding mode and original data.
  • the encoding the data block to be coded by the target coding mode includes:
  • the residual values of each two adjacent data located on both sides of the reference data are calculated respectively, and the residual value and the reference data are calculated according to
  • the code stream structure corresponding to the residual coding method is written into the coded code stream data;
  • the code stream structure corresponding to the residual coding mode includes: a flag bit of the coding mode, the reference data, the bit width occupied by the residual value, and the residual value.
  • encoding the data block to be coded by the target coding mode includes:
  • the residual value calculated for each of the data sub-blocks and the reference data are respectively written into the coded code stream data according to the code stream structure corresponding to the hybrid coding mode.
  • the code stream structure corresponding to the above-mentioned hybrid coding mode includes: the flag bit of the coding mode, the reference data, the bit width occupied by the residual value, and the residual value.
  • encoding the data block to be coded by the target coding mode includes:
  • the residual values of each two adjacent data located on both sides of the reference data are respectively calculated, and the residual value and the reference data Write the encoded code stream data according to the code stream structure corresponding to the hybrid encoding method; write the original data in the second data sub-block into the code stream structure corresponding to the hybrid encoding method Stream data;
  • the code stream structure corresponding to the hybrid encoding method in this embodiment includes: the flag bit of the encoding method, the original data of the second data block, the residual value, the reference data, and the residual value The occupied bit width and the residual value.
  • the processor reading the compressed code stream in the memory and decompressing the compressed code stream includes:
  • the processor parses the compressed code stream, determines a target encoding method corresponding to the compressed code stream among a plurality of preset first encoding methods, and determines a corresponding target decoding method according to the target encoding method;
  • the compressed code stream is decoded by the target decoding method to obtain the preset data.
  • the code stream structure of the aforementioned compressed code stream includes a flag bit; the processor determines that it corresponds to the compressed code stream according to a preset mapping relationship between the value of the flag bit and the first encoding mode The target encoding method.
  • the foregoing target decoding methods include: any one or more of run-length decoding, original data copy, residual decoding, and hybrid decoding;
  • the hybrid decoding is to divide the data block to be decoded to generate multiple data sub-blocks, and perform residual decoding or original data copy on each of the data word blocks.
  • the using the target decoding mode to decode and restore the compressed code stream includes:
  • the initial value and the run length are read from the code stream structure of the compressed code stream, and the compressed code stream is decoded and restored according to the initial value and the run length to obtain corresponding preset data.
  • the decoding and restoring the compressed code stream using the target decoding mode includes:
  • the using the target decoding mode to decode and restore the compressed code stream includes:
  • the decoding and restoring the compressed code stream using the target decoding method to obtain the preset data includes:
  • the data volume of the data obtained by decoding and restoring the compressed code stream by the target decoding method is greater than the preset data volume, discarding part of the data exceeding the preset data volume to obtain the preset data.
  • a data processing system including: a configurator 81, a memory 80, and a processor 82 connected in sequence.
  • the configurator 81 is configured to compress and encode a data block to be encoded formed by preset data to generate a compressed code stream, and write the compressed code stream into the memory 80 to reduce the write bandwidth to the memory 80.
  • the processor 82 is configured to read the compressed code stream in the memory 80, and decompress the compressed code stream, so as to reduce the read bandwidth of the memory 80.
  • an encoder is provided on the configurator 81, and a decoder is provided on the processor 82;
  • the configurator is used to compress and encode the data block to be encoded by the encoder to generate a compressed code stream, and the processor is used to perform decompression processing on the compressed code stream by the decoder.
  • the aforementioned preset data is weight data based on a convolutional neural network;
  • the configurator is used to allocate the weight data, and write the compressed code stream obtained by compressing and encoding the weight data into the memory;
  • the processor is configured to read the compressed code stream corresponding to the weight data from the memory, decompress the compressed code stream to generate the weight data, and run the convolutional neural network based on the weight data The internet.
  • the configurator, memory, and processor constitute the system architecture of the convolutional neural network.
  • the problem of the processor repeatedly reading weight data from the memory to the processor is solved. Bandwidth pressure.
  • the aforementioned processor 82 further includes a static random access memory, and the static random access memory is used to read the compressed code stream corresponding to the weight data from the memory, so that the processor can The compressed code stream is decompressed to generate the weight data, and the convolutional neural network is run based on the weight data.
  • the foregoing memory may be a double data rate synchronous dynamic random access memory (Double Data Rate Synchronous Dynamic Random Access Memory, DDR SDRAM); in this embodiment, an encoder is provided in the configurator, and the configurator is specifically The encoder is used to compress and encode the data block to be encoded to generate a code stream; the processor is provided with a decoder and a static random access memory, and the static random access memory is used to read the compression code corresponding to the preset data from the memory Stream, the processor decompresses the compressed code stream in the SRAM through the decoder to generate the original preset data.
  • DDR SDRAM Double Data Rate Synchronous Dynamic Random Access Memory
  • the aforementioned data processing system is a calculation system of a convolutional neural network
  • the aforementioned preset data is weight data of a convolutional neural network
  • the aforementioned processor is used to run the convolutional neural network based on the weight data.
  • the encoder in order to reduce the bandwidth overhead when the weight data is read from the memory to the static random access memory in the processor, the encoder decompresses the compressed weight data; accordingly, the configurator will Before the weight data is written into the memory, the encoder will compress the weight data.
  • This process uses encoders and decoders to losslessly compress and decompress the weight data, ensuring that the weight data will not introduce additional errors and do not need to retrain the neural network, while reducing the system bandwidth to a large extent. It reduces the hardware resource requirements for the operation of the convolutional neural network.
  • the above-mentioned configurator 81 is specifically configured to compress and encode the to-be-coded data block formed by the preset data in the following manner to generate a compressed code stream:
  • the configurator obtains the coding overhead of the plurality of preset first coding modes for the data block to be coded, and determines the target coding mode with the smallest coding overhead from the plurality of first coding modes;
  • the foregoing encoding overhead is the bit width occupied by the code stream data generated after encoding the data block to be encoded.
  • the above-mentioned configurator 81 is further configured to: determine whether there is a redundant data block in the original data block, and if so, encode the redundant data block according to the second coding mode;
  • the above-mentioned configurator 81 is further configured to generate the data block to be encoded in the following manner:
  • the above-mentioned configurator 81 is specifically configured to determine whether there is a redundant data block in the original data block in the following manner:
  • the foregoing second encoding manner includes run-length encoding
  • the configurator 81 is specifically configured to encode the redundant data block according to the second encoding manner in the following manner:
  • the redundant data block is written into the coded stream data according to the code stream structure corresponding to the run-length encoding; the code stream structure corresponding to the run-length encoding includes: the flag bit of the encoding mode, the start value and the run length .
  • the above-mentioned multiple first coding modes include: an original coding mode, a residual coding mode, and a mixed coding mode; wherein, the mixed coding mode is after the data block to be coded is divided into several data sub-blocks , Perform residual coding or original coding on each of the data blocks.
  • the configurator 81 is specifically configured to:
  • the original data in the data block to be coded is written into the coded stream data according to the code stream structure corresponding to the original coding mode; the code stream structure corresponding to the original coding mode includes: the flag bit of the coding mode and Raw data.
  • the configurator 81 is specifically configured to:
  • the residual values of each two adjacent data located on both sides of the reference data are calculated respectively, and the residual value and the reference data are calculated according to
  • the code stream structure corresponding to the residual coding mode is written into the coded code stream data; wherein the code stream structure corresponding to the residual coding mode includes: a flag bit of the coding mode, the reference data, and the residual The bit width occupied by the value and the residual value.
  • the configurator 81 is specifically configured to:
  • the residual value calculated for each data sub-block and the reference data are written into the coded stream data according to the code stream structure corresponding to the hybrid encoding mode; wherein the code corresponding to the hybrid encoding mode
  • the stream structure includes: a flag bit of an encoding mode, the reference data, the bit width occupied by the residual value, and the residual value.
  • the configurator is specifically configured to:
  • the residual values of each two adjacent data located on both sides of the reference data are respectively calculated, and the residual value and the reference data Write the encoded code stream data according to the code stream structure corresponding to the hybrid encoding method; write the original data in the second data sub-block into the code stream structure corresponding to the hybrid encoding method Stream data;
  • the code stream structure corresponding to the hybrid coding mode includes: the flag bit of the coding mode, the original data of the second data block, the residual value, the reference data, and the residual value Bit width and the residual value.
  • the above-mentioned configurator 81 is further configured to: before the dividing the remaining data blocks according to a specified size to obtain multiple data blocks, if the data amount of the remaining data blocks does not satisfy an integer of the specified size Times, the remaining data blocks are filled with data, so that the size of each data block obtained after division is the specified size.
  • the performing data filling on the remaining data blocks includes:
  • the foregoing processor 82 is specifically configured to read the compressed code stream in the memory in the following manner, and perform decompression processing on the compressed code stream:
  • the processor parses the compressed code stream, determines a target encoding method corresponding to the compressed code stream among a plurality of preset first encoding methods, and determines a corresponding target decoding method according to the target encoding method;
  • the compressed code stream is decoded by the target decoding method to obtain the preset data.
  • the code stream structure of the compressed code stream includes a flag bit
  • the processor is specifically configured to parse the compressed code stream in the following manner, and determine a target encoding method corresponding to the compressed code stream among a plurality of preset first encoding methods:
  • the target coding mode corresponding to the compressed code stream is obtained.
  • the target decoding mode includes any one of run-length decoding, original data copy, residual decoding, and hybrid decoding;
  • the hybrid decoding is to divide the data block to be decoded to generate multiple data sub-blocks, and perform residual decoding or original data copy on each of the data word blocks.
  • the processor when the target decoding mode is run-length decoding, the processor is specifically configured to:
  • the initial value and the run length are read from the code stream structure of the compressed code stream, and the compressed code stream is decoded and restored according to the initial value and the run length to obtain corresponding preset data.
  • the processor is specifically configured to:
  • the processor is specifically configured to:
  • the processor is specifically configured to decode and restore the compressed code stream by using the target decoding mode in the following manner to obtain the preset data, including:
  • the data volume of the data obtained by decoding and restoring the compressed code stream by the target decoding method is greater than the preset data volume, discarding part of the data exceeding the preset data volume to obtain the preset data.
  • the movable platform includes: a body 900, a power system 91, and the data processing system 92 described in the foregoing embodiment; the power system 91 is located at The body 900 and the power system 91 are used to provide power for the movable platform.
  • the aforementioned data processing system 92 may be applied to a functional system of a movable platform such as a power system, an operating system, and the like.
  • the data processing system 92 uses a convolutional neural network for data processing, it can achieve lossless compression and decompression of weight data, ensuring that the weight data will not introduce additional errors and do not need to retrain the neural network , And reduce system bandwidth overhead. It can meet the use of hardware resources with lower conditions to realize the operation of convolutional neural networks.
  • an encoder 1000 is provided in this embodiment, including: a selection circuit 101 and an encoding circuit 102;
  • the selection circuit is configured to obtain the coding overhead of the data block to be coded by the multiple first coding modes, and determine the target coding mode with the smallest coding overhead from the multiple first coding modes;
  • the encoding circuit encodes the data block to be encoded by using the target encoding method.
  • the encoder provided in the embodiment of the present invention can effectively compress data and has the advantage of high compression efficiency.
  • the foregoing encoding overhead is the bit width occupied by the code stream data generated after the data block to be encoded is encoded.
  • the above selection circuit is also used for:
  • the data block to be encoded is generated in the following manner:
  • the foregoing determining whether there is a redundant data block in the original data block includes:
  • the second encoding manner includes run-length encoding
  • encoding the redundant data block according to the second encoding manner includes:
  • the redundant data block is written into the coded stream data according to the code stream structure corresponding to the run-length encoding; the code stream structure corresponding to the run-length encoding includes: the flag bit of the encoding mode, the start value and the run length .
  • the multiple first coding modes include: an original coding mode, a residual coding mode, and a mixed coding mode; wherein the mixed coding mode is dividing the data block to be coded into several data sub-blocks Afterwards, residual coding or original coding is performed on each of the data blocks.
  • the encoding the data block to be coded by the target coding mode includes:
  • the code stream structure corresponding to the original coding mode includes: a flag bit of the coding mode and original data.
  • the encoding the data block to be coded by the target coding mode includes:
  • the code stream structure corresponding to the residual coding mode includes: a flag bit of the coding mode, the reference data, the bit width occupied by the residual value, and the residual value.
  • encoding the data block to be coded by the target coding mode includes:
  • the code stream structure corresponding to the hybrid coding mode includes: a flag bit of the coding mode, the reference data, the bit width occupied by the residual value, and the residual value.
  • encoding the data block to be coded by the target coding mode includes:
  • the residual values of each two adjacent data located on both sides of the reference data are respectively calculated, and the residual value and the reference data Write the coded code stream data according to the code stream structure corresponding to the hybrid coding mode;
  • the code stream structure corresponding to the hybrid coding mode includes: the flag bit of the coding mode, the original data of the second data block, the reference data, the bit width occupied by the residual value, and the residual value. Difference.
  • the method before dividing the remaining data blocks according to a specified size to obtain multiple data blocks, the method includes:
  • data padding is performed on the remaining data block so that the size of each data block obtained after division is the specified size.
  • the performing data filling on the remaining data blocks includes:
  • Another embodiment of the present invention also provides an encoder, which includes a residual calculation circuit and an encoding circuit.
  • the residual calculation circuit is configured to use the data in the preset position in the data block to be encoded as reference data, and respectively calculate the residual value of every two adjacent data located on both sides of the reference data;
  • the coding circuit is configured to write the residual value and the reference data into the code stream data according to the code stream structure corresponding to the residual coding mode; wherein, the code stream structure corresponding to the residual coding mode Including: the reference data, the bit width occupied by the residual value, and the residual value.
  • the data block to be encoded is generated in the following manner:
  • the original data block is divided according to a specified size to obtain multiple data sub-blocks, and each of the data sub-blocks is used as the data block to be encoded.
  • the data block to be encoded is generated in the following manner:
  • the original data block is divided according to a specified size to obtain multiple data sub-blocks, and the first data block of the multiple data blocks is used as the data block to be encoded.
  • the encoding circuit is also used for:
  • the other data blocks of the multiple data sub-blocks except the first data block are written into the code stream data according to the code stream structure corresponding to the original coding mode;
  • the code stream structure corresponding to the original coding mode includes : Flag bit and original data of encoding method.
  • Figure 18 also shows a schematic diagram of the encoder architecture.
  • the encoder architecture mainly includes four functional modules: run-length encoding module, encoding overhead calculation module, mode selection module (used to determine the target encoding method with the smallest encoding overhead) and The remaining data block encoding module (used to encode the remaining data blocks except for the redundant data block); it should be noted that this embodiment is only an example for illustration and should not be regarded as a limitation of the present invention.
  • this embodiment provides a decoder 1100, and the decoder 1100 includes: an analysis circuit 111 and a decoding circuit 112;
  • the parsing circuit 111 is configured to analyze the data to be decoded, determine a target encoding method corresponding to the data to be decoded among a plurality of preset first encoding methods, and determine a corresponding target decoding method according to the target encoding method;
  • the decoding circuit 112 is configured to decode the data to be decoded in the target decoding mode to generate decoded data.
  • the data to be decoded includes a flag bit; the analyzing the data to be decoded and determining the target encoding method corresponding to the data to be decoded among the plurality of preset first encoding methods includes:
  • the target coding mode corresponding to the data to be decoded is determined.
  • the first coding mode includes one or more of run-length coding, original data coding, residual coding, and hybrid coding;
  • the hybrid coding is to divide the data block to be coded to generate multiple data sub-blocks, and perform residual coding or original data coding on each of the data blocks.
  • the residual coding is using data at a preset position in the data block to be coded as reference data, respectively calculating the residual values of every two adjacent data on both sides of the reference data, and calculating The residual value and the reference data are written into the coded stream data according to the code stream structure corresponding to the residual coding mode.
  • the target decoding mode includes one or more of run-length decoding, original data copy, residual decoding, and hybrid decoding;
  • the hybrid decoding is to divide the data block to be decoded to generate multiple data sub-blocks, and perform residual decoding or original data copy on each of the data word blocks.
  • the residual decoding is to read the corresponding reference data, the bit length occupied by the residual value, and the residual value from the code stream structure of the data to be decoded, according to the reference data and the residual value
  • the occupied bit length and residual value perform the inverse operation of residual calculation to generate decoded data.
  • the using the target decoding mode to decode the data to be decoded includes:
  • the start value and the run length are read from the code stream structure of the data to be decoded, and the data to be decoded is decoded and restored according to the start value and the run length to obtain decoded data.
  • the using the target decoding mode to decode the data to be decoded includes:
  • the using the target decoding mode to decode the data to be decoded includes:
  • the decoding multiple pieces of to-be-decoded data to generate decoded data using the target decoding manner includes:
  • the data amount of the data obtained by decoding the data to be decoded by the target decoding method is greater than the preset data amount, discarding part of the data exceeding the preset data amount to generate the decoded data.
  • Figure 20 also shows a schematic diagram of the architecture of a decoder.
  • the decoder mainly includes the following functional modules: a pattern analysis module (used to determine the target decoding method), a run-length decoding module, a residual decoding module, and an original data decoding module. It should be noted that this embodiment is only an example and should not be regarded as a limitation of the present invention.
  • Another embodiment of the present invention also provides a decoder, which includes: a parsing circuit and a decoding circuit.
  • the parsing circuit is used to analyze the data block to be decoded to obtain reference data and residual data in the data block to be decoded; wherein the data block to be decoded is generated by encoding the original data block; and the reference data is all The data at a preset position in the original data; the residual data is the residual value of every two adjacent data located on both sides of the reference data during the encoding process.
  • the decoding circuit is used to decrement or accumulate the reference data and the residual data to generate original data on both sides of the reference data
  • the relevant part can refer to the part of the description of the method embodiment.
  • the device embodiments described above are merely illustrative.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in One place, or it can be distributed to multiple network units.
  • Some or all of the modules may be selected according to actual needs to achieve the objectives of the solutions of the embodiments. Those of ordinary skill in the art can understand and implement it without creative work.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)

Abstract

La présente invention porte sur un procédé de traitement de données, un système, un codeur et un décodeur. Le procédé consiste à : obtenir le surdébit de codage d'une pluralité de premiers modes de codage pour un bloc de données devant être codé, et déterminer un mode de codage cible avec le plus petit surdébit de codage parmi la pluralité de premiers modes de codage ; et coder ledit bloc de données dans le mode de codage cible. De cette manière, une compression et un codage efficaces de données devant être codées peuvent être mis en œuvre.
PCT/CN2019/097771 2019-07-25 2019-07-25 Procédé de traitement de données, système, codeur et décodeur WO2021012278A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201980031021.0A CN112119593A (zh) 2019-07-25 2019-07-25 一种数据的处理方法、系统、编码器及解码器
PCT/CN2019/097771 WO2021012278A1 (fr) 2019-07-25 2019-07-25 Procédé de traitement de données, système, codeur et décodeur

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/097771 WO2021012278A1 (fr) 2019-07-25 2019-07-25 Procédé de traitement de données, système, codeur et décodeur

Publications (1)

Publication Number Publication Date
WO2021012278A1 true WO2021012278A1 (fr) 2021-01-28

Family

ID=73799650

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/097771 WO2021012278A1 (fr) 2019-07-25 2019-07-25 Procédé de traitement de données, système, codeur et décodeur

Country Status (2)

Country Link
CN (1) CN112119593A (fr)
WO (1) WO2021012278A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114071154A (zh) * 2021-11-10 2022-02-18 湖南国科微电子股份有限公司 一种数据解码方法、装置、电子设备及可读存储介质

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022147745A1 (fr) * 2021-01-08 2022-07-14 深圳市大疆创新科技有限公司 Procédé de codage, procédé de décodage, appareil de codage et appareil de décodage
CN117529912A (zh) * 2021-06-04 2024-02-06 华为技术有限公司 3d地图的编解码方法及装置
CN113596470A (zh) * 2021-06-30 2021-11-02 深圳市朗强科技有限公司 应用中压缩算法的超高清视频无线发送、接收方法及设备

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101783953A (zh) * 2010-03-08 2010-07-21 北京空间机电研究所 Jpeg-ls游程编码硬件实现方法
CN106529670A (zh) * 2016-10-27 2017-03-22 中国科学院计算技术研究所 一种基于权重压缩的神经网络处理器、设计方法、芯片
CN107995494A (zh) * 2017-12-12 2018-05-04 珠海全志科技股份有限公司 视频图像数据的压缩方法与解压方法、计算机装置、计算机可读存储介质
WO2018129614A1 (fr) * 2017-01-10 2018-07-19 Blackberry Limited Procédés et dispositifs de codage de vecteurs de mouvement en compression vidéo
CN108377033A (zh) * 2018-04-20 2018-08-07 国网辽宁省电力有限公司沈阳供电公司 基于多旋翼无人机的输电线路巡检与线路图对比修改系统
CN110049321A (zh) * 2018-01-16 2019-07-23 腾讯科技（深圳）有限公司 视频编码方法、装置、设备及存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101783953A (zh) * 2010-03-08 2010-07-21 北京空间机电研究所 Jpeg-ls游程编码硬件实现方法
CN106529670A (zh) * 2016-10-27 2017-03-22 中国科学院计算技术研究所 一种基于权重压缩的神经网络处理器、设计方法、芯片
WO2018129614A1 (fr) * 2017-01-10 2018-07-19 Blackberry Limited Procédés et dispositifs de codage de vecteurs de mouvement en compression vidéo
CN107995494A (zh) * 2017-12-12 2018-05-04 珠海全志科技股份有限公司 视频图像数据的压缩方法与解压方法、计算机装置、计算机可读存储介质
CN110049321A (zh) * 2018-01-16 2019-07-23 腾讯科技（深圳）有限公司 视频编码方法、装置、设备及存储介质
CN108377033A (zh) * 2018-04-20 2018-08-07 国网辽宁省电力有限公司沈阳供电公司 基于多旋翼无人机的输电线路巡检与线路图对比修改系统

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114071154A (zh) * 2021-11-10 2022-02-18 湖南国科微电子股份有限公司 一种数据解码方法、装置、电子设备及可读存储介质

Also Published As

Publication number Publication date
CN112119593A (zh) 2020-12-22

Similar Documents

Publication Publication Date Title
WO2021012278A1 (fr) Procédé de traitement de données, système, codeur et décodeur
JP4801160B2 (ja) 逐次改善可能な格子ベクトル量子化
RU2565877C2 (ru) Способ и устройство для определения соответствия между синтаксическим элементом и кодовым словом для кодирования переменной длины
CN107565971B (zh) 一种数据压缩方法及装置
JP2006126810A (ja) 後方適応規則を用いた整数データの無損失適応Ｇｏｌｏｍｂ−Ｒｉｃｅ符号化および復号化
CN106202213B (zh) 一种fpga二进制文件压缩、解压方法及压缩、解压装置
US11184021B2 (en) Using predicates in conditional transcoder for column store
US12010352B2 (en) Coding method and related device
KR101870594B1 (ko) 스펙트럼의 피크 위치의 코딩 및 디코딩
JP2003218703A (ja) データ符号化装置及びデータ復号装置
KR100484137B1 (ko) 개선된 허프만 디코딩 방법 및 장치
KR20190040063A (ko) 인덱스 코딩 및 비트 스케줄링을 갖는 양자화기
JP2016510454A (ja) 頂点誤差訂正のための方法および装置
CN104125475A (zh) 一种多维量子数据压缩、解压缩方法及装置
CN112449191B (zh) 压缩多个图像的方法、解压缩图像的方法和装置
CN116302670A (zh) 编解码方法、编解码器、芯片、硬盘、通讯系统
CN114286104A (zh) 基于频域分层的图像压缩方法、设备及存储介质
US10931303B1 (en) Data processing system
Jagadeesh et al. An approach for image compression using adaptive Huffman coding
WO2020258188A1 (fr) Procédé de décodage, décodeur et système de décodage
CN111832257A (zh) 编码数据的条件转码
JP2001211082A (ja) Ｐｃｍ信号符号化方法及び復号化方法
KR100982666B1 (ko) 컨텍스트 기반 적응적 가변길이 코딩의 디코딩 장치 및 디코딩을 위한 테이블 탐색 방법
WO2014000443A1 (fr) Procédé et dispositif de compression et de décompression de données d'image
US20240137045A1 (en) Noniterative entropy coding

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19938171

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19938171

Country of ref document: EP

Kind code of ref document: A1