WO2021012278A1

WO2021012278A1 - Data processing method, system, encoder, and decoder

Info

Publication number: WO2021012278A1
Application number: PCT/CN2019/097771
Authority: WO
Inventors: 阮肇夏; 赵文军
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2019-07-25
Filing date: 2019-07-25
Publication date: 2021-01-28
Also published as: CN112119593A

Abstract

A data processing method, a system, an encoder, and a decoder. The method comprises: obtaining the encoding overhead of multiple first encoding modes for a data block to be encoded, and determining a target encoding mode with the smallest encoding overhead from the multiple first encoding modes; and encoding said data block in the target encoding mode . In this way, effective compression and encoding of data to be encoded can be implemented.

Description

Data processing method, system, encoder and decoder

Technical field

This application relates to the technical field of data processing, and in particular to a data processing method, system, encoder and decoder.

Background technique

With the development of neural network technology, the scale of neural network models is getting larger and larger; take CNN (Convolutional Neural Networks, convolutional neural network) network as an example, in which weight data is the main parameter, and the parameter scale is large. The medium occupies high bandwidth, and requires high hardware computing and storage capabilities, which in turn limits the application of neural network models.

Therefore, how to reduce the calculation scale of the convolutional neural network model without affecting the calculation accuracy of the network convolutional neural network model, which will speed up the calculation and processing speed of hardware devices, save storage resources and expand the convolutional neural network The scope of application is of great significance.

Summary of the invention

In view of this, one of the objectives of the present invention is to provide a data processing method, system, encoder and decoder to achieve effective compression of the data to be encoded.

In the first aspect, an embodiment of the present invention provides a data processing method, including:

Acquiring the coding overhead of the multiple first coding modes for the data block to be coded, and determining the target coding mode with the smallest coding overhead from the multiple first coding modes;

Encoding the data block to be encoded by the target encoding method.

In the second aspect, an embodiment of the present invention provides a data processing method, including:

Parsing the data to be decoded, determining a target encoding method corresponding to the data to be decoded among a plurality of preset first encoding methods, and determining a corresponding target decoding method according to the target encoding method;

The data to be decoded is decoded by the target decoding mode to generate decoded data.

In the third aspect, an embodiment of the present invention provides a data processing method, including:

Using data at a preset position in the data block to be encoded as reference data, respectively calculating residual values of every two adjacent data located on both sides of the reference data;

The residual value and the reference data are written into the coded stream data.

In a fourth aspect, an embodiment of the present invention provides a data processing method, including:

Analyze the data block to be decoded and obtain the reference data and residual data in the data block to be decoded

Performing a decrement operation or an accumulation operation on the reference data and the residual data to generate original data on both sides of the reference data.

In a fifth aspect, an embodiment of the present invention provides a data processing method, which is applied to a configurator, a memory, and a processor connected in sequence, including:

Performing compression encoding on the data block to be encoded formed by the preset data by the configurator to generate a compressed code stream, and writing the compressed code stream into the memory, so as to reduce the write bandwidth to the memory;

The processor reads the compressed code stream in the memory and performs decompression processing on the compressed code stream to reduce the read bandwidth of the memory.

In a sixth aspect, an embodiment of the present invention provides a data processing system, including: a configurator, a memory, and a processor connected in sequence;

The configurator is used to compress and encode a data block to be encoded formed by preset data to generate a compressed code stream, and write the compressed code stream into the memory, so as to reduce the write bandwidth to the memory;

The processor is configured to read the compressed code stream in the memory, and perform decompression processing on the compressed code stream, so as to reduce the read bandwidth of the memory.

In a seventh aspect, an embodiment of the present invention provides a movable platform, and the movable platform includes:

Body

A power system is provided in the body, and the power system is used to provide power to the movable platform; and, a data processing system as described in the sixth aspect.

In an eighth aspect, an embodiment of the present invention provides an encoder, including: a selection circuit and an encoding circuit;

The selection circuit is configured to obtain the coding overhead of the data block to be coded by the multiple first coding modes, and determine the target coding mode with the smallest coding overhead from the multiple first coding modes;

The encoding circuit encodes the data block to be encoded by using the target encoding method.

In a ninth aspect, an embodiment of the present invention provides an encoder, the encoder includes: a residual calculation circuit and an encoding circuit;

The residual calculation circuit is configured to use the data at a preset position in the data block to be encoded as reference data, and respectively calculate the residual value of every two adjacent data located on both sides of the reference data;

The coding circuit is configured to write the residual value and the reference data into the code stream data according to the code stream structure corresponding to the residual coding mode; wherein, the code stream structure corresponding to the residual coding mode Including: the reference data, the bit width occupied by the residual value, and the residual value.

In a tenth aspect, an embodiment of the present invention provides a decoder, the decoder including: a parsing circuit and a decoding circuit;

The parsing circuit is configured to analyze the data to be decoded, determine a target encoding method corresponding to the data to be decoded among a plurality of preset first encoding methods, and determine a corresponding target decoding method according to the target encoding method;

The decoding circuit is used to decode the data to be decoded by the target decoding mode to generate decoded data.

In an eleventh aspect, an embodiment of the present invention provides a decoder, which includes: a parsing circuit and a decoding circuit;

The analysis circuit is used to analyze the data block to be decoded, and obtain the reference data and residual data in the data block to be decoded

The decoding circuit is used to perform a decrement operation or an accumulation operation on the reference data and the residual data to generate original data on both sides of the reference data.

A data processing method, system, encoder and decoder provided by various embodiments of the present invention,

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor.

FIG. 1 is a schematic flowchart of a data processing method provided in an embodiment of the present invention;

2 is a schematic diagram of a code stream structure corresponding to a run-length encoding method provided in an embodiment of the present invention;

Fig. 3 is a schematic diagram of a code stream structure corresponding to an original encoding method provided in an embodiment of the present invention;

4 is a schematic diagram of residual calculation provided in an embodiment of the present invention;

Fig. 5 is a schematic diagram of a code stream structure corresponding to a residual coding method provided in an embodiment of the present invention;

FIG. 6 is a schematic flowchart of a method for performing data processing according to a hybrid encoding mode provided in an embodiment of the present invention;

Fig. 7 is a schematic diagram of residual calculation provided in an embodiment of the present invention;

FIG. 8 is a schematic diagram of a code stream structure corresponding to a hybrid encoding method provided in an embodiment of the present invention;

FIG. 9 is a schematic flowchart of another method for performing data processing in a hybrid encoding manner provided in an embodiment of the present invention;

FIG. 10 is a schematic diagram of a code stream structure corresponding to a hybrid encoding method provided in an embodiment of the present invention;

Figure 11 is a schematic flow chart of a method for decoding data to be decoded provided in an embodiment of the present invention;

FIG. 12 is a schematic flowchart of a method for processing data to be coded provided in an embodiment of the present invention;

FIG. 13 is a schematic flowchart of a method for processing preset data provided in an embodiment of the present invention;

Figure 14 is a schematic diagram of a system for processing preset data provided in an embodiment of the present invention;

Figure 15 is a schematic diagram of a data processing system provided in an embodiment of the present invention;

Figure 16 is a schematic structural diagram of a movable platform provided in an embodiment of the present invention;

Figure 17 is a schematic structural diagram of an encoder provided in an embodiment of the present invention;

Figure 18 is a schematic diagram of the functional architecture of an encoder provided in an embodiment of the present invention;

FIG. 19 is a schematic structural diagram of a decoder provided in an embodiment of the present invention;

FIG. 20 is a schematic diagram of the functional architecture of a decoder provided in an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

In the prior art, the huge amount of data in the neural network model limits the application of the neural network model; based on this, the embodiments of the present invention provide a data processing method, system, encoder and decoder.

Fig. 1 is a schematic flowchart of a data processing method provided in an embodiment of the present invention. Referring to Fig. 1, the method includes the following steps S1-S2:

S1. Obtain the coding overhead of the multiple first coding modes for the data block to be coded, and determine the target coding mode with the smallest coding overhead from the multiple first coding modes.

The foregoing encoding overhead refers to the bit width occupied by the code stream data generated after encoding the data block to be encoded.

Furthermore, in the embodiment of the present invention, the aforementioned target encoding method with the smallest encoding overhead refers to an encoding method with the smallest data amount of the compressed code stream obtained after compression encoding of the data to be encoded among the preset multiple first encoding methods.

S2. Encode the data block to be encoded by using the target encoding method.

In the embodiment of the present invention, multiple first coding modes are set, and then the coding mode with the least coding overhead is selected from the multiple first coding modes for the data block to be coded for coding, thereby achieving the best compression effect.

In this embodiment, the target encoding methods determined for different data blocks to be encoded may be different, and then encoding is performed by selecting a suitable encoding method with the best compression effect for the data blocks to be encoded, so that the compression obtained after compression The data volume of the code stream is the smallest.

In an embodiment of the present invention, the aforementioned data block to be encoded is obtained from an original data block, and the original data block contains data to be compressed and encoded. In this embodiment, the above method further includes the following steps A10-A20:

Step A10: Determine whether there is a redundant data block in the original data block, and if so, encode the redundant data block according to the second coding mode.

In this embodiment, it is first determined whether there is redundant data in the original data block. Optionally, it is determined whether there is a redundant data block in the original data block in the following manner:

Find out whether there is data in the original data block that appears continuously and the number of repetitions is greater than the preset value. If so, it is determined that there is a redundant data block, and the continuous data is regarded as a redundant data block; if not, it is determined that it does not exist Redundant data block.

The aforementioned preset value may be determined according to the accuracy of compression coding. Exemplarily, the foregoing preset value may be set to 4. At this time, taking the original data block as 01222223416 as an example, the repeated and consecutive data in the original data block is 2, and the number of consecutive occurrences is 5 ( Greater than 4), the redundant data block in the original data block is 22222.

Step A20: Use the remaining data block from the original data block except the redundant data block as the data block to be encoded.

In an embodiment of the present invention, after it is determined that the original data block contains the redundant data block, the redundant data block is encoded by the second encoding method, and the remaining data block of the original data block is removed from the redundant data block As the above-mentioned data block to be encoded, data processing is performed according to the above-mentioned steps S1-S2 for the data block to be encoded.

In an embodiment of the present invention, the above-mentioned second encoding method includes: run-length encoding; and in this embodiment, the redundant data block is encoded in a run-length encoding manner.

Specifically, encoding the redundant data block in a run-length encoding manner includes:

The redundant data block is written into the coded stream data according to the code stream structure corresponding to the run-length encoding; the code stream structure corresponding to the run-length encoding includes: the flag bit of the encoding mode, the start value and the run length .

Exemplarily, in the process of encoding the redundant data block using the run-length coding method, the data of each row in the original data block is scanned row by row, and the redundant data block is further obtained after scanning a certain row. The initial value and run length of the redundant data block, taking the redundant data block as 22222 as an example, the initial value of the redundant data block is 2 and the run length is 5, the initial value of the redundant data block, The run length is written into the code stream structure corresponding to the run length coding, and the code stream structure also contains a coding mode flag bit, which indicates that the coding mode adopted for the data of the code stream is run length coding. After the current line scan is completed, continue to scan the next line. Optionally, in order to improve the encoding accuracy, set not to perform cross-line scan.

For example, the code stream structure corresponding to run-length coding is shown in Figure 2:

Among them, the flag bit 00 indicates that the coding mode corresponding to the code stream is the run-length coding mode, base represents the starting value, and len represents the run length.

In another embodiment of the present invention, after the redundant data blocks are removed from the original data blocks to obtain the remaining data blocks, the remaining blocks are first divided according to a specified size to obtain multiple data blocks, and the obtained multiple data blocks Each data block has the same size, and then each divided data block is regarded as a data block to be coded, and each data block to be coded is processed and coded according to the above steps S1-S2 respectively.

Optionally, in this embodiment of the present invention, before dividing the remaining data blocks according to a specified size to obtain multiple data blocks, if the data amount of the data contained in the remaining data blocks does not satisfy an integer multiple of the foregoing specified size, It is not enough to divide the remaining data blocks into multiple data blocks of the specified size. At this time, the remaining data blocks are filled with data so that the amount of data contained in the remaining data blocks after data filling is an integer multiple of the specified size. The data volume of each data block obtained after division is the specified size.

For example, the above-mentioned specified size is 16 data. When the total number of data contained in the remaining data block is not an integer multiple of 16, that is, when the data summary in the remaining data block is not 16M, M is an integer greater than or equal to 1. Then, the remaining data block is filled with data, so that the total data of the remaining data block after filling meets to be 16M.

In an embodiment of the present invention, the foregoing method of filling the remaining data blocks includes:

Copy the last data in the N remaining data blocks, and use the copied N data to fill the remaining data blocks so that the data volume of the remaining data blocks is an integer multiple of the above specified size; where N is greater than or equal to 1 The integer.

In this embodiment, the remaining data block is filled with data by using the last data in the remaining data block; optionally, the remaining data block may be filled with data by using a predetermined data.

Exemplarily, the remaining data blocks are divided into one data block according to each 16-bit data starting from the first data in the first place. After dividing one or more data blocks, if the last remaining data is: 0122254 Since the number of data bits of the remaining data is 7, the data at the end of the remaining data block is used for data filling, and the filled data is 0122254444444444. After data filling, all the data blocks divided by the remaining data block can be obtained. Then, the target encoding method is determined for each data block according to the above method, and the target encoding method is applied to encode each data block.

In the embodiment of the present invention, the remaining data blocks are filled with data so that the obtained data blocks have the same size, which is convenient for encoding the remaining data.

In an embodiment of the present invention, the above-mentioned multiple first encoding methods include: original encoding, residual encoding, and hybrid encoding; wherein, the hybrid encoding is after the data block to be encoded is divided into several data sub-blocks, Perform residual coding or original coding on each data block.

In an embodiment of the present invention, when the determined target editing mode is the original coding mode, the original coding mode is used to encode the to-be-coded data block, which specifically includes the following step B10:

Step B10: Write the original data in the data block to be coded into the code stream data according to the code stream structure corresponding to the original coding mode.

Wherein, the code stream structure corresponding to the above-mentioned original coding mode includes: the flag bit of the coding mode and original data.

In the embodiment of the present invention, the original encoding method includes: directly writing the original data in the data block to be encoded into the encoded bitstream data according to the code stream structure corresponding to the original encoding method.

Exemplarily, the code stream structure corresponding to the original encoding method is shown in Figure 3.

In this embodiment, taking the data block to be coded containing 16 data as an example, the above flag bit 01 is used to indicate that the coding mode corresponding to the code stream structure is the original coding mode (RAW16), and the above p0, p1,, and p15 are waiting Encode the original data in the data block.

In another embodiment of the present invention, when the above-mentioned target coding mode is the residual coding mode, encoding the data block to be coded by the target coding mode includes the following step S20:

S20. Using the data at a preset position in the data block to be encoded as reference data, respectively calculate the residual value of each two adjacent data located on both sides of the reference data; compare the residual value with the reference The data is written into the coded stream data according to the code stream structure corresponding to the residual coding method.

In this embodiment, the data at a preset position in the data block to be encoded is used as the reference data, and the residual values of the data on both sides of the reference data are respectively calculated. This starts from the reference data and moves to both sides of the reference data. Compared with the prior art, the calculation method can improve the efficiency of calculation.

In an optional embodiment of the present invention, the data block to be encoded may also be further divided into multiple data sub-blocks, by setting one or more reference data in each data sub-block, and calculating the two sides of each reference data The residual value of two adjacent data, which can further improve the efficiency of calculation, and further improve the efficiency of compression coding.

The process of calculating the residual is shown in Fig. 4, taking 16 pieces of data contained in the data block to be coded as an example, the 16 pieces of data are respectively: p0, p1, p2, p3, p4,, p14, p15. Using the data p8 at the center of the data block to be encoded as the reference data, use the following array to record the residual values of pixels other than p8:

Left side: diff[0]=p8-p7, diff[1]=p7-p6,..., diff[7]=p1-p0;

Right side: diff[8]=p8-p9, diff[9]=p9-p10,..., diff[14]=p14-p15;

In this embodiment, starting with the reference data p8, the residual values of two adjacent data are calculated on both sides respectively. Compared with the data in the initial position, the residual values are calculated, which can improve the calculation efficiency.

In another embodiment, the following array can be used to record the residual values of pixels other than p8:

Left side: diff[0]=p7-p8, diff[1]=p6-p7,..., diff[7]=p0-p1;

Right side: diff[8]=p9-p8, diff[9]=p10-p9,..., diff[14]=p15-p14.

It should be noted that the calculation direction can be adjusted as needed, and is not limited to the above two methods.

After the residual is calculated, the residual value and the corresponding reference data are written into the code stream data according to the code stream structure corresponding to the residual coding method to obtain the corresponding code stream data; the residual coding method corresponds to The code stream structure includes: the flag bit of the coding mode, the reference data, the bit width occupied by the residual value, and the residual value.

Exemplarily, the code stream structure corresponding to the residual coding method is shown in FIG. 5. In this embodiment, the above-mentioned data segment to be coded contains 16 pieces of data as an example, and the flag bit 10 is used to characterize the code used in the code stream structure. The method is the residual coding method (DPCM16), p8 is the reference data, Rb is the bit width occupied by the residual value, r0-r7 is the residual value, and r10-r15 is the residual value.

In an embodiment of the present invention, referring to FIG. 6, when the above-mentioned target encoding method is hybrid encoding, encoding the data block to be encoded by the hybrid encoding method specifically includes the following steps:

S21. Divide the to-be-coded data block to obtain at least two data sub-blocks.

In this embodiment, the method for dividing the data block to be coded may be to obtain a plurality of data sub-blocks with the same size evenly, or it may be divided into data sub-blocks with different sizes according to a set size.

S22: Using data at a preset position in any of the data sub-blocks as reference data, respectively calculate residual values of every two adjacent data located on both sides of the reference data.

In this embodiment, by using the data in the preset position in the data block to be encoded as the reference data, the residual values of the two adjacent data on both sides of the reference data are respectively calculated, and then the reference data is started from the reference data and the reference data Calculation and coding are performed on both sides of the data respectively, and the two-way calculation and coding method improves coding efficiency compared with the prior art.

S23: Write the residual value calculated for each of the data sub-blocks and the reference data into the coded stream data according to the code stream structure corresponding to the hybrid coding mode.

In this embodiment, the code stream structure corresponding to the foregoing hybrid encoding method includes: a flag bit of the encoding method, the reference data, the bit width occupied by the residual value, and the residual value.

Furthermore, in the hybrid coding method provided in this embodiment, multiple data sub-blocks are obtained by dividing the data block to be coded, and each data sub-block is coded in a residual coding manner.

Exemplarily, taking the data block to be coded into two data sub-blocks as an example, the calculation method of performing residual coding on both data sub-blocks is shown in FIG. 7.

Referring to FIG. 7, in this embodiment, the data contained in the data block to be encoded is: p0, p1, p2,, p14, p15, and the data block to be encoded is divided into two data sub-blocks, which are respectively data blocks : P0p1, p7 and data block: p8p9,,, p15. For two data sub-blocks, select one data at a preset position as parameter data (respectively p4, p12 in the figure), and then calculate the residual values of two adjacent data on both sides of the reference data. And write the code stream data according to the coding structure of the hybrid coding.

Exemplarily, the encoding structure corresponding to the hybrid encoding mode is shown in FIG. 8, where, in the hybrid encoding mode, the above-mentioned flag bit 10 is used to indicate that the encoding method adopted by the code stream structure is the residual encoding method (DPCM8_1&DPCM8_2), p4 and p12 are reference data respectively, Rb is the bit width occupied by the residual value, r0-r7 is the residual value, and r8-r14 is the residual value.

In another embodiment of the present invention, the aforementioned target encoding method includes: a hybrid encoding method. In this embodiment, referring to FIG. 9, encoding the data block to be encoded by the hybrid encoding method specifically includes the following steps S21'-S23':

S21', dividing the data block to be encoded to obtain at least two data sub-blocks; wherein the at least two data word blocks include a first data sub-block and a second data sub-block.

In this embodiment, the method for dividing the data block to be coded may be to obtain a plurality of data sub-blocks with the same size evenly, or it may be divided into data sub-blocks with different sizes.

S22', using the data at a preset position in the first data sub-block as reference data, respectively calculating the residual value of each two adjacent data located on both sides of the reference data, and combining the residual value with the The reference data is written into the coded stream data according to the code stream structure corresponding to the hybrid coding mode.

In this embodiment, taking the divided data sub-blocks as an example: the first data sub-block and the second data sub-block, the first data sub-block is coded according to the above residual coding method, and then the obtained residual The value and reference data are written into the coded code stream data according to the code stream structure corresponding to the hybrid coding mode.

S23'. Write the original data in the second data sub-block into the coded code stream data according to the code stream structure corresponding to the hybrid coding mode.

In this embodiment, the original data in the second data sub-block is written into the coded stream data directly according to the code stream structure corresponding to the mixed coding mode according to the original coding mode.

Furthermore, in the embodiment of the present invention, when the hybrid coding mode is adopted, after the data block to be coded is divided to obtain multiple data sub-blocks, the first data sub-block is coded according to the residual coding mode, and the second data sub-block is Encode according to the original encoding method; the first data sub-block and the second data sub-block may be obtained by dividing all the data sub-blocks equally according to the number, and the number of the first data sub-block is the same as the number of the second data sub-block ; Of course, the number of first data sub-blocks may be different from the number of second data sub-blocks.

In this embodiment, the code stream structure corresponding to the above-mentioned hybrid encoding method includes: the flag bit of the encoding method, the original data of the second data block, the reference data, the bit width occupied by the residual value, and the residual value.

Exemplarily, taking the data block to be encoded each containing 16-bit data, and then dividing the data block to be encoded to obtain two data blocks with 8-bit data as an example for description, an embodiment of the present invention provides a hybrid encoding method The coding structure is shown in Figure 10.

10, in this embodiment, the above-mentioned flag bit 11 is used to characterize that the coding mode corresponding to the bitstream structure is hybrid coding; the bitstream structure of mixed coding includes not only the coding mode representing "hybrid coding", but also Characterize the coding mode corresponding to each sub-data block. Preferably, 1 bit can be used to mark whether the code stream of the corresponding sub-data block is original coding or residual coding, for example, 0 is used to represent the original coding, and 1 is used to represent the residual coding. In addition, it is also possible to use a combination of 2 bits or more of the common mark combination coding, for example, using 00 to represent the combination of original coding and residual coding, 01 to represent the combination of residual coding and original coding, and 10 to represent residual coding And the combination of residual coding.

The code stream structure in this hybrid coding mode includes: (1) The code stream structure (RAW8_1&DPCM8_2) for the original coding and residual coding of the data sub-blocks. The coding mode flag bit of the code stream structure is 00, where p0- p8 is the original data, p12 is the reference data, r8-r15 is the residual value, and R'b is the bit width occupied by the residual value; (2) Perform residual coding on the data sub-block and the code stream structure corresponding to the original coding (DPCM8_1&RAW8), the coding mode flag bit of this coding mode is 01, where p4 is the reference data, p8-p15 is the original data, R'b is the bit width occupied by the residual value, and r8-r15 is the residual value; (3) Perform residual coding on data sub-blocks (DPCM8_1&DPCM8_2), the coding mode flag of this coding mode is 10; among them, p4 and p12 are reference data respectively, and R'b is the bit width occupied by the residual value , R0-r7 are residual values, and r8-r14 are residual values.

In an embodiment of the present invention, taking 16-bit data contained in a data block to be encoded as an example, the encoding overhead calculated for the same data block to be encoded by each of the foregoing encoding methods can be referred to as follows:

The encoding overhead for the original encoding of the data block to be encoded is: 16*8bit; where 16 is the number of data, and the space occupied by each data is 8bit.

The coding overhead for residual coding of the data block to be coded is: 8bit+bitlen_16*15+3bit; among them, 8bit is the bit width occupied by the reference data, bitlen_16 is the bit width occupied by a residual value, and 3bit is the bitlen 16 Occupied bit width.

The coding overhead for dividing the data block to be coded into two data blocks for residual coding is: 1bit+8bit+bitlen8_1*7+3bit+1bit+8bit+bitlen8_2*7+3bit, where the calculation unit is 8 data, and 1bit is used When the mark code stream is original coding or residual coding, 8bit is the bit width occupied by the reference data, bitlen8_1 and bitlen8_2 are the bit width occupied by a residual value, and 3bit is the bit width occupied by bitlen8_1 or bitlen8_2.

The encoding overhead of dividing the data block to be encoded into two data blocks for original encoding and residual encoding is: 1bit+8bit*8+1bit+8bit+bitlen8_2*7+3bit; among them, the first 8 data are original data, each The space occupied by the original data is 8bit, 1bit is used to mark whether the code stream is original encoding or residual encoding, the second 8bit is the bit width occupied by the reference data, and bitlen8_2 is the bit width occupied by a residual value. 3bit is the bit width occupied by bitlen8_2.

The coding overhead of dividing the data block to be coded into two data blocks for residual coding and original coding is: 1bit+8bit+bitlen8_1*7+3bit+1bit+8*8bit; for the definition of the calculation formula, please refer to the above coding overhead Calculation.

It should be noted that, in this embodiment, all the data blocks to be encoded contain 16-bit data as an example; but in practice, the data blocks to be encoded may also contain data such as 32 bits, 64 bits, etc.; The present invention does not limit this.

In the foregoing embodiment of the present invention, both the first encoding method and the second encoding method are lossless encoding, which can ensure the accuracy of the data to be encoded after compression and decompression.

The data processing method provided in the foregoing embodiment of the present invention can be applied to compress the weight data of a neural network, which can greatly reduce bandwidth overhead.

In the prior art, the use of weight pruning and sharing of pair weight data to reduce the parameter scale will reduce the convergence speed of the network. In addition, pruning requires manual setting of layer sensitivity, that is, fine-tuning of hyperparameters, which can be tedious and burdensome in some applications.

In the prior art, there are also methods for lossy compression of weight parameters, such as K-means scalar quantization, fixed-point representation, and binary networks. The disadvantage of these methods is that they may cause loss of accuracy. Lossless algorithms also use text compression methods, but text compression is based on the distribution and probability of the source, especially by building a dictionary to reduce information redundancy. This method is directly used in the model, and the effect is often not good. One of the most important reasons is that the probability of two identical parameters in the parameter distribution is low. In addition, the throughput rate of the above method is reduced, which limits the performance of the computing core.

Therefore, the embodiment of the present invention does not reduce the convergence speed of the network, nor does it need to fine-tune hyperparameters, and has high efficiency; in addition, the embodiment of the present invention does not cause a loss of the accuracy of the weight data, and ensures that the weight parameters will not introduce additional errors. And there is no need to retrain the network, and the system bandwidth can be reduced to a large extent. It also does not exist that after the weight data is compressed by the method provided in the foregoing embodiment of the present invention, the convergence speed of the network will not be reduced, and the efficiency is high. And it will not cause the loss of the accuracy of the weight data.

Regarding the encoded data processed by the data processing method described in the foregoing embodiment, an embodiment of the present invention also provides a data processing method to decode the encoded data data (as data to be decoded).

Referring to FIG. 11, an embodiment of the invention provides a data processing method, which includes the following steps S41-S42:

S41. Analyze the data to be decoded, determine a target encoding method corresponding to the data to be decoded among a plurality of preset first encoding methods, and determine a corresponding target decoding method according to the target encoding method.

The above-mentioned data to be decoded includes the coding mode flag bit; the above-mentioned analysis of the data to be decoded can obtain the value of the flag bit contained in the decoded data, and then according to the preset mapping relationship between the value of the flag bit and the first coding mode, The target encoding method corresponding to the data to be decoded is determined, and the corresponding target decoding method can be determined according to the target encoding method; for example, the target decoding method is the inverse operation of the target encoding method.

For example, once again the embodiment shown in FIG. 2 is used for description. First, the flag bit 00 in the code stream structure is obtained, and the corresponding encoding method is determined to be run-length coding according to the flag bit 00. Then obtain the corresponding target decoding mode as run-length decoding.

S42. Decode the data to be decoded by using the target decoding mode to generate decoded data.

In the embodiment of the present invention, after the encoding method of the data to be decoded, the corresponding target decoding method can be determined, and the target decoding method may be the inverse operation of the encoding method.

In an embodiment of the present invention, the above-mentioned first encoding method includes one or more of run-length encoding, original data encoding, residual encoding, and hybrid encoding;

Wherein, the hybrid coding is to divide a data block to be coded to generate multiple data sub-blocks, and perform residual coding or original data coding on each of the data blocks.

The above-mentioned residual coding is to use the data at the preset position in the data block to be coded as reference data, respectively calculate the residual value of each two adjacent data located on both sides of the reference data, and combine the residual value and the The reference data is written into the coded code stream data according to the code stream structure corresponding to the residual coding mode.

Optionally, corresponding to the foregoing encoding mode, the foregoing target decoding mode includes one or more of run-length decoding, original data copy, residual decoding, and hybrid decoding;

Wherein, the above-mentioned hybrid decoding is to divide the data block to be decoded to generate multiple data sub-blocks. The division method may be divided according to flag bits, and residual decoding or original data copy is performed on each data block. .

The following describes each decoding method:

The above-mentioned residual decoding is to read the corresponding reference data, the bit length occupied by the residual value and the residual value from the bit stream structure of the data to be decoded, according to the bit length occupied by the reference data and the residual value Perform the inverse operation of the residual calculation with the residual value to generate decoded data.

When the target decoding method is residual decoding, using the residual decoding method to decode the data to be decoded includes: reading the corresponding reference data from the code stream structure of the data to be decoded and the residual value The bit length and residual value of, perform the inverse operation of residual calculation according to the reference data and residual value to obtain the original data.

When the target decoding mode is run-length decoding, using run-length decoding to decode the data to be decoded includes:

The start value and the run length are read from the code stream structure of the data to be decoded, and the data to be decoded is decoded and restored according to the start value and the run length to obtain decoded data.

When the target decoding mode is the original data copy, using the original data copy mode to decode the data to be decoded includes:

Reading the corresponding encoded data from the code stream structure of the data to be decoded, and copying the encoded data to obtain decoded data.

In the foregoing embodiment of the present invention, if the data volume of the data obtained by decoding the data to be decoded by the target decoding method is greater than the preset data volume, the part of the data exceeding the preset data volume is discarded to generate The decoded data.

Referring to FIG. 12, another embodiment of the present invention also provides a data processing method, including the following steps S51-S52:

S51: Using data at a preset position in the data block to be encoded as reference data, respectively calculating residual values of every two adjacent data located on both sides of the reference data.

Referring again to FIG. 4, still taking as an example that the data block to be encoded contains 16 data, the 16 data are: p0, p1, p2, p3, p4, ,,, p14, p15. Take the data p8 at the center of the data block to be encoded as the reference data, start with the reference data p8, calculate the residual values of the two adjacent data to both sides, and calculate the residuals relative to the data at the initial position Value, can improve the efficiency of calculation. Moreover, the intermediate data and residual data can be accumulated or decremented to both ends during decoding, which improves the parallelism of decoding.

Optionally, the foregoing data block to be encoded may be obtained in the following two ways:

Manner 1: Divide the original data block according to a specified size to obtain multiple data sub-blocks, and use each of the data sub-blocks as the data block to be encoded.

Manner 2: Divide the original data block according to the specified size to obtain multiple data sub-blocks, and use the first data block of the multiple data blocks as the data block to be encoded.

Furthermore, in the second embodiment, part of the data sub-blocks (that is, the first data block) of the multiple data sub-blocks is used as the above-mentioned data block to be coded, and the data in the preset position in the data block to be coded is used as For reference data, calculate the residual values of every two adjacent data located on both sides of the reference data.

S52. Write the residual value and the reference data into the coded stream data.

When according to the above method two, the first data block is used as the data block to be encoded, the data blocks other than the first data block among the multiple data sub-blocks obtained are based on the code stream structure corresponding to the original encoding method , Write the coded stream data; the code stream structure corresponding to the original coding mode includes: the flag bit of the coding mode and the original data.

Optionally, the above-mentioned residual value and reference data are written into the coded stream data according to the code stream structure corresponding to the residual coding mode, and the code stream structure corresponding to the residual coding mode includes: reference data, residual value And the bit width occupied by the residual value.

In this embodiment, by using the data located at the preset position in the data block to be encoded as the reference data, the residual values of the two adjacent data on both sides of the reference data are respectively calculated. Compared with the prior art, the two-way calculation method can improve the efficiency of calculation.

In an optional embodiment of the present invention, the data sub-block can also be further divided into multiple data blocks, by setting one or more reference data in each data block, and calculating the phases on both sides of each reference data. The residual value of the adjacent two data can further improve the calculation efficiency and thus the coding efficiency.

Correspondingly, when parsing the encoded data obtained by the above method as the data block to be decoded in an embodiment of the present invention, first obtain the reference data and residual data in the data block to be decoded; wherein, the data block to be decoded The data block is encoded and generated; the reference data is data at a preset position in the original data; the residual data is the residual value of every two adjacent data located on both sides of the reference data during the encoding process.

Then, the reference data and the residual data are decremented or accumulated to generate the original data on both sides of the reference data.

For example, referring to FIG. 4, taking as an example the data block to be encoded contains 16 data, the 16 data are: p0, p1, p2, p3, p4, ,, p14, p15. Using the data p8 at the center of the data block to be encoded as the reference data, use the following array to record the residual values of pixels other than p8:

Left side: diff[0]=p8-p7, diff[1]=p7-p6,..., diff[7]=p1-p0;

Right side: diff[8]=p8-p9, diff[9]=p9-p10,..., diff[14]=p14-p15.

In the decoding process, p8 and diff[8] to diff[14] are used for decrement operations to obtain p9-p14 data respectively. In the decoding process, p8 and diff[0] to diff[7] are used for decrement operations. The data of p7-p0 can be obtained separately. For example, as shown in Figure 5, decrement operations with r0 to r7 can respectively obtain data p7-p0, and decrement operations with r9 to r15 can respectively obtain data p9-p14. Exemplarily, p8-diff[0] can get p7, p8-diff[0]-diff[1] can get p6, and other things are the same.

Left side: diff[0]=p7-p8, diff[1]=p6-p7,..., diff[7]=p0-p1;

Right side: diff[8]=p9-p8, diff[9]=p10-p9,..., diff[14]=p15-p14.

In the decoding process, p8 and diff[8] to diff[14] are used for accumulation operation to obtain the data of p9-p14 respectively. In the decoding process, p8 and diff[0] to diff[7] are used for accumulation operation The data of p7-p0 can be obtained separately. For example, as shown in Figure 5, the data of p7-p0 can be obtained by accumulating operations with r0 to r7, and the data of p9-p14 can be obtained by accumulating operations with r9 to r15 respectively. Exemplarily, p8+diff[0] can get p7, p8+diff[0]+diff[1] can get p6, and the others are the same.

Furthermore, in the embodiment of the present invention, two-way simultaneous decoding operations can be performed on the data block to be decoded from both sides of the reference data, which has the positive effect of high decoding efficiency.

Referring to the embodiment shown in FIG. 13, this embodiment provides a data processing method, which is applied to a configurator, a memory, and a processor connected in sequence, and the method includes the following steps S61-S62:

S61: Perform compression encoding on the data block to be encoded formed by the preset data by the configurator to generate a compressed code stream, and write the compressed code stream into the memory, so as to reduce the write bandwidth to the memory.

In an embodiment of the present invention, the above-mentioned configurator is provided with an encoder, and the configurator specifically compresses and encodes the data block to be encoded by the encoder to generate a code stream.

S62. Read the compressed code stream in the memory by the processor, and perform decompression processing on the compressed code stream to reduce the read bandwidth of the memory.

In an embodiment of the present invention, a decoder is provided on the aforementioned processor, and the processor specifically decompresses the aforementioned compressed code stream through the decoder.

In an embodiment of the present invention, the above-mentioned preset data is weight data based on a convolutional neural network; and in this embodiment, the configurator is used to allocate the weight data and compress and encode the weight data by an encoder. The compressed code stream is written into the memory; the processor is used to read the compressed code stream corresponding to the weight data from the memory, decompress the compressed code stream through the decoder to generate weight data, and run the convolution based on the weight data Neural Networks.

Optionally, the foregoing processor further includes a static random-access memory (Static Random-Access Memory, SRAM), which is used to read the compressed code stream corresponding to the weight data from the memory, so that the processor can pass through The decoder performs decompression processing on the compressed code stream in the SRAM to generate weight data, and runs the convolutional neural network based on the weight data.

The embodiment shown in FIG. 14 shows a schematic diagram of the relationship between the configurator, the memory and the processor; referring to FIG. 14, the above-mentioned configurator is connected to the memory, and the memory is connected to the processor. In this implementation The above-mentioned memory may be a double data rate synchronous dynamic random access memory (Double Data Rate Synchronous Dynamic Random Access Memory, DDR SDRAM); in this embodiment, an encoder is provided in the configurator, and the configurator is specifically coded The processor compresses and encodes the data block to be encoded to generate a compressed code stream; the processor is provided with a decoder and a static random access memory, and the static random access memory is used to read the compressed code stream corresponding to the weight data from the memory. A decoder is used to decompress the compressed code stream in the SRAM to generate weight data, and run a convolutional neural network based on the weight data.

In the above-mentioned embodiment of the present invention, in order to reduce the bandwidth overhead when the weight data is read from the memory to the static random access memory in the processor, the encoder decompresses the compressed weight data; accordingly, the configurator will Before the weight data is written into the memory, the encoder will compress the weight data. This process uses encoders and decoders to losslessly compress and decompress the weight data, ensuring that the weight data does not introduce additional errors and does not require retraining the network. At the same time, the system bandwidth can be reduced to a large extent.

Preferably, the encoder performs lossless compression processing on the weight data to ensure that the weight parameters do not introduce additional errors and do not need to retrain the network.

In an embodiment of the present invention, in the foregoing method, the specific process of the configurator performing compression encoding on the data block to be encoded to generate a compressed code stream includes: acquiring the encoding overhead of multiple first encoding methods for the data block to be encoded, In the encoding method, determine the target encoding method with the least encoding overhead; then use the target encoding method to compress and encode the data block to be encoded formed by the preset data to generate a compressed code stream; the preset data may be the weight of the convolutional neural network data.

Wherein, the aforementioned encoding overhead is the bit width occupied by the code stream data generated after the data block to be encoded is encoded.

In this embodiment, for different data blocks to be encoded, an encoding method with the least encoding overhead is selected for encoding, which can achieve effective compression of the data blocks to be encoded.

In an embodiment of the present invention, the above-mentioned configurator first determines whether there is a redundant data block in the original data block, and if so, the redundant data block is encoded according to the second encoding method. Optionally, the second encoding method includes: run-length encoding. The specific configurator writes the redundant data block into the code stream data according to the code stream structure corresponding to the run-length encoding; the code stream corresponding to the run-length encoding The structure includes: the flag bit of the encoding method, the starting value and the run length.

After the redundant data blocks are removed from the original data blocks, the remaining data blocks can be used as the data blocks to be encoded, and the encoding method with the least encoding overhead is selected for encoding.

Alternatively, the remaining data blocks may be divided according to a specified size to obtain multiple data blocks, each data block is used as the data block to be encoded, and the encoding method with the least encoding overhead is selected for encoding; in this embodiment, if the remaining data block If the amount of data does not satisfy an integer multiple of the specified size, then the remaining data blocks are first filled with data so that the size of each data block obtained after division is the specified size.

In an embodiment of the present invention, the above-mentioned configurator determines whether there is a redundant data block in the original data block in the following manner:

The configurator searches the original data block to see whether there is data that continuously appears and the number of repetitions is greater than a preset value, and if so, determines that there is a redundant data block, and uses the continuous data as the redundant data block; if not , It is determined that there is no redundant data block.

In an embodiment of the present invention, the above-mentioned multiple first encoding methods include: an original encoding method, a residual encoding method, and a hybrid encoding method; wherein the hybrid encoding method is to divide the data block to be encoded into several data After sub-blocks, residual coding or original coding is performed on each of the data blocks.

Optionally, when the target coding mode is the original coding mode, encoding the data block to be coded by the target coding mode includes:

Writing the original data in the data block to be coded into the coded code stream data according to the code stream structure corresponding to the original coding mode;

Wherein, the code stream structure corresponding to the original coding mode includes: a flag bit of the coding mode and original data.

Optionally, when the target coding mode is the residual coding mode, the encoding the data block to be coded by the target coding mode includes:

Using the data at a preset position in the data block to be encoded as reference data, the residual values of each two adjacent data located on both sides of the reference data are calculated respectively, and the residual value and the reference data are calculated according to The code stream structure corresponding to the residual coding method is written into the coded code stream data;

Wherein, the code stream structure corresponding to the residual coding mode includes: a flag bit of the coding mode, the reference data, the bit width occupied by the residual value, and the residual value.

Optionally, when the target coding mode is a hybrid coding mode, encoding the data block to be coded by the target coding mode includes:

Dividing the data block to be encoded to obtain at least two data sub-blocks;

Using data at a preset position in any of the data sub-blocks as reference data, respectively calculating residual values of every two adjacent data located on both sides of the reference data;

The residual value calculated for each of the data sub-blocks and the reference data are respectively written into the coded code stream data according to the code stream structure corresponding to the hybrid coding mode.

Wherein, in this embodiment, the code stream structure corresponding to the above-mentioned hybrid coding mode includes: the flag bit of the coding mode, the reference data, the bit width occupied by the residual value, and the residual value.

Dividing the data block to be encoded to obtain at least two data sub-blocks; wherein the at least two data word blocks include a first data sub-block and a second data sub-block;

Using the data at a preset position in the first data sub-block as reference data, the residual values of each two adjacent data located on both sides of the reference data are respectively calculated, and the residual value and the reference data Write the encoded code stream data according to the code stream structure corresponding to the hybrid encoding method; write the original data in the second data sub-block into the code stream structure corresponding to the hybrid encoding method Stream data;

Wherein, the code stream structure corresponding to the hybrid encoding method in this embodiment includes: the flag bit of the encoding method, the original data of the second data block, the residual value, the reference data, and the residual value The occupied bit width and the residual value.

Optionally, the processor reading the compressed code stream in the memory and decompressing the compressed code stream includes:

The processor parses the compressed code stream, determines a target encoding method corresponding to the compressed code stream among a plurality of preset first encoding methods, and determines a corresponding target decoding method according to the target encoding method;

The compressed code stream is decoded by the target decoding method to obtain the preset data.

Optionally, the code stream structure of the aforementioned compressed code stream includes a flag bit; the processor determines that it corresponds to the compressed code stream according to a preset mapping relationship between the value of the flag bit and the first encoding mode The target encoding method.

The foregoing target decoding methods include: any one or more of run-length decoding, original data copy, residual decoding, and hybrid decoding;

Wherein, the hybrid decoding is to divide the data block to be decoded to generate multiple data sub-blocks, and perform residual decoding or original data copy on each of the data word blocks.

The following describes the calculation process when the target decoding method is run-length decoding, original data copy and residual decoding:

When the target decoding mode is run-length decoding, the using the target decoding mode to decode and restore the compressed code stream includes:

The initial value and the run length are read from the code stream structure of the compressed code stream, and the compressed code stream is decoded and restored according to the initial value and the run length to obtain corresponding preset data.

When the target decoding mode is the original data copy, the decoding and restoring the compressed code stream using the target decoding mode includes:

Reading the corresponding coded data from the code stream structure of the compressed code stream, and copying the coded data to obtain corresponding preset data.

When the target decoding mode is residual decoding, the using the target decoding mode to decode and restore the compressed code stream includes:

Read the corresponding reference data, the bit length occupied by the residual value and the residual value from the bit stream structure of the compressed bit stream, and execute according to the reference data, the bit length occupied by the residual value and the residual value The inverse operation of the residual calculation obtains the corresponding preset data.

The decoding and restoring the compressed code stream using the target decoding method to obtain the preset data includes:

If the data volume of the data obtained by decoding and restoring the compressed code stream by the target decoding method is greater than the preset data volume, discarding part of the data exceeding the preset data volume to obtain the preset data.

It should be noted that, for the specific execution steps of the configurator and the processor in the foregoing embodiment of the present invention, reference may be made to the introduction of the method in each of the foregoing first to fourth aspects.

In an embodiment of the present invention, referring to FIG. 15, a data processing system is also provided, including: a configurator 81, a memory 80, and a processor 82 connected in sequence.

The configurator 81 is configured to compress and encode a data block to be encoded formed by preset data to generate a compressed code stream, and write the compressed code stream into the memory 80 to reduce the write bandwidth to the memory 80.

The processor 82 is configured to read the compressed code stream in the memory 80, and decompress the compressed code stream, so as to reduce the read bandwidth of the memory 80.

Optionally, an encoder is provided on the configurator 81, and a decoder is provided on the processor 82;

The configurator is used to compress and encode the data block to be encoded by the encoder to generate a compressed code stream, and the processor is used to perform decompression processing on the compressed code stream by the decoder.

Optionally, the aforementioned preset data is weight data based on a convolutional neural network; the configurator is used to allocate the weight data, and write the compressed code stream obtained by compressing and encoding the weight data into the memory; The processor is configured to read the compressed code stream corresponding to the weight data from the memory, decompress the compressed code stream to generate the weight data, and run the convolutional neural network based on the weight data The internet.

In this embodiment, the configurator, memory, and processor constitute the system architecture of the convolutional neural network. By adding an encoder and a decoder to the system architecture, the problem of the processor repeatedly reading weight data from the memory to the processor is solved. Bandwidth pressure.

Optionally, the aforementioned processor 82 further includes a static random access memory, and the static random access memory is used to read the compressed code stream corresponding to the weight data from the memory, so that the processor can The compressed code stream is decompressed to generate the weight data, and the convolutional neural network is run based on the weight data.

In the foregoing implementation, the foregoing memory may be a double data rate synchronous dynamic random access memory (Double Data Rate Synchronous Dynamic Random Access Memory, DDR SDRAM); in this embodiment, an encoder is provided in the configurator, and the configurator is specifically The encoder is used to compress and encode the data block to be encoded to generate a code stream; the processor is provided with a decoder and a static random access memory, and the static random access memory is used to read the compression code corresponding to the preset data from the memory Stream, the processor decompresses the compressed code stream in the SRAM through the decoder to generate the original preset data. Optionally, the aforementioned data processing system is a calculation system of a convolutional neural network, the aforementioned preset data is weight data of a convolutional neural network, and the aforementioned processor is used to run the convolutional neural network based on the weight data.

In the above-mentioned embodiment of the present invention, in order to reduce the bandwidth overhead when the weight data is read from the memory to the static random access memory in the processor, the encoder decompresses the compressed weight data; accordingly, the configurator will Before the weight data is written into the memory, the encoder will compress the weight data. This process uses encoders and decoders to losslessly compress and decompress the weight data, ensuring that the weight data will not introduce additional errors and do not need to retrain the neural network, while reducing the system bandwidth to a large extent. It reduces the hardware resource requirements for the operation of the convolutional neural network.

Optionally, the above-mentioned configurator 81 is specifically configured to compress and encode the to-be-coded data block formed by the preset data in the following manner to generate a compressed code stream:

The configurator obtains the coding overhead of the plurality of preset first coding modes for the data block to be coded, and determines the target coding mode with the smallest coding overhead from the plurality of first coding modes;

Encoding the data block to be encoded by the target encoding method to obtain a compressed code stream.

Wherein, the foregoing encoding overhead is the bit width occupied by the code stream data generated after encoding the data block to be encoded.

Optionally, the above-mentioned configurator 81 is further configured to: determine whether there is a redundant data block in the original data block, and if so, encode the redundant data block according to the second coding mode;

Use the remaining data blocks in the original data blocks except for the redundant data blocks as the data blocks to be encoded.

Optionally, the above-mentioned configurator 81 is further configured to generate the data block to be encoded in the following manner:

Determine whether there is a redundant data block in the original data block, and if so, obtain the remaining data block in the original data block excluding the redundant data block;

Divide the remaining data blocks according to a specified size to obtain multiple data blocks, and use each data block as the data block to be encoded.

Optionally, the above-mentioned configurator 81 is specifically configured to determine whether there is a redundant data block in the original data block in the following manner:

Find out whether there is data that continuously appears in the original data block and the number of repetitions is greater than a preset value, if so, determine that there is a redundant data block, and use the continuous data as the redundant data block; if not, then Make sure that there are no redundant data blocks.

Optionally, the foregoing second encoding manner includes run-length encoding, and the configurator 81 is specifically configured to encode the redundant data block according to the second encoding manner in the following manner:

Optionally, the above-mentioned multiple first coding modes include: an original coding mode, a residual coding mode, and a mixed coding mode; wherein, the mixed coding mode is after the data block to be coded is divided into several data sub-blocks , Perform residual coding or original coding on each of the data blocks.

Optionally, when the target coding mode is the original coding mode, the configurator 81 is specifically configured to:

The original data in the data block to be coded is written into the coded stream data according to the code stream structure corresponding to the original coding mode; the code stream structure corresponding to the original coding mode includes: the flag bit of the coding mode and Raw data.

Optionally, when the target coding mode is the residual coding mode, the configurator 81 is specifically configured to:

Using the data at a preset position in the data block to be encoded as reference data, the residual values of each two adjacent data located on both sides of the reference data are calculated respectively, and the residual value and the reference data are calculated according to The code stream structure corresponding to the residual coding mode is written into the coded code stream data; wherein the code stream structure corresponding to the residual coding mode includes: a flag bit of the coding mode, the reference data, and the residual The bit width occupied by the value and the residual value.

Optionally, when the target coding mode is a hybrid coding mode, the configurator 81 is specifically configured to:

Dividing the data block to be encoded to obtain at least two data sub-blocks;

The residual value calculated for each data sub-block and the reference data are written into the coded stream data according to the code stream structure corresponding to the hybrid encoding mode; wherein the code corresponding to the hybrid encoding mode The stream structure includes: a flag bit of an encoding mode, the reference data, the bit width occupied by the residual value, and the residual value.

Optionally, when the target coding mode is a hybrid coding mode, the configurator is specifically configured to:

Wherein, the code stream structure corresponding to the hybrid coding mode includes: the flag bit of the coding mode, the original data of the second data block, the residual value, the reference data, and the residual value Bit width and the residual value.

Optionally, the above-mentioned configurator 81 is further configured to: before the dividing the remaining data blocks according to a specified size to obtain multiple data blocks, if the data amount of the remaining data blocks does not satisfy an integer of the specified size Times, the remaining data blocks are filled with data, so that the size of each data block obtained after division is the specified size.

Optionally, the performing data filling on the remaining data blocks includes:

Copy the last data in the N remaining data blocks, and fill the remaining data blocks with the copied N data so that the data amount of the remaining data blocks is an integer multiple of the specified size; wherein, Said N is greater than or equal to 1.

Optionally, the foregoing processor 82 is specifically configured to read the compressed code stream in the memory in the following manner, and perform decompression processing on the compressed code stream:

Optionally, the code stream structure of the compressed code stream includes a flag bit;

The processor is specifically configured to parse the compressed code stream in the following manner, and determine a target encoding method corresponding to the compressed code stream among a plurality of preset first encoding methods:

According to the preset mapping relationship between the value of the flag bit and the first coding mode, the target coding mode corresponding to the compressed code stream is obtained.

Optionally, the target decoding mode includes any one of run-length decoding, original data copy, residual decoding, and hybrid decoding;

Optionally, when the target decoding mode is run-length decoding, the processor is specifically configured to:

Optionally, when the target decoding mode is original data copy, the processor is specifically configured to:

Optionally, when the target decoding mode is residual decoding, the processor is specifically configured to:

Optionally, the processor is specifically configured to decode and restore the compressed code stream by using the target decoding mode in the following manner to obtain the preset data, including:

Referring to the embodiment shown in FIG. 16, a movable platform is provided in this embodiment. The movable platform includes: a body 900, a power system 91, and the data processing system 92 described in the foregoing embodiment; the power system 91 is located at The body 900 and the power system 91 are used to provide power for the movable platform.

Exemplarily, the aforementioned data processing system 92 may be applied to a functional system of a movable platform such as a power system, an operating system, and the like.

When the data processing system 92 provided in this embodiment uses a convolutional neural network for data processing, it can achieve lossless compression and decompression of weight data, ensuring that the weight data will not introduce additional errors and do not need to retrain the neural network , And reduce system bandwidth overhead. It can meet the use of hardware resources with lower conditions to realize the operation of convolutional neural networks.

Referring to the embodiment shown in FIG. 17, an encoder 1000 is provided in this embodiment, including: a selection circuit 101 and an encoding circuit 102;

The encoder provided in the embodiment of the present invention can effectively compress data and has the advantage of high compression efficiency.

The foregoing encoding overhead is the bit width occupied by the code stream data generated after the data block to be encoded is encoded.

Optionally, the above selection circuit is also used for:

Determine whether there is a redundant data block in the original data block, and if so, encode the redundant data block according to the second encoding mode;

Optionally, the data block to be encoded is generated in the following manner:

Optionally, the foregoing determining whether there is a redundant data block in the original data block includes:

Find out whether there is data that continuously appears in the original data block and the number of repetitions is greater than a preset value, if so, determine that there is a redundant data block, and use the continuous data as the redundant data block; Make sure that there are no redundant data blocks.

Optionally, the second encoding manner includes run-length encoding, and encoding the redundant data block according to the second encoding manner includes:

Optionally, the multiple first coding modes include: an original coding mode, a residual coding mode, and a mixed coding mode; wherein the mixed coding mode is dividing the data block to be coded into several data sub-blocks Afterwards, residual coding or original coding is performed on each of the data blocks.

Optionally, when the target coding mode is the original coding mode, the encoding the data block to be coded by the target coding mode includes:

Writing the residual value and the reference data into the coded stream data according to the code stream structure corresponding to the residual coding mode;

Dividing the data block to be encoded to obtain at least two data sub-blocks;

Respectively writing the residual value calculated for each of the data sub-blocks and the reference data into the coded code stream data according to the code stream structure corresponding to the hybrid coding mode;

Wherein, the code stream structure corresponding to the hybrid coding mode includes: a flag bit of the coding mode, the reference data, the bit width occupied by the residual value, and the residual value.

Using the data at a preset position in the first data sub-block as reference data, the residual values of each two adjacent data located on both sides of the reference data are respectively calculated, and the residual value and the reference data Write the coded code stream data according to the code stream structure corresponding to the hybrid coding mode;

Writing the original data in the second data sub-block into the coded stream data according to the code stream structure corresponding to the hybrid coding mode;

Wherein, the code stream structure corresponding to the hybrid coding mode includes: the flag bit of the coding mode, the original data of the second data block, the reference data, the bit width occupied by the residual value, and the residual value. Difference.

Optionally, before dividing the remaining data blocks according to a specified size to obtain multiple data blocks, the method includes:

If the data amount of the remaining data block does not satisfy an integer multiple of the specified size, data padding is performed on the remaining data block so that the size of each data block obtained after division is the specified size.

Optionally, the performing data filling on the remaining data blocks includes:

Copy the last data in the N remaining data blocks, and fill the remaining data blocks with the copied N data so that the data amount of the remaining data blocks is an integer multiple of the specified size; wherein, Said N is greater than or equal to 1

Another embodiment of the present invention also provides an encoder, which includes a residual calculation circuit and an encoding circuit.

Wherein, the residual calculation circuit is configured to use the data in the preset position in the data block to be encoded as reference data, and respectively calculate the residual value of every two adjacent data located on both sides of the reference data;

The data block to be encoded is generated in the following manner:

The original data block is divided according to a specified size to obtain multiple data sub-blocks, and each of the data sub-blocks is used as the data block to be encoded.

Optionally, the data block to be encoded is generated in the following manner:

The original data block is divided according to a specified size to obtain multiple data sub-blocks, and the first data block of the multiple data blocks is used as the data block to be encoded.

Optionally, the encoding circuit is also used for:

The other data blocks of the multiple data sub-blocks except the first data block are written into the code stream data according to the code stream structure corresponding to the original coding mode; the code stream structure corresponding to the original coding mode includes : Flag bit and original data of encoding method.

Figure 18 also shows a schematic diagram of the encoder architecture. The encoder architecture mainly includes four functional modules: run-length encoding module, encoding overhead calculation module, mode selection module (used to determine the target encoding method with the smallest encoding overhead) and The remaining data block encoding module (used to encode the remaining data blocks except for the redundant data block); it should be noted that this embodiment is only an example for illustration and should not be regarded as a limitation of the present invention.

Referring to the embodiment shown in FIG. 19, this embodiment provides a decoder 1100, and the decoder 1100 includes: an analysis circuit 111 and a decoding circuit 112;

The parsing circuit 111 is configured to analyze the data to be decoded, determine a target encoding method corresponding to the data to be decoded among a plurality of preset first encoding methods, and determine a corresponding target decoding method according to the target encoding method;

The decoding circuit 112 is configured to decode the data to be decoded in the target decoding mode to generate decoded data.

The data to be decoded includes a flag bit; the analyzing the data to be decoded and determining the target encoding method corresponding to the data to be decoded among the plurality of preset first encoding methods includes:

According to the preset mapping relationship between the value of the flag bit and the first coding mode, the target coding mode corresponding to the data to be decoded is determined.

Optionally, the first coding mode includes one or more of run-length coding, original data coding, residual coding, and hybrid coding;

Wherein, the hybrid coding is to divide the data block to be coded to generate multiple data sub-blocks, and perform residual coding or original data coding on each of the data blocks.

Optionally, the residual coding is using data at a preset position in the data block to be coded as reference data, respectively calculating the residual values of every two adjacent data on both sides of the reference data, and calculating The residual value and the reference data are written into the coded stream data according to the code stream structure corresponding to the residual coding mode.

Optionally, the target decoding mode includes one or more of run-length decoding, original data copy, residual decoding, and hybrid decoding;

Optionally, the residual decoding is to read the corresponding reference data, the bit length occupied by the residual value, and the residual value from the code stream structure of the data to be decoded, according to the reference data and the residual value The occupied bit length and residual value perform the inverse operation of residual calculation to generate decoded data.

Optionally, when the target decoding mode is run-length decoding, the using the target decoding mode to decode the data to be decoded includes:

Optionally, when the target decoding mode is a copy of the original data, the using the target decoding mode to decode the data to be decoded includes:

Optionally, when the target decoding mode is residual decoding, the using the target decoding mode to decode the data to be decoded includes:

Read the corresponding reference data, the bit length occupied by the residual value and the residual value from the code stream structure of the data to be decoded, and perform the inverse operation of the residual calculation according to the reference data and the residual value to obtain the original data.

Optionally, the decoding multiple pieces of to-be-decoded data to generate decoded data using the target decoding manner includes:

If the data amount of the data obtained by decoding the data to be decoded by the target decoding method is greater than the preset data amount, discarding part of the data exceeding the preset data amount to generate the decoded data.

Figure 20 also shows a schematic diagram of the architecture of a decoder. The decoder mainly includes the following functional modules: a pattern analysis module (used to determine the target decoding method), a run-length decoding module, a residual decoding module, and an original data decoding module. It should be noted that this embodiment is only an example and should not be regarded as a limitation of the present invention.

Another embodiment of the present invention also provides a decoder, which includes: a parsing circuit and a decoding circuit.

Wherein, the parsing circuit is used to analyze the data block to be decoded to obtain reference data and residual data in the data block to be decoded; wherein the data block to be decoded is generated by encoding the original data block; and the reference data is all The data at a preset position in the original data; the residual data is the residual value of every two adjacent data located on both sides of the reference data during the encoding process. The decoding circuit is used to decrement or accumulate the reference data and the residual data to generate original data on both sides of the reference data

For the device embodiment, since it basically corresponds to the method embodiment, the relevant part can refer to the part of the description of the method embodiment. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in One place, or it can be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the objectives of the solutions of the embodiments. Those of ordinary skill in the art can understand and implement it without creative work.

It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply one of these entities or operations. There is any such actual relationship or order between. The terms "include", "include", or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements that are not explicitly listed. Elements, or also include elements inherent to such processes, methods, articles, or equipment. If there are no more restrictions, the element defined by the sentence "including a..." does not exclude the existence of other same elements in the process, method, article, or equipment including the element.

The methods and devices provided by the embodiments of the present invention are described in detail above. Specific examples are used in this article to illustrate the principles and implementations of the present invention. The descriptions of the above embodiments are only used to help understand the methods and methods of the present invention. Core idea; At the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation and the scope of application. In summary, the content of this specification should not be construed as a limitation of the present invention .

Claims

A data processing method, characterized in that it comprises:

Acquiring the coding overhead of the multiple first coding modes for the data block to be coded, and determining the target coding mode with the smallest coding overhead from the multiple first coding modes;

Encoding the data block to be encoded by the target encoding method.
The method according to claim 1, wherein the encoding overhead is the bit width occupied by the code stream data generated after the data block to be encoded is encoded.
The method according to claim 1, wherein the method further comprises:

Determine whether there is a redundant data block in the original data block, and if so, encode the redundant data block according to the second encoding mode;

Use the remaining data blocks in the original data blocks except for the redundant data blocks as the data blocks to be encoded.
The method according to claim 1, wherein the data block to be encoded is generated in the following manner:

Determine whether there is a redundant data block in the original data block, and if so, obtain the remaining data block in the original data block excluding the redundant data block;

Divide the remaining data blocks according to a specified size to obtain multiple data blocks, and use each data block as the data block to be encoded.
The method according to claim 3 or 4, wherein the determining whether there is a redundant data block in the original data block comprises:

Find out whether there is data that continuously appears in the original data block and the number of repetitions is greater than a preset value, if so, determine that there is a redundant data block, and use the continuous data as the redundant data block; Make sure that there are no redundant data blocks.
The method according to claim 3, wherein the second coding mode comprises run-length coding, and the coding of the redundant data block according to the second coding mode comprises:

The redundant data block is written into the coded stream data according to the code stream structure corresponding to the run-length encoding; the code stream structure corresponding to the run-length encoding includes: the flag bit of the encoding mode, the start value and the run length .
The method according to claim 1, wherein the multiple first coding modes include: an original coding mode, a residual coding mode, and a mixed coding mode; wherein, the mixed coding mode is for the to-be-coded After the data block is divided into several data sub-blocks, residual coding or original coding is performed on each data block.
8. The method according to claim 7, wherein when the target coding mode is an original coding mode, the encoding the data block to be coded by the target coding mode comprises:

Writing the original data in the data block to be coded into the coded code stream data according to the code stream structure corresponding to the original coding mode;

Wherein, the code stream structure corresponding to the original coding mode includes: a flag bit of the coding mode and original data.
8. The method according to claim 7, wherein when the target coding mode is the residual coding mode, the encoding the data block to be coded by the target coding mode comprises:

Using data at a preset position in the data block to be encoded as reference data, respectively calculating residual values of every two adjacent data located on both sides of the reference data;

Writing the residual value and the reference data into the coded stream data according to the code stream structure corresponding to the residual coding mode;

Wherein, the code stream structure corresponding to the residual coding mode includes: a flag bit of the coding mode, the reference data, the bit width occupied by the residual value, and the residual value.
8. The method according to claim 7, wherein when the target encoding method is a hybrid encoding method, the encoding the data block to be encoded by the target encoding method comprises:

Dividing the data block to be encoded to obtain at least two data sub-blocks;

Using data at a preset position in any of the data sub-blocks as reference data, respectively calculating residual values of every two adjacent data located on both sides of the reference data;

Respectively writing the residual value calculated for each of the data sub-blocks and the reference data into the coded code stream data according to the code stream structure corresponding to the hybrid coding mode;

Wherein, the code stream structure corresponding to the hybrid coding mode includes: a flag bit of the coding mode, the reference data, the bit width occupied by the residual value, and the residual value.
8. The method according to claim 7, wherein when the target encoding method is a hybrid encoding method, the encoding the data block to be encoded by the target encoding method comprises:

Dividing the data block to be encoded to obtain at least two data sub-blocks; wherein the at least two data word blocks include a first data sub-block and a second data sub-block;

Using the data at a preset position in the first data sub-block as reference data, the residual values of each two adjacent data located on both sides of the reference data are respectively calculated, and the residual value and the reference data Write the encoded code stream data according to the code stream structure corresponding to the hybrid encoding method; = write the original data in the second data sub-block into the code stream structure corresponding to the hybrid encoding method Stream data;

Wherein, the code stream structure corresponding to the hybrid coding mode includes: the flag bit of the coding mode, the original data of the second data block, the reference data, the bit width occupied by the residual value, and the residual value. Difference.
The method according to claim 4, wherein before the dividing the remaining data blocks according to a specified size to obtain multiple data blocks, the method comprises:

If the data amount of the remaining data block does not satisfy an integer multiple of the specified size, data padding is performed on the remaining data block so that the size of each data block obtained after division is the specified size.
The method according to claim 12, wherein the performing data filling on the remaining data block comprises:

Copy the last data in the N remaining data blocks, and fill the remaining data blocks with the copied N data so that the data amount of the remaining data blocks is an integer multiple of the specified size; wherein, Said N is greater than or equal to 1.
A data processing method, characterized by comprising:

Parsing the data to be decoded, determining a target encoding method corresponding to the data to be decoded among a plurality of preset first encoding methods, and determining a corresponding target decoding method according to the target encoding method;

The data to be decoded is decoded by the target decoding mode to generate decoded data.
The method according to claim 14, wherein the data to be decoded includes a flag bit; the data to be decoded is parsed to determine a target corresponding to the data to be decoded in a plurality of preset first encoding methods Encoding methods include:

According to the preset mapping relationship between the value of the flag bit and the first coding mode, the target coding mode corresponding to the data to be decoded is determined.
The method according to claim 14, wherein the first coding mode comprises one or more of run-length coding, original data coding, residual coding and hybrid coding;

Wherein, the hybrid coding is to divide the data block to be coded to generate multiple data sub-blocks, and perform residual coding or original data coding on each of the data blocks.
The method according to claim 16, wherein the residual coding is to use data at a preset position in the data block to be coded as reference data, and respectively calculate every two adjacent ones located on both sides of the reference data. The residual value of each data, the residual value and the reference data are written into the coded code stream data according to the code stream structure corresponding to the residual coding mode.
The method according to claim 16, wherein the target decoding mode comprises one or more of run-length decoding, original data copy, residual decoding and hybrid decoding;

Wherein, the hybrid decoding is to divide the data block to be decoded to generate multiple data sub-blocks, and perform residual decoding or original data copy on each of the data word blocks.
The method according to claim 18, wherein the residual decoding comprises reading the corresponding reference data, the bit length occupied by the residual value and the residual value from the code stream structure of the data to be decoded, Perform the inverse operation of the residual calculation according to the reference data, the bit length occupied by the residual value and the residual value to generate decoded data.
The method according to claim 18, wherein when the target decoding mode is run-length decoding, the using the target decoding mode to decode the data to be decoded comprises:

The start value and the run length are read from the code stream structure of the data to be decoded, and the data to be decoded is decoded and restored according to the start value and the run length to obtain decoded data.
The method according to claim 18, wherein when the target decoding mode is a copy of the original data, the decoding the data to be decoded using the target decoding mode comprises:

Reading the corresponding encoded data from the code stream structure of the data to be decoded, and copying the encoded data to obtain decoded data.
The method according to claim 19, wherein when the target decoding mode is residual decoding, the decoding the data to be decoded using the target decoding mode comprises:

Read the corresponding reference data, the bit length occupied by the residual value and the residual value from the code stream structure of the data to be decoded, and perform the inverse operation of the residual calculation according to the reference data and the residual value to obtain the original data.
The method according to claim 14, wherein said using said target decoding mode to decode a plurality of said data to be decoded to generate decoded data comprises:

If the data amount of the data obtained by decoding the data to be decoded by the target decoding method is greater than the preset data amount, discarding part of the data exceeding the preset data amount to generate the decoded data.
A data processing method, characterized in that it comprises:

Using data at a preset position in the data block to be encoded as reference data, respectively calculating residual values of every two adjacent data located on both sides of the reference data;

The residual value and the reference data are written into the coded stream data.
The method according to claim 24, wherein the data block to be encoded is generated in the following manner:

The original data block is divided according to a specified size to obtain multiple data sub-blocks, and each of the data sub-blocks is used as the data block to be encoded.
The method according to claim 24, wherein the data block to be encoded is generated in the following manner:

The original data block is divided according to a specified size to obtain multiple data sub-blocks, and the first data block of the multiple data blocks is used as the data block to be encoded.
The method according to claim 26, further comprising:

The other data blocks of the multiple data sub-blocks except the first data block are written into the code stream data according to the code stream structure corresponding to the original coding mode; the code stream structure corresponding to the original coding mode includes : Flag bit and original data of encoding method.
A data processing method, characterized by comprising:

Analyze the data block to be decoded, and obtain the reference data and residual data in the data block to be decoded;

Perform a decrement operation or an accumulation operation on the reference data and the residual data, perform a decoding operation on the data block to be decoded, and generate original data on both sides of the reference data.
A data processing method, characterized in that it is applied to a configurator, a memory and a processor connected in sequence, and includes:

Performing compression encoding on the data block to be encoded formed by the preset data by the configurator to generate a compressed code stream, and writing the compressed code stream into the memory, so as to reduce the write bandwidth to the memory;

The processor reads the compressed code stream in the memory and performs decompression processing on the compressed code stream to reduce the read bandwidth of the memory.
The method according to claim 29, wherein an encoder is provided on the configurator, and a decoder is provided on the processor;

The configurator compresses and encodes the data block to be encoded by the encoder to generate a compressed code stream, and the processor performs decompression processing on the compressed code stream through the decoder.
The method according to claim 29, wherein the preset data is weight data based on a convolutional neural network; the configurator is used to allocate the weight data, and compress and encode the weight data The obtained compressed code stream is written into the memory; the processor is used to read the compressed code stream corresponding to the weight data from the memory, decompress the compressed code stream to generate the weight data, and generate the weight data based on The weight data runs the convolutional neural network.
The method according to claim 31, wherein the processor further comprises a static random access memory, and the static random access memory is used to read the compression code corresponding to the weight data from the memory Flow, so that the processor can decompress the compressed code stream to generate the weight data, and run the convolutional neural network based on the weight data.
The method according to claim 29, wherein the configurator compresses and encodes the to-be-encoded data block composed of preset data to generate a compressed code stream, comprising:

The configurator obtains the coding overhead of the plurality of first coding modes for the data block to be coded, and determines a target coding mode with the smallest coding overhead from the plurality of first coding modes;

Encoding the data block to be encoded by the target encoding method to obtain a compressed code stream.
The method according to claim 33, wherein the encoding overhead is the bit width occupied by the code stream data generated after the data block to be encoded is encoded.
The method of claim 33, wherein the method further comprises:

The configurator determines whether there is a redundant data block in the original data block, and if so, encodes the redundant data block according to the second encoding mode;

Use the remaining data blocks in the original data blocks except for the redundant data blocks as the data blocks to be encoded.
The method according to claim 33, wherein the data block to be encoded is generated in the following manner:

The configurator determines whether there is a redundant data block in the original data block, and if so, obtains the remaining data block in the original data block excluding the redundant data block;

Divide the remaining data blocks according to a specified size to obtain multiple data blocks, and use each data block as the data block to be encoded.
The method according to claim 35 or 36, wherein the configurator determining whether there is a redundant data block in the original data block comprises:

The configurator searches the original data block whether there is data that continuously appears and the number of repetitions is greater than a preset value, if so, determines that there is a redundant data block, and uses the continuous data as the redundant data block; If not, it is determined that there is no redundant data block.
The method according to claim 35, wherein the second coding mode comprises run-length coding, and the coding of the redundant data block according to the second coding mode comprises:

The redundant data block is written into the coded stream data according to the code stream structure corresponding to the run-length encoding; the code stream structure corresponding to the run-length encoding includes: the flag bit of the encoding mode, the start value and the run length .
The method according to claim 33, wherein the multiple first coding modes include: an original coding mode, a residual coding mode, and a mixed coding mode; wherein, the mixed coding mode is for the to-be-coded After the data block is divided into several data sub-blocks, residual coding or original coding is performed on each data block.
The method according to claim 39, wherein when the target coding mode is an original coding mode, the encoding the data block to be coded by the target coding mode comprises:

Writing the original data in the data block to be coded into the coded code stream data according to the code stream structure corresponding to the original coding mode;

Wherein, the code stream structure corresponding to the original coding mode includes: a flag bit of the coding mode and original data.
The method according to claim 39, wherein when the target coding mode is the residual coding mode, the encoding the data block to be coded by the target coding mode comprises:

Using the data at a preset position in the data block to be encoded as reference data, the residual values of each two adjacent data located on both sides of the reference data are calculated respectively, and the residual value and the reference data are calculated according to The code stream structure corresponding to the residual coding method is written into the coded code stream data;

Wherein, the code stream structure corresponding to the residual coding mode includes: a flag bit of the coding mode, the reference data, the bit width occupied by the residual value, and the residual value.
The method according to claim 39, wherein when the target coding mode is a hybrid coding mode, the encoding the data block to be coded by the target coding mode comprises:

Dividing the data block to be encoded to obtain at least two data sub-blocks;

Using data at a preset position in any of the data sub-blocks as reference data, respectively calculating residual values of every two adjacent data located on both sides of the reference data;

Respectively writing the residual value calculated for each of the data sub-blocks and the reference data into the coded code stream data according to the code stream structure corresponding to the hybrid coding mode;

Wherein, the code stream structure corresponding to the hybrid coding mode includes: a flag bit of the coding mode, the reference data, the bit width occupied by the residual value, and the residual value.
The method according to claim 39, wherein when the target coding mode is a hybrid coding mode, the encoding the data block to be coded by the target coding mode comprises:

Dividing the data block to be encoded to obtain at least two data sub-blocks; wherein the at least two data word blocks include a first data sub-block and a second data sub-block;

Using the data at a preset position in the first data sub-block as reference data, the residual values of each two adjacent data located on both sides of the reference data are respectively calculated, and the residual value and the reference data Write the encoded code stream data according to the code stream structure corresponding to the hybrid encoding method; write the original data in the second data sub-block into the code stream structure corresponding to the hybrid encoding method Stream data;

Wherein, the code stream structure corresponding to the hybrid coding mode includes: the flag bit of the coding mode, the original data of the second data block, the residual value, the reference data, and the residual value Bit width and the residual value.
The method according to claim 36, wherein before the dividing the remaining data blocks according to a specified size to obtain multiple data blocks, the method comprises:

If the data amount of the remaining data block does not satisfy an integer multiple of the specified size, data padding is performed on the remaining data block so that the size of each data block obtained after division is the specified size.
The method according to claim 44, wherein the performing data padding on the remaining data block comprises:

Copy the last data in the N remaining data blocks, and fill the remaining data blocks with the copied N data so that the data amount of the remaining data blocks is an integer multiple of the specified size; wherein, Said N is greater than or equal to 1.
The method according to claim 29, wherein the processor reads the compressed code stream in the memory, and performs decompression processing on the compressed code stream, comprising:

The processor parses the compressed code stream, determines a target encoding method corresponding to the compressed code stream among a plurality of preset first encoding methods, and determines a corresponding target decoding method according to the target encoding method;

The compressed code stream is decoded by the target decoding method to obtain the preset data.
The method of claim 46, wherein the bit stream structure of the compressed bit stream includes a flag bit;

The parsing the compressed code stream to determine the target encoding method corresponding to the compressed code stream among the plurality of preset first encoding methods includes:

According to the preset mapping relationship between the value of the flag bit and the first coding mode, the target coding mode corresponding to the compressed code stream is determined.
The method according to claim 46, wherein the target decoding mode comprises any one or more of run-length decoding, original data copy, residual decoding, and hybrid decoding;

Wherein, the hybrid decoding is to divide the data block to be decoded to generate multiple data sub-blocks, and perform residual decoding or original data copy on each of the data word blocks.
The method according to claim 48, wherein when the target decoding mode is run-length decoding, the using the target decoding mode to decode and restore the compressed code stream comprises:

The initial value and the run length are read from the code stream structure of the compressed code stream, and the compressed code stream is decoded and restored according to the initial value and the run length to obtain corresponding preset data.
The method according to claim 48, wherein when the target decoding mode is an original data copy, the using the target decoding mode to decode and restore the compressed code stream comprises:

Reading the corresponding coded data from the code stream structure of the compressed code stream, and copying the coded data to obtain corresponding preset data.
The method according to claim 48, wherein when the target decoding mode is residual decoding, the using the target decoding mode to decode and restore the compressed code stream comprises:

Read the corresponding reference data, the bit length occupied by the residual value and the residual value from the bit stream structure of the compressed bit stream, and execute according to the reference data, the bit length occupied by the residual value and the residual value The inverse operation of the residual calculation obtains the corresponding preset data.
The method according to claim 44, wherein the decoding and restoring the compressed code stream using the target decoding method to obtain the preset data comprises:

If the data volume of the data obtained by decoding and restoring the compressed code stream by the target decoding method is greater than the preset data volume, discarding part of the data exceeding the preset data volume to obtain the preset data.
A data processing system, characterized by comprising: a configurator, a memory, and a processor connected in sequence;

The configurator is used to compress and encode a data block to be encoded formed by preset data to generate a compressed code stream, and write the compressed code stream into the memory, so as to reduce the write bandwidth to the memory;

The processor is configured to read the compressed code stream in the memory, and perform decompression processing on the compressed code stream, so as to reduce the read bandwidth of the memory.
The system of claim 53, wherein the configurator is provided with an encoder, and the processor is provided with a decoder;

The configurator is used to compress and encode the data block to be encoded by the encoder to generate a compressed code stream, and the processor is used to perform decompression processing on the compressed code stream by the decoder.
The system according to claim 53, wherein the preset data is weight data based on a convolutional neural network; the configurator is used to allocate the weight data and compress and encode the weight data The obtained compressed code stream is written into the memory; the processor is used to read the compressed code stream corresponding to the weight data from the memory, perform decompression processing on the compressed code stream to generate the weight data, and based on The weight data runs the convolutional neural network.
The system according to claim 55, wherein the processor further comprises a static random access memory, and the static random access memory is used to read the compressed code stream corresponding to the weight data from the memory , So that the processor can decompress the compressed code stream to generate the weight data, and run the convolutional neural network based on the weight data.
The system according to claim 53, wherein the configurator is specifically configured to compress and encode the to-be-encoded data block composed of preset data to generate a compressed code stream in the following manner:

The configurator obtains the coding overhead of the plurality of preset first coding modes for the data block to be coded, and determines the target coding mode with the smallest coding overhead from the plurality of first coding modes;

Encoding the data block to be encoded by the target encoding method to obtain a compressed code stream.
The system according to claim 57, wherein the encoding overhead is a bit width occupied by code stream data generated after encoding the data block to be encoded.
The system according to claim 57, wherein the configurator is further configured to determine whether there is a redundant data block in the original data block, and if so, to encode the redundant data block according to the second encoding method ；

Use the remaining data blocks in the original data blocks except for the redundant data blocks as the data blocks to be encoded.
The system according to claim 57, wherein the configurator is further configured to generate the data block to be encoded in the following manner:

Determine whether there is a redundant data block in the original data block, and if so, obtain the remaining data block in the original data block excluding the redundant data block;

Divide the remaining data blocks according to a specified size to obtain multiple data blocks, and use each data block as the data block to be encoded.
The system according to claim 59 or 60, wherein the configurator is specifically configured to determine whether there is a redundant data block in the original data block in the following manner:

Find out whether there is data that continuously appears in the original data block and the number of repetitions is greater than a preset value, if so, determine that there is a redundant data block, and use the continuous data as the redundant data block; Make sure that there are no redundant data blocks.
The system according to claim 59, wherein the second encoding method comprises run-length encoding, and the configurator is specifically configured to encode the redundant data block according to the second encoding method in the following manner:

The redundant data block is written into the coded stream data according to the code stream structure corresponding to the run-length encoding; the code stream structure corresponding to the run-length encoding includes: the flag bit of the encoding mode, the start value and the run length .
The system according to claim 57, wherein the plurality of first coding modes include: an original coding mode, a residual coding mode, and a mixed coding mode; wherein, the mixed coding mode is for the to-be-coded After the data block is divided into several data sub-blocks, residual coding or original coding is performed on each data block.
The system according to claim 63, wherein when the target encoding mode is the original encoding mode, the configurator is specifically configured to:

The original data in the data block to be coded is written into the coded stream data according to the code stream structure corresponding to the original coding mode; the code stream structure corresponding to the original coding mode includes: the flag bit of the coding mode and Raw data.
The system according to claim 63, wherein when the target coding mode is the residual coding mode, the configurator is specifically configured to:

Using the data at a preset position in the data block to be encoded as reference data, the residual values of each two adjacent data located on both sides of the reference data are calculated respectively, and the residual value and the reference data are calculated according to The code stream structure corresponding to the residual coding mode is written into the coded code stream data; wherein the code stream structure corresponding to the residual coding mode includes: a flag bit of the coding mode, the reference data, and the residual The bit width occupied by the value and the residual value.
The system according to claim 63, wherein when the target coding mode is a hybrid coding mode, the configurator is specifically configured to:

Dividing the data block to be encoded to obtain at least two data sub-blocks;

Using data at a preset position in any of the data sub-blocks as reference data, respectively calculating residual values of every two adjacent data located on both sides of the reference data;

The residual value calculated for each data sub-block and the reference data are written into the coded stream data according to the code stream structure corresponding to the hybrid encoding mode; wherein the code corresponding to the hybrid encoding mode The stream structure includes: a flag bit of an encoding mode, the reference data, the bit width occupied by the residual value, and the residual value.
The system according to claim 63, wherein when the target coding mode is a hybrid coding mode, the configurator is specifically configured to:

Dividing the data block to be encoded to obtain at least two data sub-blocks; wherein the at least two data word blocks include a first data sub-block and a second data sub-block;

Using the data at a preset position in the first data sub-block as reference data, the residual values of each two adjacent data located on both sides of the reference data are respectively calculated, and the residual value and the reference data Write the encoded code stream data according to the code stream structure corresponding to the hybrid encoding method; write the original data in the second data sub-block into the code stream structure corresponding to the hybrid encoding method Stream data;

Wherein, the code stream structure corresponding to the hybrid coding mode includes: the flag bit of the coding mode, the original data of the second data block, the residual value, the reference data, and the residual value Bit width and the residual value.
The system according to claim 60, wherein the configurator is further configured to: before dividing the remaining data block according to a specified size to obtain multiple data blocks, if the data amount of the remaining data block is If it is not an integer multiple of the specified size, data padding is performed on the remaining data blocks, so that the size of each data block obtained after division is the specified size.
The system according to claim 68, wherein said performing data filling on said remaining data block comprises:

Copy the last data in the N remaining data blocks, and fill the remaining data blocks with the copied N data so that the data amount of the remaining data blocks is an integer multiple of the specified size; wherein, Said N is greater than or equal to 1.
The system according to claim 53, wherein the processor is specifically configured to read the compressed code stream in the memory in the following manner, and perform decompression processing on the compressed code stream:

The processor parses the compressed code stream, determines a target encoding method corresponding to the compressed code stream among a plurality of preset first encoding methods, and determines a corresponding target decoding method according to the target encoding method;

The compressed code stream is decoded by the target decoding method to obtain the preset data.
The system according to claim 70, wherein the bit stream structure of the compressed bit stream includes a flag bit;

The processor is specifically configured to parse the compressed code stream in the following manner, and determine a target encoding method corresponding to the compressed code stream among a plurality of preset first encoding methods:

According to the preset mapping relationship between the value of the flag bit and the first coding mode, the target coding mode corresponding to the compressed code stream is obtained.
The system according to claim 70, wherein the target decoding mode comprises any one of run-length decoding, original data copy, residual decoding and hybrid decoding;

Wherein, the hybrid decoding is to divide the data block to be decoded to generate multiple data sub-blocks, and perform residual decoding or original data copy on each of the data word blocks.
The system according to claim 72, wherein when the target decoding mode is run-length decoding, the processor is specifically configured to:

The initial value and the run length are read from the code stream structure of the compressed code stream, and the compressed code stream is decoded and restored according to the initial value and the run length to obtain corresponding preset data.
The system according to claim 72, wherein, when the target decoding mode is original data copy, the processor is specifically configured to:

Read the corresponding coded data from the code stream structure of the compressed code stream, and copy the coded data to obtain the corresponding preset data.
The system according to claim 72, wherein when the target decoding mode is residual decoding, the processor is specifically configured to:

Read the corresponding reference data, the bit length occupied by the residual value and the residual value from the bit stream structure of the compressed bit stream, and execute according to the reference data, the bit length occupied by the residual value and the residual value The inverse operation of the residual calculation obtains the corresponding preset data.
The system according to claim 72, wherein the processor is specifically configured to decode and restore the compressed code stream using the target decoding mode in the following manner to obtain the preset data, comprising:

If the data volume of the data obtained by decoding and restoring the compressed code stream by the target decoding method is greater than the preset data volume, discarding part of the data exceeding the preset data volume to obtain the preset data.
A movable platform, characterized in that, the movable platform includes:

Body

The power system is provided in the body, and the power system is used to provide power to the movable platform; and,

A data processing system according to any one of claims 53-76.
An encoder, which is characterized by comprising: a selection circuit and an encoding circuit;

The selection circuit is configured to obtain the coding overhead of the data block to be coded by the multiple first coding modes, and determine the target coding mode with the smallest coding overhead from the multiple first coding modes;

The encoding circuit encodes the data block to be encoded by using the target encoding method.
An encoder, characterized in that the encoder includes: a residual calculation circuit and an encoding circuit;

The residual calculation circuit is configured to use the data at a preset position in the data block to be encoded as reference data, and respectively calculate the residual value of every two adjacent data located on both sides of the reference data;

The coding circuit is configured to write the residual value and the reference data into the code stream data according to the code stream structure corresponding to the residual coding mode; wherein, the code stream structure corresponding to the residual coding mode Including: the reference data, the bit width occupied by the residual value, and the residual value.
A decoder, characterized in that the decoder includes: an analysis circuit and a decoding circuit;

The parsing circuit is configured to analyze the data to be decoded, determine a target encoding method corresponding to the data to be decoded among a plurality of preset first encoding methods, and determine a corresponding target decoding method according to the target encoding method;

The decoding circuit is used to decode the data to be decoded by the target decoding mode to generate decoded data.
A decoder, characterized in that the decoder includes: an analysis circuit and a decoding circuit;

The parsing circuit is used to analyze the data block to be decoded to obtain the reference data and residual data in the data block to be decoded; perform a decrement operation or an accumulation operation on the reference data and the residual data to generate the reference data Raw data on both sides.