CN116684646A

CN116684646A - Entropy-free image compression coding method, device, equipment and storage medium

Info

Publication number: CN116684646A
Application number: CN202310699130.8A
Authority: CN
Inventors: 宫久路; 谌德荣
Original assignee: Beijing Institute of Technology BIT
Current assignee: Beijing Institute of Technology BIT
Priority date: 2023-06-13
Filing date: 2023-06-13
Publication date: 2023-09-01

Abstract

The application discloses an entropy-free image compression encoding method, an entropy-free image compression encoding device, electronic equipment and a storage medium. The entropy-free image compression coding method comprises the following steps: and carrying out sampling convolution on the original image to obtain sampled and convolved image data, carrying out quantization processing on the sampled and convolved image data to obtain image quantized data, and carrying out self-adaptive run-length coding on the image quantized data to obtain image compressed data. According to the entropy-free image compression coding method provided by the embodiment of the application, the image compression data is obtained through self-adaptive run-length coding, the compression ratio is improved, the image data compression effect is improved, the compression amount obtained through the encoder can meet the requirements of practical application, and the following conditions in the related technology are improved: the compression amount obtained by the encoder in the related art is not high enough, so that the compression amount is difficult to meet the requirement of practical application, and the compression ratio needs to be further improved.

Description

Entropy-free image compression coding method, device, equipment and storage medium

Technical Field

The present application relates to the field of image processing technologies, and in particular, to an entropy-free image compression encoding method, an entropy-free image compression encoding device, an electronic apparatus, and a storage medium.

Background

In the related art image compression method, an encoder mainly comprises two parts of transformation and entropy coding, the calculation complexity is high, and the steps are generally that the transformation is firstly used for obtaining the compression amount with low information entropy, and then the entropy coding is used for coding the compression amount into a binary code stream with even distribution. The compression amount obtained by the encoder in the related art is not high enough, so that the compression amount is difficult to meet the requirement of practical application, and the compression ratio needs to be further improved.

The statements made above merely serve to provide background information related to the present disclosure and may not necessarily constitute prior art.

Disclosure of Invention

An object of the present application is to provide an entropy-free image compression encoding method, an entropy-free image compression encoding apparatus, an electronic device, and a storage medium, to improve the following conditions existing in the related art: the compression amount obtained by the encoder in the related art is not high enough, so that the compression amount is difficult to meet the requirement of practical application, and the compression ratio needs to be further improved. The following presents a simplified summary in order to provide a basic understanding of some aspects of the disclosed embodiments. This summary is not an extensive overview and is intended to neither identify key/critical elements nor delineate the scope of such embodiments. Its sole purpose is to present some concepts in a simplified form as a prelude to the more detailed description that is presented later.

According to an aspect of an embodiment of the present application, there is provided an entropy-free encoding method for image compression, which is applied to an encoder, the entropy-free encoding method including:

carrying out sampling convolution on an original image to obtain image data after sampling convolution;

carrying out quantization processing on the sampled and convolved image data to obtain image quantized data;

and carrying out self-adaptive run-length coding on the image quantized data to obtain image compressed data.

In some embodiments of the present application, performing adaptive run-length encoding on the image quantized data to obtain image compressed data, including:

performing expansion re-representation processing on the image quantized data to obtain a data segment after expansion re-representation;

determining data segments meeting preset requirements and data segments not meeting the preset requirements in the data segments after the re-representation according to the image resolution;

and performing run length coding on the data segments meeting the preset requirements, and adding the image resolution into the run length coding data without changing the data segments not meeting the preset requirements to obtain the image compression data.

In some embodiments of the present application, performing an expansion re-representation process on the image quantized data to obtain an expanded re-represented data segment, including:

Expanding the image quantization data into a one-dimensional vector;

traversing the one-dimensional vector, and recording the continuous repetition times of continuous repeated characters in the one-dimensional vector;

and for characters which exist in the quantized image data and are continuously repeated for a plurality of times, the number of times of continuous repeated for a plurality of times is represented by a number, and a data segment after being unfolded and re-represented is obtained.

In some embodiments of the application, the encoder comprises three downsampled convolutional layers and a binary mapping layer in series in sequence; the step of performing sampling convolution on the original image to obtain sampled and convolved image data comprises the following steps:

the encoder carries out downsampling convolution processing on the original image sequentially through the three downsampling convolution layers to obtain downsampled convolution processed image data;

and performing binary mapping on the image data subjected to the downsampling convolution processing by utilizing a binary mapping layer to obtain the image data subjected to the sampling convolution.

In some embodiments of the present application, the quantizing processing is performed on the sampled and convolved image data to obtain image quantized data, including:

and mapping the sampled and convolved image data to binary code streams obeying uniform distribution through quantization to obtain image quantized data.

In some embodiments of the present application, the entropy-free image compression encoding method further includes: and sending the image compression data to a decoder so that the decoder receives the image compression data from the encoder, and performing run-length decoding on the image compression data to obtain a decoded reconstructed image.

According to another aspect of an embodiment of the present application, there is provided an image compression encoding method without entropy encoding, including:

the decoder receives the image compression data from the encoder, and performs run-length decoding on the image compression data to obtain a decoded reconstructed image;

the post-processing network model processes the reconstructed image to obtain a post-processing optimized image;

the post-processing network model comprises a first convolution layer, a downsampling convolution layer, a GRDB module, an upsampling convolution layer, a CBAM module and a second convolution layer which are sequentially connected.

In some embodiments of the present application, the GRDB module includes a first RDB module, a second RDB module, a third RDB module, a splicing module, a convolution module, and an addition module sequentially connected from front to back, where an output end of the first RDB module, an output end of the second RDB module, and an output end of the third RDB module are respectively connected to an input end of the splicing module; the input end of the first RDB module is connected with the input end of the addition module.

In some embodiments of the present application, any one of the first RDB module, the second RDB module, and the third RDB module includes a first convolution activation module, a second convolution activation module, a third convolution activation module, a splice module, a convolution module, and an addition module that are sequentially connected from front to back; the input end of the first convolution activation module is respectively connected with the input end of the second convolution activation module, the input end of the third convolution activation module and the input end of the addition module; the input end of the second convolution activation module is respectively connected with the input end of the third convolution activation module and the input end of the splicing module; and the input end of the third convolution activation module is connected with the input end of the splicing module.

According to another aspect of an embodiment of the present application, there is provided an image compression encoding apparatus without entropy encoding, including an encoder, a decoder, and a post-processing network model;

the encoder is used for carrying out sampling convolution on the original image to obtain sampled and convolved image data; carrying out quantization processing on the sampled and convolved image data to obtain image quantized data; performing self-adaptive run-length coding on the image quantized data to obtain image compressed data;

The decoder is used for receiving the image compression data from the encoder, performing run-length decoding on the image compression data, and obtaining a decoded reconstructed image;

the post-processing network model is used for processing the reconstructed image to obtain a post-processing optimized image; the post-processing network model comprises a first convolution layer, a downsampling convolution layer, a GRDB module, an upsampling convolution layer, a CBAM module and a second convolution layer which are sequentially connected.

According to another aspect of an embodiment of the present application, there is provided an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor executing the computer program to implement the entropy-free image compression encoding method according to any embodiment of the present application.

According to another aspect of an embodiment of the present application, there is provided a computer-readable storage medium having stored thereon a computer program that is executed by a processor to implement the entropy-free image compression encoding method according to any of the embodiments of the present application.

One of the technical solutions provided in one aspect of the embodiments of the present application may include the following beneficial effects:

According to the entropy-free image compression coding method, sampling convolution is conducted on an original image to obtain sampled convolved image data, quantization processing is conducted on the sampled convolved image data to obtain image quantized data, self-adaptive run-length coding is conducted on the image quantized data to obtain image compression data, the image compression data are obtained through the self-adaptive run-length coding, the compression ratio is improved, the image data compression effect is improved, the compression amount obtained through an encoder can meet the needs of practical application, and the following conditions in the related technology are improved: the compression amount obtained by the encoder in the related art is not high enough, so that the compression amount is difficult to meet the requirement of practical application, and the compression ratio needs to be further improved.

The foregoing description is only an overview of the technical solutions of the embodiments of the present application, and may be implemented according to the content of the specification, so that the technical means of the embodiments of the present application can be more clearly understood, and the following specific embodiments of the present application will be more specifically described below.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments described in the present application, and other drawings may be obtained according to the drawings without inventive effort to those skilled in the art.

Fig. 1 shows a flowchart of an image compression encoding method without entropy encoding according to an embodiment of the present application.

Fig. 2 shows a flow chart of step S10 in one embodiment of the application.

Fig. 3 shows a flow chart of step S30 in one embodiment of the application.

Fig. 4 shows a schematic view of a run-length encoding process of the compression amount in one embodiment of the present application.

Fig. 5 shows a schematic diagram of adaptive run length encoding in one embodiment of the application.

Fig. 6 is a schematic diagram of an image compression process after adaptive run-length encoding is added in an embodiment of the present application.

Figure 7 shows a schematic representation of a GRDN model in one embodiment of the application.

Figure 8 shows a schematic of a GRDB module in one embodiment of the application.

FIG. 9 shows a schematic diagram of an RDB module in one embodiment of the application.

FIG. 10 shows a schematic of a CBAM module in one embodiment of the application.

FIG. 11 illustrates a channel attention schematic in a CBAM module in one embodiment of the application.

Fig. 12 shows a schematic diagram of spatial attention in a CBAM module in an embodiment of the present application.

Fig. 13 is a block diagram showing the structure of an image compression encoding apparatus without entropy encoding according to an embodiment of the present application.

Fig. 14 shows a block diagram of an electronic device according to an embodiment of the application.

FIG. 15 shows a schematic diagram of a computer-readable storage medium of one embodiment of the application.

Detailed Description

The present application will be further described with reference to the drawings and the specific embodiments in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

It will be understood by those skilled in the art that all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs unless defined otherwise. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

In the image compression method in the related art, an encoder mainly comprises two parts of transformation and entropy coding, and the compression amount obtained by the encoder in the related art is not high enough, so that the compression amount is difficult to meet the requirement of practical application, and the compression ratio needs to be further improved. In addition, in the related art, since entropy encoding is a relatively time-consuming loop in an encoder, entropy encoding results in long time-consuming image compression, and thus the related art image compression method is difficult to efficiently meet the needs of practical applications.

In view of the problems existing in the related art, referring to fig. 1, an embodiment of the present application provides an entropy-free image compression encoding method, which is applied to an encoder, and may include steps S10 to S30:

s10, carrying out sampling convolution on the original image to obtain image data after sampling convolution.

In some examples, the encoder may include three downsampled convolutional layers and a binary mapping layer in series in turn. Referring to fig. 2, the sample convolution is performed on the original image to obtain image data after the sample convolution, which may include steps S101 to S102:

s101, an encoder sequentially carries out downsampling convolution processing on an original image through three downsampling convolution layers to obtain downsampled convolution processed image data.

S102, performing binary mapping on the image data subjected to the downsampling convolution processing by utilizing a binary mapping layer to obtain the image data subjected to the sampling convolution.

S20, carrying out quantization processing on the sampled and convolved image data to obtain image quantized data.

In some embodiments, the quantization processing is performed on the sampled convolved image data to obtain image quantized data, including: and mapping the sampled and convolved image data to binary code streams obeying uniform distribution through quantization to obtain image quantized data.

In particular, since the quantization operation is not conductive, a conductive curve is used instead of a step function in training. Aiming at each value in the compression quantity, the exclusive quantization parameter corresponding to one is learned in training, so that the compression performance of the algorithm can be continuously improved.

In the test, binary quantization is used to obtain a binary code stream as shown in formula (1).

In training, the gradient of the formula (1) is discontinuous, so that the requirement of transmissibility of the gradient in counter propagation cannot be met. Therefore, the fitting function needs to be replaced by a conductive fitting function, and the previous approach is to use a Sigmoid function to fit, as shown in the formula (2), wherein η is a pre-set super-parameter.

Different values of eta bring different influences, and when eta is larger, the more the consumption formula (1.2) used for training is close to the consumption formula (1) used for testing, the smaller the output errors of training and testing are, so that accurate quantification is facilitated.

However, the larger η also makes the saturation region of the formula (2) larger, and the gradient is effective only in the region very close to the binary boundary point of 0.5, and the gradient is liable to disappear in other regions. So the value of η is not larger and better, but a compromise is needed to find the most suitable value. Experiments are carried out when eta takes different values, and finally, the reconstruction image quality is highest when eta is 64, and the reconstruction image quality is reduced when eta is smaller than 64 or larger than 64, which is the reason that eta takes 64 in some examples. However, taking into account the amount of compressionSetting the same η for all values in y, each value y in y, may not be optimal _i Should all correspond to a unique eta _i Thereby exerting the compression potential to the maximum extent, and therefore, is provided withGiven that its initial values are all 64, η is optimized in an end-to-end training as part of the model parameters.

In one specific example, an encoder processes an original image using a convolution transform to obtain a compression amount; the encoder maps the compression amount to a binary code stream subject to uniform distribution through quantization to obtain image compression data. The encoder transmits the image compression data to the decoder.

The input of the encoder is the original image to be compressed and the output is the compressed data. Based on the application background of the present embodiment, the design principle of the encoder is extremely compact, and only the most necessary parts are reserved. The convolutions that achieve downsampling are not possible to prune, and in one specific example, the encoder preferably employs a concatenation of only three downsampled convolutions of layer 0 of length 3. The first two downsampled convolution layers 0 in the encoder are followed by a RELU activation function 5, respectively, the downsampled convolution layers function to reduce the resolution of the feature map by half. Let the original image x e R of the input ^H×W×1 Output compression amountWhere cr is the base compression ratio, i.e. the ratio of x to y in the data dimension, which is not the final compression ratio. In order to enable the compression y to be optimized for even distribution in training, the last downsampling convolution is followed by a binary map.

In the test phase, the binary map is a step function and is divided by 0.5 as shown in formula (1).

The data is composed of 01 data of 1 bit after the binary mapping, so that 8 data points can occupy one byte during storage or transmission, and the final compression ratio is cr×8.

In the training phase, since the gradient of the formula (1) is discontinuous and cannot meet the requirement of transmissibility of the gradient in the back propagation, the binary mapping is replaced by the formula (2) in the training phase, wherein the value of eta is set to 64, the purpose of the binary mapping is to simulate the step, and the gradient can be continuously propagated in the back propagation in the training process at the break point.

The three downsampled convolution kernels of the encoder have a length of 3, the number of convolution kernels of the first two convolutions is set to be 64, the number of convolution kernels of the last convolution is set to be 64/cr, the compression ratio is set to be 32 times, i.e. cr=4 for example, the parameter of the encoder is set to be 46.7KB, and the image x epsilon R is set to be the same as the parameter of the encoder ^128×128×1 For example, an encoderThe floating point operand of (2) is 169.869M FLPs, and the parameter number and the calculated amount are relatively small.

The quantization parameter optimization only brings extra model parameters in the training stage, and has no influence on the testing stage. Therefore, the method can realize the great improvement of the reconstruction quality without any increase of the parameter quantity and the calculation quantity and any change of the processing flow.

S30, performing self-adaptive run-length coding on the image quantized data to obtain image compressed data.

Image compression data is obtained through self-adaptive run-length coding, the compression ratio is improved, the image data compression effect is improved, the compression amount obtained through an encoder can meet the requirements of practical application, and the following conditions in the related art are improved: the compression amount obtained by the encoder in the related art is not high enough, so that the compression amount is difficult to meet the requirement of practical application, and the compression ratio needs to be further improved.

Referring to fig. 3, in some embodiments, adaptive run-length encoding is performed on image quantized data to obtain image compressed data, which may include steps S301 to S303:

s301, performing expansion re-representation processing on the image quantized data to obtain data segments after expansion re-representation.

In some examples, performing the expanded re-representation process on the image quantized data to obtain the expanded re-represented data segment may include: expanding the image quantized data into a one-dimensional vector; traversing the one-dimensional vector, and recording the continuous repetition times of continuous repeated characters in the one-dimensional vector; and aiming at characters which are continuously repeated for a plurality of times and exist in the quantized image data, the number of times of continuous repeated for a plurality of times is represented by a number, and a data segment after being unfolded and re-represented is obtained.

S302, determining data segments meeting preset requirements and data segments not meeting the preset requirements in the data segments after the re-representation is unfolded according to the image resolution.

S303, performing run-length coding on the data segments meeting the preset requirements, and adding the image resolution into the run-length coding data without changing the data segments not meeting the preset requirements to obtain image compression data.

Specifically, the entropy of the binary compression amount y obtained by the encoder has been close to 1, i.e., its distribution has been very close to uniform distribution, the number of 0 and 1 values tends to be equal, there is little possibility of recompression from the standpoint of character statistics, but its spatial continuity is not considered. In practice, there are a number of portions of higher spatial correlation in the compression amount, for which the compression ratio can be further improved on a lossless basis by using run-length coding, and for portions of lower spatial correlation, the transmission is still performed in the original manner, and the portions to be run-length coded are adaptively selected. The run-length coding only needs to traverse data once, has very low calculation amount and has great advantages for image processing.

Compression amount continuity analysis:

performing adaptive run-length coding on the quantized image data to obtain an image compression amount, wherein the method comprises the following steps: the number of times of repeated and successive occurrences is represented by a number for repeated and successive occurrences of a character present in the quantized image data.

The idea of run-length coding is to describe a character that appears repeatedly and consecutively a number of times with "number of consecutive occurrences, a certain character". For example, for the character string "aaaaabbbbcc", it can be described as "5A4B3C" using run-length encoding, and information that originally needs 12 characters to be represented can be represented, and only 6 characters are needed to be represented after encoding, thereby realizing data compression. However, the run-length encoding is not effective in all cases, for example, for the character string "ABCD", the run-length encoding is used to describe the character string as "1A1B1C1D", 4 characters of information are originally required for representing the character string, and 8 characters are required for representing the character string after encoding, so that it is obvious that the effect of the run-length encoding has a direct relationship with the continuity of the data amount, and when the continuity of the data is better, the run-length encoding can be used for cost saving.

For example, L can be selected ₂ The continuity of the compression y of the optimized 32-fold compression basic model is explored. For original image x E R ^128×128×1 Pressure obtained by an encoderShrinkage y epsilon R ^16×16×16 Spread it into a one-dimensional vector y εR ⁴⁰⁹⁶ The size is 512 bytes. The compression amount y is subjected to the following downstream run-length encoding processing: the string of characters is replaced with the same number of consecutive occurrences of the character, as shown in fig. 4, since only character continuity is considered here, it is not necessary to distinguish which point represents character 0 or 1.

The compression y is such that the continuity of the partial areas is very good, and these areas can be further compressed by run-length encoding.

Adaptive run-length coding design:

since the compressed amount y has a region with good continuity and a region with poor continuity, the region with good continuity cannot be directly coded in a run-length mode without distinction, the region with good continuity needs to be found, the run-length coding is used here, and for the region with poor continuity, the original bit-by-bit transmission mode is still used, so that an adaptive run-length coding scheme needs to be designed.

A set of characters is set as a marker of whether run-length encoding is used, which marker cannot be exactly the same as any piece of data in the compression amount y, otherwise misinterpretation of the decoder is caused, but the marker cannot be too long, otherwise the performance of adaptive run-length encoding is reduced. If 8-bit characters are used as the markers, the situation that the characters happen to be identical is easy to happen, and the probability that the characters of 16 bits happen to be identical is extremely low, so the characters of 16 bits are very suitable as the markers. Here, "88170" is selected as the marker, and its binary character is "101100010101010". The five data sets were tested and found that none of the images contained the same character segment as "88170" in the compression amount y. For a compression amount y of 128×128 images, the longest continuous character length in the five data sets is less than 288, so it is appropriate to select 8-bit data bits to represent the length thereof, and it is necessary to consider all cases since it is not possible to determine whether the longest continuous field length of the compression amount of all test images encountered in the future will be longer. Whereas for a compression y of 256 x 256 images the longest consecutive character length in the five data sets does not exceed 2000, so a 12 bit data bit is chosen to indicate that it is appropriate. The adaptive run-length coding mode is shown in fig. 5, when encountering count continuous characters, a marker is inserted in front of the count continuous characters, then the characters are inserted, and finally the count 8-bit or 12-bit binary system is used for replacing the characters.

After adaptive run-length coding is performed on the continuous character, 32 bits or 36 bits are used to represent the segment of character, so that the run-length coding can be used only if the number of continuous repeated occurrences of the segment of character is more than 32 or 36, and the original bit-by-bit transmission mode is still used if the number of continuous occurrences is less than 32 or 36.

For 128×128 images, if the number of consecutive repetitions of the character is greater than 255, the 8-bit character cannot represent the data, the first 255 characters are run-coded as a group, then it is determined whether the subsequent number of consecutive repetitions is greater than 32 to determine whether to use run-coding, and so on, the number of bits after run-coding is as shown in equation (3).

For 256×256 images, it is verified through a large number of experiments that the number of consecutive repetitions of the character is not possible to be larger than 4095 (212-1), so that when the number of consecutive repetitions is smaller than 36, run-length encoding is not performed, direct transmission is performed, and when the number of consecutive repetitions is larger than 36, adaptive run-length encoding is performed. The number of bits after run-length encoding is shown in formula (4).

In summary, the encoding and decoding steps of the adaptive run-length encoding in the embodiment of the present application may include:

(1) Run-length coding: expanding the compression amount y into a one-dimensional vector, traversing the vector, recording the continuous repetition times of the vector, selecting the data segment meeting the requirements according to the formula (3) or (4) according to the image resolution, performing run-length coding on the data segment not meeting the requirements, and adding the image resolution into the transmission data (the information amount is small and ignored).

(2) Run-length decoding: traversing the received data, if the marker '101100010101010' is encountered, the first 8 bits of the following data are the character, selecting the last 8 bits or 12 bits as the number of continuous repetition of the character according to the image resolution, and decoding the character according to the number of continuous repetition, wherein other areas are kept unchanged.

The adaptive run-length coding adopted in the embodiment of the application only needs one traversal with the time complexity of O (n), wherein n is the dimension of the compression y, no data operation is involved, the complexity of the encoder is slightly influenced, the encoder is seamlessly inserted into a trained model, the use is simple, and the decoding performance is effectively improved. However, the compression ratio which can be improved by the method is closely related to the data characteristic of the compression amount y, so that the compression ratio cannot be accurately controlled. The adaptive run-length coding is fused into the whole image compression framework, and the flow is shown in fig. 6.

In some embodiments, the entropy-free image compression encoding method may further include:

and S40, sending the image compression data to a decoder so that the decoder receives the image compression data from the encoder, and performing run-length decoding on the image compression data to obtain a decoded reconstructed image.

According to the entropy-free image compression coding method, the image compression data is obtained through self-adaptive run-length coding, the compression ratio is improved, the image data compression effect is improved, the compression amount obtained through the coder can meet the requirements of practical application, and the following conditions in the related technology are improved: the compression amount obtained by the encoder in the related art is not high enough, so that the compression amount is difficult to meet the requirement of practical application, and the compression ratio needs to be further improved. In addition, the image compression and encoding method without entropy encoding of the embodiment of the application also realizes image compression without entropy encoding, shortens image compression time, improves image compression efficiency, and improves the condition that the image compression time is longer due to entropy encoding in the related technology, so that the image compression method of the related technology is difficult to meet the requirement of practical application in efficiency.

Another embodiment of the present application provides an image compression encoding method without entropy encoding, which may include steps 501 to 502:

step 501, a decoder receives image compression data from an encoder, and performs run-length decoding on the image compression data to obtain a decoded reconstructed image.

Specifically, in one example, run-length decoding may include: traversing the received data, if the marker '101100010101010' is encountered, the first 8 bits of the following data are the character, selecting the last 8 bits or 12 bits as the number of continuous repetition of the character according to the image resolution, and decoding the character according to the number of continuous repetition, wherein other areas are kept unchanged.

Step 502, processing the reconstructed image by the post-processing network model to obtain a post-processing optimized image; the post-processing network model comprises a first convolution layer, a downsampling convolution layer, a GRDB module, an upsampling convolution layer, a CBAM module and a second convolution layer which are sequentially connected.

The reconstructed image obtained by the decoder often has compression artifacts and smoother texture details, and needs further improvement. The presence of the post-processing network model following the decoder is reasonable and beneficial. However, the post-processing is performed on the original resolution image, the memory occupation is large, and an optimal choice needs to be made between the sustainable memory cost and the performance gain that can be brought by the memory cost, instead of increasing the complexity of the post-processing network model infinitely.

The post-processing network model can be applied in the field of end-to-end image compression. If the post-processing network model and the codec are optimized together end to end, the effect of improving the reconstruction effect is weaker than if the post-processing network model is fused into the decoder. Thus, embodiments of the present application individually train the post-processing network model, equating this task to a decoded downstream task: image compression distortion is reduced.

In the embodiment of the application, a GRDN (group residual error dense network) model is adopted to form a post-processing network model. The GRDN model can be used for image noise reduction, and its network structure is shown in fig. 7, and is composed of a first convolution layer, a downsampling convolution layer, three GRDB (Grouped Residual Dense Block) modules, an upsampling convolution layer, a CBAM module, a second convolution layer and an adding module, where the use of the downsampling convolution layer and the upsampling convolution layer helps to eliminate the influence of noise. The upsampling convolutional layer may be a transposed convolutional layer.

Illustratively, the GRDB module may include a first RDB module, a second RDB module, a third RDB module, a splicing module, a convolution module, and an addition module sequentially connected from front to back, where an output end of the first RDB module, an output end of the second RDB module, and an output end of the third RDB module are respectively connected to an input end of the splicing module; the input end of the first RDB module is connected with the input end of the addition module.

Illustratively, any one of the first, second, and third RDB modules includes a first convolution activation module, a second convolution activation module, a third convolution activation module, a splice module, a convolution module, and an addition module that are sequentially connected from front to back; the input end of the first convolution activation module is respectively connected with the input end of the second convolution activation module, the input end of the third convolution activation module and the input end of the addition module; the input end of the second convolution activation module is respectively connected with the input end of the third convolution activation module and the input end of the splicing module; the input end of the third convolution activation module is connected with the input end of the splicing module.

For example, as shown in fig. 8, the network structure of the GRDB modules is that 3 RDB (Residual Dense Block) modules extract multi-level information, and finally fusion is performed. The network structure of the RDB module is shown in fig. 9, and consists of interlayer dense connection among multiple layers of convolutions, so as to reference the design idea of DenseNet, but the connection is changed from tensor splicing to element-by-element addition, and the purpose of the RDB module is to reduce the calculation amount and the occupation of memory. Such dense connections can more effectively enhance the transfer between features.

For example, the CBAM module (Convolutional Block Attention Module, convolution block attention module) is a simple and effective attention module applied to a feedforward convolution neural network, the network structure of which is shown in fig. 10. Given a feature map, the CBAM module would sequentially compute the attention weight along two independent dimensions of channel and spatial pixels, and then weight multiply the attention weight map with the input feature map for adaptive feature optimization.

For example, as shown in fig. 11, the channel attention mechanism in the CBAM module is similar to SENet, and compresses the feature map in the space dimension to obtain a one-dimensional vector, except that not only global average pooling is considered, but also global maximum pooling is introduced as supplement, the former has feedback to each pixel point on the feature map during back propagation, and the latter only has feedback to the place with the largest response in the feature map, so that the importance of the channel dimension is extracted more effectively.

The spatial attention in the CBAM module is shown in fig. 12, and the feature map is compressed using the average pooling module and the maximum pooling module as well, except that the compression is performed in the channel dimension, so as to obtain two single-channel attention weight matrices and perform the stitching fusion.

The objective function of the post-processing network model is the original image x and the post-processing imageDistortion measure betweenUsing L ₂ The loss is shown as a formula (5), wherein H represents a post-processing network model, E _n Representative encoder, D _e Representing a decoder.

In order to further improve the compression performance of the entropy-free coding image compression algorithm, the method of the embodiment of the application firstly designs a post-processing quality enhancement network, performs independent training instead of end-to-end training, thereby obtaining 0.1dB improvement on PSNR, then optimally designs quantization parameters in training, learns unique quantization parameters for each compression amount, and realizes 0.3-0.5 dB improvement on PSNR without changing any step of a test stage. Finally, self-adaptive run-length coding is designed for the compression quantity, and different run-length coding boundaries are adopted according to the compression quantity characteristics of images with different resolutions, so that the PSNR is improved by 0.6-1 dB, and the compression performance is improved.

Referring to fig. 13, another embodiment of the present application provides an image compression encoding apparatus without entropy encoding, including an encoder, a decoder, and a post-processing network model;

In some embodiments, the encoder may also be configured to: performing expansion re-representation processing on the image quantized data to obtain a data segment after expansion re-representation; determining data segments meeting preset requirements and data segments not meeting the preset requirements in the data segments after the re-representation is unfolded according to the image resolution; and performing run length coding on the data segments meeting the preset requirements, and adding the image resolution into the run length coding data without changing the data segments not meeting the preset requirements to obtain image compression data.

In some embodiments, the encoder may also be configured to: expanding the image quantized data into a one-dimensional vector; traversing the one-dimensional vector, and recording the continuous repetition times of continuous repeated characters in the one-dimensional vector; and aiming at characters which are continuously repeated for a plurality of times and exist in the quantized image data, the number of times of continuous repeated for a plurality of times is represented by a number, and a data segment after being unfolded and re-represented is obtained.

In some embodiments, the encoder includes three downsampled convolutional layers and a binary mapping layer in series in sequence; the encoder may also be used to: the encoder sequentially carries out downsampling convolution processing on the original image through three downsampling convolution layers to obtain downsampled convolution processed image data; and performing binary mapping on the image data subjected to the downsampling convolution processing by utilizing a binary mapping layer to obtain the image data subjected to the sampling convolution.

In some embodiments, the encoder may also be configured to: and mapping the sampled and convolved image data to binary code streams obeying uniform distribution through quantization to obtain image quantized data.

In some embodiments, the encoder may also be configured to: and sending the image compression data to a decoder so that the decoder receives the image compression data from the encoder, and performing run-length decoding on the image compression data to obtain a decoded reconstructed image.

In some embodiments, the GRDB module includes a first RDB module, a second RDB module, a third RDB module, a splice module, a convolution module, and an addition module that are sequentially connected from front to back, where an output of the first RDB module, an output of the second RDB module, and an output of the third RDB module are respectively connected to an input of the splice module; the input end of the first RDB module is connected with the input end of the addition module.

In some embodiments, any one of the first, second, and third RDB modules includes a first convolution activation module, a second convolution activation module, a third convolution activation module, a splice module, a convolution module, and an addition module connected in sequence from front to back; the input end of the first convolution activation module is respectively connected with the input end of the second convolution activation module, the input end of the third convolution activation module and the input end of the addition module; the input end of the second convolution activation module is respectively connected with the input end of the third convolution activation module and the input end of the splicing module; the input end of the third convolution activation module is connected with the input end of the splicing module.

The image compression and encoding device without entropy encoding of the embodiment of the application obtains image compression data through self-adaptive run-length encoding, improves compression ratio, improves image data compression effect, can meet the requirement of practical application through compression amount obtained by an encoder, and improves the following conditions in the related art: the compression amount obtained by the encoder in the related art is not high enough, so that the compression amount is difficult to meet the requirement of practical application, and the compression ratio needs to be further improved. In addition, the entropy-free image compression and encoding device of the embodiment of the application also realizes the image compression without entropy encoding, shortens the time consumption of image compression, improves the image compression efficiency, and improves the condition that the time consumption of image compression is long and the actual application needs are difficult to meet due to the use of entropy encoding in the related technology.

Another embodiment of the application provides an electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor executing the computer program to implement the method of any of the above embodiments.

Referring to fig. 14, the electronic device 10 may include: processor 100, memory 101, bus 102 and communication interface 103, processor 100, communication interface 103 and memory 101 being connected by bus 102; the memory 101 stores a computer program executable on the processor 100, and when the processor 100 executes the computer program, the image compression encoding method without entropy encoding provided in any of the foregoing embodiments of the present application is executed.

The memory 101 may comprise a high-speed random access memory (RAM: random Access Memory) or may further comprise a non-volatile memory (non-volatile memory), such as at least one disk memory. The communication connection between the device network element and at least one other network element is achieved through at least one communication interface 103 (wired or wireless), the internet, wide area network, local network, metropolitan area network, etc. may be used.

Bus 102 may be an ISA bus, a PCI bus, an EISA bus, or the like. The buses may be divided into address buses, data buses, control buses, etc. The memory 101 is configured to store a program, and the processor 100 executes the program after receiving an execution instruction, and the method disclosed in any of the foregoing embodiments of the present application may be applied to the processor 100 or implemented by the processor 100.

The processor 100 may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in the processor 100 or by instructions in the form of software. The processor 100 may be a general-purpose processor, and may include a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), and the like; but may also be a Digital Signal Processor (DSP), application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be embodied directly in the execution of a hardware decoding processor, or in the execution of a combination of hardware and software modules in a decoding processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in the memory 101, and the processor 100 reads the information in the memory 101 and, in combination with its hardware, performs the steps of the method described above.

The electronic device provided by the embodiment of the application and the method provided by the embodiment of the application have the same beneficial effects as the method adopted, operated or realized by the electronic device and the method provided by the embodiment of the application due to the same inventive concept.

Another embodiment of the present application provides a computer-readable storage medium having stored thereon a computer program that is executed by a processor to implement the entropy-free image compression encoding method of any one of the above embodiments. Referring to fig. 15, a computer readable storage medium is shown as an optical disc 20 having a computer program (i.e., a program product) stored thereon, which, when executed by a processor, performs the method provided by any of the embodiments described above.

It should be noted that examples of the computer readable storage medium may also include, but are not limited to, a phase change memory (PRAM), a Static Random Access Memory (SRAM), a Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a flash memory, or other optical or magnetic storage medium, which will not be described in detail herein.

The computer-readable storage medium provided by the above-described embodiments of the present application has the same advantageous effects as the method adopted, operated or implemented by the application program stored therein, for the same inventive concept as the method provided by the embodiments of the present application.

It should be noted that:

the term "module" is not intended to be limited to a particular physical form. Depending on the particular application, modules may be implemented as hardware, firmware, software, and/or combinations thereof. Furthermore, different modules may share common components or even be implemented by the same components. There may or may not be clear boundaries between different modules.

The algorithms and displays presented herein are not inherently related to any particular computer, virtual machine, or other apparatus. Various general purpose devices may also be used with the examples herein. The required structure for the construction of such devices is apparent from the description above. In addition, the present application is not directed to any particular programming language. It will be appreciated that the teachings of the present application described herein may be implemented in a variety of programming languages, and the above description of specific languages is provided for disclosure of enablement and best mode of the present application.

It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, these steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited in order and may be performed in other orders, unless explicitly stated herein. Moreover, at least some of the steps in the flowcharts of the figures may include a plurality of sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, the order of their execution not necessarily being sequential, but may be performed in turn or alternately with other steps or at least a portion of the other steps or stages.

The foregoing examples merely illustrate embodiments of the application and are described in more detail and are not to be construed as limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of the application should be assessed as that of the appended claims.

Claims

1. An entropy-free image compression encoding method, which is applied to an encoder, comprising:

2. The entropy-free image compression encoding method according to claim 1, wherein performing adaptive run-length encoding on the image quantized data to obtain image compressed data comprises:

3. The entropy-free image compression encoding method according to claim 2, wherein performing the expansion re-representation processing on the image quantized data to obtain the expanded re-represented data segment, comprises:

expanding the image quantization data into a one-dimensional vector;

4. The entropy-free image compression encoding method of claim 1, wherein the encoder comprises three downsampling convolution layers and a binary mapping layer connected in series in sequence; the step of performing sampling convolution on the original image to obtain sampled and convolved image data comprises the following steps:

5. The entropy-free image compression encoding method according to claim 1, wherein the quantizing the sample convolved image data to obtain image quantized data comprises:

6. The entropy-free image compression encoding method according to claim 1, further comprising: and sending the image compression data to a decoder so that the decoder receives the image compression data from the encoder, and performing run-length decoding on the image compression data to obtain a decoded reconstructed image.

7. An image compression encoding method without entropy encoding, comprising:

8. The entropy-free image compression encoding method according to claim 7, wherein the GRDB module comprises a first RDB module, a second RDB module, a third RDB module, a splicing module, a convolution module and an addition module which are sequentially connected from front to back, wherein an output end of the first RDB module, an output end of the second RDB module and an output end of the third RDB module are respectively connected with an input end of the splicing module; the input end of the first RDB module is connected with the input end of the addition module.

9. The entropy-free image compression encoding method according to claim 8, wherein any one of the first RDB module, the second RDB module, and the third RDB module comprises a first convolution activation module, a second convolution activation module, a third convolution activation module, a splicing module, a convolution module, and an addition module, which are sequentially connected from front to back; the input end of the first convolution activation module is respectively connected with the input end of the second convolution activation module, the input end of the third convolution activation module and the input end of the addition module; the input end of the second convolution activation module is respectively connected with the input end of the third convolution activation module and the input end of the splicing module; and the input end of the third convolution activation module is connected with the input end of the splicing module.

10. An image compression coding device without entropy coding is characterized by comprising an encoder, a decoder and a post-processing network model;

11. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor executing the computer program to implement the entropy-free encoded image compression encoding method of any of claims 1-9.

12. A computer-readable storage medium having stored thereon a computer program, wherein the computer program is executed by a processor to implement the entropy-free image compression encoding method of any of claims 1-9.