CN115514964A

CN115514964A - HEVC compression noise standard deviation estimation method based on convolutional neural network

Info

Publication number: CN115514964A
Application number: CN202110699673.0A
Authority: CN
Inventors: 何小海; 孙伟恒; 熊淑华; 徐艺菲; 陈洪刚; 滕奇志; 任超
Original assignee: Sichuan University
Current assignee: Sichuan University
Priority date: 2021-06-23
Filing date: 2021-06-23
Publication date: 2022-12-23

Abstract

The invention discloses an HEVC compression noise standard deviation estimation method based on a convolutional neural network, which mainly comprises the following steps of: the method comprises the steps of dividing a video frame subjected to HEVC coding into blocks, extracting blocking information from a code stream to determine whether a current block belongs to a high texture block or a low texture block, obtaining a real noise standard deviation of a small block by subtracting an original video and a compressed video, and using the real noise standard deviation as a label of the small block to construct a data set. The invention constructs a compression noise estimation network, and trains high and low texture data sets respectively to obtain a trained model. In the testing stage, videos are tested according to HEVC standards with different resolutions, and the standard deviation level of compression noise of the videos is estimated by taking a block as a unit under different QPs and compared with real noise. Experimental results show that the method can effectively estimate the noise standard deviation of the compressed video.

Description

HEVC compression noise standard deviation estimation method based on convolutional neural network

Technical Field

The invention relates to a convolutional neural network and a video coding algorithm, in particular to an HEVC compression noise standard deviation estimation method based on the convolutional neural network, and belongs to the field of image communication.

Background

In recent years, video applications have grown rapidly, but at the same time, challenges have been presented to the transmission and storage of large amounts of video data. In this context, the International Telecommunications Union (ITU) has promulgated a High Efficiency Video Coding standard (HEVC/h.265) in cooperation with the International Organization for standardization (ISO). Compared with advanced video Coding (AVC/h.264), the HEVC standard can save about 50% of code rate under the same Coding quality. Inevitably, however, HEVC compressed video also contains many compression effects, especially in low bitrate segments, and the presence of these compression effects seriously affects the viewing experience of the user. The HEVC standard employs a block-based hybrid coding strategy, and after the predictive coding is finished, the prediction residual is transformed and quantized in units of blocks. The pixels at the block boundary are continuous, and the inconsistency of the processing parameters of the pixels distributed into different blocks causes the occurrence of blocking effect in a compressed video; quantization rounding and high-frequency zero setting cause the compressed video to lose a lot of detail information; inaccuracies in motion estimation and motion compensation further introduce compression noise. Many scholars refer to the following: accurate evaluation of the compression noise level can help the subsequent decompression effect algorithm, and setting an incorrect noise level may reduce the effect of denoising. The distribution of the compression noise is related to the compression process, so that the modeling difficulty is high, the mainstream research still regards the compression noise as additive gaussian noise at present, and the estimation of the compression noise is the estimation of the standard deviation of the gaussian noise. Convolutional neural networks have achieved significant success in the field of computer vision and image processing. Compared with the traditional image prior constructed by summarizing the image/video signal distribution, the convolutional neural network can learn the deep image characteristics in the image/video, which is also an important reason for obtaining excellent effects on various image/video processing tasks. Inspired by great success of a classification network based on CNN, the invention takes HEVC compression noise standard deviation estimation as a classification task for images of different styles.

Disclosure of Invention

The invention aims to estimate the standard deviation of HEVC compressed video compression noise.

The invention provides an HEVC compression noise standard deviation estimation method based on a convolutional neural network, which mainly comprises the following operation steps of:

(1) A training data set is constructed.

We performed 32 frame coding of 200 training videos in the data set LDV at QP =22, 27, 32, 37, the coding configuration being LDP. Then, the picture is divided into 64 × 64 small blocks, the difference is made between the small blocks and the original video to obtain a residual block of each image block, the standard deviation of the residual block is calculated, and the picture is classified according to the standard deviation, as shown in fig. 1. In general, pictures with low level of compression noise contain less texture detail, and pictures with high level of compression noise contain more texture. But we have also found special cases where some pictures with high noise levels also contain less texture, because the entire picture becomes smoother due to the coarse quantization of HEVC and the high frequency quality of the transform coefficients. For the image blocks which have less texture, the compressed image blocks represent the smooth blurring and have less compression noise, while the image blocks which have rich dense texture represent the smooth blurring and have larger compression noise. To solve this problem, we extract the blocking information from the encoded bitstream to determine whether a compressed smooth block is a high texture block or a low texture block before compression. As shown in fig. 2, when HEVC is compressed, quad-tree-based block division is adopted, dense block division is performed on a region with complex texture, sparse block division is performed on a region with simple texture, CU block information of HEVC is extracted from a code stream, an image is divided into a high texture block and a low texture block according to the depth of the CU block, the high texture block is determined when the depth is greater than 2, and the low texture block is determined when the depth is less than or equal to 2. A data set is constructed for each of the high texture blocks and the low texture blocks. The high texture blocks are divided into 15 classes, the standard deviation difference of each class is 1, from 0, the value width of each class is 0.5, if the standard deviation value interval of the first class is [0,0.5], the standard deviation value interval of the second class is [1,1.5], and the value space of the later class is analogized according to the form. For the low texture blocks, the low texture blocks are divided into 10 classes, the standard deviation difference of each class is 1, and the value width of each class is 0.7 from 0.

(2) And constructing a compression noise estimation network based on the convolutional neural network.

We propose an HEVC compression noise classification network whose network backbone is the classical classification network DENSE-NET, as shown in fig. 3, we make some modifications to the network to meet our task requirements. Since our block edge length is 64, we add a transposed convolutional layer before the first Dense block to further increase the field of view to enlarge the resolution of the feature map, and use 3 Dense blocks to achieve the balance of complexity and performance.

(3) And (3) carrying out network model training according to the data set in (1) and the network proposed in (2), and estimating the compression noise standard deviation for the HEVC compression test video block by using the trained model.

Drawings

Fig. 1 is a schematic diagram of classifying pictures based on different standard deviations of compression noise according to the present invention.

Fig. 2 is a high and low texture block distribution diagram of the present invention.

Fig. 3 is a block diagram of an HEVC compression noise standard deviation estimation network according to the present invention.

Detailed Description

In order to better illustrate the effectiveness of the present invention, videos with different resolution sizes in a standard HEVC test video are selected, noise estimation is performed on different QPs, and table one shows the difference between the estimation result of the present invention and the real noise. From the table, it can be seen that the present invention can obtain a more accurate estimation result.

Table one standard deviation difference between the invention and true noise

Claims

1. An HEVC compression noise standard deviation estimation method based on a convolutional neural network is characterized by comprising the following steps of:

the method comprises the following steps: firstly, constructing a training data set, partitioning a video compressed by HEVC, extracting partitioning information from code stream information, dividing small blocks into high texture blocks and low texture blocks, calculating a real noise standard deviation by calculating a difference value between an original video and the compressed video, taking the real noise standard deviation as a label of each small block, and finally constructing two training data sets;

step two: constructing a network, namely constructing the network on the basis of DENSE-NET, adopting 3 Dense blocks, and increasing a transposed convolution layer to enlarge the resolution of a characteristic diagram;

step three: and (3) training the data set obtained in the step one through the network provided in the step two to obtain two models aiming at the high and low texture blocks, wherein the obtained models can effectively estimate the value of the standard deviation of the HEVC compressed video.