CN103475876A - Learning-based low-bit-rate compression image super-resolution reconstruction method - Google Patents

Abstract

The invention discloses a learning-based low bit rate compressed image super-resolution reconstruction method, and the algorithm is divided into an offline part and an online part. In the offline part, the low-resolution image is first compressed with different compression quality parameter values; then the compressed image is filtered, and then the quantized distortion degree of the filtered compressed image is used as a feature, and the filtered LR image is classified according to its degree of distortion. Establish a classification sample library for multiple classes, and then use each type of sample to train the super-resolution model. In the online part, the input image is firstly filtered, and then its compression distortion category is determined. Then, according to the determination result, the sample library and super-resolution model of the corresponding category are selected to realize super-resolution restoration based on learning. Compared with other algorithms, the method of the present invention can adaptively adjust the matching sample library for input LR images with different degrees of distortion, and can effectively solve the impact of block effect distortion on image super-resolution, and directly solve the problem of low bit rate Compared with the super-resolution restoration of the distorted image, the image reconstructed by the method of the present invention has higher subjective and objective quality.

Description

A learning-based super-resolution reconstruction method for low-bit-rate compressed images

technical field

The invention relates to an image processing method, in particular to a learning-based low-bit-rate compressed image super-resolution reconstruction method.

Background technique

High-quality images and videos are increasingly becoming a mainstream demand because of their richer information and more realistic visual experience. However, the higher the quality of the image, the greater the amount of data, which brings a great burden to the storage, transmission, and processing of image information. Affected by factors such as transmission bandwidth and storage space, there is an increasing demand for low-bit compression of video and image content. However, when the JPEG compression rate is high, it usually leads to a decrease in the quality of the reconstructed image after decoding, affecting the subjective quality and automatic analysis of information. Therefore, for highly compressed low-quality images, it is of great theoretical significance and practical application value to study super-resolution restoration technology to improve image quality.

However, due to the limitations of the image compression method itself, when the super-resolution method for uncompressed images is used to directly perform super-resolution reconstruction on low-bit compressed images, severe blockiness distortion will occur. Therefore, removing blockiness becomes an important issue in the process of super-resolution restoration of low-bit compressed images. People often use post-processing methods to reduce block effects. The post-processing method is independent of the decoder when filtering, and directly operates on the decoded image, which has the advantages of flexibility, simplicity, and effectiveness. Post-processing methods based on image enhancement have been extensively studied because they do not rely on any decoding information, can independently remove blocking artifacts, and are easier to implement. Among them, spatial filtering is the most basic method, which directly processes the brightness values of pixels in the image. Since the block effect in the image is caused by the quantization error, the change of the image content will cause the expression of the block effect of the compressed image, so it is very important to adaptively select filters with different smoothing strengths for different areas of the image. . The typical spatial domain filtering method usually adopts the method of texture classification, which divides the image into flat blocks and non-flat blocks according to the human visual characteristics, and then performs adaptive filtering. Two notable features of this method are to divide the image into Different regions use different filtering methods to remove block effects. The usual method is to use high-intensity smoothing in flat areas, because there is no high-frequency information in flat areas, and high-intensity smoothing will not make the image excessively blurred. For non-flat blocks, due to the existence of more high-frequency information, the application of low-intensity smoothing filtering can better preserve high-frequency detail information. However, in this algorithm, regions containing obvious edge information are collectively classified as texture regions and are not processed separately. Experimental results show that such regions will have a greater impact on the subjective quality of the image due to quantization distortion. Therefore, using the same filter as the texture area cannot deal with the quantization distortion problem in the edge area well.

Contents of the invention

The purpose of the present invention is to solve low-resolution images with block effect distortion by using an improved post-processing filtering algorithm and a sample pre-classification method for low-bit-rate compressed images, combined with a learning-based image super-resolution method. The problem of super-resolution restoration of images.

The present invention is realized by adopting the following technical means:

A learning-based method for super-resolution reconstruction of low-bit-rate compressed images, the overall flow chart is shown in Figure 1; the algorithm is divided into an offline part and an online part; its flow charts are shown in Figure 2 and Figure 3 respectively; In the offline part, a classification sample library is established according to the degree of distortion of the compressed image; firstly, the low-resolution (LR, Low Resolution) image is compressed by using different compression quality parameters (CQ, Compressed Quality) values; then the compressed image is filtered to remove the distortion Part of the block effect in the image; then, the quantitative distortion degree of the filtered compressed image is used as a feature for K-means clustering, and the filtered LR image is divided into multiple categories according to the degree of distortion and a classification sample library is established. Perform super-resolution model training; in the online part, classify the compression degree of the input image, and then realize super-resolution restoration based on learning; first filter the input low-bit-rate compressed LR image, and then evaluate the degree of quantization distortion , select the sample library and super-resolution model of the corresponding category, and perform learning-based super-resolution restoration.

The offline part is divided into 4 steps:

(1) Apply the JPEG compression method to m LR images, perform compression processing with n different CQ values, and generate m×n sample images with different degrees of compression distortion. The LR image mentioned above refers to the high-resolution (HR, High Resolution) image, if the number of pixels per unit area of an HR image is H, then relative to this HR image, the area per unit area If the number of pixels occupied is less than or equal to 1/2×H, it is generally regarded as a low-resolution image. Here, the CQ value is the compression quality parameter when using JPEG to compress the image, and the compressed image with different compression rates can be obtained by adjusting the CQ size;

(2) The image obtained in (1) is filtered by a post-processing filtering method to remove part of the block effect in the distorted image.

(3) Evaluate the degree of image compression distortion after filtering. Calculate the blockiness distortion value of each filtered image as the evaluation value of the degree of distortion. As mentioned above, the post-processing filtering method firstly classifies the texture of the image, divides the image into edge area, texture area and flat area, and then according to the type of image block, selects different filtering methods to filter the image to remove Blocking artifacts in images. As mentioned above, the method for evaluating the degree of distortion of image block effect uses the MSDS value of the image as the evaluation value of the degree of distortion of image block effect. MSDS (MSDS, Mean Squared Difference of Slopes) is a block effect evaluation criterion of mean square slope error. The MSDS algorithm measures the degree of block effect by describing the sharpness of pixel jumps at the border of the image block. In a normal image, the brightness values of pixels at the boundary of adjacent blocks usually transition smoothly or continuously. In a normal image, edge pixel value jumps cannot always occur at the boundary of the block. If the pixels at the boundary of the image block If the value always jumps, it can be considered that there is a block effect here. MSDS reflects the pixel jump between blocks and block boundaries by calculating the pixel difference at the border of adjacent image blocks and the average brightness difference of pixels near the border. Therefore, according to the size of the MSDS value, the image block can be identified The severity of the effect distortion.

(4) Using the obtained evaluation value as a feature, the K-means clustering method is used to divide these images into N categories according to the degree of distortion of the filtered images; for the images in each category, a super-resolution restoration method is established The sample library, combined with the HR image corresponding to the LR image in the sample library, is used to train the super-resolution restoration model based on learning, and prepare for the super-resolution restoration of the input image in the online part;

The online section is divided into 3 steps:

(1) Input a low-bit-rate compressed image, and use the above-mentioned post-filtering method to filter it;

(2) Calculate the MSDS value of the filtered image to obtain the block effect distortion evaluation score of the image; calculate the similarity between the obtained score and the scores of the sample images in N categories, and judge the compression distortion category to which the input image belongs. The specific method of judging the compression distortion category of the input image is as follows: firstly filter the input distorted LR image to obtain its MSDS value, and then calculate the Euclidean distance between it and the cluster centers of N categories to obtain the Euclidean distance from the input image. The category of the cluster center with the closest Euclidean distance is the category to which the input image belongs;

(3) Select the sample in the category closest to the input image as the sample library of the input image, input the data of the filtered input image into the super-resolution restoration model that has been trained in the offline part, and realize the super-resolution of the image recovery rate;

Compared with the prior art, the present invention has the following obvious advantages and beneficial effects:

The invention firstly proposes an improved spatial domain image filtering algorithm, which utilizes the human visual sensitivity feature and divides the image texture into flat blocks, edge blocks and texture blocks. Then filter algorithms with different strengths are used for the three types of block regions to remove the block effect. When building the sample library, this improved filtering method is used to filter the distorted LR image, which better solves the problem of block effect distortion of the reconstructed image; in addition, for LR input images with different degrees of compression distortion, extracted The MSDS value of the filtered image is used as the feature of each image, and then K-means clustering is used to classify images with similar degrees of distortion into one category, and a corresponding sample library is established. When performing super-resolution restoration on input LR images with different degrees of distortion, first evaluate its MSDS value, then establish the most similar class as a sample library for input LR images, and finally apply a learning-based super-resolution algorithm for super-resolution Restoration obtains the output HR image. Compared with other algorithms, the method of the present invention can adaptively adjust the matching sample library for input LR images with different degrees of distortion, and can effectively solve the impact of block effect distortion on image super-resolution, and directly solve the problem of low bit rate Compared with the super-resolution restoration of the distorted image, the image reconstructed by the method of the present invention has higher subjective and objective quality.

Features of the present invention:

1. An improved spatial filtering algorithm is proposed, which divides the image texture into flat blocks, edge blocks and texture blocks, and uses different intensities of filtering algorithms for the three types of block areas respectively. Preserve graphical detail information well;

2. Using the MSDS eigenvalue of the distorted image, K-means clustering is used to classify the degree of distortion of the image, and the sample library that matches the input image can be adaptively adjusted, and the sample library is refined at the same time, reducing the amount of calculation;

3. The sample library used for super-resolution restoration is established after filtering, which effectively solves the problem of block effect distortion in image super-resolution reconstruction;

The following will be described in detail with reference to the accompanying drawings in conjunction with examples, in order to obtain a deeper understanding of the purpose, features and advantages of the present invention.

Description of drawings:

Fig. 1, overall flowchart of inventive method;

Fig. 2, the flow chart of the offline part of the inventive method;

Fig. 3, the flow chart of the online part of the inventive method;

Figure 4. Image texture classification results; (a) original image; (b) edge image; (c) texture image; (d) flat image;

Figure 5. Schematic diagram of block effect;

Figure 6. Schematic diagram of horizontal and vertical boundaries that need to be filtered;

Figure 7. Super-resolution reconstruction model based on example learning;

Fig. 8, method of the present invention compares with existing method result:

(a) Original high-resolution image (b) JPEG compressed image (CQ=15)

(c) Reconstruct the image directly after compression (d) Existing methods

(e) Results of the invention.

Detailed ways:

Below in conjunction with accompanying drawing of description, the embodiment of the present invention is described in detail:

The algorithm of the present invention is divided into two parts: off-line and on-line. In the offline part, a classification sample library is established according to the degree of distortion of the compressed image; firstly, the low-resolution (LR) image is compressed by using different compression quality parameters (CQ) values; Block effect; then use the quantitative distortion degree of the filtered compressed image as a feature to perform K-means clustering, divide the filtered LR image into multiple categories according to its degree of distortion and establish a classification sample library, and use various samples for super-resolution Model training; in the online part, the classification of the input image is determined to complete the super-resolution restoration based on learning; firstly, the input low bit rate compressed LR image is filtered, and then the degree of quantization distortion is evaluated, and the sample library of the corresponding category and the super-resolution are selected. Resolution model for learning-based super-resolution restoration.

(1) Offline part

(1) Compress LR images with different CQ values

Select an HR image sample library (non-compressed image), the specific resolution of the image in the sample library is 140×160 pixels, select 1000 HR images, and then directly downsample these images to generate 1000 low-resolution LR images Images make up a sample library. Select m=300 images from this LR sample library, and the specific resolution is an image of 70×80 pixels. Using JPEG compression, the m LR images are compressed with CQ=5~20, so that a total of 4800 compressed and distorted images are produced. The reason for choosing this interval for the CQ value is that when CQ is less than 5, the degree of image distortion cannot be effectively restored by super-resolution, and it is not a commonly used CQ range. When CQ is greater than 20, the subjective quality of the image is basically the same. There is no perceptible image distortion, and the size of the compressed bitstream exceeds the low bit rate range.

(2) Filter the compressed image

The image obtained in (1) is filtered by the post-processing filtering method. As mentioned above, the post-processing filtering method firstly classifies the image texture, and divides the image into edge area, texture area and flat area; then according to the image block According to the type, different filtering methods are selected to filter the image. The three regions are divided into blocks according to the size of 8×8, and then deblocking filtering is performed in different ways according to the image block types.

First, texture classification is performed on the image, the specific method is:

Select the Sobel operator to extract the gradient information of image pixels, and use the direction template as formula (1):

For an image, first use these 4 direction templates to convolve with the image respectively to obtain the gradient values in the 4 directions of the image. For each point in the image, since the point has 4 gradient values, we Select the largest of the 4 gradient values as the gradient value of the point, denoted as g _max , i corresponds to the row of the image, and j corresponds to the column of the image; then the image is divided into edge, texture and flat regions by using three component models; The specific steps are: 1. Calculate the threshold, TH ₁ =0.12×g _max and TH ₂ =0.06×g _max , where g _max is the largest gradient value among the gradient values of the entire image in the four directions obtained; TH ₁ , TH ₂ is the high and low threshold threshold value, as the basis for judging the texture classification of pixel points, 0.12 and 0.06 are empirical values; 2, adopt formula (2) to divide image pixels into edge, texture or flat area;

$\left\{\begin{array}{cc}\mathrm{<span\; class="notranslate">\; edge\; pixel</span>}<span\; class="notranslate">\; ,</span>& \mathrm{<span\; class="notranslate">\; if\; G</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ></span>\mathrm{<span\; class="notranslate">\; T</span>}{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; ,</span>\\ \mathrm{<span\; class="notranslate">\; smooth\; pixel</span>}<span\; class="notranslate">\; ,</span>& \mathrm{<span\; class="notranslate">\; if\; G</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; <</span>\mathrm{<span\; class="notranslate">\; T</span>}{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; .</span>\end{array}\right.<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

Among them, G(i, j) is the gradient value of each pixel in the image. When the gradient value of the pixel point (i, j) is greater than TH ₁ , the pixel is divided into an edge area (edgepixel); when When the gradient value of the pixel point (i, j) is less than TH ₂ , the pixel point is divided into a flat area (smooth pixel); if the gradient value of the pixel point (i, j) position is between TH ₁ and TH ₂ , the The pixels are divided into texture regions. Taking the Lena image as an example, the division results are shown in Figure 4. The first image from the left in Figure 4 is the original image, followed by the edge image, texture image and flat image. These images have been binarized, and the obvious continuous lines composed of black pixels in Fig. 4(b) represent the edge regions. The scattered black pixels in Fig. 4(c) represent texture regions. The flat areas are represented by patches of black areas in Fig. 4(d).

As mentioned above, the specific method of selecting different filtering methods to filter the image is as follows:

After the image is divided into textures, according to the type of the image block, different filtering methods are used to filter the image block. The specific method is as follows:

Since the block effect in the flat block is most easily perceived by the human eye, a high-strength filter is required to filter out this part of the block effect; use the filter with the highest filtering strength to remove the block effect of the flat block, and only select the corresponding block effect when filtering. Neighboring blocks are the boundaries of flat blocks; assuming that a and b are two flat blocks with a size of 8×8 pixels respectively, and the positional relationship between a and b is adjacent to the left and right, as shown in Figure 5, the right 4 of a Column and the left 4 columns of pixels of b form a new image block c, then the block effect between a and b will be completely retained in c, so the filtering of block effect between a and b is actually filtering c operation; let the 8 pixels in any row in c from left to right be expressed as: p ₃ , p ₂ , p ₁ , p ₀ , q ₀ , q ₁ , q ₂ , q ₃ ; as shown in the left figure of Figure 6 Shown, then use formulas (3), (4) and (5) to filter any row in c:

p' ₀ =(p ₂ +2p ₁ +2p ₀ +2q ₀ +q ₁ +4)/8 (3)

p' ₁ =(p ₂ +p ₁ +p ₀ +q ₀ +2)/4 (4)

p' ₂ =(2p ₃ +3p ₂ +p ₁ +p ₀ +q ₀ +4)/8 (5)

Among them, p' ₀ , p' ₁ and p' ₂ are the filtered results of p ₀ , p ₁ , and p ₂ , and point p ₃ is not processed; when filtering the value of point q, it is only necessary to apply the formula In (3), (4), and (5), the p-point pixel at the corresponding position is changed to q-point pixel, and the q-point pixel is changed to p-point pixel; that is, q ₀ adopts p ₀ filtering method, and q ₁ Use the filtering method of p ₁ , q ₂ adopts the filtering method of p ₂ , and q ₃ uses the filtering method of p ₃ for filtering; when the positions of a and b are adjacent up and down, only need to rotate a and b by 90 degrees at the same time Then filter according to the left and right adjacent conditions, as shown in the right figure of Figure 6;

For the case where the adjacent blocks are all edge blocks, block c is obtained by following the composition method of block c in the above. Since the brightness of the pixel in the middle of c has obvious jumps, the block effect of c is applied to a two-dimensional step function blk to simulate, as shown in formula (6):

$\mathrm{<span\; class="notranslate">\; blk</span>}<span\; class="notranslate">\; =</span>\left\{\begin{array}{c}\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 8</span>}<span\; class="notranslate">\; ;</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 4</span>}\\ <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 8</span>}<span\; class="notranslate">\; ;</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; ,</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 8</span>}\end{array}\right.<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 6</span>}<span\; class="notranslate">\; )</span>$

Among them, the values 1/2 and -1/2 represent the ladder effect in the middle of the image block c; first evaluate the block effect strength of the pixel at the center of each row in the block c, as in formula (7):

β=[c(i,6)-3×c(i,5)+3×c(i,4)-c(i,3)]/2 (7)

Among them, β represents the intensity of block effect in the middle of block c; then choose a suitable smooth function to replace the step function that produces block effect. Considering that the edge block has more detailed information and the texture structure of the position with block effect is complex, so only for The pixels at the boundary are slightly smoothed, and the smoothing function used is as formula (8):

$\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span><span\; class="notranslate">\; (</span>\frac{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; exp</span>}<span\; class="notranslate">\; (</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span>{\mathrm{<span\; class="notranslate">\; \&beta;</span>}}_{\mathrm{<span\; class="notranslate">\; level</span>}}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}$

${\mathrm{<span\; class="notranslate">\; \&beta;</span>}}_{\mathrm{<span\; class="notranslate">\; level</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; \&times;</span><span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; exp</span>}<span\; class="notranslate">\; (</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 10</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; \&beta;</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 50</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 10</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 8</span>}<span\; class="notranslate">\; )</span>$

Among them, β _level is obtained according to the block effect intensity β, which can adaptively control the shape of the smoothing function. When the value of β _level is smaller, the smoothing function is closer to the original step function; the smoothing function can minimize the change of the boundary pixel value, as slightly as possible Reduce the brightness jump at the boundary of the image block without blurring the image; discretize the smoothing function during specific processing to construct a one-dimensional smoothing function:

de_blk(j)=[f(-49)f(-35)f(-21)f(-7)f(7)f(21)f(35)f(49)]

c'(i,j)=c(i,j)+β×[de_blk(j)-blk(j)] (9)

Here, de_blk(j) is an array of 1×8 size generated by f(x), and c`(i,j) is the filtered result of row i of c, where c(i,j) refers to c blk(j) refers to the value after any row in c is simulated as blk, so that each row in the image can effectively remove the block effect after smoothing; in order not to make the entire image excessive due to filtering The boundary between edge blocks and other types of blocks will not be processed; for the boundaries between texture blocks and the boundaries between texture blocks and flat blocks, similar to the case where adjacent blocks are flat blocks The filtering method, the difference is that when filtering, the pixels at the p ₂ position and q ₂ position are not filtered;

(3) Evaluate the degree of image compression distortion after filtering

Calculate the block effect distortion (MSDS, Mean Squared Difference of Slopes) value of each filtered image as the evaluation value of the block effect distortion degree of each image; MSDS is a block effect evaluation criterion of the mean square slope error, and the MSDS algorithm passes To measure the degree of block effect by describing the sharpness of the pixel jump at the border of the image block; in a normal image, the brightness values of the pixels at the border of the adjacent block are usually smooth transition and continuous transition, and the jump of the edge pixel value in the image cannot always appear in the At the boundary of the block; therefore, according to the size of the MSDS value, the severity of the image block effect distortion can be judged; when calculating the MSDS, the pixel difference at the boundary of the adjacent image block and the average brightness difference of the pixels near the boundary are calculated value, the specific method is:

The method for forming block c is the same as that described above, assuming that block c is a new block formed by adjacent blocks a and b, then calculate the MSDS values of a and b as shown in the formula (10):

$\mathrm{<span\; class="notranslate">\; MSDS</span>}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; 7</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; \{</span><span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span><span\; class="notranslate">\; +</span><span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span><span\; class="notranslate">\; \}</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 10</span>}<span\; class="notranslate">\; )</span>$

Since each image block does not have only one adjacent image block, the MSDS value of each block is the sum of each MSDS value obtained from its adjacent blocks; while the MSDS value of an image is the sum of all blocks The average value of the MSDS value;

(4) Establishment of sample library based on K-means clustering

Due to the impact of quantization error on low bit rate images, the reconstructed image will be severely distorted, resulting in a new image structure "block effect". Therefore, when using the super-resolution restoration algorithm to restore such distorted images, if the training results of the original image library are still used, the quality of the reconstructed image will be seriously affected. When the distorted input image is magnified by 2 times, the super-resolution restoration and reconstruction results obtained under the original database and training model will produce serious distortion. Therefore, it is necessary to re-establish the sample library for distorted images. After compressing all LR images by determining a CQ value, a distorted image sample library can be established. Then use this sample library to retrain the super-resolution model to establish the relationship between the LR combination coefficient of the distorted image and the combination coefficient of the HR image, which is used to restore the HR image. However, in the experiment, when the compression rate of LR is high, the distortion of the reconstructed image is very serious, so it is impossible to obtain a better restoration result. Therefore, by adopting the post-processing filtering algorithm in the present invention, the block effect is firstly removed for the distorted samples, and then the filtered LR sample library is established, which can effectively improve the super-resolution restoration effect of the distorted images. On the other hand, since the degree of distortion of the compressed image produced by different compression ratios is significantly different, the degree of block effect distortion in the reconstructed image is also very different. Therefore, it is necessary to distinguish images with different compression ratios. The establishment of the sample library and the training of the super-resolution model are carried out. However, for an input compressed image, its compression information cannot be known, so it is necessary to predict the compression degree of the input image

According to the obtained MSDS value, the filtered samples are divided into N classes by K-means clustering. It should be pointed out here that since a total of 16 quality parameters with a CQ value of 5 to 20 are used to compress the image in the offline part, in principle, the value range of N should be 2 to 16, and the more classifications, the final The result will be better, but the corresponding operation speed will be slower. Therefore, considering the two factors of operation speed and accuracy, it is verified by experiments that N=3 is the most suitable classification for clustering. The sample images contained in each class are used to train the corresponding learning-based super-resolution restoration model.

K-means clustering is an unsupervised learning algorithm. Its core idea is to iteratively divide data samples into different clusters to minimize the objective function, so that the generated clusters are as compact and independent as possible. Given a sample set and an integer K, the K-means clustering algorithm first randomly selects K initial cluster centers, and then assigns the unselected data objects to the cluster with the smallest distance according to their Euclidean distance from each cluster center point middle. Then the average value of all samples in each cluster is used as the center point of the new cluster cluster, that is, the centroid. Repeat the above steps and iterate until the objective function converges. The commonly used objective function is the squared error criterion function:

$\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; min</span>}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; K</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; \&Element;</span>{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; c</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 11</span>}<span\; class="notranslate">\; )</span>$

Among them, x _j is the sample data, that is, the MSDS value of the LR image, C _i represents the centroid of the cluster C _j , and E represents the sum of the Euclidean distances from all samples to the centroid.

After compressing 300 images with a CQ value from 5 to 20 and performing filtering processing to generate a total of 4800 images, the MSDS values of these images are calculated as 4800 feature values, and these features are first randomly divided into 3 categories, and then K-means Clustering, after repeated iterative calculations, three cluster centers are obtained, and the approximate samples contained in each cluster center are stored as a sample library. It should be pointed out that since the training process is to train the relationship between the LR image and the corresponding HR image, the final sample library should also contain the HR image corresponding to each LR image. 300 images were selected from the three sample libraries for training, and a learning-based super-resolution restoration model was established.

(2) Online part

(1) Filter the input image.

The input image is a low-bit compressed image to be processed, which is filtered by the improved post-filtering method in step (2) of the above-mentioned online part to remove the obvious block effect in the image.

(2) Judgment of the block effect distortion category of the input image

First, calculate the MSDS value of the filtered input image according to the method for calculating the MSDS value of the image in step (3) of the offline part as described above. Then, calculate the Euclidean distance between it and the three cluster centers, and determine the corresponding cluster when the calculated Euclidean distance is the smallest as the distortion category to which the input image belongs.

(3) Realize learning-based super-resolution reconstruction

After the input image category is selected, the corresponding samples in this category are selected, and the super-resolution restoration of the image is performed by using the super-resolution model based on learning.

Learning-based super-resolution reconstruction obtains the relationship between high-resolution and low-resolution images through learning algorithms to guide the reconstruction of high-resolution images. Obtain prior knowledge from a large number of training sample sets as the basis for super-resolution restoration, and the training samples are all images that contain the same information as the input image. Here, a representative example-based learning algorithm is used to illustrate the super-resolution reconstruction process, as shown in Figure 7: first, the input image is interpolated and enlarged to the same size as the target high-resolution image; the input image is extracted LR image block, use the LR block and HR block of the training sample library obtained in the offline part, and use the learned relationship between the LR block and the HR block (that is, the model obtained in the offline part of the training) to carry out the high-frequency information of each block Prediction; then add high-frequency information to the result of interpolation and amplification to obtain an output image. In the implementation process, for the convenience of description, the input image is assumed to be LR-A. First, the LR-A is filtered to obtain the image LR-AF, and then the block effect distortion judgment is performed on the LR-AF, and it is judged that it belongs to one of the three categories; then the LR-AF is interpolated and enlarged to obtain the result set is HR-A; and then block extraction is performed on LR-AF to form the input matrix of the super-resolution model; since the offline part has already used the samples in each cluster to train the relationship model between LR samples and HR samples, so the Input the input matrix into the trained model to obtain high-frequency prediction information; finally, add the obtained high-frequency information to HR-A to obtain the final high-resolution result image.

Fig. 8 is a comparison of the results obtained by performing super-resolution reconstruction on a compressed image with CQ=15 between the method of the present invention and the existing method.

Claims (1)

1. A learning-based method for super-resolution restoration of low-bit-rate compressed images; the algorithm is divided into an offline part and an online part; the offline part includes 4 steps; first, the image samples used for training are generated using different compression ratios Images with different compression qualities are then filtered to remove some block effects in the compressed and distorted images, and then a classification sample library is established according to the degree of distortion of the compressed images and a prediction model based on learning super-resolution restoration is trained, which is the input image of the online part Prepare for super-resolution restoration; The online part includes 3 steps; realize the deblocking effect of the input low bit rate compressed image, category determination and final super-resolution restoration; For the offline part, the specific steps are as follows: 1.1 Compress LR images with different CQ values: Use the JPEG compression method for m LR images to compress n different CQ values to generate m×n sample images with different degrees of compression distortion; here LR images refer to uncompressed low-resolution images, and the CQ values is the compression quality parameter when using JPEG to compress the image; 1.2 Filter the compressed image: The image obtained in 1.1 is filtered by a post-processing filtering method to remove part of the block effect in the distorted image; the above-mentioned post-processing filtering method firstly performs texture classification on the image, and divides the image into edge areas, texture areas and flat areas. area; then according to the type of the image block, select different filtering methods to filter the image; the specific method for texture classification of the above-mentioned graphics is: Select the Sobel operator to extract the gradient information of image pixels, and use the direction template as formula (1):

For an image, first use these 4 direction templates to convolve with the image respectively to obtain the gradient values in the 4 directions of the image. For each point in the image, since the point has 4 gradient values, choose 4 The largest one of the two gradient values is taken as the gradient value of the point, which is recorded as g _max ; then the image is divided into edge, texture and flat regions by using three component models; the specific steps are: 1. Calculate the threshold value, TH ₁ =0.12× g _max and TH ₂ =0.06×g _max , where g _max is the largest gradient value among the gradient values of the entire image obtained in 4 directions; TH ₁ and TH ₂ are high and low thresholds, which are used as the basis for distinguishing pixel texture classification , 0.12 and 0.06 are empirical values; 2. Use formula (2) to divide image pixels into edge, texture or flat regions; $\left\{\begin{array}{cc}\mathrm{<span\; class="notranslate">\; edge\; pixel</span>}<span\; class="notranslate">\; ,</span>& \mathrm{<span\; class="notranslate">\; if\; G</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ></span>\mathrm{<span\; class="notranslate">\; T</span>}{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; ,</span>\\ \mathrm{<span\; class="notranslate">\; smooth\; pixel</span>}<span\; class="notranslate">\; ,</span>& \mathrm{<span\; class="notranslate">\; if\; G</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; <</span>\mathrm{<span\; class="notranslate">\; T</span>}{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; .</span>\end{array}\right.<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$ Among them, G(i, j) is the gradient value of each pixel in the image, when the gradient value of the pixel point (i, j) is greater than TH ₁ , the pixel point is divided into the edge area edgepixel; when the pixel point When the gradient value of the position (i, j) is less than TH ₂ , the pixel point is divided into a flat area smoothpixel; if the gradient value of the pixel point (i, j) position is between TH ₁ and TH ₂ , the pixel point is classified as Divided into texture areas; As mentioned above, the specific method of selecting different filtering methods to filter the image is as follows: After the image is divided into textures, according to the type of the image block, different filtering methods are used to filter the image block. The specific method is as follows: Use the filter with the largest filtering strength to remove the block effect of the flat block, and only select the boundary of the adjacent block when filtering; suppose a, b are two flat blocks with a size of 8×8 pixels, and a, b The positional relationship is adjacent to the left and right, and the right 4 columns of a and the left 4 columns of b form a new image block c, then the block effect between a and b will be completely retained in c, so for a , the filtering of the block effect between b is actually a filtering operation on c; let the 8 pixels in any row of c from left to right be expressed as: p ₃ , p ₂ , p ₁ , p ₀ , q ₀ , q ₁ , q ₂ , q ₃ ; then use formulas (3), (4), and (5) to filter any row in c: p' ₀ =(p ₂ +2p ₁ +2p ₀ +2q ₀ +q ₁ +4)/8 (3) p' ₁ =(p ₂ +p ₁ +p ₀ +q ₀ +2)/4 (4) p' ₂ =(2p ₃ +3p ₂ +p ₁ +p ₀ +q ₀ +4)/8 (5) Among them, p' ₀ , p' ₁ and p' ₂ are the filtered results of p ₀ , p ₁ , and p ₂ , and point p ₃ is not processed; when filtering the value of point q, it is only necessary to apply the formula In (3), (4), and (5), the p-point pixel at the corresponding position is changed to q-point pixel, and the q-point pixel is changed to p-point pixel; that is, q ₀ adopts p ₀ filtering method, and q ₁ Use the filtering method of p ₁ , q ₂ adopts the filtering method of p ₂ , and q ₃ uses the filtering method of p ₃ for filtering; when the positions of a and b are adjacent up and down, only need to rotate a and b by 90 degrees at the same time Then filter according to the left and right adjacent situations; For the case where the adjacent blocks are all edge blocks, block c is obtained by following the composition method of block c in the above. Since the brightness of the pixel in the middle of c has obvious jumps, the block effect of c is applied to a two-dimensional step function blk to simulate, as shown in formula (6): $\mathrm{<span\; class="notranslate">\; blk</span>}<span\; class="notranslate">\; =</span>\left\{\begin{array}{c}\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 8</span>}<span\; class="notranslate">\; ;</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 4</span>}\\ <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 8</span>}<span\; class="notranslate">\; ;</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; ,</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 8</span>}\end{array}\right.<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 6</span>}<span\; class="notranslate">\; )</span>$ Among them, the values 1/2 and -1/2 represent the ladder effect in the middle of the image block c; first evaluate the block effect intensity of the pixels in the center of each row in the block c, as in formula (7): β=[c(i,6)-3×c(i,5)+3×c(i,4)-c(i,3)]/2 (7) Among them, β represents the intensity of block effect in the middle of block c; then choose a suitable smooth function to replace the step function that produces block effect. Considering that the edge block has more detailed information and the texture structure of the position with block effect is complex, so only for The pixels at the boundary are slightly smoothed, and the smoothing function used is as formula (8): $\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span><span\; class="notranslate">\; (</span>\frac{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; exp</span>}<span\; class="notranslate">\; (</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span>{\mathrm{<span\; class="notranslate">\; \&beta;</span>}}_{\mathrm{<span\; class="notranslate">\; level</span>}}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}$ ${\mathrm{<span\; class="notranslate">\; \&beta;</span>}}_{\mathrm{<span\; class="notranslate">\; level</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; \&times;</span><span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; exp</span>}<span\; class="notranslate">\; (</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 10</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; \&beta;</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 50</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 10</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 8</span>}<span\; class="notranslate">\; )</span>$ Discretize the smooth function to construct a one-dimensional smooth function: de_blk(j)=[f(-49)f(-35)f(-21)f(-7)f(7)f(21)f(35)f(49)] c'(i,j)=c(i,j)+β×[de_blk(j)-blk(j)] (9) Here, de_blk(j) is an array of 1×8 size generated by f(x), and c`(i,j) is the filtered result of row i of c, where c(i,j) refers to c blk(j) refers to the value after any row in c is simulated as blk, so that each row in the image is smoothed to effectively remove the block effect; in order not to make the entire image excessive due to filtering Blurring phenomenon, the boundary between edge blocks and other types of blocks will not be processed; for the boundaries between texture blocks and the boundaries between texture blocks and flat blocks, similar to the case where adjacent blocks are flat blocks are used Filtering method, the difference is that the pixels at the p2 position and _q2 position are not filtered during filtering. The specific filtering process is as follows: Obtain block c by following the composition method of block c in the above; let the 8 pixels in any row in c from left to right be represented as: p ₃ , p ₂ , p ₁ , p ₀ , q ₀ , q ₁ , q ₂ , q ₃ ; then use the formula (3) and (4) to filter any row in c, where p' ₀ and p' ₁ are the results of p ₀ and p ₁ after filtering, and p ₂ and p ₃ do not Processing; when filtering the q-point value, only need to change the p-point pixel in the formula (3) and (4) to q-point pixel in the filter, and change the q-point pixel to p-point pixel in the filter; That is to use the filtering method of p ₀ for q ₀ , the filtering method of p ₁ for q ₁ , the filtering method of p ₂ for q ₂ , and the filtering method of p ₃ for q ₃ ; When adjacent, you only need to rotate a and b by 90 degrees at the same time and then filter according to the left and right adjacent conditions; 1.3 Evaluate the degree of image compression distortion after filtering: Calculate the block effect distortion MSDS value of each filtered image as the evaluation value of the block effect distortion degree of each image; when calculating MSDS, calculate the pixel difference at the border of adjacent image blocks and the average brightness difference of pixels near the border value, the specific method is: The method for forming block c is the same as that described above. Let block c be a new block formed by adjacent blocks a and b, then calculate the MSDS values of a and b as shown in formula (10): $\mathrm{<span\; class="notranslate">\; MSDS</span>}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; 7</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; \{</span><span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span><span\; class="notranslate">\; +</span><span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span><span\; class="notranslate">\; \}</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 10</span>}<span\; class="notranslate">\; )</span>$ Since each image block does not have only one adjacent image block, the MSDS value of each block is the sum of each MSDS value obtained from its adjacent blocks; while the MSDS value of an image is the sum of all blocks The average value of the MSDS value; 1.4 Establishment of sample library based on K-means clustering: Using the obtained evaluation value as a feature, the K-means clustering method is used to divide these images into N categories according to the degree of distortion of the filtered images; for the images in each category, a sample library for super-resolution restoration is established, Combining the HR images corresponding to the LR images in the sample library to train the super-resolution restoration model based on learning, to prepare for the super-resolution restoration of the input image in the online part; In the online section, the specific steps are as follows: 2.1 Filter the input image: The input image is a low-bit compressed image to be processed, and it is filtered by the post-processing filtering method in step 2.2 of the above-mentioned online part to remove part of the block effect in the image; 2.2 Judgment of the block effect distortion category of the input image: Calculate the MSDS value for the filtered image as the block effect distortion evaluation score of the image; calculate the similarity according to the obtained score and the score of the sample image in N categories, and judge the compression distortion category of the input image; determine the input image belongs to The specific method of the compression distortion category is as follows: first, filter the input image, and calculate its MSDS value, and then calculate the Euclidean distance between it and the cluster centers of N categories to obtain the Euclidean distance from the input image The category of the smallest cluster center is the category to which the input image belongs; 2.3 Implementing learning-based super-resolution reconstruction Select the samples in the category closest to the input image as the sample library of the input image, input the filtered input image data into the super-resolution restoration model that has been trained in the offline part, and realize the super-resolution restoration of the image.

Images