CN103475876A

CN103475876A - Learning-based low-bit-rate compression image super-resolution reconstruction method

Info

Publication number: CN103475876A
Application number: CN2013103791541A
Authority: CN
Inventors: 李晓光; 赵寒; 卓力; 魏振利
Original assignee: Beijing University of Technology
Current assignee: Beijing University of Technology
Priority date: 2013-08-27
Filing date: 2013-08-27
Publication date: 2013-12-25
Anticipated expiration: 2033-08-27
Also published as: CN103475876B

Abstract

The invention discloses a learning-based low-bit-rate compression image super-resolution reconstruction method. The learning-based low-bit-rate compression image super-resolution reconstruction method comprises the steps: offline part: firstly compressing low-resolution images by adopting different compression mass parameter values; then filtering the compressed images, taking the quantizing distortion degrees of the filtered compressed images as characteristics, classifying the filtered LR images according to the distortion degrees and establishing a classified sample library, and performing super-resolution model training on all samples respectively. In an online part, the flowing steps are carried out: firstly, filtering input images, then judging a compression distortion class, selecting the sample library and the super-resolution model of the corresponding class according to the judgment result, so as to realize the learning-based super-resolution restoration. Compared with other algorithms, through using the method, the sample library can be self-adaptively regulated so as to match the input LR images according to different distortion degrees, the influence of blocking effect distortion on the super-resolution of the images can be effectively overcome, and compared with the method of directly performing super-resolution restoration on the low-bit-rate distortion images, the images reconstructed by adopting the method has high subjective and objective quality.

Description

A kind of low bit rate compressed image super resolution ratio reconstruction method based on study

Technical field

The present invention relates to image processing method, particularly a kind of low bit rate compressed image super resolution ratio reconstruction method based on study.

Background technology

High-quality image and video, because it has abundanter information and more real visual experience, more and more become a kind of demand of main flow.Also just mean that data volume is larger but the quality of image is higher, this storage to image information, transmission, processing etc. have brought very large burden.Be subject to the impact of the factors such as transmission bandwidth and memory space, the demand that people hang down bit compression to video and picture material is growing.But, when the JPEG compression ratio is higher, the Quality Down of the rear reconstructed image that usually can cause decoding, affect subjective quality and the automatic analysis of information.Therefore, for the low-quality image after high compression, the research Super-Resolution Restoration from Image Sequences, the quality of raising image, have important theory significance and actual application value.

Yet, due to the limitation of method for compressing image itself, while adopting super-resolution method for non-compressed image directly low bit compression image to be carried out to super-resolution rebuilding, there will be serious blocking effect distortion.Therefore, remove blocking effect and become a major issue in low bit compression image super-resolution recuperation.People often adopt the method for reprocessing to reduce blocking effect.Post-processing approach is independent of decoder when filtering, directly decoded image is operated, and has the advantages such as flexible, simple, effective.Post-processing approach based on the figure image intensifying, because it does not rely on any decoded information, can remove blocking effect independently, is easier to realize and obtains broad research.Wherein, airspace filter is a kind of the most basic method, and the method is directly processed the brightness value of pixel in image.Due to the blocking effect in image, because quantization error causes, the variation of picture material can cause the form of expression of the blocking effect of compressed images, therefore the image zones of different is selected adaptively the filter of different level and smooth intensity, and very important meaning is arranged.Typical airspace filter method adopts the method for Texture classification usually, according to human-eye visual characteristic, image is divided into to flat block and non-flat forms piece, then carry out adaptive-filtering, two distinguishing features of the method are, according to self local message, image is divided into to different zones, to different zones, adopt different filtering methods to remove blocking effect.Usual way is to adopt high strength level and smooth at flat site, because there is not high-frequency information in flat site, the high-intensity image that smoothly can not make is excessively fuzzy.For the non-flat forms piece, owing to there being more high-frequency information, application low-intensity smothing filtering can be preserved detail of the high frequency preferably.Yet, in this algorithm, the zone unification that contains limbus information is classified as texture region, does not process separately.Experimental result shows, is subject to the impact of quantizing distortion, and larger impact can be caused to the subjective quality of image in this class zone.Therefore, adopt the filter identical with texture region can not process well the quantizing distortion problem of fringe region.

Summary of the invention

The object of the invention is to, towards the low bit rate compressed image, a kind of a kind of method that adopts improved post processing and filtering algorithm and sample to presort, in conjunction with the image super-resolution method based on study, solve the Super-Resolution problem of the low-resolution image with blocking effect distortion.

The present invention adopts following technological means to realize:

A kind of low bit rate compressed image super resolution ratio reconstruction method based on study, overall flow figure is as shown in Figure 1; Algorithm is divided into off-line part and online part; Its flow chart is respectively as shown in accompanying drawing 2 and accompanying drawing 3; The off-line part, set up the classification samples storehouse according to the compressed image distortion level; At first adopt different compression quality parameters (CQ, Compressed Quality) value to be compressed low resolution (LR, Low Resolution) image; Then compressed image is carried out to the filtering processing, remove the partial block effect in distorted image; Then the quantizing distortion degree of compressed image after filtering is carried out to the K mean cluster as feature, filtered LR image is divided into to multiclass according to its distortion level and sets up the classification samples storehouse, carry out respectively the training of super-resolution model with Different categories of samples; Online part, carry out the kind judging of compression degree to input picture, then realize the Super-Resolution based on study; At first the low bit rate compression LR image of input carried out to the filtering processing, then assessment quantizes distortion level, selects Sample Storehouse and the super-resolution model of respective classes, carries out the Super-Resolution based on study.

Described off-line partly is divided into 4 steps:

(1) m width LR image is used to the JPEG compression method, carry out the compression of n different CQ values and process, the sample image that generates different compression artefacts degree is total to m * n width.The above LR image refers to respect to high-resolution (HR, High Resolution) image, if in a width HR image as unit area, shared pixel count is H, so for this width HR image in unit are shared pixel count be less than or equal to 1/2 * H, generally just regard it as low-resolution image.Here, the compression quality parameter when CQ value is compressed image for employing JPEG, regulate the CQ size and just can obtain the compressed image of different compression ratios;

(2) to the image obtained in (1), adopt post processing and filtering method to carry out the filtering processing, remove the partial block effect in distorted image.

(3) filtered image compression distortion level is assessed.Calculate the blocking effect distortion value of every filtered image, as the distortion level assessed value.The above, post processing and filtering method, at first carry out Texture classification to image, image is divided into to fringe region, and texture region and flat site, then according to the type of image block, select targetedly different filtering methods to carry out filtering to image, remove the blocking effect in image.The above, the blocking effect distortion level assessed value method of image adopts the distortion level assessed value of the MSDS value of image as the image block effect.MSDS(MSDS, Mean Squared Difference of Slopes) be a kind of blocking effect interpretational criteria of all square slope error, the MSDS algorithm is weighed the degree of blocking effect by the severe degree of Description Image block boundary pixel jump.In normal image, it is steadily excessive or excessive continuously that the pixel brightness value on adjacent block border is generally, in normal picture, the saltus step of edge pixel value can not always appear at the boundary of piecemeal, if the pixel value of the boundary of image block always occurs that saltus step just can be thought and occurred blocking effect herein.MSDS is by the pixel value difference at calculating adjacent image block boundary place and the average luminance of pixels difference on close border, carry out the pixel jump situation between reaction block and block boundary, therefore, according to the size of MSDS value, just can differentiate the order of severity of image block effect distortion.

(4) by the assessed value obtained, be feature, the method for employing K mean cluster is divided into the N class according to the distortion level of filtered image by these images; For the image in each class, set up the Sample Storehouse for Super-Resolution, carry out the training of the Super-Resolution model based on study in conjunction with the HR image with in Sample Storehouse, the LR image is corresponding, for the input picture Super-Resolution of online part is prepared;

Described online part is divided into 3 steps:

(1) input a width low bit rate compressed image, to it, adopt the above rear filtering method to carry out the filtering processing;

(2), to filtered image calculation MSDS value, obtain the blocking effect distortion assessment mark of this figure; Calculate similarity according to the mark of the sample image in the mark obtained and N classification, the compression artefacts classification under the judgement input picture.Compression artefacts classification concrete grammar under the judgement input picture is: at first the distortion LR image of input carried out to the filtering processing, obtain its MSDS value, then calculate the Euclidean distance between the cluster centre of it and N classification, the nearest cluster centre place classification with the Euclidean distance with input picture is the classification under input picture;

(3) select sample in a class the most close with the input picture Sample Storehouse as input picture, the data of filtered input picture are input in the Super-Resolution model that off-line partly trained, realize the Super-Resolution of image;

The present invention compared with prior art, has following obvious advantage and beneficial effect:

At first the present invention has proposed a kind of improved spatial domain Image filter arithmetic, has utilized the human eye vision sensitive features, image texture is divided into to flat block, edge block and texture block.Then respectively 3 types of piece zones are adopted the filtering algorithm of varying strength, remove blocking effect.When setting up Sample Storehouse, adopt this improved filtering method to carry out the filtering processing to the LR image of distortion, preferably resolve the blocking effect problem of dtmf distortion DTMF of reconstructed image; In addition, for the LR input picture of different compression artefacts degree, extracted the feature of the MSDS value of image after filtering as every width image, the image that then adopts the K mean cluster will have close distortion level is classified as a class, and set up corresponding Sample Storehouse.When the input LR image to different distortion levels carries out Super-Resolution, at first assess its MSDS value, then establish the Sample Storehouse of a most close class as input LR image, the super-resolution algorithms of finally applying based on study is carried out the HR image that Super-Resolution obtains output.Compare other algorithms, the Sample Storehouse that method of the present invention can be complementary with it for the adjusting of the input LR image adaptive of different distortion levels, and can effectively solve the impact of blocking effect distortion on image super-resolution, with directly the low bit rate distorted image is carried out to Super-Resolution and compares, the image that the inventive method is rebuild has higher subjective and objective quality.

characteristics of the present invention:

1. propose a kind of improved spatial domain filter algorithms, image texture is divided into to flat block, edge block and texture block, to the filtering algorithm of 3 types of piece zone employing varying strengths, filtering has more specific aim, keeps better graphics details information in filtering respectively;

2. utilize the MSDS characteristic value of distorted image, adopt the K mean cluster to be classified to the distortion level of image, but self adaptation regulated the Sample Storehouse be complementary with input picture, refinement simultaneously Sample Storehouse, reduced amount of calculation;

3. for Super-Resolution, Sample Storehouse used is set up after by filtering, efficiently solves the blocking effect problem of dtmf distortion DTMF of image super-resolution rebuilding;

Below in conjunction with example, with reference to accompanying drawing, be elaborated, in the hope of purpose of the present invention, feature and advantage are obtained to more deep understanding.

The accompanying drawing explanation:

Fig. 1, inventive method overview flow chart;

Fig. 2, inventive method off-line part flow chart;

Fig. 3, the online part flow chart of inventive method;

Fig. 4, image texture classification results; (a) original image; (b) edge image; (c) texture image; (d) smooth image;

Fig. 5, blocking effect schematic diagram;

Fig. 6, level and vertical boundary schematic diagram that need to be filtered;

The Super-resolution reconstruction established model of Fig. 7, instance based learning;

Fig. 8, the inventive method and existing methods and results compare:

(a) image (CQ=15) after original high resolution image (b) JPEG compression

(c) the existing method of direct reconstructed image (d) after overcompression

(e) result of the present invention.

Embodiment:

Below in conjunction with Figure of description, embodiment of the present invention is described in detail:

Algorithm of the present invention is divided into off-line and online two parts.The off-line part, set up the classification samples storehouse according to the compressed image distortion level; At first adopt different compression quality parameters (CQ) value to be compressed low resolution (LR) image; Then compressed image is carried out to the filtering reprocessing, remove obvious blocking effect in distorted image; Then the quantizing distortion degree of compressed image after filtering is carried out to the K mean cluster as feature, filtered LR image is divided into to multiclass according to its distortion level and sets up the classification samples storehouse, carry out respectively the training of super-resolution model with Different categories of samples; Online part, carry out kind judging to input picture and complete the Super-Resolution based on study; At first the low bit rate compression LR image of input carried out to the filtering processing, then assessment quantizes distortion level, selects Sample Storehouse and the super-resolution model of respective classes, carries out the Super-Resolution based on study.

(1) off-line part

(1) different CQ values are compressed the LR image

Select a HR image pattern storehouse (non-compressed image), in Sample Storehouse, the concrete resolution of image is 140 * 160 pixels, select 1000 width HR images wherein, then these images are directly carried out to down-sampling, the LR image that generates 1000 width low resolution forms Sample Storehouse.Select m=300 width image from this LR Sample Storehouse, the image that concrete resolution is 70 * 80 pixels.Adopt the JPEG compress mode, this m width LR image is carried out to the compression of CQ=5～20, common property is given birth to 4800 width compression artefacts images like this.It is when CQ is less than 5 that the CQ value is selected this interval reason, the image fault degree can't be carried out effective Super-Resolution, and not CQ scope commonly used, CQ is greater than at 20 o'clock, the subjective quality of image is basically identical, human eye can't be discovered image fault, and the size of compressed bit stream exceeds the low bit rate scope.

(2) compressed image is carried out to the filtering processing

To the image obtained in (1), adopt post processing and filtering method to carry out the filtering processing, the above, post processing and filtering method, at first carry out Texture classification to image, and image is divided into to fringe region, texture region and flat site; Then according to the type of image block, select pointedly different filtering methods to carry out filtering to image.Block-eliminating effect filtering, according to 8 * 8 big or small piecemeals, then is carried out respectively to according to image block type in different ways in 3 kinds of zones.

At first, image is carried out to Texture classification, concrete grammar is:

Select the gradient information of Sobel operator extraction image slices vegetarian refreshments, the direction template of employing is as formula (1):

For piece image, at first with these 4 direction templates, carry out convolution with this image respectively, obtain the Grad on 4 directions of image, for each point in image, because this point has 4 Grad, so we select in 4 Grad a maximum Grad as this point, are denoted as g _max, the row of i correspondence image, the row of j correspondence image; Then adopt 3 kinds of Component Models that image is divided into to edge, texture and flat site; Specifically suddenly: 1, calculated threshold, TH ₁=0.12 * g _maxand TH ₂=0.06 * g _max, g wherein _maxit is maximum Grad in the Grad of entire image of 4 directions of gained; TH ₁, TH ₂for the height threshold value, as the foundation of differentiating the pixel Texture classification, 0.12 and 0.06 is empirical value; 2, adopt formula (2) that image pixel is divided into to edge, texture or flat site;

\{\begin{matrix} edge pixel, & if G (i, j) > T H_{1}, \\ smooth pixel, & if G (i, j) < T H_{2} . \end{matrix} - - - (2)

Wherein, G (i, j) is the Grad of each pixel in image, and the Grad when pixel (i, j) position is greater than TH ₁the time, this pixel just is divided into marginal zone (edgepixel); Grad when pixel (i, j) position is less than TH ₂the time this pixel just be divided into flat region (smoothpixel); If the Grad of pixel (i, j) position is at TH ₁and TH ₂between the time this pixel just be divided into texture area.Take the Lena image as example, and division result as shown in Figure 4.In Fig. 4 from left to right the first width image be original image, be then edge image, texture image and smooth image.These images have passed through binary conversion treatment, and the obviously continuous lines that wherein in Fig. 4 (b), black pixel point forms have represented fringe region.In Fig. 4 (c), the black pixel point of distribution at random has represented texture region.Flat site in Fig. 4 (d) black region in blocks mean.

The above, the concrete grammar of selecting targetedly different filtering methods to carry out filtering to image is:

After image is carried out to the texture division, according to the type of image block, with different filtering modes, image block is carried out to filtering pointedly, concrete grammar is as follows:

Therefore because the blocking effect in flat block is the most easily discovered by human eye, need high-intensity filter to carry out this part blocking effect of filtering; Remove the blocking effect of flat block with the filter of filtering strength maximum, only selecting adjacent block when filtering is all the border of flat block; Suppose a, b is respectively the flat block of two 8 * 8 pixel sizes, and a, the position relationship of b is that left and right is adjacent, as shown in Figure 5, a left side 4 row pixels of the right side of a 4 row and b are formed to a new image block c, the blocking effect in the middle of a, b will intactly be retained in c, therefore to a, between b, the filtering of blocking effect is that c is carried out to filtering operation actually; If in c, any a line 8 pixels from left to right are expressed as successively: p ₃, p ₂, p ₁, p ₀, q ₀, q ₁, q ₂, q ₃; As shown in figure as left as Fig. 6, to any a line in c, adopt formula (3), (4), (5) to carry out filtering so:

p' ₀=(p ₂+2p ₁+2p ₀+2q ₀+q ₁+4)/8 (3)

p’ ₁=(p ₂+p ₁+p ₀+q ₀+2)/4 (4)

p’ ₂=(2p ₃+3p ₂+p ₁+p ₀+q ₀+4)/8 (5)

P ' wherein ₀, p ' ₁and p ' ₂p ₀, p ₁, p ₂result after filtering, p ₃point is not processed; When the q point value is carried out to filtering, only need in filter, change the p point pixel of relevant position in formula (3), (4), (5) into q point pixel, it is just passable that q point pixel changes p point pixel into; Be q ₀adopt p ₀filtering mode, q ₁adopt p ₁filtering mode, q ₂adopt p ₂filtering mode, q ₃adopt p ₃filtering mode carry out filtering; Work as a, when the position of b is neighbouring, only need to be by a, b simultaneously 90-degree rotation then according to left and right adjacent situation carry out filtering and get final product, as shown in figure as right as Fig. 6;

For adjacent block, be all the situation of edge block, the building form of continuing to use aforementioned middle c piece obtains the c piece, because the pixel intensity in c centre position has obvious saltus step, this blocking effect of c is simulated with a two-dimentional step function blk, as shown in formula (6):

blk = \{\begin{matrix} 1 / 2, i = 1, . . ., 8; j = 1, . . ., 4 \\ - 1 / 2, i = 1, . . ., 8; j = 5, . . ., 8 \end{matrix} - - - (6)

Wherein, numerical value 1/2 and-1/2 has represented the middle alias of image block c; At first the block effect intensity of the every a line centre pixel in the c piece is assessed, as formula (7):

β=[c(i,6)-3×c(i,5)+3×c(i,4)-c(i,3)]/2 (7)

Wherein β represents the block effect intensity in c piece centre position; Then select an applicable smooth function to replace the step function that produces blocking effect, the detailed information of considering edge block is more, therefore only and have the position texture structure complexity of blocking effect, the boundary pixel is carried out slight smoothly, the smooth function of employing is as formula (8):

f (x) = (\frac{- 1}{1 + \exp (- (x) / β_{level})}) + 1 / 2

β_{level} = 5 \times (\frac{1}{1 + \exp (- (10 \times β - 50) / 10)}) + 5 - - - (8)

β wherein _levelaccording to block effect intensity, β obtains, and can control adaptively the shape of smooth function, works as β _levelvalue more hour smooth function approaches original step function; Smooth function can change border pixel values in Min. ground, reduces as far as possible slightly the brightness saltus step on image block border, can not cause the fuzzy of image; When concrete the processing, this smooth function is carried out to discretization, constructs the one dimension smooth function:

de_blk(j)=[f(-49)f(-35)f(-21)f(-7)f(7)f(21)f(35)f(49)]

c'(i,j)=c(i,j)+β×[de_blk(j)-blk(j)] (9)

Here, de_blk (j) is the array by 1 * 8 size of f (x) generation, c` (i, j) be the capable result after filtered of the i of c, here c (i, j) refer to the capable all elements of i of c, blk(j) refer in c the value after a line arbitrarily is modeled into blk, each in this sampled images is passed through after level and smooth can effectively remove blocking effect; In order not make the entire image blooming undue because filtering causes, the border between edge blocks and other types piece is not processed; For the border between the border between texture block and texture block and flat block, employing and adjacent block are all the similar filtering method of the situation of flat block, distinguish when being filtering p ₂position and q ₂the pixel of position does not carry out the filtering processing;

(3) filtered image compression distortion level is assessed

Calculate blocking effect distortion (MSDS, the Mean Squared Difference of Slopes) value of image after every width filtering, as the blocking effect distortion level assessed value of every width image; MSDS is a kind of blocking effect interpretational criteria of all square slope error, and the MSDS algorithm is weighed the degree of blocking effect by the severe degree of Description Image block boundary pixel jump; In normal image, the pixel brightness value on adjacent block border is generally steadily excessive and excessively continuous, and in image, the saltus step of edge pixel value can not always appear at the boundary of piecemeal; Therefore, according to the size of MSDS value, just can differentiate the order of severity of image block effect distortion; When calculating MSDS, be to calculate the pixel value difference at adjacent image block boundary place and the average luminance of pixels difference on close border, concrete grammar is:

The method of the same described composition c piece, establishing piece c is the new piece that adjacent block a and b form, the MSDS value of calculating a and b as formula as shown in (10):

MSDS = Σ_{i = 0}^{7} [c (i, 4) - c (i, 3)] - 1 / 2 {[c (i, 3) - c (i, 2)] + [c (i, 5) - c (i, 4)]} - - - (10)

Because each image block does not only have an image block be adjacent, thus the MSDS value of each piece be adjacent each MSDS value that piece tries to achieve and; The MSDS value of piece image is the mean value of the MSDS value of all;

(4) Sample Storehouse based on the K mean cluster is set up

The low bit rate image is subject to the impact of quantization error during due to compressed encoding, and reconstructed image there will be serious distortion, produces new picture structure " blocking effect ".Therefore, while adopting the Super-Resolution algorithm to be restored this class distorted image, if still adopt the training result in original image storehouse, can have a strong impact on the quality of reconstructed image.When the input picture to distortion carries out 2 times of amplifications, adopt the Super-Resolution reconstructed results obtained under original database and training pattern can produce serious distortion.Therefore need to re-establish Sample Storehouse for distorted image.After determining that a CQ value is compressed all LR images, can set up a distorted image Sample Storehouse.Then utilize this Sample Storehouse again to train the super-resolution model, with relation between the combination coefficient of the LR combination coefficient of setting up distorted image and HR image, for recovering the HR image.Yet, in experiment, when the compression ratio of LR is higher, the reconstructed image distortion situation caused is very serious, thereby can't obtain restoration result preferably.Therefore, adopt the post processing and filtering algorithm in the present invention, at first the distortion sample is removed to blocking effect, then, set up LR Sample Storehouse after filtering, just can effectively improve the Super-Resolution effect of distorted image.On the other hand, the distortion level of the compressed image of giving birth to due to different compression ratios has obvious difference, blocking effect distortion level in the reconstructed image that caused thus also has very big-difference, therefore, be necessary to carry out the foundation of Sample Storehouse and the training of super-resolution model for the image resolution of different compression ratios.Yet, for the compressed image of a secondary input, can't know its compressed information, therefore, just need to be predicted the compression degree of input picture

MSDS value according to obtaining, be divided into the N class by filtered sample by the K mean cluster.Here it is to be noted, owing to partly having adopted totally 16 mass parameters that the CQ value is 5 to 20 at off-line, image is compressed, therefore, the span of N should be 2 to 16 in principle, more results that finally obtain of classifying can be better, but corresponding arithmetic speed can be slower, therefore, consider two factors of arithmetic speed and accuracy, selecting N=3 when checking draws cluster by experiment is most suitable classification.The sample image comprised in each class is used for training the corresponding Super-Resolution model based on study.

The K mean cluster is a kind of unsupervised learning algorithm, and its core concept is by iteration, data sample to be divided in different bunches, makes the minimization of object function, thus make to generate bunch compact as much as possible and independent.Given sample set and integer K, the K means clustering algorithm is selected K initial cluster center at first at random, then by unchecked data object according to them the Euclidean distance with each clustering cluster central point, be assigned to the distance minimum bunch in.Then using the mean value of all samples in each bunch as new clustering cluster central point, i.e. barycenter.Repeat above step, by not stopping iteration until target function convergence.Usually the target function adopted is the square error criterion function:

E = \min Σ_{i = 1}^{K} \underset{x_{j} &Element; C_{j}}{Σ} {| | x_{j} - c_{i} | |}^{2} - - - (11)

Wherein, x _jfor sample data, i.e. the MSDS value of LR image, C _imean bunch C _jbarycenter, E mean all samples to the Euclidean distance of barycenter and.

Using from 5 to 20 pairs of 300 width images of CQ value compress and carried out filtering processing common property life 4800 width images, calculate the MSDS value of these images as 4800 characteristic values, these features at first arbitrarily are divided into to 3 classes, then carry out the K mean cluster, obtain 3 cluster centres after the calculating that iterates, the approximate sample that each cluster centre is comprised saves as a Sample Storehouse.It is pointed out that because training process is training LR image and the relation between corresponding HR image with it, so, in final Sample Storehouse, also should comprise the corresponding HR image of every width LR image.In 3 Sample Storehouses, select respectively 300 width images to be trained, set up the Super-Resolution model based on study.

(2) online part

(1) input picture is carried out to the filtering processing.

Input picture is a pending low bit compression image, adopts improved rear filtering mode in the step (2) of online part as mentioned above to carry out the filtering processing to it, removes in image significantly blocking effect.

(2) the blocking effect distortion kind judging of input picture

At first, according to the method for computed image MSDS value in off-line part steps (3) as mentioned above, the MSDS value of the input picture after calculation of filtered.Then, calculate the Euclidean distance between it and 3 cluster centres, by calculate the gained Euclidean distance hour corresponding cluster be defined as the distortion classification under input picture.

(3) realize the super-resolution rebuilding based on study

After the input picture classification is selected, select corresponding sample in such, the super-resolution model of utilization based on study carries out the Super-Resolution of image.

Super-resolution rebuilding based on study obtains the relation between high-resolution and low-resolution image by learning algorithm, instructs the reconstruction of high-definition picture.Concentrate and obtain the foundation of priori as Super-Resolution from a large amount of training samples, training sample is all the image that comprises same category information with input picture.Here, with the algorithm of representative instance based learning, the super-resolution rebuilding process is described, as shown in Figure 7: at first to input picture carry out interpolation amplification to and target high-definition picture in the same size; The image of inputting is extracted to the LR image block, and LR piece and the HR piece of the training sample database of using off-line partly to obtain, use LR piece and the relation between the HR piece (being the model that off-line training partly obtains) learnt to carry out the high-frequency information prediction of each piece; Then high-frequency information is added in the result of interpolation amplification, obtains output image.In implementation procedure, for convenience of describing, establishing input picture is LR-A.At first, after the filtered processing of LR-A, obtain image LR-AF, then LR-AF is carried out to blocking effect distortion judgement, judge it and belong to a certain class in 3 classifications; Then LR-AF being carried out to interpolation amplification obtains result and is made as HR-A; Then LR-AF is carried out to piece and extract the input matrix that forms the super-resolution model; Because the off-line part has utilized the sample training in each cluster to get well the relational model between LR sample and HR sample, so input matrix is input in the model trained and just can obtains the high frequency information of forecasting; Finally the high-frequency information obtained is added on HR-A and has just obtained last high-resolution result images.

To be the inventive method and existing method carry out super-resolution rebuilding, the comparison of acquired results to the compressed image of CQ=15 to Fig. 8.

Claims

One kind based on study low bit rate compressed image Super-Resolution method; Algorithm is divided into off-line part and online part; Described off-line partly comprises 4 steps; At first will for the image pattern of training, adopt different compression ratios to generate the image with different compression qualities, then carry out filtering and process the partial block effect of removing in the compression artefacts image, then according to the compressed image distortion level, set up classification samples storehouse the forecast model of training based in the study Super-Resolution, for the input picture Super-Resolution of online part is prepared;

Described online part comprises 3 steps; Realize the deblocking effect of input low bit rate compressed image, kind judging and final Super-Resolution;

Described off-line part, concrete steps are as follows:

1.1 different CQ values are compressed the LR image:

M width LR image is used to the JPEG compression method, carry out the compression of n different CQ values and process, the sample image that generates different compression artefacts degree is total to m * n width; Here the LR image refers to the low-resolution image of uncompressed, the compression quality parameter when CQ value is compressed image for employing JPEG;

1.2 compressed image is carried out to the filtering processing:

To the image obtained in 1.1, adopt post processing and filtering method to carry out the filtering processing, remove the partial block effect in distorted image; The above post processing and filtering method, at first carry out Texture classification to image, and image is divided into to fringe region, texture region and flat site; Then according to the type of image block, select pointedly different filtering methods to carry out filtering to image; The above concrete grammar that figure is carried out to Texture classification is:

Select the gradient information of Sobel operator extraction image slices vegetarian refreshments, the direction template of employing is as formula (1):

For piece image, at first with these 4 direction templates, carry out convolution with this image respectively, obtain the Grad on 4 directions of image, for each point in image, because this point has 4 Grad, select a Grad as this point maximum in 4 Grad, be denoted as g _max; Then adopt 3 kinds of Component Models that image is divided into to edge, texture and flat site; Specifically suddenly: 1, calculated threshold, TH ₁=0.12 * g _maxand TH ₂=0.06 * g _max, g wherein _maxit is maximum Grad in the Grad of entire image of 4 directions of gained; TH ₁, TH ₂for the height threshold value, as the foundation of differentiating the pixel Texture classification, 0.12 and 0.06 is empirical value; 2, adopt formula (2) that image pixel is divided into to edge, texture or flat site;

$\{\begin{matrix} edge pixel, & if G (i, j) > T H_{1}, \\ smooth pixel, & if G (i, j) < T H_{2} . \end{matrix} - - - (2)$

Wherein, G (i, j) is the Grad of each pixel in image, and the Grad when pixel (i, j) position is greater than TH ₁the time, this pixel just is divided into marginal zone edgepixel; Grad when pixel (i, j) position is less than TH ₂the time this pixel just be divided into flat region smoothpixel; If the Grad of pixel (i, j) position is at TH ₁and TH ₂between the time this pixel just be divided into texture area;

The above, the concrete grammar of selecting targetedly different filtering methods to carry out filtering to image is:

After image is carried out to the texture division, according to the type of image block, with different filtering modes, image block is carried out to filtering pointedly, concrete grammar is as follows:

Remove the blocking effect of flat block with the filter of filtering strength maximum, only selecting adjacent block when filtering is all the border of flat block; Suppose a, b is respectively the flat block of two 8 * 8 pixel sizes, and a, the position relationship of b is that left and right is adjacent, the a left side 4 row pixels of the right side of a 4 row and b are formed to a new image block c, the blocking effect in the middle of a, b will intactly be retained in c, and therefore to a, between b, the filtering of blocking effect is that c is carried out to filtering operation actually; If in c, any a line 8 pixels from left to right are expressed as successively: p ₃, p ₂, p ₁, p ₀, q ₀, q ₁, q ₂, q ₃; To any a line in c, adopt formula (3), (4), (5) to carry out filtering so:

p' ₀=(p ₂+2p ₁+2p ₀+2q ₀+q ₁+4)/8 (3)

p’ ₁=(p ₂+p ₁+p ₀+q ₀+2)/4 (4)

p' ₂=(2p ₃+3p ₂+p ₁+p ₀+q ₀+4)/8 (5)

P ' wherein ₀, p ' ₁and p ' ₂p ₀, p ₁, p ₂result after filtering, p ₃point is not processed; When the q point value is carried out to filtering, only need in filter, change the p point pixel of relevant position in formula (3), (4), (5) into q point pixel, it is just passable that q point pixel changes p point pixel into; Be q ₀adopt p ₀filtering mode, q ₁adopt p ₁filtering mode, q ₂adopt p ₂filtering mode, q ₃adopt p ₃filtering mode carry out filtering; Work as a, when the position of b is neighbouring, only need to be by a, b simultaneously 90-degree rotation then according to left and right adjacent situation carry out filtering and get final product;

For adjacent block, be all the situation of edge block, the building form of continuing to use aforementioned middle c piece obtains the c piece, because the pixel intensity in c centre position has obvious saltus step, this blocking effect of c is simulated with a two-dimentional step function blk, as shown in formula (6):

$blk = \{\begin{matrix} 1 / 2, i = 1, . . ., 8; j = 1, . . ., 4 \\ - 1 / 2, i = 1, . . ., 8; j = 5, . . ., 8 \end{matrix} - - - (6)$

Wherein, numerical value 1/2 and-1/2 has represented the middle alias of image block c; At first the block effect intensity of the every a line centre pixel in the c piece is assessed, as formula (7):

β=[c(i,6)-3×c(i,5)+3×c(i,4)-c(i,3)]/2 (7)

Wherein β represents the block effect intensity in c piece centre position; Then select an applicable smooth function to replace the step function that produces blocking effect, the detailed information of considering edge block is more, therefore only and have the position texture structure complexity of blocking effect, the boundary pixel is carried out slight smoothly, the smooth function of employing is as formula (8):

$f (x) = (\frac{- 1}{1 + \exp (- (x) / β_{level})}) + 1 / 2$

$β_{level} = 5 \times (\frac{1}{1 + \exp (- (10 \times β - 50) / 10)}) + 5 - - - (8)$

This smooth function is carried out to discretization, constructs the one dimension smooth function:

de_blk(j)=[f(-49)f(-35)f(-21)f(-7)f(7)f(21)f(35)f(49)]

c'(i,j)=c(i,j)+β×[de_blk(j)-blk(j)] (9)

Here, de_blk (j) is the array by 1 * 8 size of f (x) generation, c` (i, j) be the capable result after filtered of the i of c, here c (i, j) refer to the capable all elements of i of c, blk(j) refer in c the value after a line arbitrarily is modeled into blk, each in this sampled images was passed through after level and smooth and was effectively removed blocking effect; In order not make the entire image blooming undue because filtering causes, the border between edge blocks and other types piece is not processed; For the border between the border between texture block and texture block and flat block, employing and adjacent block are all the similar filtering method of the situation of flat block, distinguish while being filtering p2 position and q ₂the pixel of position does not carry out the filtering processing, and concrete filtering is as follows:

The building form of continuing to use aforementioned middle c piece obtains the c piece; If in c, any a line 8 pixels from left to right are expressed as successively: p ₃, p ₂, p ₁, p ₀, q ₀, q ₁, q ₂, q ₃; So in c, filtering, wherein p ' are carried out in a line employing formula (3), (4) arbitrarily ₀, p ' ₁p ₀, p ₁result after filtering, p ₂, p ₃point is not processed; When the q point value is carried out to filtering, only need in filter, change the p point pixel of relevant position in formula (3), (4) into q point pixel, it is just passable that q point pixel changes p point pixel into; Be q ₀adopt p ₀filtering mode, q ₁adopt p ₁filtering mode, q ₂adopt p ₂filtering mode, q ₃adopt p ₃filtering mode carry out filtering; Work as a, when the position of b is neighbouring, only need to be by a, b simultaneously 90-degree rotation then according to left and right adjacent situation carry out filtering and get final product;

1.3 filtered image compression distortion level is assessed:

Calculate the blocking effect distortion MSDS value of image after every width filtering, as the blocking effect distortion level assessed value of every width image; When calculating MSDS, be to calculate the pixel value difference at adjacent image block boundary place and the average luminance of pixels difference on close border, concrete grammar is:

The method of the same described composition c piece, establishing piece c is the new piece that adjacent block a and b form, the MSDS value of calculating a and b as formula as shown in (10):

$MSDS = Σ_{i = 0}^{7} [c (i, 4) - c (i, 3)] - 1 / 2 {[c (i, 3) - c (i, 2)] + [c (i, 5) - c (i, 4)]} - - - (10)$

Because each image block does not only have an image block be adjacent, thus the MSDS value of each piece be adjacent each MSDS value that piece tries to achieve and; The MSDS value of piece image is the mean value of the MSDS value of all;

1.4 the Sample Storehouse based on the K mean cluster is set up:

By the assessed value obtained, be feature, the method for employing K mean cluster is divided into the N class according to the distortion level of filtered image by these images; For the image in each class, set up the Sample Storehouse for Super-Resolution, carry out the training of the Super-Resolution model based on study in conjunction with the HR image with in Sample Storehouse, the LR image is corresponding, for the input picture Super-Resolution of online part is prepared;

Described online part, concrete steps are as follows:

2.1 input picture is carried out to the filtering processing:

Input picture is a pending low bit compression image, adopts the post processing and filtering mode in the step 2.2 of online part as mentioned above to carry out the filtering processing to it, removes the partial block effect in image;

2.2 the blocking effect distortion kind judging of input picture:

To filtered image calculation MSDS value, as the blocking effect distortion assessment mark of this image; Calculate similarity according to the mark of the sample image in the mark obtained and N classification, the compression artefacts classification under the judgement input picture; Compression artefacts classification concrete grammar under the judgement input picture is: at first, image to input carries out the filtering processing, and calculating obtains its MSDS value, then calculate the Euclidean distance between the cluster centre of it and N classification, with the cluster centre place classification of the Euclidean distance minimum with input picture, it is the classification under input picture;

2.3 realize the super-resolution rebuilding based on study

Select sample in a class the most close with the input picture Sample Storehouse as input picture, the data of filtered input picture are input in the Super-Resolution model that the off-line part trained, realize the Super-Resolution of image.