CN110298789B

CN110298789B - Single image super-resolution reconstruction method based on TV prior

Info

Publication number: CN110298789B
Application number: CN201910475134.1A
Authority: CN
Inventors: 傅博; 李艺; 傅世林; 张湘怡; 王相海
Original assignee: Liaoning Normal University
Current assignee: Dalian Houren Technology Co ltd
Priority date: 2019-06-03
Filing date: 2019-06-03
Publication date: 2023-04-14
Anticipated expiration: 2039-06-03
Also published as: CN110298789A

Abstract

The invention discloses a super-resolution reconstruction method for a single image based on TV prior, which comprises the steps of firstly utilizing a bicubic interpolation algorithm oms3 to perform up-sampling pretreatment on an image with resolution to be improved, extracting TV prior information from the pretreated image according to 28 TV direction templates, and finally introducing the TV prior information into a non-local regression frame, so that texture and edge information of a super-resolution image are better reserved, the phenomenon that the edge information of the image generated by the traditional interpolation algorithm is insufficient is overcome, the super-resolution reconstructed image is obtained, and the super-resolution effect is greatly improved.

Description

Single image super-resolution reconstruction method based on TV prior

Technical Field

The invention relates to a single image super-resolution reconstruction method, in particular to a single image super-resolution reconstruction method based on TV prior, which can enhance the image edge.

Background

At present, image super-resolution reconstruction is a quite active research field, and provides a solution to the low resolution limit from mobile phone imaging, remote sensing imaging and medical imaging. Low quality images are generally caused by low cost sensor capture, narrow band transmission and poor light interference, and when low quality images are widely used for high definition display, the visual analysis and recognition tasks tend to be displayed in higher resolution versions, so that it is necessary to estimate the high resolution image from the content of the low resolution image, i.e. image super-resolution reconstruction.

At present, people only can excavate prior knowledge as much as possible, establish a model, observe the mapping relation from a low-resolution image to a high-resolution image, and design an ultra-resolution reconstruction algorithm on the basis of the generated model. The super-resolution reconstruction method uses a simple linear function to perform up-sampling on an image, such as bilinear interpolation, bicubic interpolation, s-spline and the like, when a larger amplification factor is needed, the reconstruction-based super-resolution reconstruction algorithm cannot achieve a good effect, and has some defects in the aspects of image edge preservation and the like. There are also methods that add gradient prior information to the local image structure, which usually introduce more complex structures (patches, etc.) and auxiliary judgment information (gradients, edge detection information, etc.), but most methods fix a set of invariant model parameters for different image data contents. Therefore, when these methods process natural images having different texture contents, an over-smoothing or burring phenomenon tends to occur.

Total Variation (Total Variation) models were first used to denoise noise contaminated images, and Chan et al, 2002, promoted the TV model to image inpainting. The TV model can effectively decompose structural information and textures in an image without particularly specifying whether the textures are regular or symmetrical. In other words, the method is generic and arbitrary, and is applicable to non-uniform or anisotropic textures. However, there is no report about the application of the TV model to super-resolution reconstruction of single images.

Disclosure of Invention

The invention aims to solve the technical problems in the prior art and provides a single image super-resolution reconstruction method based on TV prior, which can enhance the image edge.

The technical solution of the invention is as follows: a single image super-resolution reconstruction method based on TV prior is sequentially carried out according to the following steps:

step 011: acquiring an Image with resolution to be improved, and recording the Image as Image _ ori;

step 012: performing up-sampling on the Image Image _ ori by a bicubic interpolation oms3 algorithm, and recording an obtained Image as Image _ oms3;

step 021: setting the size of the Image _ oms3 as M × N, moving the Image _ oms3 point by point from the upper left corner of the Image to obtain M × N blocks; starting from the first block, 28 TV direction templates with the size of 3 × 3 are established by taking the pixel to be repaired as the center, and are marked as o (n), n =1,2,3 \8230, 28, wherein n is the serial number of the direction template, 9 pixels in the template are respectively marked as { d1, d2, d3, d4, d5, d6, d7, d8, d9} from top to bottom and from left to right, and then 28 direction templates are respectively marked as: o1{ d1, d5, d6}, o2{ d4, d5, d3},3 medium d3, d5, d6}, o4{ d4, d5, d1}

o5{d6,d5,d7},o6{d4,d5,d9},o7{d6,d5,d9},o8{d4,d5,d7},o9{d2,d5,d7}，o10{d2,d5,d9},o11{d2,d5,d3},o12{d2,d5,d1},o13{d8,d5,d1},o14{d8,d5,d3}o15{d8,d5,d9},o16{d8,d5,d7},o17{d4,d5,d2},o18{d6,d5,d2},o19{d4,d5,d8}o20{d6,d5,d8},o21{d1,d5,d3},o22{d1,d5,d7},o23{d3,d5,d9},o24{d7,d5,d9}o25{d4,d5,d6}，o26{d2,d5,d8},o27{d1,d5,d9},o28{d7,d5,d3}；

Step 022: respectively calculating the mean value of three pixels in each direction template, and recording as s (n);

step 023: calculating the difference between all s (n) and the central pixel value d5, taking the absolute value of the difference as t (n), and selecting the template with the sequence number n corresponding to the minimum value in t (n) as the optimal direction template;

step 031: starting from a first block, establishing a target block with the scale of 3 x 3 by taking a pixel to be repaired as a center, searching 25 search blocks p (m) around the target block, wherein m =1,2,3 \8230 \ 823025, calculating Euclidean distances between the 25 search blocks and the target block to be recorded as D (m), and obtaining the weight of each search block to be recorded as w (m) according to the similarity between the target block and the search blocks;

step 032: in 25 search blocks, calculating the median value of three pixel values in the direction as M (M) according to the optimal direction template;

step 033: and correspondingly multiplying the obtained w (M) and M (M) and then summing to obtain a value which is the pixel value after the current pixel point is repaired.

Step 034: and circularly traversing the whole Image _ images 3, recording the repaired Image as Image _ result and storing the Image _ result.

According to the method, firstly, images to be improved in resolution ratio are subjected to up-sampling pretreatment by utilizing a bicubic interpolation algorithm omoms3, TV prior information is extracted from the pretreated images according to 28 TV direction templates, and finally the TV prior information is introduced into a non-local regression frame, so that texture and edge information of super-resolution images are well reserved, the phenomenon that image edge information generated by a traditional interpolation algorithm is insufficient is overcome, super-resolution reconstructed images are obtained, and the super-resolution effect is greatly improved.

Drawings

Fig. 1 is a schematic diagram of performing bicubic upsampling on an original image according to an embodiment of the present invention.

FIG. 2 is a schematic diagram of an embodiment of the present invention introducing TV prior information into a non-local regression framework.

FIG. 3 is an image to be improved in resolution according to an embodiment of the present invention.

Fig. 4 is an image of the bicubic algorithm after resolution enhancement.

Fig. 5 is an image with improved resolution according to an embodiment of the invention.

Detailed Description

The invention discloses a single image super-resolution reconstruction method based on TV prior, which is sequentially carried out according to the following steps:

a step 011: acquiring an Image with resolution to be improved, and recording the Image as Image _ ori;

step 012: as shown in fig. 1, the Image _ ori is up-sampled by a bicubic interpolation oms3 algorithm, and the obtained Image is marked as Image _ oms3;

step 021: setting the size of the Image _ oms3 as M × N, moving the Image _ oms3 point by point from the upper left corner of the Image to obtain M × N blocks; starting from the first block, as shown in fig. 2, 28 TV direction templates with a scale of 3 × 3 are established with the pixel to be repaired as the center, and are denoted as o (n), n =1,2,3 \8230 \ 823028, where n is the serial number of the direction template, 9 pixels in the template are denoted as { d1, d2, d3, d4, d5, d6, d7, d8, d9} from top to bottom and from left to right, respectively, and then 28 direction templates are denoted as: o1{ d1, d5, d6}, o2{ d4, d5, d3},3 retaining ports d3, d5, d6}, o4{ d4, d5, d1}

step 031: starting from a first block, establishing a target block p0 with the scale of 3-3 by taking a pixel to be repaired as the center, searching 25 search blocks p (m) around the target block p0, wherein m =1,2,3 \8230, 25, calculating the Euclidean distance between the 25 search blocks and the target block to be recorded as D (m), and obtaining the weight of each search block to be recorded as w (m) according to the similarity between the target block and the search block;

for the target block p0, the calculation formula of the weight of the search block p1 is:

wherein, the formula of x (p 0, p 1) is: />

In the formula:

is the square of the weighted euclidean distance of block p0 to block p1, device for selecting or keeping>

Used for controlling the corrosion degree of the filter;

Step 034: and circularly traversing the whole Image _ elements 3, and recording the repaired Image as Image _ result and storing the Image.

Fig. 4 is an image of the bicubic algorithm after resolution enhancement.

FIG. 5 is an image restored by a single image super-resolution reconstruction method based on TV prior in an embodiment of the present invention.

The following table shows PSNR values after different image reconstructions by the embodiment of the present invention (NT _ oms 3) and bicubic algorithm.

/>

Claims

1. A single image super-resolution reconstruction method based on TV prior is characterized by comprising the following steps in sequence:

step 021: setting the size of the Image _ oms3 as M x N, moving the Image _ oms3 point by point from the upper left corner of the Image to obtain M x N blocks; starting from the first block, 28 TV direction templates with the size of 3 × 3 are established by taking the pixel to be repaired as the center, and are marked as o (n), n =1,2,3 \8230, 28, wherein n is the serial number of the direction template, 9 pixels in the template are respectively marked as { d1, d2, d3, d4, d5, d6, d7, d8, d9} from top to bottom and from left to right, and then 28 direction templates are respectively marked as: o1{ d1, d5, d6}, o2{ d4, d5, d3},3 retaining ports d3, d5, d6}, o4{ d4, d5, d1}

step 032: in 25 search blocks, calculating the median of three pixel values in the direction as M (M) according to the optimal direction template;

step 033: correspondingly multiplying the obtained w (M) and M (M) and summing to obtain a value which is the pixel value of the current pixel point after restoration;