CN105976318A - Image super-resolution reconstruction method - Google Patents

Abstract

The invention discloses an image super-resolution reconstruction method and belongs to the image processing technical field. The method includes the following steps that: a low-resolution image Y in a training image database is extracted; Bi-cubic interpolation enlargement is performed on the obtained low-resolution image, and the low-resolution image is enlarged to a required size; a convolutional neural network containing dynamic convolutional layers is designed; the low-resolution image Y is inputted into the pre-trained convolutional neural network B, so that filters SH1 and filters SV1 can be obtained; the low-resolution image Y, the filters SV1 and the filters SH1 are inputted into the pre-trained convolutional neural network B containing the dynamic convolutional layers; and a high-resolution image X is reconstructed. As indicated by an experimental result, the method provided by the invention have more significant effects in visual effect and in objective evaluation criterion compared with other three excellent algorithms, and has an excellent super resolution reconstruction performance.

Description

A kind of image super-resolution rebuilding method

Technical field

The present invention relates to technical field of image processing, be specifically related to a kind of image super-resolution rebuilding method.

Background technology

Image Super-resolution Reconstruction refers to, by the way of software algorithm, existing low-resolution image is converted into high-resolution Rate image.The fields such as it prints at image, video monitoring, Medical Image Processing, satellite imagery, Criminal Detecting are widely used And having emerged in large numbers a large amount of outstanding algorithm, these algorithms are roughly divided into three classes: Super-Resolution of Images Based based on interpolation, based on weight The Image Super-resolution Reconstruction algorithm of structure, Image Super-resolution Reconstruction algorithm based on study.

In recent years, degree of depth theory of learning develops rapidly, different from the feature extraction algorithm that tradition relies on priori, the degree of depth Neutral net can training data drive under adaptively construction feature describe, there is higher motility and universality.As Realizing an important technology of degree of depth study, convolutional neural networks has the developing history of decades, and degree of depth convolutional neural networks is Closely become a volatile study hotspot due to its outstanding performance in image is classified, be applied successfully to other and calculated Machine visual field.So can also utilize convolutional neural networks that image is carried out super-resolution rebuilding, with original image it is directly Input, obtains feature description by autonomic learning under the driving of training data, improves computing effect while simplifying characteristic model Rate.Convolutional neural networks is utilized image to carry out super-resolution rebuilding, directly between study low resolution and high-definition picture Map end to end.

Summary of the invention

It is an object of the invention to propose a kind of image super-resolution rebuilding method, the method comprises the following steps:

S1 carries out fuzzy and down-sampling according to image degradation model, the image concentrating high-resolution training image, obtains Corresponding low resolution training image collection, is designated as Y by low-resolution image；

The low-resolution image of acquisition is carried out bi-cubic interpolation amplification by S2, is amplified to required size；

S3 designs a convolutional neural networks containing dynamic volume lamination；

The convolutional neural networks B that S4 low-resolution image Y input pre-training is good, obtains wave filter SH1 and SV1；

S5 is by the god of the convolution containing dynamic volume lamination good for low-resolution image Y and wave filter SV1, SH1 input pre-training Through network；

S6 rebuilds high-definition picture X；

Described S3, the convolutional neural networks containing dynamic volume lamination includes three parts:

S3.1 extracts image block from low-resolution image Y, and each image block is expressed as high dimension vector.These vectors include One stack features mapping graph, its number is equal to the dimension of vector:

F1 (Y)=max (0, W1*Y+B1)

W1 is weight, and B1 is deviation, and symbol * is convolution, and W1 corresponds to n1 wave filter, and i.e. n1 convolution acts on figure Picture, the size of convolution kernel is c1 × f1 × f1, and wave filter is from independent convolutional network B.

S3.2 each high dimension vector nonlinear mapping is to another high dimension vector.Conceptive, each map vector is one The representative of high-definition picture block, these vectors include another stack features figure；

F2 (Y)=max (0, W2*F1 (Y)+B2)

W2 is weight, and B2 is deviation, and symbol * is convolution, and W2 corresponds to n2 wave filter, and i.e. n2 convolution acts on figure Picture, the size of each convolution kernel is n1 × f2 × f2, and wave filter is from independent convolutional network B.

S3.3 is polymerized above-mentioned high-definition picture block, produces final high-definition picture；

F (Y)=W3*F2 (Y)+B3

W3 is weight, and B3 is deviation, and symbol * is convolution, and W3 corresponds to n3 wave filter, and i.e. n3 convolution acts on figure Picture, the size of convolution kernel is n2 × f3 × f3.

S5 includes as follows

S5.1 propagated forward

IfInput for i & lt characteristic pattern based on sample t,Export for jth based on sample t time characteristic pattern,For Convolution kernel, its computing formula is

$y_j^t=\underset{i}{\Σ}{k}_{ij}^t*X_i^t$

Unlike conventional roll lamination, the convolution kernel of each convolutional layer in the convolutional neural networks containing dynamic volume lamination Different.

S5.2 back-propagating

Gradient loss function L relative to

$\frac{\∂L}{\∂x_i^t}=\underset{j}{\Σ}\left(\frac{\∂L}{\∂y_j^t}\right)\underset{\‾}{*}\left(k_{i_j}^t\right)$

Symbol*Represent zero padding convolution.

Gradient loss function L relative to

$\frac{\∂L}{\∂k_{ij}^t}=\left(\frac{\∂L}{\∂y_j^t}\right)*{\tilde{X}}_i^t$

ForTransposition.

S4 includes as follows

Different from traditional convolutional layer, dynamic volume lamination accepts two inputs.First input is the characteristic pattern of last layer, Second input is wave filter.Characteristic pattern is from convolutional network A, and wave filter is from independent convolutional network B.

Convolutional network B structure:

1) convolutional layer C1, inputs the low resolution training data identical with convolutional network A, by n1 size c1 × f1 × The wave filter of f1, exports n1 characteristic pattern

2) maximum-down-sampling layer M1, C1 layer n1 the characteristic pattern produced, be 2 by step-length, the window of size 2 × 2

3) convolutional layer C2, inputs n1 characteristic pattern, by n2 size n1 × f1 × f1 wave filter, exports n2 characteristic pattern

4) maximum-down-sampling layer M2, C2 layer n2 the characteristic pattern produced, be 2 by step-length, the window of size 2 × 2 Mouthful；

5) convolutional layer C3, inputs n2 characteristic pattern, by n3 big n2 × f1 × f1 wave filter, exports n3 characteristic pattern；

6) maximum-down-sampling layer M3, C3 layer n3 the characteristic pattern produced, be 2 by step-length, the window of size 2 × 2 Mouthful；

7) output of M3 is converted to an one-dimensional row vector H1:1 × h1 by a full articulamentum；

8) output of M3 is converted to dimensional vector V1:v1 × 1 by a full articulamentum；

9) H1 and V1 is used Softmax function, obtain vector SH1 and SV1；

10) wave filter SV1 is applied to dynamic volume lamination；

11) wave filter SH1 is applied to dynamic volume lamination.

Accompanying drawing explanation

Fig. 1 is the convolutional neural networks Image Super-resolution Reconstruction algorithm frame that the present invention contains dynamic volume lamination；

Fig. 2 is the convolutional neural networks B framework that the present invention obtains dynamic volume lamination wave filter SV1 and SH1；

Fig. 3 is that the image using the present invention to amplify 2 times processes through the convolutional neural networks containing dynamic volume lamination Reconstructed results and other three kinds of method comparison；Wherein, a is artwork, and b is bicubic interpolation, and c is that the anchor point neighbour improved returns calculation Method, d is super-resolution rebuilding algorithm based on convolutional neural networks, and e is the present invention.

Fig. 4 is that the image using the present invention to amplify 2 times processes through the convolutional neural networks containing dynamic volume lamination Reconstructed results and other three kinds of method comparison；Wherein, a is artwork, and b is bicubic interpolation, and c is that the anchor point neighbour improved returns calculation Method, d is super-resolution rebuilding algorithm based on convolutional neural networks, and e is the present invention

Detailed description of the invention

With reference to Fig. 1, the framework of the present invention is

Step 1, inputs low-resolution image Y；

Step 2, utilizes the imresize function in Matlab software that the image of this low resolution carries out double cubes of 2 times Interpolation amplification, obtains low-resolution image Y；

Step 3, the convolutional network nerve B that low-resolution image Y input pre-training is good, obtain wave filter SH1 and SV1；

Step 4, inputs the god of the convolution containing dynamic volume lamination that pre-training is good by low-resolution image Y and wave filter SV1 First dynamic volume lamination through network；

Step 5, the of the output in previous step and the wave filter SH1 input convolutional neural networks containing dynamic volume lamination Two dynamic volume laminations；

Step 6, the convolutional neural networks third layer containing dynamic volume lamination of the output input in previous step, obtain high score Resolution image；

Step 7, rebuilds high-definition picture.

For the effectiveness of verification algorithm, on test library set5 and test library 14, respectively with other three kinds of outstanding algorithms Compare.The four width images of Fig. 3 are artwork respectively, Bicubic bicubic interpolation algorithm, and A+ is that the anchor point neighbour improved returns Algorithm, SRCNN is super-resolution rebuilding algorithm based on convolutional neural networks, image super-resolution rebuilding algorithm of the present invention.Fig. 4 Four width images be artwork respectively, Bicubic bicubic interpolation algorithm, A+ be improve anchor point neighbour's regression algorithm, SRCNN is Super-resolution rebuilding algorithm based on convolutional neural networks, image super-resolution rebuilding algorithm of the present invention.

Table 1 is the structural similarity (SSIM) of Fig. 3 reconstructed results and Y-PSNR (PSNR) compares.

Table 1

Table 2 is the structural similarity (SSIM) of Fig. 4 reconstructed results and Y-PSNR (PSNR) compares.

Table 2

Test result indicate that, the algorithm that the present invention proposes, not only in visual effect but also in objective evaluation standard all Achieve than other three kinds of significant effects of outstanding algorithm, present outstanding super-resolution rebuilding performance.

Claims (4)

1. an image super-resolution rebuilding method, it is characterised in that: the method comprises the following steps,

S1 extracts the low-resolution image Y in training image storehouse；

The low-resolution image of acquisition is carried out bi-cubic interpolation amplification by S2, is amplified to required size；

S3 designs a convolutional neural networks containing dynamic volume lamination；

The convolutional neural networks B that S4 low-resolution image Y input pre-training is good, obtains wave filter SH1 and SV1；

S5 is by the convolutional Neural net containing dynamic volume lamination good for low-resolution image Y and wave filter SV1, SH1 input pre-training Network；

S6 rebuilds high-definition picture X.

A kind of image super-resolution rebuilding method the most according to claim 1, it is characterised in that: described S3, containing dynamically The convolutional neural networks of convolutional layer includes three parts:

S3.1 extracts image block from low-resolution image Y, and each image block is expressed as high dimension vector；These vectors include one group Feature Mapping figure, its number is equal to the dimension of vector:

F1 (Y)=max (0, W1*Y+B1)

W1 is weight, and B1 is deviation, and symbol * is convolution, and W1 corresponds to n1 wave filter, and i.e. n1 convolution acts on image, often The size of individual convolution kernel is c1 × f1 × f1, and n1 wave filter is from independent convolutional network B；

S3.2 each high dimension vector nonlinear mapping is to another high dimension vector；Conceptive, each map vector is a high score The representative of resolution image block, these vectors include another stack features figure；

F2 (Y)=max (0, W2*F1 (Y)+B2)

W2 is weight, and B2 is deviation, and symbol * is convolution, and W2 corresponds to n2 wave filter, and i.e. n2 convolution acts on image, often The size of individual convolution kernel is n1 × f2 × f2, and n2 wave filter is from independent convolutional network B；

S3.3 is polymerized above-mentioned high-definition picture block, produces final high-definition picture；

F (Y)=W3*F2 (Y)+B3

W3 is weight, and B3 is deviation, and symbol * is convolution, and W3 acts on image corresponding to i.e. n3 convolution of n3 wave filter, each The size of convolution kernel is n2 × f3 × f3.

A kind of image super-resolution rebuilding method the most according to claim 1, it is characterised in that: S5 includes as follows,

S5.1 propagated forward

IfInput for i & lt characteristic pattern based on sample t,Export for jth based on sample t time characteristic pattern,For convolution Core, its computing formula is

$y_j^t=\underset{i}{\Σ}{k}_{ij}^t*X_i^t$

Unlike conventional roll lamination, in the convolutional neural networks containing dynamic volume lamination, the convolution kernel of each convolutional layer is not With；

S5.2 back-propagating

Gradient loss function L relative to

$\frac{\∂L}{\∂x_i^t}=\underset{j}{\Σ}\left(\frac{\∂L}{\∂y_j^t}\right)\underset{\‾}{*}\left(k_{i_j}^t\right)$

Symbol*Represent zero padding convolution；

Gradient loss function L relative to

$\frac{\∂L}{\∂k_{ij}^t}=\left(\frac{\∂L}{\∂y_j^t}\right)*\widetilde{X_i^t}$

ForTransposition.

A kind of image super-resolution rebuilding method the most according to claim 1, it is characterised in that: S4 includes as follows, with biography The convolutional layer of system is different, and dynamic volume lamination accepts two inputs；First input is the characteristic pattern of last layer, and second input is Wave filter；Characteristic pattern is from convolutional network A, and wave filter is from independent convolutional network B；

Convolutional network B structure:

1) convolutional layer C1, inputs the low resolution training data identical with convolutional network A identical with convolutional network A, by n1 The wave filter of size c1 × f1 × f1, exports n1 characteristic pattern

2) maximum-down-sampling layer M1, C1 layer the n1 characteristic pattern produced, be 2 by step-length, the window of size 2 × 2

3) convolutional layer C2, inputs n1 characteristic pattern, by the wave filter of n2 size c2 × f2 × f2, exports n2 characteristic pattern

4) maximum-down-sampling layer M2, C2 layer the n2 characteristic pattern produced, be 2 by step-length, the window of size 2 × 2；

5) convolutional layer C3, inputs n2 characteristic pattern, by the wave filter of n3 size c3 × f3 × f3, exports n3 characteristic pattern；

6) maximum-down-sampling layer M3, C3 layer the n3 characteristic pattern produced, be 2 by step-length, the window of size 2 × 2；

7) output of M3 is converted to an one-dimensional row vector H1:1 × h1 by a full articulamentum；

8) output of M3 is converted to dimensional vector V1:v1 × 1 by a full articulamentum；

9) H1 and V1 is used Softmax function, obtain vector SH1 and SV1；

10) wave filter SV1 is applied to dynamic volume lamination；

11) wave filter SH1 is applied to dynamic volume lamination.