CN110223234A - Depth residual error network image super resolution ratio reconstruction method based on cascade shrinkage expansion - Google Patents
Depth residual error network image super resolution ratio reconstruction method based on cascade shrinkage expansion Download PDFInfo
- Publication number
- CN110223234A CN110223234A CN201910506498.1A CN201910506498A CN110223234A CN 110223234 A CN110223234 A CN 110223234A CN 201910506498 A CN201910506498 A CN 201910506498A CN 110223234 A CN110223234 A CN 110223234A
- Authority
- CN
- China
- Prior art keywords
- layer
- network
- image
- shrinkage
- sub
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000012549 training Methods 0.000 claims abstract description 23
- 238000013528 artificial neural network Methods 0.000 claims abstract description 17
- 238000005070 sampling Methods 0.000 claims description 38
- 230000004913 activation Effects 0.000 claims description 22
- 230000008602 contraction Effects 0.000 claims description 11
- 230000003252 repetitive effect Effects 0.000 claims description 4
- 210000005036 nerve Anatomy 0.000 claims 1
- 238000004422 calculation algorithm Methods 0.000 abstract description 52
- 238000011084 recovery Methods 0.000 abstract description 2
- 230000007812 deficiency Effects 0.000 abstract 1
- 230000006870 function Effects 0.000 description 25
- 238000012360 testing method Methods 0.000 description 9
- 230000000694 effects Effects 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 8
- 238000013507 mapping Methods 0.000 description 8
- 210000002569 neuron Anatomy 0.000 description 8
- 238000011478 gradient descent method Methods 0.000 description 6
- 230000000644 propagated effect Effects 0.000 description 6
- 230000003321 amplification Effects 0.000 description 4
- 238000013527 convolutional neural network Methods 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 4
- 238000003199 nucleic acid amplification method Methods 0.000 description 4
- 238000010276 construction Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 210000004027 cell Anatomy 0.000 description 2
- 238000009795 derivation Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- 230000001537 neural effect Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 239000011248 coating agent Substances 0.000 description 1
- 238000000576 coating method Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000003475 lamination Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4053—Super resolution, i.e. output image resolution higher than sensor resolution
Abstract
The invention discloses a kind of depth residual error network image super resolution ratio reconstruction methods based on cascade shrinkage expansion, it include: to obtain low resolution observed image sequence first, bicubic difference operation is carried out to piece image any in the sequence, obtains the initial valuation of high-definition picture；It is then based on the depth residual error neural network of cascade shrinkage expansion, the training of depth residual error neural network is carried out to initial valuation image, by trained characteristic pattern and initial valuation image addition, recovers the corresponding high-definition picture of low-resolution image.The present invention efficiently solves the problems, such as network structure redundancy, training complexity, image reconstruction details deficiency in the prior art, realizes the better detail recovery of super-resolution image, and algorithm is simple, the speed of service is fast, practical.
Description
Technical field
The present invention relates to multimedia technology fields, particularly belong to digital picture and digital technical field of video processing, especially
It is related to a kind of depth residual error network image super resolution ratio reconstruction method based on cascade shrinkage expansion.
Background technique
The super-resolution rebuilding technology of image is widely applied to every field, including medical image, satellite image, video
Monitoring, HD video and detection and identification mission.In satellite image field, due to by remote sensing satellite launch cost, at image quality
Amount etc. conditions restriction, while there is also satellite image transmission process by uncertain atmospheric perturbation and the outer space electricity
Magnetic signal interference, the resolution ratio of the remote sensing satellite image of acquisition are often unable to reach the demand of many tasks.Use image reconstruction
Technology improves HR level, can effectively save satellite resource cost, it helps improve identification and the reconnaissance capability of remote sensing satellite.
The purpose of image super-resolution is in the case where given one or several low-resolution images, and reconstruct obtains a super-resolution
Image.In the past more than ten years, researcher proposes many image super-resolution rebuilding algorithms, according to low-resolution image
Number, image super-resolution rebuilding algorithm can be divided into the super-resolution rebuilding algorithm of multiple image and the oversubscription of single image
Resolution algorithm for reconstructing.
For the super-resolution rebuilding of single image, presently, there are many different algorithms.Based on traditional interpolation side
Method is all by a basic function, such as bilinear interpolation, bicubic interpolation and nearest neighbor algorithm, it is generally the case that these
Method is all simple and effective, but since the picture element interpolation of these algorithms operates, will lead to edge and aliasing effect occurs in high-frequency region
It should be with fuzzy distortion.
The image super-resolution rebuilding technology based on deep learning method achieves very big development and effect in recent years,
It rebuilds in effect and time efficiency and has surmounted traditional method based on rebuilding and based on rarefaction representation.But it is based on depth
The super-resolution rebuilding technology of habit still can not preferably handle practical problem at present, therefore there are also very big rooms for promotion.Mesh
Before, there are the following problems for the image super-resolution rebuilding technology based on depth convolutional neural networks:
1) there are redundancies for the structure of reconstruction network:
The super resolution ratio reconstruction method for being currently based on deep learning still has many problems, one of them is due to depth
The level of network structure is deep, relatively more, the bigger problem of calculation amount that leads to that there are parameters.Image super-resolution weight at present
Build that frame is widely used is the convolutional neural networks based on residual error study, and has been achieved for preferable effect.But such as
What design residual error structure, does not increase with abundant super-resolution rebuilding details and network size, is still problem to be solved.
2) reconstruction image details is insufficient:
Image super-resolution rebuilding is an ill-conditioning problem, because of the height corresponding for each low-resolution image
The possible solution domain of image in different resolution is possible very big.Currently based on the single image high-resolution of depth convolutional neural networks
The accuracy and speed that rate is rebuild have breakthrough, and traditional image super-resolution rebuilding algorithm usually makes for increasing receptive field
It is realized with more convolutional layers or using expansion convolution.But using more layers mean network it is huge and complicated and
It is more difficult to train.In addition, frequently can lead to grid effect using expansion convolution in image super-resolution rebuilding.
Summary of the invention
It is provided the purpose of the present invention is reducing the limitation that cost overcomes hardware by Image Reconstruction Technology, while for people
A kind of effective and feasible algorithm solution proposes a kind of depth residual error network image oversubscription based on cascade shrinkage expansion
Resolution method for reconstructing realizes the better detail recovery of super-resolution image, algorithm to solve the above-mentioned problems of the prior art
Simply, the speed of service is fast, practical.
To achieve the above object, the present invention provides a kind of depth residual error network image super-resolution based on cascade shrinkage expansion
Rate method for reconstructing, specific technical solution are as follows:
A kind of depth residual error network image super resolution ratio reconstruction method based on cascade shrinkage expansion, comprising:
Step S1 obtains low resolution observed image sequence, and it is poor to carry out bicubic to piece image any in the sequence
It is worth operation, obtains the initial valuation of high-definition picture；
Step S2 carries out depth residual error to initial valuation image based on the depth residual error neural network of cascade shrinkage expansion
Neural network training, by trained characteristic pattern and initial valuation image addition, recovers the corresponding high score of low-resolution image
Resolution image.
Preferably, in step S2, the training of depth residual error neural network is carried out to initial valuation image, comprising: use first
It shrinks sub-network and shrinks initial valuation characteristics of image, then rebuild by the characteristics of image of contraction by extension sub-network defeated
Out, the corresponding high-definition picture of low-resolution image is recovered finally by reconstruction sub-network.
Preferably, shrinking sub-network collapse step is to be input to initial valuation image and shrink sub-network first order convolution
Layer obtains the first shrinkage layer C1；First shrinkage layer C1 is operated by down-sampling, is shunk using the study of convolutional layer and is obtained the
Two shrinkage layer C2；Second shrinkage layer C2 is operated by down-sampling, learns to obtain third shrinkage layer C3 using a convolutional layer；The
Three shrinkage layer C3 are operated by down-sampling, learn to obtain the 4th shrinkage layer C4 using cubic convolution layer, the 4th shrinkage layer C4 makees
For the first extension layer D1 in extension sub-network.
Preferably, extension sub-network spread step be, the first extension layer D1 through up-sampling operation after with third shrinkage layer C3
It is added, obtains the second extension layer D2；Second extension layer D2 is received after a convolutional layer learns and up-samples operation, then with second
Contracting layer C2 is added, and obtains third extension layer D3；Third extension layer D3 by a convolutional layer learns and up-sample operation after, then with
First shrinkage layer C1 is added, and obtains the 4th extension layer D4, the 4th extension layer D4 is by a convolutional layer study, repeated contraction step
And spread step.
Preferably, rebuilding sub-network reconstruction procedures is, in repetitive extension sub-network, the second extension layer D2 passes through on twice
Sampling operation exports third feature figure R3 to the 4th extension layer D4；Third extension layer D3 is by primary up-sampling operation to the 4th
Extension layer D4 exports second feature figure R2；By extending sub-network spread step, obtained the 4th extension layer D4 output first is special
Sign figure R1；Fisrt feature figure R1, second feature figure R2 are added to obtain trained characteristic pattern with third feature figure R3, trained
After characteristic pattern carries out convolution operation, then with initial valuation image addition, obtain the corresponding high-definition picture of low-resolution image.
Preferably, the convolution down-sampling operation for executing that step-length is 2 in sub-network is shunk, executing step-length in extension sub-network is
2 deconvolution up-samples operation, and each layer in network is made of 3 × 3 filters and activation primitive operation.
Preferably, shrink sub-network in, the second shrinkage layer C2 and the first shrinkage layer C1 sampling after the completion of, by fast connecting
Through extension sub-network respective layer reaches the next stage shrinkage layer for shrinking sub-network.
Compared with prior art, beneficial effects of the present invention are specific as follows:
It is first the invention discloses a kind of depth residual error network image super resolution ratio reconstruction method based on cascade shrinkage expansion
First to input low-resolution image, bicubic difference operation is carried out, the initial valuation of high-definition picture is obtained, then passes through contraction
The operation of sub-network down-sampling carries out feature shrink, increases receptive field, and the shrinkage characteristic that will be finally obtained is more by extension sub-network
Grade up-sampling realizes that details is rebuild and multi-stage characteristics export, and does not increase for feature-rich and computation burden, by fast connecting
It connects and combines the Feature Mapping for shrinking sub-network with the Feature Mapping of extension sub-network, merged finally by sub-network is rebuild
The feature of different scale, recovers the corresponding high-definition picture of low-resolution image, and the present invention can learn plan according to residual error
Slightly realize that residual information is transmitted using sub-network multi-grade remnant learning structure is shunk, meanwhile, using expansion sub-network multistage weight
The mode of building exports different levels detail textures, preferably recovers the details of low-resolution image.And inventive algorithm and its
He compares mainstream algorithm, with good performance and efficiency, and when obtaining optimal performance and efficiency, this algorithm is compared with other mainstreams
Algorithm time-consuming is shorter, and this algorithm network structure is simple, and the computation complexity speed of service is fast, while reducing calculating cost
Preferable performance is achieved, resultant effect has preferable promotion, practical.
Detailed description of the invention
It in order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, below will be to institute in embodiment
Attached drawing to be used is needed to be briefly described, it should be apparent that, the accompanying drawings in the following description is only some implementations of the invention
Example, for those of ordinary skill in the art, without any creative labor, can also be according to these attached drawings
Obtain other attached drawings.
Fig. 1 is inventive algorithm frame diagram；
Fig. 2 is the single neuronal structure figure of the present invention；
Fig. 3 is that the present invention is based on the image super-resolution rebuilding frames of the depth residual error network of cascade shrinkage expansion；
Fig. 4 is that inventive algorithm and current main-stream algorithm rebuild picture comparison diagram；
Fig. 5 is test run time analysis figure；
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
In order to make the foregoing objectives, features and advantages of the present invention clearer and more comprehensible, with reference to the accompanying drawing and specific real
Applying mode, the present invention is described in further detail.
Referring to Fig. 2, a kind of depth residual error network image super resolution ratio reconstruction method based on cascade shrinkage expansion of the present invention,
Include:
1. the nonlinear fitting of neural network
Neural network is exactly the multiple neurons connected according to certain rule, each neuron node therein in fact
Receive input value of the output valve of upper one layer of neuron as this neuron, and input value is passed into next layer, input layer mind
Attribute value can be directly passed to next layer (or output layer) through first node.In multilayer neural network, upper layer node
There is a functional relation, this function is known as activation primitive between output and the input of lower level node.Single neuronal structure
As shown in Fig. 2, single neuron knot mathematic(al) representation can be described as it is as follows:
For formula (1), f represents the neuron, wherein xjFor j-th of input of neuron, wjFor the jth of corresponding input
A weight, b are bias term, and y is the output of neuron, which includes n input and an output.One neural network
There is weights in each connection, these weights are exactly the parameter of neural network model in fact, that is, model to be learnt
Thing.However, these parameters such as the connection type of a neural network, the number of plies of network, every layer of number of nodes, learning rate, then
It is artificial set in advance.Hyper parameter (Hyper-Parameters) is normally referred to as these parameters being artificially arranged.
2. propagated forward
The input that propagated forward algorithm may be summarized to be each layer in neural network is weighted with corresponding connection weight
And operation, result add a bias term, but in some neural networks bias term it is not necessary to.Then, then
By a nonlinear activation function, such as ReLu, Sigmoid, Tanh function, obtained each layer of output result.Finally
Constantly constantly operation forward by this method, obtains output layer result.For propagated forward, regardless of how high dimension is,
Its process can be indicated with following formula:
Wherein,Indicating the weight inputted for j-th of l layers of i-th cell, f indicates activation primitive,It indicates
The activation output valve of l layers of i-th cell.Since relatively simple, forward direction is propagated in the output therefore preceding paragraph that only need to calculate each layer
Then start the backpropagation of network after propagation.
3. backpropagation
In order to which the parameter to network is updated, must just carry out gradient backpropagation (Back-Propagation,
BP amplitude) is updated to calculate each parameter, back propagation is specific implementation of the gradient descent method on depth network.It is logical
Often, gradient is calculated the objective function of depth network, and minimizing target function gradient descent method is a kind of Neural Network Optimization
Algorithm, central idea are minimum (or most to wish to reach objective function along the direction undated parameter value of target function gradient
Greatly).Gradient descent method is the most common optimization algorithm of deep learning network.
Back-propagation algorithm is usually used to trained CNN, i.e., is updated by backpropagation loss to each layer of gradient
The corresponding parameter of network, network parameter update iteration is complete primary, is further continued for calculating the loss of next iteration by propagated forward,
Then undated parameter is repeated, until reaching setting the number of iterations or reaching setting penalty values, so that the e-learning of training arrives
Mapping relations as close possible to real goal map.Calculate the chain type derivation formula expression of gradient are as follows:
It include fixed sample the collection { (x of m sample equipped with one(1),y(1)),…,(x(m),y(m)), use batch gradient
Descent method (Batch Gradient Descent) solves neural network.Specifically, to single sample (x, y) if using normal
Two norms calculate its loss, then its cost function indicates are as follows:
In order to reduce the amplitude of weight, over-fitting is prevented, by the way that regularization term is added, wherein weight attenuation parameter λ is used for
Control two in formula relative importances, loss function statement are as follows:
M indicates the batch size of batch gradient descent method, updates loss by constantly iterating to calculate, target is for parameter
W and b asks the smallest cost J (W, b).Each iteration is updated W and b all in accordance with following formula in gradient descent method:
(W, b)=(W(1),b(1),W(2),b(2)…,W(n),b(n))………………8
Wherein α is learning rate, by calculating separately loss function to the partial derivative of weight W and biasing b multiplied by study
Rate has then obtained the paces that parameter needs to update every time.Wherein, formula (8) is to need Optimal Parameters set, i.e. weight W and partially
B is set, so far, gradient back-propagation algorithm is returning to propagated forward step after having updated all parameters, constantly with this
Reciprocation cycle terminates until training.
Under normal conditions, one layer of active coating can be all followed by after convolutional layer.Activation primitive be for be added it is non-linear because
Element, because the ability to express of linear model cannot achieve complicated Nonlinear Mapping.Output after convolution is passed through into activation letter
Number is mapped to the value of activation primitive above threshold, and activation primitive exports the Nonlinear Mapping of feature in order to realize.
Correcting linear unit (Rectified Linear unit, ReLU) is also a common nonlinear activation function,
ReLU function is linear in Zheng Banqu, that is, realize input mapping be itself, in the area Fu Ban, the functional value is then 0, derivative namely
It is 0.The function is substantially better than the function of front two, is present most popular function.The function is relatively simple, reversed derivation
Also very easy, mathematic(al) representation is as follows:
F (x)=max (0, x) ... ... ... ... ... ... ... 9
Non-linear behavior based on activation primitive, neural network may be implemented advanced by combined use activation primitive
Nonlinear Mapping.Above-mentioned common activation primitive also include the more advanced activation primitive of some variations such as: Leaky
It is the activation primitive extended that ReLU function, ELU function, MaxOut function, PReLU function etc., which are with ReLU, and Softmax function is
Polytypic activation primitive is used for the extension of Sigmoid function.
Construction step such as Fig. 3 of depth residual error network algorithm based on cascade shrinkage expansion, comprising: shrink sub-network, expand
It opens up sub-network and rebuilds sub-network.It shrinks and extension sub-network is respectively all comprising there are four the features of level to describe.It uses first
Low-resolution image after bicubic interpolation operation is by shrinking sub-network increase receptive field and being used to expand by shrinkage characteristic expression
Sub-network is opened up, the feature shrink of multistage down-sampling layer is then constructed；Extension sub-network constructs and to shrink sub-network corresponding
The Fusion Features of multistage up-sampling layer are to realize that image detail restores；Subnet will be shunk by fast connection in rebuilding sub-network
Network combines with the Feature Mapping of extension sub-network to realize that multistage reconstruction features export.
Algorithm implementation of the invention as shown in figure 3, each rectangular block represents one layer of convolutional layer in figure,
Downsampling indicate execute step-length be 2 convolution realize down-sampling operation, Upsampling indicate execute step-length be 2 it is anti-
Convolution realizes up-sampling operation, and each layer in network is made of 3 × 3 filters and activation primitive operation.Activation primitive uses
Linear amending unit ReLU is parameterized, the expression ability of the activation primitive energy maximization network does not use the last layer then
Activation primitive is to keep the layer as linear convergent rate.Specific each section is described as follows:
(1) sub-network is shunk
Sub-network is shunk in building；Shown in its structure such as Fig. 3 left hand half point, contraction sub-network is shunk every time is all by step-length
2 convolutional layer realizes down-sampling operation, is first input to initial valuation image and shrinks sub-network first order convolutional layer and obtain the
One shrinkage layer C1；First shrinkage layer C1 is operated by down-sampling, is shunk using a convolutional layer study and is obtained the second shrinkage layer
C2；Second shrinkage layer C2 is operated by down-sampling, learns to obtain third shrinkage layer C3 using a convolutional layer；Third shrinkage layer
C3 is operated by down-sampling, learns to obtain the 4th shrinkage layer C4 using cubic convolution layer, the 4th shrinkage layer C4 is as extension
The first extension layer D1 in network.
(2) sub-network is extended
Building extension sub-network；As shown in the half part to the right of the left side Fig. 3, extension sub-network extends all by step its structure every time
A length of 2 deconvolution executes up-sampling operation, and realization gradually restores characteristic pattern spatial resolution, first by the first extension
Layer D1 is added after up-sampling operation with third shrinkage layer C3, obtains the second extension layer D2；Second extension layer D2 passes through a secondary volume
It after lamination study and up-sampling operation, then is added with the second shrinkage layer C2, obtains third extension layer D3；Third extension layer D3 passes through
It after the study of convolutional layer and up-sampling operation, then is added with the first shrinkage layer C1, obtains the 4th extension layer D4, the 4th extension layer
D4 is by a convolutional layer study, repeated contraction step and spread step, and in repetitive extension step, the first order extends sub-network
Phase add operation in upper sampling process has only used the first order and has shunk the block that sub-network down-sampling corresponds to size, expanded in the second level
It opens up during sub-network up-sampling is added, had not only used the second level to shrink corresponding piece of sub-network down-sampling, but also used the
The block of level-one contraction sub-network down-sampling.
(3) sub-network is rebuild
In repetitive extension sub-network, the second extension layer D2 is by up-sampling operation twice to the 4th extension layer D4, output
Third feature figure R3；Third extension layer D3, to the 4th extension layer D4, exports second feature figure R2 by primary up-sampling operation；Through
Extension sub-network spread step is crossed, obtained the 4th extension layer D4 output fisrt feature figure R1；Fisrt feature figure R1, second feature
Figure R2 is added to obtain trained characteristic pattern with third feature figure R3, after trained characteristic pattern carries out convolution operation, then and just
Beginning valuation image addition obtains the corresponding high-definition picture of low-resolution image.
The training of depth residual error network algorithm and test based on cascade shrinkage expansion
(1) algorithm training
The image reconstruction of the multiple dimensioned factor may be implemented in algorithm for reconstructing of the present invention.Using with multiple scale factor models
Benefit is that all parameters of network can be shared on different amplification factors.It is similar to VDSR, EDSR and DRNN method, we
Model can rebuild the HR image that amplification factor is 2,3 and 4 times.It needs compared to other algorithms for each scale factor training list
Only model, our method reduce many network parameters, can reduce the ruler of image reconstruction model in practical applications
It is very little.In order to preferably train network, the network of proposition, DIV2K data are trained using a large-scale image reconstruction data set
Collection contains the training image and authentication image of 1000 high-resolution image constructions.Usually by the cromogram in all data sets
As being converted to YCbCr space from rgb space, an extract light intensity level is trained and tests, and color component is inserted by bicubic
Value amplification obtains.Network will be used as loss function using mean square error function (MSE), while the constraint to parameter is added to prevent
Network over-fitting:
Wherein, Θ is network parameter, and n is training set image number, y(i)For HR image, x(i)For LR image, Y (x(i)) be
Network output, λ is regularization coefficient.This part implementation steps is as follows: carrying out random Gaussian core blur degradation to training image collection
Operation, then carries out down-sampling and obtains multiple LR images, and HR image construction training image pair corresponding in training set, and uses number
Expand training set according to enhancing；By obtained LR and HR image to being sent in contraction expansion depth residual error network, to train weight
Establishing network；The Color Channel after the luminance channel and interpolation after reconstruction is finally transformed into rgb space again.
Usually in data prediction, training image is cut into the block that size is 64 × 64 preferably to accelerate to instruct
Practice.Training data is expanded by two ways: by 90 °, 180 ° or 270 ° of image Random-Rotation and horizontal and vertical being turned over
Turn image.Since the picture size in DIV2K data set is larger, we only randomly choose a kind of enhancing processing and expand training set
To original 2 times.In addition to the last layer, each layer is all by the convolution algorithm of 64 3 × 3 filters and an activation primitive operation
Composition.So filter initialization mode, which uses, is based on He-normal method.Using batch gradient descent algorithm instruction in training process
Practice network, every batch of image number of blocks is set as 64.Training also uses learning rate decaying strategy, and initial learning rate is set as
0.0001,1/10th are decayed to every 15 wheels, decays four times in total until 60 wheel of training terminates.All experiments exist
It is realized on TensorFlow platform using Python3.6 environment.
(2) test of heuristics
In order to further illustrate the validity of invention algorithm, we are compared this method and other image rebuilding methods
Compared with objectively evaluating index, Y-PSNR (Peak Signal to Noise Ratio) abbreviation PSNR using common image
And measure structural similarity (Structural Similarity index) abbreviation SSIM of two images index of similarity.
Experimental result has published the test code from different authors, including Bicubic, SRCNN, VDSR, DRCN,
LapSRN, ARN and DWSR.On BSD100 and Urban100 assessment data set, method and existing some depth to proposition
The image rebuilding method of study is tested, using the HR image by bicubic linear interpolation arithmetic as initial estimation,
Luminance channel calculates PSNR the and SSIM value of reconstruction image, objectively evaluates the performance of image reconstruction algorithm, table 1 gives 2 bases
The average PSNR and SSIM value of 7 kinds of algorithm for reconstructing, can be seen that algorithm for reconstructing of the present invention from the data of table 1 and exists on quasi- data set
The PSNR value of the test result of four disclosed image reconstruction evaluation data sets is high compared with other mainstreams algorithms, obtained PSNR and
SSIM value has reached preferable effect, and image synthesis effect is improved；
1 inventive algorithm of table and current main-stream algorithm are compared in the PSNR and SSIM of different data collection
Comparing result on open test data set is as shown in figure 4, can be with by the detail section amplification to reconstruction image
Find out, algorithm for reconstructing of the invention can preferably rebuild the information such as detail textures, realize the better details of super-resolution image
Restore, it is practical.
The average PSNR and mean test time of Urban100 test data set when amplifying 4 times as shown in Figure 5
Tetra- Riming time of algorithm of VDSR, DRCN, LAPSRN, DWSR of relationship, algorithm for reconstructing of the present invention and deep learning method carry out
Comparison, algorithm for reconstructing of the present invention is with good performance and efficiency, when obtaining optimal performance and efficiency, inventive algorithm
Time close to DWSR and LAPSR algorithm, still, the PSNR value of inventive algorithm is higher than DWSR and LAPSR algorithm.And
Inventive algorithm is more shorter than other three kinds of algorithm time-consumings, and PSNR is apparently higher than other three kinds of algorithms, especially in SCRNN
Inventive algorithm reduce calculate cost while achieve preferable performance, the low speed of service of computation complexity is fast, algorithm and
Network structure is simple.
Embodiment described above is only that the preferred embodiment of invention is described, and is not defined to the range of invention,
Without departing from the spirit of the design of the present invention, those of ordinary skill in the art make technical solution of the present invention various
Modification and improvement are each fallen in the protection scope that claims of the present invention determines.
Claims (7)
1. a kind of depth residual error network image super resolution ratio reconstruction method based on cascade shrinkage expansion, characterized by comprising:
Step S1 obtains low resolution observed image sequence, carries out bicubic difference fortune to piece image any in the sequence
It calculates, obtains the initial valuation of high-definition picture；
Step S2 carries out depth residual error nerve to initial valuation image based on the depth residual error neural network of cascade shrinkage expansion
Network training, by trained characteristic pattern and initial valuation image addition, recovers the corresponding high-resolution of low-resolution image
Image.
2. the depth residual error network image super resolution ratio reconstruction method according to claim 1 based on cascade shrinkage expansion,
It is characterized by: carrying out the training of depth residual error neural network to initial valuation image, comprising: first using contraction in step S2
Sub-network shrinks initial valuation characteristics of image, the characteristics of image of contraction is then extended output by extension sub-network, most
The corresponding high-definition picture of low-resolution image is recovered by rebuilding sub-network afterwards.
3. the depth residual error network image super resolution ratio reconstruction method according to claim 2 based on cascade shrinkage expansion,
It is characterized by: shrinking sub-network collapse step is that initial valuation image is input to contraction sub-network first order convolutional layer and is obtained
To the first shrinkage layer C1；First shrinkage layer C1 is operated by down-sampling, is shunk using a convolutional layer study and is obtained the second receipts
Contracting layer C2；Second shrinkage layer C2 is operated by down-sampling, learns to obtain third shrinkage layer C3 using a convolutional layer；Third is received
Contracting layer C3 is operated by down-sampling, learns to obtain the 4th shrinkage layer C4 using cubic convolution layer, the 4th shrinkage layer C4 is as expansion
Open up the first extension layer D1 in sub-network.
4. the depth residual error network image super resolution ratio reconstruction method according to claim 2 based on cascade shrinkage expansion,
It is characterized by: extension sub-network spread step is, the first extension layer D1 is added after up-sampling operation with third shrinkage layer C3,
Obtain the second extension layer D2；Second extension layer D2 by a convolutional layer learns and up-sample operation after, then with the second shrinkage layer
C2 is added, and obtains third extension layer D3；Third extension layer D3 is after a convolutional layer learns and up-samples operation, then with first
Shrinkage layer C1 is added, and obtains the 4th extension layer D4, the 4th extension layer D4 is by a convolutional layer study, repeated contraction step and expansion
Open up step.
5. the depth residual error network image super resolution ratio reconstruction method according to claim 2 based on cascade shrinkage expansion,
It is characterized by: rebuilding sub-network reconstruction procedures is that in repetitive extension sub-network, the second extension layer D2 by up-sampling twice
It operates to the 4th extension layer D4, exports third feature figure R3；Third extension layer D3 is by primary up-sampling operation to the 4th extension
Layer D4, exports second feature figure R2；By extending sub-network spread step, obtained the 4th extension layer D4 output fisrt feature figure
R1；Fisrt feature figure R1, second feature figure R2 are added to obtain trained characteristic pattern, trained feature with third feature figure R3
After figure carries out convolution operation, then with initial valuation image addition, obtain the corresponding high-definition picture of low-resolution image.
6. the depth residual error network image super-resolution rebuilding side based on cascade shrinkage expansion according to claim 3-4
Method, it is characterised in that: shrink the convolution down-sampling operation for executing that step-length is 2 in sub-network, executing step-length in extension sub-network is 2
Deconvolution up-sample operation, each layer in network is made of 3 × 3 filters and activation primitive operation.
7. the depth residual error network image super resolution ratio reconstruction method according to claim 3 based on cascade shrinkage expansion,
It is characterized by: shrink sub-network in, the second shrinkage layer C2 and the first shrinkage layer C1 sampling after the completion of, it is through by fast connecting
It extends sub-network respective layer or reaches the next stage shrinkage layer for shrinking sub-network.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910506498.1A CN110223234A (en) | 2019-06-12 | 2019-06-12 | Depth residual error network image super resolution ratio reconstruction method based on cascade shrinkage expansion |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910506498.1A CN110223234A (en) | 2019-06-12 | 2019-06-12 | Depth residual error network image super resolution ratio reconstruction method based on cascade shrinkage expansion |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110223234A true CN110223234A (en) | 2019-09-10 |
Family
ID=67816658
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910506498.1A Pending CN110223234A (en) | 2019-06-12 | 2019-06-12 | Depth residual error network image super resolution ratio reconstruction method based on cascade shrinkage expansion |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110223234A (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110648282A (en) * | 2019-09-29 | 2020-01-03 | 燕山大学 | Image super-resolution reconstruction method and system based on width neural network |
| CN110930327A (en) * | 2019-11-18 | 2020-03-27 | 浙江工商大学 | Video denoising method based on cascade depth residual error network |
| CN110991337A (en) * | 2019-12-02 | 2020-04-10 | 山东浪潮人工智能研究院有限公司 | Vehicle detection method based on self-adaptive double-path detection network |
| CN111028147A (en) * | 2019-11-25 | 2020-04-17 | 天津大学 | Three-dimensional medical image super-resolution reconstruction method based on cascade cavity convolution |
| CN111275686A (en) * | 2020-01-20 | 2020-06-12 | 中山大学 | Method and device for generating medical image data for artificial neural network training |
| CN111382845A (en) * | 2020-03-12 | 2020-07-07 | 成都信息工程大学 | Template reconstruction method based on self-attention mechanism |
| CN111563843A (en) * | 2020-04-30 | 2020-08-21 | 苏州大学 | Image super-resolution reconstruction method, system and related device |
| CN111915481A (en) * | 2020-06-08 | 2020-11-10 | 北京大米未来科技有限公司 | Image processing method, image processing apparatus, electronic device, and medium |
| CN111915490A (en) * | 2020-08-14 | 2020-11-10 | 深圳清研智城科技有限公司 | License plate image super-resolution reconstruction model and method based on multi-scale features |
| CN112508013A (en) * | 2020-12-02 | 2021-03-16 | 哈尔滨市科佳通用机电股份有限公司 | Lock catch loss fault detection method, system and device |
| CN113033704A (en) * | 2021-04-22 | 2021-06-25 | 江西理工大学 | Intelligent judging method for copper converter converting copper making period end point based on pattern recognition |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107358575A (en) * | 2017-06-08 | 2017-11-17 | 清华大学 | A kind of single image super resolution ratio reconstruction method based on depth residual error network |
| CN109146784A (en) * | 2018-07-27 | 2019-01-04 | 徐州工程学院 | A kind of image super-resolution rebuilding method based on multiple dimensioned generation confrontation network |
| CN109559297A (en) * | 2018-11-15 | 2019-04-02 | 大连大学 | A method of generating the Lung neoplasm detection of network based on 3D region |
| CN109767386A (en) * | 2018-12-22 | 2019-05-17 | 昆明理工大学 | A kind of rapid image super resolution ratio reconstruction method based on deep learning |
-
2019
- 2019-06-12 CN CN201910506498.1A patent/CN110223234A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107358575A (en) * | 2017-06-08 | 2017-11-17 | 清华大学 | A kind of single image super resolution ratio reconstruction method based on depth residual error network |
| CN109146784A (en) * | 2018-07-27 | 2019-01-04 | 徐州工程学院 | A kind of image super-resolution rebuilding method based on multiple dimensioned generation confrontation network |
| CN109559297A (en) * | 2018-11-15 | 2019-04-02 | 大连大学 | A method of generating the Lung neoplasm detection of network based on 3D region |
| CN109767386A (en) * | 2018-12-22 | 2019-05-17 | 昆明理工大学 | A kind of rapid image super resolution ratio reconstruction method based on deep learning |
Non-Patent Citations (5)
| Title |
|---|
| KANGFU MEI等: "An Effective Single-Image Super-Resolution Model Using Squeeze-and-Excitation Networks", 《10.48550/ARXIV.1810.01831》 * |
| KANGFU MEI等: "An Effective Single-Image Super-Resolution Model Using Squeeze-and-Excitation Networks", 《10.48550/ARXIV.1810.01831》, 31 October 2018 (2018-10-31), pages 1 - 12 * |
| 房雪键: "基于深度学习的图像分类算法研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》, pages 13 - 17 * |
| 郑启财: "基于深度学习的图像检索技术的研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》, pages 21 - 23 * |
| 雷晓静: "基于深度学习的图像分类研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110648282B (en) * | 2019-09-29 | 2021-03-23 | 燕山大学 | Image super-resolution reconstruction method and system based on width neural network |
| CN110648282A (en) * | 2019-09-29 | 2020-01-03 | 燕山大学 | Image super-resolution reconstruction method and system based on width neural network |
| CN110930327A (en) * | 2019-11-18 | 2020-03-27 | 浙江工商大学 | Video denoising method based on cascade depth residual error network |
| CN110930327B (en) * | 2019-11-18 | 2022-09-23 | 浙江工商大学 | Video denoising method based on cascade depth residual error network |
| CN111028147A (en) * | 2019-11-25 | 2020-04-17 | 天津大学 | Three-dimensional medical image super-resolution reconstruction method based on cascade cavity convolution |
| CN111028147B (en) * | 2019-11-25 | 2024-02-02 | 天津大学 | Three-dimensional medical image super-resolution reconstruction method based on cascade cavity convolution |
| CN110991337A (en) * | 2019-12-02 | 2020-04-10 | 山东浪潮人工智能研究院有限公司 | Vehicle detection method based on self-adaptive double-path detection network |
| CN110991337B (en) * | 2019-12-02 | 2023-08-25 | 山东浪潮科学研究院有限公司 | Vehicle detection method based on self-adaptive two-way detection network |
| CN111275686B (en) * | 2020-01-20 | 2023-05-26 | 中山大学 | Method and device for generating medical image data for artificial neural network training |
| CN111275686A (en) * | 2020-01-20 | 2020-06-12 | 中山大学 | Method and device for generating medical image data for artificial neural network training |
| CN111382845B (en) * | 2020-03-12 | 2022-09-02 | 成都信息工程大学 | Template reconstruction method based on self-attention mechanism |
| CN111382845A (en) * | 2020-03-12 | 2020-07-07 | 成都信息工程大学 | Template reconstruction method based on self-attention mechanism |
| CN111563843A (en) * | 2020-04-30 | 2020-08-21 | 苏州大学 | Image super-resolution reconstruction method, system and related device |
| CN111563843B (en) * | 2020-04-30 | 2023-11-28 | 苏州大学 | Image super-resolution reconstruction method, system and related device |
| CN111915481A (en) * | 2020-06-08 | 2020-11-10 | 北京大米未来科技有限公司 | Image processing method, image processing apparatus, electronic device, and medium |
| CN111915481B (en) * | 2020-06-08 | 2024-03-29 | 北京大米未来科技有限公司 | Image processing method, device, electronic equipment and medium |
| CN111915490A (en) * | 2020-08-14 | 2020-11-10 | 深圳清研智城科技有限公司 | License plate image super-resolution reconstruction model and method based on multi-scale features |
| CN112508013A (en) * | 2020-12-02 | 2021-03-16 | 哈尔滨市科佳通用机电股份有限公司 | Lock catch loss fault detection method, system and device |
| CN112508013B (en) * | 2020-12-02 | 2021-07-16 | 哈尔滨市科佳通用机电股份有限公司 | Lock catch loss fault detection method, system and device |
| CN113033704A (en) * | 2021-04-22 | 2021-06-25 | 江西理工大学 | Intelligent judging method for copper converter converting copper making period end point based on pattern recognition |
| CN113033704B (en) * | 2021-04-22 | 2023-11-07 | 江西理工大学 | Intelligent judging method and system for copper converter converting copper-making final point based on pattern recognition |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110223234A (en) | Depth residual error network image super resolution ratio reconstruction method based on cascade shrinkage expansion | |
| Gao et al. | Image super-resolution using knowledge distillation | |
| CN107123089B (en) | Remote sensing image super-resolution reconstruction method and system based on depth convolution network | |
| CN110363716B (en) | High-quality reconstruction method for generating confrontation network composite degraded image based on conditions | |
| CN112489164B (en) | Image coloring method based on improved depth separable convolutional neural network | |
| CN110570363A (en) | Image defogging method based on Cycle-GAN with pyramid pooling and multi-scale discriminator | |
| CN111275171B (en) | Small target detection method based on parameter sharing multi-scale super-division reconstruction | |
| Chen et al. | Single image super-resolution using deep CNN with dense skip connections and inception-resnet | |
| CN107423747A (en) | A kind of conspicuousness object detection method based on depth convolutional network | |
| CN109598676A (en) | A kind of single image super-resolution method based on Hadamard transform | |
| CN109636721A (en) | Video super-resolution method based on confrontation study and attention mechanism | |
| CN108734675A (en) | Image recovery method based on mixing sparse prior model | |
| CN104408697B (en) | Image Super-resolution Reconstruction method based on genetic algorithm and canonical prior model | |
| CN107240136A (en) | A kind of Still Image Compression Methods based on deep learning model | |
| CN113538234A (en) | Remote sensing image super-resolution reconstruction method based on lightweight generation model | |
| CN113870124B (en) | Weak supervision-based double-network mutual excitation learning shadow removing method | |
| CN111461325A (en) | Multi-target layered reinforcement learning algorithm for sparse rewarding environment problem | |
| CN116822382B (en) | Sea surface temperature prediction method and network based on space-time multiple characteristic diagram convolution | |
| CN116311186A (en) | Plant leaf lesion identification method based on improved transducer model | |
| CN112560624A (en) | High-resolution remote sensing image semantic segmentation method based on model depth integration | |
| CN115601236A (en) | Remote sensing image super-resolution reconstruction method based on characteristic information distillation network | |
| Zhou et al. | MSAR‐DefogNet: Lightweight cloud removal network for high resolution remote sensing images based on multi scale convolution | |
| CN112686817B (en) | Image completion method based on uncertainty estimation | |
| CN115797181A (en) | Image super-resolution reconstruction method for mine fuzzy environment | |
| CN116206214A (en) | Automatic landslide recognition method, system, equipment and medium based on lightweight convolutional neural network and double attention |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190910 |