CN113034386B - Image processing method, system and medium based on deep neural network - Google Patents
Image processing method, system and medium based on deep neural network Download PDFInfo
- Publication number
- CN113034386B CN113034386B CN202110234328.XA CN202110234328A CN113034386B CN 113034386 B CN113034386 B CN 113034386B CN 202110234328 A CN202110234328 A CN 202110234328A CN 113034386 B CN113034386 B CN 113034386B
- Authority
- CN
- China
- Prior art keywords
- image
- neural network
- deep neural
- network
- blurred image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 50
- 238000003672 processing method Methods 0.000 title claims abstract description 16
- 238000012549 training Methods 0.000 claims abstract description 35
- 238000012360 testing method Methods 0.000 claims abstract description 32
- 238000012545 processing Methods 0.000 claims abstract description 17
- 238000000034 method Methods 0.000 claims description 34
- 230000000694 effects Effects 0.000 description 11
- 230000006870 function Effects 0.000 description 9
- 238000005070 sampling Methods 0.000 description 7
- 238000013527 convolutional neural network Methods 0.000 description 6
- 238000000605 extraction Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 5
- 238000011160 research Methods 0.000 description 5
- 238000013135 deep learning Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000008034 disappearance Effects 0.000 description 4
- 238000011176 pooling Methods 0.000 description 3
- 239000000654 additive Substances 0.000 description 2
- 230000000996 additive effect Effects 0.000 description 2
- 238000007796 conventional method Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000003062 neural network model Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000010355 oscillation Effects 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 238000010200 validation analysis Methods 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 description 1
- 238000007476 Maximum Likelihood Methods 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000003321 amplification Effects 0.000 description 1
- 238000010009 beating Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 238000000275 quality assurance Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G06T5/73—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Abstract
The invention provides an image processing method, system and medium based on a deep neural network. The image processing is to restore motion blurred images. The image processing method comprises the following steps: step S1, a sample database is established, wherein the sample database stores the motion blurred image and comprises a training set and a testing set; step S2, constructing and optimizing the deep neural network for restoring the motion blurred image based on a twin network framework of a ResNet residual network by utilizing the training set; and S3, restoring the motion blurred image in the test set by using the deep neural network.
Description
Technical Field
The present invention relates to the field of image processing, and in particular, to an image processing method, system and medium based on a deep neural network.
Background
Image restoration techniques aim to repair degraded low quality images, thereby obtaining more effective information. The existing restoration method has certain limitations such as slow restoration speed, low restoration definition and the like when realizing the restoration of the motion blurred image. When a clear image is collected at ordinary times, the image is blurred due to interference of various factors, such as when a certain severe weather condition or a shooting scene with poor light is met, and a certain relative motion is generated between a camera lens and the shooting scene due to factors such as shaking of the lens during shooting, so that the image is blurred.
The image restoration technique aims at restoring a low-quality image due to bad environmental factors of a photographing process, and makes the low-quality image clear with best effort, thereby presenting more effective information. The restoration technology of the motion blur image has important roles in various industries, such as license plate number resolution, face recognition and monitoring system recognition, and the fields of traffic safety, photographic work, target object recognition and tracking, astronomy, medical science, industrial control and the like.
In the following restoration technique, deep learning is increasingly dominant, and can quickly and efficiently restore low-quality images, and the restoration method is more diversified than the traditional method. Meanwhile, the deep learning method is adopted to restore the motion blurred image, so that the problems of large calculated amount, large network parameter training difficulty and the like exist, the method is limited to the performance of computer hardware equipment, and the development speed of the deep learning is slow.
At present, a plurality of remarks about motion blur image restoration technology at home and abroad exist, and the existing traditional restoration method is generally adopted and is based on an image restoration method proposed by the American Massachu institute of technology, and the method is called a maximum posterior probability method. Based on the research of the above method, the related research of the problem of blurred image restoration has been further developed. In the early stage, lucy and Richardson respectively proposed a Richardson-Lucy algorithm, and the algorithm achieves the purpose of recovering a blurred image mainly by constructing noise in the blurred image into poisson distribution and then calculating the value of a maximum likelihood function. Fergus et al independently established a model called the variational Bayesian, and used the model to estimate the blur kernel, thereby realizing the deblurring operation of the blurred image. In the conventional methods, for example, R-L deconvolution methods, these methods have the defect of serious Ringing effect (Ringing Artifact, which means oscillation generated at a place where the gray value of the output image varies greatly, such as air oscillation wave generated at the time of beating clock). Therefore, shan et al have made a certain study on this basis and found the intrinsic cause of the problem, namely, a large error is generated when the fuzzy kernel is calculated. After knowing the cause of this effect, the Shan et al have made further studies to make it possible to well suppress the ringing effect in the restored image, but there is still a phenomenon of slow convergence speed, and at the same time, the parameter setting is also affected to some extent.
Deep learning currently uses restored blurred images in many industries and achieves more remarkable research results than the research of the conventional method of image restoration. For example schulter et al propose using two layers of CNN to extract features of the training image, and further repairing the image with these features as a priori knowledge. Xu and Jia et al propose a neural network of non-blind deconvolution, which also first performs blur kernel extraction on the blurred image and then recovers the blurred image as a priori knowledge, differing from schulter et al in that a priori knowledge is processed. Jain et al use a 4-layer CNN model to apply to restoration of blurred images, but the experimental result shows that the CNN with a shallower network layer number has a poor effect on image restoration.
Disclosure of Invention
The invention aims to provide an image processing scheme based on a deep neural network so as to solve the technical problems in the prior art.
The first aspect of the present invention provides an image processing method based on a deep neural network, the image processing being to restore motion blurred images, the image processing method comprising: step S1, a sample database is established, wherein the sample database stores the motion blurred image and comprises a training set and a testing set; step S2, constructing and optimizing the deep neural network for restoring the motion blurred image based on a twin network framework of a ResNet residual network by utilizing the training set; and S3, restoring the motion blurred image in the test set by using the deep neural network.
According to the method provided by the first aspect of the present invention, in the step S1, the motion blurred image is acquired by using the following method: cutting the original image to obtain an image block with definition meeting preset requirements; performing convolution operation on the image block and the fuzzy kernel; and adding Gaussian noise into the convolved image to generate the motion blurred image.
According to the method provided by the first aspect of the present invention, the twin network has two branches, and the two branches both adopt a res net residual network, and the step S2 specifically includes: extracting features of the blur kernel by using one of the two branches, and extracting features of the blur image by using the other of the two branches; performing convolution operation on the features of the blur kernel and the features of the blurred image; and adjusting parameters of the deep neural network so that the convolved image is consistent with the motion blurred image to optimize the deep neural network.
According to the method provided by the first aspect of the invention, in step S3, the motion blurred image in the test set is amplified using an upsampling operation, said upsampling operation having an interpolation value.
A second aspect of the present invention provides an image processing system based on a deep neural network, the image processing being to restore motion blurred images, the image processing system comprising: a first module configured to build a sample database storing the motion blurred image, including a training set and a testing set; a second module configured to construct and optimize the deep neural network for restoring the motion blurred image based on a twin network framework of a res net residual network using the training set; and a third module configured to reconstruct a motion blurred image in the test set using the deep neural network.
According to a second aspect of the present invention, the first module is specifically configured to obtain the motion blurred image by: cutting the original image to obtain an image block with definition meeting preset requirements; performing convolution operation on the image block and the fuzzy kernel; and adding Gaussian noise into the convolved image to generate the motion blurred image.
According to a system provided by the second aspect of the present invention, the twin network has two branches, the two branches each employ a res net residual network, and the second module is specifically configured to: extracting features of the blur kernel by using one of the two branches, and extracting features of the blur image by using the other of the two branches; performing convolution operation on the features of the blur kernel and the features of the blurred image; and adjusting parameters of the deep neural network so that the convolved image is consistent with the motion blurred image to optimize the deep neural network.
According to a system provided by the second aspect of the invention, the third module is specifically configured to amplify the motion blurred image in the test set with an upsampling operation, the upsampling operation having an interpolation value.
A third aspect of the invention provides a non-transitory computer readable medium storing instructions which, when executed by a processor, perform steps in a deep neural network based image processing method according to the first aspect of the invention.
In summary, the method realizes the end-to-end non-blind restoration network with known fuzzy kernels by performing feature extraction on the fuzzy kernels and the fuzzy images and deconvolution operation on the fuzzy images by using the twin network. The twin network is composed of a pair of ResNet networks of improved versions of unshared weights, the blurred image is subjected to non-blind deconvolution operation, restoration of the blurred image is achieved, and meanwhile the size of an output image is adjusted through up-sampling operation, so that the output size is identical to the input size; the loss function uses a mean square error function to avoid the disappearance of gradients to train the network. The method has the advantages that the fuzzy picture is improved to a great extent, the restoration effect is obvious, and the restoration speed is high.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings which are required in the description of the embodiments or the prior art will be briefly described, it being obvious that the drawings in the description below are some embodiments of the invention and that other drawings may be obtained from these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic diagram of a flow of an image processing method based on a deep neural network according to an embodiment of the present invention;
FIGS. 2a-2c are training results according to embodiments of the present invention;
FIGS. 2d, 2f, 2h are motion blurred images in a test set in accordance with an embodiment of the present invention;
FIGS. 2e, 2g, 2i are restored images according to embodiments of the present invention; and
fig. 3 is a block diagram of an image processing system based on a deep neural network according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made apparent and fully in view of the accompanying drawings, in which some, but not all embodiments of the invention are shown. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The first aspect of the invention provides an image processing method based on a deep neural network, wherein the image processing is to restore motion blur. Fig. 1 is a schematic diagram of a flow of an image processing method based on a deep neural network according to an embodiment of the present invention, as shown in fig. 1, the image processing method includes: step S1, a sample database is established, wherein the sample database stores the motion blurred image and comprises a training set and a testing set; step S2, constructing and optimizing the deep neural network for restoring the motion blurred image based on a twin network framework of a ResNet residual network by utilizing the training set; and S3, restoring the motion blurred image in the test set by using the deep neural network.
In step S1, a sample database is established, said sample database storing said motion blurred image, comprising a training set and a test set. Wherein the motion blurred image is acquired using the following method: cutting the original image to obtain an image block with definition meeting preset requirements; performing convolution operation on the image block and the fuzzy kernel; and adding Gaussian noise into the convolved image to generate the motion blurred image.
In order to generate enough motion blur images for training, the training set of the sample database uses a BSDS500 data set, and the data set not only has the support of the number of image samples, but also has quality assurance, thereby providing a data base for the research of a plurality of students. This dataset had a total of 500 sharp images, which 500 sharp images were randomly cropped into image blocks of size 225 x 3 pixels as sharp images. The blur kernel is generated according to the method of the blur kernel made by a. Chakrabarti in its literature, and has a size of 113×113×3. After the generated blur kernel is obtained, the clear image is convolved with the blur kernel, and additive Gaussian noise is added to simulate the required motion blur image. The test set includes 80 clear images and 8 blur kernels as clear images and blur kernels of the test data. Thus, there are 640 blurred images in total. Whether the training set or the test set, after obtaining the blurred image, 0.1% gaussian noise is added to it for simulating the motion blurred image.
In step S2, the deep neural network for restoring the motion blurred image is constructed and optimized based on a twin network framework of a res net residual network using the training set. The twin network has two branches, both of which employ a ResNet residual network. The step S2 specifically includes: extracting features of the blur kernel by using one of the two branches, and extracting features of the blur image by using the other of the two branches; performing convolution operation on the features of the blur kernel and the features of the blurred image; and adjusting parameters of the deep neural network so that the convolved image is consistent with the motion blurred image to optimize the deep neural network.
In order to obtain an effective and rapid neural network model for restoring a motion blurred image, a known blurred image restoration method based on a twin network blur kernel is provided. Firstly, designing a twin network framework which is not shared by weight values, wherein the framework mainly comprises two parts, each part is formed by a ResNet residual network as a network structure, compared with the ResNet34 structure, the two residual networks have certain modification, one branch of the twin network is used for realizing the feature extraction of a fuzzy core, the other branch is used for carrying out the feature extraction of a fuzzy image, carrying out convolution operation on the feature images extracted by the two parts, and further adjusting the size of a convolution result feature map to ensure that the output restored image is identical with the size of an input image; and then adopting a mean square error optimization network to train, and adding an image fidelity term to improve the restoration effect.
Firstly, taking the manufactured fuzzy image and the fuzzy kernel as the input of a neural network, carrying out feature extraction on the input image through a twin network, extracting a feature map of the fuzzy image from the fuzzy image, and extracting a feature map of the fuzzy kernel from the fuzzy kernel; then, carrying out convolution operation on the two middle feature images, and adjusting the number of channels to obtain more feature images; and finally, an up-sampling operation is adopted for the obtained characteristic image, wherein the purpose of the up-sampling operation is not just to enlarge the image. The up-sampling operation also improves the quality of the image to some extent by using an interpolation method, resulting in a restored image of better quality, outputting a restored clear image of the same size as the original input blurred image. The whole process belongs to a deconvolution process, and is a recovery process of non-blind deconvolution because the blur kernel of a blurred image is known.
In some embodiments, the twin network: i.e. blurred images restore the first half of the network structure. The multi-branch parameter non-sharing twin network structure is provided with two input ends, the upper end is a PSF of 3 multiplied by 113, the PSF is used as input by fuzzy cores in a data set, and a training set and a testing set respectively use different fuzzy cores; the lower end is a 3×225×225 blurred image, which is obtained by convolving a clear image in a dataset with a blur kernel and adding additive noise, and the two inputs respectively pass through a CNN network with unshared parameters.
Wherein the CNN network is composed of a modified version of the res net network.
TABLE 1
As shown in table 1, the res net used consisted essentially of one first layer and three layer layers. The first layer includes 3 blocks of residual blocks, the second includes 4 blocks, and the third includes 6 blocks. Each layer will initially amplify the channel count of the feature map, thereby extracting more features.
The twin network part constructs a convolutional neural network consisting of a convolutional layer, a pooling layer and a residual layer. Except for the first layer which has a pooling layer, none of the other layers has a pooling layer, and after each convolution is completed, BN layers are added and the ReLU function is used as an activation function. The BN layer is added, so that the overfitting can be effectively inhibited, and the network learning speed can be increased. The twin network framework uses a res net residual network as the neural network structure. When the number of network layers is deepened, the training error becomes larger, the phenomenon of gradient disappearance is more obvious, and the problem of gradient disappearance can be effectively avoided by using a residual network as a neural network structure. The two input ends of the twin network are used for extracting the characteristics of the images to obtain two characteristic images with the sizes of 8 multiplied by 256 and 15 multiplied by 256, the two characteristic images are subjected to convolution operation and the number of channels is increased to obtain a characteristic image with the size of 8 multiplied by 512, and the characteristic image is used as the input of the up-sampling process.
And performing size interpolation amplification on the input feature image through an up-sampling stage to obtain a restored image with the same size as the original blurred image. The loss values are then calculated for back propagation to adjust the parameters. And repeatedly iterating the process, and finally training to obtain a group of weight parameters with lower loss values.
TABLE 2
As shown in table 2, the parameter setting for the network structure is specifically composed of 5 image amplifying operations, each of which is composed of one sample, one reflection pad, and one conv operation. The up sample operation can double the width and the height of the image, and the channel number is unchanged; the reflection pad operation adds a pad to the outermost side of the feature map, namely, the width and the height are increased by 2 respectively; the conv is a convolution operation with a convolution kernel with size of 4×4, padding of 0, stride of 1, in order to adjust the size of the feature map, the five convolution kernels are all the same in size, but the number of output channels is changed accordingly.
At step S3, the motion blurred image in the test set is restored using the deep neural network. In particular, the motion blurred image in the test set is amplified with an upsampling operation having an interpolation value.
Specific test examples
Based on the method of the first aspect of the disclosure, in this example, a deep neural network model for restoring motion blurred images is built, and three network models are trained for such network structures.
The first network model (model 1), the initial learning rate is set to 0.001, 300 epochs are iterated in total, the learning rate of the first 200 epochs is 0.001, and the last 100 iterations are periodically decreased; a second network model (model 2), wherein the initial learning rate is set to be 0.01, the total iteration of the network is 300 epochs, and the learning rate is fixed; the third network model (model 3), the initial learning rate was set to 0.001 and was fixed, and the network trained a total of 500 epochs. The lower graph is a Loss-epoch change graph of three network models on the training set and the validation set. All three models use Adam optimizers, and the loss function uses root mean square error function. In addition, the training sets are all 1 ten thousand blurred pictures. FIGS. 2a-2c show training results according to an embodiment of the present invention, as shown in FIG. 2a, model1 has convergence on the training set, but the convergence effect is not very obvious, and the convergence trend is not shown on the test set, so that the training set is analyzed and no preprocessing operation is found on the data, and thus the training error can be considered. After the network model parameters are adjusted and the data preprocessing operation is added, two groups of network model parameters are trained, as shown in fig. 2b and 2c, the convergence effect of the model2 and the model3 on the verification set is greatly improved compared with that of the model1, and the network on the training set has obvious convergence trend. Comparing model2 with model3, the clarity of the restored pictures of model3 is better than model2, and in order to enhance the restoring effect, 200 epochs are continuously trained on the basis of the original model3, so that the network is trained for 500 times in total.
From the training set and validation set Loss-epoch plots trained on the three models above, model1 had a Loss of 0.03158 at 300 epochs, model2 had a Loss of 0.01409 at 300 epochs, and model3 had a Loss of 0.01121 at 300 epochs. It can be seen that model1 has the slowest convergence rate on the training set, and that model3 has a better convergence rate than model2, and that model2 is centered. From this, model3 is the model choice that is currently more suitable for convergence effects.
And repeatedly training and modifying the selected network model3 through multiple optimization and adjustment, so that the average loss value of the model on a test set is 0.0099, and storing the model reaching the lowest loss value, thereby facilitating the call when the motion blurred image is restored. And applying the selected model to the test set, and performing restoration operation on the blurred image. Referring specifically to fig. 2d-2e, 2f-2g and 2h-2i, fig. 2d, 2f, 2h are motion blurred images in the test set, and fig. 2e, 2g, 2i are restored images.
In summary, the method of the first aspect of the disclosure implements an end-to-end non-blind restoration network where the blur kernel is known by performing feature extraction on the blur kernel and the blur image using a twin network, and performing a deconvolution operation on the blur image. The twin network is composed of a pair of ResNet networks of improved versions of unshared weights, the blurred image is subjected to non-blind deconvolution operation, restoration of the blurred image is achieved, and meanwhile the size of an output image is adjusted through up-sampling operation, so that the output size is identical to the input size; the loss function uses a mean square error function to avoid the disappearance of gradients to train the network. The method has the advantages that the fuzzy picture is improved to a great extent, the restoration effect is obvious, and the restoration speed is high.
A second aspect of the present invention provides an image processing system based on a deep neural network, fig. 3 is a block diagram of the image processing system based on the deep neural network according to an embodiment of the present invention, as shown in fig. 3, the image processing is to restore a motion blurred image, and the image processing system 300 includes: a first module 301 configured to build a sample database storing the motion blurred image, comprising a training set and a test set; a second module 302 configured to construct and optimize the deep neural network for restoring the motion blurred image based on a twin network framework of a res net residual network using the training set; and a third module 303 configured to reconstruct a motion blurred image in the test set using the deep neural network.
According to the system provided by the second aspect of the present invention, the first module 301 is specifically configured to obtain the motion blurred image by: cutting the original image to obtain an image block with definition meeting preset requirements; performing convolution operation on the image block and the fuzzy kernel; and adding Gaussian noise into the convolved image to generate the motion blurred image.
According to the system provided by the second aspect of the present invention, the twin network has two branches, the two branches each employ a res net residual network, and the second module 302 is specifically configured to: extracting features of the blur kernel by using one of the two branches, and extracting features of the blur image by using the other of the two branches; performing convolution operation on the features of the blur kernel and the features of the blurred image; and adjusting parameters of the deep neural network so that the convolved image is consistent with the motion blurred image to optimize the deep neural network.
According to the system provided by the second aspect of the invention, the third module 303 is specifically configured to amplify the motion blurred image in the test set with an upsampling operation having an interpolation value.
A third aspect of the invention provides a non-transitory computer readable medium storing instructions which, when executed by a processor, perform steps in a deep neural network based image processing method according to the first aspect of the invention.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the invention.
Claims (3)
1. An image processing method based on a deep neural network, wherein the image processing is to restore motion blur, the image processing method comprising:
step S1, a sample database is established, wherein the sample database stores the motion blurred image and comprises a training set and a testing set;
wherein the motion blurred image is acquired using the following method:
cutting the original image to obtain an image block with definition meeting preset requirements;
performing convolution operation on the image block and the fuzzy kernel; and
adding Gaussian noise into the convolved image to generate the motion blurred image;
step S2, constructing and optimizing the deep neural network for restoring the motion blurred image based on a twin network framework of a ResNet residual network by utilizing the training set;
the twin network has two branches, both of which adopt a res net residual network, and the step S2 specifically includes:
extracting features of the blur kernel by using one of the two branches, and extracting features of the blur image by using the other of the two branches;
performing convolution operation on the features of the blur kernel and the features of the blurred image;
adjusting parameters of the deep neural network to enable the convolved image to be consistent with the motion blurred image so as to optimize the deep neural network;
s3, restoring the motion blurred image in the test set by using the deep neural network;
wherein the motion blurred image in the test set is amplified using an upsampling operation, said upsampling operation having an interpolated value.
2. An image processing system based on a deep neural network, wherein the image processing is to recover motion blur, the image processing system comprising:
a first module configured to build a sample database storing the motion blurred image, including a training set and a testing set;
the first module is specifically configured to acquire the motion blurred image using:
cutting the original image to obtain an image block with definition meeting preset requirements;
performing convolution operation on the image block and the fuzzy kernel; and
adding Gaussian noise into the convolved image to generate the motion blurred image;
a second module configured to construct and optimize the deep neural network for restoring the motion blurred image based on a twin network framework of a res net residual network using the training set;
the twin network has two branches, both of which employ a res net residual network, the second module being specifically configured to:
extracting features of the blur kernel by using one of the two branches, and extracting features of the blur image by using the other of the two branches;
performing convolution operation on the features of the blur kernel and the features of the blurred image;
adjusting parameters of the deep neural network to enable the convolved image to be consistent with the motion blurred image so as to optimize the deep neural network;
a third module configured to reconstruct a motion blurred image in the test set using the deep neural network;
the third module is specifically configured to amplify the motion blurred image in the test set with an upsampling operation having an interpolation value.
3. A non-transitory computer readable medium storing instructions which, when executed by a processor, perform the steps in the deep neural network based image processing method of claim 1.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110234328.XA CN113034386B (en) | 2021-03-03 | 2021-03-03 | Image processing method, system and medium based on deep neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110234328.XA CN113034386B (en) | 2021-03-03 | 2021-03-03 | Image processing method, system and medium based on deep neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113034386A CN113034386A (en) | 2021-06-25 |
| CN113034386B true CN113034386B (en) | 2024-01-23 |
Family
ID=76465784
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110234328.XA Active CN113034386B (en) | 2021-03-03 | 2021-03-03 | Image processing method, system and medium based on deep neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113034386B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018045602A1 (en) * | 2016-09-07 | 2018-03-15 | 华中科技大学 | Blur kernel size estimation method and system based on deep learning |
| CN111462019A (en) * | 2020-04-20 | 2020-07-28 | 武汉大学 | Image deblurring method and system based on deep neural network parameter estimation |
-
2021
- 2021-03-03 CN CN202110234328.XA patent/CN113034386B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018045602A1 (en) * | 2016-09-07 | 2018-03-15 | 华中科技大学 | Blur kernel size estimation method and system based on deep learning |
| CN111462019A (en) * | 2020-04-20 | 2020-07-28 | 武汉大学 | Image deblurring method and system based on deep neural network parameter estimation |
Non-Patent Citations (2)
| Title |
|---|
| 吴梦婷 ; 李伟红 ; 龚卫国 ; .双框架卷积神经网络用于运动模糊图像盲复原.计算机辅助设计与图形学学报.2018,(12),全文. * |
| 高新波 ; 路文 ; 查林 ; 惠政 ; 亓统帅 ; 姜建德 ; .超高清视频画质提升技术及其芯片化方案.重庆邮电大学学报(自然科学版).2020,(05),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113034386A (en) | 2021-06-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Liang et al. | Details or artifacts: A locally discriminative learning approach to realistic image super-resolution | |
| CN111028177B (en) | Edge-based deep learning image motion blur removing method | |
| CN108596841B (en) | Method for realizing image super-resolution and deblurring in parallel | |
| CN111275643B (en) | Real noise blind denoising network system and method based on channel and space attention | |
| CN112419184B (en) | Spatial attention map image denoising method integrating local information and global information | |
| CN114140353A (en) | Swin-Transformer image denoising method and system based on channel attention | |
| CN110060204B (en) | Single image super-resolution method based on reversible network | |
| CN111223062A (en) | Image deblurring method based on generation countermeasure network | |
| CN113421187B (en) | Super-resolution reconstruction method, system, storage medium and equipment | |
| CN113837959B (en) | Image denoising model training method, image denoising method and system | |
| CN113450290A (en) | Low-illumination image enhancement method and system based on image inpainting technology | |
| CN113191983A (en) | Image denoising method and device based on deep learning attention mechanism | |
| CN112598587A (en) | Image processing system and method combining face mask removal and super-resolution | |
| CN110503608B (en) | Image denoising method based on multi-view convolutional neural network | |
| Bai et al. | MSPNet: Multi-stage progressive network for image denoising | |
| CN113096032B (en) | Non-uniform blurring removal method based on image region division | |
| CN113034386B (en) | Image processing method, system and medium based on deep neural network | |
| CN116883265A (en) | Image deblurring method based on enhanced feature fusion mechanism | |
| CN101567079B (en) | Method for restoring motion blurred image based on Hopfield neural network | |
| CN116091315A (en) | Face super-resolution reconstruction method based on progressive training and face semantic segmentation | |
| CN116188313A (en) | Dynamic scene blind deblurring method based on asymmetric U-Net network | |
| CN115345791A (en) | Infrared image deblurring algorithm based on attention mechanism residual error network model | |
| CN112016456B (en) | Video super-resolution method and system based on adaptive back projection depth learning | |
| CN111626945B (en) | Depth image restoration method based on pixel-level self-similar model | |
| Hou et al. | Residual dilated network with attention for image blind denoising |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |