CN108550118B - Motion blur image blur processing method, device, equipment and storage medium - Google Patents
Motion blur image blur processing method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN108550118B CN108550118B CN201810240092.9A CN201810240092A CN108550118B CN 108550118 B CN108550118 B CN 108550118B CN 201810240092 A CN201810240092 A CN 201810240092A CN 108550118 B CN108550118 B CN 108550118B
- Authority
- CN
- China
- Prior art keywords
- network
- blurred image
- image
- motion
- unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000003672 processing method Methods 0.000 title abstract description 11
- 238000012545 processing Methods 0.000 claims abstract description 62
- 230000005284 excitation Effects 0.000 claims abstract description 35
- 230000006835 compression Effects 0.000 claims abstract description 20
- 238000007906 compression Methods 0.000 claims abstract description 20
- 238000000605 extraction Methods 0.000 claims abstract description 19
- 238000000034 method Methods 0.000 claims abstract description 18
- 238000012549 training Methods 0.000 claims description 36
- 238000007781 pre-processing Methods 0.000 claims description 25
- 238000004590 computer program Methods 0.000 claims description 14
- 230000003042 antagnostic effect Effects 0.000 claims 2
- 230000000694 effects Effects 0.000 abstract description 13
- 238000010586 diagram Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000011176 pooling Methods 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/047—Probabilistic or stochastic networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/60—Image enhancement or restoration using machine learning, e.g. neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20172—Image enhancement details
- G06T2207/20201—Motion blur correction
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Probability & Statistics with Applications (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention is suitable for the technical field of image processing, and provides a motion blurred image blurring processing method, a device, equipment and a storage medium, wherein the method comprises the following steps: when a blurring processing request for a motion blurred image is received, the motion blurred image is input into a generator of a pre-trained enhanced generation countermeasure network, the generator comprises a compression excitation residual error network unit and a scaling convolution unit, the motion blurred image is subjected to feature extraction through the compression excitation residual error network unit to obtain a feature image corresponding to the motion blurred image, and the feature image is subjected to blurring processing through the scaling convolution unit to obtain a clear image corresponding to the motion blurred image, so that a chessboard effect in blurring processing of the motion blurred image is reduced, the restoring definition of the motion blurred image is improved, and the generalization performance of the enhanced generation countermeasure network is improved.
Description
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to a motion blur image blur processing method, device, equipment and storage medium.
Background
Motion blur is a ubiquitous phenomenon in an imaging process, when images are shot on an airplane or a running automobile in a walking process, the relative speed of a shot object is too high or the image is shaken during shooting, motion blur is generated, and the motion blur of the images has a serious influence on application in the fields of astronomy, military, road traffic, medical images and the like, so that the motion blur removal of the images is always an important problem in the field of computer vision.
With the rise of artificial intelligence algorithms represented by deep learning algorithms, research fields such as image processing, image recognition, language signal processing, natural language processing and the like have been rapidly developed, and research on blurred image restoration by using the deep learning algorithms has also achieved some achievements, for example, generation of a countermeasure network (GAN) proposed by students such as Ian Goodfellow in 2014, and once proposed, the method has become one of the most popular research directions in the deep learning field. Subsequently, the university of OrestKupyn, et al, in the 2017 paper, proposed an end-to-end learning method based on a conditional-impedance generation network (CGAN) and content loss (content loss), which is a network model of the DeblurGAN network that is obvious in the effect of removing the blurring phenomenon caused by the motion of the object on the image, however, the network model of the DeblurGAN network may generate a "checkerboard-like artifact" (i.e., a checkerboard effect) after removing the motion-blurred image, especially the dark part of the image, and thus the restoration effect of the motion-blurred image is affected.
Disclosure of Invention
The invention aims to provide a motion blurred image blurring processing method, a motion blurred image blurring processing device, motion blurred image blurring processing equipment and a storage medium, and aims to solve the problems that a chessboard effect is obvious and user experience is poor after blurring a motion blurred image due to the fact that an effective motion blurred image blurring processing method cannot be provided in the prior art.
In one aspect, the present invention provides a method for blur processing of a motion-blurred image, the method comprising the steps of:
when a blurring processing request for a motion blurred image is received, inputting the motion blurred image into a generator of a pre-trained enhanced generation countermeasure network, wherein the generator comprises a compression excitation residual error network unit and a scaling convolution unit;
performing feature extraction on the motion blurred image through the compressed excitation residual error network unit to obtain a feature image corresponding to the motion blurred image;
and carrying out blurring processing on the characteristic image through the scaling convolution unit to obtain a clear image corresponding to the motion blurred image.
In another aspect, the present invention provides a motion-blurred image blur processing apparatus, including:
the image input unit is used for inputting the motion blurred image into a generator of a pre-trained enhanced generation countermeasure network when a blurring processing request for the motion blurred image is received, and the generator comprises a compression excitation residual error network unit and a scaling convolution unit;
the characteristic extraction unit is used for extracting the characteristics of the motion blurred image through the compressed excitation residual error network unit to obtain a characteristic image corresponding to the motion blurred image; and
and the blurring processing unit is used for blurring the characteristic image through the scaling convolution unit so as to obtain a clear image corresponding to the motion blurred image.
In another aspect, the present invention also provides a computing device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method when executing the computer program.
In another aspect, the present invention also provides a computer-readable storage medium, in which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of the method as set forth above.
When a fuzzy processing request for a motion blurred image is received, the motion blurred image is input into a generator of a pre-trained enhanced generation countermeasure network, the generator comprises a compression excitation residual error network unit and a scaling convolution unit, the motion blurred image is subjected to feature extraction through the compression excitation residual error network unit to obtain a feature image corresponding to the motion blurred image, and the feature image is subjected to fuzzy processing through the scaling convolution unit to obtain a clear image corresponding to the motion blurred image, so that a chessboard effect in fuzzy processing of the motion blurred image is reduced, the restoration definition of the motion blurred image is improved, and the generalization performance of the enhanced generation countermeasure network is improved.
Drawings
Fig. 1 is a flowchart of an implementation of a blurring processing method for a motion-blurred image according to an embodiment of the present invention;
fig. 2 is a schematic flowchart of feature extraction performed by a compressed excitation residual error network unit in the motion blur processing method for motion blurred images according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a motion-blurred image blur processing apparatus according to a second embodiment of the present invention;
fig. 4 is a schematic structural diagram of a motion-blurred image blur processing apparatus according to a third embodiment of the present invention; and
fig. 5 is a schematic structural diagram of a computing device according to a fourth embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The following detailed description of specific implementations of the present invention is provided in conjunction with specific embodiments:
the first embodiment is as follows:
fig. 1 shows an implementation flow of a motion-blurred image blur processing method provided in an embodiment of the present invention, and for convenience of description, only the parts related to the embodiment of the present invention are shown, which are detailed as follows:
in step S101, when a blur processing request for a motion-blurred image is received, the motion-blurred image is input to a generator of an enhanced generation countermeasure network trained in advance, the generator including a compression excitation residual network unit and a scaling convolution unit.
The embodiment of the invention is suitable for computing equipment, such as a personal computer, a smart phone, a tablet and the like. Before the motion blurred image is input into a generator of a pre-trained enhanced generation countermeasure network (EDGAN), preferably, a preset formatting preprocessing layer is used for formatting the motion blurred image, and the motion blurred image subjected to formatting preprocessing is subjected to image enhancement processing to obtain an enhanced motion blurred image, so that the effect of performing blurring processing on the motion blurred image by the enhanced generation countermeasure network is improved.
Further preferably, the preset formatting preprocessing layer performs formatting preprocessing on the motion blurred image by using a three-dimensional Block Matching (Block-formatting 3D, BM3D for short) or an extended Patch Log Likelihood expectation (EPLL for short) or Weighted Nuclear Norm Minimization (WNNM) denoising algorithm, so as to denoise the motion blurred image, thereby improving the degree of significance of detail features in the motion blurred image.
Still preferably, the pre-trained enhanced generation countermeasure network is set as a formatting preprocessing layer, and the motion-blurred image is subjected to formatting preprocessing so as to be denoised, thereby improving the effect of noise filtering of the motion-blurred image.
Further preferably, when the motion-blurred image subjected to the formatting preprocessing is subjected to image enhancement processing, the image may be scaled by a preset resolution scaling ratio, and then the scaled image is randomly cropped to obtain an enhanced motion-blurred image, for example, the blurred image with 2560 × 720 resolution is scaled to 640 × 360 resolution, and then is randomly cropped to an image with 256 × 256 resolution, so as to improve the training efficiency of the enhanced generation of the countermeasure network according to the embodiment of the present invention, and improve the efficiency of the subsequent blurring processing on the motion-blurred image.
When a blurring processing request for a motion blurred image is received, before the motion blurred image is input into a generator of a pre-trained enhanced generation countermeasure network, preferably, two generation countermeasure networks are constructed in advance, each generation countermeasure network comprises two convolution units, 9 compression excitation residual error network units and two scaling convolution units, so that generalization performance of the network is improved, and the pre-trained enhanced generation countermeasure network can be obtained through training of the two generation countermeasure networks. For ease of description, the two generative countermeasure networks are referred to herein as a first generative countermeasure network and a second generative countermeasure network.
Further preferably, when two generators for generating the countermeasure network are respectively constructed, all the Batch Norm or the Instance Norm layers in the generators are removed, so that the training speed of the network and the stability of the network are improved, and the original contrast information of the image is prevented from being damaged.
After the two generative confrontation networks are initially constructed, the two generative confrontation networks are trained to obtain a pre-trained enhanced generative confrontation network. Preferably, when two generative confrontation networks are trained to obtain a pre-trained enhanced generative confrontation network, a first training sample is input into a pre-constructed first generative confrontation network for training to obtain a trained first generative confrontation network, a second training sample is subjected to formatting preprocessing by the trained first generative confrontation network, the second training sample subjected to formatting preprocessing is subjected to image enhancement processing to obtain an enhanced second training sample, the enhanced second training sample is input into the pre-constructed second generative confrontation network for training to obtain a trained second generative confrontation network, and the trained second generative confrontation network is set as the enhanced generative confrontation network (i.e., the pre-trained enhanced generative confrontation network). The enhanced generation countermeasure network trained by the embodiment of the invention carries out fuzzy processing on the motion blurred image, and improves the peak signal-to-noise ratio (PSNR) and the Structural Similarity (SSIM) of the restored motion blurred image.
In step S102, feature extraction is performed on the motion-blurred image through the compressed excitation residual error network unit to obtain a feature image corresponding to the motion-blurred image.
In the embodiment of the present invention, preferably, the compressively-excited residual network unit is composed of a residual network and a compressively-excited network, the residual network does not include a Batch Norm layer or an Instance Norm layer, and the compressively-excited network includes a global average pooling layer, 2 full-link layers, a Relu layer, and a Sigmoid layer, so as to improve the performance of feature extraction.
As an example, fig. 2 is a schematic flow diagram of feature extraction performed by a compressed excitation residual network unit, where an image input to the compressed excitation residual network unit (SE-ResBlock) is first convolved by two convolution layers (3 × 3Conv) with convolution kernel sizes of 3 × 3 in a residual network and an excitation layer Relu operation connected between the two convolution layers to obtain a preliminary feature image, then the feature image is input to a Global average Pooling layer (Global Pooling) of the compressed excitation network for compression, the compressed feature image is input to a fully connected layer (FC), the fully connected layer reduces the feature dimension of the feature image to 1/16 for input, then the feature dimension of the feature image is increased back to the original dimension by the fully connected layer (FC) after activation by a Relu, and then the feature image is excited by a Sigmoid, and finally, weighting the normalized weight to each channel feature through Scale operation, and finally outputting a feature image corresponding to the motion blurred image, thereby improving the performance of feature extraction.
In step S103, the feature image is blurred by the scaling convolution unit to obtain a sharp image corresponding to the motion-blurred image.
In the embodiment of the present invention, when the feature image is blurred by the scaling convolution unit, preferably, first, the size of the feature image is scaled according to a preset proportion by a preset nearest neighbor interpolation value or a bilinear interpolation value, and then, the scaled feature image is up-sampled by the convolution operation to obtain a sharp image corresponding to the feature image, so that a checkerboard effect in the obtained sharp image is reduced, and the obtained sharp image can be clearly displayed on a display device with a higher resolution.
As an example, table 1 shows peak signal-to-noise ratio (PSNR) and Structural Similarity (SSIM) obtained through experiments of the GOPRO data set and the Lai data set in the dellugan network and the enhanced generative countermeasure network according to the embodiment of the present invention, and it can be seen that the peak signal-to-noise ratio and the structural similarity obtained through the enhanced generative countermeasure network according to the embodiment of the present invention are significantly improved compared with those obtained through the dellugan network.
TABLE 1
In the embodiment of the invention, when a blurring processing request for a motion blurred image is received, the motion blurred image is input into a generator of a pre-trained enhanced generation countermeasure network, the generator comprises a compression excitation residual error network unit and a scaling convolution unit, the motion blurred image is subjected to feature extraction through the compression excitation residual error network unit to obtain a feature image corresponding to the motion blurred image, and the feature image is subjected to blurring processing through the scaling convolution unit to obtain a clear image corresponding to the motion blurred image, so that a chessboard effect in blurring processing of the motion blurred image is reduced, the restoring definition of the motion blurred image is improved, and the generalization performance of the enhanced generation countermeasure network in the embodiment of the invention is improved.
Example two:
fig. 3 shows a structure of a motion-blurred image blur processing apparatus according to a second embodiment of the present invention, and for convenience of description, only the parts related to the second embodiment of the present invention are shown, where the structure includes:
an image input unit 31, configured to input the motion-blurred image into a generator of a pre-trained enhanced generation countermeasure network when a blurring processing request for the motion-blurred image is received, where the generator includes a compression excitation residual network unit and a scaling convolution unit.
The embodiment of the invention is suitable for computing equipment, such as a personal computer, a smart phone, a tablet and the like. When a blurring processing request for a motion blurred image is received, before the motion blurred image is input into a generator of a pre-trained enhanced generation countermeasure network, the enhanced generation countermeasure network is preferably constructed in advance and comprises two convolution units, 9 compression excitation residual error network units and two scaling convolution units, so that the network generalization performance is improved.
Further preferably, when the generator of the enhanced generation countermeasure network is constructed, all the Batch Norm or Instance Norm layers in the generator are removed, so that the training speed of the network and the stability of the network are improved, and the original contrast information of the image is prevented from being damaged.
And the feature extraction unit 32 is configured to perform feature extraction on the motion-blurred image through the compressed excitation residual error network unit to obtain a feature image corresponding to the motion-blurred image.
In the embodiment of the present invention, preferably, the compressively-excited residual network unit is composed of a residual network and a compressively-excited network, the residual network does not include a Batch Norm layer or an Instance Norm layer, and the compressively-excited network includes a global average pooling layer, 2 full-link layers, a Relu layer, and a Sigmoid layer, so as to improve the performance of feature extraction.
And the blurring processing unit 33 is used for performing blurring processing on the characteristic image through the scaling convolution unit to obtain a clear image corresponding to the motion blurred image.
In the embodiment of the present invention, when the feature image is blurred by the scaling convolution unit, preferably, first, the size of the feature image is scaled according to a preset proportion by a preset nearest neighbor interpolation value or a bilinear interpolation value, and then, the scaled feature image is up-sampled by the convolution operation to obtain a sharp image corresponding to the feature image, so that a checkerboard effect in the obtained sharp image is reduced, and the obtained sharp image can be clearly displayed on a display device with a higher resolution.
In the embodiment of the present invention, each unit of the motion blur processing apparatus for motion blur images may be implemented by a corresponding hardware or software unit, and each unit may be an independent software or hardware unit, or may be integrated into a software or hardware unit, which is not limited herein.
Example three:
fig. 4 shows a structure of a motion-blurred image blur processing apparatus according to a third embodiment of the present invention, and for convenience of description, only the parts related to the third embodiment of the present invention are shown, where the structure includes:
a first network training unit 41, configured to input a first training sample into a first generative pairwise anti network constructed in advance for training, so as to obtain a trained first generative pairwise anti network;
a sample preprocessing unit 42, configured to perform formatting preprocessing on the second training sample through the first generative confrontation network;
a sample enhancement unit 43, configured to perform image enhancement processing on the formatted and preprocessed second training sample to obtain an enhanced second training sample;
a second network training unit 44, configured to input the enhanced second training sample into a second generative confrontation network constructed in advance for training, so as to obtain a trained second generative confrontation network, and set the trained second generative confrontation network as an enhanced generative confrontation network (i.e., the enhanced generative confrontation network trained in advance);
a formatting preprocessing unit 45, configured to perform formatting preprocessing on the motion-blurred image through a preset formatting preprocessing layer when a blurring processing request for the motion-blurred image is received;
an image enhancement unit 46, configured to perform image enhancement processing on the motion-blurred image subjected to the formatting preprocessing, so as to obtain an enhanced motion-blurred image;
an image input unit 47, configured to input the enhanced motion-blurred image into a generator of a pre-trained enhanced generation countermeasure network, where the generator includes a compression excitation residual network unit and a scaling convolution unit;
the feature extraction unit 48 is configured to perform feature extraction on the motion-blurred image through the compressed excitation residual error network unit to obtain a feature image corresponding to the motion-blurred image; and
and the blurring processing unit 49 is used for performing blurring processing on the characteristic image through the scaling convolution unit so as to obtain a clear image corresponding to the motion blurred image.
In the embodiment of the present invention, each unit of the motion blur processing apparatus for motion blur images may be implemented by a corresponding hardware or software unit, and each unit may be an independent software or hardware unit, or may be integrated into a software or hardware unit, which is not limited herein. The detailed implementation of each unit can refer to the description of the first embodiment, and is not repeated herein.
Example four:
fig. 5 shows a structure of a computing device according to a fourth embodiment of the present invention, and for convenience of explanation, only a part related to the embodiment of the present invention is shown.
The computing device 5 of an embodiment of the invention comprises a processor 50, a memory 51 and a computer program 52 stored in the memory 51 and executable on the processor 50. The processor 50, when executing the computer program 52, implements the steps in the above-described motion-blurred image blur processing method embodiments, such as steps S101 to S103 shown in fig. 1. Alternatively, the processor 50, when executing the computer program 52, implements the functions of the units in the above-described device embodiments, such as the functions of the units 31 to 33 shown in fig. 3.
In the embodiment of the invention, when a blurring processing request for a motion blurred image is received, the motion blurred image is input into a generator of a pre-trained enhanced generation countermeasure network, the generator comprises a compression excitation residual error network unit and a scaling convolution unit, the motion blurred image is subjected to feature extraction through the compression excitation residual error network unit to obtain a feature image corresponding to the motion blurred image, and the feature image is subjected to blurring processing through the scaling convolution unit to obtain a clear image corresponding to the motion blurred image, so that a chessboard effect in blurring processing of the motion blurred image is reduced, the restoring definition of the motion blurred image is improved, and the generalization performance of the enhanced generation countermeasure network in the embodiment of the invention is improved.
The computing equipment of the embodiment of the invention can be a personal computer, a smart phone and a tablet. The steps implemented when the processor 50 in the computing device 5 executes the computer program 52 to implement the blurring processing method for the motion-blurred image can refer to the description of the foregoing method embodiments, and are not described herein again.
Example five:
in an embodiment of the present invention, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps in the above-described embodiment of the method for blur processing of a motion-blurred image, for example, steps S101 to S103 shown in fig. 1. Alternatively, the computer program may be adapted to perform the functions of the units of the above-described device embodiments, such as the functions of the units 31 to 33 shown in fig. 3, when executed by the processor.
In the embodiment of the invention, when a blurring processing request for a motion blurred image is received, the motion blurred image is input into a generator of a pre-trained enhanced generation countermeasure network, the generator comprises a compression excitation residual error network unit and a scaling convolution unit, the motion blurred image is subjected to feature extraction through the compression excitation residual error network unit to obtain a feature image corresponding to the motion blurred image, and the feature image is subjected to blurring processing through the scaling convolution unit to obtain a clear image corresponding to the motion blurred image, so that a chessboard effect in blurring processing of the motion blurred image is reduced, the restoring definition of the motion blurred image is improved, and the generalization performance of the enhanced generation countermeasure network in the embodiment of the invention is improved.
The computer readable storage medium of the embodiments of the present invention may include any entity or device capable of carrying computer program code, a recording medium, such as a ROM/RAM, a magnetic disk, an optical disk, a flash memory, or the like.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (8)
1. A method of blur processing of a motion blurred image, said method comprising the steps of:
when a blurring processing request for a motion blurred image is received, inputting the motion blurred image into a generator of a pre-trained enhanced generation countermeasure network, wherein the generator comprises a compression excitation residual error network unit and a scaling convolution unit;
performing feature extraction on the motion blurred image through the compressed excitation residual error network unit to obtain a feature image corresponding to the motion blurred image;
blurring the characteristic image through the scaling convolution unit to obtain a clear image corresponding to the motion blurred image;
before the step of inputting the motion-blurred image into the generator of the pre-trained enhanced generation countermeasure network, the method comprises the following steps:
inputting a first training sample into a first pre-constructed generation pairing-defense network for training so as to obtain the trained first generation pairing-defense network;
performing formatting preprocessing on a second training sample through the first generation countermeasure network;
performing image enhancement processing on the second training sample subjected to formatting preprocessing to obtain an enhanced second training sample;
inputting the enhanced second training sample into a second generative confrontation network which is constructed in advance for training to obtain the trained second generative confrontation network, and setting the trained second generative confrontation network as the enhanced generative confrontation network.
2. The method of claim 1, wherein the step of inputting the motion-blurred image into a pre-trained generator of an enhanced generative countermeasure network is preceded by the steps of:
when a blurring processing request for a motion blurred image is received, formatting preprocessing is carried out on the motion blurred image through a preset formatting preprocessing layer;
and performing image enhancement processing on the motion blurred image subjected to the formatting pretreatment to obtain the enhanced motion blurred image.
3. The method of claim 1, wherein the compressed excitation residual network unit consists of a residual network and a compressed excitation network.
4. A blur processing apparatus for a motion-blurred image, the apparatus comprising:
the image input unit is used for inputting the motion blurred image into a generator of a pre-trained enhanced generation countermeasure network when a blurring processing request for the motion blurred image is received, and the generator comprises a compression excitation residual error network unit and a scaling convolution unit;
the characteristic extraction unit is used for extracting the characteristics of the motion blurred image through the compressed excitation residual error network unit to obtain a characteristic image corresponding to the motion blurred image; and
the blurring processing unit is used for blurring the characteristic image through the scaling convolution unit to obtain a clear image corresponding to the motion blurred image;
the device further comprises:
the first network training unit is used for inputting a first training sample into a first constructed generative antagonistic network for training so as to obtain a trained first generative antagonistic network;
the sample preprocessing unit is used for carrying out formatting preprocessing on a second training sample through the first generation pairing-resisting network;
the sample enhancement unit is used for carrying out image enhancement processing on the second training sample subjected to formatting preprocessing so as to obtain an enhanced second training sample; and
and the second network training unit is used for inputting the enhanced second training sample into a second generative confrontation network which is constructed in advance for training so as to obtain the trained second generative confrontation network, and setting the trained second generative confrontation network as the enhanced generative confrontation network.
5. The apparatus of claim 4, wherein the apparatus further comprises:
the formatting preprocessing unit is used for performing formatting preprocessing on the motion blurred image through a preset formatting preprocessing layer when a blurring processing request for the motion blurred image is received; and
and the image enhancement unit is used for carrying out image enhancement processing on the motion blurred image subjected to the formatting pretreatment to obtain the enhanced motion blurred image.
6. The apparatus of claim 4, in which the compressed excitation residual network unit consists of a residual network and a compressed excitation network.
7. A computing device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any of claims 1 to 3 when executing the computer program.
8. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 3.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810240092.9A CN108550118B (en) | 2018-03-22 | 2018-03-22 | Motion blur image blur processing method, device, equipment and storage medium |
| PCT/CN2018/081710 WO2019178893A1 (en) | 2018-03-22 | 2018-04-03 | Motion blur image sharpening method and device, apparatus, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810240092.9A CN108550118B (en) | 2018-03-22 | 2018-03-22 | Motion blur image blur processing method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108550118A CN108550118A (en) | 2018-09-18 |
| CN108550118B true CN108550118B (en) | 2022-02-22 |
Family
ID=63516720
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810240092.9A Expired - Fee Related CN108550118B (en) | 2018-03-22 | 2018-03-22 | Motion blur image blur processing method, device, equipment and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN108550118B (en) |
| WO (1) | WO2019178893A1 (en) |
Families Citing this family (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109712092B (en) * | 2018-12-18 | 2021-01-05 | 上海信联信息发展股份有限公司 | File scanning image restoration method and device and electronic equipment |
| CN109859141B (en) * | 2019-02-18 | 2022-05-27 | 安徽理工大学 | Deep vertical shaft well wall image denoising method |
| CN110070517B (en) * | 2019-03-14 | 2021-05-25 | 安徽艾睿思智能科技有限公司 | Blurred image synthesis method based on degradation imaging mechanism and generation countermeasure mechanism |
| CN110012145B (en) * | 2019-04-08 | 2021-01-05 | 北京易诚高科科技发展有限公司 | Mobile phone anti-shake function evaluation method based on image fuzziness |
| CN110074813B (en) * | 2019-04-26 | 2022-03-04 | 深圳大学 | Ultrasonic image reconstruction method and system |
| CN111340716B (en) * | 2019-11-20 | 2022-12-27 | 电子科技大学成都学院 | Image deblurring method for improving double-discrimination countermeasure network model |
| CN112904548B (en) * | 2019-12-03 | 2023-06-09 | 精微视达医疗科技（武汉）有限公司 | Endoscope focusing method and device |
| CN111275637B (en) * | 2020-01-15 | 2024-01-30 | 北京工业大学 | Attention model-based non-uniform motion blurred image self-adaptive restoration method |
| CN111369451B (en) * | 2020-02-24 | 2023-08-01 | 黑蜂智造(深圳)科技有限公司 | Image restoration model, method and device based on complex task decomposition regularization |
| CN111460939A (en) * | 2020-03-20 | 2020-07-28 | 深圳市优必选科技股份有限公司 | Deblurring face recognition method and system and inspection robot |
| CN111768826B (en) * | 2020-06-30 | 2023-06-27 | 深圳平安智慧医健科技有限公司 | Electronic health case generation method, device, terminal equipment and storage medium |
| CN112435179A (en) * | 2020-11-11 | 2021-03-02 | 北京工业大学 | Fuzzy pollen particle picture processing method and device and electronic equipment |
| CN112419201A (en) * | 2020-12-04 | 2021-02-26 | 珠海亿智电子科技有限公司 | Image deblurring method based on residual error network |
| CN112598593B (en) * | 2020-12-25 | 2022-05-27 | 吉林大学 | Seismic noise suppression method based on non-equilibrium depth expectation block log-likelihood network |
| CN112651894A (en) * | 2020-12-29 | 2021-04-13 | 湖北工业大学 | Image deblurring algorithm for generating countermeasure network based on RRDB |
| CN112949460B (en) * | 2021-02-26 | 2024-02-13 | 陕西理工大学 | Human behavior network model based on video and identification method |
| CN113313180B (en) * | 2021-06-04 | 2022-08-16 | 太原理工大学 | Remote sensing image semantic segmentation method based on deep confrontation learning |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105118031A (en) * | 2015-08-11 | 2015-12-02 | 中国科学院计算技术研究所 | Image processing method for recovering depth information |
| CN106204467A (en) * | 2016-06-27 | 2016-12-07 | 深圳市未来媒体技术研究院 | A kind of image de-noising method based on cascade residual error neutral net |
| CN106952239A (en) * | 2017-03-28 | 2017-07-14 | 厦门幻世网络科技有限公司 | image generating method and device |
| CN107527044A (en) * | 2017-09-18 | 2017-12-29 | 北京邮电大学 | A kind of multiple car plate clarification methods and device based on search |
| CN107590774A (en) * | 2017-09-18 | 2018-01-16 | 北京邮电大学 | A kind of car plate clarification method and device based on generation confrontation network |
| CN107609560A (en) * | 2017-09-27 | 2018-01-19 | 北京小米移动软件有限公司 | Character recognition method and device |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104091340A (en) * | 2014-07-18 | 2014-10-08 | 厦门美图之家科技有限公司 | Blurred image rapid detection method |
| CN107330954A (en) * | 2017-07-14 | 2017-11-07 | 深圳市唯特视科技有限公司 | A kind of method based on attenuation network by sliding attribute manipulation image |
| CN107730458A (en) * | 2017-09-05 | 2018-02-23 | 北京飞搜科技有限公司 | A kind of fuzzy facial reconstruction method and system based on production confrontation network |
-
2018
- 2018-03-22 CN CN201810240092.9A patent/CN108550118B/en not_active Expired - Fee Related
- 2018-04-03 WO PCT/CN2018/081710 patent/WO2019178893A1/en active Application Filing
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105118031A (en) * | 2015-08-11 | 2015-12-02 | 中国科学院计算技术研究所 | Image processing method for recovering depth information |
| CN106204467A (en) * | 2016-06-27 | 2016-12-07 | 深圳市未来媒体技术研究院 | A kind of image de-noising method based on cascade residual error neutral net |
| CN106952239A (en) * | 2017-03-28 | 2017-07-14 | 厦门幻世网络科技有限公司 | image generating method and device |
| CN107527044A (en) * | 2017-09-18 | 2017-12-29 | 北京邮电大学 | A kind of multiple car plate clarification methods and device based on search |
| CN107590774A (en) * | 2017-09-18 | 2018-01-16 | 北京邮电大学 | A kind of car plate clarification method and device based on generation confrontation network |
| CN107609560A (en) * | 2017-09-27 | 2018-01-19 | 北京小米移动软件有限公司 | Character recognition method and device |
Non-Patent Citations (1)
| Title |
|---|
| 深度图像下基于头部多特征的人数统计算法;刘磊;《深圳大学学报(理工版)》;20171130;第34卷(第6期);第584-590页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108550118A (en) | 2018-09-18 |
| WO2019178893A1 (en) | 2019-09-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108550118B (en) | Motion blur image blur processing method, device, equipment and storage medium | |
| Hussein et al. | Image-adaptive GAN based reconstruction | |
| Shen et al. | Deep semantic face deblurring | |
| Xu et al. | Learning to super-resolve blurry face and text images | |
| Xu et al. | Motion blur kernel estimation via deep learning | |
| Yu et al. | A unified learning framework for single image super-resolution | |
| Sun et al. | Lightweight image super-resolution via weighted multi-scale residual network | |
| Ren et al. | Face video deblurring using 3D facial priors | |
| CN107392852B (en) | Super-resolution reconstruction method, device and equipment for depth image and storage medium | |
| Zhang et al. | One-two-one networks for compression artifacts reduction in remote sensing | |
| Ren et al. | Partial deconvolution with inaccurate blur kernel | |
| Min et al. | Blind deblurring via a novel recursive deep CNN improved by wavelet transform | |
| Lin et al. | Learning to deblur face images via sketch synthesis | |
| CN112509144A (en) | Face image processing method and device, electronic equipment and storage medium | |
| Lee et al. | Progressive semantic face deblurring | |
| Liu et al. | Facial image inpainting using multi-level generative network | |
| Qi et al. | Blind face images deblurring with enhancement | |
| Dong et al. | Multi-scale Residual Low-Pass Filter Network for Image Deblurring | |
| Guo et al. | Image blind deblurring using an adaptive patch prior | |
| CN113763268A (en) | Blind restoration method and system for face image | |
| Lee et al. | Wide receptive field and channel attention network for jpeg compressed image deblurring | |
| Yang et al. | Deblurring and super-resolution using deep gated fusion attention networks for face images | |
| Pattanaik et al. | Mish-dctgan based combined image super-resolution and deblurring approach for blurry license plates | |
| Liu et al. | Poisson noise removal based on nonlocal total variation with Euler’s elastica pre-processing | |
| Song et al. | MBD-GAN: Model-based image deblurring with a generative adversarial network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20220222 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |