CN115760589A - Image optimization method and device for motion blurred image - Google Patents
Image optimization method and device for motion blurred image Download PDFInfo
- Publication number
- CN115760589A CN115760589A CN202211209802.4A CN202211209802A CN115760589A CN 115760589 A CN115760589 A CN 115760589A CN 202211209802 A CN202211209802 A CN 202211209802A CN 115760589 A CN115760589 A CN 115760589A
- Authority
- CN
- China
- Prior art keywords
- image
- network
- motion
- blurred
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Abstract
The invention discloses an image optimization method for a motion blurred image, which comprises the following steps: step 1, synthesizing a corresponding motion fuzzy image set according to a randomly generated motion trajectory vector and a clear sample image set, and forming the clear sample image set and the corresponding motion fuzzy image set into a training set; step 2, training the pre-constructed generation countermeasure network by utilizing the training set in the step 1 to obtain an image optimization model for generating a blur-free image; and 3, inputting the motion blurred image into the image optimization model obtained in the step 2, and outputting a corresponding real non-blurred image. The invention also provides an image optimization device. The method provided by the invention can effectively optimize the motion blurred image.
Description
Technical Field
The present invention relates to the field of image data processing, and in particular, to an image optimization method and apparatus for motion-blurred images.
Background
Image data-based detection techniques are widely used in the fields of environmental detection, agricultural detection, road condition assessment, civil engineering construction monitoring, structural health detection, and the like. Generally, mobile measurement platforms such as unmanned aerial vehicles are used for collecting image data of detected targets, and the method has the advantages of great flexibility and efficiency. However, the measurement precision mainly depends on the quality of the image obtained by the unmanned aerial vehicle, if the image is not clear enough or part of the content is fuzzy, the accuracy of the final result is affected, and in actual operation, the unmanned aerial vehicle is easily affected by external environmental factors, so that a motion fuzzy part exists in the acquired image data.
Most of the existing deblurring methods are based on classical convolution models, generally adopt iterative computation flow, utilize an image pyramid to compute a blur kernel and eliminate blur from top to bottom, and consume a large amount of computation time. In addition, such a convolution model-based method aims at uniform blurring on the global scale of an image, is difficult to describe by using a single blurring kernel for non-global uniform blurring, such as image rotation blurring caused by rotation of an unmanned aerial vehicle, and is even incapable of handling local blurring caused by object motion in a scene.
Patent document CN111275637A discloses an attention model-based non-uniform motion blurred image self-adaptive restoration method, which adopts a condition combined with an attention mechanism to generate a confrontation network, wherein the generation network is a coding and decoding structure, a dense connection network is adopted in a coding stage to extract features, the feature utilization rate is improved, the propagation of the features is enhanced, and a visual attention mechanism is added, so that the network can adaptively adjust network parameters for different input images, and image blur is dynamically removed. The method gives corresponding weight to areas with different fuzzy degrees, improves the deblurring effect of the traditional restoration method, but has the advantages that the input is a single image, the image information acquisition source is single, and in the deblurring process, the information is easy to lose, the irreversible effect is caused, and the final restoration effect is influenced.
Patent document CN114820299A discloses a method and a device for restoring a non-uniform motion blur super-resolution image, wherein the method comprises the following steps: s1, constructing a data set; s2, inputting the preprocessed data set into a generator to obtain a primary restored image; s3, distinguishing the preliminary restored image and the real image through a discriminator to obtain a distinguishing result, wherein the discriminator is a Markov distinguishing network; s4, optimizing the countermeasure network by using a loss function to obtain a network result with optimal performance and obtain an optimal restored image, wherein the generated countermeasure network comprises a generator and a discriminator; and S5, outputting a restoration result. According to the method, the acquisition amount of image information is increased in a rotating mode, so that the final restoration definition is improved, but the method has high requirements on the accuracy of a generator in the training process, and if the definition of the preprocessed primary restoration image is not enough, the training effect of a subsequent model is influenced.
Disclosure of Invention
In order to solve the problems, the invention provides an image optimization method which is simple to operate and stable in model, and the method can effectively optimize a motion blurred image.
An image optimization method for motion blurred images, comprising:
and 3, inputting the motion blurred image into the image optimization model obtained in the step 2, and outputting a corresponding real non-blurred image.
The method is different from the traditional method, deconvolution denoising is carried out on a blurred image under the condition that a blur kernel is known, so that a high-quality training set is obtained, meanwhile, a pre-constructed generation countermeasure network is trained by adopting single network training and an optimized combined loss function, and a final image optimization model is guided to have stable deblurring capability.
Preferably, in step 1, the synthesizing process of the motion-blurred image set specifically includes: and generating a fuzzy core according to the randomly generated motion track vector and the corresponding two-dimensional plane motion track, and performing fuzzy calculation on the clear sample image set by using the fuzzy core to obtain a corresponding motion fuzzy image set, so that the image quality of the training set is improved.
Specifically, in step 2, the generation network is configured to receive motion blur image input, and regenerate a corresponding sharp image through feature extraction and deconvolution algorithm, and the discrimination network is configured to perform true and false determination on the regenerated sharp image and the sharp sample image, so as to update a parameter state of the generation countermeasure network.
Specifically, the generation network comprises a three-channel blurred image feature extractor, a convolution layer, a batch normalization layer and a ReLU layer.
Specifically, the discrimination network includes a discriminator composed of a plurality of groups of convolution and full connection layers.
Preferably, in step 2, the training includes training of an individual network and optimization of a joint loss function, so as to ensure that the generated network and the discriminant network reach a balanced state.
Preferably, in step 2, the specific process of the training is as follows:
step 2-1, freezing the discrimination network, and training the generation network for 3 periods;
step 2-2, freezing the generated network, and training the discrimination network for 1 period;
and 2-3, repeating the steps 2-1 to 2-2 until the loss function of the generated network is minimized, and judging the loss function of the network to be maximized.
The invention also provides an image optimization apparatus comprising a computer memory in which is executed an image optimization model as described above, a computer processor and a computer program stored in and executable on said computer memory;
the computer processor, when executing the computer program, performs the steps of: and inputting the motion blurred image into an image optimization model, and outputting a corresponding high-definition non-blurred image.
Compared with the prior art, the invention has the following beneficial effects:
the method is different from the traditional method that under the condition that the fuzzy kernel is known, the deconvolution denoising is carried out on the fuzzy image. The method is based on the trainable generation of the confrontation network, breaks through the limitation of the traditional convolution model, guides the generation model to have stable deblurring capability, and can effectively optimize the image with motion blur.
Drawings
FIG. 1 is a flowchart illustrating an image optimization method for motion-blurred images according to the present invention;
fig. 2 is a structural diagram of a generation network according to the present embodiment;
fig. 3 is a structural diagram of the discrimination network provided in the present embodiment;
fig. 4 is an exemplary diagram before and after motion blur removal.
Detailed Description
As shown in fig. 1, an image optimization method for a motion-blurred image includes:
The resultant data set was as follows 8:2, randomly dividing to obtain a training set and a verification set.
the generation of the countermeasure network comprises a generation network and a discrimination network, wherein the generation network is used for receiving motion blurred image input, extracting image characteristics through a characteristic extraction path, then reconstructing by using a deconvolution network to generate a corresponding clear image, and the discrimination network carries out true and false judgment on the generated image and the real clear image so as to guide the generation network to output a more real non-blurred image;
as shown in fig. 2, to generate a detailed structure of the network:
generating a first layer of the network, and sequentially passing through a convolution layer, a batch normalization layer and a ReLU layer, wherein the convolution layer has convolution kernels of 5 × 5, a step length of 1 and an output dimension of 32;
generating a second layer of the network, and sequentially passing through a convolution layer, a batch normalization layer and a ReLU layer, wherein the convolution kernel of the convolution layer is 3 x 3, the step length is 2, and the output dimension is 64;
a third layer of the generated network sequentially passes through a convolution layer, a batch normalization layer and a ReLU layer, wherein the convolution kernel of the convolution layer is 3 x 3, the step length is 2, and the output dimension is 128;
in the fourth layer of the generated network, sequentially passing through a convolution layer, a batch normalization layer and a ReLU layer, wherein the convolution kernel of the convolution layer is 3 x 3, the step length is 2, and the output dimension is 256;
the fifth layer of the generated network sequentially passes through an deconvolution layer, a batch normalization layer and a ReLU layer, wherein the convolution kernel of the convolution layer is 3 x 3, the step length is 2, and the output dimension is 128;
in the sixth layer of the generated network, sequentially passing through an deconvolution layer, a batch normalization layer and a ReLU layer, wherein the convolution kernel of the convolution layer is 3 x 3, the step length is 2, and the output dimension is 64;
in the seventh layer of the generated network, sequentially passing through an deconvolution layer, a batch normalization layer and a ReLU layer, wherein the convolution kernel of the convolution layer is 3 x 3, the step length is 2, and the output dimension is 32;
and in the eighth layer of the generated network, sequentially passing through a convolution layer, a batch normalization layer and a ReLU layer, wherein the convolution kernel of the convolution layer is 5 × 5, the step length is 1, and the output dimension is 3.
As shown in fig. 3, in order to determine the specific structure of the network:
sequentially passing a convolution layer, a batch normalization layer and a ReLU layer in a first layer of the discrimination network, wherein the convolution kernel of the convolution layer is 3 x 3, the step length is 2, and the output dimension is 32;
sequentially passing through a convolution layer, a batch normalization layer and a ReLU layer at a second layer of the discrimination network, wherein the convolution kernel of the convolution layer is 3 x 3, the step length is 2, and the output dimension is 64;
sequentially passing through a convolution layer, a batch normalization layer and a ReLU layer on a third layer of the discrimination network, wherein the convolution kernel of the convolution layer is 3 x 3, the step length is 2, and the output dimension is 128;
in the fourth layer of the discrimination network, a convolution layer, a batch normalization layer and a ReLU layer are sequentially arranged, wherein the convolution kernel of the convolution layer is 3 x 3, the step length is 2, and the output dimension is 256;
in the fifth layer of the discrimination network, through a full connection layer and a ReLU layer, the output dimension is 1, and the length is 512;
and on the sixth layer of the discrimination network, sequentially passing through a full connection layer and a sigmoid activation layer, wherein the output dimension is 1, the length is 2, and the output result is the binary classification probability for judging whether the input image is a deblurred image or a real clear image.
And establishing a joint loss function, wherein the joint loss function is expressed as the sum of the network loss and the judgment network loss.
The loss of the generated network is the mean square error of the deblurred picture and the real clear picture output by the generated network; and judging the loss of the network as the cross entropy of the two classification probabilities output by the network and the source label of the input picture.
And meanwhile, the two losses are considered, so that the judgment network can judge whether the deblurred picture output by the generation network is true or false, and the generation network can be further promoted to output a more real deblurred picture.
When training the generated network, freezing the discrimination network to minimize the loss function of the generated network; when the discriminant network is trained, the generated network is frozen, so that the loss function of the discriminant network is maximized.
And updating network parameters by adopting a random gradient descent algorithm as an optimizer, wherein the weight attenuation factor is 0.0005, and the momentum is 0.9.
The initial learning rate was set to 0.001, and after each cycle, the rate was reduced to 0.95 times the previous rate, and the batch size was 12.
And (4) iterating the training process, and tracking the change of the two loss values in the training process so as to make the losses of the two networks converge.
And during model verification, the generation network carries out deblurring operation on the blurred picture, outputs the deblurred picture, and carries out calculation of peak signal-to-noise ratio and structural similarity with the real clear picture.
Comparing with a set peak signal-to-noise ratio threshold value 25 and a set structural similarity threshold value 0.8, and if the model verification results are greater than two threshold values, judging that the generated network has good generalization capability on a deblurring task; otherwise, the iterative training is continuously carried out until the verification result is qualified.
And 3, inputting the motion blurred image into the image optimization model obtained in the step 2, and outputting a corresponding real non-blurred image.
The present embodiment also provides an image optimization apparatus, comprising a computer memory, a computer processor, and a computer program stored in the computer memory and executable on the computer processor, the computer memory having the image optimization model as described above implemented therein;
the computer processor when executing the computer program implements the steps of: and inputting the motion blurred image into an image optimization model, and outputting a corresponding high-definition non-blurred image.
As shown in fig. 4, the left side of the figure is the input motion blurred image and the right side of the figure is the output non-blurred image.
Claims (8)
1. An image optimization method for motion blurred images, comprising:
step 1, synthesizing a corresponding motion fuzzy image set according to a randomly generated motion trajectory vector and a clear sample image set, and forming the clear sample image set and the corresponding motion fuzzy image set into a training set;
step 2, training a pre-constructed generation countermeasure network by utilizing the training set in the step 1 to obtain an image optimization model for generating a blur-free image, wherein the generation countermeasure network comprises a generation network and a judgment network;
and 3, inputting the motion blurred image into the image optimization model obtained in the step 2, and outputting a corresponding real non-blurred image.
2. The image optimization method for motion-blurred images according to claim 1, wherein in step 1, the synthesis process of the motion-blurred image set specifically comprises: and generating a fuzzy core according to the randomly generated motion track vector and the corresponding two-dimensional plane motion track, and performing fuzzy calculation on the clear sample image set by using the fuzzy core to obtain a corresponding motion fuzzy image set.
3. The image optimization method for motion blurred images as claimed in claim 1, wherein in step 2, the generation network is configured to receive the motion blurred image input, and to reproduce the corresponding sharp image through feature extraction and deconvolution algorithm, and the discrimination network is configured to perform true and false determination on the reproduced sharp image and the sharp sample image, so as to update the parameter state of the generation countermeasure network.
4. The image optimization method for motion blurred images of claim 3, wherein the generation network comprises a three-channel blurred image feature extractor, a convolution layer, a batch normalization layer and a ReLU layer.
5. The image optimization method for motion blurred images of claim 3, wherein the discriminant network comprises a discriminator, and the discriminator is composed of a plurality of groups of convolution and full connection layers.
6. The image optimization method for motion blurred images as claimed in claim 1, wherein in the step 2, the training comprises individual network training and optimization of a joint loss function, so as to ensure that the generation network and the discrimination network reach an equilibrium state.
7. The image optimization method for motion-blurred images as claimed in claim 1 or 6, wherein in step 2, the specific process of training is as follows:
step 2-1, freezing the discrimination network, and training the generation network for 3 periods;
step 2-2, freezing the generated network, and training the discrimination network for 1 period;
and 2-3, repeating the steps 2-1 to 2-2 until the loss function of the generated network is minimized, and judging the loss function maximization of the network.
8. An image optimization apparatus comprising a computer memory, a computer processor, and a computer program stored in the computer memory and executable on the computer processor, wherein the image optimization model of claim 1 is executed in the computer memory;
the computer processor, when executing the computer program, performs the steps of: and inputting the motion blurred image into an image optimization model, and outputting a corresponding high-definition non-blurred image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211209802.4A CN115760589A (en) | 2022-09-30 | 2022-09-30 | Image optimization method and device for motion blurred image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211209802.4A CN115760589A (en) | 2022-09-30 | 2022-09-30 | Image optimization method and device for motion blurred image |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115760589A true CN115760589A (en) | 2023-03-07 |
Family
ID=85350720
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211209802.4A Pending CN115760589A (en) | 2022-09-30 | 2022-09-30 | Image optimization method and device for motion blurred image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115760589A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116432534A (en) * | 2023-04-18 | 2023-07-14 | 中海石油(中国)有限公司上海分公司 | Data-driven TOC sample prediction method |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020087607A1 (en) * | 2018-11-02 | 2020-05-07 | 北京大学深圳研究生院 | Bi-skip-net-based image deblurring method |
| CN111612703A (en) * | 2020-04-22 | 2020-09-01 | 杭州电子科技大学 | Image blind deblurring method based on generation countermeasure network |
| CN111626956A (en) * | 2020-05-26 | 2020-09-04 | 北京百度网讯科技有限公司 | Image deblurring method and device |
| CN112686119A (en) * | 2020-12-25 | 2021-04-20 | 陕西师范大学 | License plate motion blurred image processing method based on self-attention generation countermeasure network |
| CN112927171A (en) * | 2021-04-15 | 2021-06-08 | 重庆邮电大学 | Single image deblurring method based on generation countermeasure network |
| CN113947589A (en) * | 2021-10-26 | 2022-01-18 | 北京理工大学 | Missile-borne image deblurring method based on countermeasure generation network |
-
2022
- 2022-09-30 CN CN202211209802.4A patent/CN115760589A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020087607A1 (en) * | 2018-11-02 | 2020-05-07 | 北京大学深圳研究生院 | Bi-skip-net-based image deblurring method |
| CN111612703A (en) * | 2020-04-22 | 2020-09-01 | 杭州电子科技大学 | Image blind deblurring method based on generation countermeasure network |
| CN111626956A (en) * | 2020-05-26 | 2020-09-04 | 北京百度网讯科技有限公司 | Image deblurring method and device |
| CN112686119A (en) * | 2020-12-25 | 2021-04-20 | 陕西师范大学 | License plate motion blurred image processing method based on self-attention generation countermeasure network |
| CN112927171A (en) * | 2021-04-15 | 2021-06-08 | 重庆邮电大学 | Single image deblurring method based on generation countermeasure network |
| CN113947589A (en) * | 2021-10-26 | 2022-01-18 | 北京理工大学 | Missile-borne image deblurring method based on countermeasure generation network |
Non-Patent Citations (4)
| Title |
|---|
| OREST KUPYN 等: "DeblurGAN:Blind Motion Deblurring Using COnditional Adversarial Networks", 《2018 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION》, pages 8183 - 8192 * |
| ZE-MING CHEN 等: "Blind Motion Deblurring via Inceptionresdensenet by Using GAN Model", 《ICASSP 2019》, pages 1463 - 1467 * |
| 易诗;吴志娟;朱竞铭;李欣荣;袁学松;: "基于多尺度生成对抗网络的运动散焦红外图像复原", 《电子与信息学报》, vol. 42, no. 07, pages 1766 - 1773 * |
| 裴慧坤 等: "基于生成对抗网络的无人机图像去模糊方法", 《地理空间信息》, vol. 17, no. 12, pages 4 - 9 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116432534A (en) * | 2023-04-18 | 2023-07-14 | 中海石油(中国)有限公司上海分公司 | Data-driven TOC sample prediction method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110135366B (en) | Shielded pedestrian re-identification method based on multi-scale generation countermeasure network | |
| CN112233038B (en) | True image denoising method based on multi-scale fusion and edge enhancement | |
| CN110378844B (en) | Image blind motion blur removing method based on cyclic multi-scale generation countermeasure network | |
| CN108154222B (en) | Deep neural network training method and system and electronic equipment | |
| CN108921877B (en) | Long-term target tracking method based on width learning | |
| Chen et al. | Local patch network with global attention for infrared small target detection | |
| CN109242097B (en) | Visual representation learning system and method for unsupervised learning | |
| CN111489304B (en) | Image deblurring method based on attention mechanism | |
| CN111681188B (en) | Image deblurring method based on combination of image pixel prior and image gradient prior | |
| CN114463218B (en) | Video deblurring method based on event data driving | |
| CN112419171A (en) | Image restoration method for multi-residual-block conditional generation countermeasure network | |
| CN116152591A (en) | Model training method, infrared small target detection method and device and electronic equipment | |
| CN115760589A (en) | Image optimization method and device for motion blurred image | |
| Kan et al. | A GAN-based input-size flexibility model for single image dehazing | |
| CN113393385B (en) | Multi-scale fusion-based unsupervised rain removing method, system, device and medium | |
| CN114202473A (en) | Image restoration method and device based on multi-scale features and attention mechanism | |
| CN116385281A (en) | Remote sensing image denoising method based on real noise model and generated countermeasure network | |
| CN115601257A (en) | Image deblurring method based on local features and non-local features | |
| CN115731447A (en) | Decompressed image target detection method and system based on attention mechanism distillation | |
| CN115223033A (en) | Synthetic aperture sonar image target classification method and system | |
| CN113012072A (en) | Image motion deblurring method based on attention network | |
| CN115605924A (en) | Class-agnostic repeat counting in video using temporal self-similarity matrix | |
| CN117475481B (en) | Domain migration-based night infrared image animal identification method and system | |
| Xing et al. | Image super-resolution using aggregated residual transformation networks with spatial attention | |
| CN112288738B (en) | Single image snowflake removing method and device based on deep learning and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |