CN116229081A - Unmanned aerial vehicle panoramic image denoising method based on attention mechanism - Google Patents
Unmanned aerial vehicle panoramic image denoising method based on attention mechanism Download PDFInfo
- Publication number
- CN116229081A CN116229081A CN202310221981.1A CN202310221981A CN116229081A CN 116229081 A CN116229081 A CN 116229081A CN 202310221981 A CN202310221981 A CN 202310221981A CN 116229081 A CN116229081 A CN 116229081A
- Authority
- CN
- China
- Prior art keywords
- image
- panoramic image
- convolution
- denoising
- feature map
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 230000007246 mechanism Effects 0.000 title claims abstract description 27
- 230000004913 activation Effects 0.000 claims abstract description 18
- 238000005286 illumination Methods 0.000 claims abstract description 18
- 238000011176 pooling Methods 0.000 claims abstract description 13
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 4
- 238000012549 training Methods 0.000 claims description 21
- 238000012937 correction Methods 0.000 claims description 19
- 238000010586 diagram Methods 0.000 claims description 12
- 238000000605 extraction Methods 0.000 claims description 10
- 230000006870 function Effects 0.000 claims description 10
- 238000012360 testing method Methods 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 6
- 230000001537 neural effect Effects 0.000 claims description 5
- 238000007781 pre-processing Methods 0.000 claims description 3
- 238000012545 processing Methods 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 claims description 2
- 230000000694 effects Effects 0.000 abstract description 7
- 238000001994 activation Methods 0.000 description 14
- 238000003384 imaging method Methods 0.000 description 6
- 238000011160 research Methods 0.000 description 3
- 230000035945 sensitivity Effects 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 230000003213 activating effect Effects 0.000 description 1
- 230000003321 amplification Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000593 degrading effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/30—Noise filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/34—Smoothing or thinning of the pattern; Morphological operations; Skeletonisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/17—Terrestrial scenes taken from planes or by drones
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Remote Sensing (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an unmanned aerial vehicle panoramic image denoising method based on an attention mechanism, which comprises the following steps: the convolutional neural network codec structure performs codec operation on an input image through operations such as depth separable convolution, activation pooling, deconvolution and the like; the depth can be separated, the convolution operation on the feature map is realized through the depth convolution and the point-by-point convolution, and the parameter number of the denoising network is reduced; the channel attention module is used for giving weight to the coded feature map on the channel to improve the network denoising performance; the residual attention module with multi-scale jump connection further improves the network denoising performance by paying attention to multi-scale information of the feature map during jump connection and giving weight to the feature map on a channel. According to the method, the panoramic image shot by the unmanned aerial vehicle in the complex illumination environment is subjected to image denoising, and the problems that the noise level needs to be estimated, time is consumed, and the image denoising effect is poor in the complex illumination environment in the conventional denoising method are solved.
Description
Technical Field
The invention relates to an image denoising method, in particular to an unmanned aerial vehicle panoramic image denoising method based on an attention mechanism.
Background
The panoramic imaging technology has the imaging characteristics of large visual field, low distortion and high resolution, and can be used as eyes of an unmanned aerial vehicle to realize real-time perception of a 360-degree scene. Panoramic images obtained by panoramic cameras on unmanned aerial vehicles in complex illumination environments have the problems of partial underexposure and partial underexposure, so that the panoramic images are degraded, and the perception effect of the unmanned aerial vehicles is seriously influenced. Noise, especially at low illumination, has been close to the signal intensity, overwhelming the useful detail information of the image, severely degrading the image quality. If signal amplification is carried out only, noise is further amplified, and the signal to noise ratio of a camera cannot be improved, so that the denoising of the panoramic image of the unmanned aerial vehicle in a low-light environment is particularly important.
In recent years, many researches on image denoising are carried out, but most researches on denoising are carried out on common images of a mobile phone camera and a digital camera, and researches on denoising of panoramic images of an unmanned aerial vehicle are not carried out. The image noise distribution in a complex illumination environment is related to factors such as illumination intensity, and a single noise model is difficult to cover all image noise. The prior denoising method needs to estimate the noise level, consumes time and has poor denoising effect in a complex illumination environment, which limit the application of the panoramic imaging system in the unmanned plane field.
Disclosure of Invention
Aiming at the situation, in order to overcome the defects of the prior art, the invention provides an unmanned aerial vehicle panoramic image denoising method based on an attention mechanism.
The invention solves the technical problems by the following technical proposal: an unmanned aerial vehicle panoramic image denoising method based on an attention mechanism is characterized by comprising the following steps:
the convolutional neural network coding and decoding structure comprises an encoder and a decoder, wherein the encoder comprises five convolution units, the convolution and activation operation is carried out on an image to finish image feature extraction, the pooling operation reduces the number of parameters, the decoder comprises four convolution units, the feature up-sampling is carried out on the image by deconvolution, the image size is gradually restored, and then the convolution and activation operation are repeated to gradually restore a denoised image;
depth separable convolutions, including depth convolutions and point-by-point convolutions, the depth convolutions convolve each channel of the input feature map with one convolution kernel, and then stack the results of all convolutions. The point-to-point convolution adjusts the channel number of the feature map according to the number of the convolution kernels, and better fuses the connection between the channels. The depth separable convolution can greatly reduce the network parameter quantity and accelerate the model operation speed;
the channel attention module comprises an average pooling module, a maximum pooling module, a shared MLP (multi-level processor) module, a feature map summation module, an activation module and a feature map product module, wherein the feature map with weight is obtained by carrying out maximum pooling and average pooling calculation on an input feature map and inputting the shared MLP module, and is summed, and finally the feature map with weight is multiplied with the original feature map, so that the weight is given to the feature map on the channel;
the residual attention module with the multi-scale jump connection comprises a common convolution module, a residual connection module and a channel attention module, wherein the input characteristic diagram is subjected to the common convolution once, the channel attention module is input, the obtained result is subjected to the residual connection with the input characteristic diagram, and finally the obtained result is subjected to multi-scale channel stacking with the characteristic diagram of the decoder.
The invention provides an unmanned aerial vehicle panoramic image denoising method based on an attention mechanism, which is characterized by comprising the following steps of:
step one, adjusting parameters of a panoramic camera on the unmanned aerial vehicle to proper values, and collecting and storing an original panoramic image;
secondly, extracting noise from the collected panoramic image, adding the extracted noise into other clean data sets to form a noise image, matching the noise image with a corresponding clean reference image, and constructing a training image data set for denoising the panoramic image of the unmanned aerial vehicle;
step three, carrying out histogram equalization and gamma correction on the original panoramic image, and adjusting the brightness and contrast of the original image to obtain a panoramic image after brightness and contrast correction;
step four, carrying out color correction on the panoramic image with corrected brightness and contrast to obtain an image with corrected color, and completing the pretreatment work of all panoramic images;
constructing a convolutional neural warp decoding network and a loss function based on an attention mechanism, importing training images into a denoising network in a batch mode, optimizing the loss function, and performing supervised learning training;
step six, the denoising test flow is to input the preprocessed panoramic image into a trained denoising network and output the denoised panoramic image.
Preferably, the codec uses depth separable convolution to perform convolution operation, so as to reduce the parameter amount required by the model operation process and accelerate the model operation speed. The encoder performs forward propagation calculations; the decoder restores the image size through deconvolution calculation mode, and the residual attention module with multi-scale jump connection stacks the characteristic graphs of different output scales of the encoder and the decoder on the channel.
Preferably, the encoder comprises five convolution units and the decoder comprises four convolution units.
Preferably, the channel attention module is located after the encoder and before the decoder; the residual attention module is between the convolution units of the encoder and decoder.
Preferably, the complex illumination environment comprises very weak visible light and exposed visible light.
Preferably, the CMOS camera uses back-illuminated, sensitivity-enhanced, micro-wedge techniques, and the like, which can image in more extreme complex illumination environments than conventional CMOS cameras.
Preferably, the noise extraction is performed by selecting a smooth region in the image, and subtracting the image mean of the region to obtain a noise block. The training data set is generated using noise extraction, and the training data set is input in a batch manner.
Preferably, the image preprocessing operation includes image enhancement and color correction processing, the image enhancement controls image darkness or overexposure through histogram equalization and gamma correction, and the color correction adopts a gray world method.
Preferably, the training includes forward propagation and backward propagation operations. The forward propagation operation calculates the feature map of each layer according to the network layer settings, and the backward propagation operation calculates the gradient and inversely updates the parameters of the previous convolution layer according to the final feature map and the loss function value of the target image.
Preferably, the denoising test inputs the preprocessed image to be denoised, and directly outputs the denoised image on the GPU by calling the trained model and the trained parameter configuration file.
The invention has the positive progress effects that: the method comprises the steps of constructing a convolutional neural warp-knitted decoding network based on an attention mechanism as a denoising network, improving the convolutional form to be depth-separable convolutional, adding a channel attention module and a residual attention module with multi-scale jump connection, training a denoising network model, inputting an image to be denoised into a trained denoising network, and outputting the denoised image. According to the method, according to the requirements of application scenes, the panoramic camera on the unmanned aerial vehicle is used for denoising pictures under a complex illumination environment, and the problems that in the prior denoising method, the noise level needs to be estimated, time is consumed, and the denoising effect is poor under a dark light environment are solved.
Drawings
Fig. 1 is a flowchart of an unmanned aerial vehicle panoramic image denoising method based on an attention mechanism.
Fig. 2 is a block diagram of the overall structure of the attention-based codec denoising network according to the present invention.
Fig. 3 is a schematic diagram of a depth separable convolution module of the present invention.
Fig. 4 is a schematic diagram of a channel attention module of the present invention.
Fig. 5 is a schematic diagram of a residual attention module with multi-scale jump connection of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments.
The image denoising method of the traditional method generally focuses on removing Gaussian noise and ISO noise, and the method uses noise extraction to generate a training data set, and the characteristic learned by the method using deep learning is specific to the training data, so that the method meets the panoramic image denoising requirement of the unmanned aerial vehicle in a real complex illumination environment. In addition, the deep learning has the advantages of strong generalization capability, and the GPU performance is fully utilized by reading in a batch mode, so that the image processing speed is increased. The method solves the problems that the prior image denoising method needs to estimate the noise level, consumes time and has poor denoising effect in a dark light environment.
As shown in fig. 1, the unmanned aerial vehicle panoramic image denoising method based on the attention mechanism comprises the following steps:
shooting a panoramic image by using a panoramic camera on an unmanned aerial vehicle in a complex illumination environment, adjusting parameters such as aperture, focal length, exposure time and the like of the camera to proper values, and collecting and storing an original panoramic image;
secondly, extracting noise from the collected panoramic image, adding the extracted noise into other clean data sets to form a noise panoramic image, matching the noise panoramic image with a corresponding clean reference panoramic image, and constructing a training image data set for denoising the panoramic image of the unmanned aerial vehicle under a complex illumination condition;
thirdly, carrying out histogram equalization and gamma correction on the original panoramic image, and adjusting the brightness and contrast of the original panoramic image to obtain a panoramic image after brightness and contrast correction;
step four, carrying out color correction on the panoramic image subjected to brightness and contrast correction to obtain a panoramic image subjected to color correction, and completing pretreatment work of all the panoramic images;
constructing a convolutional neural warp decoding network and a loss function based on an attention mechanism, importing training images into a denoising network in a batch mode, optimizing the loss function, and performing supervised learning training;
step six, the denoising test flow is to input the preprocessed panoramic image into a trained denoising network and output the denoised panoramic image.
In order to perform effective imaging in a complex illumination environment, the used CMOS camera adopts the techniques of back illumination, sensitivity enhancement, micro-optical wedge and the like, so that the light capturing area and the light sensing sensitivity of the CMOS are improved. For imaging in more extreme low-light environments, additional imaging with invisible near-infrared light may also be used. The collected original image is used for obtaining a noise model of the CMOS camera through a noise extraction method, and a noise block is obtained through selecting a smooth area in the image and subtracting an image mean value of the area. The determination mode of the flat sliding block is specifically as follows:
wherein P is a selected image block, s 1 Sum s 2 The length and width of the image block, respectively, p is the smaller test block, h 1 And h 2 The length and width of an image block, respectively, C represents three color channels, i and j are the sequence numbers of the blocks. Mean (-) represents the calculated average, hist (-) represents the calculated histogram, λ low And lambda (lambda) low Is a threshold that determines whether smoothing is met. Therefore, the noise block is specifically expressed as follows:
wherein P is k Is the smoothed image block selected and N is the noise block. The training data set is generated using noise extraction.
And performing image preprocessing operation on the original image, controlling the image to be too dark or too exposed through histogram equalization and gamma correction in image enhancement, and performing image color correction through a gray world method.
As shown in fig. 2, the overall structure of the attention mechanism-based codec denoising network of the present invention includes:
the size of the input image is 128 x 128, and the convolutional neural network codec structure includes an encoder and a decoder. The encoder performs forward propagation by convolving and activating the image to complete extraction of image features and reduce the amount of computation. The encoder comprises five layers of convolution units:
the first layer convolution unit includes three 3x3 size, 2 step size, "same" fill-wise depth separable convolutions and ReLU activations followed by 2 x 2 size max pooled downsampling.
The second layer convolution unit includes three 3x3 size, 2 step size, "sarne" fill-style depth separable convolutions and ReLU activations followed by 2 x 2 size maximum pooled downsampling.
The third layer of convolution units consists of three 3x3 size, 2 step size, "sarne" fill-wise depth separable convolutions and ReLU activations followed by 2 x 2 size max pooled downsampling.
The fourth layer convolution unit includes three 3x3 size, 2 step size, "sarne" fill-style depth separable convolutions and ReLU activations followed by 2 x 2 size maximum pooled downsampling.
The fifth convolution unit includes a 3x3 size twice, a step size of 2, a depth separable convolution of the "sanne" fill mode, and a ReLU activation.
The encoded profile is input to a channel attention module and then to a decoder. The decoder gradually restores the image size by performing deconvolution, convolution and activation operations on the image, the deconvolution performs characteristic up-sampling, the convolution and activation gradually restore the de-noised image, and finally the de-noised image is output, and the decoder comprises four layers of convolution units:
the first layer of convolution units includes 2 x 2 deconvolution upsampling followed by two layers of 3x3 size, step size 2, "same" fill-wise deep separable convolution and ReLU activation.
The second layer convolution unit includes 2 x 2 deconvolution upsampling followed by two layers of 3x3 size, step size 2, "same" fill-wise depth separable convolution and ReLU activation.
The third layer of convolution units includes 2 x 2 deconvolution upsampling followed by two layers of 3x3 size, step size 2, "same" fill-wise depth separable convolution and ReLU activation.
The fourth layer convolution unit includes 2 x 2 deconvolution upsampling followed by two layers of 3x3 size, step size 2, "same" fill-wise depth separable convolution and ReLU activation.
And adding a residual attention module with multi-scale jump connection between convolution units of the encoder and the decoder, and performing multi-scale feature map connection, wherein the size of a final output image is 128×128.
As shown in fig. 3, the calculation method of the depth separable convolution module of the present invention includes:
the method comprises the steps of deep convolution and point-by-point convolution, wherein the deep convolution uses a convolution kernel to convolve each channel of an input feature map, and then channel stacking is carried out on the convolved results. If the input feature map has M channels in total, the depth convolution contains M3×3 convolution kernels in total, each convolution kernel convolves the feature map of only one channel, and then stacks all the convolved results. The point-to-point convolution adjusts the channel number of the feature map according to the number of the convolution kernels, and fuses the links among the channels. If the channel of the output feature map is N, N convolution kernels of 1×1 are used for point-by-point convolution, each convolution kernel is convolved with all channels of the input feature map, and the results are added to obtain one channel of the output feature map, so that N channels are completed in total. The depth separable convolution can greatly reduce the network parameter quantity and accelerate the model operation speed;
as shown in fig. 4, the channel attention module of the present invention includes:
the method comprises the steps of carrying out maximum pooling and average pooling calculation on an input feature map, inputting a shared MLP to obtain two feature maps with weights, setting the reduction ratio of the shared MLP to be 16, and finally multiplying the feature map with the weights by an original feature map to give weights to the feature map on a channel.
As shown in fig. 5, the residual attention module with multi-scale jump connection of the present invention comprises:
the device comprises a common convolution module, a residual connection module and a channel attention module, wherein the common convolution module is used for carrying out common convolution on a feature image, the residual connection is carried out on the feature image and an input feature image, the output result is subjected to downsampling of different multiplying powers, and the downsampled feature image and the feature image of a decoder are subjected to multi-scale channel stacking.
Unmanned aerial vehicle panoramic image denoising method based on attention mechanism trains dataset in batch mode input to the coding and decoding denoising network based on attention mechanism and supervises learning training, uses L 1 Loss as a function of loss, specifically formula (1):
L 1 ＝|I GT -I pre |...(4)
wherein I is GT For reference pictures, I pre Is a predictive image. And calculating gradients according to the final feature map and the loss function value of the target image, and back-propagating, and updating the parameters of the previous convolution. The batch size was set to 4 and the training dataset and validation set ratio was set to 8:2, a total of 30 epochs were trained, with a learning rate set to 0.0001. Adam optimizer is selected and used, parameter beta 1 ＝0.9，β 2 ＝0.999，epsilon＝1×10 -8 。
And the denoising test inputs the preprocessed image to be denoised into a trained denoising network, and outputs a denoised image.
According to the invention, a training data set is constructed based on the panoramic image of the unmanned aerial vehicle, and a convolutional neural warp-knitted decoding network based on an attention mechanism is used as a denoising network. According to the method and the device, according to the requirements of application scenes, the panoramic picture shot by the panoramic camera on the unmanned aerial vehicle under the complex illumination condition is subjected to denoising treatment, so that the problems that the noise level needs to be estimated, time is consumed, and the denoising effect is poor under the complex illumination environment in the conventional denoising algorithm are solved.
The invention can be combined with other subsequent computer vision methods, such as target detection, semantic segmentation and other algorithms, according to actual demands, and the accuracy of the advanced computer vision recognition algorithm can be further improved. The above-described embodiments are merely illustrative of the present invention and are not intended to limit the scope of the present invention, and those skilled in the art can make various modifications and variations without departing from the spirit of the invention, but these modifications and variations are intended to fall within the scope of the invention as defined in the appended claims.
Claims (10)
1. An unmanned aerial vehicle panoramic image denoising method based on an attention mechanism is characterized by comprising the following steps:
the convolutional neural network codec structure comprises an encoder and a decoder, wherein the encoder comprises five convolution units, the convolution and activation operation is carried out on an image to finish image feature extraction, the pooling operation reduces the number of parameters, the decoder comprises four convolution units, the feature up-sampling is carried out on the image by deconvolution, the image size is gradually restored, and then the convolution and activation operation is repeated to gradually restore a denoised image;
depth separable convolutions, including depth convolutions and point-by-point convolutions, the depth convolutions convolve each channel of the input feature map with one convolution kernel, and then stack the results of all convolutions. The point-by-point convolution adjusts the channel number of the feature map according to the number of the convolution kernels, and better fuses the connection between the channels;
the channel attention module comprises an average pooling module, a maximum pooling module, a shared MLP (multi-level processor) module, a feature map summation module, an activation module and a feature map product module, wherein the feature map with weight is obtained by carrying out maximum pooling and average pooling calculation on an input feature map and inputting the shared MLP module, and is summed, and finally the feature map with weight is multiplied with the original feature map, so that the weight is given to the feature map on the channel;
the residual attention module with the multi-scale jump connection comprises a common convolution module, a residual connection module and a channel attention module, wherein the input characteristic diagram is subjected to the common convolution once, the channel attention module is input, the obtained result is subjected to the residual connection with the input characteristic diagram, and finally the obtained result is subjected to multi-scale channel stacking with the characteristic diagram of the decoder.
2. The unmanned aerial vehicle panoramic image denoising method based on the attention mechanism is characterized by comprising the following steps of:
step one, adjusting parameters of a panoramic camera on an unmanned aerial vehicle to proper values, and collecting and storing an original panoramic image;
secondly, extracting noise from the collected panoramic image, adding the extracted noise into other clean data sets to form a noise image, and matching the noise image with a corresponding clean reference image to construct a training image data set for denoising the panoramic image;
thirdly, carrying out histogram equalization and gamma correction on the original panoramic image, and adjusting the brightness and contrast of the original panoramic image to obtain a panoramic image after brightness and contrast correction;
step four, carrying out color correction on the panoramic image with corrected brightness and contrast to obtain an image with corrected color, and completing the pretreatment work of all panoramic images;
constructing a convolutional neural warp decoding network and a loss function based on an attention mechanism, importing training panoramic images into a denoising network in a batch mode, optimizing the loss function, and performing supervised learning training;
step six, the denoising test flow is to input the preprocessed panoramic image into a trained denoising network and output the denoised panoramic image.
3. The unmanned aerial vehicle panoramic image denoising method based on the attention mechanism of claim 1, wherein the encoder and decoder uses depth separable convolution to carry out convolution operation, so that the parameter amount required by the model operation process is reduced, and the model operation speed is increased; the channel attention module is positioned after the encoder and before the decoder; the residual attention module is used for connecting the characteristic diagrams of the encoder and the decoder, and the result of the residual attention module is subjected to downsampling of different multiplying factors, and the characteristic diagrams of different scales are connected on the channel.
4. The unmanned aerial vehicle panoramic image denoising method based on the attention mechanism of claim 2, wherein the complex illumination environment comprises weak visible light and overexposed visible light.
5. The unmanned aerial vehicle panoramic image denoising method based on the attention mechanism of claim 2, wherein the noise extraction is used to generate a training data set, which is input in a batch manner.
6. The unmanned aerial vehicle panoramic image denoising method based on the attention mechanism of claim 2, wherein the image preprocessing operation comprises image enhancement and color correction processing.
7. The unmanned aerial vehicle panoramic image denoising method based on an attention mechanism of claim 2, wherein the training comprises forward propagation and backward propagation operations. The forward propagation operation calculates the feature map of each layer according to the network layer settings, and the backward propagation operation calculates the gradient and inversely updates the parameters of the previous convolution layer according to the final feature map and the loss function value of the target image.
8. The unmanned aerial vehicle panoramic image denoising method based on the attention mechanism of claim 2, wherein the denoising test flow is to preprocess the panoramic image to be denoised by inputting, and calculate the panoramic image directly after denoising on the GPU in parallel by calling the trained model and the parameter configuration file.
9. The unmanned aerial vehicle panoramic image denoising method based on the attention mechanism of claim 6, wherein the noise extraction obtains a noise block by selecting a smooth region in the image and subtracting an image mean value of the region.
10. The unmanned aerial vehicle panoramic image denoising method based on the attention mechanism of claim 7, wherein the image enhancement controls the image to be too dark or too exposed through histogram equalization and gamma correction, and the color correction adopts a gray world method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310221981.1A CN116229081A (en) | 2023-03-09 | 2023-03-09 | Unmanned aerial vehicle panoramic image denoising method based on attention mechanism |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310221981.1A CN116229081A (en) | 2023-03-09 | 2023-03-09 | Unmanned aerial vehicle panoramic image denoising method based on attention mechanism |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116229081A true CN116229081A (en) | 2023-06-06 |
Family
ID=86580365
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310221981.1A Pending CN116229081A (en) | 2023-03-09 | 2023-03-09 | Unmanned aerial vehicle panoramic image denoising method based on attention mechanism |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116229081A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116681618A (en) * | 2023-06-13 | 2023-09-01 | 强联智创（北京）科技有限公司 | Image denoising method, electronic device and storage medium |
-
2023
- 2023-03-09 CN CN202310221981.1A patent/CN116229081A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116681618A (en) * | 2023-06-13 | 2023-09-01 | 强联智创（北京）科技有限公司 | Image denoising method, electronic device and storage medium |
| CN116681618B (en) * | 2023-06-13 | 2024-08-02 | 强联智创（北京）科技有限公司 | Image denoising method, electronic device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111968044B (en) | Low-illumination image enhancement method based on Retinex and deep learning | |
| CN115442515B (en) | Image processing method and apparatus | |
| CN112233038B (en) | True image denoising method based on multi-scale fusion and edge enhancement | |
| CN112541877B (en) | Defuzzification method, system, equipment and medium for generating countermeasure network based on condition | |
| CN111402146B (en) | Image processing method and image processing apparatus | |
| CN114119378A (en) | Image fusion method, and training method and device of image fusion model | |
| CN112164011B (en) | Motion image deblurring method based on self-adaptive residual error and recursive cross attention | |
| CN111079764B (en) | Low-illumination license plate image recognition method and device based on deep learning | |
| CN113129236B (en) | Single low-light image enhancement method and system based on Retinex and convolutional neural network | |
| CN113450290B (en) | Low-illumination image enhancement method and system based on image inpainting technology | |
| CN110428389B (en) | Low-light-level image enhancement method based on MSR theory and exposure fusion | |
| Moghimi et al. | Real-time underwater image resolution enhancement using super-resolution with deep convolutional neural networks | |
| CN116596799A (en) | Low-illumination image enhancement method based on channel space compound attention | |
| CN116229081A (en) | Unmanned aerial vehicle panoramic image denoising method based on attention mechanism | |
| CN112927162A (en) | Low-illumination image oriented enhancement method and system | |
| CN115797205A (en) | Unsupervised single image enhancement method and system based on Retinex fractional order variation network | |
| Saleem et al. | A non-reference evaluation of underwater image enhancement methods using a new underwater image dataset | |
| CN117351216A (en) | Image self-adaptive denoising method based on supervised deep learning | |
| CN113379861B (en) | Color low-light-level image reconstruction method based on color recovery block | |
| Li et al. | LDNet: low-light image enhancement with joint lighting and denoising | |
| CN114140361A (en) | Generation type anti-network image defogging method fusing multi-stage features | |
| CN117974459A (en) | Low-illumination image enhancement method integrating physical model and priori | |
| CN115311149A (en) | Image denoising method, model, computer-readable storage medium and terminal device | |
| CN117392036A (en) | Low-light image enhancement method based on illumination amplitude | |
| CN117391987A (en) | Dim light image processing method based on multi-stage joint enhancement mechanism |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |