CN112907479A - Residual single image rain removing method based on attention mechanism - Google Patents
Residual single image rain removing method based on attention mechanism Download PDFInfo
- Publication number
- CN112907479A CN112907479A CN202110244785.7A CN202110244785A CN112907479A CN 112907479 A CN112907479 A CN 112907479A CN 202110244785 A CN202110244785 A CN 202110244785A CN 112907479 A CN112907479 A CN 112907479A
- Authority
- CN
- China
- Prior art keywords
- image
- rain
- neural network
- attention
- network model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 32
- 230000007246 mechanism Effects 0.000 title claims abstract description 11
- 238000003062 neural network model Methods 0.000 claims abstract description 35
- 238000012545 processing Methods 0.000 claims abstract description 26
- 238000012549 training Methods 0.000 claims abstract description 8
- 238000007781 pre-processing Methods 0.000 claims abstract description 5
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims description 8
- 238000005070 sampling Methods 0.000 claims description 6
- 230000004913 activation Effects 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 3
- 230000000452 restraining effect Effects 0.000 claims description 2
- 230000000694 effects Effects 0.000 abstract description 4
- 230000006870 function Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 238000013527 convolutional neural network Methods 0.000 description 3
- 238000013136 deep learning model Methods 0.000 description 2
- 239000000428 dust Substances 0.000 description 2
- 238000002156 mixing Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 241001085205 Prenanthella exigua Species 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000000779 smoke Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000013179 statistical model Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a residual single image rain removing method based on an attention mechanism, which mainly solves the problems of limitation and unsatisfactory processing effect of the existing single image rain removing technology. The scheme comprises the following steps: 1) preprocessing an input image to obtain a preprocessed image; 2) constructing an attention residual error neural network model comprising a residual error network module and a coder-decoder network module; 3) inputting the preprocessed image into an attention residual error neural network model for training, constraining the attention residual error neural network model by using a loss function, and then performing back propagation for parameter updating to obtain a trained rain-removing neural network model; 4) and inputting the rain image to be processed into a rain-removing neural network model for image processing to obtain a clear image without rain. The method can effectively remove rain marks in a single rain-containing image to obtain a clear image; meanwhile, the background information in the original image is fully reserved.
Description
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to a residual single image rain removing method based on an attention mechanism, which can be used for the sharpening processing of single images.
Background
Vision, one of the most important human perception systems, is the main source from which we obtain knowledge. As pictures and videos are increasingly abundant in the internet age, images become an important part of our lives. Therefore, computer vision is a very important discipline in the background of the modern times, and is an important component of all fields in our lives, such as manufacturing, military industry and various intelligent system fields.
In a severe weather environment, pictures or videos shot by people are often disturbed and blurred by rain, snow and fog. The acquired picture or video information is damaged to different degrees, and even the main body of the picture is seriously interfered. The variable weather mainly includes rain, snow, smoke and dust. For smog and dust, rain is a more disordered picture noise, which changes constantly and also causes different effects on the picture along with the magnitude of the rain. For example, light rain can cause bright white rain lines in some areas of the picture, while heavy rain can cause occlusion or severe blurring of the entire picture. Therefore, the rainwater removing work is a work with high difficulty in the fields of image denoising and image restoration.
Rain is a dynamic weather-because its constituent particles are relatively large, it can be easily captured by the camera. Due to the introduction of unintended visual artifacts in outdoor vision systems, computer vision tasks such as subsequent image matching can be adversely affected. Raindrops typically produce dense streaks on the image that are unpredictable, such as direction, intensity of rain, uneven rain density, and the like. Thus, the complex pixel variations and additional gradients caused by these dense fringes can obscure the advantageous information conveyed in the image, preventing efficient detection of reliable features.
However, rainwater, the most common natural phenomenon in our lives, occurs very frequently, and particularly in the southern plum rain season, continuous rainfall occurs. Therefore, the method has practical significance for the research of image rain removal.
In recent years, the image rain-removing task has a very high attention, and in general, the image rain-removing task can be roughly divided into two types: 1) video-based methods and 2) single-image based methods.
The patent of Chen et al, "a video rain removing method based on a multi-scale mixing index model" (application number: CN201910298090.X, publication number: CN110070506A) discloses a video rain removing method based on a multi-scale mixing index model. A rain removing video is obtained by optimizing a rain video statistical model under a single scale, pyramid decomposition reconstruction is further adopted on the rain removing video of each scale, and a final result is obtained by adding and averaging; for the video shot by a static camera, the method can effectively remove the rain stripes, however, for the video shot by a mobile camera, the method has a great limitation and is not suitable.
Tsinging discloses a single image rain removing method based on a convolutional neural network in the patent of 'a single image rain removing method based on a convolutional neural network' (application number: CN201610592134.6, publication number: CN 106204499A). the method artificially adds rain to a clean clear image through a screen blend model to establish an image library; then, taking out image block pairs with rain or without rain from the image library, and performing network training by taking the image block pairs as training samples; and finally, taking blocks of the single rain-containing image in an overlapping mode, inputting the blocks into a trained rain-removing filtering system, obtaining corresponding rain-free image blocks, and carrying out weighted average to obtain a rain-free image. The method has the disadvantage that the designed convolutional neural network is too simple in structure and cannot achieve a good removal effect.
Disclosure of Invention
The invention aims to provide a residual single image rain removing method based on an attention mechanism aiming at the defects of the prior art, which is used for improving the rain removing performance of a single image; firstly, preprocessing an input image, and then inputting the preprocessed image into a constructed attention residual error neural network model for training; constraining the attention residual error neural network model through a loss function, and then performing parameter updating through back propagation to obtain a trained rain-removing neural network model; and inputting the rain image into the trained rain removing neural network model to obtain a clear rain-free image. The method can effectively remove rain marks in a single rain-containing image to obtain a clear image; meanwhile, the background information in the original image is fully reserved.
The invention realizes the aim as follows:
(1) preprocessing an input image to obtain a preprocessed rainy image;
(2) constructing an attention residual error neural network model:
(2.1) adopting a convolution layer, a convolution residual block formed by short jump connection and a residual network module formed by long jump connection, and extracting image characteristics of the preprocessed rain image;
(2.2) constructing a coding and decoding network module:
(2.2.1) an encoder network is formed by a down-sampling and channel attention module SE _ Block, and a decoder network is formed by an up-sampling and channel attention module SE _ Block;
(2.2.2) the output characteristics of the bottommost layer of the encoder network enter a decoder network after being processed by a channel attention module SE _ Block and a convolutional layer, and the output characteristic information of decoder networks on other layers and the corresponding layers of the encoder network are overlapped and fused in a hopping connection mode to form an encoding and decoding network module;
(2.3) the output of the residual error network module is connected with the input of an encoder network in the encoding and decoding network module, and the output of a decoder network in the encoding and decoding network module is rain print information;
and (2.4) carrying out difference processing on the preprocessed rain image obtained in the step (1) and the rain print information output by the coding and decoding network module, outputting a rain-free image after rain is removed, and completing construction of an attention residual error neural network model.
(3) Training the attention residual error neural network model by adopting the preprocessed rainy image, restraining the attention residual error neural network model by using a loss function, and updating parameters by adopting back propagation to obtain a rain-removing neural network model;
(4) and taking the rain image to be processed as the input of a rain removing neural network model, carrying out rain removing processing on the image by using the rain removing neural network model, and outputting a clean image after rain removing.
Compared with the prior art, the invention has the following advantages:
firstly, the network model adopts a residual error network structure, so that the network is deepened, the degradation problem of a deep neural network is avoided, richer image characteristic information can be extracted, and the network training process is simple;
secondly, as the attention mechanism is introduced, the interference of background information is reduced, so that the model focuses more on the detailed information of the rain streak in the image, and meanwhile, the implementation is simpler and the rain removing effect is good.
Drawings
FIG. 1 is a flow chart of an implementation of the method of the present invention;
FIG. 2 is a schematic diagram of a residual error network module according to the present invention;
FIG. 3 is a schematic diagram of a convolution residual block of the present invention;
FIG. 4 is a schematic diagram of a deep learning model of the method of the present invention;
fig. 5 is a schematic diagram of a channel attention module SE _ Block according to the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described with reference to the accompanying drawings.
Example 1:
referring to the attached figure 1, the invention provides a residual single image rain removing method based on an attention mechanism, which comprises the following steps:
step 1: preprocessing an input image to obtain a preprocessed rainy image;
the pretreatment specifically comprises the following steps: the pixel values of the input image are normalized to [0, 1], and the pixel size is clipped to 256 × 3.
Step 2: building an attention residual error neural network model;
(2.1) adopting a convolution layer, a convolution residual block formed by short jump connection and a residual network module formed by long jump connection, and extracting image characteristics of the preprocessed rain image;
as shown in fig. 2, in this embodiment, it is preferable that 1 convolutional layer, 5 convolutional residual blocks composed of short-hop connections, and a long-hop connection constitute a residual network module, which is used to perform image feature extraction on the preprocessed rain image; the structure of the convolution residual block is shown in fig. 3, and the convolution calculation formula is as follows:
xL+1=xL+F(xL+WL),
wherein x isL+1Is the convolution result of the L +1 th convolutional layer, xLAs a result of convolution of the L-th convolutional layer, WLIs the weight of the L-th convolutional layer, F (x)L+WL) Representing the residual part.
(2.2) constructing a coding and decoding network module:
(2.2.1) an encoder network is formed by a down-sampling and channel attention module SE _ Block, and a decoder network is formed by an up-sampling and channel attention module SE _ Block; in the embodiment, 3 downsampling and 3 channel attention modules SE _ Block are preferably selected to form an encoder network, which is used for extracting the rainprint features in the image features step by step; 3 times of upsampling and 3 channel attention modules SE _ Block form a decoder network; the system is used for fusing the rainprint features extracted by the encoder network step by step to obtain fused rainprint information; the upsampling is specifically composed of a convolutional layer with the size of 3x3 and the step length of 2; the downsampling is specifically composed of a transposed convolutional layer with the size of 3x3 and the step size of 2; the channel attention module SE _ Block consists of a global pooling layer, a full connection layer, a ReLU activation layer, a full connection layer and a Sigmoid layer.
(2.2.2) the output characteristics of the bottommost layer of the encoder network enter a decoder network after being processed by a channel attention module SE _ Block and three convolution layers, and the output characteristic information of decoder networks on other layers and the corresponding layers of the encoder network are overlapped and fused in a jump connection mode to form an encoding and decoding network module;
(2.3) the output of the residual error network module is connected with the input of an encoder network in the encoding and decoding network module, and the output of a decoder network in the encoding and decoding network module is rain print information;
and (2.4) carrying out difference processing on the preprocessed rain image obtained in the step (1) and the rain print information output by the coding and decoding network module, outputting a rain-free image after rain is removed, and completing construction of an attention residual error neural network model.
And step 3: training the attention residual error neural network model built in the step 2 by adopting the preprocessed rainy image obtained in the step 1, constraining the attention residual error neural network model by using a loss function, and updating parameters by adopting back propagation to obtain the trained attention residual error neural network model, namely a rain-removing neural network model;
the expression of the above loss function is:
Loss=LossSmooth_L1+λLossSSIM,
wherein LossSmooth_L1Represents a smooth mean absolute error loss function expressed as follows:
wherein, yiShowing the ith rainless image of the input, xiIndicating the input ith rained image, f (x)i) Representing the output of the ith rained image after the attention residual error neural network model processing;
LossSSIMrepresenting the structural similarity loss function, the expression is as follows:
LossSSIM=1-SSIM(Y,F(x)),
wherein, Y and x represent the input no-rain image and the input rain image respectively, F (x) represents the output of the input rain image x after the processing of the attention residual error neural network model;
SSIM denotes structural similarity, expressed as follows:
SSIM=[g(Y,F(x))]α·[c(Y,F(x))]β·[s(Y,F(x))]γ,
where g, c, and s represent brightness, contrast, and texture, respectively, and α, β, and γ represent a brightness coefficient, a contrast coefficient, and a texture coefficient, respectively.
And 4, step 4: and (3) taking the rain image to be processed as the input of the trained network model in the step (3), carrying out rain removing treatment on the image by using a rain removing neural network model, and outputting a clean image after rain removing.
Example 2:
referring to FIG. 4, a deep learning model of the method of the present invention is schematically illustrated;
the method for removing rain from a residual single image based on an attention mechanism is the same as that in embodiment 1, wherein the attention residual neural network model constructed in the step 2 specifically comprises the following image processing processes:
4.1, inputting a rain image, performing first convolutional layer processing, outputting a characteristic F1 with the channel number of 32, processing the output characteristic F1 by 5 convolutional residual blocks formed by short jump connection, and outputting a characteristic F2 with the channel number of 32;
4.2, the output characteristic F1 is connected with the output characteristic F2 through long jump to output the characteristic F3;
4.3, the output characteristic F3 is processed by a channel attention module SE _ Block, and the output characteristic F4;
4.4, the output characteristic F4 is subjected to the first downsampling processing by the downsampling convolutional layer, the output characteristic F5 is output, and the number of characteristic channels is 64;
4.5, the output characteristic F5 is processed by a channel attention module SE _ Block, and the output characteristic F6;
4.6, the output characteristic F6 is subjected to the second downsampling processing by the downsampling convolutional layer, the output characteristic F7 is output, and the number of characteristic channels is 128;
4.7, the output characteristic F7 is processed by a channel attention module SE _ Block, and the output characteristic F8;
4.6, carrying out third downsampling processing on the output characteristic F8 after downsampling convolutional layers, and outputting a characteristic F9 with the characteristic channel number of 256;
4.8, processing the output characteristic F9 by a channel attention module SE _ Block, and outputting a characteristic F10;
4.9, outputting a characteristic F11 after the output characteristic F10 is processed by three convolutional layers;
4.10, the output characteristic F11 is processed by a channel attention module SE _ Block, and the output characteristic F12;
4.11, performing first upsampling processing on the output characteristic F12 through an upsampling convolutional layer to obtain an output characteristic F13, wherein the number of characteristic channels is 128;
4.12, channel connection is carried out on the output characteristic F7 and the output characteristic F13, and the output characteristic F14 is processed through a fifth convolutional layer;
4.13, the output characteristic F14 passes through a channel attention module SE _ Block to output a characteristic F15;
4.14, performing second upsampling processing on the output characteristic F15 through the upsampling convolutional layer to obtain an output characteristic F16 with the characteristic channel number of 64;
4.15, channel connection is carried out on the output characteristic F5 and the output characteristic F16, and the output characteristic F17 is processed through a sixth convolutional layer;
4.16, the output feature F17 passes through a channel attention module SE _ Block to output a feature F18;
4.17, performing third upsampling processing on the output characteristic F18 through the upsampling convolutional layer to obtain an output characteristic F19, wherein the number of characteristic channels is 32;
4.18, connecting the output characteristic F3 and the output characteristic F19 by a channel, and processing the output characteristic F20 by a seventh convolution layer;
4.19, processing the output characteristic F20 through an eighth convolutional layer to output a characteristic F21, wherein the number of characteristic channels is 3;
4.20, the input image of step 4.1, i.e. the rained image, subtracts the output characteristic F21, and finally outputs a clear image after raining.
Example 3:
the method for removing rain from the residual single image based on the attention mechanism is the same as that of the channel attention module SE _ Block described in the embodiment 1-2 and the step (2.2.1), and is shown in FIG. 5; the specific steps of processing the image features by a channel attention module SE _ Block comprise:
5.1, outputting the weight of each channel after the image features of the input image sequentially pass through a global pooling layer, a complete connection layer, a ReLU activation layer, a complete connection layer and a Sigmoid layer;
and 5.2, multiplying each channel weight output in the step 5.1 with the input image characteristic in the step 5.1 channel by channel, and outputting the processed image characteristic.
The invention extracts more detailed image characteristics by utilizing the residual error network and the coding and decoding network structure, fully obtains the rainprint information of the rain image, and further obtains more detailed information of the rainprint in the rain image by adopting an attention mechanism, thereby effectively obtaining a clean clear image without rain and providing more convenience for subsequent high-level computer vision tasks.
The invention has not been described in detail in part of the common general knowledge of those skilled in the art.
While the invention has been particularly shown and described with reference to a preferred embodiment, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention.
Claims (7)
1. A residual single image rain removing method based on an attention mechanism is characterized by comprising the following specific steps:
(1) preprocessing an input image to obtain a preprocessed rainy image;
(2) constructing an attention residual error neural network model:
(2.1) adopting a convolution layer, a convolution residual block formed by short jump connection and a residual network module formed by long jump connection, and extracting image characteristics of the preprocessed rain image;
(2.2) constructing a coding and decoding network module:
(2.2.1) an encoder network is formed by a down-sampling and channel attention module SE _ Block, and a decoder network is formed by an up-sampling and channel attention module SE _ Block;
(2.2.2) the output characteristics of the bottommost layer of the encoder network enter a decoder network after being processed by a channel attention module SE _ Block and a convolutional layer, and the output characteristic information of decoder networks on other layers and the corresponding layers of the encoder network are overlapped and fused in a hopping connection mode to form an encoding and decoding network module;
(2.3) the output of the residual error network module is connected with the input of an encoder network in the encoding and decoding network module, and the output of a decoder network in the encoding and decoding network module is rain print information;
and (2.4) carrying out difference processing on the preprocessed rain image obtained in the step (1) and the rain print information output by the coding and decoding network module, outputting a rain-free image after rain is removed, and completing construction of an attention residual error neural network model.
(3) Training the attention residual error neural network model by adopting the preprocessed rainy image, restraining the attention residual error neural network model by using a loss function, and updating parameters by adopting back propagation to obtain a rain-removing neural network model;
(4) and taking the rain image to be processed as the input of a rain removing neural network model, carrying out rain removing processing on the image by using the rain removing neural network model, and outputting a clean image after rain removing.
2. The method of claim 1, wherein: the pretreatment in the step (1) is specifically as follows: the pixel values of the input image are normalized to [0, 1], and the pixel size is clipped to 256 × 3.
3. The method of claim 1, wherein: and (3) the residual error network module in the step (2.1) is formed by 1 convolution layer, 5 convolution residual error blocks formed by short jump connection and 1 long jump connection.
4. The method of claim 3, wherein: the convolution residual block formula is as follows:
xL+1=xL+F(xL+WL),
wherein x isL+1Is the convolution result of the L +1 th convolutional layer, xLAs a result of convolution of the L-th convolutional layer, WLIs the weight of the L-th convolutional layer, F (x)L+WL) Representing the residual part.
5. The method of claim 1, wherein: in the step (2.2.1), the encoder network consists of 3 downsampling and 3 channel attention modules SE _ Block and is used for extracting the rainprint features in the image features step by step; the decoder network consists of 3 times of upsampling and 3 channel attention modules SE _ Block and is used for fusing the rainprint features extracted by the encoder network step by step to obtain rainprint information.
6. The method of claim 5, wherein: the upsampling is specifically composed of a convolutional layer with the size of 3x3 and the step length of 2; the downsampling is specifically composed of a transposed convolutional layer with the size of 3x3 and the step size of 2; the channel attention module SE _ Block consists of a global pooling layer, a full connection layer, a ReLU activation layer, a full connection layer and a Sigmoid layer.
7. The method of claim 1, wherein: the loss function in the step (3) is specifically as follows:
Loss=LossSmooth_L1+λLossSSIM,
wherein LossSmooth_L1Represents a smooth mean absolute error loss function expressed as follows:
wherein, yiShowing the ith rainless image of the input, xiIndicating the input ith rained image, f (x)i) Representing the output of the ith rained image after the attention residual error neural network model processing;
LossSSIMrepresenting the structural similarity loss function, the expression is as follows:
LossSSIM=1-SSIM(Y,F(x)),
wherein, Y and x represent the input no-rain image and the input rain image respectively, F (x) represents the output of the input rain image x after the processing of the attention residual error neural network model;
SSIM denotes structural similarity, expressed as follows:
SSIM=[g(Y,F(x))]α·[c(Y,F(x))]β·[s(Y,F(x))]γ,
where g, c, and s represent brightness, contrast, and texture, respectively, and α, β, and γ represent a brightness coefficient, a contrast coefficient, and a texture coefficient, respectively.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110244785.7A CN112907479B (en) | 2021-03-05 | 2021-03-05 | Residual single image rain removing method based on attention mechanism |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110244785.7A CN112907479B (en) | 2021-03-05 | 2021-03-05 | Residual single image rain removing method based on attention mechanism |
Publications (2)
Publication Number | Publication Date |
---|---|
CN112907479A true CN112907479A (en) | 2021-06-04 |
CN112907479B CN112907479B (en) | 2023-04-18 |
Family
ID=76107761
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202110244785.7A Active CN112907479B (en) | 2021-03-05 | 2021-03-05 | Residual single image rain removing method based on attention mechanism |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN112907479B (en) |
Cited By (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113436101A (en) * | 2021-06-28 | 2021-09-24 | 杭州电子科技大学 | Method for removing rain of Longge Kuta module based on efficient channel attention mechanism |
CN113450288A (en) * | 2021-08-04 | 2021-09-28 | 广东工业大学 | Single image rain removing method and system based on deep convolutional neural network and storage medium |
CN113450278A (en) * | 2021-06-30 | 2021-09-28 | 中国矿业大学 | Image rain removing method based on cross-domain collaborative learning |
CN113658074A (en) * | 2021-08-16 | 2021-11-16 | 福州大学 | Single-image raindrop removing method based on LAB color space multi-scale fusion network |
CN113673590A (en) * | 2021-08-13 | 2021-11-19 | 广东工业大学 | Rain removing method, system and medium based on multi-scale hourglass dense connection network |
CN113706420A (en) * | 2021-10-20 | 2021-11-26 | 常州微亿智造科技有限公司 | Rain line removing device and rain line removing method in industrial detection |
CN114137541A (en) * | 2021-11-18 | 2022-03-04 | 国家超级计算深圳中心(深圳云计算中心) | Method, device and storage medium for predicting short rainfall on basis of Transformer-IRB |
CN114387436A (en) * | 2021-12-28 | 2022-04-22 | 北京安德医智科技有限公司 | Wall coronary artery detection method and device, electronic device and storage medium |
CN114529470A (en) * | 2022-02-21 | 2022-05-24 | 南通大学 | Single image rain removing method based on end-to-end neural network |
CN114881871A (en) * | 2022-04-12 | 2022-08-09 | 华南农业大学 | Attention-fused single image rain removing method |
CN114998138A (en) * | 2022-06-01 | 2022-09-02 | 北京理工大学 | High dynamic range image artifact removing method based on attention mechanism |
CN115345801A (en) * | 2022-10-18 | 2022-11-15 | 中科方寸知微(南京)科技有限公司 | Image compression and filter removal method and system based on image denoising idea |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109447918A (en) * | 2018-11-02 | 2019-03-08 | 北京交通大学 | Removing rain based on single image method based on attention mechanism |
AU2020100196A4 (en) * | 2020-02-08 | 2020-03-19 | Juwei Guan | A method of removing rain from single image based on detail supplement |
CN112070690A (en) * | 2020-08-25 | 2020-12-11 | 西安理工大学 | Single image rain removing method based on convolutional neural network double-branch attention generation |
-
2021
- 2021-03-05 CN CN202110244785.7A patent/CN112907479B/en active Active
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109447918A (en) * | 2018-11-02 | 2019-03-08 | 北京交通大学 | Removing rain based on single image method based on attention mechanism |
AU2020100196A4 (en) * | 2020-02-08 | 2020-03-19 | Juwei Guan | A method of removing rain from single image based on detail supplement |
CN112070690A (en) * | 2020-08-25 | 2020-12-11 | 西安理工大学 | Single image rain removing method based on convolutional neural network double-branch attention generation |
Non-Patent Citations (4)
Title |
---|
JUN FU ET AL.: "Residual Squeeze-and-Excitation Network for Fast Image Deraining", 《ARXIV[EESS.IV]》 * |
XUEYANG FU ET AL.: "Removing rain from single images via a deep detail network", 《2017 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR)》 * |
徐爱生 等: "注意力残差网络的单图像去雨方法研究", 《小型微型计算机系统》 * |
盖杉 等: "基于深度学习的非局部注意力增强网络图像去雨算法研究", 《电子学报》 * |
Cited By (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113436101A (en) * | 2021-06-28 | 2021-09-24 | 杭州电子科技大学 | Method for removing rain of Longge Kuta module based on efficient channel attention mechanism |
CN113436101B (en) * | 2021-06-28 | 2023-11-10 | 杭州电子科技大学 | Method for removing rain by Dragon lattice tower module based on efficient channel attention mechanism |
CN113450278A (en) * | 2021-06-30 | 2021-09-28 | 中国矿业大学 | Image rain removing method based on cross-domain collaborative learning |
CN113450278B (en) * | 2021-06-30 | 2022-02-18 | 中国矿业大学 | Image rain removing method based on cross-domain collaborative learning |
CN113450288A (en) * | 2021-08-04 | 2021-09-28 | 广东工业大学 | Single image rain removing method and system based on deep convolutional neural network and storage medium |
CN113673590A (en) * | 2021-08-13 | 2021-11-19 | 广东工业大学 | Rain removing method, system and medium based on multi-scale hourglass dense connection network |
CN113658074B (en) * | 2021-08-16 | 2023-07-28 | 福州大学 | Single image raindrop removing method based on LAB color space multi-scale fusion network |
CN113658074A (en) * | 2021-08-16 | 2021-11-16 | 福州大学 | Single-image raindrop removing method based on LAB color space multi-scale fusion network |
CN113706420A (en) * | 2021-10-20 | 2021-11-26 | 常州微亿智造科技有限公司 | Rain line removing device and rain line removing method in industrial detection |
CN114137541A (en) * | 2021-11-18 | 2022-03-04 | 国家超级计算深圳中心(深圳云计算中心) | Method, device and storage medium for predicting short rainfall on basis of Transformer-IRB |
CN114387436A (en) * | 2021-12-28 | 2022-04-22 | 北京安德医智科技有限公司 | Wall coronary artery detection method and device, electronic device and storage medium |
CN114529470A (en) * | 2022-02-21 | 2022-05-24 | 南通大学 | Single image rain removing method based on end-to-end neural network |
CN114881871A (en) * | 2022-04-12 | 2022-08-09 | 华南农业大学 | Attention-fused single image rain removing method |
CN114998138A (en) * | 2022-06-01 | 2022-09-02 | 北京理工大学 | High dynamic range image artifact removing method based on attention mechanism |
CN114998138B (en) * | 2022-06-01 | 2024-05-28 | 北京理工大学 | High dynamic range image artifact removal method based on attention mechanism |
CN115345801A (en) * | 2022-10-18 | 2022-11-15 | 中科方寸知微(南京)科技有限公司 | Image compression and filter removal method and system based on image denoising idea |
Also Published As
Publication number | Publication date |
---|---|
CN112907479B (en) | 2023-04-18 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN112907479B (en) | Residual single image rain removing method based on attention mechanism | |
CN111915530B (en) | End-to-end-based haze concentration self-adaptive neural network image defogging method | |
CN109360155B (en) | Single-frame image rain removing method based on multi-scale feature fusion | |
CN110009580B (en) | Single-picture bidirectional rain removing method based on picture block rain drop concentration | |
CN110517203B (en) | Defogging method based on reference image reconstruction | |
CN110378849B (en) | Image defogging and rain removing method based on depth residual error network | |
CN109993804A (en) | A kind of road scene defogging method generating confrontation network based on condition | |
CN108269244B (en) | Image defogging system based on deep learning and prior constraint | |
CN111462013A (en) | Single-image rain removing method based on structured residual learning | |
CN113284064B (en) | Cross-scale context low-illumination image enhancement method based on attention mechanism | |
CN111709888B (en) | Aerial image defogging method based on improved generation countermeasure network | |
CN112308085B (en) | Light field image denoising method based on convolutional neural network | |
CN112164010A (en) | Multi-scale fusion convolution neural network image defogging method | |
Bansal et al. | A review of image restoration based image defogging algorithms | |
CN113052776A (en) | Unsupervised image defogging method based on multi-scale depth image prior | |
Qian et al. | CIASM-Net: a novel convolutional neural network for dehazing image | |
CN111598793A (en) | Method and system for defogging image of power transmission line and storage medium | |
Pham et al. | Low-light image enhancement for autonomous driving systems using DriveRetinex-Net | |
CN113947538A (en) | Multi-scale efficient convolution self-attention single image rain removing method | |
WO2024178979A1 (en) | Single-image defogging method based on detail restoration | |
CN115471414A (en) | Image rain and snow removing method based on exposure imaging model and modular depth network | |
Wu et al. | MSDB-based CNN architecture for image dehazing in driverless cars | |
CN117764988B (en) | Road crack detection method and system based on heteronuclear convolution multi-receptive field network | |
Huang et al. | An End-to-End Network for Single Image Dedusting | |
Li et al. | Single I mage Haze Removal Based on Concentration Scale Prior |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |