WO2023040462A1 - 图像去雾方法、装置及设备 - Google Patents

图像去雾方法、装置及设备 Download PDF

Info

Publication number
WO2023040462A1
WO2023040462A1 PCT/CN2022/107282 CN2022107282W WO2023040462A1 WO 2023040462 A1 WO2023040462 A1 WO 2023040462A1 CN 2022107282 W CN2022107282 W CN 2022107282W WO 2023040462 A1 WO2023040462 A1 WO 2023040462A1
Authority
WO
WIPO (PCT)
Prior art keywords
residual
image
attention
module
output
Prior art date
Application number
PCT/CN2022/107282
Other languages
English (en)
French (fr)
Inventor
张登银
朱虹
韩文生
严伟丹
寇英杰
Original Assignee
南京邮电大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 南京邮电大学 filed Critical 南京邮电大学
Priority to US17/987,763 priority Critical patent/US11663705B2/en
Publication of WO2023040462A1 publication Critical patent/WO2023040462A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/73—Deblurring; Sharpening
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30181—Earth observation
    • G06T2207/30192—Weather; Meteorology

Definitions

  • the present application mainly relates to the technical field of image processing, and more specifically relates to an image defogging method, device and equipment.
  • the image dehazing algorithm is mainly based on the method of deep learning.
  • the dehazing network based on deep learning treats the channel and pixel features equally, but the haze is heterogeneous, such as thin fog and dense fog, and the pixel weight of the near view and the distant view should be obvious. Different, therefore, the dehazing network treats the channel and pixel features equally, resulting in poor dehazing effect, and the dehazed image still inevitably maintains the deep haze of the image and loses the details of the image.
  • the purpose of the present invention is to overcome the deficiencies in the prior art and provide an image defogging method, device and equipment.
  • the present invention provides an image defogging method, the method comprising:
  • the foggy image to be processed is input into a pre-trained dehazing model to obtain the fog-free image corresponding to the foggy image to be processed;
  • the pre-trained dehazing model includes a plurality of residual groups, and the residual groups include several series-connected residual dual-attention fusion modules, and the residual dual-attention fusion modules include residual Block, first convolutional layer, channel attention module, pixel attention module and second convolutional layer, the output of the residual block is connected with the channel attention module and pixel attention module respectively through the first convolution layer
  • the input connection of the channel attention module and the output of the pixel attention module is fused and then output processing is used to realize the output of the residual dual attention fusion module to obtain pixel features while enhancing the global dependence of each channel map.
  • the dehazing model includes 3 residual groups, and the 3 residual groups are connected in-channel according to the output from back to front.
  • the residual group includes 3 residual dual attention fusion modules.
  • the output of the channel attention module and the pixel attention module and the input of the residual block are added element by element and input to the second convolutional layer for fusion to obtain the output of the residual dual attention fusion module.
  • the dehazing model also includes a feature extraction convolution layer, a channel attention module, a pixel attention module, and an output convolution layer, and the foggy image to be processed enters the residual after the features are extracted by the feature extraction convolution layer Group, after the residual group processing, it enters the channel attention module, pixel attention module and output convolution layer for processing to obtain the output features, and the output features are added element-by-element to the foggy image to be processed to obtain Fog-free images.
  • the training of the dehazing model includes:
  • loss function L of the neural network is expressed as:
  • N is the number of training samples, is the real clear image of the i-th training sample, is the haze-free image estimated by the neural network for the i-th training sample.
  • the present invention also provides an image defogging device, the device comprising:
  • the foggy image acquisition module to be processed is used to obtain the foggy image to be processed
  • a defogging processing module configured to input the image to be defogged into a defogging model for processing, and output a fog-free image corresponding to the foggy image to be processed;
  • the dehazing model includes a plurality of residual groups, and the residual group includes several series-connected residual dual attention fusion modules, and the residual dual attention fusion module includes a residual block, a first convolutional layer , a channel attention module, a pixel attention module and a second convolutional layer, the output of the residual block is respectively connected to the input of the channel attention module and the pixel attention module through the first convolution layer, and the residual
  • the output of the difference block is respectively connected to the input of the channel attention module and the pixel attention module through the first convolutional layer, and the output of the channel attention module and the pixel attention module are fused to realize residual double attention
  • the output of the fusion module obtains pixel-wise features while enhancing the global dependencies of each channel map.
  • the output of the channel attention module and the pixel attention module and the input of the residual block are added element by element and input to the second convolutional layer for fusion to obtain the output of the residual dual attention fusion module.
  • the present invention also provides a device, including a memory, a processor, and a computer program stored in the memory and operable on the processor, when the processor executes the computer program, the first
  • a device including a memory, a processor, and a computer program stored in the memory and operable on the processor, when the processor executes the computer program, the first
  • the image defogging method described in any one aspect.
  • the present invention improves the convolutional neural network with a fixed receptive field, uses the residual dual attention fusion module as the basic module, and the residual dual attention fusion module consists of a residual block, a channel attention module and a pixel attention module Fusion composition, combining the relevant features of different channel maps, each channel map enhances the global dependence, and obtains pixel features at the same time, while reducing the number of parameters while retaining better details and improving the dehazing effect;
  • the present invention adopts an end-to-end dehazing network, and only three residual dual attention fusion modules are set in the residual group, which reduces the complexity of the model and improves the efficiency of model training.
  • FIG. 1 is a flowchart of an image defogging method provided by an embodiment of the present invention
  • Fig. 2 is a schematic structural diagram of a defogging model provided by an embodiment of the present invention
  • Fig. 3 is a schematic diagram of a channel attention module provided by an embodiment of the present invention.
  • Fig. 4 is a schematic diagram of a pixel attention module provided by an embodiment of the present invention.
  • FIG. 5 is a schematic diagram of a residual group provided by an embodiment of the present invention.
  • Fig. 6 is a schematic diagram of a residual dual attention fusion module provided by an embodiment of the present invention.
  • FIG. 7 is a comparison diagram of the defogging effect of the image defogging method provided by the embodiment of the present invention and other methods.
  • the present invention provides an image defogging method.
  • This method establishes an image defogging model based on a neural network, but improves the convolutional neural network with a fixed receptive field, and uses the residual dual attention fusion module as The basic module, Residual Dual Attention Fusion module consists of residual block and channel-to-pixel attention module fusion.
  • the dehazing model includes 1 feature extraction convolutional layer, 3 residual groups, channel attention module, pixel attention module and 2 output convolutional layers, 3 residual group Groups are connected in the channel according to the output from back to front; the foggy image to be processed enters the residual group after the features are extracted by the feature extraction convolution layer, and then enters the channel attention module and the pixel attention module after the residual group processing After processing by the module and two output convolutional layers, the output features are obtained, and the output features are added element-by-element to the foggy image to be processed to obtain a fog-free image.
  • the channel attention module inputs the input features into the global average pooling, convolutional layer, ReLU activation function, convolutional layer, and Sigmoid activation function, and the obtained output weight and input features are multiplied element-wise to obtain channel attention
  • the output of the force module its expression is as follows:
  • Z c (x, y) represents the pixel value of the input Z c of the c channel at the position (x, y), c ⁇ ⁇ R, G, B ⁇ ; after the global average pooling, the dimension of the feature map From C ⁇ H ⁇ W to C ⁇ 1 ⁇ 1; ⁇ is the ReLU activation function, ⁇ is the Sigmoid activation function, is an element-wise multiplication; channel attention module input F c to channel attention module output The mapping function between is H CAB .
  • the first convolutional layer of the channel attention module uses 8 convolutional kernels of size 1*1, and the second convolutional layer uses 64 convolutional kernels of size 1*1.
  • the pixel attention module is input from the input feature to the convolutional layer, the ReLU activation function, the convolutional layer, and the Sigmoid activation function, and the obtained output weight and the input feature are multiplied element-by-element to obtain the output of the pixel attention module. Its expression is as follows:
  • F PA is the output feature weight
  • the dimension is changed from C ⁇ H ⁇ W to 1 ⁇ H ⁇ W
  • the mapping function between the input F of the pixel attention module and the output F PAB of the channel attention module is H PAB .
  • the first convolutional layer of the pixel attention module uses 8 convolutional kernels of size 1*1, and the second convolutional layer uses 1 convolutional kernel of size 1*1.
  • the other convolutional layers use 64 convolutional kernels of size 3*3.
  • the residual group includes 3 series-connected residual dual attention fusion modules and 1 convolutional layer, the input features are input residual dual attention fusion modules and convolutional layers, and the output results and input features Perform element-by-element addition to obtain the output of the residual group, and the output expression of the residual group is as follows:
  • the mapping function between the input F g,m-1 of the residual double attention fusion module and the output F g,m of the residual double attention fusion module is H RDAFM ; by the residual group
  • the mapping function between the input F g,0 of and the output F g of the residual group is HRG .
  • the residual dual attention fusion module includes a residual block, a first convolutional layer, a channel attention module, a pixel attention module, and a second convolutional layer
  • the residual block includes a convolutional layer and a ReLU Activation function
  • the output of the residual block is respectively connected to the input of the channel attention module and the pixel attention module through the first convolution layer
  • the output fusion of the channel attention module and the pixel attention module is used to realize residual
  • the output of the difference double attention fusion module and the input of the residual block are added element by element and input to the second convolutional layer for fusion, and the output of the residual double attention fusion module is obtained, so that the residual double attention fusion module
  • the output of the pixel feature is obtained while enhancing the global dependence of each channel map, and the output of the residual dual attention fusion module is expressed as follows:
  • F RB is the output of the residual block
  • F * is the input of the attention module, from the input F of the residual dual attention fusion module to the output F RDAFM of the residual dual attention fusion module
  • the mapping function is H RDAFM .
  • the training of the dehazing model includes the following steps: obtain the RESIDE data set, randomly select 6000 pairs of foggy images and non-fogging images from the RESIDE data set to construct a training sample set, and use the training sample set to train the neural network to obtain the dehazing model. , get the foggy image to be processed and input it into the dehazing model to obtain the fog-free image.
  • the loss function L of the neural network is expressed as:
  • N is the number of training samples, is the real clear image of the i-th training sample, is the haze-free image estimated by the neural network for the i-th training sample.
  • the Adam optimizer is used to initialize the weight parameters of the network, where the default values of ⁇ 1 and ⁇ 2 are 0.9 and 0.999 respectively; the initial learning rate ⁇ is set to 1 ⁇ 10 -4 , and the cosine annealing strategy is used To update the learning rate, adjust the learning rate from the initial value to 0:
  • T is the total number of batches
  • ⁇ is the initial learning rate
  • t is the current batch
  • ⁇ t is the learning rate of adaptive update.
  • I is the foggy input image
  • an image defogging device is also provided, and the device includes:
  • Image acquisition module used to acquire foggy images to be processed
  • An image defogging module configured to input the image to be defogged into a defogging model for processing, and output a fog-free image corresponding to the foggy image to be processed;
  • the dehazing model includes a plurality of residual groups, and the residual group includes several series-connected residual dual attention fusion modules, and the residual dual attention fusion module includes a residual block, a first convolutional layer , a channel attention module, a pixel attention module and a second convolutional layer, the output of the residual block is respectively connected to the input of the channel attention module and the pixel attention module through the first convolution layer, and the residual The output of the difference block is respectively connected to the input of the channel attention module and the pixel attention module through the first convolutional layer, and the output of the channel attention module and the pixel attention module is connected to the input of the residual block by successive After the elements are added, they are input to the second convolutional layer for fusion, and the output of the residual dual attention fusion module is obtained, which is used to realize the output of the residual dual attention fusion module to obtain pixel features while enhancing the global dependence of each channel map.
  • This embodiment also provides a device, including a memory, a processor, and a computer program stored in the memory and operable on the processor.
  • the processor executes the computer program, the computer program described in Embodiment 1 is implemented. image defogging method.
  • the embodiments of the present application may be provided as methods, systems, or computer program products. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
  • computer-usable storage media including but not limited to disk storage, CD-ROM, optical storage, etc.
  • These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions
  • the device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

本发明公开了一种图像去雾方法、装置及设备，所述方法包括：获取待处理有雾图像，将有雾图像输入预先训练的去雾模型，获得有雾图像对应的无雾图像；所述去雾模型包括多个残差群组，所述残差群组包括若干串联的残差双重注意力融合模块，残差双重注意力融合模块包括残差块、第一卷积层、通道注意力模块、像素注意力模块和第二卷积层，所述残差块的输出经第一卷积层分别与所述通道注意力模块和像素注意力模块的输入连接，所述通道注意力模块和像素注意力模块的输出融合再进行输出处理，用于实现残差双重注意力融合模块的输出在每个通道图增强全局依赖的同时获得像素特征。本发明将残差双重注意力融合模块作为神经网络的基本模块，提高了去雾效果。

Description

图像去雾方法、装置及设备 技术领域
本申请主要涉及图像处理技术领域，更具体地说是涉及一种图像去雾方法、装置及设备。
背景技术
近年来，随着社会的发展，雾霾已经成为当前的一种大气现象，在计算机视觉中普遍存在。由于众多悬浮颗粒的存在，光线在传播过程中被反射，导致户外图像模糊、色彩失真、对比度降低。高级图像处理任务，如目标检测、目标识别和工业物联网，都需要清晰的图像作为输入，有雾的图像将影响后续高级任务的质量和鲁棒性；因此，作为一个预处理的图像任务，图像去雾是一个经典的图像处理问题，一直是研究者们研究的热点。
目前，图像去雾算法主要基于深度学习的方法，基于深度学习的去雾网络同等对待通道和像素特征，但是雾霾是非均质的，如薄雾和浓雾，近景和远景的像素权重应明显不同，因此，去雾网络同等对待通道和像素特征导致去雾效果不佳，去雾化后的图像仍然不可避免地保持着图像的深层雾霾，失去了图像的细节。
发明内容
本发明的目的在于克服现有技术中的不足，提供一种图像去雾方法、装置及设备。
第一方面，本发明提供了一种图像去雾方法，所述方法包括：
获取待处理有雾图像；
将所述待处理有雾图像输入预先训练的去雾模型，获得所述待处理有雾图 像对应的无雾图像；
其特征在于，所述预先训练的去雾模型包括多个残差群组，所述残差群组包括若干串联的残差双重注意力融合模块，所述残差双重注意力融合模块包括残差块、第一卷积层、通道注意力模块、像素注意力模块和第二卷积层，所述残差块的输出经第一卷积层分别与所述通道注意力模块和像素注意力模块的输入连接，所述通道注意力模块和像素注意力模块的输出融合再进行输出处理，用于实现残差双重注意力融合模块的输出在每个通道图增强全局依赖的同时获得像素特征。
进一步的，所述去雾模型包括3个残差群组，所述3个残差群组按照由后到前的输出进行通道内连接。
进一步的，所述残差群组包括3个残差双重注意力融合模块。
进一步的，所述通道注意力模块和像素注意力模块的输出与所述残差块的输入经逐元素相加后输入第二卷积层进行融合，获得残差双重注意力融合模块的输出。
进一步的，所述去雾模型还包括特征提取卷积层、通道注意力模块、像素注意力模块和输出卷积层，所述待处理有雾图像经特征提取卷积层提取特征后进入残差群组，经残差群组处理后依次进入通道注意力模块、像素注意力模块和输出卷积层进行处理后得到输出特征，输出特征与所述待处理有雾图像进行逐元素相加，获得无雾图像。
进一步的，所述去雾模型的训练，包括：
获取RESIDE数据集，从RESIDE数据集中随机选取6000对有雾图像和无雾图像构建训练样本集；
利用训练样本集对预先设立的神经网络进行训练。
进一步的，所述神经网络的损失函数L表示为：
Figure PCTCN2022107282-appb-000001
式中，N为训练样本的个数，
Figure PCTCN2022107282-appb-000002
为第i个训练样本的真实的清晰图像,
Figure PCTCN2022107282-appb-000003
为第i个训练样本由所述神经网络估计得到的无雾图像。
第二方面，本发明还提供了一种图像去雾装置，所述装置包括：
待处理有雾图像获取模块，用于获取待处理有雾图像；
去雾处理模块，用于将所述待去雾图像输入去雾模型进行处理，输出所述待处理有雾图像对应的无雾图像；
所述去雾模型包括多个残差群组，所述残差群组包括若干串联的残差双重注意力融合模块，所述残差双重注意力融合模块包括残差块、第一卷积层、通道注意力模块、像素注意力模块和第二卷积层，所述残差块的输出经第一卷积层分别与所述通道注意力模块和像素注意力模块的输入连接，所述残差块的输出经第一卷积层分别与所述通道注意力模块和像素注意力模块的输入连接，所述通道注意力模块和像素注意力模块的输出融合，用于实现残差双重注意力融合模块的输出在每个通道图增强全局依赖的同时获得像素特征。
进一步的，所述通道注意力模块和像素注意力模块的输出与所述残差块的输入经逐元素相加后输入第二卷积层进行融合，获得残差双重注意力融合模块的输出。
第三方面，本发明还提供了一种设备，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现第一方面任一项所述的图像去雾方法。
与现有技术相比，本发明的有益效果为：
第一，本发明改进了具有固定感受野的卷积神经网络，将残差双重注意力融合模块作为基本模块，残差双重注意力融合模块由残差块、通道注意力模块和像素注意力模块融合组成，结合不同通道图的相关特征，每个通道图增强全局依赖性，同时获得像素特征，在减少参数数量的同时保留了较好的细节，提高了去雾效果；
第二，本发明采用端到端的去雾网络，残差群组内部仅设置了3个残差双重注意力融合模块，减少了模型复杂程度，提高了模型训练的效率。
附图说明
图1是本发明实施例提供的图像去雾方法流程图；
图2是本发明实施例提供的去雾模型结构示意图；
图3是本发明实施例提供的通道注意力模块的示意图；
图4是本发明实施例提供的像素注意力模块的示意图；
图5是本发明实施例提供的残差群组的示意图；
图6是本发明实施例提供的残差双重注意力融合模块的示意图；
图7是本发明实施例提供的图像去雾方法与其他方法的去雾效果对比图。
具体实施方式
下面结合附图对本发明作进一步描述。以下实施例仅用于更加清楚地说明本发明的技术方案，而不能以此来限制本发明的保护范围。
实施例1：
如图1所示，本发明提供了一种图像去雾方法，本方法基于神经网络建立图像去雾模型，但是改进了具有固定感受野的卷积神经网络，将残差双重注意 力融合模块作为基本模块，残差双重注意力融合模块由残差块和通道与像素的注意力模块融合组成。
具体的，如图2所示，去雾模型包括1个特征提取卷积层、3个残差群组、通道注意力模块、像素注意力模块和2个输出卷积层，3个残差群组按照由后到前的输出进行通道内连接；待处理有雾图像经特征提取卷积层提取特征后进入残差群组，经残差群组处理后依次进入通道注意力模块、像素注意力模块和2个输出卷积层处理后得到输出特征，输出特征与所述待处理有雾图像进行逐元素相加，获得无雾图像。
如图3所示，通道注意力模块由输入特征输入全局平均池化、卷积层、ReLU激活函数、卷积层、Sigmoid激活函数，得到的输出权重和输入特征进行逐元素相乘得到通道注意力模块的输出，其表达式如下：
Figure PCTCN2022107282-appb-000004
Figure PCTCN2022107282-appb-000005
Figure PCTCN2022107282-appb-000006
式中，Z c(x,y)代表在(x,y)位置的c通道的输入Z c的像素值，c∈{R,G,B}；在全局平均池化后，特征图的维度由C×H×W变为C×1×1；δ是ReLU激活函数，σ是Sigmoid激活函数，
Figure PCTCN2022107282-appb-000007
是逐元素相乘；通道注意力模块输入F c到通道注意力模块输出
Figure PCTCN2022107282-appb-000008
之间的映射函数是H CAB。
通道注意力模块的第一个卷积层使用8个大小为1*1的卷积核，第二个卷积层使用64个大小为1*1的卷积核。
如图4所示，像素注意力模块由输入特征输入卷积层、ReLU激活函数、卷积层、Sigmoid激活函数，得到的输出权重和输入的特征进行逐元素相乘得到像 素注意力模块的输出其表达式如下：
F PA＝σ(Conv(δ(Conv(F))))     (4)
Figure PCTCN2022107282-appb-000009
F PAB＝H PAB(F)       (6)
式中F PA为输出的特征权重，维度由C×H×W变为1×H×W，像素注意力模块输入F到通道注意力模块输出F PAB之间的映射函数是H PAB。
像素注意力模块的第一个卷积层使用8个大小为1*1的卷积核，第二个卷积层使用1个大小为1*1的卷积核。其他卷积层使用64个大小为3*3的卷积核。
如图5所示，残差群组包括3个串联的残差双重注意力融合模块和1个卷积层，输入特征输入残差双重注意力融合模块和卷积层，输出的结果和输入特征进行逐元素相加得到残差群组的输出，残差群组的输出表达式如下：
F g,m＝H RDAFM(F g,m-1)      (7)
Figure PCTCN2022107282-appb-000010
F g＝H RG(F g,0)         (9)
式中，F g,m-1和F g,m分布是第g个残差群组中的第m个残差双重注意力融合模块的输入、输出，g＝1,2,3，m＝1,2,3；由残差双重注意力融合模块的输入F g,m-1到残差双重注意力融合模块的输出F g,m之间的映射函数是H RDAFM；由残差群组的输入F g,0到残差群组的输出F g之间的映射函数是H RG。
如图6所示，残差双重注意力融合模块包括残差块、第一卷积层、通道注意力模块、像素注意力模块和第二卷积层，残差块包括一个卷积层和ReLU激活函数，残差块的输出经第一卷积层分别与所述通道注意力模块和像素注意力 模块的输入连接，所述通道注意力模块和像素注意力模块的输出融合，用于实现残差双重注意力融合模块的输出与所述残差块的输入经逐元素相加后输入第二卷积层进行融合，获得残差双重注意力融合模块的输出，使得残差双重注意力融合模块的输出在每个通道图增强全局依赖的同时获得像素特征，残差双重注意力融合模块的输出，其表达式如下：
Figure PCTCN2022107282-appb-000011
F *＝Conv(F RB)        (11)
Figure PCTCN2022107282-appb-000012
F RDAFM＝H RDAFM(F)       (13)
其中，
Figure PCTCN2022107282-appb-000013
是逐元素相加，F RB是残差块的输出，F *是注意力模块的输入，由残差双重注意力融合模块的输入F到残差双重注意力融合模块的输出F RDAFM之间的映射函数是H RDAFM。
去雾模型的训练包括以下步骤：获取RESIDE数据集，从RESIDE数据集中随机选取6000对有雾图像和无雾图像构建训练样本集，利用训练样本集对神经网络进行训练得到去雾模型，使用时，获取待处理有雾图像输入去雾模型，获得无雾图像。
神经网络的损失函数L表示为：
Figure PCTCN2022107282-appb-000014
其中，N为训练样本的个数，
Figure PCTCN2022107282-appb-000015
为第i个训练样本的真实的清晰图像,
Figure PCTCN2022107282-appb-000016
为第i个训练样本由所述神经网络估计得到的无雾图像。
在所述神经网络中，使用Adam优化器对网络的权重参数进行初始化，其中β 1和β 2的默认值分别为0.9和0.999；初始学习率α设置为1×10 -4，使用余弦 退火策略来更新学习率，将学习率从初始值调整为0：
Figure PCTCN2022107282-appb-000017
式中，T为批次总数，α为初始学习率，t为当前的批次，α t为自适应更新的学习率。
对每个输入去雾网络模型的训练集中的样本图像，先利用前向传播求出真实的清晰图像与网络恢复的去雾的差异的总损失，再根据Adam优化器对权重参数进行更新；总共训练1×10 5步，每200步为一个批次，总共500个批次，重复上述步骤直至达到设定的最大步长，得到训练好的去雾网络模型，其表达式为：
F 0＝Conv(I)        (16)
F g＝H RG(F g-1)        (17)
Figure PCTCN2022107282-appb-000018
Figure PCTCN2022107282-appb-000019
其中，I是有雾的输入图像，F g-1和F g分别是第g个残差群组的输入、输出，g＝1,2,3；
Figure PCTCN2022107282-appb-000020
表示通道内的连接操作，
Figure PCTCN2022107282-appb-000021
是恢复的输出图像。
实施例2：
本实施例中，还提供一种图像去雾装置，所述装置包括：
图像获取模块，用于获取待处理有雾图像；
图像去雾模块，用于将所述待去雾图像输入去雾模型进行处理，输出所述待处理有雾图像对应的无雾图像；
所述去雾模型包括多个残差群组，所述残差群组包括若干串联的残差双重注意力融合模块，所述残差双重注意力融合模块包括残差块、第一卷积层、通 道注意力模块、像素注意力模块和第二卷积层，所述残差块的输出经第一卷积层分别与所述通道注意力模块和像素注意力模块的输入连接，所述残差块的输出经第一卷积层分别与所述通道注意力模块和像素注意力模块的输入连接，所述通道注意力模块和像素注意力模块的输出与所述残差块的输入经逐元素相加后输入第二卷积层进行融合，获得残差双重注意力融合模块的输出，用于实现残差双重注意力融合模块的输出在每个通道图增强全局依赖的同时获得像素特征。
实施例3：
本实施例还提供一种设备，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现实施例1所述的图像去雾方法。
本领域内的技术人员应明白，本申请的实施例可提供为方法、系统、或计算机程序产品。因此，本申请可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本申请可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本申请是参照根据本申请实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流 程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
以上所述仅是本发明的优选实施方式，应当指出，对于本技术领域的普通技术人员来说，在不脱离本发明技术原理的前提下，还可以做出若干改进和变形，这些改进和变形也应视为本发明的保护范围。

Claims (10)

  1. 一种图像去雾方法，所述方法包括：
    获取待处理有雾图像；
    将所述待处理有雾图像输入预先训练的去雾模型，获得所述待处理有雾图像对应的无雾图像；
    其特征在于，所述预先训练的去雾模型包括多个残差群组，所述残差群组包括若干串联的残差双重注意力融合模块，所述残差双重注意力融合模块包括残差块、第一卷积层、通道注意力模块、像素注意力模块和第二卷积层，所述残差块的输出经第一卷积层分别与所述通道注意力模块和像素注意力模块的输入连接，所述通道注意力模块和像素注意力模块的输出融合后再进行输出处理，用于实现残差双重注意力融合模块的输出在每个通道图增强全局依赖的同时获得像素特征。
  2. 根据权利要求1所述的图像去雾方法，其特征在于，所述去雾模型包括3个残差群组，所述3个残差群组按照由后到前的输出进行通道内连接。
  3. 根据权利要求1所述的图像去雾方法，其特征在于，所述残差群组包括3个残差双重注意力融合模块。
  4. 根据权利要求1所述的图像去雾方法，其特征在于，所述通道注意力模块和像素注意力模块的输出与所述残差块的输入经逐元素相加后输入第二卷积层进行融合，获得残差双重注意力融合模块的输出。
  5. 根据权利1所述的图像去雾方法，其特征在于，所述去雾模型还包括特征提取卷积层、通道注意力模块、像素注意力模块和输出卷积层，所述待处理 有雾图像经特征提取卷积层提取特征后进入残差群组，经残差群组处理后依次进入通道注意力模块、像素注意力模块和输出卷积层进行处理后得到输出特征，输出特征与所述待处理有雾图像进行逐元素相加，获得无雾图像。
  6. 根据权利要求1所述的图像去雾方法，其特征在于，所述去雾模型的训练，包括：
    获取RESIDE数据集，从RESIDE数据集中随机选取6000对有雾图像和无雾图像构建训练样本集；
    利用训练样本集对预先设立的神经网络进行训练。
  7. 根据权利要求6所述的图像去雾方法，其特征在于，所述神经网络的损失函数L表示为：
    Figure PCTCN2022107282-appb-100001
    式中，N为训练样本的个数，
    Figure PCTCN2022107282-appb-100002
    为第i个训练样本的真实的清晰图像,
    Figure PCTCN2022107282-appb-100003
    为第i个训练样本由所述神经网络估计得到的无雾图像。
  8. 一种图像去雾装置，所述装置包括：
    图像获取模块，用于获取待处理有雾图像；
    图像去雾模块，用于将所述待去雾图像输入去雾模型进行处理，输出所述待处理有雾图像对应的无雾图像；
    其特征在于，所述去雾模型包括多个残差群组，所述残差群组包括若干串联的残差双重注意力融合模块，所述残差双重注意力融合模块包括残差块、第一卷积层、通道注意力模块、像素注意力模块和第二卷积层，所述残差块的输出经第一卷积层分别与所述通道注意力模块和像素注意力模块的输入连接，所述残差块的输出经第一卷积层分别与所述通道注意力模块和像素注意力模块的 输入连接，所述通道注意力模块和像素注意力模块的输出融合，用于实现残差双重注意力融合模块的输出在每个通道图增强全局依赖的同时获得像素特征。
  9. 根据权利要求8所述的图像去雾装置，其特征在于，所述通道注意力模块和像素注意力模块的输出与所述残差块的输入经逐元素相加后输入第二卷积层进行融合，获得残差双重注意力融合模块的输出。
  10. 一种设备，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，其特征在于，所述处理器执行所述计算机程序时实现权利要求1-7任一项所述的图像去雾方法。
PCT/CN2022/107282 2021-09-17 2022-07-22 图像去雾方法、装置及设备 WO2023040462A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/987,763 US11663705B2 (en) 2021-09-17 2022-11-15 Image haze removal method and apparatus, and device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111091274.2 2021-09-17
CN202111091274.2A CN113947537A (zh) 2021-09-17 2021-09-17 图像去雾方法、装置及设备

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/987,763 Continuation US11663705B2 (en) 2021-09-17 2022-11-15 Image haze removal method and apparatus, and device

Publications (1)

Publication Number Publication Date
WO2023040462A1 true WO2023040462A1 (zh) 2023-03-23

Family

ID=79328700

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/107282 WO2023040462A1 (zh) 2021-09-17 2022-07-22 图像去雾方法、装置及设备

Country Status (2)

Country Link
CN (1) CN113947537A (zh)
WO (1) WO2023040462A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116596792A (zh) * 2023-05-22 2023-08-15 武汉理工大学 一种面向智能船舶的内河雾天场景恢复方法、系统及设备
CN117649439A (zh) * 2024-01-30 2024-03-05 山东省海洋资源与环境研究院(山东省海洋环境监测中心、山东省水产品质量检验中心) 一种海草床面积的获取方法、系统、设备和存储介质
CN117911282A (zh) * 2024-03-19 2024-04-19 华中科技大学 一种图像去雾模型的构建方法及应用

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11663705B2 (en) 2021-09-17 2023-05-30 Nanjing University Of Posts And Telecommunications Image haze removal method and apparatus, and device
CN113947537A (zh) * 2021-09-17 2022-01-18 南京邮电大学 图像去雾方法、装置及设备
CN114648467B (zh) * 2022-05-18 2022-08-16 中山大学深圳研究院 图像的去雾方法、装置、终端设备及计算机可读存储介质
CN115330631A (zh) * 2022-08-22 2022-11-11 南京邮电大学 一种基于堆叠沙漏网络的多尺度融合去雾方法
CN115908206B (zh) * 2023-03-13 2023-06-16 中国石油大学(华东) 一种基于动态特征注意力网络的遥感图像去雾方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190294931A1 (en) * 2018-03-26 2019-09-26 Artomatix Ltd. Systems and Methods for Generative Ensemble Networks
CN111814753A (zh) * 2020-08-18 2020-10-23 深延科技(北京)有限公司 针对雾天条件下的目标检测方法和装置
CN113269685A (zh) * 2021-05-12 2021-08-17 南通大学 一种融合多注意力机制的图像去雾方法
CN113284070A (zh) * 2021-06-16 2021-08-20 河南理工大学 基于传递注意力机制的非均匀雾图去雾算法
CN113947537A (zh) * 2021-09-17 2022-01-18 南京邮电大学 图像去雾方法、装置及设备

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190294931A1 (en) * 2018-03-26 2019-09-26 Artomatix Ltd. Systems and Methods for Generative Ensemble Networks
CN111814753A (zh) * 2020-08-18 2020-10-23 深延科技(北京)有限公司 针对雾天条件下的目标检测方法和装置
CN113269685A (zh) * 2021-05-12 2021-08-17 南通大学 一种融合多注意力机制的图像去雾方法
CN113284070A (zh) * 2021-06-16 2021-08-20 河南理工大学 基于传递注意力机制的非均匀雾图去雾算法
CN113947537A (zh) * 2021-09-17 2022-01-18 南京邮电大学 图像去雾方法、装置及设备

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
ZHU HONG; ZHANG DENGYIN; KOU YINGJIE: "Dual Attention Fusion Network for Single Image Dehazing", 2021 13TH INTERNATIONAL CONFERENCE ON WIRELESS COMMUNICATIONS AND SIGNAL PROCESSING (WCSP), IEEE, 20 October 2021 (2021-10-20), pages 1 - 5, XP034033166, DOI: 10.1109/WCSP52459.2021.9613170 *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116596792A (zh) * 2023-05-22 2023-08-15 武汉理工大学 一种面向智能船舶的内河雾天场景恢复方法、系统及设备
CN116596792B (zh) * 2023-05-22 2023-12-29 武汉理工大学 一种面向智能船舶的内河雾天场景恢复方法、系统及设备
CN117649439A (zh) * 2024-01-30 2024-03-05 山东省海洋资源与环境研究院(山东省海洋环境监测中心、山东省水产品质量检验中心) 一种海草床面积的获取方法、系统、设备和存储介质
CN117649439B (zh) * 2024-01-30 2024-04-09 山东省海洋资源与环境研究院(山东省海洋环境监测中心、山东省水产品质量检验中心) 一种海草床面积的获取方法、系统、设备和存储介质
CN117911282A (zh) * 2024-03-19 2024-04-19 华中科技大学 一种图像去雾模型的构建方法及应用
CN117911282B (zh) * 2024-03-19 2024-05-28 华中科技大学 一种图像去雾模型的构建方法及应用

Also Published As

Publication number Publication date
CN113947537A (zh) 2022-01-18

Similar Documents

Publication Publication Date Title
WO2023040462A1 (zh) 图像去雾方法、装置及设备
CN112288658B (zh) 一种基于多残差联合学习的水下图像增强方法
CN109360171B (zh) 一种基于神经网络的视频图像实时去模糊方法
CN110097519B (zh) 基于深度学习的双监督图像去雾方法、系统、介质和设备
CN109410127B (zh) 一种基于深度学习与多尺度图像增强的图像去噪方法
CN108596053B (zh) 一种基于ssd和车辆姿态分类的车辆检测方法和系统
CN108564549B (zh) 一种基于多尺度稠密连接网络的图像去雾方法
CN109272455B (zh) 基于弱监督生成对抗网络的图像去雾方法
CN109978807B (zh) 一种基于生成式对抗网络的阴影去除方法
JP2021517681A (ja) ロボットの目標物体把持位置の検出方法
US11663705B2 (en) Image haze removal method and apparatus, and device
WO2019019086A1 (zh) 增强图像对比度的方法、设备及存储介质
CN109509248B (zh) 一种基于神经网络的光子映射渲染方法和系统
CN109493300B (zh) 基于fpga卷积神经网络的航拍图像实时去雾方法及无人机
WO2021238420A1 (zh) 图像去雾方法、终端及计算机存储介质
CN112419191B (zh) 基于卷积神经网络的图像运动模糊去除方法
CN112686119A (zh) 基于自注意力生成对抗网络的车牌运动模糊图像处理方法
CN111091122B (zh) 一种多尺度特征卷积神经网络的训练和检测方法、装置
CN113724155A (zh) 用于自监督单目深度估计的自提升学习方法、装置及设备
CN110751271B (zh) 一种基于深度神经网络的图像溯源特征表征方法
CN116363011A (zh) 一种基于频域分频的多分支低照度图像增强方法
CN108876750B (zh) 一种基于非局部相似性低秩矩阵的图像去除泊松噪声的方法
CN109859111A (zh) 一种基于map方法的单幅图像盲去模糊方法
CN111161228B (zh) 一种基于迁移学习的纽扣表面缺陷检测方法
CN109800793B (zh) 一种基于深度学习的目标检测方法和系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22868839

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE